Get the changed HTML content after it's updated by Javascript? (htmlunit)

htmlunit, java, scala

Solution

I figured it out!

`HtmlPage` objects have an `executeJavaScript(String)` which can be used to kick off the `showTime` script. Then, once the script has actually started, that's when `waitForBackgroundJavaScript` becomes relevant.

The code I ended up with:

import com.gargoylesoftware.htmlunit._
import com.gargoylesoftware.htmlunit.html.HtmlPage
import com.gargoylesoftware.htmlunit.html.DomElement

object AtomicTime {

  def main(args: Array[String]): Unit = {
    val url = "http://tycho.usno.navy.mil/what.html"
    val client = new WebClient(BrowserVersion.CHROME)

    var response: HtmlPage = client.getPage(url)
    response.executeJavaScript("showTime")

    printf("Current AtomicTime: %s", getUpdatedRespose(response, client))
  }

  def getUpdatedRespose(page: HtmlPage, client: WebClient): String = {
    while (page.getElementById("USNOclk").asText() == "Loading...") {
      client.waitForBackgroundJavaScript(200)
    }
    return page.getElementById("USNOclk").asText()
  }
}

Problem

I'm having some trouble figuring out how to get the content of some HTML after javascript has updated it. Specifically, I'm trying to get the current time from US Naval Observatory Master Clock. It has an `h1` element with the `ID` of `USNOclk` in which it displays the current time. When the page first loads, this element is set to display "Loading...", and then javascript kicks in and updates it to the current time via ``` function showTime() { document.getElementById('USNOclk').innerHTML="Loading...<br />"; xmlHttp=GetXmlHttpObject(); if (xmlHttp==null){ document.getElementById('USNOclk').innerHTML="Sorry, browser incapatible. <BR />"; return; } refresher = 0; startResponse = new Date().getTime(); var url="http://tycho.usno.navy.mil/cgi-bin/time.pl?n="+ startResponse; xmlHttp.onreadystatechange=stateChanged; xmlHttp.open("GET",url,true); xmlHttp.send(null); } ``` So, the problem is that I'm not sure how to get the updated time. When I check the element, I see the "Loading..." as the content of the `h1` element. I've double checked that javascript is enabled, and I've tried calling the `waitForBackgroundJavaScript` function on the `webclient` as well hoping that it would give the javascript time to start updating stuff. However, no success as of yet. My Current Code: ``` import com.gargoylesoftware.htmlunit._ import com.gargoylesoftware.htmlunit.html.HtmlPage object AtomicTime { def main(args: Array[String]): Unit = { val url = "http://tycho.usno.navy.mil/what.html" val client = new WebClient(BrowserVersion.CHROME) println(client.isJavaScriptEnabled()) // returns true client.waitForBackgroundJavaScript(10000) // client.waitForBackgroundJavaScriptStartingBefore(10000) //tried this one too without success var response: HtmlPage = client.getPage(url) println(response.asText()) } } ``` How do I trigger the javascript to update the HTML?

Original source