Error Executing javascript using HtmlUnit

Question

I am trying to execute a JavaScript on the following URL: `http://steamcommunity.com/id/bobcatchris/inventory#730`. I can do it using google chrome by opening console by pressing ctrl+shift+j and pasting the following script: ``` var list = []; var size = Object.keys(g_ActiveInventory.rgInventory).size(); var counter = 0; while (counter < size) { list.push(g_ActiveInventory.rgInventory[Object.keys(g_ActiveInventory.rgInventory)[counter]].market_name); counter +=1; } ``` It returns: 150 Then in the next line, i write: `>list` and it returns an array of length 150. When i try to do it in `HtmlUnit` as: ``` public static void main(String[] args) throws IOException { WebClient webClient=new WebClient(BrowserVersion.FIREFOX_17); HtmlPage page=webClient.getPage("http://steamcommunity.com/id/bobcatchris/inventory#730"); String script="var list = [];
" + "
" + "
" + "var size = Object.keys(g_ActiveInventory.rgInventory).size();
" + "
" + "
" + "
" + "var counter = 0;
" + "
" + "while…

Accepted Answer

These are your `javascript` script errors translated to `Java exceptions` by your framework.

That's because you have not explicitly set the `setThrowExceptionOnScriptError` option of your `WebClient` to `false`.

Unless there are absolutely no errors in the `js`, it is useful to always set this value as false unless it interferes with the results your are looking for.

Typically, these are my `webclient` settings when dealing with `js` and `ajax` through `HtmlUnit`.

```
final WebClient webClient = new WebClient(BrowserVersion.FIREFOX_17,
                PROXY_HOST, PROXY_PORT);

WebRequest request = new WebRequest(new URL(
                "http://steamcommunity.com/id/bobcatchris/inventory#730"));

webClient.getOptions().setThrowExceptionOnScriptError(false);
webClient.setJavaScriptTimeout(10000);
webClient.getOptions().setJavaScriptEnabled(true);
webClient.setAjaxController(new NicelyResynchronizingAjaxController());
webClient.getOptions().setTimeout(10000);

HtmlPage page = webClient.getPage(request);

String script="var list = [];
" +
                    "
" +
                    "
" +
                    "var size = Object.keys(g_ActiveInventory.rgInventory).size();
" +
                    "
" +
                    "
" +
                    "
" +
                    "var counter = 0;
" +
                    "
" +
                    "while (counter < size) {
" +
                    " list.push(g_ActiveInventory.rgInventory[Object.keys(g_ActiveInventory.rgInventory)[counter]].market_name);
" +
                    " counter +=1;
" +
                    "}";
            Object result = page.executeJavaScript(script).getJavaScriptResult();
            System.out.println(result);
```

If I try your code, with the above settings I get `150.0` printed to the console which I assume is working as expected.

EDIT:

To traverse over the complete array `list`:

```
String script="var list = [];
" +
                    "
" +
                    "
" +
                    "var size = Object.keys(g_ActiveInventory.rgInventory).size();
" +
                    "
" +
                    "
" +
                    "
" +
                    "var counter = 0;
" +
                    "
" +
                    "while (counter < size) {
" +
                    " list.push(g_ActiveInventory.rgInventory[Object.keys(g_ActiveInventory.rgInventory)[counter]].market_name);
" +
                    " counter +=1;
" +
                    "}"
                    + "list";

Object result = page.executeJavaScript(script).getJavaScriptResult();
            if (result instanceof NativeArray) {
                for (Object obj : (NativeArray)result) {
                    System.out.println(obj);    
                }

}
```

Above, I have change the `js` to include list as the return parameter and iterate over `NativeArray` to get each element.

Output:

```
P2000 | Scorpion (Factory New)
AK-47 | Black Laminate (Field-Tested)
★ StatTrak™ Karambit | Case Hardened (Minimal Wear)
CS:GO Case Key
CS:GO Case Key
```

You can read much on their ajax settings at their FAQs - here.

Error Executing javascript using HtmlUnit

Solution

Problem