Java 6 NTLM proxy authentication and HTTPS - has anyone got it to work?

Question

I have a Java application (not an applet) that needs to access a web service. Proxies for the web service have been generated with JAX-WS, and seem to work fine. In one scenario it needs to talk through a web proxy server (actually Squid 3.0), which is set to require NTLM authentication. Running on Sun's JRE 1.6.0_14, everything works fine for accessing HTTP URLs, without requiring any changes: the built-in NTLM authenticator kicks in and it all works seemlessly. If, however, the web service URL is a HTTPS URL, the web service call fails deep inside Sun's code: ``` com.sun.xml.internal.ws.client.ClientTransportException: HTTP transport error: java.lang.NullPointerException at com.sun.xml.internal.ws.transport.http.client.HttpClientTransport.getOutput(HttpClientTransport.java:121) at com.sun.xml.internal.ws.transport.http.client.HttpTransportPipe.process(HttpTransportPipe.java:142) at com.sun.xml.internal.ws.transport.http.client.HttpTransportPipe.processRequest(HttpTransportPipe.java:…

Accepted Answer

After some debugging, this seems to be a flaw in the JRE class libraries, specifically in `sun.net.www.protocol.http.HttpURLConnection`.

Studying the HTTP requests and responses in the cases of HTTP and HTTPS endpoints showed that, in the successful HTTP case, the requests had a header `Proxy-Connection=keep-alive`, which was missing on the failing HTTPS case. Reading more generally, there seems to be some confusion on whether one should use "Proxy-Connection" or just "Connection", too ...

Anyway, it is notable that in the HTTP case, the code goes through `HttpURLConnection.writeRequests()`, which contains the following code snippet

```
    /*
     * For HTTP/1.1 the default behavior is to keep connections alive.
     * However, we may be talking to a 1.0 server so we should set
     * keep-alive just in case, except if we have encountered an error
     * or if keep alive is disabled via a system property
     */

// Try keep-alive only on first attempt
    if (!failedOnce && http.getHttpKeepAliveSet()) {
    if (http.usingProxy) {
        requests.setIfNotSet("Proxy-Connection", "keep-alive");
    } else {
        requests.setIfNotSet("Connection", "keep-alive");
    }
```

There's no such code when creating a tunnel through the proxy for HTTPS, which causes Squid to get upset during the NTLM authentication conversation.

To work around this, in `HttpURLConnection.sendCONNECTRequest()`, I added

```
if (http.getHttpKeepAliveSet()) {
    if (http.usingProxy) {
        requests.setIfNotSet("Proxy-Connection", "keep-alive");
    }
}
```

just before

```
setPreemptiveProxyAuthentication(requests);
http.writeRequests(requests, null);
```

I inject my modified `HttpURLConnection.class` into the JRE using the "-Xbootclasspath/p" flag, and now it works! Not exactly elegant, but there we are.

Java 6 NTLM proxy authentication and HTTPS - has anyone got it to work?

Solution

Problem