Commons HTTPClient 3.1 and 401 Unauthorized Error

Commons http client is a library provides the client-side implementation of most of the HTTP standards and recommendations. It is mainly used while building HTTP aware client applications such as web browsers, web service clients etc.

Recently while accessing the sql server reporting services, we faced an authentication error on one of the client environments. The stacktrace found was as below

2011-08-10 03:17:12,868 INFO  [http-8080-2](HTTPSender.java:192) - Unable to sendViaPost to url[http://machine-name/ReportServer/ReportService2005.asmx]
org.apache.axis2.AxisFault: Transport error: 401 Error: Unauthorized
at org.apache.axis2.transport.http.HTTPSender.handleResponse(HTTPSender.java:296)
at org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:190)
at org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:75)
at org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:371)
at org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:209)
at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:448)
at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:401)
at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:228)
at org.apache.axis2.client.OperationClient.execute(OperationClient.java:163)

We use axis2 stubs for accessing the reporting services which internally uses commons http client library for authentication. Our application used 3.1 version of the library.  The code for specifying the authentication details is as below

ReportExecutionServiceStub stub = new ReportExecutionServiceStub(args[0] + "/ReportExecution2005.asmx");
ServiceClient sc = stub._getServiceClient();

HttpTransportProperties.Authenticator authentication = new HttpTransportProperties.Authenticator();
authentication.setUsername(args[1]);
authentication.setPassword(args[2]);
authentication.setHost(args[3]);

authentication.setRealm(AuthScope.ANY.toString());
authentication.setDomain(args[4]);

sc.getOptions().setProperty(HTTPConstants.AUTHENTICATE, authentication);

As part of initial troubleshooting one should verify the username password and whether the host name is given in the right format. By right format I mean the host name should match with the host name specified in the report server url (i.e. args[0] and args[3] from the above code). If the host name specified in “test-machine-01.domain-name” then the report server url should be http://test-machine-01.domain-name/reportserver .

Report Server Trace Logs

Report Server provides detailed logging which can also help in the troubleshooting of the issue. Enabling the Report Server Service Trace Log and Report Server HTTP Log can provide more in-sights to the issue. In my case I found the below log message in the Report Server Service Trace Logs

http!rshost !d6c!<date-time>:: v VERBOSE: Authentication failed with error state: 14

Having a look at the event viewer logs on the sql server machine showed that a Failed Audit event was logging with an message “Username or password is incorrect”.

On capturing the packets (through Wireshark) exchanged between the client and the server machine I found that the commons http-client 3.1 version of the library implemented the most basic version of the NTLM protocol which is called the LAN Manager authentication protocol.

NTLM Authentication Protocol

NTLM is a challenge-response authentication protocol which uses three messages to authenticate a client in a connection oriented environment (connectionless is similar), and a fourth additional message if integrity is desired. First, the client establishes a network path to the server and sends a NEGOTIATE_MESSAGE advertising its capabilities. Next, the server responds with CHALLENGE_MESSAGE which is used to establish the identity of the client. Finally, the client responds to the challenge with a AUTHENTICATE_MESSAGE.

There are three variants of this protocol – LM, NTMLv1 and NTLMv2. The message flow for all three is the same; the only differences are the function used to compute various response fields from the challenge, and which response fields are set.

In terms of security, this is the lowest level at which any Windows computer can operate. NTLMv1 is an improvement over LM, but is still not as secure as the newest version of NTLM. This is the latest version of the available Windows authentication protocols, and is the most secure.

Packet Trace & Source Code

The packet trace from wireshark indicated that the NT Response in the AUTHENTICATE_MESSAGE was empty.

Also the getType3Message() method implementation in the NTLM class sets the NT response length as zero as per the protocol specification.

Root Cause

The interesting part was that this protocol implementation worked everywhere else but failed on one specific windows machine. On investigating further I found that there are two windows control settings which lead to this authentication failure

1. LAN Manager authentication level
2. NoLMHashPolicy

LAN Manager authentication level

The LAN Manager Authentication Level setting governs which protocols Windows accepts. Authentication via the Kerberos protocal is also available, but because Kerberos is not configurable through the LAN Manager Authentication Level setting, it will not be included here.

Look at the domain controller (for a domain account) or the server machine (for a local account to know the value of this setting. Below are the steps to follow :

1. Click Start, point to Programs, and then click Administrative Tools.
2. Under Local Security Settings, expand Local Policies.
3. Click Security Options.
4. Double-click Network Security: LAN manager authentication level, and then click an appropriate value in the list.

You can also have a look at this registry key – HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\LSA\LMCompatibilityLevel.

This setting can have a value from 0-5 which are very well explained here and here. The simpler way to think about the settings is that levels 0-3 affect the computer when acting as a client, while levels 4 and 5 affect the machine when acting as the authentication server.

Because not all clients can use the highest security level (LM or NTLMv1 or NTLMv2) available, the LAN Manager Authentication Level can be set relatively low to ensure compatibility with computers using other authentication protocols. However, increasing compatibility also increases vulnerability, as the older LM and NTLMv1 protocols are now considered insecure.

LM authentication is not as strong as Windows NT authentication so some customers may want to disable its use, because an attacker eavesdropping on network traffic will attack the weaker protocol. A successful attack can compromise the user’s password. To disable LM authentication, one can set the LAN Manager Authentication Level to 4 (Send NTLMv2 response only/refuse LM). This would mean that the authentication request from the commons http client library (version 3.1) would be rejected with an authentication error.

NoLMHashPolicy

Once you force the systems to accept only NTLMv1 and NTLMv2 authentication requests you’ll still needto force the systems to remove the LM hash from their database. Instead of storing your user account password in clear-text, Windows generates and stores user account passwords by using two different password representations, generally known as “hashes.” When you set or change the password for a user account to a password that contains fewer than 15 characters, Windows generates both a LAN Manager hash (LM hash) and a Windows NT hash (NT hash) of the password. These hashes are stored in the local Security Accounts Manager (SAM) database or in Active Directory.
The LM hash is relatively weak compared to the NT hash, and it is therefore prone to fast brute force attack. Therefore, you may want to prevent Windows from storing an LM hash of your password. This can be done by updating the Local Group Policy  (Windows XP or Windows Server 2003) or in a Windows Server 2003 Active Directory environment by using Group Policy in Active Directory.

1. In Group Policy, expand Computer Configuration, expand Windows Settings, expand Security Settings, expand Local Policies, and then click Security Options.
2. In the list of available policies, double-click Network security: Do not store LAN Manager hash value on next password change.
3. Click Enabled, and then click OK.

Setting either the LAN Manager Authentication Level to 4 or enabling the NoLMHashPolicy could lead to an authentication failure when using the commons httpclient library.

There are a couple of solutions to this issue i.e. either upgrading the commons http-client library version from 3.1 to the latest version for e.g. 4.1.2 which supports all the three versions of the NTLM protocol or patching the commons httpclient 3.1 version of the library to compute the NT Response before sending the Type3 message to the server. I will be following the later solution in my next post.

Advertisement