Sunday 17 January 2016

IndexOutOfBoundsException when using commons BeanUtils in Java 8

When migrating from Java 7 to Java 8, we noticed a strange issue with apache commons BeanUtils.populate(target, properties) method, when collections were populated from properties. They started failing with the following exception


Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.rangeCheck(ArrayList.java:653)
at java.util.ArrayList.get(ArrayList.java:429)
at org.apache.commons.beanutils.PropertyUtilsBean.getIndexedProperty(PropertyUtilsBean.java:513)
at org.apache.commons.beanutils.PropertyUtilsBean.getIndexedProperty(PropertyUtilsBean.java:410)
at org.apache.commons.beanutils.PropertyUtilsBean.getNestedProperty(PropertyUtilsBean.java:768)
at org.apache.commons.beanutils.PropertyUtilsBean.getProperty(PropertyUtilsBean.java:846)
at org.apache.commons.beanutils.BeanUtilsBean.setProperty(BeanUtilsBean.java:903)
at org.apache.commons.beanutils.BeanUtilsBean.populate(BeanUtilsBean.java:830)
at org.apache.commons.beanutils.BeanUtils.populate(BeanUtils.java:433)
at sample.test.BeanPopulateTest.main(BeanPopulateTest.java:16)

To make things clear, I am going to use the example below.

Employee has a collection of Address objects. Note that Employee object has additional methods, to get the address at a given index and set address at an index. These methods are used by the BeanPopulator, to populate the values from OGNL expressions.

Employee.java

import java.util.ArrayList;
import java.util.List;

public class Employee {

  private List addresses = new ArrayList<Address>();

  public List<Address> getAddresses() {
    return addresses;
  }

  public void setAddresses(List<Address> addresses) {
    this.addresses = addresses;
  }

  public Address getAddresses(final int index) {
    if (index <= addresses.size()) {
       for (int i = addresses.size(); i <= index; i++) {
         addresses.add(new Address());
       }
    }
    return addresses.get(index);
  }

  public void setAddresses(final int index, final Address address) {
    this.addresses.add(index, address);
  }
}

Address.java

public class Address {
 private String postCode;

 public Address() {
 }

 public String getPostCode() {
  return postCode;
 }

 public void setPostCode(String postCode) {
  this.postCode = postCode;
 }
} 

A simple test to populate the Address collection revealed what was happening

import java.util.HashMap;
import java.util.Map;
import org.apache.commons.beanutils.BeanUtils;

public class BeanPopulateTest {
   public static void main(String[] args) throws Exception {
 
        Employee employee = new Employee();
 
        Map<String,String> properties = new HashMap<String,String>();
 properties.put("addresses[0].postCode", "TES456");

 BeanUtils.populate(employee, properties);

 System.out.println(employee.getAddresses().get(0).getPostCode());
   }
}

After a few hours of investigation and looking at the BeanUtils code, I realized that Java 8 property descriptor for Employee.addresses (collections) field is different from Java 7.

BeanInfo info = Introspector.getBeanInfo(Employee.class);
PropertyDescriptor[] descriptors = info.getPropertyDescriptors();
for (int i = 0; i < descriptors.length; i++) {
  System.out.println(descriptors[i].getName() + " " + descriptors[i].getClass().getName());
}

The output from the above code is

Java 8
java.beans.PropertyDescriptor:addresses

Java 7
java.beans.IndexedPropertyDescriptor:addresses

This causes BeanUtils to process the address collection as a non-indexed property and invoke the get(index) method on the collection, causing the IndexOutOfBoundsException.

A simple workaround for the issue is to change the names of the default getter/setter methods for the addresses collection, from getAddresses/setAddresses to getAddressList/setAddressList. All references to the existing method calls will have to be replaced with the new method names.

Employee.java (fix)

import java.util.ArrayList;
import java.util.List;

public class Employee {

 private List addresses = new ArrayList<Address>();

 public List<Address> getAddressList() {
  return addresses;
 }

 public void setAddressList(List<Address> addresses) {
  this.addresses = addresses;
 }

 public Address getAddresses(final int index) {
  if (index <= addresses.size()) {
   for (int i = addresses.size(); i <= index; i++) {
    addresses.add(new Address());
   }
  }
  return addresses.get(index);
 }

 public void setAddresses(final int index, final Address address) {
  this.addresses.add(index, address);
 }
}




Thursday 24 December 2015

Java SSL handshake with Server Name Identification (SNI)

SNI (Server Name Indication) was an extension added to TLS, to support multiple digital certificates per host name on a single IP. The SNI headers indicates which host is the client trying to connect as, allowing the server to return the appropriate digital certificate to the client.

Java handling of SNI 

 

Support for SNI extension was added in Java 7. With this addition, every time a SSL connection handshake was initiated, the server_name extension header was added by default, which ensure the client receives the correct server certificate(public key). This support exists in Java 8, however, there is a specific scenario where this header isn't sent.

When connecting to a server over a HTTPS url, the client performs a SSL handshake with the server, to validate the server's identity. The server sends a  server certificate (public key) and the client validates the certificate against an existing trust store (a storage for public keys of servers that the client trusts). The involves, apart from other things, matching the host name in the URL with the common name(CN) provided in the server certificate. If the host name in the URL does not match the server certificate common name (CN), you can provide a host name verifier implementation to perform custom validation and accordingly accept or deny the connection.

HttpsURLConnection.setDefaultHostnameVerifier(new HostnameVerifier() {
   @Override
   public boolean verify(String hostName, SSLSession session) {
      return true;
   }
});

This is where things start to go wrong. Whenever a custom HostNameVerifier is provided, java 8 fails to add the SNI extension header and throws the following exception.

javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Exception in thread "main" javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
    at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1949)
    at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:302)
    at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296)
    at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1506)
    at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)
    at sun.security.ssl.Handshaker.processLoop(Handshaker.java:979)
    at sun.security.ssl.Handshaker.process_record(Handshaker.java:914)
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1062)
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
    at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
    at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1512)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1440)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)

Note that this problem does not occur if the domain name is the only name mapped to the IP.

 

Solution 

 

To fix the problem, we need to ensure that each handshake includes the SNI header. This can be accomplished by implementing a SSLSocketFactory wrapper that adds supports adding custom parameters for every socket created

import java.io.IOException;
import java.net.InetAddress;
import java.net.Socket;
import java.net.UnknownHostException;
import javax.net.ssl.SSLParameters;
import javax.net.ssl.SSLSocket;
import javax.net.ssl.SSLSocketFactory;

public class SSLSocketFactoryWrapper extends SSLSocketFactory {

    private final SSLSocketFactory wrappedFactory;
    private final SSLParameters sslParameters;

    public SSLSocketFactoryWrapper(SSLSocketFactory factory, SSLParameters sslParameters) {
        this.wrappedFactory = factory;
        this.sslParameters = sslParameters;
    }

    @Override
    public Socket createSocket(String host, int port) throws IOException, UnknownHostException {
        SSLSocket socket = (SSLSocket) wrappedFactory.createSocket(host, port);
        setParameters(socket);
        return socket;
    }

    @Override
    public Socket createSocket(String host, int port, InetAddress localHost, int localPort)
            throws IOException, UnknownHostException {
        SSLSocket socket = (SSLSocket) wrappedFactory.createSocket(host, port, localHost, localPort);
        setParameters(socket);
        return socket;
    }


    @Override
    public Socket createSocket(InetAddress host, int port) throws IOException {
        SSLSocket socket = (SSLSocket) wrappedFactory.createSocket(host, port);
        setParameters(socket);
        return socket;
    }

    @Override
    public Socket createSocket(InetAddress address, int port, InetAddress localAddress, int localPort) throws IOException {
        SSLSocket socket = (SSLSocket) wrappedFactory.createSocket(address, port, localAddress, localPort);
        setParameters(socket);
        return socket;

    }

    @Override
    public Socket createSocket() throws IOException {
        SSLSocket socket = (SSLSocket) wrappedFactory.createSocket();
        setParameters(socket);
        return socket;
    }

    @Override
    public String[] getDefaultCipherSuites() {
        return wrappedFactory.getDefaultCipherSuites();
    }

    @Override
    public String[] getSupportedCipherSuites() {
        return wrappedFactory.getSupportedCipherSuites();
    }

    @Override
    public Socket createSocket(Socket s, String host, int port, boolean autoClose) throws IOException {
        SSLSocket socket = (SSLSocket) wrappedFactory.createSocket(s, host, port, autoClose);
        setParameters(socket);
        return socket;
    }

    private void setParameters(SSLSocket socket) {
        socket.setSSLParameters(sslParameters);
    }

}
This wrapper factory can be used in place of the default SSL Socket Factory as below

URL url = new URL("https://host.com");
SSLParameters sslParameters = new SSLParameters();
List sniHostNames = new ArrayList(1);
sniHostNames.add(new SNIHostName(url.getHost()));
sslParameters.setServerNames(sniHostNames);
SSLSocketFactory wrappedSSLSocketFactory = new SSLSocketFactoryWrapper(sslContext.getSocketFactory(), sslParameters);
HttpsURLConnection.setDefaultSSLSocketFactory(wrappedSSLSocketFactory); 
Any HTTPS connections from now on will have the server_name extension set to the host name. This can be verified by running the program with SSL debug flag on (-Djavax.net.debug=ssl:handshake)

Extension elliptic_curves, curve names: {secp256r1, sect163k1, sect163r2, secp192r1, secp224r1, sect233k1, sect233r1, sect283k1, sect283r1, secp384r1, sect409k1, sect409r1, secp521r1, sect571k1, sect571r1, secp160k1, secp160r1, secp160r2, sect163r1, secp192k1, sect193r1, sect193r2, secp224k1, sect239k1, secp256k1}
Extension ec_point_formats, formats: [uncompressed]
Extension signature_algorithms, signature_algorithms: SHA512withECDSA, SHA512withRSA, SHA384withECDSA, SHA384withRSA, SHA256withECDSA, SHA256withRSA, SHA224withECDSA, SHA224withRSA, SHA1withECDSA, SHA1withRSA, SHA1withDSA, MD5withRSA
Extension server_name, server_name: [type=host_name (0), value=host.com]

Note: I came across this error when creating a SOAP client using apache CXF SOAP module. The CXF SOAP client by default overrides the default host name verifier with its custom implementation.