Why does SSL handshake give 'Could not generat

Question

When I make an SSL connection with some IRC servers (but not others - presumably due to the server's preferred encryption method) I get the following exception:

Caused by: java.lang.RuntimeException: Could not generate DH keypair
    at com.sun.net.ssl.internal.ssl.DHCrypt.<init>(DHCrypt.java:106)
    at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverKeyExchange(ClientHandshaker.java:556)
    at com.sun.net.ssl.internal.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:183)
    at com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Handshaker.java:593)
    at com.sun.net.ssl.internal.ssl.Handshaker.process_record(Handshaker.java:529)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:893)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1138)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1165)
    ... 3 more

Final cause:

Caused by: java.security.InvalidAlgorithmParameterException: Prime size must be multiple of 64, and can only range from 512 to 1024 (inclusive)
    at com.sun.crypto.provider.DHKeyPairGenerator.initialize(DashoA13*..)
    at java.security.KeyPairGenerator$Delegate.initialize(KeyPairGenerator.java:627)
    at com.sun.net.ssl.internal.ssl.DHCrypt.<init>(DHCrypt.java:100)
    ... 10 more

An example of a server that demonstrates this problem is aperture.esper.net:6697 (this is an IRC server). An example of a server that does not demonstrate the problem is kornbluth.freenode.net:6697. [Not surprisingly, all servers on each network share the same respective behaviour.]

My code (which as noted does work when connecting to some SSL servers) is:

    SSLContext sslContext = SSLContext.getInstance("SSL");
    sslContext.init(null, trustAllCerts, new SecureRandom());
    s = (SSLSocket)sslContext.getSocketFactory().createSocket();
    s.connect(new InetSocketAddress(host, port), timeout);
    s.setSoTimeout(0);
    ((SSLSocket)s).startHandshake();

It's that last startHandshake that throws the exception. And yes there is some magic going on with the 'trustAllCerts'; that code forces the SSL system not to validate certs. (So... not a cert problem.)

Obviously one possibility is that esper's server is misconfigured, but I searched and didn't find any other references to people having problems with esper's SSL ports, and 'openssl' connects to it (see below). So I'm wondering if this is a limitation of Java default SSL support, or something. Any suggestions?

Here's what happens when I connect to aperture.esper.net 6697 using 'openssl' from commandline:

~ $ openssl s_client -connect aperture.esper.net:6697
CONNECTED(00000003)
depth=0 /C=GB/ST=England/L=London/O=EsperNet/OU=aperture.esper.net/CN=*.esper.net/emailAddress=[e-mail]
verify error:num=18:self signed certificate
verify return:1
depth=0 /C=GB/ST=England/L=London/O=EsperNet/OU=aperture.esper.net/CN=*.esper.net/emailAddress=[e-mail]
verify return:1
---
Certificate chain
 0 s:/C=GB/ST=England/L=London/O=EsperNet/OU=aperture.esper.net/CN=*.esper.net/emailAddress=[e-mail]
   i:/C=GB/ST=England/L=London/O=EsperNet/OU=aperture.esper.net/CN=*.esper.net/emailAddress=[e-mail]
---
Server certificate
-----BEGIN CERTIFICATE-----
[There was a certificate here, but I deleted it to save space]
-----END CERTIFICATE-----
subject=/C=GB/ST=England/L=London/O=EsperNet/OU=aperture.esper.net/CN=*.esper.net/emailAddress=[e-mail]
issuer=/C=GB/ST=England/L=London/O=EsperNet/OU=aperture.esper.net/CN=*.esper.net/emailAddress=[e-mail]
---
No client certificate CA names sent
---
SSL handshake has read 2178 bytes and written 468 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1
    Cipher    : DHE-RSA-AES256-SHA
    Session-ID: 51F1D40A1B044700365D3BD1C61ABC745FB0C347A334E1410946DCB5EFE37AFD
    Session-ID-ctx: 
    Master-Key: DF8194F6A60B073E049C87284856B5561476315145B55E35811028C4D97F77696F676DB019BB6E271E9965F289A99083
    Key-Arg   : None
    Start Time: 1311801833
    Timeout   : 300 (sec)
    Verify return code: 18 (self signed certificate)
---

As noted, after all that, it does connect successfully which is more than you can say for my Java app.

Should it be relevant, I'm using OS X 10.6.8, Java version 1.6.0_26.

Answer 1

We got the same exact exception error returned, to fix it was easy after hours surfing the internet.

We downloaded the highest version of jdk we could find on oracle.com, installed it and pointed Jboss application server to the directory of the installed new jdk.

Restarted Jboss, reprocessed, problemo fixed!!!

Answer 2

The answer above is correct, but in terms of the workaround, I had problems with the BouncyCastle implementation when I set it as preferred provider:

java.lang.ArrayIndexOutOfBoundsException: 64
    at com.sun.crypto.provider.TlsPrfGenerator.expand(DashoA13*..)

This is also discussed in one forum thread I found, which doesn't mention a solution. http://www.javakb.com/Uwe/Forum.aspx/java-programmer/47512/TLS-problems

I found an alternative solution which works for my case, although I'm not at all happy with it. The solution is to set it so that the Diffie-Hellman algorithm is not available at all. Then, supposing the server supports an alternative algorithm, it will be selecting during normal negotiation. Obviously the downside of this is that if somebody somehow manages to find a server that only supports Diffie-Hellman at 1024 bits or less then this actually means it will not work where it used to work before.

Here is code which works given an SSLSocket (before you connect it):

List<String> limited = new LinkedList<String>();
for(String suite : ((SSLSocket)s).getEnabledCipherSuites())
{
    if(!suite.contains("_DHE_"))
    {
        limited.add(suite);
    }
}
((SSLSocket)s).setEnabledCipherSuites(limited.toArray(
    new String[limited.size()]));

Nasty.

Answer 3

I've got this error with Bamboo 5.7 + Gradle project + Apache. Gradle tried to get some dependencies from one of our servers via SSL.

Solution:

1. Generate DH Param:

with OpenSSL:

openssl dhparam 1024

example output:

-----BEGIN DH PARAMETERS-----
MIGHfoGBALxpfMrDpImEuPlhopxYX4L2CFqQov+FomjKyHJrzj/EkTP0T3oAkjnS
oCGh6p07kwSLS8WCtYJn1GzItiZ05LoAzPs7T3ST2dWrEYFg/dldt+arifj6oWo+
vctDyDqIjlevUE+vyR9MF6B+Rfm4Zs8VGkxmsgXuz0gp/9lmftY7AgEC
-----END DH PARAMETERS-----

2. Append output to certificate file (for Apache - SSLCertificateFile param)

3. Restart apache

4. Restart Bamboo

5. Try to build project again

Answer 4

This is a quite old post, but if you use Apache HTTPD, you can limit the DH size. See http://httpd.apache.org/docs/current/ssl/ssl_faq.html#javadh

Answer 5

For me, the following command line fixed the issue:

java -jar -Dhttps.protocols=TLSv1.2 -Ddeployment.security.TLSv1.2=true -Djavax.net.debug=ssl:handshake XXXXX.jar

I am using JDK 1.7.0_79

Answer 6

The "Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files" answer did not work for me but The BouncyCastle's JCE provider suggestion did.

Here are the steps I took using Java 1.6.0_65-b14-462 on Mac OSC 10.7.5

1) Download these jars:

• bcprov-jdk15on-154.jar

• bcprov-ext-jdk15on-154.jar

2) move these jars to $JAVA_HOME/lib/ext

3) edit $JAVA_HOME/lib/security/java.security as follows: security.provider.1=org.bouncycastle.jce.provider.BouncyCastleProvider

restart app using JRE and give it a try