For a program I am writing, I would like to use TLS (or something similar) to encapsulate my application's protocol. This will minimize both the amount of work I have to do as well as the number of vulnerabilities I could accidentally create.
My program is designed to be peer-to-peer although one or more servers provide some services to help one user locate another (it registers IP address/port combos) but do little else. I want to make this system very fault-tolerant so having these servers act as a Certificate Authority is unacceptable because a compromise of a server or its key would affect too many users. Therefore I plan on using a Web of Trust.
The main problem with using TLS is that the original TLS 1.2 specification (RFC 5246) does not provide for using OpenPGP certificates. It seems to be very x.509 centric. RFC 6091, which obsoletes RFC 5081 and extends RFC 5246, makes provisions for an extension to TLS that does what I want. The problem is that I don't think BouncyCastle implements this extension and I can't find a Java crypto library that does. I also don't want to write my own / contribute to BC because I'm really bad at not making mistakes and I'm also very lazy.
Another problem with this is that BouncyCastle provides "a light weight client-side TLS API" but because this software is P2P, a server-side API is also necessary so that I can use TLS by making it believe that the peer originating the connection is the client. I'm pretty sure that once the handshake is complete that it's the same.
Questions:
Is there any way that I can still use TLS (which I highly doubt)? Is there a protocol like TLS that is designed for P2P, or at least can function in this way (like I believe TLS can), but can work with an OpenPGP certificate? If neither is the case, should I pursue the idea explained in this question and implement my own layer taking concepts from TLS?
Links to RFCs: RFC 5246 and RFC 6091
The only library that I know to support RFC 6091 (i.e. TLS with openpgp certificates) is GnuTLS but I don't know whether you can use something like that in Java. Alternatively you could replicate the SSH semantics, where you store the public keys of your peers using self-signed
X.509 certificates.
In TLS, the X.509 parts are actually handled as opaque blobs:
- The server sends its certificate (and some helper certificates, if it wishes so) as (a list of) opaque string(s) of bytes (a three-byte length, followed by the encoded certificate as arbitrary bytes).
- When the server asks for public key client authentication, it sends a list of "names" which are supposed to be the encoded X.500 names of the root CA the server will recognize -- there again, opaque blobs (two-byte length).
- The client, when (if) it sends a certificate (chain), uses the same format than the server.
As TLS is defined, both client and server are supposed to use the peer public key, which they get in any way they see fit and that's mostly out of scope of the TLS specification: the certificates exchanged over the wire are considered as mere helpers. So there would be no problem in actually sending OpenPGP encoded public keys in those blobs, as long as both client and server expect it -- and since you control code on both, this should be no issue.
Your problem then "simply" becomes a matter of making a TLS implementation accept to hand you the blobs without choking on them. I know of no existing Java-only TLS implementation which will fit the bill, so you may have to write a bit of code -- but I urge you not to fiddle with TLS protocol details except processing of the certificate blobs. Those things are subtle and weaknesses are sooo easy to create...
As far as I know, the Sun/Oracle JSSE implementation only deals with X.509 TrustManagers (which you can customize to handle certain extensions, but would still expect a structurally valid X.509 certificate.
It might be possible to use Java's security API to implement RFC 6091, but I'm not sure how. It's definitely more work than just tweaking the TrustManagers, as you would have to go deeper into Java's implementation of TLS.
Alternatively, if it's for a bespoke server, you could re-use the key material from PGP certificates into X.509 certificates and put the initial PGP certificate (with all its signatures) as a blob in a custom X.509 extension (as it's more or less done here). The problem here would be interoperability, since such an extension wouldn't be a standard. Implementing a TrustManager in Java that is able to understand extension is definitely feasible, and you wouldn't need to dig into the internals of Java's TLS stack, you'd only have to deal with custom TrustManagers to initialize your SSLContexts.