In a TLS handshake configured with a client authentication, there is a step where the server receives the client's certificate and choose to trust it or not (for instance, in Java it is done via a TrustManager).
I would like to know if the eventual "trust failure" message from the server is sent before or after the server made sure that the client really own that public key (for example, by receiving first some messages from the handshake encoded with the client's private key).
The purpose of my question is to see if it is possible for a third party to check if the server trust a client, by pretending to be this client and by using his public key.
Note: The risk is real when TLS is used in a context with specific security requirements. For instance, let's suppose a P2P application which uses TLS between peers, and which use the TrustManager as a way to authenticate peers from his contact list. This contact list is supposed to be private. An ISP can list the IPs with who a node communicates, then get his public certificate by starting a TLS handshake with it, then he can try to connect each another nodes on the IP list. In the end, the ISP can get a big part of the contact list which was supposed to be private.
OpenSSL verifies the client certificate, too, immediately upon receiving it in the Client Certificate message.
But it is as Eugene says, if the server sends meaningful alerts, then it does not matter if you send bad_certificate
right away or only after having verified the signature in the Certificate Verify message. This would only prevent someone from finding out whether a certificate is trusted or not if they additionally send a malformed signature (e.g. by using the wrong key). But if a server were implemented that way, all you had to do is sign your Certificate Verify message with a private key you just generated. Then the signature will be valid and the server will then dutifully validate the certificate you sent, revealing the same information as before.
To mitigate this situation you would really have to use a customized server that does not send the corresponding alert at all, but rather something less revealing.
This depends on implementation. Our implementation sends the error immediately, as for other implementations - I guess most do the same.
However it doesn't matter: the server sends specific error code (BadCertificate) if the certificate is not valid, so no matter when this code is sent, the attacker would know that the certificate has not been accepted. Protecting the server from this attack would require the server send a different error code and this would confuse legitimate clients.
The risk (or unpleasant consequences) of detecting that the certificate is accepted by the server or not is questionable. If this matters to you, you can change the error code and build your custom version of OpenSSL or other SSL server module you use.