I cannot get the users to create real certs for their servers but I'd like to do some security checks. So the following is too light because, as I read it, there is no checking on the certs.
ServicePointManager.ServerCertificateValidationCallback = delegate { return true; };
What do you recommend that I have the clients check on the x509 cert? Given that I'm using a .NET language (c#/f#).
If you're using self signed certs then the only errors you should expect is a chain error on the root (Cert. Issuer). I would suggest something like this that traps for that chain error specifically and lets all other errors fall through.
if you can check certs you could put your own validation logic in the function ValidateRemoteCertificate
If you can't get the clients to create real certs you should at least try to get them to create certs using your server. Then you can check that the certificate is valid or at least from your CA because you'll know if your CA has been compromised. If you're trusting any and all CAs there's really nothing worth checking.