SAML 2.0 - How to verify the sender certificate?

I implement a SAML SP in Java.
I send an AuthnRequest to SAML 2.0 IDP and gets an encrypted response.
My question is:
How do I make sure that the response indeed comes from the IDP and not from a hacker?
It is not enough to validate the signature, since this only tells me that the sender has a matching pair of private/public keys, but it could be anyone.
So, I need the IDP to supply me in advance a certificate which I upload to a jks file, and compare it each time to the certificate I extract from the ds:X509Certificate element of the response.
Now, is there a standard way of comparing the sender's certificates with the one stored in my keystore?
I saw the following code:

 KeyStore keyStore = getKS();
 PKIXParameters params = new PKIXParameters(keyStore);
 params.setRevocationEnabled(false);
 CertPath certPath = certificateFactory.generateCertPath(Arrays.asList(certFromResponse));
 CertPathValidator certPathValidator = CertPathValidator.getInstance(CertPathValidator.getDefaultType());
 CertPathValidatorResult result = certPathValidator.validate(certPath, params);

Is it enough? If the validation doesn't throw an exception it verifies the sender's identity?

This is the way i have solved the verification of signatures with OpenSAML

http://blog.samlsecurity.com/2012/11/verifying-signatures-with-opensaml.html

I have also written a book, A Guide to OpenSAML, where I explain in detail encryption and signing and more using OpenSAML.

What is important with the OpenSAML verification methods is that they only verify the cryptographic validity of the signature (That the content has not been changed). It does not however verify that the sender is someone that you trust.

The Signature validator is instantiated with the public key of the sender to validate against, the public key of the sender. This is normally exchanged is the setup of an identity federation using SAML Metadata