Qt Server+Client App: encryption fails (updated wi

问题:

I have been struggling with all sorts of problems when designing the Server+Client with secure TSL connection in Qt. I am using:

Qt 5.2.0 (QSslSocket) XCA (based on OpenSSL 32-bit)

For testing I have created private key (RSA, 1024) and CA. Then a certificate signed by this CA with another private key (RSA, 1024). Common name set to 'localhost' as I am testing on one machine. I have the needed DLLs of OpenSSL in both client and server apps.

However no matter how I set things up the encrypted connection just does not work. Regular connection works fine but attempting the encryption always results in errors. Most common errors:

QSskError::CertificateSignatureFailed "The signature of the certificate is invalid"
QAbstractSocket::SslInternalError
QAbstractSocket::SslHandshakeFailedError

I might be missing something obvious or have overlooked something but I am at my wits end trying to solve this for some time now. Especially since I don't know if the problem is in my code, misusing openSSL, certificates or combination of these or something else in the mix I am not aware of.

I would appreciate any pointers to both common errors and especially reliable guides how to do this properly.

Thank you!

EDIT4: Provided minimalist code to demonstrate the problem.

EDIT3: If I move the loading of the CA certificate on client side before the connection is made and start the handshake immidiatelly after connection it actually works. However delayed handshake still does not with the same error as before (SslHandshakeFailedError). Documentation states only that CA must be loaded before handshake, not before connection...

EDIT2: This command succeeded in verifying the certificate (Verify return 0 (Ok)):

openssl s_client -connect <host>:<port> -CAfile <ca-file>

EDIT: On request, I have whipped up client app and server app using the same code as in real production minus all the stuff around it like threading, logging etc.

CLIENT

mainwindow.h

class MainWindow : public QMainWindow
{
    Q_OBJECT

public:
    explicit MainWindow(QWidget *parent = 0);
    ~MainWindow();

private slots:
    void on_pushButton_Connect_released();
    void on_pushButton_Encrypt_released();

    void socketEncrypted() { qDebug() << "Encrypted"; }

private:
    QSslSocket m_Socket;
    Ui::MainWindow *ui;

    void socketWrite(const QByteArray &data);
};

mainwindow.cpp

MainWindow::MainWindow(QWidget *parent) :
    QMainWindow(parent),
    ui(new Ui::MainWindow)
{
    ui->setupUi(this);

    connect(&m_Socket,&QSslSocket::encrypted,
            this,&MainWindow::socketEncrypted);
}

MainWindow::~MainWindow()
{
    delete ui;
}

void MainWindow::on_pushButton_Connect_released()
{
    m_Socket.connectToHost("localhost", 1234);
}

void MainWindow::on_pushButton_Encrypt_released()
{
    QFile file("PM_Client_CA.crt");
    file.open(QIODevice::ReadOnly);
    QSslCertificate certificate(&file);

    m_Socket.addCaCertificate(certificate);
    m_Socket.startClientEncryption();

    QByteArray data("encrypt");
    socketWrite(data);
}

void MainWindow::socketWrite(const QByteArray &data)
{
    m_Socket.write(data);
    m_Socket.flush();
}

SERVER

server.h

class Server : public QTcpServer
{
    Q_OBJECT
public:
    explicit Server(QObject *parent = 0);

public slots:
    void clientEncrypted() { qDebug() << "Encrypted"; }
    void clientRead();

protected:
    void incomingConnection(qintptr handle);

private:
    QSslSocket m_Socket;
};

server.cpp

Server::Server(QObject *parent) :
    QTcpServer(parent)
{
    QFile file("PM_Server.crt");
    file.open(QIODevice::ReadOnly);
    QSslCertificate certificate(&file);
    file.close();

    file.setFileName("PM_Server.pem");
    file.open(QIODevice::ReadOnly);
    QSslKey key(&file, QSsl::Rsa);

    m_Socket.setLocalCertificate(certificate);
    m_Socket.setPrivateKey(key);

    connect(&m_Socket,&QSslSocket::encrypted,
            this,&Server::clientEncrypted);

    listen(QHostAddress::Any, 1234);
}

void Server::incomingConnection(qintptr handle)
{
    m_Socket.setSocketDescriptor(handle);

    connect(&m_Socket,&QSslSocket::readyRead,
            this,&Server::clientRead);
}

void Server::clientRead()
{
    disconnect(&m_Socket,&QSslSocket::readyRead,
               this,&Server::clientRead);

    m_Socket.startServerEncryption();
}

回答1:

There are some faults in your code. First you should call the function startServerEncryption() in server side at incomingConnection after a client connects to server. If you don't call startServerEncryption() the connection is not established and the readRead signal will not emit.

You should also connect the readyRead signal of server in the slot clientEncrypted. So it should be like:

void Server::incomingConnection(qintptr handle)
{
    m_Socket.setSocketDescriptor(handle);

    m_Socket.startServerEncryption();

}

void Server::clientEncrypted()
{
    connect(&m_Socket,&QSslSocket::readyRead,
            this,&Server::clientRead);

}

Another issue is that there is no need to load certificate on client side. The cerfificate is usually loaded at server.

The last point is that you should connect the signal sslErrors of sockets to slots and call ignoreSslErrors() from there. This should be done for both client side and server side. It is like:

connect( &m_Socket, SIGNAL(sslErrors(QList<QSslError>)),
            this, SLOT(onSslError(QList<QSslError>)) );

void MainWindow::sslError(QList<QSslError> errors)
{
    m_Socket.ignoreSslErrors();
}

You can see the errors by:

void Server::sslError(QList<QSslError> errors)
{
    QString erroStr="";
    foreach (const QSslError &e, errors)
        erroStr.append(e.errorString()).append("\n");

    QMessageBox::warning( (QWidget *)this->parent(), tr("Error"),erroStr );

    m_Socket.ignoreSslErrors();
}

The last point is about loading the certificate. I have used the following code for loading license in SSL3 connection successfully. You can try it:

m_Socket.setProtocol(QSsl::SslV3);

QByteArray key;
QByteArray cert;

QFile file_key("server.key");
if(file_key.open(QIODevice::ReadOnly))
{
    key = file_key.readAll();
    file_key.close();
}
else
{
    qDebug() << file_key.errorString();
}

QFile file_cert("server.crt");
if(file_cert.open(QIODevice::ReadOnly))
{
    cert = file_cert.readAll();
    file_cert.close();
}
else
{
    qDebug() << file_cert.errorString();
}


QSslKey ssl_key(key, QSsl::Rsa,QSsl::Pem,QSsl::PrivateKey,"server");

QSslCertificate ssl_cert(cert);

m_Socket.addCaCertificate(ssl_cert);
m_Socket.setLocalCertificate(ssl_cert);
m_Socket.setPrivateKey(ssl_key);

回答2:

It seems I have figured it out. First I must thank @jww for helping me verify that the issue were not the certificates.

In fact, the issue was me setting up the socket in the wrong order AND reading up the encryption data and doing stuff based on that (like starting second handshake etc.).

Anyway, this seems to be answered now but I have run into new load of issues with the handshake. I have opened the Qt forum thread for it: http://qt-project.org/forums/viewthread/41293/

Thanks again to everyone who tried to help!