SSL handshake failures when no data was sent over

I start looking at implementing explicit FTP by extending the current Twisted FTP.

Most of the code was straight forward and implementing AUTH, PBSZ, PROT was easy and I got a working secured control channel.

My problem is with the data channel.

The client side error is : SSL routines', 'SSL3_READ_BYTES', 'ssl handshake failure'

It looks like SSL handshake and shutdown are called only when some data was send over the data channel. This affect the case when sending empty files or listing empty folders, since before closing the connection, the client will call the SSL shutdown.

I am looking after some suggestion for how and where I should search for fixing the TLS handshake from Twisted TLS when no data is sent.

This code works when listing folders that are not empty... but will fail if the folder contains no files or folders.

Many thanks!

def getDTPPort(self, factory):
    """
    Return a port for passive access, using C{self.passivePortRange}
    attribute.
    """
    for portn in self.passivePortRange:
        try:
            if self.protected_data:
                dtpPort = reactor.listenSSL(
                    port=portn, factory=factory,
                    contextFactory=self.ssl_context)
            else:
                dtpPort = self.listenFactory(portn, factory)

        except error.CannotListenError:
            continue
        else:
            return dtpPort
    raise error.CannotListenError('', portn,
        "No port available in range %s" %
        (self.passivePortRange,))

Update 1

I will update this text since comments are not well formated:

So I ended up with:

def getDTPPort(self, factory):
    """
    Return a port for passive access, using C{self.passivePortRange}
    attribute.
    """
    for portn in self.passivePortRange:
        try:
            if self.protected_data:
                tls_factory = TLSMemoryBIOFactory(
                    contextFactory=self.ssl_context,
                    isClient=False,
                    wrappedFactory=factory)
                dtpPort = reactor.listenTCP(
                    port=portn, factory=tls_factory)
            else:
                dtpPort = self.listenFactory(portn, factory)

        except error.CannotListenError:
            continue
        else:
            return dtpPort
    raise error.CannotListenError('', portn,
        "No port available in range %s" %
        (self.passivePortRange,))

Update 2

The problem is caused by the fact that the connection is closed while the handshake is still running. I don't know how check on an empty connection that the SSL handshake was done.

So I ended up with this stupid code

def loseConnection(self):
    """
    Send a TLS close alert and close the underlying connection.
    """
    self.disconnecting = True

    def close_connection():
        if not self._writeBlockedOnRead:
            self._tlsConnection.shutdown()
            self._flushSendBIO()
            self.transport.loseConnection()

    # If we don't know if the handshake was done, we wait for a bit
    # and the close the connection.
    # This is done to avoid closing the connection in the middle of a
    # handshake.
    if not self._handshakeDone:
        reactor.callLater(0.1, close_connection)
    else:
        close_connection()

The SSL handshake is initiated by the do_handshake method of the pyOpenSSL Connection object. It can also be initiated implicitly by a send or recv call. The transport set up by reactor.connectSSL and reactor.listenSSL relies on the latter. So your conclusion is correct - the handshake is never performed if no data is sent over the connection.

However, twisted.protocols.tls calls do_handshake as soon as the connection is made. If you set up your SSL server with that API instead, I think you'll see your problem resolved.

There is also a plan to reimplement the former using the latter, since the latter seems to work better in general.