can't use gpg-agent as an ssh agent

I've been trying to get gpg-agent to work as an ssh agent, but without any luck. I'm running the KDE version of Linux Mint (Sarah).

First, I disabled the built-in ssh agent in /etc/X11/Xsession.options:

# $Id: Xsession.options 189 2005-06-11 00:04:27Z branden $
#
# configuration options for /etc/X11/Xsession
# See Xsession.options(5) for an explanation of the available options.
allow-failsafe
allow-user-resources
allow-user-xsession
#use-ssh-agent
use-session-dbus

Then, I enabled the ssh-agent support in gpg-agent:

> $ cat ~/.gnupg/gpg-agent.conf                                                                                                                            
enable-ssh-support

At this point I restarted my pc, and checked my environment variables:

> $ echo $GPG_AGENT_INFO                                                                                                                                   
/home/jeroen/.gnupg/S.gpg-agent:0:1

> $ echo $SSH_AUTH_SOCK                                                                                                                                    
/home/jeroen/.gnupg/S.gpg-agent.ssh

Just to be sure no other agents are running:

> $ ps aux | grep agent                                                                                                                                    
jeroen    5072  0.0  0.0 282604   764 ?        Ss   Nov24   0:15 /usr/bin/gpg-agent --daemon
jeroen    5265  0.0  0.1 697044 29032 ?        Sl   Nov24   0:01 /usr/lib/x86_64-linux-gnu/libexec/polkit-kde-authentication-agent-1
jeroen    9574  0.0  0.0 130292  1016 pts/6    S+   12:15   0:00 grep --color=auto --exclude-dir=.bzr --exclude-dir=CVS --exclude-dir=.git --exclude-dir=.hg --exclude-dir=.svn agent

So far, everything looks good. So let's take a look at my gpg keys:

pub   rsa2048/03AB1CB5 2015-06-12 [SC]
uid         [ultimate] Jeroen Jacobs <jeroen.jacobs@REDACTED.be>
uid         [ultimate] Jeroen Jacobs <jeroen.jacobs@REDACTED.be>
sub   rsa2048/014F9774 2015-06-12 [E]
sub   rsa2048/0FEF1A6F 2017-11-23 [A]

As you can see, an authentication subkey is present. sure gpg-agent would have picked this up, right?

ssh-add -l                                                                                                                                             
The agent has no identities.

Well, it doesn't... Already rebooted my pc again, but doesn't make a difference. I've reading almost any tutorial on the topic, and I think I did everything by the book, but I can't get it to work.

Just in case you need it, the versions of gpg2 and gpg-agent:

> $ gpg2 --version                                                                                                                                         
gpg (GnuPG) 2.1.11
libgcrypt 1.6.5
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Home: ~/.gnupg
Supported algorithms:
Pubkey: RSA, ELG, DSA, ECDH, ECDSA, EDDSA
Cipher: IDEA, 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH,
        CAMELLIA128, CAMELLIA192, CAMELLIA256
Hash: SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224
Compression: Uncompressed, ZIP, ZLIB, BZIP2

> $ gpg-agent --version                                                                                                                                    
gpg-agent (GnuPG) 2.1.11
libgcrypt 1.6.5
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Append the keygrip of your authentication subkey into ~/.gnupg/sshcontrol file.

gpg -K --with-keygrip is the command which lists all the secret keys with their keygrips.