multiple heroku accounts not working

I'll be very detailed, as I followed the advice on a related question and it didn't work.

I'm on Mac OS X 10.7 trying to make two heroku accounts work. Let's call these accounts first and second

To start clean, I removed the keys in ~.ssh/, run heroku keys:remove on both accounts to get rid of existing keys, and run ssh-add -D to remove existing entries (in other words: ~.ssh/ is empty, except for known_hosts; heroku keys says I have no keys, and ssh-add -L says there are no entries.

I heroku login into the first account and I run these:

ssh-keygen -t rsa -C "first@example.com" -f  ~/.ssh/id_rsa_first
ssh-add ~/.ssh/id_rsa_first
heroku keys:add ~/.ssh/id_rsa_first.pub

then I heroku logout and heroku login into the second account and I run these:

ssh-keygen -t rsa -C "second@example.com" -f  ~/.ssh/id_rsa_second
ssh-add ~/.ssh/id_rsa_second
heroku keys:add ~/.ssh/id_rsa_second.pub

In both cases I note down the key fingerprints, which I'll call fingerprint_first and fingerprint_second.

Now I heroku login to first and git push heroku master works fine, then heroku login to second on a second app and git push heroku master fails, with the error

!  Your key with fingerprint fingerprint_first is not authorized to access second-app-name.

If I remove that from the ssh list with ssh-add -d ~/.ssh/id_rsa_first, then git push heroku master works on the second app, but now if I go back to the first app and heroku login to the first account, git push heroku master of course fails.

Please help :)