It seems to show this dialog two or three times. If I cancel it goes away and I'm still able to commit changes.
How do I prevent it from popping up each time and where do I find my passphrase if I have to reenter it and click "Remember to keychain" (which I've done before but will try again)?
In the latest version of macOS - 10.12.2/Sierra - this is an easy fix.
Just edit your ~/.ssh/config and enable the UseKeychain option:
Host *
UseKeychain yes
Save and solved.
If your ssh key has been protected with a passphrase, then you need to run an ssh-agent first, in order to avoid entering said passphrase for every connections.
See GitHub article "Working with SSH key passphrases".
Initializing new SSH agent...
# succeeded
# Enter passphrase for /c/Users/you/.ssh/id_rsa:
# Identity added: /c/Users/you/.ssh/id_rsa (/c/Users/you/.ssh/id_rsa)
# Welcome to Git (version 1.6.0.2-preview20080923)
#
# Run 'git help git' to display the help index.
# Run 'git help ' to display help for specific commands.
If you don't have your passphrase anymore, it is better to re-create a new set of public/private ssh keys (protected, if you want, with a new passphrase), and publish that new public key on your GitHub account.
Ricardo's solution is correct for macOS Sierra v. 10.12.2. I still needed to enter the passphrase the first time I ran the ssh-agent
process, but not again after that. If you are doing this in a terminal instead of a GUI, be sure to either restart your session or reinitialize it with . ~/.ssh/config