jenkins - can't ssh to remote server (key - pe

Question

I have Jenkins running on my local machine trying to figure out remote ssh problem I'm having on a server. I'm getting this permission denied error which indicates a problem with the key, yet from the same user account on the shell, I can definitely connect.

Started by user anonymous
Building in workspace /Users/jgoodwin/jenkins/workspace/app
[postprocessor] $ /bin/sh -xe /var/folders/b0/h_wtmzss6cx11p6153y9h2cr0000gn/T/hudson4163212101874527747.sh
+ echo /Users/jgoodwin
/Users/jgoodwin
+ whoami
jgoodwin
+ ssh -i /Users/jgoodwin/.ssh/id_rsa remoteuser@server 'echo success'
Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
Build step 'Execute shell' marked build as failure
Finished: FAILURE

This is run right on the shell:

Jasons-MacBook-Air:~ jgoodwin$ echo $HOME
/Users/jgoodwin
Jasons-MacBook-Air:~ jgoodwin$ whoami
jgoodwin
Jasons-MacBook-Air:~ jgoodwin$ ssh -i /Users/jgoodwin/.ssh/id_rsa remoteuser@server 'echo success'
success

I'm fairly stumped - I've done quite a bit of work with hudson in the past and I don't think I had any issues doing this type of work. The error indicates a problem with the keys but they are clearly fine.

Edit:

Verbose log as per request

OpenSSH_5.9p1, OpenSSL 0.9.8x 10 May 2012
debug1: Reading configuration data /etc/ssh_config
debug1: /etc/ssh_config line 20: Applying options for *
debug1: /etc/ssh_config line 53: Applying options for *
debug1: Connecting to hostname [ip] port 22.
debug1: Connection established.
debug1: identity file /Users/jgoodwin/.ssh/id_rsa type 1
debug1: identity file /Users/jgoodwin/.ssh/id_rsa-cert type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3
debug1: match: OpenSSH_5.3 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.9
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Server host key: RSA ed:d4:92:3f:33:bd:dd:b9:eb:d1:b2:19:4c:f1:70:e9
debug1: Host 'hostname' is known and matches the RSA host key.
debug1: Found key in /Users/jgoodwin/.ssh/known_hosts:6
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /Users/jgoodwin/.ssh/id_rsa
debug1: Server accepts key: pkalg ssh-rsa blen 279
debug1: key_parse_private_pem: PEM_read_PrivateKey failed
debug1: read PEM private key done: type <unknown>
debug1: read_passphrase: can't open /dev/tty: Device not configured
debug1: No more authentication methods to try.
Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
Build step 'Execute shell' marked build as failure

Edit: successful attempt added 8/15

OpenSSH_5.9p1, OpenSSL 0.9.8x 10 May 2012
debug1: Reading configuration data /etc/ssh_config
debug1: /etc/ssh_config line 20: Applying options for *
debug1: /etc/ssh_config line 53: Applying options for *
debug1: Connecting to hostname [ip] port 22.
debug1: Connection established.
debug1: identity file /Users/jgoodwin/.ssh/id_rsa type 1
debug1: identity file /Users/jgoodwin/.ssh/id_rsa-cert type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3
debug1: match: OpenSSH_5.3 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.9
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Server host key: RSA 40:bf:b5:74:1c:5f:b6:93:00:4b:ca:1d:fc:0f:39:ec
debug1: Host 'hostname' is known and matches the RSA host key.
debug1: Found key in /Users/jgoodwin/.ssh/known_hosts:3
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /Users/jgoodwin/.ssh/id_rsa
debug1: Server accepts key: pkalg ssh-rsa blen 279
debug1: Authentication succeeded (publickey).
Authenticated to hostname ([54.226.250.218]:22).
debug1: channel 0: new [client-session]
debug1: Requesting [e-mail]
debug1: Entering interactive session.
debug1: Sending environment.
debug1: Sending env LANG = en_CA.UTF-8
Last login: Thu Aug 15 13:09:32 2013 from 66.199.39.230

Answer 1

Check whether the home directory is encrypted : http://gopukrish.wordpress.com/2013/04/24/ssh-cant-connect-authentications-that-can-continue-publickeygssapi-keyexgssapi-with-micpassword/

Answer 2

In my situation, I was using a keychain (as @coffeebreaks suggested) which was being setup when I sourced in the jenkins user .bashrc file. Unfortunately, the jenkins backend doesn't seem to source this file like a standard shell login does.

The solution was to add the following code prior to the scp call in the Jenkins pipeline:

. ~/.bashrc

Answer 3

Multiple reasons could cause this behavior, like key caching with agent / keychain manager, etc.

I would suggest to use the -v argument to compare the 2 outputs:

ssh -v -i /Users/jgoodwin/.ssh/id_rsa remoteuser@server

This will let you compare what is going in in a more verbose way. If you still cannot solve it, please post your verbose outputs to compare.

Note: you can add up to 3 -v arguments for increased verbosity.

Updated:

@JasonG From what I see the details of the failure are:

debug1: Offering RSA public key: /Users/jgoodwin/.ssh/id_rsa debug1: Server accepts key: pkalg ssh-rsa blen 279 debug1: key_parse_private_pem: PEM_read_PrivateKey failed debug1: read PEM private key done: type debug1: read_passphrase: can't open /dev/tty: Device not configured

It looks like your key has a passphrase and the passphrase cannot be entered because we are not in an interactive shell. The command line from your standard shell may benefit from the Keycahin which would 'type the passphrase' for you.

If you could generate the same verbosity for your successful command so that we could compare...

Answer 4

Jenkins run shell scripts is slight different with cmd line on environment

there are some environment difference in your case, which we didn't notice. like initial script, path setting.

besides the method provided by @coffeebreaks, try below

• check the system environment, like show command env
• write above steps into bash scripts and run the scripts both in cmd line and jenkins jobs
• use another user instead of your initial user who starts jenkins instance