Embedding an expect inside a bash script

Question

I realise similar questions have been asked before, I looked at them and try to apply what I learned and have the following script:

#!/bin/bash

if [ `hostname` = 'EXAMPLE' ]
then

/usr/bin/expect << EOD

spawn scp -rp host:~/outfiles/ /home/USERNAME/outfiles/
expect "id_rsa':"
send "PASSWORD\r"
interact

spawn scp -rp host:~/errfiles/ /home/USERNAME/errfiles/
expect "id_rsa':"
send "PASSWORD\r"
interact

expect eof

EOD

echo 'Successful download'
fi

Unfortunately it doesn't seem to work and I get an error message:

spawn scp -rp host:~/outfiles/ /home/USERNAME/outfiles/
Enter passphrase for key '/home/USERNAME/.ssh/id_rsa': interact: spawn id exp0 not open
    while executing
"interact"

I don't know what it means and why it doesn't work. However, when I wrote the above code using a not-embedded expect script:

#!/usr/bin/expect

spawn scp -rp host:~/outfiles/ /home/USERNAME/outfiles/
expect "id_rsa':"
send "PASSWORD\r"
interact

spawn scp -rp host:~/errfiles/ /home/USERNAME/errfiles/
expect "id_rsa':"
send "PASSWORD\r"
interact

It worked without any problems. So what am I doing wrong?

NOTE: Often when someone posts a question about using expect to use scp or ssh the answer given is to use rsa keys. I tried, unfortunately on one of my computers there is some crappy bug with the gnome keyring that means that I can't remove my password from the rsa-key, which is exactly why I'm trying to write the above script with an if statement. So please don't tell me to use rsa keys.

Answer 1

Your bash script is passing the expect commands on the standard input of expect. That is what the here-document <<EOD does. However, expect... expects its commands to be provided in a file, or as the argument of a -c, per the man page. Three options are below. Caveat emptor; none have been tested.

1. Process substitution with here-document:

   expect <(cat <<'EOD'
   spawn ... (your script here)
   EOD
   )
   

   The EOD ends the here-document, and then the whole thing is wrapped in a <( ) process substitution block. The result is that expect will see a temporary filename including the contents of your here-document.

   As @Aserre noted, the quotes in <<'EOD' mean that everything in your here-document will be treated literally. Leave them off to expand bash variables and the like inside the script, if that's what you want.

2. Edit Variable+here-document:

   IFS= read -r -d '' expect_commands <<'EOD' 
   spawn ... (your script here)
   interact
   EOD
   
   expect -c "${expect_commands//
   /;}"
   

   Yes, that is a real newline after // - it's not obvious to me how to escape it. That turns newlines into semicolons, which the man page says is required.

   Thanks to this answer for the read+heredoc combo.

3. Shell variable

   expect_commands='
   spawn ... (your script here)
   interact'
   expect -c "${expect_commands//
   /;}"
   

   Note that any ' in the expect commands (e.g., after id_rsa) will need to be replaced with '\'' to leave the single-quote block, add a literal apostrophe, and then re-enter the single-quote block. The newline after // is the same as in the previous option.