Passing S3cmd commands As User Data To Ec2

Question

i am having one AWS EC2 instance. From this EC2 instance i am creating slave EC2 instances.

And while creating slave instances i am passing user data to new slave instance.In that user data i have written code for creating new directory in EC2 instance and downloading file from S3 bucket.

but problem is that, script creates new directory on EC2 instance but it Fails to download file from S3 bucket.

User Data Script :-

#! /bin/bash
cd /home 
mkdir pravin 
s3cmd get s3://bucket/usr.sh >> download.log

As shown above,in this code mkdir pravin create directory but s3cmd get s3://bucket/usr.sh fails to download file and download.log file also gets created but it remains empty.

How can i solve this proble, ? (AMI used for this is preconfigured with s3cmd)

Answer 1

Are you by chance running Ubuntu? Then Shlomo Swidler's question Python s3cmd only runs from login shell, not during startup sequence might apply exactly:

The s3cmd Python script (this one: http://s3tools.org/s3cmd ) seems to only work when run via an interactive login session, but not when run via scripts during the boot process.

Mitch Garnaat suggests that one should always beware of environmental differences inflicted by executing code within User-Data Scripts:

It's probably related to some difference in your environment when you are logged in as opposed to when the script is running as part of the startup sequence. I have run into similar problems with cron jobs.

This turned out to be the problem indeed, Shlomo Swidler summarizes the 'root cause' and a solution further down in this thread:

Mitch, your comment helped me realize what's different about the startup sequence: the operative user is root. When I log in, I'm the "ubuntu" user.

s3cmd looks in the current user's ~/.s3cfg - which didn't exist in /root/.s3cfg, only in /home/ubuntu/.s3cfg.

Luckily s3cmd allow you to specify the config file's location with --config /home/ubuntu/.s3cfg .