EC2 startup script gets stuck on wget

Question

I have the following script for some number crunching

#!/bin/bash
sudo apt-get update -y
sudo apt-get upgrade -y
sudo apt-get install -y r-base r-base-dev htop s3cmd p7zip-full
wget https://s3.amazonaws.com/#######/###.7z
7z e ###.7z
sudo R CMD BATCH --slave --no-timing --vanilla "--args 0 1 100 200 500 2" SOME-ROUTINE.R
s3cmd put *.results s3://#########/

on EC2. I upload the script as file at the Launch Instance->Instance Details->User Data

The machine fires up, updates and upgrades but then it does not execute wget and does not download the file. When i SSH in the Instance and run the exact same commands the process completes without problems.

Any ideas why wget does not work?

Any other alternatives?

EC

Answer 1

It is always a bit of guessing, but here is how I would debug this:

My first suggestion would be to check for special characters in the S3 URL. This might cause the wget call to fail.

Second, I would give an explicit output path to wget with the -O option. While you are editing the command, you can also add -o to output logging information.

Last step is to check your access rights to the S3 bucket. Perhaps you can try to put the file on another webspace to see if the command executes then.