How do I stop Elastic Beanstalk (apparently) remov

Question

I have these container commands in an .ebextensions .config file:

container_commands:
  download_geography_data_01:
    command: "python scripts/download_geography_data.py"
  download_geography_data_02:
    command: "file /opt/python/current/app/data/geography/geography.sqlite"

And these execute perfectly when I use eb create - the second command is just there to put an entry in the log proving that the first script ran correctly and the file was created. I get a line like this in eb_activity.log, exactly as I would expect:

Completed activity. Result: /opt/python/current/app/data/geography/geography.sqlite: SQLite 3.x database

But 2 minutes later, it has my Python web server app is up and running and when it attempts to read that file, it fails:

IOError: geography data file not found. Path searched was /opt/python/current/app/data/geography/geography.sqlite

I then use eb ssh to log in to my environment, and it's right - the file is not there at all. It has seemingly disappeared between the configuration stage and the execution stage.

Anybody have any idea what could be going wrong? Could it somehow be clearing this file away?

I'm using the "64bit Amazon Linux 2015.03 v1.4.3 running Python 2.7" platform, if that is relevant.

Answer 1

AWS Elastic Beanstalk have never removed your file, except for the app directory.

As AWS Documentation said in Customizing Software on Linux Servers:

You can use the container_commands key to execute commands for your container. The commands in container_commands are processed in alphabetical order by name. They run after the application and web server have been set up and the application version file has been extracted, but before the application version is deployed. They also have access to environment variables such as your AWS security credentials. Additionally, you can use leader_only. One instance is chosen to be the leader in an Auto Scaling group. If the leader_only value is set to true, the command runs only on the instance that is marked as the leader.

You put your SQLite database inside your app directory. When you update to a new one, your change into your old directory will be destroyed.

Before deploying:

[ec2-user@ip-172-31-52-184 python]$ ll /opt/python/
total 20
drwxr-xr-x 2 root root 4096 Jul  30 09:09 bin
drwxr-xr-x 3 root root 4096 Jul  30 09:12 bundle
lrwxrwxrwx 1 root root   20 Jul  30 09:12 current -> /opt/python/bundle/3
drwxr-xr-x 2 root root 4096 Jul  30 09:09 etc
drwxrwxr-x 3 root wsgi 4096 Jul  30 09:10 log
drwxr-xr-x 4 root root 4096 Jul  30 09:10 run

On deploying:

[ec2-user@ip-172-31-52-184 python]$ ll /opt/python/
total 20
drwxr-xr-x 2 root root 4096 Jul  30 09:09 bin
drwxr-xr-x 4 root root 4096 Jul  30 09:18 bundle
lrwxrwxrwx 1 root root   20 Jul  30 09:12 current -> /opt/python/bundle/3
drwxr-xr-x 2 root root 4096 Jul  30 09:09 etc
drwxrwxr-x 3 root wsgi 4096 Jul  30 09:10 log
lrwxrwxrwx 1 root root   20 Jul  30 09:18 ondeck -> /opt/python/bundle/4

After deployed:

[ec2-user@ip-172-31-52-184 python]$ ll /opt/python/
total 20
drwxr-xr-x 2 root root 4096 Jul  30 09:09 bin
drwxr-xr-x 3 root root 4096 Jul  30 09:18 bundle
lrwxrwxrwx 1 root root   20 Jul  30 09:18 current -> /opt/python/bundle/4
drwxr-xr-x 2 root root 4096 Jul  30 09:09 etc
drwxrwxr-x 3 root wsgi 4096 Jul  30 09:10 log
drwxr-xr-x 4 root root 4096 Jul  30 09:10 run

Do you notice your /opt/python/current/app changed? While on deploying, your /opt/python/current/app is pointed to /opt/python/bundle/3. And then after deployed, it is pointed to /opt/python/bundle/4. And then, the /opt/python/bundle/3 (your old app) is removed.

So, the solution might be:

• Don't put your database inside your app directory. Just put anywhere outside your app directory.
• Run the command after deployed (see here).

UPDATE: First time deployment and first time update

TLDR: If you create a new environment, the /opt/python/current was pointed to /opt/python/bundle/1 and then it will be updated to /opt/python/bundle/2. So, before your app is deployed, there is a "default" app inside the instance.

I create a simple .ebextensions script to list all object inside /opt/python:

.ebextensions/env-check.config

files:
  "/opt/env-checker/env-checker.sh" :
    content : |
      #!/usr/bin/env bash
      TIMESTAMP=`date +%Y%m%d_%H%M%S_%N`
      if [ -d /opt/python ]
      then
        echo "PWD: $PWD" >> /opt/env-checker/${TIMESTAMP}_${1}
        ls -al /opt/python >> /opt/env-checker/${TIMESTAMP}_${1}
      else
        echo "PWD: $PWD" >> /opt/env-checker/${TIMESTAMP}_${1}
        ls -al /opt >> /opt/env-checker/${TIMESTAMP}_${1}
      fi
    mode : "000755"
    owner : root
    group : root
  "/opt/elasticbeanstalk/hooks/appdeploy/post/post-check.sh":
    mode: "000755"
    owner: root
    group: root
    content: |
      #!/usr/bin/env bash
      /opt/env-checker/env-checker.sh 02_post_deploy

commands:
  000_create_post_dir:
    command: "mkdir /opt/elasticbeanstalk/hooks/appdeploy/post"
    ignoreErrors: true
  100_commands_envc:
    command: /opt/env-checker/env-checker.sh 00_commands

container_commands:
  101_container_commands_envc:
    command: /opt/env-checker/env-checker.sh 01_container_commands

In the first deployment (create a new environment using the above .ebentensions), I got this:

20150730_141343_068692315_00_commands:

PWD: /
total 28
drwxr-xr-x 7 root root 4096 Jul 30 14:13 .
drwxr-xr-x 6 root root 4096 Jul 30 14:13 ..
drwxr-xr-x 2 root root 4096 Jul 30 14:13 bin
drwxr-xr-x 3 root root 4096 Jul 30 14:13 bundle
lrwxrwxrwx 1 root root   20 Jul 30 14:13 current -> /opt/python/bundle/1
drwxr-xr-x 2 root root 4096 Jul 30 14:13 etc
drwxrwxr-x 3 root wsgi 4096 Jul 30 14:13 log
drwxr-xr-x 4 root root 4096 Jul 30 14:13 run

20150730_141350_596104854_01_container_commands

PWD: /opt/python/bundle/2/app
total 28
drwxr-xr-x 7 root root 4096 Jul 30 14:13 .
drwxr-xr-x 6 root root 4096 Jul 30 14:13 ..
drwxr-xr-x 2 root root 4096 Jul 30 14:13 bin
drwxr-xr-x 4 root root 4096 Jul 30 14:13 bundle
lrwxrwxrwx 1 root root   20 Jul 30 14:13 current -> /opt/python/bundle/1
drwxr-xr-x 2 root root 4096 Jul 30 14:13 etc
drwxrwxr-x 3 root wsgi 4096 Jul 30 14:13 log
lrwxrwxrwx 1 root root   20 Jul 30 14:13 ondeck -> /opt/python/bundle/2
drwxr-xr-x 4 root root 4096 Jul 30 14:13 run

20150730_141355_731024404_02_post_deploy

PWD: /
total 28
drwxr-xr-x 7 root root 4096 Jul 30 14:13 .
drwxr-xr-x 6 root root 4096 Jul 30 14:13 ..
drwxr-xr-x 2 root root 4096 Jul 30 14:13 bin
drwxr-xr-x 3 root root 4096 Jul 30 14:13 bundle
lrwxrwxrwx 1 root root   20 Jul 30 14:13 current -> /opt/python/bundle/2
drwxr-xr-x 2 root root 4096 Jul 30 14:13 etc
drwxrwxr-x 3 root wsgi 4096 Jul 30 14:13 log
drwxr-xr-x 4 root root 4096 Jul 30 14:13 run

In the second deployment (first update), I got this:

20150730_141535_631911395_00_commands

PWD: /
total 28
drwxr-xr-x 7 root root 4096 Jul 30 14:13 .
drwxr-xr-x 6 root root 4096 Jul 30 14:13 ..
drwxr-xr-x 2 root root 4096 Jul 30 14:13 bin
drwxr-xr-x 3 root root 4096 Jul 30 14:13 bundle
lrwxrwxrwx 1 root root   20 Jul 30 14:13 current -> /opt/python/bundle/2
drwxr-xr-x 2 root root 4096 Jul 30 14:13 etc
drwxrwxr-x 3 root wsgi 4096 Jul 30 14:13 log
drwxr-xr-x 4 root root 4096 Jul 30 14:13 run

20150730_141542_258594223_01_container_commands

PWD: /opt/python/bundle/3/app
total 28
drwxr-xr-x 7 root root 4096 Jul 30 14:15 .
drwxr-xr-x 6 root root 4096 Jul 30 14:13 ..
drwxr-xr-x 2 root root 4096 Jul 30 14:13 bin
drwxr-xr-x 4 root root 4096 Jul 30 14:15 bundle
lrwxrwxrwx 1 root root   20 Jul 30 14:13 current -> /opt/python/bundle/2
drwxr-xr-x 2 root root 4096 Jul 30 14:13 etc
drwxrwxr-x 3 root wsgi 4096 Jul 30 14:13 log
lrwxrwxrwx 1 root root   20 Jul 30 14:15 ondeck -> /opt/python/bundle/3
drwxr-xr-x 4 root root 4096 Jul 30 14:13 run

20150730_141547_336930605_02_post_deploy

PWD: /
total 28
drwxr-xr-x 7 root root 4096 Jul 30 14:15 .
drwxr-xr-x 6 root root 4096 Jul 30 14:13 ..
drwxr-xr-x 2 root root 4096 Jul 30 14:13 bin
drwxr-xr-x 3 root root 4096 Jul 30 14:15 bundle
lrwxrwxrwx 1 root root   20 Jul 30 14:15 current -> /opt/python/bundle/3
drwxr-xr-x 2 root root 4096 Jul 30 14:13 etc
drwxrwxr-x 3 root wsgi 4096 Jul 30 14:13 log
drwxr-xr-x 4 root root 4096 Jul 30 14:13 run

From the above result and some checking inside EC2 instance, I can conclude that:

• There is default app /opt/python/bundle/1 directory before your app come into /opt/python.
• The commands in .ebextensions will be executed before your app file is in /opt/python.
• The working directory of commands is in /.
• The container_commands in .ebextensions will be executed after your app file is extracted in /opt/python.
• The working directory of container_commands is in your latest app directory in /opt/python.
• EB will remove your old app directory after the app is deployed.

I thought, another solution: you can put your SQLite database using current working directory of your script.