Aws Beanstalk nginx killed when tried to add new e

2019-03-03 18:36发布

问题:

I receive this error every time that I need to add a new environment variable from AWS EBS panel:

AWS Beanstalk events:

2018-02-16 14:49:21 UTC-0200  INFO  The environment was reverted to the previous configuration setting.
2018-02-16 14:48:49 UTC-0200  ERROR During an aborted deployment, some instances may have deployed the new application version. To ensure all instances are running the same version, re-deploy the appropriate application version.
2018-02-16 14:48:49 UTC-0200  ERROR Failed to deploy configuration.
2018-02-16 14:48:49 UTC-0200  ERROR Unsuccessful command execution on instance id(s) 'i-xxxxxxxxxxxxxx'. Aborting the operation.
2018-02-16 14:48:49 UTC-0200  INFO  Command execution completed on all instances. Summary: [Successful: 0, Failed: 1].

eb-activity.log:

Successfully execute hooks in directory /opt/elasticbeanstalk/hooks/configdeploy/enact.
[2018-02-16T16:21:18.921Z] INFO [8550] – [Configuration update app-0_0_10-180216_141535@104/ConfigDeployStage1/ConfigDeployPostHook] : Starting activity…
[2018-02-16T16:21:18.921Z] INFO [8550] – [Configuration update app-0_0_10-180216_141535@104/ConfigDeployStage1/ConfigDeployPostHook/99_kill_default_nginx.sh] : Starting activity…
[2018-02-16T16:21:19.164Z] INFO [8550] – [Configuration update app-0_0_10-180216_141535@104/ConfigDeployStage1/ConfigDeployPostHook/99_kill_default_nginx.sh] : Activity execution failed, because: + rm -f /etc/nginx/conf.d/00_elastic_beanstalk_proxy.conf
+ service nginx stop
Stopping nginx: /sbin/service: line 66: 8986 Killed env -i PATH=”$PATH” TERM=”$TERM” “${SERVICEDIR}/${SERVICE}” ${OPTIONS} (ElasticBeanstalk::ExternalInvocationError)
caused by: + rm -f /etc/nginx/conf.d/00_elastic_beanstalk_proxy.conf
+ service nginx stop
Stopping nginx: /sbin/service: line 66: 8986 Killed env -i PATH=”$PATH” TERM=”$TERM” “${SERVICEDIR}/${SERVICE}” ${OPTIONS} (Executor::NonZeroExitStatus)

[2018-02-16T16:21:19.164Z] INFO [8550] – [Configuration update app-0_0_10-180216_141535@104/ConfigDeployStage1/ConfigDeployPostHook/99_kill_default_nginx.sh] : Activity failed.
[2018-02-16T16:21:19.165Z] INFO [8550] – [Configuration update app-0_0_10-180216_141535@104/ConfigDeployStage1/ConfigDeployPostHook] : Activity failed.
[2018-02-16T16:21:19.165Z] INFO [8550] – [Configuration update app-0_0_10-180216_141535@104/ConfigDeployStage1] : Activity failed.
[2018-02-16T16:21:19.165Z] INFO [8550] – [Configuration update app-0_0_10-180216_141535@104] : Completed activity. Result:
Configuration update – Command CMD-ConfigDeploy failed

Edit: Added stack-https.config file

eb-activity.log:

Command 01_copy_conf_file] : Activity execution failed, because:  (ElasticBeanstalk::ExternalInvocationError

Starting activity...
[2018-02-16T20:38:30.476Z] INFO  [2536]  - [Application deployment app-0_0_10-1-gb633-180216_175029@124/StartupStage0/EbExtensionPostBuild/Infra-EmbeddedPostBuild/postbuild_0_paneladm_api_stack_SampleApplication_W4FJ8W83X64B] : Starting activity...
[2018-02-16T20:38:32.456Z] INFO  [2536]  - [Application deployment app-0_0_10-1-gb633-180216_175029@124/StartupStage0/EbExtensionPostBuild/Infra-EmbeddedPostBuild/postbuild_0_paneladm_api_stack_SampleApplication_W4FJ8W83X64B/Command 00_removeconfig] : Starting activity...
[2018-02-16T20:38:32.463Z] INFO  [2536]  - [Application deployment app-0_0_10-1-gb633-180216_175029@124/StartupStage0/EbExtensionPostBuild/Infra-EmbeddedPostBuild/postbuild_0_paneladm_api_stack_SampleApplication_W4FJ8W83X64B/Command 00_removeconfig] : Completed activity.
[2018-02-16T20:38:34.493Z] INFO  [2536]  - [Application deployment app-0_0_10-1-gb633-180216_175029@124/StartupStage0/EbExtensionPostBuild/Infra-EmbeddedPostBuild/postbuild_0_paneladm_api_stack_SampleApplication_W4FJ8W83X64B/Command 01_copy_conf_file] : Starting activity...
[2018-02-16T20:38:34.538Z] INFO  [2536]  - [Application deployment app-0_0_10-1-gb633-180216_175029@124/StartupStage0/EbExtensionPostBuild/Infra-EmbeddedPostBuild/postbuild_0_paneladm_api_stack_SampleApplication_W4FJ8W83X64B/Command 01_copy_conf_file] : Activity execution failed, because:  (ElasticBeanstalk::ExternalInvocationError)

I don't know if the problem is because I previous removed the default elastic_beanstalk_proxy.conf file with my commands as below:

Resources:
  sslSecurityGroupIngress:
    Type: AWS::EC2::SecurityGroupIngress
    Properties:
      GroupId: {"Fn::GetAtt" : ["AWSEBSecurityGroup", "GroupId"]}
      IpProtocol: tcp
      ToPort: 443
      FromPort: 443
      CidrIp: 0.0.0.0/0

files:

  /etc/letsencrypt/configs/http_proxy.pre:
    mode: "000644"
    owner: root
    group: root
    content: |
      # Elastic Beanstalk Managed

      upstream nodejs {
          server 127.0.0.1:8081;
          keepalive 256;
      }

      server {
        listen 8080;

        access_log /var/log/nginx/access.log  main;

        location /.well-known {
          allow all;
          root /usr/share/nginx/html;
        }

        # Redirect non-https traffic to https.
        location / {
          if ($scheme   != "https") { 
            return 301 https://$host$request_uri;
          } #   managed by Certbot
        }
      }

# The Nginx config forces https, and is meant as an example only. 
  /etc/letsencrypt/configs/https_custom.pos:
    mode: "000644"
    owner: root
    group: root
    content: |
      # HTTPS server
      server {
        listen       443 default ssl;
        server_name  localhost;
        error_page   497 https://$host$request_uri;

        ssl_certificate      /etc/letsencrypt/live/ebcert/fullchain.pem;
        ssl_certificate_key  /etc/letsencrypt/live/ebcert/privkey.pem;
        ssl_session_timeout  5m;
        ssl_protocols  TLSv1.1 TLSv1.2;
        ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
        ssl_prefer_server_ciphers   on;

        if ($ssl_protocol = "") {
          rewrite ^ https://$host$request_uri? permanent;
        }

        location / {
            proxy_pass  http://nodejs;
            proxy_set_header Connection "";
            proxy_http_version 1.1;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
        }

        gzip on;
        gzip_comp_level 4;
        gzip_types text/plain text/css application/json application/javascript application/x-javascript text/xml application/xml application/xml+rss text/javascript;
      }

  /etc/letsencrypt/configs/generate-cert.sh:
    mode: "000664"
    owner: root
    group: root
    content: |
      #!/bin/sh

      _EMAIL=
      _DOMAIN=

      while getopts ":e:d:" OPTION;
      do
        case "${OPTION}" in
          "e") _EMAIL="${OPTARG}";;
          "d") _DOMAIN="${OPTARG}";;
        esac
      done

      if [ -z "${_EMAIL}" ]; then
        echo "Param email isn't specified!"
      fi

      if [ -z "${_DOMAIN}" ]; then
        echo "Param domain isn't specified!"
      fi

      if [ -n "$_EMAIL" ] && [ -n "$_DOMAIN" ]; then
        cd /opt/certbot/
        ./certbot-auto certonly \
          --debug --non-interactive --email ${_EMAIL} \
          --webroot -w /usr/share/nginx/html --agree-tos -d ${_DOMAIN} --keep-until-expiring
      fi

      if [ $? -ne 0 ]
        then
          ERRORLOG="/var/log/letsencrypt/letsencrypt.log"
          echo "The Let's Encrypt cert has not been renewed!\n" >> $ERRORLOG
        else
          /etc/init.d/nginx reload
        fi

      exit 0

  /opt/elasticbeanstalk/hooks/configdeploy/post/99_kill_default_nginx.sh:
    mode: "000755"
    owner: root
    group: root
    content: |
      #!/bin/bash -xe
      rm -f /etc/nginx/conf.d/00_elastic_beanstalk_proxy.conf
      service nginx stop 
      service nginx start

packages:
  yum:
    epel-release: []

container_commands:
  00_removeconfig:
    command: "rm -f /tmp/deployment/config/#etc#nginx#conf.d#00_elastic_beanstalk_proxy.conf /etc/nginx/conf.d/00_elastic_beanstalk_proxy.conf"
  01_copy_conf_file:
    command: "cp /etc/letsencrypt/configs/http_proxy.pre /etc/nginx/conf.d/http_proxy.conf; /etc/init.d/nginx reload"
  02_createdir:
    command: "mkdir /opt/certbot || true"
  03_installcertbot:
    command: "wget https://dl.eff.org/certbot-auto -O /opt/certbot/certbot-auto"
  04_permission:
    command: "chmod a+x /opt/certbot/certbot-auto"
  05_getcert:
    command: "sudo sh /etc/letsencrypt/configs/generate-cert.sh -e ${CERT_EMAIL} -d ${CERT_DOMAIN}"
  06_link:
    command: "ln -sf /etc/letsencrypt/live/${CERT_DOMAIN} /etc/letsencrypt/live/ebcert"
  07_copy_ssl_conf_file:
    command: "cp /etc/letsencrypt/configs/https_custom.pos /etc/nginx/conf.d/https_custom.conf; /etc/init.d/nginx reload"
  08_cronjob_renew:
    command: "sudo sh /etc/letsencrypt/configs/generate-cert.sh -e ${CERT_EMAIL} -d ${CERT_DOMAIN}"

I'm doing this because I replace this file to my own proxy.conf file.

Please I need your help.

References:

  • awslabs/elastic-beanstalk-sampes/https-redirect-nodejs.config

  • AWS EBS - Environment Properties and Other Software Settings

回答1:

I had this problem as well and Amazon acknowledged the error in the documentation. This is a working restart script that you can use in your .ebextensions config file.

/opt/elasticbeanstalk/hooks/configdeploy/post/99_kill_default_nginx.sh:
mode: "000755"
owner: root
group: root
content: |
  #!/bin/bash -xe
  rm -f /etc/nginx/conf.d/00_elastic_beanstalk_proxy.conf
  status=`/sbin/status nginx`

  if [[ $status = *"start/running"* ]]; then
    echo "stopping nginx..."
    stop nginx
    echo "starting nginx..."
    start nginx
  else
    echo "nginx is not running... starting it..."
    start nginx
  fi