How can I get Capistrano to stop failing at deploy

Question

At least that's where it looks like capistrano is failing. It makes it all the way through the deployment and at the end. here is the output.

* executing `deploy:create_symlink'
* executing "rm -f ~/xxx.xx.xx/test/current && ln -s ~/xxx.xx.xx/test/releases/20120525193307 ~/xxx.xx.xx/test/current"
servers: ["test.xxx.xx.xx"]
["test.xxx.xx.xx"] executing command
 ** [out :: test.xxx.xx.xx] rm: cannot remove `/var/www/vhosts/xxx.xx.xx/test/current': I command finished in 460ms
*** [deploy:create_symlink] rolling back
*** no previous release to rollback to, rollback of symlink skipped
* executing "rm -rf /var/www/vhosts/xxx.xx.xx/test/releases/20120525195909; true"
servers: ["test.xxx.xx.xx"]
[test.xxx.xx.xx] executing command
command finished in 524ms
failed: "rvm_path=$HOME/.rvm/ $HOME/.rvm/bin/rvm-shell 'default' -c 'rm -f /var/www/vhosts/xxx.xx.xx/test/current && ln -s /var/www/vhosts/xxx.xx.xx/test/releases/20120525195909 /var/www/vhosts/xxx.xx.xx/test/current'" on xxx.xx.xx

the app is using capistrano (2.12.0) capistrano-ext (1.2.1) obviously there are more gems just trying to put what seems relevant, please let me know if more info would be helpful.

Here is the deploy.rb

require "rvm/capistrano"
require 'bundler/capistrano'
require 'capistrano/ext/multistage'

set :stages, %w(staging production)
set :default_stage, 'staging'

set :application, 'xxx'

# trying to not use sudo on the deployment
#set :use_sudo, false

#set :copy_exclude, [".git", "spec"]

set :repository,  '~/git/xxx.git'
set :local_repository, "~/rorwork/xxx/.git"
set :scm, :git
set :user, 'xxx'
set :group, 'xxxx'
ssh_options[:forward_agent] = true
set :branch, 'master'
set :deploy_via, :remote_cache
set :scm_command, "/usr/local/bin/git"
set :local_scm_command, :default
default_run_options[:pty] = true
set :normalize_asset_timestamps, false #for asset piple

set :dbuser, 'xxx'
set :dbpass, 'xxx'

# if you want to clean up old releases on each deploy uncomment this:
# after "deploy:restart", "deploy:cleanup"

# if you're still using the script/reaper helper you will need
# these http://github.com/rails/irs_process_scripts

# If you are using Passenger mod_rails uncomment this:
 namespace :deploy do
   task :start do ; end
   task :stop do ; end
   task :restart, :roles => :app, :except => { :no_release => true } do
     run "#{try_sudo} touch #{File.join(current_path,'tmp','restart.txt')}"
   end
 end

and the staging.rb

set :domain, 'test.xxx.xx.xx'

role :web, domain                          # Your HTTP server, Apache/etc
role :app, domain                          # This may be the same as your `Web` server
role :db,  domain, :primary => true        # This is where Rails migrations will run

set :deploy_to, "/var/www/vhosts/xxx.xx.xx/test"
set :rails_env, 'staging'
set :rack_env, rails_env

set :dbname, "xxx_staging"

#set :bundle_without, [:test, :development, :production]

i manually create the 'current', 'shared' and 'release' folders in the deploy dir and assign the appropriate user:group. originally was getting the permissions issues on dirs but got that sorted out. kinda at a loss here, much searching for solutions is netting nothing as of yet. any suggestions or experience here is much appreciated!

in testing cap staging deploy thought i would try on production as well cap production deploy bails at the same time... something clever about consistency.

Answer 1

UPDATE: Found it

Deleting the 'current' directory and letting capistrano create the 'current' symlink to the latest release fixed it.

I'll try to clear it up a bit. I was actually creating the folders 'current' and such, 'mkdir current' like. which just gets in the way of capistrano creating the symlinks. try it, make a folder called 'bob' then in the same directory make a symlink called 'bob' and point it somewhere it will probably put the symlink 'bob' inside the directory 'bob.'

when i removed the 'current' 'shared' etc, directories, capistrano could create the 'current' and other necessary symlinks. So, concretely removing the 'current' directory allowed capistrano to create the 'current' symlink that just points to the latest deploy under the 'releases' dir.

note the below error, which is also in the question.

** [out :: test.xxx.xx.xx] rm: cannot remove `/var/www/vhosts/xxx.xx.xx/test/current': I command finished in 460ms

the reason it failed. I had created the folder named 'current.' this is the problem. it is NOT supposed to be a directory, it is supposed to be a symlink that capistrano creates.

By creating those folders it broke the deploy. Removing those folders allowed capistrano to create the symlinks of the same name. Smooth sailing ever since.

Answer 2

I just tracked down a similar problem.

Here's the error first error I had:

cap aborted! Errno::ECONNREFUSED: Connection refused - connect(2) for "{my-ip-address}" port {my-ssh-port}

I would also get this similar error:

Tasks: TOP => git:create_release (See full trace by running task with --trace) The deploy has failed with an error: #<Errno::ECONNREFUSED: Connection refused - connect(2) for "my-ip-address" port {my-port}>

It turns out it was an issue with concurrent SSH sessions as my server runs Fail2Ban. To solve that I simply did the following:

edit the jail that contains SSH configurations

$ sudo nano /etc/fail2ban/jail.local

look for [SSH] and set enabled = false find [ssh-ddos] and set enabled = false

Remember to restart Fail2Ban after your changes and open-ssh (if thats what your using)

$ sudo service fail2ban reload
$ sudo /etc/init.d/ssh reload

Its worth noting that the connection would be refused at different steps (tasks) in the deployment. For example after a reboot and a quick bundle exec cap production deploy:check everything appeared fine. Then I tried to deploy and received the same error, but during the execution of a different task. I also use UFW which I disabled and reenabled without issues. UFW was not the cause of the above problem.

After I solved that problem Capistrano would fail to create the symlink from the current directory to the latest release. The error is below:

Tasks: TOP => deploy:symlink:release The deploy has failed with an error: #<SSHKit::Command::Failed: rm exit status: 1 rm stdout: Nothing written rm stderr: Nothing written > ** Invoke deploy:failed (first_time) ** Execute deploy:failed

After reading extensively, changing the sshkit version, downgrading capistrano, eliminating gems, etc, I navigated to my document root and listed the directory contents ls -la I noticed that the current directory was not in the deploy user's group, I deleted the current folder (sudo rm -rf current)and everything worked fine. Good luck.