Initialize the Delayed Jobs gem by starting the wo

Question

I am using Ruby on Rails 3.0.9 and I am trying to setup the delay_job gem. All works if, after rebooting the Apache2 server, I run in the Terminal\Console following commands:

RAILS_ENV=development script/delayed_job stop
RAILS_ENV=development script/delayed_job -n 2 start

However, since I always want to start the workers on application start, in my config/initializers/delayed_job.rb I add the following code (that handles both development and production mode):

if Rails.env.development?
  system 'RAILS_ENV=development script/delayed_job stop'
  system 'RAILS_ENV=development script/delayed_job -n 2 start'
elsif Rails.env.production?
  system 'RAILS_ENV=production script/delayed_job stop'
  system 'RAILS_ENV=production script/delayed_job -n 2 start'
end

However, by using the above code and after re-rebooting the Apache2 server, the DJ gem doesn't work anymore as expected. That is, it doesn't process jobs as it makes when I run the above command lines in the Terminal\Console.

How can I make DJ to work correctly? What is the problem?

P.S.: I would like to do that in order to automatize processes.

---

It seams that the above code in the config/initializers/delayed_job.rb file doesn't "create" the "pids" files related to DJ in the RAILS_ROOT/tmp/pids directory. Those are created only by running the above command lines manually. Why this happens?

---

UPDATE for @Devin M

My config/initializers/delayed_job.rb contains:

# Options
Delayed::Worker.destroy_failed_jobs = false
Delayed::Worker.sleep_delay = 2
Delayed::Worker.max_attempts = 5
Delayed::Worker.max_run_time = 1.hour
Delayed::Worker.delay_jobs = !Rails.env.test?

if Rails.env.development?
  system "RAILS_ENV=development #{Rails.root.join('script','delayed_job')} stop"
  system "RAILS_ENV=development #{Rails.root.join('script','delayed_job')} -n 2 start"
elsif Rails.env.production?
  system "RAILS_ENV=production #{Rails.root.join('script','delayed_job')} stop"
  system "RAILS_ENV=production #{Rails.root.join('script','delayed_job')} -n 2 start"
end

Answer 1

Try using this code:

system "RAILS_ENV=production #{Rails.root.join('script','delayed_job')} stop"
system "RAILS_ENV=production #{Rails.root.join('script','delayed_job')} -n 2 start"

Testing it in the console produced this output which indicates it should work:

Loading development environment (Rails 3.0.9)
ruby-1.9.2-p290 :001 > Rails.root.join('script','delayed_job')
 => #<Pathname:/home/devin/testsoapp/script/delayed_job> 
ruby-1.9.2-p290 :002 > "RAILS_ENV=production #{Rails.root.join('script','delayed_job')} -n 2 start"
 => "RAILS_ENV=production /home/devin/testsoapp/script/delayed_job -n 2 start" 
ruby-1.9.2-p290 :003 > 

Answer 2

I don't think you can do that because when you start the '/script/delayed_job', the rails environment will be loaded, causing the 'config/initializers/delayed_job.rb' file to be executed again. You can see this results in an infinite loop. Also each time you invoke rake, for example: 'rake db:migrate', it would initialize delayed_jobs.

You can hack around it with this:

if Rails.env.production?
  if(!File.exists?(Rails.root.join('tmp','pids', 'delayed_job.pid')))
    system "echo \"Starting delayed_jobs...\""
    system "./script/delayed_job start &"
  else
    system "echo \"delayed_jobs is running\""
  end
end

With the '&' the delayed_job script is run in background, on a separate process than rails. Subsequent calls to rake will skip launching delayed_jobs if its already running. You'll still have some problems if, for some reason, that file is not deleted when delayed_jobs terminates.

The command /script/delayed_job status will detect if that's the case, but you can't run it inside this 'config/initializers/delayed_job.rb' file because it would cause an infinite loop :(

Answer 3

I had a problem where the initializing delayed_job process was running though its initializers before creating the pid file (i.e. delayed_job.pid in case of a single worker and delayed_job.0.pid delayed_job.1.pid etc. in case of many workers)

So I resorted to creating my own lock file as such:

Delayed::Worker.destroy_failed_jobs = false
Delayed::Worker.sleep_delay = 2
Delayed::Worker.max_attempts = 5
Delayed::Worker.max_run_time = 4.hour
Delayed::Worker.delay_jobs = !Rails.env.test?

workers = 2


if Rails.env.production? || Rails.env.development?
  # Check if the delayed job process is already running
  # Since the process loads the rails env, this file will be called over and over
  # Unless this condition is set.
  pids = Dir.glob(Rails.root.join('tmp','pids','*'))

  system "echo \"delayed_jobs INIT check\""
  if pids.select{|pid| pid.start_with?(Rails.root.join('tmp','pids','delayed_job.init').to_s)}.empty?

    f = File.open(Rails.root.join('tmp','pids','delayed_job.init'), "w+") 
    f.write(".")
    f.close
    system "echo \"Restatring delayed_jobs...\""
    system "RAILS_ENV=#{Rails.env} #{Rails.root.join('bin','delayed_job')} stop"
    system "RAILS_ENV=#{Rails.env} #{Rails.root.join('bin','delayed_job')} -n #{workers} start"
    system "echo \"delayed_jobs Workers Initiated\""
    File.delete(Rails.root.join('tmp','pids','delayed_job.init')) if File.exist?(Rails.root.join('tmp','pids','delayed_job.init'))

  else
    system "echo \"delayed_jobs is running\""
  end
end

file: config\initializers\delayed_job.rb

Note: will not work for windows!