Requests get stuck in PG::Connection#async_exec

2019-07-29 13:44发布

问题:

I'm trying to make a rails application work under passenger with no luck so far. It uses sequel to connect to postgresql. Requests randomly take too much time to complete. I was able to find where the app is spending its time:

 %self      total      self      wait     child     calls  name
 99.92     65.713    65.713     0.000     0.000        5   PG::Connection#async_exec
  0.00      0.002     0.002     0.000     0.000      264   Set#delete
...

Supposedly, it doesn't happen with webrick, and I'm going to try and run it under unicorn.

Here are some related links.

I'm using passenger-5.0.30, ruby-2.3.0, postgresql-9.4, pg-0.19.0, rails-5.0.0.1. The same goes for unicorn-5.1.0. The app is running in lxc container. Host and guest OSs are ubuntu/xenial.

回答1:

The issue was resolved after we started reestablishing connections to database. We were using sequel on the project, and Passenger handles the issue itself only when ActiveRecord is used.

To expand on it, by default Passenger uses smart spawning for ruby apps. It first spawns preloader process, which loads framework and all the libraries. After that preloader process spawns worker processes when the need arises, which handle the requests. When spawning worker processes, all file descriptors are inherited from preloader process. So if you don't reestablish connection to database, all worker process share one, established by preloader. Which leads to all sorts of weird behavior. Like requests taking too long.

Before that, we also moved handling WebSocket connections to a separate process. Not sure if it contributed to the issue however.

More on that here:

Disconnect If Using Forking Webserver with Code Preloading
Unintentional file descriptor sharing
Running the Action Cable server on the same host and port, under a sub-URI