Apache mod_wsgi django enable multi-thread multi-p

Question

Question :

1. how can i know if my apache server has already run in multi-process & multi-thread mode ? because when i load tested it, it giving me same result time with different threads count. i run test with 25 threads and 50 threads.

2. Is there any adjustment that i have to do in the Django code for the multi-thread / multi-process work?

3. Am I have to change the MPM config (/conf/extra/httpd-mpm.conf) ?

below is my server detail and configuration :

Server redhat enterprise 6.9
Apache server 2.4.33
Postgre 9.6.6
Python 3.6
Virtualbox RAM 8Gb, 2 Core (4 vCpu).

my httpd.conf :

Listen 8000

LoadModule wsgi_module modules/mod_wsgi.so

Include conf/extra/httpd-vhosts.conf

WSGIScriptAlias / /home/applmgr/Harpa/HarpaBackend/harpa/wsgi.py
WSGIPythonHome /home/applmgr/Harpa/pyenv_sl
WSGIPythonPath /home/applmgr/Harpa/HarpaBackend
WSGIPassAuthorization On

<Directory /home/applmgr/Harpa/HarpaBackend/harpa>
<Files wsgi.py>
Require all granted
</Files>
</Directory>

my httpd-vhosts.conf :

<VirtualHost *:8000>

    Alias /static /home/applmgr/Harpa/HarpaBackend/static
    <Directory /home/applmgr/Harpa/HarpaBackend/static>
        Require all granted
    </Directory>

    WSGIDaemonProcess harpa python-home=/home/applmgr/Harpa/pyenv_sl processes=15 threads=50 python-path=/home/applmgr/Harpa/HarpaBackend  
    WSGIProcessGroup harpa

    <Directory /home/applmgr/Harpa/HarpaBackend/harpa>
        <Files wsgi.py>
            Require all granted
        </Files>
    </Directory>
</VirtualHost>

ps -ef | grep httpd :

applmgr   2817     1  0 14:18 ?        00:00:00 /opt/apache_http/bin/httpd -k start
applmgr   2818  2817  0 14:18 ?        00:00:04 /opt/apache_http/bin/httpd -k start
applmgr   2819  2817  0 14:18 ?        00:00:00 /opt/apache_http/bin/httpd -k start
applmgr   2820  2817  0 14:18 ?        00:00:03 /opt/apache_http/bin/httpd -k start
applmgr   2821  2817  0 14:18 ?        00:00:06 /opt/apache_http/bin/httpd -k start
applmgr   2822  2817  0 14:18 ?        00:00:03 /opt/apache_http/bin/httpd -k start
applmgr   2823  2817 26 14:18 ?        00:34:21 /opt/apache_http/bin/httpd -k start
applmgr   2824  2817  0 14:18 ?        00:00:06 /opt/apache_http/bin/httpd -k start
applmgr   2825  2817 47 14:18 ?        01:01:16 /opt/apache_http/bin/httpd -k start
applmgr   2826  2817  0 14:18 ?        00:00:00 /opt/apache_http/bin/httpd -k start
applmgr   2827  2817 25 14:18 ?        00:33:00 /opt/apache_http/bin/httpd -k start
applmgr   2828  2817  0 14:18 ?        00:00:00 /opt/apache_http/bin/httpd -k start
applmgr   2829  2817  0 14:18 ?        00:00:03 /opt/apache_http/bin/httpd -k start
applmgr   2830  2817  0 14:18 ?        00:00:03 /opt/apache_http/bin/httpd -k start
applmgr   2831  2817  0 14:18 ?        00:00:00 /opt/apache_http/bin/httpd -k start
applmgr   2832  2817  0 14:18 ?        00:00:03 /opt/apache_http/bin/httpd -k start
applmgr   2833  2817  0 14:18 ?        00:00:00 /opt/apache_http/bin/httpd -k start
applmgr   2834  2817  0 14:18 ?        00:00:00 /opt/apache_http/bin/httpd -k start
applmgr   2835  2817  0 14:18 ?        00:00:00 /opt/apache_http/bin/httpd -k start
applmgr   3875  2817  0 14:23 ?        00:00:00 /opt/apache_http/bin/httpd -k start
applmgr   4979  2642  0 16:26 pts/1    00:00:00 grep httpd

./apachectl -V :

Server version: Apache/2.4.33 (Unix)
Server built:   Apr  9 2018 16:42:03
Server's Module Magic Number: 20120211:76
Server loaded:  APR 1.6.3, APR-UTIL 1.6.1
Compiled using: APR 1.6.3, APR-UTIL 1.6.1
Architecture:   64-bit
Server MPM:     event
threaded:     yes (fixed thread count)
forked:     yes (variable process count)
Server compiled with....
-D APR_HAS_SENDFILE
-D APR_HAS_MMAP
-D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
-D APR_USE_SYSVSEM_SERIALIZE
-D APR_USE_PTHREAD_SERIALIZE
-D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
-D APR_HAS_OTHER_CHILD
-D AP_HAVE_RELIABLE_PIPED_LOGS
-D DYNAMIC_MODULE_LIMIT=256
-D HTTPD_ROOT="/opt/apache_http"
-D SUEXEC_BIN="/opt/apache_http/bin/suexec"
-D DEFAULT_PIDLOG="logs/httpd.pid"
-D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
-D DEFAULT_ERRORLOG="logs/error_log"
-D AP_TYPES_CONFIG_FILE="conf/mime.types"
-D SERVER_CONFIG_FILE="conf/httpd.conf"

Answer 1

Setting a high number of threads in Python is usually a bad idea because of the Python GIL. And doing benchmark testing where you excessively overload the system is even worse as it jut exacerbates things and gives unreliable results. I'd recommend not running a Python web server which uses threading at over 40-60% capacity because once you push throughput higher, especially if more CPU bound, then things go rapidly down hill. Benchmarks which send max requests, do this very quickly and so are meaningless.

I would suggest watching the following two conference talk videos which goes into some of the issues.

• https://www.youtube.com/watch?v=k6Erh7oHvns&t=1s
• https://www.youtube.com/watch?v=SGleKfigMsk&t=133s

As a general rule of thumb, do the following:

• Use daemon mode.
• Disable embedded mode using WSGIRestrictEmbedded On.
• Use at most 5 threads per process, unless excessively I/O bound.
• Use processes over threads, but don't get carried away with processes either.
• Scale across multiple machines.
• Instrument the WSGI server and application so now how it performs for real system.

For adding metrics to monitor mod_wsgi see:

• http://blog.dscpl.com.au/2015/06/implementing-request-monitoring-within.html

If want to discuss metrics, use the mod_wsgi mailing list for latest information.

---

UPDATE 1

Also watch:

• https://www.youtube.com/watch?v=H6Q3l11fjU0

At the end of this it talks about various settings daemon mode you should look at and set. Also see the end of:

• http://modwsgi.readthedocs.io/en/develop/configuration-directives/WSGIDaemonProcess.html

for some recommended defaults for daemon process settings.