What to do when a py.test hangs silently?

Question

While using py.test, I have some tests that run fine with SQLite but hang silently when I switch to Postgresql. How would I go about debugging something like that? Is there a "verbose" mode I can run my tests in, or set a breakpoint ? More generally, what is the standard plan of attack when pytest stalls silently? I've tried using the pytest-timeout, and ran the test with $ py.test --timeout=300, but the tests still hang with no activity on the screen whatsoever

Answer 1

I had a similar problem with pytest and Postgresql while testing a Flask app that used SQLAlchemy. It seems pytest has a hard time running a teardown using its request.addfinalizer method with Postgresql.

Previously I had:

@pytest.fixture
def db(app, request):
    def teardown():
        _db.drop_all()

    _db.app = app
    _db.create_all()

    request.addfinalizer(teardown)

    return _db

( _db is an instance of SQLAlchemy I import from extensions.py ) But if I drop the database every time the database fixture is called:

@pytest.fixture
def db(app, request):
    _db.app = app
    _db.drop_all()
    _db.create_all()
    return _db

Then pytest won't hang after your first test.

Answer 2

I ran into the same SQLite/Postgres problem with Flask and SQLAlchemy, similar to Gordon Fierce. However, my solution was different. Postgres is strict about table locks and connections, so explicitly closing the session connection on teardown solved the problem for me.

My working code:

@pytest.yield_fixture(scope='function')
def db(app):
    # app is an instance of a flask app, _db a SQLAlchemy DB
    _db.app = app
    with app.app_context():
        _db.create_all()

    yield _db

    # Explicitly close DB connection
    _db.session.close()

    _db.drop_all()

SQLAlchemy reference: http://docs.sqlalchemy.org/en/rel_0_8/faq.html#my-program-is-hanging-when-i-say-table-drop-metadata-drop-all

Answer 3

In my case the Flask application did not check if __name__ == '__main__': so it executed app.start() when that was not my intention.

You can read many more details here.

Answer 4

To answer the question "How would I go about debugging something like that?"

1. Run with py.test -m trace --trace to get trace of python calls.

2. One option (useful for any stuck unix binary) is to attach to process using strace -p <PID>. See what system call it might be stuck on or loop of system calls. e.g. stuck calling gettimeofday

3. For more verbose py.test output install pytest-sugar. pip install pytest-sugar And run test with pytest.py --verbose . . . https://pypi.python.org/pypi/pytest-sugar

Answer 5

Not knowing what is breaking in the code, the best way is to isolate the test that is failing and set a breakpoint in it to have a look. Note: I use pudb instead of pdb, because it's really the best way to debug python if you are not using an IDE.

For example, you can the following in your test file:

import pudb
...

def test_create_product(session):
    pudb.set_trace()
    # Create the Product instance
    # Create a  Price instance
    # Add the Product instance to the session. 
    ...

Then run it with

py.test -s --capture=no test_my_stuff.py

Now you'll be able to see exactly where the script locks up, and examine the stack and the database at this particular moment of execution. Otherwise it's like looking for a needle in a haystack.