Lighthouse has a new layout. Prefer the old one? Return to the old layout, and switch back any time from the link at the top of each page.

This project is archived and is in readonly mode.

Psycopg2 hung queries

#112

Submitted by: anoop

We have a python program which uses psycopg2 to make a connection every 2 seconds to postgresql-9.0 and batch insert/update records. The connection is closed after use.

It's part of a server software which run continuously for months. Some times the query processes is getting stuck (for ever) in pgWaitTimed.

But pg_stat_activity on postgresql indicates that no query is running.

StackTrace
-----------
>> pqWaitTimed
>> PQgetResult

The processes has to be restarted to fix the issue.

Reported by Psycopg website · June 23rd, 2012 @ 02:44 PM

State: invalid
Milestone: none
Assigned to: nobody

Activity

  1. Josh Berkus
    Josh Berkus

    I'm seeing the exact same problem in a multithreaded Python application. It makes and drops connections rapidfire for a few minutes, then hangs. GDB indicates the hang is in psycopg2.

    We have a 100% reproduceable test case for this on our test environment; let me know what you want to trace etc. in order to diagnose this.

    June 28th, 2012 @ 09:19 PM

  2. Daniele Varrazzo
    Daniele Varrazzo
    • State changed from new to open

    Josh, are you using the latest, 2.4.5? There have been a few multithread bug fixed in that version.

    Anoop, from your tag I see you are using 2.4.1: can you try updating and repeat your test?

    Josh, a stack trace would be a start. Are you creating one cursor per connection or many cursors? Is the same connection used by different threads from the different cursors? (which should work, but here is where a bug may lurk). Is the same cursor used by different threads? (which is not to be done).

    It would be killer if you could synthesize your test case into something that you can provide us.

    Thank you.

    June 28th, 2012 @ 09:51 PM

  3. Daniele Varrazzo
    Daniele Varrazzo

    Josh, another possible source of help: try compiling and running psycopg with debug info (setting PSYCOPG_DEBUG in setup.cfg and running the program with the PSYCOPG_DEBUG env variable set). You will have plenty of debug info on stderr.

    June 28th, 2012 @ 10:07 PM

  4. Josh Berkus
    Josh Berkus

    Daniele,

    The original bug isn't mine, I'm just adding to it because it seems to be the exact same symptoms. It's Anoop's bug originally.

    I am using 2.4.5.

    We'll enable the trace and get back to you.

    June 28th, 2012 @ 10:21 PM

  5. Daniele Varrazzo
    Daniele Varrazzo
    • State changed from open to invalid

    Closing for lack of further information and tests.

    April 20th, 2013 @ 01:36 AM