#113 ✓ resolved
Psycopg website

Psycopg2 - Eventlet connection hang

Reported by Psycopg website | June 23rd, 2012 @ 02:47 PM

Submitted by: anoop

I have a program which keeps a persistent connection to a postgresql server. Suppose the connection is blocked due to network issues(can be simulated by bringing down network interface), the process which invoked query using eventlet is getting stuck. This is preventing other eventlets also from executing. Attaching gdb indicated that processes is blocked in psycopg2 C code. I tried postgresql TCP keepalives. But this will also fail if a query is invoked as soon as the connection is lost.

Is there any way to prevent eventlet from getting in to this state ? Is there any way to break eventlet out of this state.

GDB

[#0](/projects/62710/tickets/0 "Ticket #0")  0x00002b4e7836be46 in poll () from /lib64/libc.so.6
[#1](/projects/62710/tickets/1 "Ticket #1")  0x00002b4e835dec4f in ?? () from /usr/pgsql-9.0/lib/libpq.so.5
[#2](/projects/62710/tickets/2 "Ticket #2")  0x00002b4e835ded50 in ?? () from /usr/pgsql-9.0/lib/libpq.so.5
[#3](/projects/62710/tickets/3 "Ticket #3")  0x00002b4e835dd28e in PQgetResult () from /usr/pgsql-9.0/lib/libpq.so.5
[#4](/projects/62710/tickets/4 "Ticket #4")  0x00002b4e833a4284 in psyco_exec_green (conn=0x21f42510, command=<value optimized out>) at psycopg/green.c:195
[#5](/projects/62710/tickets/5 "Ticket #5")  0x00002b4e833a5387 in pq_execute_command_locked (conn=0x21f42510, 
    query=0x2b4e833b5de8 "BEGIN; SET TRANSACTION ISOLATION LEVEL READ COMMITTED", pgres=0x7fff84c38448, 
    error=0x7fff84c38440, 
    tstate=0x7fff84c38438) at psycopg/pqpath.c:342
[#6](/projects/62710/tickets/6 "Ticket #6")  0x00002b4e833a58f7 in pq_begin_locked (conn=0x21f42510, pgres=0xffffffffffffffff, error=0xffffffffffffffff, tstate=0x0)
    at psycopg/pqpath.c:424
[#7](/projects/62710/tickets/7 "Ticket #7")  0x00002b4e833a59a0 in pq_execute (curs=0x1f487528, query=0x2248c894 "SELECT 1", async=0) at psycopg/pqpath.c:764
[#8](/projects/62710/tickets/8 "Ticket #8")  0x00002b4e833ab15b in _psyco_curs_execute (self=0x1f487528, operation=0x0, vars=0x21ec9a80, async=0)
    at psycopg/cursor_type.c:419
[#9](/projects/62710/tickets/9 "Ticket #9")  0x00002b4e833ab7b1 in psyco_curs_execute (self=0x1f487528, args=<value optimized out>, kwargs=<value optimized out>)
    at psycopg/cursor_type.c:475

Comments and changes to this ticket

  • Daniele Varrazzo

    Daniele Varrazzo June 28th, 2012 @ 10:03 PM

    • State changed from “new” to “open”

    The problem is in psyco_clear_result_blocking: it is called because the network problem has been detected. That function was supposed to be called upon an error in the green callback: it looks like calling it in presence of a network problem is not a good idea.

    We could have to rethink the way we deal with errors during green reception of data: if we cannot disambiguate between a python error and a network error we'd rather mark the connection unusable.

  • passing
  • Daniele Varrazzo

    Daniele Varrazzo October 6th, 2012 @ 12:12 PM

    Working in the fix-113 branch of my repos.

    The current best solution is just to close the connection on callback error. PQcancel is blocking as well, PQreset may leave the connection in unexpected state.

  • Daniele Varrazzo

    Daniele Varrazzo October 11th, 2012 @ 10:42 PM

    • State changed from “open” to “resolved”

    Merged in devel branch.

Please Sign in or create a free account to add a new ticket.

With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.

New-ticket Create new ticket

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile »

Psycopg is the most used PostgreSQL adapter for the Python programming language. At the core it fully implements the Python DB API 2.0 specifications. Several extensions allow access to many of the features offered by PostgreSQL.

Shared Ticket Bins

Pages