This project is archived and is in readonly mode.

#82 ✓resolved
Gavin Panella

psycopg went psycotic without error set

Reported by Gavin Panella | December 7th, 2011 @ 12:18 PM

In Storm https://launchpad.net/storm we have various tests for
behaviour when a connection to the database is broken. One of them is
breaking with "psycopg went psycotic without error set" on psycopg2
2.4.1 and 2.4.2. It does not break with 2.2.2 and 2.0.14.

Here's a fragment of the test showing what we're trying to do:

def setUp(self):
    super(...).setUp()
    database_uri = URI(...)
    database_user = database_uri.username
    database_dsn = make_dsn(database_uri)
    # Create a pgbouncer fixture.
    self.pgbouncer = pgbouncer.fixture.PGBouncerFixture()
    self.pgbouncer.databases[database_uri.database] = database_dsn
    self.pgbouncer.users[database_user] = "trusted"
    self.pgbouncer.admin_users = [database_user]
    self.useFixture(self.pgbouncer)
    # Create a Database that uses pgbouncer.
    pgbouncer_uri = database_uri.copy()
    pgbouncer_uri.host = self.pgbouncer.host
    pgbouncer_uri.port = self.pgbouncer.port
    # create_database() returns an object that wraps psycopg2.
    self.database = create_database(pgbouncer_uri)

def test_pgbouncer_stopped(self):
    # The error raised from a connection that is no longer connected
    # because pgbouncer has been immediately shutdown (via SIGTERM; see
    # man 1 pgbouncer) is considered a disconnection error.
    connection = self.database.connect()
    self.pgbouncer.stop()
    # DisconnectionError is from Storm.
    self.assertRaises(
        DisconnectionError, connection.execute,
        "SELECT current_database()")

I tracked down the problem using a debug build of psycopg2 2.4.2. The
problem appears to lie in pg_raise in psycopg/pqpath.c:

if (pgres) {
    err = PQresultErrorMessage(pgres);

PQresultErrorMessage() is returning an empty string.

The Storm bug is https://bugs.launchpad.net/storm/+bug/900702, and the
test that's breaking is PostgresDisconnectionTestWithPGBouncer in
tests/databases/postgres.py.

rel-2.4.1, rel-2.4.2, os-linux

Comments and changes to this ticket

  • Daniele Varrazzo

    Daniele Varrazzo December 11th, 2011 @ 09:20 PM

    Hello, thank you for the report.

    Could you please provide a couple of details? You said with recent releases you get a "psycopg went psycotic", while in 2.2.2 it wasn't "broken". What do you mean? I assume the connection is dropped anyway as it's the purpose of your test. What is the behaviour of 2.2.2? The "DisconnectionError" you test for is something raised by your wrapper: what is the psycopg does differently in the two versions?

    Thank you very much.

  • Daniele Varrazzo

    Daniele Varrazzo December 15th, 2011 @ 06:10 PM

    • State changed from “new” to “resolved”

    The bug should be fixed in my devel branch: a DatabaseError should be raised instead of an Error when we hit that case.

  • Gavin Panella

    Gavin Panella December 18th, 2011 @ 10:48 PM

    Hi Daniele,

    Sorry for the wait. I've tested Storm trunk (r427) with https://github.com/dvarrazzo/psycopg (2af563227a95a86e7d303716323d21294716b4a5) and it no longer goes psycotic \o/

    Instead I get:

      psycopg2.DatabaseError: error with no message from the libpq

    which seems fine. However, is this safe to treat as a disconnection error? I assume so, but perhaps there's something else I should be aware of.

    Thank you enormously for working on this!

    Gavin.

  • Daniele Varrazzo

    Daniele Varrazzo December 18th, 2011 @ 11:21 PM

    The class and message you get is actually what intended, so the patch worked as expected.

    DatabaseError is not actually a synonym for a disconnection: it's rather a catch-all - see the exceptions inheritance layout at http://initd.org/psycopg/docs/module.html#exceptions. It is relatively safe to assume that whatever error you get at connection time... is a connection error :) it's still too early to get more complicated errors :) (assuming we are talking about the catch statement at storm/database.py:347 - tell me if I'm wrong) However, in the DBAPI exception tree, Error is a yet more generic exception: catching it wouldn't have raised the problem in first place.

    You have probably made more tests than me about what happens brutally disconnecting the connection. Without using pgbouncer I've not been able to reproduce the condition raising that exception - in fact the author thought if you got there it would have been a psycopg bug instead - hence the cheeky error message. I would have rather raised an InterfaceError, but now that we know there is actually a way to trick the libpq to return an error without a message, we will keep it as DatabaseError.

    I can only ask you to report any weird behaviour you get in corner cases: I'll try to make psycopg the most consistent I can. Test cases are also welcome for inclusion in our test suite.

    Cheers!

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile ยป

<b>WARNING:</b> the informations in this tracker are archived. Please submit new tickets or comments to <a href="https://github.com/psycopg/psycopg2/issues">the new tracker</a>.
<br/>
Psycopg is the most used PostgreSQL adapter for the Python programming language. At the core it fully implements the Python DB API 2.0 specifications. Several extensions allow access to many of the features offered by PostgreSQL.

Shared Ticket Bins

People watching this ticket

Pages