This project is archived and is in readonly mode.

psycopg went psycotic without error set

#82

In Storm https://launchpad.net/storm we have various tests for
behaviour when a connection to the database is broken. One of them is
breaking with "psycopg went psycotic without error set" on psycopg2
2.4.1 and 2.4.2. It does not break with 2.2.2 and 2.0.14.

Here's a fragment of the test showing what we're trying to do:

def setUp(self):
    super(...).setUp()
    database_uri = URI(...)
    database_user = database_uri.username
    database_dsn = make_dsn(database_uri)
    # Create a pgbouncer fixture.
    self.pgbouncer = pgbouncer.fixture.PGBouncerFixture()
    self.pgbouncer.databases[database_uri.database] = database_dsn
    self.pgbouncer.users[database_user] = "trusted"
    self.pgbouncer.admin_users = [database_user]
    self.useFixture(self.pgbouncer)
    # Create a Database that uses pgbouncer.
    pgbouncer_uri = database_uri.copy()
    pgbouncer_uri.host = self.pgbouncer.host
    pgbouncer_uri.port = self.pgbouncer.port
    # create_database() returns an object that wraps psycopg2.
    self.database = create_database(pgbouncer_uri)

def test_pgbouncer_stopped(self):
    # The error raised from a connection that is no longer connected
    # because pgbouncer has been immediately shutdown (via SIGTERM; see
    # man 1 pgbouncer) is considered a disconnection error.
    connection = self.database.connect()
    self.pgbouncer.stop()
    # DisconnectionError is from Storm.
    self.assertRaises(
        DisconnectionError, connection.execute,
        "SELECT current_database()")

I tracked down the problem using a debug build of psycopg2 2.4.2. The
problem appears to lie in pg_raise in psycopg/pqpath.c:

if (pgres) {
    err = PQresultErrorMessage(pgres);

PQresultErrorMessage() is returning an empty string.

The Storm bug is https://bugs.launchpad.net/storm/+bug/900702, and the
test that's breaking is PostgresDisconnectionTestWithPGBouncer in
tests/databases/postgres.py.

rel-2.4.1, rel-2.4.2, os-linux

Reported by Gavin Panella · December 7th, 2011 @ 12:18 PM

Activity

Daniele Varrazzo
Hello, thank you for the report.

Could you please provide a couple of details? You said with recent releases you get a "psycopg went psycotic", while in 2.2.2 it wasn't "broken". What do you mean? I assume the connection is dropped anyway as it's the purpose of your test. What is the behaviour of 2.2.2? The "DisconnectionError" you test for is something raised by your wrapper: what is the psycopg does differently in the two versions?

Thank you very much.

December 11th, 2011 @ 09:20 PM
Daniele Varrazzo
- State changed from new to resolved
The bug should be fixed in my devel branch: a DatabaseError should be raised instead of an Error when we hit that case.

December 15th, 2011 @ 06:10 PM
Gavin Panella
Hi Daniele,

Sorry for the wait. I've tested Storm trunk (r427) with https://github.com/dvarrazzo/psycopg (2af563227a95a86e7d303716323d21294716b4a5) and it no longer goes psycotic \o/

Instead I get:
```
  psycopg2.DatabaseError: error with no message from the libpq
```
which seems fine. However, is this safe to treat as a disconnection error? I assume so, but perhaps there's something else I should be aware of.

Thank you enormously for working on this!

Gavin.
December 18th, 2011 @ 10:48 PM
Daniele Varrazzo
The class and message you get is actually what intended, so the patch worked as expected.

DatabaseError is not actually a synonym for a disconnection: it's rather a catch-all - see the exceptions inheritance layout at http://initd.org/psycopg/docs/module.html#exceptions. It is relatively safe to assume that whatever error you get at connection time... is a connection error :) it's still too early to get more complicated errors :) (assuming we are talking about the catch statement at storm/database.py:347 - tell me if I'm wrong) However, in the DBAPI exception tree, Error is a yet more generic exception: catching it wouldn't have raised the problem in first place.

You have probably made more tests than me about what happens brutally disconnecting the connection. Without using pgbouncer I've not been able to reproduce the condition raising that exception - in fact the author thought if you got there it would have been a psycopg bug instead - hence the cheeky error message. I would have rather raised an InterfaceError, but now that we know there is actually a way to trick the libpq to return an error without a message, we will keep it as DatabaseError.

I can only ask you to report any weird behaviour you get in corner cases: I'll try to make psycopg the most consistent I can. Test cases are also welcome for inclusion in our test suite.

Cheers!

December 18th, 2011 @ 11:21 PM