Psycopg psycopg → psycopg2 2.5.3

Comments and changes to this ticket

• 

  Daniele Varrazzo February 6th, 2014 @ 02:05 PM

  • State changed from “new” to “open”

  I would say no: users of that property don't expect an extra query to be performed to check the connection is still up. For one, it would become a blocking operation where people don't expect to be blocked.

  You can subclass the connection and execute a "select 1" on cnn.closed to perform your own probe. Or call cnn.poll() and catch any error.

  Note: looking into that I've found a couple of buglets:

  1) poll() on a broken connection doesn't mark the connection closed (so, if you want to play with poll, you better just trust the exception, or check cnn.get_transaction_status().

  In [4]: cnn.poll()
  ---------------------------------------------------------------------------
  OperationalError                          Traceback (most recent call last)
  /home/piro/<ipython-input-4-2d99764f7d0a> in <module>()
  ----> 1 cnn.poll()
  
  OperationalError: SSL SYSCALL error: EOF detected
  
  
  In [5]: cnn.closed
  Out[5]: 0
  

  2) the connection.closed docstring is wrong: it claims to be a boolean whereas it assumes values 0, 1, 2 where 2 means connection broken. Also, 2 is not mentioned in the docs.

  Leaving this bug open to fix these two points.

  Note: in an ideal world we shouldn't have cnn.closed and cnn.status attributes on the connection structure but only rely on PQtransactionStatus (the property exposed to python as get_transaction_status). I'm tempted to give a go at dropping the C attributes, reimplementing these properties reading PQtransactionStatus and see if anything breaks in the test suite.

• 

  Daniele Varrazzo March 6th, 2014 @ 06:19 PM

  • State changed from “open” to “resolved”

  Pushed a change for this bug in a github branch.

  https://github.com/dvarrazzo/psycopg/commits/connection-closed

  Will merge to master and release in 2.5.3 but first I want to fix poll too. Zaar, if you can test it it would be welcome.

• You flagged this item as spam.
  

  Zaar Hai March 10th, 2014 @ 07:03 AM

  Master does not compile:

  x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.6.dev0 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x09010C -DPG_VERSION_HEX=0x09010C -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/9.1/server -c psycopg/connection_type.c -o build/temp.linux-x86_64-2.7/psycopg/connection_type.o -Wdeclaration-after-statement -Wdeclaration-after-statement
  
  psycopg/connection_type.c:1037:13: error: ‘psyco_conn_get_exception’ undeclared here (not in a function)
  
       { #exc, psyco_conn_get_exception, NULL, exc ## _doc, &exc }
  
               ^
  
  psycopg/connection_type.c:1041:5: note: in expansion of macro ‘EXCEPTION_GETTER’
  
       EXCEPTION_GETTER(Error),
  
       ^
  
  error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
  

• 

  Daniele Varrazzo March 10th, 2014 @ 11:19 AM

  It does for me. psyco_conn_get_exception is declared in the same file at line 835 in current master.

• You flagged this item as spam.
  

  Zaar Hai March 10th, 2014 @ 02:36 PM

  I knew that :)

  Don't know what is the problem. Just run

  pip install -e git+https://github.com/dvarrazzo/psycopg.git#egg=psycopg2
  

  Simple pip install psycopg2 works flawlessly.

  I guess I'll wait for official 2.5.3.

  Thanks for fixing this issue!

  Zaar

• 

  Daniele Varrazzo March 10th, 2014 @ 02:44 PM

  Uhm... I was going to reply you that you can pip-install a project from a branch: you could have tested my branch with:

  pip install -e git+https://github.com/dvarrazzo/psycopg.git@connection-closed#egg=psycopg2
  

  but actually I tried that... and triggered your bug. So I'll check how to fix it.

  gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DHAVE_MXDATETIME=1 -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.5.3.dev0 (dt dec mx pq3 ext)" -DPG_VERSION_HEX=0x090303 -DPG_VERSION_HEX=0x090303 -I/usr/include/python2.7/mx -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/9.3/server -c psycopg/connection_type.c -o build/temp.linux-x86_64-2.7/psycopg/connection_type.o -Wdeclaration-after-statement -Wdeclaration-after-statement
  
  psycopg/connection_type.c:1039:5: error: ‘psyco_conn_get_exception’ undeclared here (not in a function)
  
  error: command 'gcc' failed with exit status 1
  

  first attempt to reproduce the issue failed: "make" and "python setup.py build" in the checkout dir work. mmm...

• 

  Daniele Varrazzo March 10th, 2014 @ 03:01 PM

  The difference is that the failing version is building without:

  -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1
  

  and we haven't really tested using no PSYCOPG_EXTENSIONS in this century. I wonder why the setup.py/.cfg is not doing what expected: that's new.

• 

  Daniele Varrazzo March 10th, 2014 @ 04:40 PM

  For what I can see it's a pip bug which doesn't take in consideration the setup.cfg. I've reported the issue at https://github.com/pypa/pip/issues/1630

  Zaar, if you can help me testing, just check out psycopg from the branch in github and install it with a python setup.py install. That should just work.

• 

  Daniele Varrazzo March 10th, 2014 @ 05:28 PM

  • State changed from “resolved” to “open”
  • Milestone set to “psycopg2 2.5.3”

  This bug was closed by mistake: the one to be closed should have been the #196. To be reviewed for 2.5.3

• You flagged this item as spam.
  

  Dave March 25th, 2014 @ 01:22 AM

  Daniele,
  Don't know if this is related but when opening a connection I do the following.

  class Conn(psycopg2.extensions.connection):
      def __init__(self, dsnStr):
          try:
              super(Conn, self).__init__(dsnStr)
              print "Connection using < %s > is successful" % self.dsn
          except:
              self.close() # explicit close otherwise 'connection.closed' reads as open still
              print "Connection using < %s > failed" % self.dsn
      
      # bunch of db utility methods here
      
  conn = Conn('user=dpostgres password=fght98 host=localhost')
  if conn.closed == 0:
      # Do something with the db
  else:
      # Ask user to correct dsn parameters and try again
  

  If the connection attempt fails, the exception is caught.... but the closed attribute
  has already been set to 0 which should be 1 at this point. That's why I do a close().

  Maybe I'm doing it wrong and should not attempt to do anything with the class if
  the connection fails and just return False; but having access to the dsn attribute is a good thing.

  Cheers
  :)

• 

  Daniele Varrazzo March 25th, 2014 @ 08:39 AM

  Yes, I'd say it's related. It's a sort of corner case but I'll try to iron out that too for next release.

• You flagged this item as spam.
  

  Dave March 26th, 2014 @ 10:40 PM

  On the same subject,
  

  connection.dsn
  

  will normally return a hidden password, however if the connection class instantiation is partial due to a connection error as per my previous example,
  

  connection.dsn
  

  will now expose the password used in the attempted connection.
• 

  Daniele Varrazzo April 3rd, 2014 @ 12:43 AM

  Fixed the problems with dsn and closed in connection subclasses. Please open a different issue if you have something else to report :)

• 

  Daniele Varrazzo April 3rd, 2014 @ 10:35 AM

  Open ticket #204 to deal with the pip -e problem

• 

  Daniele Varrazzo April 5th, 2014 @ 02:41 PM

  • State changed from “open” to “invalid”

  Fixed the poll() and docs reworded. However poll() doesn't notice immediately either that the connection got closed: it takes another roundtrip. Example:

  connect and restart the server

  In [2]: import psycopg2    
  In [3]: cnn = psycopg2.connect('')
  
  In [4]: !sudo /etc/init.d/postgresql stop 9.2
   * Stopping PostgreSQL 9.2 database server   ...  [ OK ]
  

  the first poll succeeds, although a notice is sent that the server restarted

  In [5]: cnn.poll()
  Out[5]: 0
  
  In [6]: cnn.notices
  Out[6]: ['FATAL:  terminating connection due to administrator command\n']
  

  the libpq still thinks the connection is good

  In [7]: cnn.get_transaction_status()
  Out[7]: 0
  

  the following poll finds the connection broken

  In [8]: cnn.poll()
  ---------------------------------------------------------------------------
  OperationalError                          Traceback (most recent call last)
  /home/piro/dev/psycopg2/build/lib.2.7/<ipython-input-8-2d99764f7d0a> in <module>()
  ----> 1 cnn.poll()
  
  OperationalError: 
  
  In [9]: cnn.get_transaction_status()
  Out[9]: 4
  

  but at least now (psycopg 2.5.3 the connection is marked closed)

  In [10]: cnn.closed
  Out[10]: 2
  

  Closing this bug as I've cleaned up all the glitches that cropped out of it but closed will stay a client-side only operation. Cheers!