Psycopg psycopg

Comments and changes to this ticket

• 

  Daniele Varrazzo March 9th, 2012 @ 11:45 AM

  • State changed from “new” to “resolved”
  • Tag set to “lo_truncate”

  Hello Ángel,

  thank you for testing 2.4.5, it should be released as stable in the next days.

  The error you report is not any fixed in 2.4.5: lo_truncate is a function available in the libpq (the postgres client library psycopg wraps) only from version 8.3. When psycopg is built, if the libpq version is >=8.3, then it is used, otherwise the functions using it are just not available. The problem is likely that psycopg has been compiled with lo_truncate support, but at import time it finds an older version of the library, not exposing the function, making the import fail.

  To check what is the library psycopg tries to import, run:

  ldd /usr/local/lib/python2.7/site-packages/psycopg2-2.4.4-py2.7-linux-x86_64.egg/psycopg2/_psycopg.so | grep libpq
  

  About configuration, you may have the postgresql files in very specific directories, and the setup.cfg is there exactly to allow you to customize the build. However, if the postgres installation is correct, the pg_config should be able to tell the setup where everything is. Try running:

  python setup.py build_ext --pg-config=/path/to/pg/config build
  sudo python setup.py install
  

  If it doesn't work, let me know, but it can be only expected to work if postgres has been configured, built and installed properly, so that pg_config --libdir/--includedir and similar commands return the correct files.

  I will check if the documentation requires some update to explain this problem. Thanks.

• 

  Daniele Varrazzo March 9th, 2012 @ 12:29 PM

  • Tag cleared.

  I have updated the docs with the warning box in:

  http://initd.org/psycopg/docs/extensions.html#psycopg2.extensions.l...

  and the FAQ:

  http://initd.org/psycopg/docs/faq.html#faq-lo-truncate

  Hope this helps.

• You flagged this item as spam.
  

  Ángel Guzmán Maeso March 9th, 2012 @ 04:09 PM

  Hi Daniele,

  Thanks for your detailed explanations. I appreciate a lot the feedback.

  But, if you let me clear some thinks before to mark as resolved, I would give you more info because I think that the installer is doing something wrong when try to simlink the libraries.

  This is my info for 2.4.4:

  ldd /usr/local/lib/python2.7/site-packages/psycopg2-2.4.4-py2.7-linux-x86_64.egg/psycopg2/_psycopg.so | grep libpq
      libpq.so.4 => /usr/lib64/libpq.so.4 (0x00002b523d5ee000)
  

  With 2.5.0dev all work fine:

  # python -c 'import psycopg2; print psycopg2.__version__'
  2.4.5.dev0 (dt dec pq3 ext)
  

  # ldd /usr/local/lib/python2.7/site-packages/psycopg2/_psycopg.so | grep libpq.so
      libpq.so.5 => /usr/pgsql-9.1/lib/libpq.so.5 (0x00002b52b5154000)
  

  For install 2.5.0dev I made with:

  python setup.py build
  python setup.py install
  

  With 2.4.4 it was installed from easy_install downloading the tar.gz, and running "easy_install ." because "easy_install psycopg2" show me a 404 downloading the tar.gz

  So the config info requested:

  pg_config --libdir
  /usr/lib
  

  pg_config --includedir
  /usr/include
  

  locate libpq.so
  /usr/lib/libpq.so.4
  /usr/lib/libpq.so.4.1
  /usr/lib64/libpq.so.4
  /usr/lib64/libpq.so.4.1
  /usr/local/cpanel/lib64/libpq.so.3
  /usr/local/pgsql/lib/libpq.so
  /usr/local/pgsql/lib/libpq.so.5
  /usr/local/pgsql/lib/libpq.so.5.4
  /usr/pgsql-9.1/lib/libpq.so
  /usr/pgsql-9.1/lib/libpq.so.5
  /usr/pgsql-9.1/lib/libpq.so.5.4
  /var/lib/pgsql/lib/libpq.so
  /var/lib/pgsql/lib/libpq.so.5
  /var/lib/pgsql/lib/libpq.so.5.4
  

  I made a script for find all the libpq on every _psycopg.so:

  #!/bin/env bash
  
  for lib in `locate _psycopg.so`;
  do
      echo "File: $lib"
          ldd $lib | grep libpq;
  done;
  

  Then running:

  ./libpqgrep.sh
  File: /root/psycopg2/build/lib.2.7/psycopg2/_psycopg.so
      libpq.so.5 => /usr/pgsql-9.1/lib/libpq.so.5 (0x00002ad74a207000)
  File: /root/psycopg2/build/lib.linux-x86_64-2.7/psycopg2/_psycopg.so
      libpq.so.5 => /usr/pgsql-9.1/lib/libpq.so.5 (0x00002ac9929dd000)
  File: /root/psycopg2-2.4.4/build/lib.2.7/psycopg2/_psycopg.so
      libpq.so.5 => /usr/pgsql-9.1/lib/libpq.so.5 (0x00002b31f33c5000)
  File: /root/psycopg2-2.4.4/build/lib.linux-x86_64-2.7/psycopg2/_psycopg.so
      libpq.so.4 => /usr/lib64/libpq.so.4 (0x00002b4518f4a000)
  File: /usr/lib/python2.4/site-packages/psycopg2-2.4.4-py2.4-linux-x86_64.egg/psycopg2/_psycopg.so
      libpq.so.4 => /usr/lib64/libpq.so.4 (0x00002b9db0f35000)
  File: /usr/lib64/python2.4/site-packages/psycopg2/_psycopg.so
      libpq.so.5 => /usr/pgsql-9.1/lib/libpq.so.5 (0x00002afd2c8b4000)
  File: /usr/local/lib/python2.7/site-packages/psycopg2/_psycopg.so
      libpq.so.5 => /usr/pgsql-9.1/lib/libpq.so.5 (0x00002b2f61ec3000)
  File: /usr/local/lib/python2.7/site-packages/psycopg2-2.4.4-py2.7-linux-x86_64.egg/psycopg2/_psycopg.so
      libpq.so.4 => /usr/lib64/libpq.so.4 (0x00002abc1a1ab000)
  

  As you can see I have several libpq.so versions. Some are related to lib64 (libpq.so.4) and others are from /usr/pgsql-9.1/lib/.

  The library on /usr/lib64 or /usr/pgsql-9.1/lib/ seems that doesn't have support for lo_truncate

  I try running:

   readelf -Ws /usr/lib64/libpq.so.4 | grep io_truncate
      objdump -TC /usr/lib64/libpq.so.4 | grep io_truncate
      readelf -Ws /usr/lib64/libpq.so.4 | grep io_truncate
  

  Also for /usr/pgsql-9.1/lib/libpq.so.5

  But it doesn't show any symbol related to io_truncate, so at some point e I am getting the io_truncate from libpq.so.5 but I don't know where.

  I have fear to compile again and lose the working psycopg2 because I don't know how it is associating the libpq support

• 

  Daniele Varrazzo March 9th, 2012 @ 04:28 PM

  The problem seems that when you have compiled the psycopg2 2.4.4 it has somehow found the libpq.so.5: it could have been because the relative pg_config was ahead in the PATH, for instance, or because it has been explicitly selected with "build_ext --pg-config". But now, because your libpq.so.5 is in a non standard location, it is not found. If you put the libpq.so in "/usr/pgsql-9.1/lib/" you then have to be prepared to use LD_LIBRARY_PATH to allow the dynamic linker to find it.

  Also note that "pg_config --libdir" returns "/usr/lib", so either it's not the libpq.so.5 version of pg_config or your postgres is not installed properly, i.e. where it was configured.

  So, yours is not a bug for 2.4.4: just try and install it, downloading the source package from the website if easy_istall doesn't work, and you'll see it works as good as 2.4.5 w.r.t. lo_truncate. Your problem is just a messed-up installation: you have three different version of the libpq in four places, of which three non standard, and the linker gets unsurprisingly confused.