Lighthouse has a new layout. Prefer the old one? Return to the old layout, and switch back any time from the link at the top of each page.

This project is archived and is in readonly mode.

_psycopg.so: undefined symbol: lo_truncate on 2.4.4 (no 2.4.5.dev0)

#104

I get this traceback when import psycopg2 bases on 2.4.4:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/psycopg2-2.4.4-py2.7-linux-x86_64.egg/psycopg2/__init__.py", line 67, in <module>
    from psycopg2._psycopg import BINARY, NUMBER, STRING, DATETIME, ROWID
ImportError: /usr/local/lib/python2.7/site-packages/psycopg2-2.4.4-py2.7-linux-x86_64.egg/psycopg2/_psycopg.so: undefined symbol: lo_truncate

I clone the git repo and build with setup.py and the version 2.4.5.dev0 is not affected. So please it would be nice release 2.4.5 as stable.

Maybe some change related to file psycopg/lobject_int.c solve the problem:
psycopg/lobject_int.c:478: retvalue = lo_truncate(self->conn->pgconn, self->fd, len);

Also I have to change the setup.cfg values because it doesn't found properly my config and it would fail on load libpq.so or libpq-int.h and:

pg_config=/usr/bin/pg_config
library_dirs=/usr/pgsql-9.1/lib/:/usr/lib64/:/usr/local/:/usr/local/pgsql/include/server/libpq/
include_dirs=/usr/include/pgsql/:/usr/include/pgsql/server/:/usr/local/pgsql/include/:/usr/local/pgsql/include/server/libpq/:/usr/pgsql-9.1/include/server/:/usr/local/pgsql/include/internal/

Addiontal info:

$ cat /etc/redhat-release CentOS release 5.8 (Final)

$ arch x86_64

Posgresql (compiles by sources, no RPM)
$ psql --version psql (PostgreSQL) 9.1.2
contains support for command-line editing

$ pg_config BINDIR = /usr/bin
DOCDIR = /usr/share/doc
HTMLDIR = /usr/share/doc
INCLUDEDIR = /usr/include
PKGINCLUDEDIR = /usr/include
INCLUDEDIR-SERVER = /usr/include/server
LIBDIR = /usr/lib
PKGLIBDIR = /usr/lib
LOCALEDIR = /usr/share/locale
MANDIR = /usr/share/man
SHAREDIR = /usr/share
SYSCONFDIR = /usr/etc
PGXS = /usr/lib/pgxs/src/makefiles/pgxs.mk
CONFIGURE =
CC = gcc
CPPFLAGS = -D_GNU_SOURCE
CFLAGS = -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wformat-security -fno-strict-aliasing -fwrapv
CFLAGS_SL = -fpic
LDFLAGS = -Wl,-rpath,'/usr/local/pgsql/lib',--enable-new-dtags
LDFLAGS_EX =
LDFLAGS_SL =
LIBS = -lpgport -lz -lreadline -ltermcap -lcrypt -ldl -lm
VERSION = PostgreSQL 9.1.2

Reported by Ángel Guzmán Maeso · March 9th, 2012 @ 04:25 AM

State: resolved
Milestone: none
Assigned to: nobody

Activity

  1. Daniele Varrazzo
    Daniele Varrazzo
    • State changed from new to resolved
    • Tag set to lo_truncate

    Hello Ángel,

    thank you for testing 2.4.5, it should be released as stable in the next days.

    The error you report is not any fixed in 2.4.5: lo_truncate is a function available in the libpq (the postgres client library psycopg wraps) only from version 8.3. When psycopg is built, if the libpq version is >=8.3, then it is used, otherwise the functions using it are just not available. The problem is likely that psycopg has been compiled with lo_truncate support, but at import time it finds an older version of the library, not exposing the function, making the import fail.

    To check what is the library psycopg tries to import, run:

    ldd /usr/local/lib/python2.7/site-packages/psycopg2-2.4.4-py2.7-linux-x86_64.egg/psycopg2/_psycopg.so | grep libpq

    About configuration, you may have the postgresql files in very specific directories, and the setup.cfg is there exactly to allow you to customize the build. However, if the postgres installation is correct, the pg_config should be able to tell the setup where everything is. Try running:

    python setup.py build_ext --pg-config=/path/to/pg/config build
sudo python setup.py install

    If it doesn't work, let me know, but it can be only expected to work if postgres has been configured, built and installed properly, so that pg_config --libdir/--includedir and similar commands return the correct files.

    I will check if the documentation requires some update to explain this problem. Thanks.

    March 9th, 2012 @ 11:45 AM

  2. Daniele Varrazzo
    Daniele Varrazzo
    • Tag cleared.

    March 9th, 2012 @ 12:29 PM

  3. Ángel Guzmán Maeso
    Ángel Guzmán Maeso

    Hi Daniele,

    Thanks for your detailed explanations. I appreciate a lot the feedback.

    But, if you let me clear some thinks before to mark as resolved, I would give you more info because I think that the installer is doing something wrong when try to simlink the libraries.

    This is my info for 2.4.4:

    ldd /usr/local/lib/python2.7/site-packages/psycopg2-2.4.4-py2.7-linux-x86_64.egg/psycopg2/_psycopg.so | grep libpq
    libpq.so.4 => /usr/lib64/libpq.so.4 (0x00002b523d5ee000)

    With 2.5.0dev all work fine:

    # python -c 'import psycopg2; print psycopg2.__version__'
2.4.5.dev0 (dt dec pq3 ext)

    # ldd /usr/local/lib/python2.7/site-packages/psycopg2/_psycopg.so | grep libpq.so
    libpq.so.5 => /usr/pgsql-9.1/lib/libpq.so.5 (0x00002b52b5154000)

    For install 2.5.0dev I made with:

    python setup.py build
python setup.py install

    With 2.4.4 it was installed from easy_install downloading the tar.gz, and running "easy_install ." because "easy_install psycopg2" show me a 404 downloading the tar.gz

    So the config info requested:

    pg_config --libdir
/usr/lib

    pg_config --includedir
/usr/include

    locate libpq.so
/usr/lib/libpq.so.4
/usr/lib/libpq.so.4.1
/usr/lib64/libpq.so.4
/usr/lib64/libpq.so.4.1
/usr/local/cpanel/lib64/libpq.so.3
/usr/local/pgsql/lib/libpq.so
/usr/local/pgsql/lib/libpq.so.5
/usr/local/pgsql/lib/libpq.so.5.4
/usr/pgsql-9.1/lib/libpq.so
/usr/pgsql-9.1/lib/libpq.so.5
/usr/pgsql-9.1/lib/libpq.so.5.4
/var/lib/pgsql/lib/libpq.so
/var/lib/pgsql/lib/libpq.so.5
/var/lib/pgsql/lib/libpq.so.5.4

    I made a script for find all the libpq on every _psycopg.so:

    #!/bin/env bash

for lib in `locate _psycopg.so`;
do
    echo "File: $lib"
        ldd $lib | grep libpq;
done;

    Then running:

    ./libpqgrep.sh
File: /root/psycopg2/build/lib.2.7/psycopg2/_psycopg.so
    libpq.so.5 => /usr/pgsql-9.1/lib/libpq.so.5 (0x00002ad74a207000)
File: /root/psycopg2/build/lib.linux-x86_64-2.7/psycopg2/_psycopg.so
    libpq.so.5 => /usr/pgsql-9.1/lib/libpq.so.5 (0x00002ac9929dd000)
File: /root/psycopg2-2.4.4/build/lib.2.7/psycopg2/_psycopg.so
    libpq.so.5 => /usr/pgsql-9.1/lib/libpq.so.5 (0x00002b31f33c5000)
File: /root/psycopg2-2.4.4/build/lib.linux-x86_64-2.7/psycopg2/_psycopg.so
    libpq.so.4 => /usr/lib64/libpq.so.4 (0x00002b4518f4a000)
File: /usr/lib/python2.4/site-packages/psycopg2-2.4.4-py2.4-linux-x86_64.egg/psycopg2/_psycopg.so
    libpq.so.4 => /usr/lib64/libpq.so.4 (0x00002b9db0f35000)
File: /usr/lib64/python2.4/site-packages/psycopg2/_psycopg.so
    libpq.so.5 => /usr/pgsql-9.1/lib/libpq.so.5 (0x00002afd2c8b4000)
File: /usr/local/lib/python2.7/site-packages/psycopg2/_psycopg.so
    libpq.so.5 => /usr/pgsql-9.1/lib/libpq.so.5 (0x00002b2f61ec3000)
File: /usr/local/lib/python2.7/site-packages/psycopg2-2.4.4-py2.7-linux-x86_64.egg/psycopg2/_psycopg.so
    libpq.so.4 => /usr/lib64/libpq.so.4 (0x00002abc1a1ab000)

    As you can see I have several libpq.so versions. Some are related to lib64 (libpq.so.4) and others are from /usr/pgsql-9.1/lib/.

    The library on /usr/lib64 or /usr/pgsql-9.1/lib/ seems that doesn't have support for lo_truncate

    I try running:

     readelf -Ws /usr/lib64/libpq.so.4 | grep io_truncate
    objdump -TC /usr/lib64/libpq.so.4 | grep io_truncate
    readelf -Ws /usr/lib64/libpq.so.4 | grep io_truncate

    Also for /usr/pgsql-9.1/lib/libpq.so.5

    But it doesn't show any symbol related to io_truncate, so at some point e I am getting the io_truncate from libpq.so.5 but I don't know where.

    I have fear to compile again and lose the working psycopg2 because I don't know how it is associating the libpq support

    March 9th, 2012 @ 04:09 PM

  4. Daniele Varrazzo
    Daniele Varrazzo

    The problem seems that when you have compiled the psycopg2 2.4.4 it has somehow found the libpq.so.5: it could have been because the relative pg_config was ahead in the PATH, for instance, or because it has been explicitly selected with "build_ext --pg-config". But now, because your libpq.so.5 is in a non standard location, it is not found. If you put the libpq.so in "/usr/pgsql-9.1/lib/" you then have to be prepared to use LD_LIBRARY_PATH to allow the dynamic linker to find it.

    Also note that "pg_config --libdir" returns "/usr/lib", so either it's not the libpq.so.5 version of pg_config or your postgres is not installed properly, i.e. where it was configured.

    So, yours is not a bug for 2.4.4: just try and install it, downloading the source package from the website if easy_istall doesn't work, and you'll see it works as good as 2.4.5 w.r.t. lo_truncate. Your problem is just a messed-up installation: you have three different version of the libpq in four places, of which three non standard, and the linker gets unsurprisingly confused.

    March 9th, 2012 @ 04:28 PM