This project is archived and is in readonly mode.

#119 ✓invalid
Lou Picciano

psycopg2-2.4.5 - and PostgreSQL 9.1.4, cannot get linked in

Reported by Lou Picciano | August 17th, 2012 @ 03:12 AM

Ciao, Daniele!

Trying to make wider use of psycopg now, with builds of latest version. With psycopg2-2.4.5 - and PostgreSQL 9.1.4, cannot get linked into psycopg2.

Have tried both the clean (build, install) approach and the
$ make clean, make approach

with similar results. Have also modified setup.cfg extensively to include /usr/pgsql/9.1/bin/pg_config, various lib directories, etc.

Please note that build and install work without any error. At runtime, however:

import psycopg2 Traceback (most recent call last):
File "", line 1, in File "/usr/lib/python2.6/site-packages/psycopg2/", line 67, in

from psycopg2._psycopg import BINARY, NUMBER, STRING, DATETIME, ROWID

ImportError: isapython2.6: fatal: open failed: No such file or directory

Of course, placing the lib in LD_LIBRARY_PATH works in the local environment.

How to get the linker to '-I' the libpq library, preferably via the setup.cfg file? (Changing the runtime environment, globally, is not an option in this context; we may have several versions of both PostgreSQL and Python in play at any given moment).

Tks in advance, Lou

Comments and changes to this ticket

  • Federico Di Gregorio

    Federico Di Gregorio August 17th, 2012 @ 07:28 AM

    • State changed from “new” to “invalid”

    Build-time linking and run-time linking are different things. This is how run-time (dynamic) linking works: you need to put the libpq library into one of the default locations (see /etc/ and "man or setup your environment to make sure the dynamic linker finds it (LD_LIBRARY_PATH is fine for that).

  • Lou Picciano

    Lou Picciano August 17th, 2012 @ 11:37 AM

    Federico, Tks for that response. As I mentioned, I am able to direct psycopg2 to libpq at runtime with the LD_LIBRARY_PATH (meant to be only a 'technology demonstration' (sic)!) Doing this, psycopg runs fine.

    I'd like to link in libpq at build-time, however, specific to this pairing of PG and Python. There isn't a standard location I can put things into (this is messy, anyway), as we will have several different versions of both PostgreSQL and Python.

    Solaris doesn't use the /etc/ approach; we've got the crle tool - which we also avoid like the plague! It wouldn't be appropriate to make such changes in the global environment.

    Is there some magic for helping a python build link to libpq?

  • Federico Di Gregorio

    Federico Di Gregorio August 17th, 2012 @ 11:50 AM is linking in libpq. What you want is to embed a statically linked version of libpq into the module. You can try to pass the --static-libpq option to but I don't know if it will have any effect on Solaris because I never tried it before there.

  • Lou Picciano

    Lou Picciano August 17th, 2012 @ 12:25 PM

    Thanks for your help here.
    I've tried with static_libpq=1 in setup.cfg; after a successful build, output of ldd is:

    $ ldd build/lib.solaris-2.11-i86pc-2.6/psycopg2/   
   =>   /usr/lib/
   =>    (file not found)
   =>         /usr/sfw/lib/
   =>    /lib/
   =>     /lib/
   =>     /lib/

    However, isn't static linking exactly what I don't want to do? We build various versions of PostgreSQL - even within a given version - sometimes several times a week, doing various testing. I wouldn't want to rebuild the entire chain each time...

    (I want you to know, btw; we do use psycopg2 quite a bit already - it's great! So far, though we've had to work around this issue by setting the runtime path to libpq - this is what I am trying to compile my way around now!)

  • Federico Di Gregorio

    Federico Di Gregorio August 17th, 2012 @ 01:10 PM

    You're getting it wrong. Let me try to explain. Under Linux (and Solaris I guess) you can do dynamic or static linking. Static linking means including a version of the library (libpq in our case) into the target executale ( in our case). It is good because you don't have runtime dependencies but it is bad bacause any change to the linked library needs a rebuild. The you have dynamic linking: you depend on your OS runtime linker to find the correct libraries. In your case the libpq shared object is in a non-standard location so your runtime linker can't find it. You have two possibilities:

    1. Use LD_LIBRARY_PATH &c, as you already do.
    2. Use RPATH, i.e., force the linker (the gcc/ld linked, not the runtime one) to encode into the executable the full path to the dynamic library. I don't know how to that under Linux (except when using libtool) and all distros agree that This Is Bad (TM), but if you want to try this on Solaris you can start here:

    Note that we don't currently support RPATH in but we'll gladly accept a patch if you ever happen to make one.

    Also, note that you don't need multiple versions of libpq: the newest one can connect to all (older( database versions.

  • Lou Picciano

    Lou Picciano August 19th, 2012 @ 12:54 PM

    Federico, Again, thank you for your notes.

    Respectfully, I think I may be actually getting it right... I do wish to link dynamically to libpq

    Showing results with LD_LIBRARY_PATH was only meant to be illustrative; problem is: cannot get this linking to persist beyond the current environment. In local terms, I can establish the link with either -L or -R flags at build/runtime. No, have no plan to hack this with a fragile runtime solution.

    More maddening: the build proceeds without error; in fact, the gcc output even indicates successful linking to lpq, and finds the appropriate, as I recall...)

    We do need multiple versions of libpq - for reasons which have nothing to do with with back end server versions! This is essentially a testbed/sandbox environment, with multiple versions of everything.

    Which lead me to the solution: as various bits of our food chain have been built with various compilers, the problem was there. As soon as I linked in the appropriate libgcc runtime, things started working!

    Again, thanks for your feedback. The sounding board/feedback helped me get there.

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile ยป

<b>WARNING:</b> the informations in this tracker are archived. Please submit new tickets or comments to <a href="">the new tracker</a>.
Psycopg is the most used PostgreSQL adapter for the Python programming language. At the core it fully implements the Python DB API 2.0 specifications. Several extensions allow access to many of the features offered by PostgreSQL.

Shared Ticket Bins

People watching this ticket