Psycopg psycopg

Comments and changes to this ticket

• 

  Federico Di Gregorio August 17th, 2012 @ 07:28 AM

  • State changed from “new” to “invalid”

  Build-time linking and run-time linking are different things. This is how run-time (dynamic) linking works: you need to put the libpq library into one of the default locations (see /etc/ld.so.conf and "man ld.so) or setup your environment to make sure the dynamic linker finds it (LD_LIBRARY_PATH is fine for that).

• You flagged this item as spam.
  

  Lou Picciano August 17th, 2012 @ 11:37 AM

  Federico, Tks for that response. As I mentioned, I am able to direct psycopg2 to libpq at runtime with the LD_LIBRARY_PATH (meant to be only a 'technology demonstration' (sic)!) Doing this, psycopg runs fine.

  I'd like to link in libpq at build-time, however, specific to this pairing of PG and Python. There isn't a standard location I can put things into (this is messy, anyway), as we will have several different versions of both PostgreSQL and Python.

  Solaris doesn't use the /etc/ld.so.conf approach; we've got the crle tool - which we also avoid like the plague! It wouldn't be appropriate to make such changes in the global environment.

  Is there some magic for helping a python setup.py build link to libpq?

• 

  Federico Di Gregorio August 17th, 2012 @ 11:50 AM

  setup.py is linking in libpq. What you want is to embed a statically linked version of libpq into the _psycopg.so module. You can try to pass the --static-libpq option to setup.py but I don't know if it will have any effect on Solaris because I never tried it before there.

• You flagged this item as spam.
  

  Lou Picciano August 17th, 2012 @ 12:25 PM

  Thanks for your help here.
  I've tried with static_libpq=1 in setup.cfg; after a successful build, output of ldd is:

  $ ldd build/lib.solaris-2.11-i86pc-2.6/psycopg2/_psycopg.so   
          libpython2.6.so.1.0 =>   /usr/lib/libpython2.6.so.1.0
          libpq.so.5 =>    (file not found)
          libgcc_s.so.1 =>         /usr/sfw/lib/libgcc_s.so.1
          libdl.so.1 =>    /lib/libdl.so.1
          libm.so.2 =>     /lib/libm.so.2
          libc.so.1 =>     /lib/libc.so.1
  

  However, isn't static linking exactly what I don't want to do? We build various versions of PostgreSQL - even within a given version - sometimes several times a week, doing various testing. I wouldn't want to rebuild the entire chain each time...

  (I want you to know, btw; we do use psycopg2 quite a bit already - it's great! So far, though we've had to work around this issue by setting the runtime path to libpq - this is what I am trying to compile my way around now!)

• 

  Federico Di Gregorio August 17th, 2012 @ 01:10 PM

  You're getting it wrong. Let me try to explain. Under Linux (and Solaris I guess) you can do dynamic or static linking. Static linking means including a version of the library (libpq in our case) into the target executale (_psycopg.so in our case). It is good because you don't have runtime dependencies but it is bad bacause any change to the linked library needs a rebuild. The you have dynamic linking: you depend on your OS runtime linker to find the correct libraries. In your case the libpq shared object is in a non-standard location so your runtime linker can't find it. You have two possibilities:

  1. Use LD_LIBRARY_PATH &c, as you already do.
  2. Use RPATH, i.e., force the linker (the gcc/ld linked, not the runtime one) to encode into the executable the full path to the dynamic library. I don't know how to that under Linux (except when using libtool) and all distros agree that This Is Bad (TM), but if you want to try this on Solaris you can start here: http://www.eyrie.org/~eagle/notes/rpath.html

  Note that we don't currently support RPATH in setup.py but we'll gladly accept a patch if you ever happen to make one.

  Also, note that you don't need multiple versions of libpq: the newest one can connect to all (older( database versions.

• You flagged this item as spam.
  

  Lou Picciano August 19th, 2012 @ 12:54 PM

  Federico, Again, thank you for your notes.

  Respectfully, I think I may be actually getting it right... I do wish to link dynamically to libpq

  Showing results with LD_LIBRARY_PATH was only meant to be illustrative; problem is: cannot get this linking to persist beyond the current environment. In local terms, I can establish the link with either -L or -R flags at build/runtime. No, have no plan to hack this with a fragile runtime solution.

  More maddening: the build proceeds without error; in fact, the gcc output even indicates successful linking to lpq, and finds the appropriate libpq.so.(.5?, as I recall...)

  We do need multiple versions of libpq - for reasons which have nothing to do with with back end server versions! This is essentially a testbed/sandbox environment, with multiple versions of everything.

  Which lead me to the solution: as various bits of our food chain have been built with various compilers, the problem was there. As soon as I linked in the appropriate libgcc runtime, things started working!

  Again, thanks for your feedback. The sounding board/feedback helped me get there.