This project is archived and is in readonly mode.

#153 ✓resolved
Kridsada Thanabulpong

psycopg2's fix_b causing other module using build_py_2to3 to fail

Reported by Kridsada Thanabulpong | March 17th, 2013 @ 10:01 PM

psycopg2 monkey-patching of lib2to3.refactor (setup.py line 65) is causing any module that rely on build_py_2to3 to fail if it installs after psycopg2 (i.e. as dependencies). To demonstrate this setup.py will fails under Python 3 (BeautifulSoup4 as of 4.1.3 is using build_py_2to3, WebTest2 has BS4 as dependencies):

from setuptools import setup, find_packages

setup(name='foomod',
        version='0.0',
        description='Test',
        long_description='Just a test',
        packages=find_packages(),
        install_requires=['psycopg2==2.4.6', 'webtest==2.0.2'])

with these error message:

Traceback (most recent call last):
  File "setup.py", line 13, in <module>
    install_requires=['psycopg2==2.4.6', 'webtest==2.0.2'])
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/dist.py", line 917, in run_commands
    self.run_command(cmd)
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/dist.py", line 936, in run_command
    cmd_obj.run()
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/setuptools/command/develop.py", line 27, in run
    self.install_for_development()
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/setuptools/command/develop.py", line 129, in install_for_development
    self.process_distribution(None, self.dist, not self.no_deps)
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/setuptools/command/easy_install.py", line 682, in process_distribution
    [requirement], self.local_index, self.easy_install
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/pkg_resources.py", line 596, in resolve
    dist = best[req.key] = env.best_match(req, ws, installer)
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/pkg_resources.py", line 842, in best_match
    return self.obtain(req, installer) # try and download/install
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/pkg_resources.py", line 854, in obtain
    return installer(requirement)
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/setuptools/command/easy_install.py", line 598, in easy_install
    return self.install_item(spec, dist.location, tmpdir, deps)
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/setuptools/command/easy_install.py", line 628, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/setuptools/command/easy_install.py", line 823, in install_eggs
    return self.build_and_install(setup_script, setup_base)
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/setuptools/command/easy_install.py", line 1103, in build_and_install
    self.run_setup(setup_script, setup_base, args)
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/setuptools/command/easy_install.py", line 1089, in run_setup
    run_setup(setup_script, args)
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/setuptools/sandbox.py", line 31, in run_setup
    lambda: exec(compile(open(
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/setuptools/sandbox.py", line 79, in run
    return func()
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/setuptools/sandbox.py", line 34, in <lambda>
    {'__file__':setup_script, '__name__':'__main__'})
  File "setup.py", line 27, in <module>
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/dist.py", line 917, in run_commands
    self.run_command(cmd)
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/dist.py", line 936, in run_command
    cmd_obj.run()
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/setuptools/command/bdist_egg.py", line 179, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/setuptools/command/bdist_egg.py", line 166, in call_command
    self.run_command(cmdname)
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/dist.py", line 936, in run_command
    cmd_obj.run()
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/setuptools/command/install_lib.py", line 20, in run
    self.build()
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/command/install_lib.py", line 105, in build
    self.run_command('build_py')
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/dist.py", line 936, in run_command
    cmd_obj.run()
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/command/build_py.py", line 405, in run
    self.run_2to3(self.updated_files)
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/util.py", line 573, in run_2to3
    return run_2to3(files, self.fixer_names, self.options, self.explicit)
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/util.py", line 520, in run_2to3
    r = DistutilsRefactoringTool(fixer_names, options=options)
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/lib2to3/refactor.py", line 210, in __init__
    self.pre_order, self.post_order = self.get_fixers()
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/lib2to3/refactor.py", line 245, in get_fixers
    mod = __import__(fix_mod_path, {}, {}, ["*"])
ImportError: No module named 'fix_b'

I've been able to reproduce this under Python 3.3 (Distribute 0.6.31) on Mac OS X and Python 3.2 (Distribute 0.6.34) on Ubuntu. Installation went through without any problems on both platnforms if I remove psycopg2 from install_requires.

Comments and changes to this ticket

  • Daniele Varrazzo

    Daniele Varrazzo March 17th, 2013 @ 10:48 PM

    Is there a different way to customize the list of fixers which wouldn't suffer of the shortcoming?

  • Daniele Varrazzo

    Daniele Varrazzo March 17th, 2013 @ 10:57 PM

    Can you test if this fix works?

    diff --git a/setup.py b/setup.py
index 3791c14..bd0a8f3 100644
--- a/setup.py
+++ b/setup.py
@@ -55,10 +55,14 @@ from distutils.ccompiler import get_default_compiler
 from distutils.util import get_platform

 try:
-    from distutils.command.build_py import build_py_2to3 as build_py
+    from distutils.command.build_py import build_py_2to3
 except ImportError:
     from distutils.command.build_py import build_py
 else:
+    class build_py(build_py_2to3):
+        # workaround subclass for ticket [#153](/projects/62710/tickets/153 "Ticket #153")
+        pass
+
     # Configure distutils to run our custom 2to3 fixers as well
     from lib2to3.refactor import get_fixers_from_package
     build_py.fixer_names = get_fixers_from_package('lib2to3.fixes')
  • Daniele Varrazzo

    Daniele Varrazzo March 17th, 2013 @ 11:01 PM

    Ignore the above patch: the problem is the list changed inplace

  • Daniele Varrazzo

    Daniele Varrazzo March 17th, 2013 @ 11:04 PM

    What about this patch instead?

    Maybe subclassing build_py is not necessary but it doesn't hurt.

    diff --git a/setup.py b/setup.py
index 3791c14..96cd237 100644
--- a/setup.py
+++ b/setup.py
@@ -55,14 +55,18 @@ from distutils.ccompiler import get_default_compiler
 from distutils.util import get_platform

 try:
-    from distutils.command.build_py import build_py_2to3 as build_py
+    from distutils.command.build_py import build_py_2to3
 except ImportError:
     from distutils.command.build_py import build_py
 else:
+    class build_py(build_py_2to3):
+        # workaround subclass for ticket [#153](/projects/62710/tickets/153 "Ticket #153")
+        pass
+
     # Configure distutils to run our custom 2to3 fixers as well
     from lib2to3.refactor import get_fixers_from_package
-    build_py.fixer_names = get_fixers_from_package('lib2to3.fixes')
-    build_py.fixer_names.append('fix_b')
+    build_py.fixer_names = get_fixers_from_package('lib2to3.fixes') \
+        + [ 'fix_b' ]
     sys.path.insert(0, 'scripts')

 try:

    Let me know: if it works for you I'll commit it for next version and release in a few days.

  • Kridsada Thanabulpong

    Kridsada Thanabulpong March 17th, 2013 @ 11:18 PM

    Both patch works, I can see foomod went through without any problems.

    Thanks!

  • Daniele Varrazzo

    Daniele Varrazzo March 18th, 2013 @ 02:26 AM

    • State changed from “new” to “resolved”

    Fixed in devel, to be released soon.

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile ยป

<b>WARNING:</b> the informations in this tracker are archived. Please submit new tickets or comments to <a href="https://github.com/psycopg/psycopg2/issues">the new tracker</a>.
<br/>
Psycopg is the most used PostgreSQL adapter for the Python programming language. At the core it fully implements the Python DB API 2.0 specifications. Several extensions allow access to many of the features offered by PostgreSQL.

Shared Ticket Bins

People watching this ticket

Pages