Lighthouse has a new layout. Prefer the old one? Return to the old layout, and switch back any time from the link at the top of each page.

This project is archived and is in readonly mode.

psycopg2's fix_b causing other module using build_py_2to3 to fail

#153

psycopg2 monkey-patching of lib2to3.refactor (setup.py line 65) is causing any module that rely on build_py_2to3 to fail if it installs after psycopg2 (i.e. as dependencies). To demonstrate this setup.py will fails under Python 3 (BeautifulSoup4 as of 4.1.3 is using build_py_2to3, WebTest2 has BS4 as dependencies):

from setuptools import setup, find_packages

setup(name='foomod',
        version='0.0',
        description='Test',
        long_description='Just a test',
        packages=find_packages(),
        install_requires=['psycopg2==2.4.6', 'webtest==2.0.2'])

with these error message:

Traceback (most recent call last):
  File "setup.py", line 13, in <module>
    install_requires=['psycopg2==2.4.6', 'webtest==2.0.2'])
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/dist.py", line 917, in run_commands
    self.run_command(cmd)
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/dist.py", line 936, in run_command
    cmd_obj.run()
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/setuptools/command/develop.py", line 27, in run
    self.install_for_development()
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/setuptools/command/develop.py", line 129, in install_for_development
    self.process_distribution(None, self.dist, not self.no_deps)
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/setuptools/command/easy_install.py", line 682, in process_distribution
    [requirement], self.local_index, self.easy_install
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/pkg_resources.py", line 596, in resolve
    dist = best[req.key] = env.best_match(req, ws, installer)
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/pkg_resources.py", line 842, in best_match
    return self.obtain(req, installer) # try and download/install
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/pkg_resources.py", line 854, in obtain
    return installer(requirement)
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/setuptools/command/easy_install.py", line 598, in easy_install
    return self.install_item(spec, dist.location, tmpdir, deps)
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/setuptools/command/easy_install.py", line 628, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/setuptools/command/easy_install.py", line 823, in install_eggs
    return self.build_and_install(setup_script, setup_base)
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/setuptools/command/easy_install.py", line 1103, in build_and_install
    self.run_setup(setup_script, setup_base, args)
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/setuptools/command/easy_install.py", line 1089, in run_setup
    run_setup(setup_script, args)
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/setuptools/sandbox.py", line 31, in run_setup
    lambda: exec(compile(open(
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/setuptools/sandbox.py", line 79, in run
    return func()
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/setuptools/sandbox.py", line 34, in <lambda>
    {'__file__':setup_script, '__name__':'__main__'})
  File "setup.py", line 27, in <module>
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/dist.py", line 917, in run_commands
    self.run_command(cmd)
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/dist.py", line 936, in run_command
    cmd_obj.run()
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/setuptools/command/bdist_egg.py", line 179, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/setuptools/command/bdist_egg.py", line 166, in call_command
    self.run_command(cmdname)
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/dist.py", line 936, in run_command
    cmd_obj.run()
  File "/Users/sirn/.virtualenvs/foo/lib/python3.3/site-packages/distribute-0.6.31-py3.3.egg/setuptools/command/install_lib.py", line 20, in run
    self.build()
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/command/install_lib.py", line 105, in build
    self.run_command('build_py')
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/dist.py", line 936, in run_command
    cmd_obj.run()
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/command/build_py.py", line 405, in run
    self.run_2to3(self.updated_files)
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/util.py", line 573, in run_2to3
    return run_2to3(files, self.fixer_names, self.options, self.explicit)
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/util.py", line 520, in run_2to3
    r = DistutilsRefactoringTool(fixer_names, options=options)
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/lib2to3/refactor.py", line 210, in __init__
    self.pre_order, self.post_order = self.get_fixers()
  File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/lib2to3/refactor.py", line 245, in get_fixers
    mod = __import__(fix_mod_path, {}, {}, ["*"])
ImportError: No module named 'fix_b'

I've been able to reproduce this under Python 3.3 (Distribute 0.6.31) on Mac OS X and Python 3.2 (Distribute 0.6.34) on Ubuntu. Installation went through without any problems on both platnforms if I remove psycopg2 from install_requires.

Reported by Kridsada Thanabulpong · March 17th, 2013 @ 10:01 PM

State: resolved
Milestone: none
Assigned to: nobody

Activity

  1. Daniele Varrazzo
    Daniele Varrazzo

    Is there a different way to customize the list of fixers which wouldn't suffer of the shortcoming?

    March 17th, 2013 @ 10:48 PM

  2. Daniele Varrazzo
    Daniele Varrazzo

    Can you test if this fix works?

    diff --git a/setup.py b/setup.py
index 3791c14..bd0a8f3 100644
--- a/setup.py
+++ b/setup.py
@@ -55,10 +55,14 @@ from distutils.ccompiler import get_default_compiler
 from distutils.util import get_platform

 try:
-    from distutils.command.build_py import build_py_2to3 as build_py
+    from distutils.command.build_py import build_py_2to3
 except ImportError:
     from distutils.command.build_py import build_py
 else:
+    class build_py(build_py_2to3):
+        # workaround subclass for ticket [#153](/projects/62710/tickets/153 "Ticket #153")
+        pass
+
     # Configure distutils to run our custom 2to3 fixers as well
     from lib2to3.refactor import get_fixers_from_package
     build_py.fixer_names = get_fixers_from_package('lib2to3.fixes')

    March 17th, 2013 @ 10:57 PM

  3. Daniele Varrazzo
    Daniele Varrazzo

    Ignore the above patch: the problem is the list changed inplace

    March 17th, 2013 @ 11:01 PM

  4. Daniele Varrazzo
    Daniele Varrazzo

    What about this patch instead?

    Maybe subclassing build_py is not necessary but it doesn't hurt.

    diff --git a/setup.py b/setup.py
index 3791c14..96cd237 100644
--- a/setup.py
+++ b/setup.py
@@ -55,14 +55,18 @@ from distutils.ccompiler import get_default_compiler
 from distutils.util import get_platform

 try:
-    from distutils.command.build_py import build_py_2to3 as build_py
+    from distutils.command.build_py import build_py_2to3
 except ImportError:
     from distutils.command.build_py import build_py
 else:
+    class build_py(build_py_2to3):
+        # workaround subclass for ticket [#153](/projects/62710/tickets/153 "Ticket #153")
+        pass
+
     # Configure distutils to run our custom 2to3 fixers as well
     from lib2to3.refactor import get_fixers_from_package
-    build_py.fixer_names = get_fixers_from_package('lib2to3.fixes')
-    build_py.fixer_names.append('fix_b')
+    build_py.fixer_names = get_fixers_from_package('lib2to3.fixes') \
+        + [ 'fix_b' ]
     sys.path.insert(0, 'scripts')

 try:

    Let me know: if it works for you I'll commit it for next version and release in a few days.

    March 17th, 2013 @ 11:04 PM

  5. Kridsada Thanabulpong
    Kridsada Thanabulpong

    Both patch works, I can see foomod went through without any problems.

    Thanks!

    March 17th, 2013 @ 11:18 PM

  6. Daniele Varrazzo
    Daniele Varrazzo
    • State changed from new to resolved

    Fixed in devel, to be released soon.

    March 18th, 2013 @ 02:26 AM