I have a package with a rather extensive test suite, which I maintain with very low frequency of changes. From a time to time I forget to install a component needed for testing or that my changes break the test code. And very often when this happens, setuptools causes a hiding of the real cause of the problem.
Here is an example:
$ python setup.py test
running test
running egg_info
writing requirements to pwman3.egg-info/requires.txt
writing pwman3.egg-info/PKG-INFO
writing top-level names to pwman3.egg-info/top_level.txt
writing dependency_links to pwman3.egg-info/dependency_links.txt
writing entry points to pwman3.egg-info/entry_points.txt
reading manifest file 'pwman3.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'pwman3.egg-info/SOURCES.txt'
running build_ext
Traceback (most recent call last):
File "setup.py", line 361, in <module>
'console_scripts': ['pwman3 = pwman.ui.cli:main']
File "/usr/lib64/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/usr/lib64/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/usr/lib64/python2.7/site-packages/setuptools/command/test.py", line 146, in run
self.with_project_on_sys_path(self.run_tests)
File "/usr/lib64/python2.7/site-packages/setuptools/command/test.py", line 127, in with_project_on_sys_path
func()
File "/usr/lib64/python2.7/site-packages/setuptools/command/test.py", line 167, in run_tests
testRunner=self._resolve_as_ep(self.test_runner),
File "/usr/lib64/python2.7/unittest/main.py", line 94, in __init__
self.parseArgs(argv)
File "/usr/lib64/python2.7/unittest/main.py", line 149, in parseArgs
self.createTests()
File "/usr/lib64/python2.7/unittest/main.py", line 158, in createTests
self.module)
File "/usr/lib64/python2.7/unittest/loader.py", line 130, in loadTestsFromNames
suites = [self.loadTestsFromName(name, module) for name in names]
File "/usr/lib64/python2.7/unittest/loader.py", line 100, in loadTestsFromName
parent, obj = obj, getattr(obj, part)
AttributeError: 'module' object has no attribute 'test_pwman'
Sometimes, I spend a few frustrating minutes trying to understand why I get this error, despite have a python module called test_pwman inside the test directory.
The relevant code inside setup.py is:
from setuptools.command.install import install
setup(name=pwman.appname,
...
test_suite='tests.test_pwman.suite',
...
)
After spending some time staring at the screen, I would remember I can run the test suite like this:
$ python -m tests.test_pwman
Which reveals, for example a real cause for tests not running:
Traceback (most recent call last):
File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/home/ozn/Software/pwman3/tests/test_pwman.py", line 27, in <module>
from .test_postgresql import TestPostGresql
File "tests/test_postgresql.py", line 26, in <module>
import psycopg2 as pg
File "/home/ozn/.virtualenvs/pwman3/lib/python2.7/site-packages/psycopg2/__init__.py", line 50, in <module>
from psycopg2._psycopg import BINARY, NUMBER, STRING, DATETIME, ROWID
ImportError: libpq.so.5: cannot open shared object file: No such file or directory
After installing Postgresql with Python support, I could run my tests. So my question is:
How do you cause setuptools to propagate the real exception?
Is my code calling the test suite from setup.py wrong?
See http://bugs.python.org/issue7559, it's a bug in unittest module.
As pointed out in iElectric's answer, it is a know bug that has been fixed in python 3.5. However, I see you are using python 2.7.
The bug fix has been back ported to unittest2 which will solve the problem for you in python 2.7.
To use unittest2, install it (pip install unittest2) simply replace import unittest with import unittest2.
The unittest2 is maintained by Robert Collins who is listed a an official expert of the stdlib unitest module, so you can be assured that you are in good hands.
Related
I'm attempting to briefcase a hello-world type script, from a virtual environment created using pipenv. My original python installation building using Anaconda, though I don't really need it as I don't use any of the scientific computing stack. I am not sure what I'm experiencing is a pipenv error, a pip error, or a briefcase error. If you could help me sort this, I would really appreciate it.
Briefcase
(root) C:\Users\stmwr\Dropbox\SoftwareProjects\helloworld-br\helloworld>python setup.py windows
running windows
Traceback (most recent call last):
File "setup.py", line 73, in <module>
'app_requires': [
File "C:\Users\stmwr\Anaconda3\lib\site-packages\setuptools\__init__.py", line 129, in setup
return distutils.core.setup(**attrs)
File "C:\Users\stmwr\Anaconda3\lib\distutils\core.py", line 148, in setup
dist.run_commands()
File "C:\Users\stmwr\Anaconda3\lib\distutils\dist.py", line 955, in run_commands
self.run_command(cmd)
File "C:\Users\stmwr\Anaconda3\lib\distutils\dist.py", line 973, in run_command
cmd_obj.ensure_finalized()
File "C:\Users\stmwr\Anaconda3\lib\distutils\cmd.py", line 107, in ensure_finalized
self.finalize_options()
File "C:\Users\stmwr\Anaconda3\lib\site-packages\briefcase\windows.py", line 18, in finalize_options
finalized = self.get_finalized_command('app')
File "C:\Users\stmwr\Anaconda3\lib\distutils\cmd.py", line 299, in get_finalized_command
cmd_obj.ensure_finalized()
File "C:\Users\stmwr\Anaconda3\lib\distutils\cmd.py", line 107, in ensure_finalized
self.finalize_options()
File "C:\Users\stmwr\Anaconda3\lib\site-packages\briefcase\app.py", line 123, in finalize_options
pip.utils.ensure_dir(self.download_dir)
AttributeError: module 'pip' has no attribute 'utils'
It's likely that this is an issue with Pipenv not supporting Pip 10 yet; in Pip 10, all internal APIs were moved around, which broke all applications that depended on them. I believe the aim is to have a release out soon.
If you can wait a couple of days and then update Pipenv, that will probably be easiest. If you can't wait, you could try to downgrade Pip to version 9.0.3, which should work again.
I have recently played around with different versions of Python 2.7 and I ended up breaking PyCharm.
Firstly I have uninstalled and reinstalled Python and Pycharm multiple times with no luck.
I get the feeling that setup tools is the main issue but I can't seem to get it uninstalled correctly...
Below is the error Pycharm is giving and I am seeing similar errors trying to fix setuptools manually.
Anybody have a clue what is wrong with my environment?
Install package failed.
The following command was executed:
c:\users\ask\appdata\local\temp\tmpf3zyjdpycharm-management\setuptools-1.1.5\setup.py install
The error output of the command:
Traceback (most recent call last):
File "c:\users\ask\appdata\local\temp\tmpf3zyjdpycharm-management\setuptools-1.1.5\setup.py", line 204, in <module>
dist = setuptools.setup(**setup_params)
File "C:\Python27\lib\distutils\core.py", line 152, in setup
dist.run_commands()
File "C:\Python27\lib\distutils\dist.py", line 953, in run_commands
self.run_command(cmd)
File "C:\Python27\lib\distutils\dist.py", line 972, in run_command
cmd_obj.run()
File "c:\users\ask\appdata\local\temp\tmpf3zyjdpycharm-management\setuptools-1.1.5\setuptools\command\install.py", line 73, in run
self.do_egg_install()
File "c:\users\ask\appdata\local\temp\tmpf3zyjdpycharm-management\setuptools-1.1.5\setuptools\command\install.py", line 82, in do_egg_install
easy_install = self.distribution.get_command_class('easy_install')
File "c:\users\ask\appdata\local\temp\tmpf3zyjdpycharm-management\setuptools-1.1.5\setuptools\dist.py", line 363, in get_command_class
self.cmdclass[command] = cmdclass = ep.load()
File "c:\users\ask\appdata\local\temp\tmpf3zyjdpycharm-management\setuptools-1.1.5\pkg_resources.py", line 2108, in load
entry = __import__(self.module_name, globals(),globals(), ['__name__'])
File "c:\users\ask\appdata\local\temp\tmpf3zyjdpycharm-management\setuptools-1.1.5\setuptools\command\easy_install.py", line 34, in <module>
from setuptools.sandbox import run_setup
File "c:\users\ask\appdata\local\temp\tmpf3zyjdpycharm-management\setuptools-1.1.5\setuptools\sandbox.py", line 209, in <module>
_EXCEPTIONS.append(GetGeneratePath())
File "win32com\client\gencache.pyc", line 131, in GetGeneratePath
AssertionError: Why do you want the genpath for a readonly store?
I finally managed to get this resolved. I installed a library called Automa about the same time I started getting the issue. With the install I had to add a PYTHONPATH entry and when I removed it today setup tools was able to install again.
I have added the PYTHONPATH back again and I am still able to install libraries with setupTools/Pip. So it must have not liked the path while installing.
I very happy to be back on track again!!!! :)
I'm trying to run Hadoopy, but am getting a compiling error on OS X:
ImportError: Building module failed: ["CompileError: command 'llvm-gcc-4.2' failed with exit status 1\n"
I have /Developer/usr/bin in my $PATH, and am running latest version of XCode on OS X Lion 10.7. Cython was installed via easy_install.
Full output:
>>> import pyximport; pyximport.install()
>>> import hadoopy
/Users/dolan/.pyxbld/temp.macosx-10.7-intel-2.7/pyrex/hadoopy/_main.c:236:22: error: getdelim.h: No such file or directory
/Users/dolan/.pyxbld/temp.macosx-10.7-intel-2.7/pyrex/hadoopy/_main.c:236:22: error: getdelim.h: No such file or directory
/Users/dolan/.pyxbld/temp.macosx-10.7-intel-2.7/pyrex/hadoopy/_main.c: In function ‘__pyx_f_7hadoopy_5_main_11HadoopyTask_read_offset_value_text’:
/Users/dolan/.pyxbld/temp.macosx-10.7-intel-2.7/pyrex/hadoopy/_main.c:4399: warning: implicit conversion shortens 64-bit value into a 32-bit value
lipo: can't open input file: /var/folders/8b/n0j5pn_13qn_x8p2v4f848zh0000gn/T//ccC8x2Ex.out (No such file or directory)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Python/2.7/site-packages/hadoopy/__init__.py", line 22, in <module>
from _main import run, print_doc_quit
File "/Library/Python/2.7/site-packages/Cython-0.15.1-py2.7-macosx-10.7-intel.egg/pyximport/pyximport.py", line 335, in load_module
self.pyxbuild_dir)
File "/Library/Python/2.7/site-packages/Cython-0.15.1-py2.7-macosx-10.7-intel.egg/pyximport/pyximport.py", line 183, in load_module
so_path = build_module(name, pyxfilename, pyxbuild_dir)
File "/Library/Python/2.7/site-packages/Cython-0.15.1-py2.7-macosx-10.7-intel.egg/pyximport/pyximport.py", line 167, in build_module
reload_support=pyxargs.reload_support)
File "/Library/Python/2.7/site-packages/Cython-0.15.1-py2.7-macosx-10.7-intel.egg/pyximport/pyxbuild.py", line 85, in pyx_to_dll
dist.run_commands()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/Library/Python/2.7/site-packages/Cython-0.15.1-py2.7-macosx-10.7-intel.egg/Cython/Distutils/build_ext.py", line 135, in run
_build_ext.build_ext.run(self)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/command/build_ext.py", line 340, in run
self.build_extensions()
File "/Library/Python/2.7/site-packages/Cython-0.15.1-py2.7-macosx-10.7-intel.egg/Cython/Distutils/build_ext.py", line 143, in build_extensions
self.build_extension(ext)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/command/build_ext.py", line 499, in build_extension
depends=ext.depends)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/ccompiler.py", line 624, in compile
self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/unixccompiler.py", line 180, in _compile
raise CompileError, msg
ImportError: Building module failed: ["CompileError: command 'llvm-gcc-4.2' failed with exit status 1\n"
For me it was an installation issue and i had fixed it sometime back using below mentioned steps:
pip install argparse
git clone https://github.com/bwhite/hadoopy.git
cd hadoopy
python setup.py install
Instead of using pyximport, build the extension modules in place with python setup.py build_ext --inplace (or install for in-place development with python setup.py develop, or just a regular install via python setup.py install). For packages you almost always want to run the setup, which will properly configure the compilation environment, build, and installation process.
pyximport is good for your personal scripts if you're using Cython to speed up your code (e.g. for scientific computing). Even then you might need to bring in other libraries, and build from multiple sources. In that case, you can use a pyxbld file to set up the sources and include_dirs. For example, say you have foo.pyx, then you can place build instructions in a foo.pyxbld in the same directory. For example:
#foo.pyxbld
def make_ext(modname, pyxfilename):
from distutils.extension import Extension
return Extension(name = modname,
sources=[pyxfilename, 'bar.c'],
include_dirs=['/myinclude'] )
def make_setup_args():
return dict(script_args=["--compiler=mingw32"])
Now using foo is as simple as the following:
import pyximport
pyximport.install()
import foo
Presuming there's no compilation errors, it's virtually transparent but for the delay the first time you import the module. Subsequent imports will look for the built extension in the .pyxbld subdirectory of your home/profile directory.
I'm not sure I'm organizing my package structure correctly or am using the right options in setup.py because I'm getting errors when I try to run unit tests.
I have a structure like this:
/project
/bin
/src
/pkgname
__init__.py
module1.py
module2.py
/tests
__init__.py
test1.py
test2.py
My setup.py looks like this:
#!/usr/bin/env python
from setuptools import setup, find_packages
setup(version='0.1',
description='Trend following library',
author='Nate Reed',
author_email='nate#natereed.com',
packages=find_packages(),
install_requires=['numpy'],
test_suite="tests",
)
When I run 'python setup.py test' I get:
nate#nate-desktop:~/PycharmProjects/trendfollowing$ sudo python setup.py test
running test
running egg_info
writing requirements to UNKNOWN.egg-info/requires.txt
writing UNKNOWN.egg-info/PKG-INFO
writing top-level names to UNKNOWN.egg-info/top_level.txt
writing dependency_links to UNKNOWN.egg-info/dependency_links.txt
reading manifest file 'UNKNOWN.egg-info/SOURCES.txt'
writing manifest file 'UNKNOWN.egg-info/SOURCES.txt'
running build_ext
Traceback (most recent call last):
File "setup.py", line 11, in <module>
test_suite="tests",
File "/usr/lib/python2.6/distutils/core.py", line 152, in setup
dist.run_commands()
File "/usr/lib/python2.6/distutils/dist.py", line 975, in run_commands
self.run_command(cmd)
File "/usr/lib/python2.6/distutils/dist.py", line 995, in run_command
cmd_obj.run()
File "/usr/lib/python2.6/dist-packages/setuptools/command/test.py", line 137, in run
self.with_project_on_sys_path(self.run_tests)
File "/usr/lib/python2.6/dist-packages/setuptools/command/test.py", line 117, in with_project_on_sys_path
func()
File "/usr/lib/python2.6/dist-packages/setuptools/command/test.py", line 146, in run_tests
testLoader = loader_class()
File "/usr/lib/python2.6/unittest.py", line 816, in __init__
self.parseArgs(argv)
File "/usr/lib/python2.6/unittest.py", line 843, in parseArgs
self.createTests()
File "/usr/lib/python2.6/unittest.py", line 849, in createTests
self.module)
File "/usr/lib/python2.6/unittest.py", line 613, in loadTestsFromNames
suites = [self.loadTestsFromName(name, module) for name in names]
File "/usr/lib/python2.6/unittest.py", line 587, in loadTestsFromName
return self.loadTestsFromModule(obj)
File "/usr/lib/python2.6/dist-packages/setuptools/command/test.py", line 34, in loadTestsFromModule
tests.append(self.loadTestsFromName(submodule))
File "/usr/lib/python2.6/unittest.py", line 584, in loadTestsFromName
parent, obj = obj, getattr(obj, part)
AttributeError: 'module' object has no attribute 'test1'
Do the test names need to match module names? Are there other conventions I need to follow in my package structure?
Through some trial and error, I found the cause of this problem. Test names should match module names. If there is a "foo_test.py" test, there needs to be a corresponding module foo.py.
I found some guidelines on organizing package structure, which helped me reorganize my package into a structure I was confident in.
I am trying to install distribute using ActivePython 3.1.2 on Windows.
Running python distribute_setup.py as described on the cheese shop give me:
No setuptools distribution found
running install
Traceback (most recent call last):
File "setup.py", line 177, in
scripts = scripts,
File "C:\Dev\Python_x86\3.1\lib\distutils\core.py", line 149, in setup
dist.run_commands()
File "C:\Dev\Python_x86\3.1\lib\distutils\dist.py", line 919, in run_commands
self.run_command(cmd)
File "C:\Dev\Python_x86\3.1\lib\distutils\dist.py", line 938, in run_command
cmd_obj.run()
File "build\src\setuptools\command\install.py", line 73, in run
self.do_egg_install()
File "build\src\setuptools\command\install.py", line 82, in do_egg_install
easy_install = self.distribution.get_command_class('easy_install')
File "build\src\setuptools\dist.py", line 361, in get_command_class
self.cmdclass[command] = cmdclass = ep.load()
File "build\src\pkg_resources.py", line 1953, in load
entry = import(self.module_name, globals(),globals(), ['name'])
File "build\src\setuptools\command\easy_install.py", line 16, in
from setuptools.sandbox import run_setup
File "build\src\setuptools\sandbox.py", line 164, in
fromlist=['name']).file)
AttributeError: 'module' object has no attribute 'file'
Something went wrong during the installation.
See the error message above.
Is there possibly an unknown dependency that I'm missing?
Downloading the source tarball and executing python setup.py install produces the exact same output.
Edit: Added the full stack trace for running the installer.
So apparently the python.org version of Python3 is different from the ActiveState version of Python3. (You should file a bug to someone (I'm not sure to whom))
The fix I have (I'm not sure of all the repercussions)
Download:
http://pypi.python.org/packages/source/d/distribute/distribute-0.6.12.tar.gz#md5=5a52e961f8d8799d243fe8220f9d760e
and then extracting it and modify:
distribute-0.6.12\setuptools\sandbox.py:165
from:
except ImportError:
to
except (ImportError, AttributeError):
that will silence the error and allow you to run:
python setup.py install
It took me awhile to find a package from http://pypi.python.org/pypi?:action=browse&c=533&show=all that would actually install on either version of Python3. "files" was the first package, and since it installed I am pretty sure that easy_install is working for both copies of Python3.
...hope it works! (That's all I can help you with)
this is a bug with Distribute http://bitbucket.org/tarek/distribute/issue/151 ... it should be fixed by next release (0.6.13). It is only reproducible with PyWin32 installed; and ActivePython comes bundled with PyWin32.