I'm using tox to test a module that has a script entry point defined in my setuptools-based setup.py script.
entry_points={
'console_scripts': [
'myscript = mypackage:main',
],
}
tox is configured thus:
[testenv]
commands = pytest --cov=mypackage --cov-report=html --cov-append tests/
deps =
pytest
pytest-cov
usedevelop = True
As expected, the file .tox/py38/lib/python3.8/site-packages/mypackage.egg-link points back to my development directory, which obviously has the new version number. The script installed at .tox/py38/bin/myscript.py includes the below reference to the old version number, and tox isn't updating it, so when I re-run tox after updating the version I end up getting a VersionConflict error.
==================================== ERRORS ====================================
_____________________ ERROR collecting tests/test_args.py ______________________
tests/test_args.py:6: in <module>
import mypackage
.tox/py38/bin/mypackage.py:4: in <module>
__import__('pkg_resources').require('mypackage==1.0.0b1')
.tox/py38/lib/python3.8/site-packages/pkg_resources/__init__.py:900: in require
needed = self.resolve(parse_requirements(requirements))
.tox/py38/lib/python3.8/site-packages/pkg_resources/__init__.py:791: in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
E pkg_resources.VersionConflict: (mypackage 1.0.0b2 (/Users/matt/devel/mypackage), Requirement.parse('mypackage==1.0.0b1'))
tox is installing the following two scripts in its bin directory which don't seem to get updated, and are triggering the above error:
% cat mypackage.py
#!/Users/matt/devel/mypackage/.tox/py38/bin/python
# EASY-INSTALL-DEV-SCRIPT: 'mypackage==1.0.0b1','mypackage.py'
__requires__ = 'mypackage==1.0.0b1'
__import__('pkg_resources').require('mypackage==1.0.0b1')
__file__ = '/Users/matt/devel/mypackage/mypackage.py'
with open(__file__) as f:
exec(compile(f.read(), __file__, 'exec'))
% cat myscript
#!/Users/matt/devel/mypackage/.tox/py38/bin/python
# EASY-INSTALL-ENTRY-SCRIPT: 'mypackage','console_scripts','myscript'
__requires__ = 'mypackage'
import re
import sys
from pkg_resources import load_entry_point
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
sys.exit(
load_entry_point('mypackage', 'console_scripts', 'myscript')()
)
Is there a way to convince tox to do a proper upgrade of the console script in its environment without manually wiping out all of the .tox/ subdirectories every time I change the version number?
Related
is it possible to detect if current python code is running from package?
if yes - is it possible to get package metadata (name, version, description)?
package is created with this kind of setup.py
import os
from setuptools import setup, find_packages
setup(
name='my-pack-name',
description='my description ' + os.getenv('GIT_COMMIT', '*')[:7],
version=os.getenv('BUILD_VERSION', '0.0.0dev'),
packages=find_packages(),
)
build: python3 setup.py bdist_wheel -d ./artifact
install on target: pip3 install "my-pack-name-x.x.x.whl" --upgrade
now from my_pack_name/app.py that was inside my-pack-name-x.x.x.whl i want to detect that i'm running from installed package
and if so then get package metadata defined during setup.py execution
For Python >=3.8
https://docs.python.org/es/3.10/library/importlib.metadata.html
You can get the metadata for a package by:
from importlib.metadata import metadata
md = metadata("your package name")
author = md["Author"]
# etc ...
For Python <3.8
This is just an idea (not tested).
What about having the package metadata in a different module, and try relative import it in the app.py module?
# metadata.py
name='my-pack-name',
description='my description ' + os.getenv('GIT_COMMIT', '*')[:7],
version=os.getenv('BUILD_VERSION', '0.0.0dev')
In setup.py you could reuse that:
# setup.py
import os
from setuptools import setup, find_packages
import .metadata as md
setup(
name=md.name,
description=md.description + os.getenv('GIT_COMMIT', '*')[:7],
version=md.version,
packages=find_packages(),
)
And the in app.py
def get_metadata():
try:
import .metadata as md
except ImportError:
return None
else:
# return metadata here
That way if get_metadata returns None, you were not able to import the module, so you're not executing the app.py in your package, otherwise, you are in your package and as a bonus you got your metadata.
For the test if python code is running from an installed package (vs. from a development location), I use
if 'site-packages' in __file__:
...
I don't know if that's a good approach; seems to work thus far.
I want to run a bash shell script before running setup is called in a setup.py:
from numpy.distutils.core import setup, Extension
from subprocess import call
err = call('sh dependencies.sh',shell=True)
if err:
raise Exception('The dependencies failed to compile.')
extensions = [...]
setup(name = 'Package',
packages=['Package'],
ext_modules=extensions)
When I run python -m pip install . -v, everything works. HOWEVER, the script dependencies.sh is run two times, compiling the dependencies two times. How do I do this properly? Thanks!
I have added tox to my project and my tox.ini is very simple:
[tox]
envlist = py37
[testenv]
deps =
-r{toxinidir}/requirements_test.txt
commands =
pytest -v
But when I run tox, I get the following error:
ERROR: invocation failed (exit code 1), logfile: /path/to/my_project/.tox/py37/log/py37-2.log
========================================================================================= log start ==========================================================================================
Processing ./.tox/.tmp/package/1/my_project-0+untagged.30.g6909bfa.dirty.zip
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-req-build-ywna_4ks/setup.py", line 15, in <module>
with open(requirements_path) as requirements_file:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-req-build-ywna_4ks/requirements.txt'
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-req-build-ywna_4ks/
You are using pip version 10.0.1, however version 19.2.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
========================================================================================== log end ===========================================================================================
__________________________________________________________________________________________ summary ___________________________________________________________________________________________
ERROR: py37: InvocationError for command /path/to/my_project/.tox/py37/bin/python -m pip install --exists-action w .tox/.tmp/package/1/my_project-0+untagged.30.g6909bfa.dirty.zip (exited with code 1)
Here is my setup.py:
-*- coding: utf-8 -*-
import os
import sys
from setuptools import setup, find_packages
import versioneer
here = os.path.abspath(os.path.dirname(__file__))
sys.path.insert(0, here)
requirements_path = os.path.join(os.path.dirname(os.path.realpath(__file__)), 'requirements.txt')
with open(requirements_path) as requirements_file:
requires = requirements_file.readlines()
setup(
name='my_project',
version=versioneer.get_version(),
cmdclass=versioneer.get_cmdclass(),
maintainer='Hamed',
license='BSD',
py_modules=['my_project'],
packages=find_packages(),
package_data={'': ['*.csv', '*.yml', '*.html']},
include_package_data=True,
install_requires=requires,
long_description=open('README.md').read(),
zip_safe=False
]
},
)
python setup.py install works fine.
It seems that tox is looking for requirements in the tmp dir, but can't find it there. Is there something wrong with my configurations?
I am using tox==3.12.1, python==3.7.3, setuptools==41.0.1, and conda==4.6.9
I've tested this on Arch and SLES 12 and got the same result with both.
Based on the point from #phd, I found out that requirements.txt was not present in the source distribution. Adding requirements.txt to the MANIFEST.in solved the issue!
Complementing Hamed2005's answer:
I have my requirements split into different files (base_requirements.txt, dev_requirements.txt, etc), all of them in a requirements directory. In this case, you need to add this directory in the MANIFEST.in as
recursive-include requirements *
I'm creating a library, but the library is also an application. Therefore, I'm trying to make the code as painless as possible for people with no real background in programming. I decided to go along with suggestions in this post. However, I'm running into all sorts of issues. I want users to do
$ make config
$ make install
I need to do this because it's actually a C++/Python code, so I'm mixing them with Swig and Cmake. My objective is then to make those config and install targets so installing is a breeze. The first question I have is the following: If the contents of my setup.py file is as simple as
from setuptools import setup, find_packages
def read(*names, **kwargs):
return io.open(
join(dirname(__file__), *names),
encoding=kwargs.get("encoding", "utf8")
).read()
setup(
name="myproject",
version="0.1.0",
)
and I type
$ python setup.py build
How is it possible that the code is actually installed within pip?
$ pip freeze
-e git+https://svn.3me.tudelft.nl/git/myproject.git#92f08bfcbaf1f78a6acdf5f03b5c7a36e87800eb#egg=myproject
I improved the setup.py so that it also has dependencies:
setup(
name="myproject",
version="0.1.0",
install_requires=[
# eg: "aspectlib==1.1.1", "six>=1.7",
"numpy", "scipy", "Sphinx", "pytest", "matplotlib", "dill"
],
)
and I put a requirements.txt as discussed in the link above:
--index-url https://pypi.python.org/simple/
-e .
and I run pip install -r requirements.txt it actually installs all dependencies needed, but if I rerun it, it fails with
Exception:
Traceback (most recent call last):
File "/Users/aaragon/.virtualenvs/test7/lib/python3.5/site-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/Users/aaragon/.virtualenvs/test7/lib/python3.5/site-packages/pip/commands/install.py", line 342, in run
prefix=options.prefix_path,
File "/Users/aaragon/.virtualenvs/test7/lib/python3.5/site-packages/pip/req/req_set.py", line 778, in install
requirement.uninstall(auto_confirm=True)
File "/Users/aaragon/.virtualenvs/test7/lib/python3.5/site-packages/pip/req/req_install.py", line 703, in uninstall
'(at %s)' % (link_pointer, self.name, dist.location)
AssertionError: Egg-link /Users/aaragon/Local/myproject does not match installed location of myproject (at /Users/aaragon/Local/myproject/src)
why am I getting this error?
I'm trying to install PyOpenCV on Ubuntu 14.04 using setuptools. When I tried
python setup.py config
I got the error
ImportError: cannot import name Library
I found in an answer to a previous question that the fix was to change
from setuptools import Library
to
from setuptools.extension import Library
in setup.py. Now when I run setup.py, the error occurs in the config.py script it generates:
$ python setup.py config
Configuring PyOpenCV via CMake...
<snip>
-- Configuring done
-- Generating done
-- Build files have been written to: /home/saul/Downloads/pyopencv/build
Traceback (most recent call last):
File "setup.py", line 137, in <module>
import config as C
File "/home/saul/Downloads/pyopencv/config.py", line 1, in <module>
from setuptools import Extension, Library
ImportError: cannot import name Library
The first line of config.py contains the same import error. I can correct config.py of course, but I don't know how to resume the build process.
The text of setup.py from the point of failure is:
import config as C
setup(
name = "pyopencv",
version = C.PYOPENCV_VERSION,
description = DOCLINES[0],
author = 'Minh-Tri Pham',
author_email = 'pmtri80#gmail.com',
url = 'http://code.google.com/p/pyopencv/',
license = 'New BSD License',
platforms = 'OS Independent, Windows, Linux, MacOS',
classifiers = filter(None, CLASSIFIERS.split('\n')),
long_description = "\n".join(DOCLINES[2:]),
ext_modules=C.extension_list,
install_requires = ['numpy>=1.2.0'],
package_data = {'pyopencv': ['*.dll']},
include_package_data = True,
# zip_safe = (os.name!='nt'), # thanks to ffmpeg dependency
package_dir={'':'package'},
packages = find_packages('package'),
)
Can you tell me how to fix this, or suggest an alternative way to install pyOpenCV?
There is much simple way to do it, open terminal and type :
sudo apt-get install python-opencv