I have run
python setup.py sdist --formats=gztar,zip bdist_wheel
and then
python setup.py install
The result is that the egg files are created in the site-packages directory but not the <package-name>/<package-source files>:
$ls /usr/local/lib/python3.7/site-packages/infix*
/usr/local/lib/python3.7/site-packages/infixpy-0.0.3-py3.7.egg
/usr/local/lib/python3.7/site-packages/infixpy.egg-link
/usr/local/lib/python3.7/site-packages/infixpy-0.0.4-py3.7.egg
Notice that the directory infix was not created - and thus none of the source code was copied. What am I missing / not understanding in this local installation process?
Update When I had run
pip3 install infixpy
there was an additional directory infix and the source code was included in that directory. Running the local or devel modes of setup.py install was not causing that code to be updated and - crucially - the stacktraces from running any python code (even in a completely new ipython repl) was showing only the older / pip3 installed code. In particular the file __init__.py So my observation has been that the source file :
/usr/local/lib/python3.7/site-packages/infixpy/__init__.py
is an accurate reflection of what the python executable were using. #phd is mentioning that the source code is already included in the egg. So then I do not understand the relationship between the source code in the egg and the source code in that subdirectory - which in the lastest run of mine is completely missing.
The following commands all yield slightly different results:
pip install .: installed as uncompressed package directories and a XXX.dist-info directory
pip install infixpy: same as previous, but installed from an (remote) index (per default PyPI), not from the local directory
python setup.py install: installed as a zipped file XXX.egg
pip install --editable . or python setup.py develop: not installed, but linked as a XXX.egg-link file
So depending on the commands entered, the content of site-packages is different.
Now this is what you say you have:
$ls /usr/local/lib/python3.7/site-packages/infix*
/usr/local/lib/python3.7/site-packages/infixpy-0.0.3-py3.7.egg
/usr/local/lib/python3.7/site-packages/infixpy.egg-link
/usr/local/lib/python3.7/site-packages/infixpy-0.0.4-py3.7.egg```
This is a bit surprising, since theoretically there are 3 versions of your project that are importable (0.0.3, 0.0.4, and develop/editable). I am not sure which one is used by the Python interpreter in this case. You might want to run pip uninstall infixpy a couple of times to start fresh and alleviate these uncertainties. You can then experiment with the commands mentioned above and see how they impact the content of site-packages along with inspecting the result of pip show infixpy.
Related
I have successfully built a Python package that uses CMake combined with pybind11 to create a shared object (.so - assuming only Linux usage at the moment) file. The implementation works but I am unable to remove this shared object file using pip uninstall .
My setup command in setup.py file looks like this taken from the pybind/cmake_example repository:
setup(
name='package',
version='0.0.1',
author='-',
author_email='-',
description='A test project using pybind11 and CMake',
long_description='',
ext_modules=[CMakeExtension('packagebindings')],
cmdclass=dict(build_ext=CMakeBuild),
zip_safe=False,
packages=setuptools.find_packages()
)
My CMakeLists.txt file has an install instruction that looks like this:
install(TARGETS packagebindings COMPONENT python LIBRARY DESTINATION ${Python_SITELIB})
To summarise, here are the files that are created when running pip install .:
path/to/site-packages/package/* - removed by pip uninstall package
path/to/site-packages/package-0.0.1.dist-info/* - removed by pip uninstall package
path/to/site-packages/packagebindings.cpython-37m-x86_64-linux-gnu.so - still present after pip uninstall package
I would like to know how make it so that running pip uninstall . removes the .so file.
If a further MRE is required, I can link to a repository.
Your CMake install target seems to place the .so directly into the python installation directory (DESTINATION ${Python_SITE_LIB}). I'm guessing this stops the .so from being registered by Python proper, so it is not removed when uninstalling. I would suggest to make CMake place the .so in a distribution directory, and then add the following option to setup():
data_files = [("installation_bin", ["distribution_bin/library.so"])]
This will let the .so be tracked by the Python package manager. The first string is a directory relative to the installation prefix. The second string is the .so file in your distribution, relative to the setup.py script.
On Ubuntu 16.04.4, I suspect some recent update of some Python system package of having broken my Python 2.7 configuration. Whatever package I try to install or reinstall with a basic sudo python setup.py install it always fails because of gitignore:
running install
running bdist_egg
running egg_info
[...]
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
error: Error: setup script specifies an absolute path:
/home/me/some_repo/.gitignore
setup() arguments must *always* be /-separated paths relative to the
setup.py directory, *never* absolute paths.
Lately I found a temporary workaround by manually cleaning the /usr/local/lib/python2.7/dist-packages/some_package directory manually before installing some_package. However tonight I'm facing the same issue on another repository and it keeps failing whatever I clean up. I also tried cleaning all compiled folders .egg-info/ build/ dist/ without success.
Notes: The setup script does not actually specify an absolute path to gitignore. An example of failing repo is https://github.com/philchristensen/python-artnet/blob/master/setup.py This repo has a setuptools_git entry which might lead to a clue, but other packages without this git entry point also fail because of the gitignore while a couple of months ago I never faced such issue with the same repos. Deleting the gitignore causes the setup to fail because of another non-py local file.
Any clue?
It looks like some other package that I had previously installed broke my system-wide Python.
Here's how I fixed in order to install package xyz:
Browsed /usr/local/lib/python2.7/dist-packages in search for occurences of "gitignore"
Deleted folders of all matching occurences (including setuptools_git itself that matches "gitignore")
pip install setuptools_git
In package xyz, rm -rf dist/ build/ *.egg-info/
Reinstall package xyz, which now succeeds to install
Use virtual envs as a lesson
I'm running python 3.6 via anaconda 3, using Visual Studio Code.
I followed instructions like these (Interactive Brokers API install) and downloaded the package to a local directory of mine say: c:\dev\pyib, so now the code is in c:\dev\pyib\IbPy-master
I open that directory in command line and run
python setup.py install
All runs ok.
But then my program, which is in c:\dev\pyib says Module not found. (In my case ibapi). The linter is also showing red.
There is no other python installed on this pc.
Where did the package install to? and how do I check that? What will I find where the package installed itself to that shows me its there?
Or do I have to use a trial-and-error with the linter and sys.path.append()? (I tried that with the directory where the files are downloaded to - to no avail)
I'm trying to set up the PYTHONPATH using the "env" in launch.json from Visual Studio Code, as shown in this unaccepted answer.
Current sys.path:
'c:\\dev\\pyIb',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\python36.zip',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\DLLs',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\lib',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-
packages',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\Babel-2.5.0-py3.6.egg',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\win32',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\win32\\lib',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\Pythonwin'
I deleted the ib directory and re-ran the install. The last line says: Writing C:\Users\user\AppData\Local\Continuum\anaconda3\Lib\site-packages\IbPy2-0.8.0-py3.6.egg-info So is the location of the egg-info the location of my undetected module? The actual folder in the site-packages is called ib.
Or could my problems be because of a difference in Lib vs. lib with the lowercase in the sys.path and the uppercase in the actual directory?
But the real question here is still: HOW DO I KNOW WHERE the package was installed what should I search for?
This answer is specific for anaconda3 Python and packages installed using python setup.py install (which is actually using distutils)
Take a look at anaconda3\Lib\site-packages you should see a directory for the package you installed.
The way to know for sure where your package is, is by doing a pip list then trying to pip uninstall and re-install again using the python setup.py install: Here are the detailed instructions:
When uninstalling, pip will tell you it cannot because it was done via distutils.
You'll get a message like this:
DEPRECATION: Uninstalling a distutils installed project (ibpy2) has been deprecated and will be removed in a future version.
This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
You'll be prompted to continue anyway. If you choose No, then you can find the directory in
C:\Users\<yourusername>\AppData\Local\Continuum\anaconda3\Lib\site-packages
Thanks to Emanuel Mtali for pointing me in the right direction
Some more information:
The problem I had was due to a stupid mistake of mine. I was running setup of a different (but related) package not used anymore. IbPy2 instead of TwsAPI. I was supposed to run the setup.py of the package installed via the latest version of the MSI from IB, and NOT the IbPy2 package. :-(
I am a newbie to installing python extensions working on Windows 7, running Python 2.6 - I need to install the Levenshtein library from
http://code.google.com/p/pylevenshtein/downloads/detail?name=python-Levenshtein-0.10.1.tar.bz2&can=2&q=
When I unzip the downloaded file, it gives me the following list of files:
COPYING
gendoc.sh
Levenshtein.c
Levenshtein.h
MANIFEST
NEWS
PKG-INFO
README
setup.cfg
setup.py
StringMatcher.py
How do I install the Levenshtein library so I could import and use it into my python code?
Assuming you have Python already installed on on you PATH, you can do this:
python setup.py install
However, it seems to have a compiled extension so you will probably also need a complete Windows development environment to install that (it is a source distribution). So if you don't it may not work. Your best bet would be to find that as an MSI package, if you can.
Here is quite a large section of the documentation easily found by doing some research.
http://docs.python.org/install/index.html
It appears that you will want to run:
python setup.py install --prefix="\Temp\Python"
to install modules to the \Temp\Python directory on the current drive.
Some more info:
If you don’t choose an installation directory—i.e., if you just run
setup.py install—then the install command installs to the standard
location for third-party Python modules.
The default installation directory on Windows was C:\Program Files\Python under Python 1.6a1, 1.5.2, and earlier.
Goals:
Make use of modern Python packaging toolsets to deploy/install proprietary packages into some virtualenv.
The installed packages should include compiled *.pyc(or *.pyo) only without source files.
There are a couple of packages, and a vendor name (here we choose dgmx for our studio) is used as the package names. Therefore, the installed packages would be something like dgmx/alucard, dgmx/banshee, dgmx/carmilla, ...
The file hierarchy of installed packages should be like ones by python setup.py install --single-version-externally-managed or pip install. Refer to How come I can't get the exactly result to *pip install* by manually *python setup.py install*?
Question in short:
I like to deploy proprietary namespaced packages into a virtualenv by only compiled *.pyc(or *.pyo) files, in which the file/directory hierarchy just reflects the namespace with polluting sys.path by lots of ooxx.egg paths.
Something I have tried:
python setup.py bdist_egg --exclude-source-files then easy_install ooxx.egg.
pollute "sys.path" for each namespace package.
python setup.py install --single-version-externally-managed.
not *.pyc only.
the "install_requires" got ignored!
need to manually put a ooxx.egg-info/installed-files.txt to make uninstall work correctly.
pip install . in the location of "setup.py".
not *.pyc only.
pysetup install . in the location of "setup.py".
not *.pyc only.
Update:
My current idea is to follow method 2.
python setup.py egg_info --egg-base . # get requires.txt
python setup.py install --single-version-externally-managed --record installed-files.txt # get installed-files.txt
manually install other dependencies through "requires.txt"
manually delete installed source files (*.py) through "installed-files.txt"
remove source files (*.py) from "installed-files.txt" and put it into deployed "ooxx.egg-info/installed-files.txt"
References:
Migrating to pip+virtualenv from setuptools
installing only .pyc (python compiled) with setuptools
Can I deploy Python .pyc files only to Google App Engine?
How come I can't get the exactly result to *pip install* by manually *python setup.py install*?
Some trick may help:
Compile your source into .pyc, zip them up in a single .zip file.
Write a new module with a simple module all it does is to add the .zip to the sys.path.
So when you import this module, the .zip is in the path. All you have to do is in a custom step in setup.py, copy the zip file to the proper place.