We Keep Coding

Installing shared library with python package not separately

I have successfully built a Python package that uses CMake combined with pybind11 to create a shared object (.so - assuming only Linux usage at the moment) file. The implementation works but I am unable to remove this shared object file using pip uninstall .
My setup command in setup.py file looks like this taken from the pybind/cmake_example repository:
setup(
name='package',
version='0.0.1',
author='-',
author_email='-',
description='A test project using pybind11 and CMake',
long_description='',
ext_modules=[CMakeExtension('packagebindings')],
cmdclass=dict(build_ext=CMakeBuild),
zip_safe=False,
packages=setuptools.find_packages()
)
My CMakeLists.txt file has an install instruction that looks like this:
install(TARGETS packagebindings COMPONENT python LIBRARY DESTINATION ${Python_SITELIB})
To summarise, here are the files that are created when running pip install .:
path/to/site-packages/package/* - removed by pip uninstall package
path/to/site-packages/package-0.0.1.dist-info/* - removed by pip uninstall package
path/to/site-packages/packagebindings.cpython-37m-x86_64-linux-gnu.so - still present after pip uninstall package
I would like to know how make it so that running pip uninstall . removes the .so file.
If a further MRE is required, I can link to a repository.

Your CMake install target seems to place the .so directly into the python installation directory (DESTINATION ${Python_SITE_LIB}). I'm guessing this stops the .so from being registered by Python proper, so it is not removed when uninstalling. I would suggest to make CMake place the .so in a distribution directory, and then add the following option to setup():
data_files = [("installation_bin", ["distribution_bin/library.so"])]
This will let the .so be tracked by the Python package manager. The first string is a directory relative to the installation prefix. The second string is the .so file in your distribution, relative to the setup.py script.

pip does not install my package dependencies

I have developed a python package on github that I released on PyPi. It installs with pip install PACKAGENAME, but does not do anything with the dependencies that are stated in the "install_requires" of the setup.py file.
Weirdly enough, the zip file of the associated release does install all dependencies.. I tried with different virtual environments and on different computers but it never installs the dependencies.. Any help appreciated.

pip install pythutils downloads a wheel if it's available — and it's available for your package.
When generating a wheel setuptools runs python setup.py locally but doesn't include setup.py into the wheel. Download your wheel file and unzip it (it's just a zip archive) — there is your main package directory pythutils and a directory with metadata pythutils-1.1.1.dist-info. In the metadata directory there is a file METADATA that usually lists static dependencies but your file doesn't list any. Because when you were generating wheels all your dependencies has already been installed so all your dynamic code paths were skipped.
The archive that you downloaded from Github release install dependencies because it's not a wheel so pip runs python setup.py install and your dynamic dependencies work.
What you can do? My advice is to avoid dynamic dependencies. Declare static dependencies and allow pip to decide what versions to install:
install_requires=[
'numpy==1.16.5; python_version>="2" and python_version<"3"',
'numpy; python_version>="3"',
],
Another approach would be to create version-specific wheel files — one for Python 2 and another for Python 3 — with fixed dependencies.
Yet another approach is to not publish wheels at all and only publish sdist (source distribution). Then pip is forced to run python setup.py install on the target machine. That not the best approach and it certainly will be problematic for packages with C extensions (user must have a compiler and developer tools to install from sources).

Your setup.py does a series of checks like
try:
import numpy
except ImportError:
if sys.version_info[0] == 2:
install_requires.append('numpy==1.16.5')
if sys.version_info[0] == 3:
install_requires.append("numpy")
Presumably the system where you ran it had all the required modules already installed, and so ended up with an empty list in install_requires. But this is the wrong way to do it anyway; you should simply make a static list (or two static lists, one each for Python 2 and Python 3 if you really want to support both in the same package).

How to upgrade/uninstall distutils packages (PyYAML) in windows OS

I am working in WIN10 , with python 2.7.15
I am try to install package, during the installation process I received the following error .
Cannot uninstall 'PyYAML'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
I try to uninstall with pip (18.1) command and I received the same error.
pip uninstall PyYAML
How I can uninstall/upgrade distutils packge in win10 OS.

Base distutils functionality doesn't leave any information about which files belong to a package -- thus it cannot be reliably uninstalled. That's what the message is telling you. Moreover, it doesn't have dependency metadata, so it can't be "upgraded" reliably, either. All those features are additions by setuptools (and some by wheel and pip itself).
This can happen if you installed the package directly from source with setup.py install if setup.py is distutils- rather than setuptools-based. Or if you installed it manually from some types of packages by copying/extracting files.
Unless the way you installed it provides an own uninstaller, you'll have to manually figure out which files belong to the package and delete them from Python directories.
Usually, these are:
site-packages\<package_name>* directories and/or
site-packages\<package_name>*.py for standalone modules
optionally, a site-packages\<package_name>.pth file
Generally, look for anything that bears the package's name on it.
If you can build the same package from source, you can use the build process to get a hint: build a binaly package that you can look into (e.g. setup.py bdist_wheel -- .whl is a ZIP archive) and see what files it has in it.

Where did my python module install to?

I'm running python 3.6 via anaconda 3, using Visual Studio Code.
I followed instructions like these (Interactive Brokers API install) and downloaded the package to a local directory of mine say: c:\dev\pyib, so now the code is in c:\dev\pyib\IbPy-master
I open that directory in command line and run
python setup.py install
All runs ok.
But then my program, which is in c:\dev\pyib says Module not found. (In my case ibapi). The linter is also showing red.
There is no other python installed on this pc.
Where did the package install to? and how do I check that? What will I find where the package installed itself to that shows me its there?
Or do I have to use a trial-and-error with the linter and sys.path.append()? (I tried that with the directory where the files are downloaded to - to no avail)
I'm trying to set up the PYTHONPATH using the "env" in launch.json from Visual Studio Code, as shown in this unaccepted answer.
Current sys.path:
'c:\\dev\\pyIb',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\python36.zip',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\DLLs',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\lib',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-
packages',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\Babel-2.5.0-py3.6.egg',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\win32',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\win32\\lib',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\Pythonwin'
I deleted the ib directory and re-ran the install. The last line says: Writing C:\Users\user\AppData\Local\Continuum\anaconda3\Lib\site-pac‌​kages\IbPy2-0.8.0-py‌​3.6.egg-info So is the location of the egg-info the location of my undetected module? The actual folder in the site-packages is called ib.
Or could my problems be because of a difference in Lib vs. lib with the lowercase in the sys.path and the uppercase in the actual directory?
But the real question here is still: HOW DO I KNOW WHERE the package was installed what should I search for?

This answer is specific for anaconda3 Python and packages installed using python setup.py install (which is actually using distutils)
Take a look at anaconda3\Lib\site-packages you should see a directory for the package you installed.
The way to know for sure where your package is, is by doing a pip list then trying to pip uninstall and re-install again using the python setup.py install: Here are the detailed instructions:
When uninstalling, pip will tell you it cannot because it was done via distutils.
You'll get a message like this:
DEPRECATION: Uninstalling a distutils installed project (ibpy2) has been deprecated and will be removed in a future version.
This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
You'll be prompted to continue anyway. If you choose No, then you can find the directory in
C:\Users\<yourusername>\AppData\Local\Continuum\anaconda3\Lib\site-packages
Thanks to Emanuel Mtali for pointing me in the right direction
Some more information:
The problem I had was due to a stupid mistake of mine. I was running setup of a different (but related) package not used anymore. IbPy2 instead of TwsAPI. I was supposed to run the setup.py of the package installed via the latest version of the MSI from IB, and NOT the IbPy2 package. :-(

Python-Asurv Installation

When I try to install python-asurv using setup.py, (typing "path"\python "path"\setup.py install in the command prompt), I get the following error:
building extension "twokm" sources target build\src.win32-2.7\twokmmodule.c does not exist: Assuming twokmmodule.c was generated with "build_src --inplace" command. error: '.\\twokmmodule.c' missing`
I am on windows 7 64bit with 32bit python2.7
In the zip file that I downloaded, there is setup.py, asurv.py, asurv.pyc, two licences, a readme, and twokm.pyf and twokm.f, which I think are in fortran format (don't know anything about fortran). The README just says type python setup.py install.
I think that for the twokm.pyf and twokm.f files I need to use f2py to convert them to .py files, am I right?
How can I install python-asurv?

Make sure you have installed numpy at it is dependency of python-asurv
Do pip install numpy and then Do python setup.py install in the directory you have downloaded.