We Keep Coding

Python manually install package to virtual environment with setup.py

I am developing using Python 3.7.
I'm trying to write a small Python package and install it into a virtual environment for testing. The library is structured like this:
my_package/
src/
my_package/
__init__.py
utilities.py
some_data_files/
setup.py
README.md
...
When I create virtual environment and try to install:
(env) C:\Users\path\to\my_package python setup.py install
The output is:
running install
running bdist_egg
running egg_info
writing my_package.egg-info\PKG-INFO
writing dependency_links to my_package.egg-info\dependency_links.txt
writing top-level names to my_package.egg-info\top_level.txt
reading manifest file 'my_package.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'my_package.egg-info\SOURCES.txt'
installing library code to build\bdist.win-amd64\egg
running install_lib
warning: install_lib: 'build\lib' does not exist -- no Python modules to install
creating build\bdist.win-amd64\egg
creating build\bdist.win-amd64\egg\EGG-INFO
copying my_package.egg-info\PKG-INFO -> build\bdist.win-amd64\egg\EGG-INFO
copying my_package.egg-info\SOURCES.txt -> build\bdist.win-amd64\egg\EGG-INFO
copying my_package.egg-info\dependency_links.txt -> build\bdist.win-amd64\egg\EGG-INFO
copying my_package.egg-info\top_level.txt -> build\bdist.win-amd64\egg\EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating 'dist\my_package-0.0.1-py3.7.egg' and adding 'build\bdist.win-amd64\egg' to it
removing 'build\bdist.win-amd64\egg' (and everything under it)
Processing my_package-0.0.1-py3.7.egg
Copying my_package-0.0.1-py3.7.egg to c:\users\jon\development\my_package_env\env\lib\site-p
ackages
Adding my_package 0.0.1 to easy-install.pth file
Installed c:\users\jon\development\my_package_env\env\lib\site-packages\my_package-0.0.1-py3
.7.egg
Processing dependencies for my_package==0.0.1
Finished processing dependencies for my_package==0.0.1
There is an egg file in site-packages but no directory containing any of the files from the my_package directory. This is my first time writing a Python package for distribution, and I'm using the Python documentation as a guide, but I'm not ready to upload anywhere and would like to test locally. Can someone let me know what steps I must take to be able to test my package locally in a virtual environment?

Create a source distribution and a built distribution of your project, with these commands:
(env) C:\Users\jon\development\my_package_env>python3 setup.py sdist
(env) C:\Users\jon\development\my_package_env>python3 setup.py bdist_wheel
Note the two newly created archives in the dist/ directory
Try to install these distributions in fresh virtual environments:
C:\Users\jon\development\my_package_env>python3 -m venv sdist-env
C:\Users\jon\development\my_package_env>sdist-env\Scripts\activate.bat
(sdist-env) C:\Users\jon\development\my_package_env>pip install dist\my_package-0.0.1.tar.gz
(sdist-env) C:\Users\jon\development\my_package_env>deactivate
C:\Users\jon\development\my_package_env>python3 -m venv wheel-env
C:\Users\jon\development\my_package_env>wheel-env\Scripts\activate.bat
(wheel-env) C:\Users\jon\development\my_package_env>pip install dist\my_package-0.0.1-py3-none-any.whl
Once you're confident with this, you might want to look at tox to automate the testing of your project in multiple virtual environments.

How can I best install a Python package (python-ldap3 2.5.1) without easy_install (followup to an older thread)?

I am trying to figure how to install a Python package (ldap3) using a .egg file (fyi, the version of python-ldap3 we need does not have a .whl file) on a system that does not have easy_install available.
I found from an earlier thread (How can I install a .egg Python package on Windows (attempt using easy_install not working)) that I might be able to:
Untar the python-ldap3 tar.gz
Change dir to the untarred dir
run "python setup.py install"
So I tried the above, and it does seem to deploy/install the python-ldap3, but the installation hangs because it looks like it is trying to download something:
# python setup.py install
running install
running bdist_egg
running egg_info
writing requirements to ldap3.egg-info/requires.txt
writing ldap3.egg-info/PKG-INFO
writing top-level names to ldap3.egg-info/top_level.txt
writing dependency_links to ldap3.egg-info/dependency_links.txt
reading manifest file 'ldap3.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'ldap3.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib
creating build/lib/ldap3
copying ./ldap3/__init__.py -> build/lib/ldap3
copying ./ldap3/version.py -> build/lib/ldap3
creating build/lib/ldap3/abstract
copying ./ldap3/abstract/cursor.py -> build/lib/ldap3/abstract
.
.
.
Installed /usr/lib/python2.6/site-packages/ldap3-2.5.1-py2.6.egg
Processing dependencies for ldap3==2.5.1
Searching for pyasn1>=0.1.8
Reading https://pypi.python.org/simple/pyasn1/
The "python setup.py install" hangs at that last line above ("Reading...."), but if I kill the process then check the python-ldap3 version, it looks like it installed all right:
# pip freeze | grep ldap3
ldap3==2.5.1
Also, FYI, our Python version is 2.6, and the pyasn1 is:
#pip freeze | grep pyasn1
pyasn1==0.4.3
I am not that familiar with Python and Python packaging, so I was wondering:
Is it all right to do the ldap3 2.5.1 installation per above (using
the 'python setup.py install', and then just killing the script)?
If not, what would be a safer/better way to get the ldap3 2.5.1 installed?
Thanks in advance!
Jim
EDIT: I forgot to mention that I think that the reason the script hung at the point it did was that we have to do this installation on an "isolated network", with no access to the external Internet, so we actually prefer/need to do the ldap3 installation without any external access.
EDIT 2: I guess that one other thing that is puzzling is why is the "python setup.py install" trying to get "pyasn1>=0.1.8" when "0.4.3" is already installed, since, from the release history, https://pypi.org/project/pyasn1/#history, "0.1.8" is OLDER than "0.4.3"?
EDIT 3: TO be clear, FYI, the package I am working with it the "ldap3" package, from https://github.com/cannatag/ldap3.

Python setuptools' "setup.py install" does not automatically call build?

I am trying to package a Python 3.6 project which additionally requires compiling CPP code. My test setup.py file currently looks like this:
print("Hello world from setup.py")
from distutils.command.build import build
from setuptools import setup
class TestBuild(build):
def run(self):
print("Hello world from TestBuild")
setup(
name="Test",
cmdclass={
'build': TestBuild,
},
)
The idea here is to place the CPP-compiling code inside TestBuild. When I run python setup.py build it seems to work well:
> python setup.py build
Hello world from setup.py
running build
Hello world from TestBuild
However, running python setup.py install does not seem to run the build step at all:
> python setup.py install
Hello world from setup.py
running install
running bdist_egg
running egg_info
writing Test.egg-info/PKG-INFO
writing top-level names to Test.egg-info/top_level.txt
writing dependency_links to Test.egg-info/dependency_links.txt
reading manifest file 'Test.egg-info/SOURCES.txt'
writing manifest file 'Test.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
warning: install_lib: 'build/lib' does not exist -- no Python modules to install
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying Test.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying Test.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying Test.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying Test.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating 'dist/Test-0.0.0-py3.5.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing Test-0.0.0-py3.5.egg
My questions:
Does the install command automatically invokes the build command?
If it does, what did I do wrong? If it does not, how can I ensure the build command runs?
Where can I find detailed description of the install process? I tried to dig inside the setuptools documentation and did not find anything like this.

Does the install command automatically invokes the build command?
Yes but it doesn't run python setup.py build — it calls the
corresponding code right inside one process hence you never see the
second print. If you really need to print it you have to override setuptools commands with custom classes; create such classes for build, bdist_egg and install commands.
Where can I find detailed description of the install process? I tried to dig inside the setuptools documentation and did not find anything like this.
I'm afraid distutils and setuptools source code is the only authoritative reference.
bdist_egg doesn't run build directly; it runs egg_info, build_clib and install_lib. install_lib calls distutils.build which runs build_py and build_ext. So it seems you have to override build_py, not build.

Why won't my python module install?

I am trying to build a pulp distributor plugin which will execute a bash script containing arbitrary code so that I may trigger actions after an RPM repo is published.
These plugins are generally created using distutils. However, when I attempt to install my module, I receive the error:
warning: install_lib: 'build/lib' does not exist -- no Python modules to install
Typically, this means that the working directory is incorrect or __init.py__ is missing. In my case however, I am attempting to install from the correct working directory and I did create __init.py__ files (see the repo here).
I suspect that I am running into a pathing issue having something to do with my code being in a subdirectory so far removed from setup.py. What am I doing wrong? Why won't my module install?

When you face errors like this, one of the first things to check is what packages are actually being added to your distribution when you build it. In your case the package list is empty, but should contain at least the pulp_hook package:
$ python -c "from setuptools import find_packages; print(find_packages())"
[]
So why does setuptools not recognize pulp_hook as a regular package? Look at its structure: you have added file named __init.py__, but its name should be __init__.py. Once you rename the files, the pulp_hook and its subdirectories become regular packages:
$ python -c "from setuptools import find_packages; print(find_packages())"
['pulp_hook', 'pulp_hook.plugins', 'pulp_hook.plugins.distributors']
Now build/lib will be created because now distutils finds at least one package to install:
$ python setup.py install_lib
running install_lib
running build_py
creating build
creating build/lib
creating build/lib/pulp_hook
copying pulp_hook/__init__.py -> build/lib/pulp_hook
creating build/lib/pulp_hook/plugins
copying pulp_hook/plugins/__init__.py -> build/lib/pulp_hook/plugins
creating build/lib/pulp_hook/plugins/distributors
copying pulp_hook/plugins/distributors/distributionhook.py -> build/lib/pulp_hook/plugins/distributors
copying pulp_hook/plugins/distributors/__init__.py -> build/lib/pulp_hook/plugins/distributors

running install_lib warning: install_lib: 'build\lib' does not exist -- no Python modules to install

c:\Python33>python.exe D:\program_python\module_setup\setups.py install
running install
running build
running build_py
running install_lib
**warning: install_lib: 'build\lib' does not exist -- no Python modules to in**
running install_egg_info
Removing c:\Python33\Lib\site-packages\find_elements_of_list-3.3.3-py3.3.eg
o
Writing c:\Python33\Lib\site-packages\find_elements_of_list-3.3.3-py3.3.egg
I am getting this error ( is in bold ) can you suggest ( when I am using window 8 -32 bit o/s with python33 as well as when using Red Hat Linux-6 with python 2.6 )
c:\Python33>update modules list
'update' is not recognized as an internal or external command,
operable program or batch file.

You need to change into the directory of the application and then run the file.
c:\Python33>cd D:\program_python\module_setup
c:\Python33>cd D:
D:\program_python\module_setup> C:\Python33\python.exe setup.py install

I had to do 2 things:
Add __init__.py (can be empty) in the setup directory
./main/setup.py
./main/setup/__init__.py
Install cmake (if it's not installed yet)
sudo apt-get install cmake