We Keep Coding

Python 3 `venv` and the `wheel` package

Starting with python3.8, I noticed that the wheel package seems to be required to pip install packages without errors (at least they look like errors, but behave like warnings).
Note: in all of my examples, I am cleaning up in between by deactivating my environment, removing the environment, and clearing my pip cache:
deactivate
rm test_env/ -rf
rm ~/.cache/pip/ -rf
Example 1: The problem
python3.8 -m venv test_env
source test_env/bin/activate
pip install markuppy
I get the following output:
Collecting markuppy
Downloading MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
Building wheel for markuppy (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/k/test_env/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-tfm9bgxv/markuppy/setup.py'"'"'; __file__='"'"'/tmp/pip-install-tfm9bgxv/markuppy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-cpx9gxcn
cwd: /tmp/pip-install-tfm9bgxv/markuppy/
Complete output (6 lines):
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'bdist_wheel'
----------------------------------------
ERROR: Failed building wheel for markuppy
Running setup.py clean for markuppy
Failed to build markuppy
Installing collected packages: markuppy
Running setup.py install for markuppy ... done
Successfully installed markuppy-1.14
It errored because there is no wheel package installed. This has been addressed in other posts on SO.
Example 2: Fixing the problem by installing wheel into the venv
If I install the wheel package directly after creating a new venv, this does not happen:
python3.8 -m venv test_env
source test_env/bin/activate
pip install wheel
pip install markuppy
Output:
Collecting markuppy
Using cached MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
Building wheel for markuppy (setup.py) ... done
Created wheel for markuppy: filename=MarkupPy-1.14-py3-none-any.whl size=7413 sha256=52b3e5c3e317ae21724acd871fe3deb85dde9df305b20d16f2c5592c43b11e91
Stored in directory: /home/k/.cache/pip/wheels/95/13/60/31c9d5f4cd012e491aeac154ef8b0ec964916523623eb02f0b
Successfully built markuppy
Installing collected packages: markuppy
Successfully installed markuppy-1.14
This also works given a requirements.txt file.
requirements.txt:
markuppy
python3.8 -m venv test_env
source test_env/bin/activate
pip install wheel
pip install -r requirements.txt
Output:
Collecting markuppy
Using cached MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
Building wheel for markuppy (setup.py) ... done
Created wheel for markuppy: filename=MarkupPy-1.14-py3-none-any.whl size=7413 sha256=52b3e5c3e317ae21724acd871fe3deb85dde9df305b20d16f2c5592c43b11e91
Stored in directory: /home/k/.cache/pip/wheels/95/13/60/31c9d5f4cd012e491aeac154ef8b0ec964916523623eb02f0b
Successfully built markuppy
Installing collected packages: markuppy
Successfully installed markuppy-1.14
Example 3: Adding the wheel package to requirements.txt does not help
It's important to know that if you place the wheel package into the requirements.txt file, and do not install it separately, you get the same problem as in Example 1:
requirements.txt:
wheel
markuppy
python3.8 -m venv test_env
source test_env/bin/activate
pip install -r requirements.txt
Output:
Collecting wheel
Downloading wheel-0.37.0-py2.py3-none-any.whl (35 kB)
Collecting markuppy
Downloading MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
Building wheel for markuppy (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/k/test_env/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-z7cqzaej/markuppy/setup.py'"'"'; __file__='"'"'/tmp/pip-install-z7cqzaej/markuppy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-kw3_8ayt
cwd: /tmp/pip-install-z7cqzaej/markuppy/
Complete output (6 lines):
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'bdist_wheel'
----------------------------------------
ERROR: Failed building wheel for markuppy
Running setup.py clean for markuppy
Failed to build markuppy
Installing collected packages: wheel, markuppy
Running setup.py install for markuppy ... done
Successfully installed markuppy-1.14 wheel-0.37.0
Wheel installed properly, but was not there in time to install the markuppy package.
Example 4: Installing wheel at the system level does not help
If I install the wheel package directly into the system itself, the venv that was created does not have access to it, so I get the same result as in Example 1.
sudo apt install python3-wheel
python3.8 -m venv test_env
source test_env/bin/activate
pip install markuppy
Output:
Collecting markuppy
Downloading MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
Building wheel for markuppy (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/k/test_env/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-5xkssq1l/markuppy/setup.py'"'"'; __file__='"'"'/tmp/pip-install-5xkssq1l/markuppy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-aunof5xf
cwd: /tmp/pip-install-5xkssq1l/markuppy/
Complete output (6 lines):
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'bdist_wheel'
----------------------------------------
ERROR: Failed building wheel for markuppy
Running setup.py clean for markuppy
Failed to build markuppy
Installing collected packages: markuppy
Running setup.py install for markuppy ... done
Successfully installed markuppy-1.14
Example 5: Installing wheel at the system level and granting venv access to system site packages works
If I install wheel at the system level, then grant my venv access to system site packages (note the flag on the venv command) when I create it, then this issue does not happen.
sudo apt install python3-wheel
python3.8 -m venv test_env --system-site-packages
source test_env/bin/activate
pip install markuppy
Output:
Collecting markuppy
Downloading MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
Building wheel for markuppy (setup.py) ... done
Created wheel for markuppy: filename=MarkupPy-1.14-py3-none-any.whl size=7414 sha256=cefe8d9f20cecaf72253cab1e18acbdcb6d30827d7a3fd5a74bbef4935bc2e44
Stored in directory: /home/k/.cache/pip/wheels/95/13/60/31c9d5f4cd012e491aeac154ef8b0ec964916523623eb02f0b
Successfully built markuppy
Installing collected packages: markuppy
Successfully installed markuppy-1.14
The obvious downside to this approach is that it breaks isolation between your project's venv and the python packages installed on the system itself.
Example 6: None of this happens in Python3.7
python3.7 -m venv test_env
source test_env/bin/activate
pip install -U pip
pip install markuppy
Output:
Collecting markuppy
Downloading MarkupPy-1.14.tar.gz (6.8 kB)
Using legacy 'setup.py install' for markuppy, since package 'wheel' is not installed.
Installing collected packages: markuppy
Running setup.py install for markuppy ... done
Successfully installed markuppy-1.14
Although it does provide a warning that pip used setup.py install in lieu of wheel.
The Question
How are we supposed to deal with this wheel package? It seems that it's expected that we all have this package installed in most (all?) environments, but the environment tooling in the standard library does not automatically provide it.
I like the idea of keeping my project environments completely isolated from the system python environment, so it looks like the only options that I can see are:
Manually install the wheel package immediately after creating a new venv.
Using another tool besides venv to manage environments.

When setting up a venv, before running anything else, just run pip install --upgrade pip wheel first. This was a good practice anyhow to insure you're using the latest version of pip to resolve your dependencies, hence the warning:
WARNING: You are using pip version 22.0.4; however, version 22.2.2 is available.
You should consider upgrading via the '/home/user/tmp/test_env/bin/python3 -m pip install --upgrade pip' command.
It does seem wheel is no longer included by default, as I was able to replicate on 3.10 as well. Specifying --system-site-packages to use system wheel would be expected, the whole point of venv is to avoid touching the system python. Not being used during a requirements.txt install also makes sense, as it's unlikely any of the packages have wheel listed as a dependency, so it's probably being installed in parallel.

pip cmake Failed to build dlib package

I'm trying to install dlib using cmd.
I installed cmake, wheel, opencv-python, cv2 and face_recognition successfully,
but I have trouble to install dlib.
If tried pip install dlib but it's not working.
I also download dlib-19.22.0.tar.
I'm getting this error message:
Collecting dlib
Using cached dlib-19.22.0.tar.gz (7.4 MB)
Building wheels for collected packages: dlib
Building wheel for dlib (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: 'd:\program files\python39\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\washi\\AppData\\Local\\Temp\\pip-install-_kij9b6z\\dlib\\setup.py'"'"'; __file__='"'"'C:\\Users\\washi\\AppData\\Local\\Temp\\pip-install-_kij9b6z\\dlib\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\washi\AppData\Local\Temp\pip-wheel-yhq98ysd'
cwd: C:\Users\washi\AppData\Local\Temp\pip-install-_kij9b6z\dlib\
Complete output (55 lines):
running bdist_wheel
running build
running build_py
package init file 'tools\python\dlib\__init__.py' not found (or not a regular file)
running build_ext
Building extension for Python 3.9.4 (tags/v3.9.4:1f2e308, Apr 6 2021, 13:40:21) [MSC v.1928 64 bit (AMD64)]
Invoking CMake setup: 'cmake C:\Users\washi\AppData\Local\Temp\pip-install-_kij9b6z\dlib\tools\python -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=C:\Users\washi\AppData\Local\Temp\pip-install-_kij9b6z\dlib\build\lib.win-amd64-3.9 -DPYTHON_EXECUTABLE=d:\program files\python39\python.exe -DCMAKE_LIBRARY_OUTPUT_DIRECTORY_RELEASE=C:\Users\washi\AppData\Local\Temp\pip-install-_kij9b6z\dlib\build\lib.win-amd64-3.9 -A x64'
-- Building for: Visual Studio 16 2019
-- Selecting Windows SDK version to target Windows 10.0.19041.
-- The C compiler identification is unknown
-- The CXX compiler identification is unknown
CMake Error at CMakeLists.txt:14 (project):
No CMAKE_C_COMPILER could be found.
CMake Error at CMakeLists.txt:14 (project):
No CMAKE_CXX_COMPILER could be found.
--Configuring incomplete, errors occurred!
----------------------------------------
ERROR: Failed building wheel for dlib
Running setup.py clean for dlib
Failed to build dlib
Installing collected packages: dlib
Running setup.py install for dlib ... error

How to fix `error: invalid command 'bdist_wheel'`?

I try to install watchman on Ubuntu 20.04.01:
guettli#yoga15:~/tmp$ python3 -m venv pywatchman-test
guettli#yoga15:~/tmp$ cd pywatchman-test
guettli#yoga15:~/tmp/pywatchman-test$ . bin/activate
(pywatchman-test) guettli#yoga15:~/tmp/pywatchman-test$ pip install pywatchman
Fails:
Collecting pywatchman
Using cached pywatchman-1.4.1.tar.gz (29 kB)
Building wheels for collected packages: pywatchman
Building wheel for pywatchman (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/guettli/tmp/pywatchman-test/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-on_zbadt/pywatchman/setup.py'"'"'; __file__='"'"'/tmp/pip-install-on_zbadt/pywatchman/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-9np2rv_b
cwd: /tmp/pip-install-on_zbadt/pywatchman/
Complete output (6 lines):
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'bdist_wheel'
----------------------------------------
ERROR: Failed building wheel for pywatchman
Running setup.py clean for pywatchman
Failed to build pywatchman
Installing collected packages: pywatchman
Running setup.py install for pywatchman ... done
Successfully installed pywatchman-1.4.1
I found a way to solve this: If I uninstall pywatchman and install wheel it works fine.
I would like to create a patch for watchman, so that this works right of the box.
How could this be solved, so that in future this does not happen any more?
Related: https://github.com/facebook/watchman/issues/876

Running
pip install wheel
or
pip3 install wheel
or
python3 -m pip install wheel
solved the issue for me

Cannot run bdist_wheel build command on my setup.py

I have created a pypi package for educational purposes, and I wanted to do an upgrade. I did through the usual steps(do the changes, run setup.py etc)
but when I do:
python3 setup.py bdist_wheel
I am getting the error
python3 setup.py build bdist_wheel
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'bdist_wheel'
Wheel and pip are installed and up to date:
pip3 install wheel
Requirement already satisfied: wheel in /usr/local/lib/python3.4/dist-
packages (0.31.0)
The help of the command, does not seem to support bdist_wheel.
Am I missing something?
python3 setup.py --help-commands
Standard commands:
build build everything needed to install
build_py "build" pure Python modules (copy to build directory)
build_ext build C/C++ extensions (compile/link to build directory)
build_clib build C/C++ libraries used by Python extensions
build_scripts "build" scripts (copy and fixup #! line)
clean clean up temporary files from 'build' command
install install everything from build directory
install_lib install all Python modules (extensions and pure Python)
install_headers install C/C++ header files
install_scripts install scripts (Python or otherwise)
install_data install data files
sdist create a source distribution (tarball, zip file, etc.)
register register the distribution with the Python package index
bdist create a built (binary) distribution
bdist_dumb create a "dumb" built distribution
bdist_rpm create an RPM distribution
bdist_wininst create an executable installer for MS Windows
check perform some checks on the package
upload upload binary package to PyPI
Extra commands:
alias define a shortcut to invoke one or more commands
bdist_egg create an "egg" distribution
develop install package in 'development mode'
easy_install Find/get/install Python packages
egg_info create a distribution's .egg-info directory
install_egg_info Install an .egg-info directory for the package
rotate delete older distributions, keeping N newest files
saveopts save supplied options to setup.cfg or other config file
setopt set an option in setup.cfg or another config file
test run unit tests after in-place build
upload_docs Upload documentation to PyPI
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
EDIT:
python3 --version
Python 3.6.3
pip3 --version
pip 10.0.1 from /usr/local/lib/python3.4/dist-packages/pip (python 3.4)

Looks like pip version and python version didnt match...fixing this removed the problem.

PYMC installation --fcompiler not recognized

I'm trying to install pymc on OSX Mountain lion.
I have installed gfortran (from http://gcc.gnu.org/wiki/GFortranBinaries#MacOS) and cloned pymc.
I tried what has previously been working: 'setup.py config --fcompiler=gfortran build', but this gives an error:
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --fcompiler not recognized
If I just do: 'sudo python setup.py install' then I get this
.
.
.
clang: warning: argument unused during compilation: '-mno-fused-madd'
In file included from src/ft2font.cpp:3:
In file included from src/ft2font.h:16:
/usr/X11/include/ft2build.h:56:10: fatal error: 'freetype/config/ftheader.h' file not found
include
^
1 error generated.
error: Setup script exited with error: command 'clang' failed with exit status 1
Any ideas what's going on?
Thanks,
Jen

Which version of PyMC are you trying to install, the current development version on GitHub master (PyMC 3) or the release version (PyMC 2.3)?
If you are trying to install PyMC 3, you do not need a Fortran compiler. You do need Theano, however. Can you try the following?
pip install --no-deps git+git://github.com/Theano/Theano.git
pip install --no-deps git+git://github.com/pymc-devs/pymc.git