We Keep Coding

Python 3 `venv` and the `wheel` package

Starting with python3.8, I noticed that the wheel package seems to be required to pip install packages without errors (at least they look like errors, but behave like warnings).
Note: in all of my examples, I am cleaning up in between by deactivating my environment, removing the environment, and clearing my pip cache:
deactivate
rm test_env/ -rf
rm ~/.cache/pip/ -rf
Example 1: The problem
python3.8 -m venv test_env
source test_env/bin/activate
pip install markuppy
I get the following output:
Collecting markuppy
Downloading MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
Building wheel for markuppy (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/k/test_env/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-tfm9bgxv/markuppy/setup.py'"'"'; __file__='"'"'/tmp/pip-install-tfm9bgxv/markuppy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-cpx9gxcn
cwd: /tmp/pip-install-tfm9bgxv/markuppy/
Complete output (6 lines):
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'bdist_wheel'
----------------------------------------
ERROR: Failed building wheel for markuppy
Running setup.py clean for markuppy
Failed to build markuppy
Installing collected packages: markuppy
Running setup.py install for markuppy ... done
Successfully installed markuppy-1.14
It errored because there is no wheel package installed. This has been addressed in other posts on SO.
Example 2: Fixing the problem by installing wheel into the venv
If I install the wheel package directly after creating a new venv, this does not happen:
python3.8 -m venv test_env
source test_env/bin/activate
pip install wheel
pip install markuppy
Output:
Collecting markuppy
Using cached MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
Building wheel for markuppy (setup.py) ... done
Created wheel for markuppy: filename=MarkupPy-1.14-py3-none-any.whl size=7413 sha256=52b3e5c3e317ae21724acd871fe3deb85dde9df305b20d16f2c5592c43b11e91
Stored in directory: /home/k/.cache/pip/wheels/95/13/60/31c9d5f4cd012e491aeac154ef8b0ec964916523623eb02f0b
Successfully built markuppy
Installing collected packages: markuppy
Successfully installed markuppy-1.14
This also works given a requirements.txt file.
requirements.txt:
markuppy
python3.8 -m venv test_env
source test_env/bin/activate
pip install wheel
pip install -r requirements.txt
Output:
Collecting markuppy
Using cached MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
Building wheel for markuppy (setup.py) ... done
Created wheel for markuppy: filename=MarkupPy-1.14-py3-none-any.whl size=7413 sha256=52b3e5c3e317ae21724acd871fe3deb85dde9df305b20d16f2c5592c43b11e91
Stored in directory: /home/k/.cache/pip/wheels/95/13/60/31c9d5f4cd012e491aeac154ef8b0ec964916523623eb02f0b
Successfully built markuppy
Installing collected packages: markuppy
Successfully installed markuppy-1.14
Example 3: Adding the wheel package to requirements.txt does not help
It's important to know that if you place the wheel package into the requirements.txt file, and do not install it separately, you get the same problem as in Example 1:
requirements.txt:
wheel
markuppy
python3.8 -m venv test_env
source test_env/bin/activate
pip install -r requirements.txt
Output:
Collecting wheel
Downloading wheel-0.37.0-py2.py3-none-any.whl (35 kB)
Collecting markuppy
Downloading MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
Building wheel for markuppy (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/k/test_env/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-z7cqzaej/markuppy/setup.py'"'"'; __file__='"'"'/tmp/pip-install-z7cqzaej/markuppy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-kw3_8ayt
cwd: /tmp/pip-install-z7cqzaej/markuppy/
Complete output (6 lines):
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'bdist_wheel'
----------------------------------------
ERROR: Failed building wheel for markuppy
Running setup.py clean for markuppy
Failed to build markuppy
Installing collected packages: wheel, markuppy
Running setup.py install for markuppy ... done
Successfully installed markuppy-1.14 wheel-0.37.0
Wheel installed properly, but was not there in time to install the markuppy package.
Example 4: Installing wheel at the system level does not help
If I install the wheel package directly into the system itself, the venv that was created does not have access to it, so I get the same result as in Example 1.
sudo apt install python3-wheel
python3.8 -m venv test_env
source test_env/bin/activate
pip install markuppy
Output:
Collecting markuppy
Downloading MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
Building wheel for markuppy (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/k/test_env/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-5xkssq1l/markuppy/setup.py'"'"'; __file__='"'"'/tmp/pip-install-5xkssq1l/markuppy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-aunof5xf
cwd: /tmp/pip-install-5xkssq1l/markuppy/
Complete output (6 lines):
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'bdist_wheel'
----------------------------------------
ERROR: Failed building wheel for markuppy
Running setup.py clean for markuppy
Failed to build markuppy
Installing collected packages: markuppy
Running setup.py install for markuppy ... done
Successfully installed markuppy-1.14
Example 5: Installing wheel at the system level and granting venv access to system site packages works
If I install wheel at the system level, then grant my venv access to system site packages (note the flag on the venv command) when I create it, then this issue does not happen.
sudo apt install python3-wheel
python3.8 -m venv test_env --system-site-packages
source test_env/bin/activate
pip install markuppy
Output:
Collecting markuppy
Downloading MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
Building wheel for markuppy (setup.py) ... done
Created wheel for markuppy: filename=MarkupPy-1.14-py3-none-any.whl size=7414 sha256=cefe8d9f20cecaf72253cab1e18acbdcb6d30827d7a3fd5a74bbef4935bc2e44
Stored in directory: /home/k/.cache/pip/wheels/95/13/60/31c9d5f4cd012e491aeac154ef8b0ec964916523623eb02f0b
Successfully built markuppy
Installing collected packages: markuppy
Successfully installed markuppy-1.14
The obvious downside to this approach is that it breaks isolation between your project's venv and the python packages installed on the system itself.
Example 6: None of this happens in Python3.7
python3.7 -m venv test_env
source test_env/bin/activate
pip install -U pip
pip install markuppy
Output:
Collecting markuppy
Downloading MarkupPy-1.14.tar.gz (6.8 kB)
Using legacy 'setup.py install' for markuppy, since package 'wheel' is not installed.
Installing collected packages: markuppy
Running setup.py install for markuppy ... done
Successfully installed markuppy-1.14
Although it does provide a warning that pip used setup.py install in lieu of wheel.
The Question
How are we supposed to deal with this wheel package? It seems that it's expected that we all have this package installed in most (all?) environments, but the environment tooling in the standard library does not automatically provide it.
I like the idea of keeping my project environments completely isolated from the system python environment, so it looks like the only options that I can see are:
Manually install the wheel package immediately after creating a new venv.
Using another tool besides venv to manage environments.

When setting up a venv, before running anything else, just run pip install --upgrade pip wheel first. This was a good practice anyhow to insure you're using the latest version of pip to resolve your dependencies, hence the warning:
WARNING: You are using pip version 22.0.4; however, version 22.2.2 is available.
You should consider upgrading via the '/home/user/tmp/test_env/bin/python3 -m pip install --upgrade pip' command.
It does seem wheel is no longer included by default, as I was able to replicate on 3.10 as well. Specifying --system-site-packages to use system wheel would be expected, the whole point of venv is to avoid touching the system python. Not being used during a requirements.txt install also makes sense, as it's unlikely any of the packages have wheel listed as a dependency, so it's probably being installed in parallel.

Installing dryscrape (python library for web scraping) in Google Colab

I was trying to install dryscrape and when i write !pip install dryscrape
in google colab i get the following output:
Collecting dryscrape
Downloading https://files.pythonhosted.org/packages/b5/75/c45f796ec5bc7f98c38b9ae425390ef5f4a76153c8b5af946adb97e7e622/dryscrape-1.0.tar.gz
Collecting webkit_server>=1.0
Downloading https://files.pythonhosted.org/packages/29/f2/f4f454cccde75e95359e91fa58f14497350dc97e58534f9003c77eca3dff/webkit-server-1.0.tar.gz (41kB)
|████████████████████████████████| 51kB 2.5MB/s
Requirement already satisfied: lxml in /usr/local/lib/python3.7/dist-packages (from dryscrape) (4.2.6)
Collecting xvfbwrapper
Downloading https://files.pythonhosted.org/packages/57/b6/4920eabda9b49630dea58745e79f9919aba6408d460afe758bf6e9b21a04/xvfbwrapper-0.2.9.tar.gz
Building wheels for collected packages: dryscrape, webkit-server, xvfbwrapper
Building wheel for dryscrape (setup.py) ... done
Created wheel for dryscrape: filename=dryscrape-1.0-cp37-none-any.whl size=5440 sha256=8e8469df960b731a8cb688d85b0cd4abcdd0aa4bcf5805d817af8e91b57091c1
Stored in directory: /root/.cache/pip/wheels/d5/7e/24/0b5b37166c524082a6fb722bc14c6f885ebb7fcfc7e1563f3e
Building wheel for webkit-server (setup.py) ... error
ERROR: Failed building wheel for webkit-server
Running setup.py clean for webkit-server
Building wheel for xvfbwrapper (setup.py) ... done
Created wheel for xvfbwrapper: filename=xvfbwrapper-0.2.9-cp37-none-any.whl size=5009 sha256=0df17305d7fbff2973dddb1560c1e232f19fcc2a73be1dca2ed2a00d1a2c50f5
Stored in directory: /root/.cache/pip/wheels/10/f2/61/cacfaf84b352c223761ea8d19616e3b5ac5c27364da72863f0
Successfully built dryscrape xvfbwrapper
Failed to build webkit-server
Installing collected packages: webkit-server, xvfbwrapper, dryscrape
Running setup.py install for webkit-server ... error
ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-irrofp8e/webkit-server/setup.py'"'"'; file='"'"'/tmp/pip-install-irrofp8e/webkit-server/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-ufv_nemh/install-record.txt --single-version-externally-managed --compile Check the logs for full command output.
I tried installing from the github repository and i get the same output when i write !pip install -r requirements.txt, i was using the offical documentation.

Seeing to the logs error is regarding the webkit-server library and seeing in the GitHub issues of webkit-server the library came to the pull request, which helped me install webkit-server and then install dryscrape follow the below commands to install it in your Colab.
Command 1:
!pip install git+https://github.com/niklasb/webkit-server.git#refs/pull/35/head
Command 2:
!pip install dryscrape
This will successfully install dryscrape.
Output:
Let me know if you have any questions :)

installing scipy version 1.1.0

I need to install scipy version 1.1.0.
After I run the command pip install scipy==1.1.0, I get the following error:
Collecting scipy==1.1.0
Using cached scipy-1.1.0.tar.gz (15.6 MB)
Using legacy 'setup.py install' for scipy, since package 'wheel' is not installed.
Installing collected packages: scipy
Running setup.py install for scipy ... error
ERROR: Command errored out with exit status 1:
command: 'c:\users\myuser\appdata\local\programs\python\python38\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\myuser\\AppData\\Local\\Temp\\pip-install-n02rqpmt\\scipy\\setup.py'"'"'; __file__='"'"'C:\\Users\\myuser\\AppData\\Local\\Temp\\pip-install-n02rqpmt\\scipy\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\myuser\AppData\Local\Temp\pip-record-qa6zm3ga\install-record.txt' --single-version-externally-managed --compile --install-headers 'c:\users\myuser\appdata\local\programs\python\python38\Include\scipy'
cwd: C:\Users\myuser\AppData\Local\Temp\pip-install-n02rqpmt\scipy\
Complete output (147 lines):
Note: if you need reliable uninstall behavior, then install
with pip instead of using `setup.py install`:
- `pip install .` (from a git repo or downloaded source
release)
- `pip install scipy` (last SciPy release on PyPI)
help me please

You must first install the Python version compatible with scipy.
I could not use pip to install scipy version 1.1 [ I think this version is no longer supported on pip] and used conda instead:
conda install -c anaconda scipy==1.1.0

if you are using pip make sure that, of python2 is installed. or else try with the python3
pip install scipy==1.1.0 for python
pip3 install scipy==1.1.0 for python3
it will install successfully
pip install scipy==1.1.0
Collecting scipy==1.1.0
Downloading https://files.pythonhosted.org/packages/2a/f3/de9c1bd16311982711209edaa8c6caa962db30ebb6a8cc6f1dcd2d3ef616/scipy-1.1.0-cp27-cp27mu-manylinux1_x86_64.whl (30.8MB)
100% |████████████████████████████████| 30.8MB 36kB/s
Collecting numpy>=1.8.2 (from scipy==1.1.0)
Downloading https://files.pythonhosted.org/packages/3a/5f/47e578b3ae79e2624e205445ab77a1848acdaa2929a00eeef6b16eaaeb20/numpy-1.16.6-cp27-cp27mu-manylinux1_x86_64.whl (17.0MB)
100% |████████████████████████████████| 17.0MB 67kB/s
Installing collected packages: numpy, scipy
Successfully installed numpy-1.16.6 scipy-1.1.0

Python package installation via pip error

I’m trying to install few python packages via pip3, but at most of them I’m geting Errors that I don’t understand. For example:
root#turris:~# pip3 install pyshark
Collecting pyshark
Downloading pyshark-0.4.2.9-py3-none-any.whl (31 kB)
Collecting py
Downloading py-1.8.1-py2.py3-none-any.whl (83 kB)
|████████████████████████████████| 83 kB 1.6 MB/s
Collecting lxml
Downloading lxml-4.5.0.tar.gz (4.5 MB)
|████████████████████████████████| 4.5 MB 3.6 MB/s
Installing collected packages: py, lxml, pyshark
Running setup.py install for lxml ... error
ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-eln_tufx/lxml/setup.py'"'"' ; __file__='"'"'/tmp/pip-install-eln_tufx/lxml/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.c lose();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-0uhmqqdg/install-record.txt --single-version-externally-managed --compile --insta ll-headers /usr/include/python3.6/lxml Check the logs for full command output.
I have python3.6 installed and pip 20.0.2. I already updated setuptools and wheel. Now i get this:
root#turris:~/skripty# python3.6 -m pip install numpy
Collecting numpy
Using cached numpy-1.18.2.zip (5.4 MB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... error
ERROR: Command errored out with exit status 1: /usr/bin/python3.6 /usr/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpf_8xl1dr Check the logs for full command output.
I'm using TurrisOS (modificated OpenWRT distro). Does anyone know what is wrong? Thanks.
EDIT:
Ok, I fixed the error for Pyshark package by this:
opkg update
opkg install python3-lxml
pip3 install pyshark
But I’m still getting another errors when installing e.g. numpy.
root#turris:~# pip3 install numpy
Collecting numpy
Using cached numpy-1.18.2.zip (5.4 MB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... error
ERROR: Command errored out with exit status 1: /usr/bin/python3 /usr/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmp6bzl_666 Check the logs for full command output.
Or another package and different error:
root#turris:~# pip3 install matplotlib
Collecting matplotlib
Downloading matplotlib-3.2.1.tar.gz (40.3 MB)
|████████████████████████████████| 40.3 MB 11.3 MB/s
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Python Flask Install - Are these errors bad?

When installing Flask I get errors but it still says successful. Are these errors going to cause problems for me?
Using Python3 on Debian 9
I tried upgrading setuptools, but still get the same errors.
Collecting flask
Using cached Flask-0.12.2-py2.py3-none-any.whl
Collecting itsdangerous>=0.21 (from flask)
Using cached itsdangerous-0.24.tar.gz
Collecting Werkzeug>=0.7 (from flask)
Using cached Werkzeug-0.12.2-py2.py3-none-any.whl
Collecting click>=2.0 (from flask)
Using cached click-6.7-py2.py3-none-any.whl
Collecting Jinja2>=2.4 (from flask)
Using cached Jinja2-2.10-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23 (from Jinja2>=2.4->flask)
Using cached MarkupSafe-1.0.tar.gz
Building wheels for collected packages: itsdangerous, MarkupSafe
Running setup.py bdist_wheel for itsdangerous ... error
Complete output from command /home/ev/Desktop/DNIT/venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-blrz3kyb/itsdangerous/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmprpoc62sipip-wheel- --python-tag cp35:
usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: -c --help [cmd1 cmd2 ...]
or: -c --help-commands
or: -c cmd --help
error: invalid command 'bdist_wheel'
----------------------------------------
Failed building wheel for itsdangerous
Running setup.py clean for itsdangerous
Running setup.py bdist_wheel for MarkupSafe ... error
Complete output from command /home/ev/Desktop/DNIT/venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-blrz3kyb/MarkupSafe/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmp8419jrrypip-wheel- --python-tag cp35:
usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: -c --help [cmd1 cmd2 ...]
or: -c --help-commands
or: -c cmd --help
error: invalid command 'bdist_wheel'
----------------------------------------
Failed building wheel for MarkupSafe
Running setup.py clean for MarkupSafe
Failed to build itsdangerous MarkupSafe
Installing collected packages: itsdangerous, Werkzeug, click, MarkupSafe, Jinja2, flask
Running setup.py install for itsdangerous ... done
Running setup.py install for MarkupSafe ... done
Successfully installed Jinja2-2.10 MarkupSafe-1.0 Werkzeug-0.12.2 click-6.7 flask-0.12.2 itsdangerous-0.24

this is how a solve the problem
pip install wheel
you can remove the venv folder and restart like this
python3 -m venv ./venv
pip install wheel
pip install flask
this will solve the problem