We Keep Coding

Python 3 `venv` and the `wheel` package

Starting with python3.8, I noticed that the wheel package seems to be required to pip install packages without errors (at least they look like errors, but behave like warnings).
Note: in all of my examples, I am cleaning up in between by deactivating my environment, removing the environment, and clearing my pip cache:
deactivate
rm test_env/ -rf
rm ~/.cache/pip/ -rf
Example 1: The problem
python3.8 -m venv test_env
source test_env/bin/activate
pip install markuppy
I get the following output:
Collecting markuppy
Downloading MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
Building wheel for markuppy (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/k/test_env/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-tfm9bgxv/markuppy/setup.py'"'"'; __file__='"'"'/tmp/pip-install-tfm9bgxv/markuppy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-cpx9gxcn
cwd: /tmp/pip-install-tfm9bgxv/markuppy/
Complete output (6 lines):
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'bdist_wheel'
----------------------------------------
ERROR: Failed building wheel for markuppy
Running setup.py clean for markuppy
Failed to build markuppy
Installing collected packages: markuppy
Running setup.py install for markuppy ... done
Successfully installed markuppy-1.14
It errored because there is no wheel package installed. This has been addressed in other posts on SO.
Example 2: Fixing the problem by installing wheel into the venv
If I install the wheel package directly after creating a new venv, this does not happen:
python3.8 -m venv test_env
source test_env/bin/activate
pip install wheel
pip install markuppy
Output:
Collecting markuppy
Using cached MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
Building wheel for markuppy (setup.py) ... done
Created wheel for markuppy: filename=MarkupPy-1.14-py3-none-any.whl size=7413 sha256=52b3e5c3e317ae21724acd871fe3deb85dde9df305b20d16f2c5592c43b11e91
Stored in directory: /home/k/.cache/pip/wheels/95/13/60/31c9d5f4cd012e491aeac154ef8b0ec964916523623eb02f0b
Successfully built markuppy
Installing collected packages: markuppy
Successfully installed markuppy-1.14
This also works given a requirements.txt file.
requirements.txt:
markuppy
python3.8 -m venv test_env
source test_env/bin/activate
pip install wheel
pip install -r requirements.txt
Output:
Collecting markuppy
Using cached MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
Building wheel for markuppy (setup.py) ... done
Created wheel for markuppy: filename=MarkupPy-1.14-py3-none-any.whl size=7413 sha256=52b3e5c3e317ae21724acd871fe3deb85dde9df305b20d16f2c5592c43b11e91
Stored in directory: /home/k/.cache/pip/wheels/95/13/60/31c9d5f4cd012e491aeac154ef8b0ec964916523623eb02f0b
Successfully built markuppy
Installing collected packages: markuppy
Successfully installed markuppy-1.14
Example 3: Adding the wheel package to requirements.txt does not help
It's important to know that if you place the wheel package into the requirements.txt file, and do not install it separately, you get the same problem as in Example 1:
requirements.txt:
wheel
markuppy
python3.8 -m venv test_env
source test_env/bin/activate
pip install -r requirements.txt
Output:
Collecting wheel
Downloading wheel-0.37.0-py2.py3-none-any.whl (35 kB)
Collecting markuppy
Downloading MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
Building wheel for markuppy (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/k/test_env/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-z7cqzaej/markuppy/setup.py'"'"'; __file__='"'"'/tmp/pip-install-z7cqzaej/markuppy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-kw3_8ayt
cwd: /tmp/pip-install-z7cqzaej/markuppy/
Complete output (6 lines):
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'bdist_wheel'
----------------------------------------
ERROR: Failed building wheel for markuppy
Running setup.py clean for markuppy
Failed to build markuppy
Installing collected packages: wheel, markuppy
Running setup.py install for markuppy ... done
Successfully installed markuppy-1.14 wheel-0.37.0
Wheel installed properly, but was not there in time to install the markuppy package.
Example 4: Installing wheel at the system level does not help
If I install the wheel package directly into the system itself, the venv that was created does not have access to it, so I get the same result as in Example 1.
sudo apt install python3-wheel
python3.8 -m venv test_env
source test_env/bin/activate
pip install markuppy
Output:
Collecting markuppy
Downloading MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
Building wheel for markuppy (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/k/test_env/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-5xkssq1l/markuppy/setup.py'"'"'; __file__='"'"'/tmp/pip-install-5xkssq1l/markuppy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-aunof5xf
cwd: /tmp/pip-install-5xkssq1l/markuppy/
Complete output (6 lines):
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'bdist_wheel'
----------------------------------------
ERROR: Failed building wheel for markuppy
Running setup.py clean for markuppy
Failed to build markuppy
Installing collected packages: markuppy
Running setup.py install for markuppy ... done
Successfully installed markuppy-1.14
Example 5: Installing wheel at the system level and granting venv access to system site packages works
If I install wheel at the system level, then grant my venv access to system site packages (note the flag on the venv command) when I create it, then this issue does not happen.
sudo apt install python3-wheel
python3.8 -m venv test_env --system-site-packages
source test_env/bin/activate
pip install markuppy
Output:
Collecting markuppy
Downloading MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
Building wheel for markuppy (setup.py) ... done
Created wheel for markuppy: filename=MarkupPy-1.14-py3-none-any.whl size=7414 sha256=cefe8d9f20cecaf72253cab1e18acbdcb6d30827d7a3fd5a74bbef4935bc2e44
Stored in directory: /home/k/.cache/pip/wheels/95/13/60/31c9d5f4cd012e491aeac154ef8b0ec964916523623eb02f0b
Successfully built markuppy
Installing collected packages: markuppy
Successfully installed markuppy-1.14
The obvious downside to this approach is that it breaks isolation between your project's venv and the python packages installed on the system itself.
Example 6: None of this happens in Python3.7
python3.7 -m venv test_env
source test_env/bin/activate
pip install -U pip
pip install markuppy
Output:
Collecting markuppy
Downloading MarkupPy-1.14.tar.gz (6.8 kB)
Using legacy 'setup.py install' for markuppy, since package 'wheel' is not installed.
Installing collected packages: markuppy
Running setup.py install for markuppy ... done
Successfully installed markuppy-1.14
Although it does provide a warning that pip used setup.py install in lieu of wheel.
The Question
How are we supposed to deal with this wheel package? It seems that it's expected that we all have this package installed in most (all?) environments, but the environment tooling in the standard library does not automatically provide it.
I like the idea of keeping my project environments completely isolated from the system python environment, so it looks like the only options that I can see are:
Manually install the wheel package immediately after creating a new venv.
Using another tool besides venv to manage environments.

When setting up a venv, before running anything else, just run pip install --upgrade pip wheel first. This was a good practice anyhow to insure you're using the latest version of pip to resolve your dependencies, hence the warning:
WARNING: You are using pip version 22.0.4; however, version 22.2.2 is available.
You should consider upgrading via the '/home/user/tmp/test_env/bin/python3 -m pip install --upgrade pip' command.
It does seem wheel is no longer included by default, as I was able to replicate on 3.10 as well. Specifying --system-site-packages to use system wheel would be expected, the whole point of venv is to avoid touching the system python. Not being used during a requirements.txt install also makes sense, as it's unlikely any of the packages have wheel listed as a dependency, so it's probably being installed in parallel.

Error executing: pip install opencv-python

So im trying to install OpenCV-Python using my administrator command prompt in windows 10 and my version of python is 3.9 and pip is fully updated.. here is the (massive) error I get after trying to execute pip install opencv-python
This is what happens:
C:\WINDOWS\system32>pip install opencv-python
Collecting opencv-python
Using cached opencv-python-4.4.0.44.tar.gz (88.9 MB)
Installing build dependencies ... error
ERROR: Command errored out with exit status 1:
command: 'C:\Users\inbet\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\python.exe' 'C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.240.0_x64__qbz5n2kfra8p0\lib\site-packages\pip' install --ignore-installed --no-user --prefix 'C:\Users\inbet\AppData\Local\Temp\pip-build-env-prbqcb63\overlay' --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel scikit-build cmake pip 'numpy==1.11.3; python_version=='"'"'3.5'"'"'' 'numpy==1.13.3; python_version=='"'"'3.6'"'"'' 'numpy==1.14.5; python_version=='"'"'3.7'"'"'' 'numpy==1.17.3; python_version>='"'"'3.8'"'"''
cwd: None
Complete output (310 lines):
Ignoring numpy: markers 'python_version == "3.5"' don't match your environment
Ignoring numpy: markers 'python_version == "3.6"' don't match your environment
Ignoring numpy: markers 'python_version == "3.7"' don't match your environment
Collecting setuptools
Using cached setuptools-50.3.1-py3-none-any.whl (785 kB)
Collecting wheel
Using cached wheel-0.35.1-py2.py3-none-any.whl (33 kB)
Collecting scikit-build
Using cached scikit_build-0.11.1-py2.py3-none-any.whl (72 kB)
Collecting cmake
Using cached cmake-3.18.2.post1-py3-none-win_amd64.whl (34.7 MB)
Collecting pip
Using cached pip-20.2.3-py2.py3-none-any.whl (1.5 MB)
Collecting numpy==1.17.3
Using cached numpy-1.17.3.zip (6.4 MB)
Collecting distro
Using cached distro-1.5.0-py2.py3-none-any.whl (18 kB)
Collecting packaging
Using cached packaging-20.4-py2.py3-none-any.whl (37 kB)
Collecting six
Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting pyparsing>=2.0.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Using legacy 'setup.py install' for numpy, since package 'wheel' is not installed.
Installing collected packages: setuptools, wheel, distro, six, pyparsing, packaging, scikit-build, cmake, pip, numpy
Running setup.py install for numpy: started
Running setup.py install for numpy: finished with status 'error'
ERROR: Command errored out with exit status 1:
command: 'C:\Users\inbet\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\inbet\\AppData\\Local\\Temp\\pip-install-dqkz6qyf\\numpy\\setup.py'"'"'; __file__='"'"'C:\\Users\\inbet\\AppData\\Local\\Temp\\pip-install-dqkz6qyf\\numpy\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\inbet\AppData\Local\Temp\pip-record-zko205u_\install-record.txt' --single-version-externally-managed --prefix 'C:\Users\inbet\AppData\Local\Temp\pip-build-env-prbqcb63\overlay' --compile --install-headers 'C:\Users\inbet\AppData\Local\Temp\pip-build-env-prbqcb63\overlay\Include\numpy'
cwd: C:\Users\inbet\AppData\Local\Temp\pip-install-dqkz6qyf\numpy\
Complete output (277 lines):
Running from numpy source directory.
There is a LOT more after that but I am unable to post the error in its entirety as it says "it looks like your post is mostly code. please add some more details"
My gut feeling is that all of this is occurring because my python directory isn't in C:\ProgramFiles\Python or maybe it has to do with the fact that I installed python via the windows app store.
I know its a lot of code and I am literally just beginning to get into this because I am trying to create a script that will scrape eBay thumbnails and download the images from a start page with search terms entered, if and only if those images contain numbers. Hence why I am trying to install OpenCV.
Any help would be greatly appreciated. The last time I ever coded anything was when I was 12 years old playing Never Winter Nights which used something like C+ so I generally know my way around under the hood but am totally new to python specific commands and am hoping this little project I am attempting will teach me a useful amount about python and also automate this whole "needle in a haystack" search for images with numbers in them.
Cheers

opencv require install wheel before that

installing scipy version 1.1.0

I need to install scipy version 1.1.0.
After I run the command pip install scipy==1.1.0, I get the following error:
Collecting scipy==1.1.0
Using cached scipy-1.1.0.tar.gz (15.6 MB)
Using legacy 'setup.py install' for scipy, since package 'wheel' is not installed.
Installing collected packages: scipy
Running setup.py install for scipy ... error
ERROR: Command errored out with exit status 1:
command: 'c:\users\myuser\appdata\local\programs\python\python38\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\myuser\\AppData\\Local\\Temp\\pip-install-n02rqpmt\\scipy\\setup.py'"'"'; __file__='"'"'C:\\Users\\myuser\\AppData\\Local\\Temp\\pip-install-n02rqpmt\\scipy\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\myuser\AppData\Local\Temp\pip-record-qa6zm3ga\install-record.txt' --single-version-externally-managed --compile --install-headers 'c:\users\myuser\appdata\local\programs\python\python38\Include\scipy'
cwd: C:\Users\myuser\AppData\Local\Temp\pip-install-n02rqpmt\scipy\
Complete output (147 lines):
Note: if you need reliable uninstall behavior, then install
with pip instead of using `setup.py install`:
- `pip install .` (from a git repo or downloaded source
release)
- `pip install scipy` (last SciPy release on PyPI)
help me please

You must first install the Python version compatible with scipy.
I could not use pip to install scipy version 1.1 [ I think this version is no longer supported on pip] and used conda instead:
conda install -c anaconda scipy==1.1.0

if you are using pip make sure that, of python2 is installed. or else try with the python3
pip install scipy==1.1.0 for python
pip3 install scipy==1.1.0 for python3
it will install successfully
pip install scipy==1.1.0
Collecting scipy==1.1.0
Downloading https://files.pythonhosted.org/packages/2a/f3/de9c1bd16311982711209edaa8c6caa962db30ebb6a8cc6f1dcd2d3ef616/scipy-1.1.0-cp27-cp27mu-manylinux1_x86_64.whl (30.8MB)
100% |████████████████████████████████| 30.8MB 36kB/s
Collecting numpy>=1.8.2 (from scipy==1.1.0)
Downloading https://files.pythonhosted.org/packages/3a/5f/47e578b3ae79e2624e205445ab77a1848acdaa2929a00eeef6b16eaaeb20/numpy-1.16.6-cp27-cp27mu-manylinux1_x86_64.whl (17.0MB)
100% |████████████████████████████████| 17.0MB 67kB/s
Installing collected packages: numpy, scipy
Successfully installed numpy-1.16.6 scipy-1.1.0

Python package installation via pip error

I’m trying to install few python packages via pip3, but at most of them I’m geting Errors that I don’t understand. For example:
root#turris:~# pip3 install pyshark
Collecting pyshark
Downloading pyshark-0.4.2.9-py3-none-any.whl (31 kB)
Collecting py
Downloading py-1.8.1-py2.py3-none-any.whl (83 kB)
|████████████████████████████████| 83 kB 1.6 MB/s
Collecting lxml
Downloading lxml-4.5.0.tar.gz (4.5 MB)
|████████████████████████████████| 4.5 MB 3.6 MB/s
Installing collected packages: py, lxml, pyshark
Running setup.py install for lxml ... error
ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-eln_tufx/lxml/setup.py'"'"' ; __file__='"'"'/tmp/pip-install-eln_tufx/lxml/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.c lose();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-0uhmqqdg/install-record.txt --single-version-externally-managed --compile --insta ll-headers /usr/include/python3.6/lxml Check the logs for full command output.
I have python3.6 installed and pip 20.0.2. I already updated setuptools and wheel. Now i get this:
root#turris:~/skripty# python3.6 -m pip install numpy
Collecting numpy
Using cached numpy-1.18.2.zip (5.4 MB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... error
ERROR: Command errored out with exit status 1: /usr/bin/python3.6 /usr/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpf_8xl1dr Check the logs for full command output.
I'm using TurrisOS (modificated OpenWRT distro). Does anyone know what is wrong? Thanks.
EDIT:
Ok, I fixed the error for Pyshark package by this:
opkg update
opkg install python3-lxml
pip3 install pyshark
But I’m still getting another errors when installing e.g. numpy.
root#turris:~# pip3 install numpy
Collecting numpy
Using cached numpy-1.18.2.zip (5.4 MB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... error
ERROR: Command errored out with exit status 1: /usr/bin/python3 /usr/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmp6bzl_666 Check the logs for full command output.
Or another package and different error:
root#turris:~# pip3 install matplotlib
Collecting matplotlib
Downloading matplotlib-3.2.1.tar.gz (40.3 MB)
|████████████████████████████████| 40.3 MB 11.3 MB/s
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Google Colaboratory Python Installation Error: TA-Lib

Recently, I have attempted to complete a neural network and technical analysis program for the prediction of fluctuations within the financial equities markets available via utilised databases; I am currently utilising the module Quandl for the purposes of financial information retrieval, while the program itself remains within the Google Colaboratory integrated development environment. For the purposes of this project, I have recently attempted to install a Python wrapper for the module TA-Lib, which contains a portion designed for the analysis of candlestick chart patterns; the documentation for the library and page are shown below:
https://github.com/mrjbq7/ta-lib
https://mrjbq7.github.io/ta-lib/func_groups/pattern_recognition.html
Within the Colaboratory editor, I attempted to install the program via the command:
pip install TA-Lib
as specified; however, the integrated development environment provided the following error message:
Collecting TA-Lib
Using cached https://files.pythonhosted.org/packages/90/05/d4c6a778d7a7de0be366bc4a850b4ffaeac2abad927f95fa8ba6f355a082/TA-Lib-0.4.17.tar.gz
Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from TA-Lib) (1.17.5)
Building wheels for collected packages: TA-Lib
Building wheel for TA-Lib (setup.py) ... error
ERROR: Failed building wheel for TA-Lib
Running setup.py clean for TA-Lib
Failed to build TA-Lib
Installing collected packages: TA-Lib
Running setup.py install for TA-Lib ... error
ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-npc2f3yq/TA-Lib/setup.py'"'"'; __file__='"'"'/tmp/pip-install-npc2f3yq/TA-Lib/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-oh1la09j/install-record.txt --single-version-externally-managed --compile Check the logs for full command output.
Is there an alternative method responsible for the avoidance of this inevitable error? If not, is there a method to download this module within Colaboratory which would bypass such processes? Thank you for your assistance.

Try this
!wget http://prdownloads.sourceforge.net/ta-lib/ta-lib-0.4.0-src.tar.gz
!tar -xzvf ta-lib-0.4.0-src.tar.gz
%cd ta-lib
!./configure --prefix=/usr
!make
!make install
!pip install Ta-Lib
import talib