docker cannot build scipy - python

I am trying to create a docker container for my python application (this is the first time I am trying to use docker). I looked at the online tutorials and created a DockerFile as follows:
FROM python:3.6-alpine
COPY . /app
WORKDIR /app
RUN apk --update add --no-cache \
lapack-dev \
gcc \
freetype-dev
# Install dependencies
RUN apk add --no-cache --virtual .build-deps \
gfortran \
musl-dev \
g++
RUN pip3 install -r requirements.txt
RUN python3 setup.py install
RUN apk del .build-deps
ENTRYPOINT python3 testapp.py
My project requirements are:
numpy==1.13.3
Cython==0.28.2
nibabel==2.2.1
scipy==1.0.0
I build the docker file as: docker build -t myimg .
So, the docker file progresses but scipy fails to build with the following error:
Collecting numpy==1.13.3 (from -r requirements.txt (line 1))
Downloading https://files.pythonhosted.org/packages/bf/2d/005e45738ab07a26e621c9c12dc97381f372e06678adf7dc3356a69b5960/numpy-1.13.3.zip (5.0MB)
Collecting Cython==0.28.2 (from -r requirements.txt (line 2))
Downloading https://files.pythonhosted.org/packages/79/9d/dea8c5181cdb77d32e20a44dd5346b0e4bac23c4858f2f66ad64bbcf4de8/Cython-0.28.2.tar.gz (1.9MB)
Collecting nibabel==2.2.1 (from -r requirements.txt (line 3))
Downloading https://files.pythonhosted.org/packages/d7/de/1d96fd0b118c9047bf35f02090db8ef8fd3927dfce635f09a6f7d5b572e6/nibabel-2.2.1.zip (4.2MB)
Collecting scipy==1.0.0 (from -r requirements.txt (line 4))
Downloading https://files.pythonhosted.org/packages/d0/73/76fc6ea21818eed0de8dd38e1e9586725578864169a2b31acdeffb9131c8/scipy-1.0.0.tar.gz (15.2MB)
Collecting six>=1.3 (from nibabel==2.2.1->-r requirements.txt (line 3))
Downloading https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Building wheels for collected packages: numpy, Cython, nibabel, scipy
Running setup.py bdist_wheel for numpy: started
Running setup.py bdist_wheel for numpy: still running...
Running setup.py bdist_wheel for numpy: still running...
Running setup.py bdist_wheel for numpy: still running...
Running setup.py bdist_wheel for numpy: finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/b6/10/65/189b772e73b4505109d5a1e6671b07e65797023718777295e0
Running setup.py bdist_wheel for Cython: started
Running setup.py bdist_wheel for Cython: still running...
Running setup.py bdist_wheel for Cython: still running...
Running setup.py bdist_wheel for Cython: still running...
Running setup.py bdist_wheel for Cython: finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/6f/24/5d/def09ad0aed8ba26186f2a38070906f70ab4b2287bf64d4414
Running setup.py bdist_wheel for nibabel: started
Running setup.py bdist_wheel for nibabel: finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/46/50/8d/bcb0b8f7c030da5bac1752fbe9cc375cbf5725fa93ba79ad84
Running setup.py bdist_wheel for scipy: started
Running setup.py bdist_wheel for scipy: finished with status 'error'
Complete output from command /usr/local/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-boosbyfg/scipy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-cczhwdqj --python-tag cp36:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-boosbyfg/scipy/setup.py", line 418, in <module>
setup_package()
File "/tmp/pip-install-boosbyfg/scipy/setup.py", line 398, in setup_package
from numpy.distutils.core import setup
ModuleNotFoundError: No module named 'numpy'
Not sure why it is having issue finding numpy as it was installed as part of the requirements?

Because to build scipy wheel you need to have numpy installed. However, only numpy wheel build was complete by the time pip attempts to build scipy wheel.
You will have to install the dependencies first. There are multiple ways to do this.
1) Use a shell script like the one below, copy it and RUN it instead of RUN pip install -r requirements.txt:
#!/bin/sh
while read module; do
pip install $module
done < requirements.txt
2) Install scipy in a seperate RUN command.
3) apk add py-numpy#community as discussed in this answer.

Related

Python 3 `venv` and the `wheel` package

Starting with python3.8, I noticed that the wheel package seems to be required to pip install packages without errors (at least they look like errors, but behave like warnings).
Note: in all of my examples, I am cleaning up in between by deactivating my environment, removing the environment, and clearing my pip cache:
deactivate
rm test_env/ -rf
rm ~/.cache/pip/ -rf
Example 1: The problem
python3.8 -m venv test_env
source test_env/bin/activate
pip install markuppy
I get the following output:
Collecting markuppy
Downloading MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
Building wheel for markuppy (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/k/test_env/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-tfm9bgxv/markuppy/setup.py'"'"'; __file__='"'"'/tmp/pip-install-tfm9bgxv/markuppy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-cpx9gxcn
cwd: /tmp/pip-install-tfm9bgxv/markuppy/
Complete output (6 lines):
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'bdist_wheel'
----------------------------------------
ERROR: Failed building wheel for markuppy
Running setup.py clean for markuppy
Failed to build markuppy
Installing collected packages: markuppy
Running setup.py install for markuppy ... done
Successfully installed markuppy-1.14
It errored because there is no wheel package installed. This has been addressed in other posts on SO.
Example 2: Fixing the problem by installing wheel into the venv
If I install the wheel package directly after creating a new venv, this does not happen:
python3.8 -m venv test_env
source test_env/bin/activate
pip install wheel
pip install markuppy
Output:
Collecting markuppy
Using cached MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
Building wheel for markuppy (setup.py) ... done
Created wheel for markuppy: filename=MarkupPy-1.14-py3-none-any.whl size=7413 sha256=52b3e5c3e317ae21724acd871fe3deb85dde9df305b20d16f2c5592c43b11e91
Stored in directory: /home/k/.cache/pip/wheels/95/13/60/31c9d5f4cd012e491aeac154ef8b0ec964916523623eb02f0b
Successfully built markuppy
Installing collected packages: markuppy
Successfully installed markuppy-1.14
This also works given a requirements.txt file.
requirements.txt:
markuppy
python3.8 -m venv test_env
source test_env/bin/activate
pip install wheel
pip install -r requirements.txt
Output:
Collecting markuppy
Using cached MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
Building wheel for markuppy (setup.py) ... done
Created wheel for markuppy: filename=MarkupPy-1.14-py3-none-any.whl size=7413 sha256=52b3e5c3e317ae21724acd871fe3deb85dde9df305b20d16f2c5592c43b11e91
Stored in directory: /home/k/.cache/pip/wheels/95/13/60/31c9d5f4cd012e491aeac154ef8b0ec964916523623eb02f0b
Successfully built markuppy
Installing collected packages: markuppy
Successfully installed markuppy-1.14
Example 3: Adding the wheel package to requirements.txt does not help
It's important to know that if you place the wheel package into the requirements.txt file, and do not install it separately, you get the same problem as in Example 1:
requirements.txt:
wheel
markuppy
python3.8 -m venv test_env
source test_env/bin/activate
pip install -r requirements.txt
Output:
Collecting wheel
Downloading wheel-0.37.0-py2.py3-none-any.whl (35 kB)
Collecting markuppy
Downloading MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
Building wheel for markuppy (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/k/test_env/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-z7cqzaej/markuppy/setup.py'"'"'; __file__='"'"'/tmp/pip-install-z7cqzaej/markuppy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-kw3_8ayt
cwd: /tmp/pip-install-z7cqzaej/markuppy/
Complete output (6 lines):
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'bdist_wheel'
----------------------------------------
ERROR: Failed building wheel for markuppy
Running setup.py clean for markuppy
Failed to build markuppy
Installing collected packages: wheel, markuppy
Running setup.py install for markuppy ... done
Successfully installed markuppy-1.14 wheel-0.37.0
Wheel installed properly, but was not there in time to install the markuppy package.
Example 4: Installing wheel at the system level does not help
If I install the wheel package directly into the system itself, the venv that was created does not have access to it, so I get the same result as in Example 1.
sudo apt install python3-wheel
python3.8 -m venv test_env
source test_env/bin/activate
pip install markuppy
Output:
Collecting markuppy
Downloading MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
Building wheel for markuppy (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/k/test_env/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-5xkssq1l/markuppy/setup.py'"'"'; __file__='"'"'/tmp/pip-install-5xkssq1l/markuppy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-aunof5xf
cwd: /tmp/pip-install-5xkssq1l/markuppy/
Complete output (6 lines):
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'bdist_wheel'
----------------------------------------
ERROR: Failed building wheel for markuppy
Running setup.py clean for markuppy
Failed to build markuppy
Installing collected packages: markuppy
Running setup.py install for markuppy ... done
Successfully installed markuppy-1.14
Example 5: Installing wheel at the system level and granting venv access to system site packages works
If I install wheel at the system level, then grant my venv access to system site packages (note the flag on the venv command) when I create it, then this issue does not happen.
sudo apt install python3-wheel
python3.8 -m venv test_env --system-site-packages
source test_env/bin/activate
pip install markuppy
Output:
Collecting markuppy
Downloading MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
Building wheel for markuppy (setup.py) ... done
Created wheel for markuppy: filename=MarkupPy-1.14-py3-none-any.whl size=7414 sha256=cefe8d9f20cecaf72253cab1e18acbdcb6d30827d7a3fd5a74bbef4935bc2e44
Stored in directory: /home/k/.cache/pip/wheels/95/13/60/31c9d5f4cd012e491aeac154ef8b0ec964916523623eb02f0b
Successfully built markuppy
Installing collected packages: markuppy
Successfully installed markuppy-1.14
The obvious downside to this approach is that it breaks isolation between your project's venv and the python packages installed on the system itself.
Example 6: None of this happens in Python3.7
python3.7 -m venv test_env
source test_env/bin/activate
pip install -U pip
pip install markuppy
Output:
Collecting markuppy
Downloading MarkupPy-1.14.tar.gz (6.8 kB)
Using legacy 'setup.py install' for markuppy, since package 'wheel' is not installed.
Installing collected packages: markuppy
Running setup.py install for markuppy ... done
Successfully installed markuppy-1.14
Although it does provide a warning that pip used setup.py install in lieu of wheel.
The Question
How are we supposed to deal with this wheel package? It seems that it's expected that we all have this package installed in most (all?) environments, but the environment tooling in the standard library does not automatically provide it.
I like the idea of keeping my project environments completely isolated from the system python environment, so it looks like the only options that I can see are:
Manually install the wheel package immediately after creating a new venv.
Using another tool besides venv to manage environments.
When setting up a venv, before running anything else, just run pip install --upgrade pip wheel first. This was a good practice anyhow to insure you're using the latest version of pip to resolve your dependencies, hence the warning:
WARNING: You are using pip version 22.0.4; however, version 22.2.2 is available.
You should consider upgrading via the '/home/user/tmp/test_env/bin/python3 -m pip install --upgrade pip' command.
It does seem wheel is no longer included by default, as I was able to replicate on 3.10 as well. Specifying --system-site-packages to use system wheel would be expected, the whole point of venv is to avoid touching the system python. Not being used during a requirements.txt install also makes sense, as it's unlikely any of the packages have wheel listed as a dependency, so it's probably being installed in parallel.

pip3 install isal fails in Ubuntu 18 but works fine in Ubuntu 20 (dockerized scenario)

I would greatly appreciate help in tackling a problem that drives me crazy (I do need Ubuntu 18.04 and python 3). I did try using different scenarios, but everything fails when installing PyPI package isal on Ubuntu 18.04:
FROM ubuntu:18.04
RUN apt update -y && apt upgrade -y
RUN apt install -y python3 python3-pip
RUN pip3 install isal
docker build . fails with:
Step 3/3 : RUN pip3 install isal
---> Running in 71a47c31d97c
Collecting isal
Downloading https://files.pythonhosted.org/packages/d6/72/b997fd8ba95a0820edcd5da268505705a5518fd860d64bf28a7c1c343a3a/isal-0.11.0.tar.gz (680kB)
Building wheels for collected packages: isal
Running setup.py bdist_wheel for isal: started
Running setup.py bdist_wheel for isal: finished with status 'error'
....
running build_ext
/tmp/tmpk3o08f96/autogen.sh: 3: /tmp/tmpk3o08f96/autogen.sh: autoreconf: not found
error: [Errno 2] No such file or directory: '/tmp/tmpk3o08f96/configure': '/tmp/tmpk3o08f96/configure'
----------------------------------------
Failed building wheel for isal
Running setup.py clean for isal
Failed to build isal
Installing collected packages: isal
Running setup.py install for isal: started
Running setup.py install for isal: finished with status 'error'
Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-qa68yevk/isal/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-8qioc11y-record/install-record.txt --single-version-externally-managed --compile:
/usr/lib/python3.6/distutils/dist.py:261: UserWarning: Unknown distribution option: 'long_description_content_type'
warnings.warn(msg)
....
running build_ext
/tmp/tmpnfxsy9ug/autogen.sh: 3: /tmp/tmpnfxsy9ug/autogen.sh: autoreconf: not found
error: [Errno 2] No such file or directory: '/tmp/tmpnfxsy9ug/configure': '/tmp/tmpnfxsy9ug/configure'
----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-qa68yevk/isal/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-8qioc11y-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-qa68yevk/isal/
The command '/bin/sh -c pip3 install isal' returned a non-zero code: 1
while FROM ubuntu:20.04 everything works fine
pip also fails:
FROM ubuntu:18.04
RUN apt update -y && apt upgrade -y \
&& apt install -y python python-pip
RUN pip install isal
Step 3/3 : RUN pip install isal
---> Running in 6e157d7d965a
Collecting isal
Could not find a version that satisfies the requirement isal (from versions: )
No matching distribution found for isal
The command '/bin/sh -c pip install isal' returned a non-zero code: 1
you can try apt install autoconf to resolve autoreconf: not found

error Collecting pandas==1.3.0 in alpine image

I am trying to install apache-airflow inside a docker image with alpine, the docker file is the following:
FROM python:3.9.5-alpine3.13
WORKDIR /usr/app
RUN pip install pipenv
COPY Pipfile* ./
RUN apk add --no-cache libressl-dev musl-dev libffi-dev libressl-dev musl-dev libffi-dev gcc build-base
RUN apk add gcc musl-dev libffi-dev openssl-dev python3-dev
RUN apk update && apk add libressl-dev postgresql-dev libffi-dev gcc musl-dev python3-dev
RUN pipenv install --system --deploy --ignore-pipfile
RUN airflow db init
RUN airflow users create --username admin --password admin --firstname Anonymous --lastname Admin --role Admin --email admin#example.org
RUN cp dags ~/airflow/dags/
RUN airflow webserver
RUN airflow scheduler
COPY ./src ./src
COPY ./src/.env.docker ./src/.env
CMD ["python3", "src/main.py"]
But when I execute that I got the following error:
An error occurred while installing cryptography==3.4.7; python_version >= '3.6' --hash=sha256:3d10de8116d25649631977cb37da6c
I tried solved installing another a lot of libraries but I still cannot install any idea?
EDIT
It is necessary to install rust in alpine, but that alpine has an old version.
EDIT 2
After update the docker file like this:
FROM python:3.9.6-alpine3.14
WORKDIR /usr/app
RUN pip install pipenv
COPY Pipfile* ./
RUN apk add --no-cache gcc musl-dev python3-dev libffi-dev openssl-dev cargo
RUN pipenv install --system --deploy --ignore-pipfile
RUN airflow db init
RUN airflow users create --username admin --password admin --firstname Anonymous --lastname Admin --role Admin --email admin#example.org
RUN cp dags ~/airflow/dags/
RUN airflow webserver
RUN airflow scheduler
COPY ./src ./src
COPY ./src/.env.docker ./src/.env
CMD ["python3", "src/main.py"]
I got the following error:
[pipenv.exceptions.InstallError]: Collecting pandas==1.3.0
[pipenv.exceptions.InstallError]: Using cached pandas-1.3.0.tar.gz
(4.7 MB) [pipenv.exceptions.InstallError]: ERROR: Command errored
out with exit status 1: [pipenv.exceptions.InstallError]:
command: /usr/local/bin/python -c 'import io, os, sys, setuptools,
tokenize; sys.argv[0] =
'"'"'/tmp/pip-install-lfncf2u8/pandas_6f9f84af90264ff59c98fca45e89ed74/setup.py'"'"';
file='"'"'/tmp/pip-install-lfncf2u8/pandas_6f9f84af90264ff59c98fca45e89ed74/setup.py'"'"';f
= getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import
setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"',
'"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))'
egg_info --egg-base /tmp/pip-pip-egg-info-s8ljky3j
[pipenv.exceptions.InstallError]: cwd:
/tmp/pip-install-lfncf2u8/pandas_6f9f84af90264ff59c98fca45e89ed74/
[pipenv.exceptions.InstallError]: Complete output (7 lines):
[pipenv.exceptions.InstallError]: Traceback (most recent call
last): [pipenv.exceptions.InstallError]: File "", line
1, in [pipenv.exceptions.InstallError]: File
"/tmp/pip-install-lfncf2u8/pandas_6f9f84af90264ff59c98fca45e89ed74/setup.py",
line 650, in [pipenv.exceptions.InstallError]:
ext_modules=maybe_cythonize(extensions,
compiler_directives=directives), [pipenv.exceptions.InstallError]:
File
"/tmp/pip-install-lfncf2u8/pandas_6f9f84af90264ff59c98fca45e89ed74/setup.py",
line 414, in maybe_cythonize [pipenv.exceptions.InstallError]:
raise RuntimeError("Cannot cythonize without Cython installed.")
[pipenv.exceptions.InstallError]: RuntimeError: Cannot cythonize
without Cython installed.
Thanks
Answer for main question
Do you try python3 -m pip install cryptography==3.4.7 ?
Answer for EDIT 2
You probably need to install Cython.
I redirect you to this page about Cython in Alpine:
https://pkgs.alpinelinux.org/package/v3.3/main/x86/cython
You can also try to run python3 -m pip install Cython.

DockerSkeleton python ML module installation error

I am trying to install python ml modules via dockerSkeleton so i can invoke docker action from openwhisk cli but i am keep getting below errors and couldn't find much help.
Collecting scikit-learn (from sklearn->-r requirements.txt (line 4))
Downloading https://files.pythonhosted.org/packages/25/b6 /454cf208be93efa3db50ce06b732328c57ede005d1dcfa71d9a1548530b0/scikit-learn-0.19.2.tar.gz (9.7MB)
Building wheels for collected packages: numpy, elm, scipy, sklearn, deap, optunity, scikit-learn
Running setup.py bdist_wheel for numpy: started
Running setup.py bdist_wheel for numpy: still running...
Running setup.py bdist_wheel for numpy: still running...
Running setup.py bdist_wheel for numpy: still running...
Running setup.py bdist_wheel for numpy: finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/29/74/5c/331da973b82cedd300e088ad58bd241c4c8857dd6faf5433eb
Running setup.py bdist_wheel for elm: started
Running setup.py bdist_wheel for elm: finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/31/17/21/20fd245c9d2b1db0e4805d9e3a9b36c2767ce4816378548cf7
Running setup.py bdist_wheel for scipy: started
Running setup.py bdist_wheel for scipy: finished with status 'error'
Complete output from command /usr/local/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-jprwp998/scipy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-st6v_gb2 --python-tag cp36:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-jprwp998/scipy/setup.py", line 474, in <module>
setup_package()
File "/tmp/pip-install-jprwp998/scipy/setup.py", line 450, in setup_package
from numpy.distutils.core import setup
ModuleNotFoundError: No module named 'numpy'
----------------------------------------
Failed building wheel for scipy
Running setup.py clean for scipy
Complete output from command /usr/local/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-jprwp998/scipy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" clean --all:
`setup.py clean` is not supported, use one of the following instead:
- `git clean -xdf` (cleans all files)
- `git clean -Xdf` (cleans all versioned files, doesn't touch
files that aren't checked into the git repo)
Add `--force` to your command to use it anyway if you must (unsupported).
----------------------------------------
Failed cleaning build dir for scipy
Running setup.py bdist_wheel for sklearn: started
Running setup.py bdist_wheel for sklearn: finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/76/03/bb/589d421d27431bcd2c6da284d5f2286c8e3b2ea3cf1594c074
Running setup.py bdist_wheel for deap: started
Running setup.py bdist_wheel for deap: finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/22/ea/bf/dc7c8a2262025a0ab5da9ef02282c198be88902791ca0c6658
Running setup.py bdist_wheel for optunity: started
Running setup.py bdist_wheel for optunity: finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/1c/a1/79/8c2f85ed743981107a5b98afcc95bab74d5f85335ec70c7995
Running setup.py bdist_wheel for scikit-learn: started
Running setup.py bdist_wheel for scikit-learn: finished with status 'error'
Complete output from command /usr/local/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-jprwp998/scikit-learn/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-mhy2zrpk --python-tag cp36:
Partial import of sklearn during the build process.
Traceback (most recent call last):
File "/tmp/pip-install-jprwp998/scikit-learn/setup.py", line 168, in get_numpy_status
import numpy
ModuleNotFoundError: No module named 'numpy'
Traceback (most recent call last):
File "/tmp/pip-install-jprwp998/scikit-learn/setup.py", line 148, in get_scipy_status
import scipy
ModuleNotFoundError: No module named 'scipy'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-jprwp998/scikit-learn/setup.py", line 269, in <module>
setup_package()
File "/tmp/pip-install-jprwp998/scikit-learn/setup.py", line 249, in setup_package
.format(numpy_req_str, instructions))
ImportError: Numerical Python (NumPy) is not installed.
scikit-learn requires NumPy >= 1.8.2.
Installation instructions are available on the scikit-learn website: http://scikit-learn.org/stable/install.html
----------------------------------------
Failed building wheel for scikit-learn
Running setup.py clean for scikit-learn
and the docker file i am using to build this is
# Dockerfile for Python whisk docker action
FROM openwhisk/dockerskeleton
ENV FLASK_PROXY_PORT 8080
# lapack-dev is available in community repo.
RUN echo "http://dl-4.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories
# add package build dependencies
RUN apk add --no-cache \
g++ \
lapack-dev \
gfortran
# Install our action's Python dependencies
ADD requirements.txt /action/requirements.txt
RUN cd /action; pip install -r requirements.txt
# Add all source assets
ADD . /action
# Rename our executable Python action
ADD main.py /action/exec
CMD ["/bin/bash", "-c", "cd actionProxy && python -u actionproxy.py"]
requirement file is
numpy==1.14.3
elm
scipy
sklearn
i have searched alot of websites but nothing is helping, if anyone have any idea or experience with openwhisk docker python actions please any help would be appreciated.
I have checked it by just installing numpy and its working perfectly but somehow when i add scipy and numpy both to install it throws me error that numpy module not found.
Thanks
Try using the python runtime kind python-jessie:3 it already contains some ML related packages like numpy, scipy, pandas, and scikit-learn
For example using the CLI to create an action
ibmcloud fn action update mymlaction mymlaction.py --kind python-jessie:3
More details on the python runtime here https://console.bluemix.net/docs/openwhisk/openwhisk_reference.html#openwhisk_ref_python_environments

Python3: install in virtualenv fails

I tried to create Python3 virtualenv and test some Python code in it:
python3 -m venv tbzuploader-py3env
cd tbzuploader-py3env
. ./bin/activate
pip install -e git+https://github.com/guettli/tbzuploader.git#egg=tbzuploader
Here is the output I get:
Obtaining tbzuploader from git+https://github.com/guettli/tbzuploader.git#egg=tbzuploader
Cloning https://github.com/guettli/tbzuploader.git to ./src/tbzuploader
Collecting requests (from tbzuploader)
Using cached requests-2.18.4-py2.py3-none-any.whl
Collecting future (from tbzuploader)
Using cached future-0.16.0.tar.gz
Collecting urllib3<1.23,>=1.21.1 (from requests->tbzuploader)
Using cached urllib3-1.22-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2 (from requests->tbzuploader)
Using cached chardet-3.0.4-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests->tbzuploader)
Using cached certifi-2017.7.27.1-py2.py3-none-any.whl
Collecting idna<2.7,>=2.5 (from requests->tbzuploader)
Using cached idna-2.6-py2.py3-none-any.whl
Building wheels for collected packages: future
Running setup.py bdist_wheel for future ... error
Complete output from command /home/tguettler/projects/tbzuploader-py3env/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-4iiy_oby/future/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d /tmp/tmpukjvdwtmpip-wheel- --python-tag cp35:
usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: -c --help [cmd1 cmd2 ...]
or: -c --help-commands
or: -c cmd --help
error: invalid command 'bdist_wheel'
----------------------------------------
Failed building wheel for future
Running setup.py clean for future
Failed to build future
Installing collected packages: urllib3, chardet, certifi, idna, requests, future, tbzuploader
Running setup.py install for future ... done
Running setup.py develop for tbzuploader
Successfully installed certifi-2017.7.27.1 chardet-3.0.4 future-0.16.0 idna-2.6 requests-2.18.4 tbzuploader urllib3-1.22
You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
I use Python from Ubuntu 16.04:
===> python --version
Python 3.5.2
What am I doing wrong?
By default venv installed pip version 8. You should update it: pip install --upgrade pip
You need to update pip version by following command.
pip install --upgrade pip
When you install python3, pip3 gets installed. And if you don't have another python installation(like python2.7) then a link is created which points pip to pip3. pip generally points to the first installation.

Categories