We Keep Coding

Docker :openjdk:8-jdk-slim is unable to find pip even after the installation

I was using the below docker file for quite sometime it was working well, But today when i tried to build the image it is throwing that pip is not installed, even though it is installed in the previous step
/bin/sh: 1: pip: not found
The command '/bin/sh -c pip install setuptools wheel' returned a non-zero code: 127
Below is the text of Dockerfile till the point where the error occurred:
FROM openjdk:8-jdk-slim
LABEL maintainer="abc#ab.com"
ARG RELEASE=2.13.2
ARG UID=1000
ARG GID=1000
RUN apt-get update
RUN apt-get install --no-install-recommends -y
RUN apt-get install tzdata -y
RUN apt-get install nano -y
RUN apt-get install curl -y
RUN apt-get install vim -y
RUN apt-get install python3 -y
RUN apt-get install python3-pip -y
RUN apt-get install unzip -y
RUN ln -s `which python3` /usr/bin/python
RUN pip3 install --upgrade pip
RUN pip install setuptools wheel
RUN pip install Flask flask-swagger-ui requests
RUN apt-get install --reinstall procps -y
I am not sure what happened suddenly , my attempt into looking for similar issues didn't yield a reason so for :
Docker issue: /bin/sh: pip: not found
Install pip in docker
Docker Build can't find pip

Docker build fails if pip is installed and updated to 10.0.1 in a single RUN section

How would you explain docker build failure with Dockerfile1, and it's success with Dockerfile2 (see below).
1)
// Dockerfile1
FROM ubuntu:16.04
RUN apt-get -y update && \
apt-get -y install python-pip python-dev build-essential && \
pip install --upgrade pip && \
pip install --upgrade virtualenv
docker build . fails with the following err
Collecting pip
Downloading
https://files.pythonhosted.org/packages/0f/74/ecd13431bcc456ed390b44c8a6e917c1820365cbebcb6a8974d1cd045ab4/pip-10.0.1-py2.py3-none-any.whl (1.3MB)
Installing collected packages: pip
Found existing installation: pip 8.1.1
Not uninstalling pip at /usr/lib/python2.7/dist-packages, outside
environment /usr
Successfully installed pip-10.0.1
Traceback (most recent call last):
File "/usr/bin/pip", line 9, in <module>
from pip import main
ImportError: cannot import name main
The command '/bin/sh -c apt-get -y update && apt-get -y install
python-pip python-dev build-essential && pip install --upgrade pip && pip install --upgrade virtualenv && virtualenv /venv' returned a non-zero code: 1
However, it succeeds if we split it into two RUN.
2)
// Dockerfile2
FROM ubuntu:16.04
RUN apt-get -y update && \
apt-get -y install python-pip python-dev build-essential && \
pip install --upgrade pip
RUN pip install --upgrade virtualenv
The installation failure for pip is related to this reported issue. So my questions:
Why does docker build fail in the first case? If we just run those command in bash, there wont be any error.
Why does docker build succeed in the second case? How is it related to layering concept in docker?
Why specifying pip version in Dockerfile1 (i.e. pip install --upgrade pip=0.9.3) solves the problem too?
Update (May 6, 2018):
I've figured out the issue. What happens here is as below:
apt-get -y install python-pip installs an old version of pip whose shim script import pip's main directly.
pip install --upgrade pip installs pip 10.0.1 and moves main into an internal directory _internal. It adds its shim script to PATH.
Calling pip fails as it still calls the old shim script as it's path is cached. Running hash -d pip in between fixes the issue.
So apparently, splitting install and update into two RUN sections has similar effect as hash -d pip. Workarounds (also suggested by Andriy Maletsky) are 1) pin pip update to 9.0.3, or 2) install (latest) pip from source in the first place, or 3) use hash -r in between, or 4) use another RUN command for later use of pip.

The problem is that pip executable (/usr/bin/pip) breaks while updating pip from version 9 to version 10.
Possible solutions:
1. Do not update and use pip v9
2. Do not use apt-get to install pip. Download it manually.
Why does docker build fail in the first case? If we just run those command in bash, there wont be any error.
No, there will be an error. I ran those commands inside docker run --rm -it ubuntu:16.04 bash and got it.
Why does docker build succeed in the second case? How is it related to layering concept in docker?
I believe you made a mistake somewhere in second RUN and it's silencing an error (in that place which you didn't provide). For example, this will work (because ; used instead of && and execution doesn't break after bad command):
RUN pip install --upgrade virtualenv && \
virtualenv /venv; source /venv/bin/activate
Why specifying pip version in Dockerfile1 (i.e. pip install --upgrade pip=0.9.3) solves the problem too?
Because this pip bug appeared in version 10.
P.S. You should not update or manually change files you added to your system via apt-get (you are doing this via pip install --upgrade pip).

pyconfig.h missing during "pip install cryptography"

I wanna set up scrapy cluster follow this link scrapy-cluster,Everything is ok before I run this command:
pip install -r requirements.txt
The requirements.txt looks like:
cffi==1.2.1
characteristic==14.3.0
ConcurrentLogHandler>=0.9.1
cryptography==0.9.1
...
I guess the above command means to install packages in requirements.txt.But I don't want it to specify the version,So I change it to this:
cat requirements.txt | while read line; do pip install ${line%%[>=]*} --user;done
When install cryptography,it gives me the error:
build/temp.linux-x86_64-2.7/_openssl.c:12:24:fatal error:pyconfig.h:No such file or directory
#include <pyconfig.h>
I don't know how to solved this , I have tried a lot of methods ,but failed. my system is centos 7, and the version of python is 2.7.5(default).
Besides, Is there any other scrapy frame which is appliable for a large number of urls . Thanks in advance

For Ubuntu, python2
apt-get install python-dev
For Ubuntu, python3
apt-get install python3-dev

I have solved it by myself. for the default python of centos, there is only a file named pyconfg-64.h in usr/include/python2.7/,So run the command
yum install python-devel
Then it works.

for python3.6,
apt-get install python3.6-dev
and
apt-get install libssl-dev libffi-dev

i use python 2 on ubuntu and got the same problem when installing cryptography.
after i run this command
apt-get install python-dev libssl-dev libffi-dev
then it works.

For Python 3.7 on Debian, the following works for me.
apt-get install python3.7-dev
and
apt-get install libssl-dev
You may also need:
apt-get install libffi-dev

On a Debian based distro (AntiX distro), together with apt-get install python3-dev, I also installed rust, to complete successfully the pip3 install cryptography command. So, I gave:
$ sudo apt-get install build-essential curl python3-dev libssl-dev libffi-dev
$ sudo curl https://sh.rustup.rs -sSf | sh
When prompted (Figure 1), type 1 and hit Enter on your keyboard.
Once it completes, you have to give following commands:
$ source $HOME/.cargo/env
$ source ~/.profile
$ pip3 install cryptography

Unable to locate package virtualenv in ubuntu-13 on a virtual-machine

When i try to run the command:
sudo apt-get install virtualenv
The error I get in response is:
E: Unable to locate package virtualenv

The Ubuntu package is called python-virtualenv, not "virtualenv".

Try This :
sudo apt-get install software-properties-common
sudo apt-add-repository universe
sudo apt-get update
sudo apt-get install virtualenv

It's also possible that you may not have run sudo apt-get update. It worked for me.

You need to add python before virtualenv because ubuntu package is
python-virtualenv not virtualenv.
sudo apt-get install python-virtualenv

you can install it with this instruction :
curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.10.1.tar.gz
tar xvfz virtualenv-1.10.1.tar.gz
cd virtualenv-1.10.1
sudo python setup.py install

sudo apt-get update
sudo apt-get install virtualenv

Yess, if any one wants to have a look at ( installing virtualenv: a basis for the installation of ) django installation, please run this command when the shell opens up in Ubuntu:
sudo apt-get install python-setuptools
sudo apt-get install python-easy_install virtualenv
and then run the commands to simply start the instructions followed on the following link(starting from the section "Setting up a new environment"):
http://www.django-rest-framework.org/tutorial/1-serialization/

How to check whether python package is installed or not in Docker?

I used Dockerfile successfully built a container. However, my code doesn't work in the container. It does work if I install all the packages manually. I'm assuming I messed up something that cause docker didn't install the packages properly. So, I want to check whether python package is installed or not in Docker container. What is the best way to check it?
The Dockerfile I used:
# Update the sources list
RUN sudo apt-get update
# Install basic applications
RUN sudo apt-get install -y tar git curl nano wget dialog net-tools build-essential
# First install ZeroMQ
RUN sudo apt-get install -y libzmq-dev
# Install libevent
RUN sudo apt-get install -y libevent-dev
# Install Python and Basic Python Tools
RUN sudo apt-get install -y python python-dev python-setuptools
RUN sudo apt-get install -y python-pip
# Add the current directory to the container
ADD . /root/code
# Get pip to download and install requirements:
RUN sudo pip install -r /root/code/requirements.txt
# Expose ports
EXPOSE 80 4242
# Define working directory.
WORKDIR /root/code
# Start the tcp server.
CMD python app.py
The requirements.txt I used:
gevent==1.0.1
greenlet==0.4.5
msgpack-python==0.4.2
pyzmq==13.1.0
wsgiref==0.1.2
zerorpc==0.4.4

I figured out.
docker exec <container ID> pip list