We Keep Coding

Can't install pip and python & ansible using Dockerfile in CentOS

I am trying to install python and pip & Ansible using Dockerfile but I get this error
/bin/sh: 1: python: not found
The command '/bin/sh -c curl -O https://bootstrap.pypa.io/pip/2.7/get-pip.py && python get-pip.py && python -m pip install --upgrade "pip < 21.0" && pip install ansible --upgrade' returned a non-zero code: 127
ERROR: Service 'jenkins' failed to build : Build failed
Here is my Dockerfile:
FROM jenkins/jenkins
USER root
RUN curl -O https://bootstrap.pypa.io/pip/2.7/get-pip.py && \
python get-pip.py && \
python -m pip install --upgrade "pip < 21.0" && \
pip install ansible --upgrade
USER jenkins
Note: I used the same instructions on another Dockerfile and it went without errors. Here is the Dockerfile from CentOS image:
FROM centos:7
RUN yum update -y && \
yum -y install openssh-server && \
yum install -y passwd
RUN useradd remote_user && \
echo "password" | passwd remote_user --stdin && \
mkdir /home/remote_user/.ssh && \
chmod 700 /home/remote_user/.ssh
COPY remote-key.pub /home/remote_user/.ssh/authorized_keys
RUN chown remote_user:remote_user -R /home/remote_user && \
chmod 600 /home/remote_user/.ssh/authorized_keys
RUN /usr/sbin/sshd-keygen
RUN yum -y install mysql
RUN curl -O https://bootstrap.pypa.io/pip/2.7/get-pip.py && \
python get-pip.py && \
python -m pip install --upgrade "pip < 21.0" && \
pip install awscli --upgrade
CMD /usr/sbin/sshd -D

Since I'm not entirely sure my comments were fully understandable, here is how I would install ansible in your current base image jenkins/jenkins.
Notes:
I fixed the tag to lts since building from latest is a bit on the edge. You can change that to whatever tag suits your needs.
That base image is itself based on Ubuntu and not CentOS as reported in your title (hence using apt and not yum/dnf)
I used two RUN directives (one for installing python, the other for ansible) but you can merge them in a single instruction if you want to further limit the number of layers.
FROM jenkins/jenkins:lts
USER root
RUN apt-get update && \
apt-get install -y python3-pip && \
rm -rf /var/lib/apt/lists/*
RUN pip install --upgrade pip && \
pip install ansible && \
pip cache purge
USER jenkins

I deleted RUN instructions and replaced it with :
RUN apt-get update
RUN apt-get install -y ansible
Worked like a charm.

Make Docker container use newest version of Python installed

I have a couple of Python modules that I use inside my Docker container and they require a higher version of Python that what's being used. I install Python and install the modules using:
RUN apt-get update || : && apt-get install python3 -y
RUN apt-get install -y python3-pip
COPY requirements.txt /project
RUN pip3 install -r requirements.txt
Expecting I would be using the latest version of Python in my Docker container but when I go into it's shell and run python3 --version is comes as 3.4.2 which is incredibly old for my program. How do I make the default Python to be the latest I installed above without messing over the System-level python?
The image runtime I'm using for the Docker container is: node:9-slim

I don't think you can find a prebuilt python3.9 package on a debian 8 distribution as your environment is pretty old.
The only solution is you build the python3.9 out from source code in your base container. A full workable Dockerfile as next:
FROM node:9-slim
RUN apt update; \
apt install -y build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev; \
wget https://www.python.org/ftp/python/3.9.7/Python-3.9.7.tgz; \
tar -zxvf Python-3.9.7.tgz; \
cd Python-3.9.7; \
./configure --prefix=/usr/local/python3; \
make && make install; \
ln -sf /usr/local/python3/bin/python3.9 /usr/bin/python3; \
ln -sf /usr/local/python3/bin/pip3.9 /usr/bin/pip3
Verify it:
$ docker build -t myimage:1 .
$ docker run --rm -it myimage:1 python3 --version
Python 3.9.7
$ docker run --rm -it myimage:1 pip3 --version
pip 21.2.3 from /usr/local/python3/lib/python3.9/site-packages/pip (python 3.9)

Docker how to make python 3.8 as default

I'm trying to update an existing Dockerfile to switch from python3.5 to python3.8, previously it was creating a symlink for python3.5 and pip3 like this:
RUN ln -s /usr/bin/pip3 /usr/bin/pip
RUN ln -s /usr/bin/python3 /usr/bin/python
I've updated the Dockerfile to install python3.8 from deadsnakes:ppa
apt-get install python3-pip python3.8-dev python3.8-distutils python3.8-venv
if I remove python3-pip, it complains about gcc
C compiler or Python headers are not installed on this system. Try to run: sudo apt-get install gcc python3-dev
with these installations in place I'm trying to update existing symlink creation something like this:
RUN ln -s /usr/bin/pip3 /usr/local/lib/python3.8/dist-packages/pip
RUN ln -s /usr/bin/pip /usr/local/lib/python3.8/dist-packages/pip
RUN ln -s /usr/bin/python3.8 /usr/bin/python3
it fails, saying
ln: failed to create symbolic link '/usr/bin/python3': File exists
which I assume fails because python3 points to python3.6.
if I try: RUN ln -s /usr/bin/python3.8 /usr/bin/python it doesn't complain about symlink and image gets build successfully, but fails while installing requirements later (we use Makefile targets to install dependencies inside the container using pip and pip-sync):
ERROR: Cannot uninstall 'python-apt'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
which I assume because python-apt gets installed as part of the default python3.6 installation and python3.8 pip can't uninstall it.
PS: my Dockerfile image is based on Ubunut 18.04 which comes with python3.6 as default.
How can I properly switch Dockerfile / image from python3.5 to python3.8? so I can later use pip directly and it points to python3.8's pip

Replacing the system python in this way is usually not a good idea (as it can break operating-system-level programs which depend on those executables) -- I go over that a little bit in this video I made "why not global pip / virtualenv?"
A better way is to create a prefix and put that on the PATH earlier (this allows system executables to continue to work, but bare python / python3 / etc. will use your other executable)
in the case of deadsnakes which it seems like you're using, something like this should work:
FROM ubuntu:bionic
RUN : \
&& apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
software-properties-common \
&& add-apt-repository -y ppa:deadsnakes \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
python3.8-venv \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* \
&& :
RUN python3.8 -m venv /venv
ENV PATH=/venv/bin:$PATH
the ENV line is the key here, that puts the virtualenv on the beginning of the path
$ docker build -t test .
...
$ docker run --rm -ti test bash -c 'which python && python --version && which pip && pip --version'
/venv/bin/python
Python 3.8.5
/venv/bin/pip
pip 20.1.1 from /venv/lib/python3.8/site-packages/pip (python 3.8)
disclaimer: I'm the maintainer of deadsnakes

Why not just build a new image from ubuntu:18.04 with the desired config you need?
Like this:
FROM ubuntu:18.04
RUN apt update && apt install software-properties-common -y
RUN add-apt-repository ppa:deadsnakes/ppa && install python3.8 -y
RUN ln -s /usr/bin/pip3 /usr/bin/pip && \
ln -s /usr/bin/python3.8 /usr/bin/python

You can install and enable your python version.
# Python 3.8 and pip3
RUN apt-get update
RUN apt-get install -y software-properties-common
RUN add-apt-repository ppa:deadsnakes/ppa -y
RUN apt-get install -y python3.8
RUN ln -s /usr/bin/python3.8 /usr/bin/python
RUN apt-get install -y python3-pip

Sometimes, modifying the OS (like getting new Ubuntu clean os) is not favorable, because the current OS is too complicated. For example, my base OS is FROM ufoym/deepo:all-cu101.
So, to modify the existing python (3.6) to python 3.8, I added these 2 lines:
RUN apt-get update -qq && apt-get install -y -qq python3.8
RUN rm /usr/bin/python && rm /usr/bin/python3 && ln -s /usr/bin/python3.8 /usr/bin/python && ln -s /usr/bin/python3.8 /usr/bin/python3 \
&& rm /usr/local/bin/python && rm /usr/local/bin/python3 && ln -s /usr/bin/python3.8 /usr/local/bin/python && ln -s /usr/bin/python3.8 /usr/local/bin/python3 \
&& apt-get install -y python3-pip python-dev python3.8-dev && python3 -m pip install pip --upgrade
The first step is to install the python3.8;
The second step is to modify the softlink of python and python3 to point to python3.8
After that, install python3-pip, and update it to make sure the pip is using the current python 3.8 environment.

Opencv not being installed in Raspbian Stretch

I am installing opencv in Raspbian stretch using Raspberry Pi 3, following the tutorial given by Dr. Adrian Rosebrock as mentioned in the code section.
sudo apt-get update && sudo apt-get upgrade
sudo apt-get install build-essential cmake pkg-config
sudo apt-get install libjpeg-dev libtiff5-dev libjasper-dev libpng12-
dev
sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev
libv4l-dev
sudo apt-get install libxvidcore-dev libx264-dev
sudo apt-get install libgtk2.0-dev libgtk-3-dev
sudo apt-get install libatlas-base-dev gfortran
sudo apt-get install python2.7-dev python3-dev
cd ~
wget -O opencv.zip
https://github.com/Itseez/opencv/archive/3.3.0.zip
unzip opencv.zip
wget -O opencv_contrib.zip
https://github.com/Itseez/opencv_contrib/archive/3.3.0.zip
unzip opencv_contrib.zip
wget https://bootstrap.pypa.io/get-pip.py
sudo python get-pip.py
sudo python3 get-pip.py
sudo pip install virtualenv virtualenvwrapper
sudo rm -rf ~/.cache/pip
export WORKON_HOME=$HOME/.virtualenvs
export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3
source /usr/local/bin/virtualenvwrapper.sh
echo -e "\n# virtualenv and virtualenvwrapper" >> ~/.profile
echo "export WORKON_HOME=$HOME/.virtualenvs" >> ~/.profile
echo "export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3" >> ~/.profile
echo "source /usr/local/bin/virtualenvwrapper.sh" >> ~/.profile
mkvirtualenv cv -p python2
mkvirtualenv cv -p python3
pip install numpy
workon cv
cd ~/opencv-3.3.0/
mkdir build
cd build
cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D INSTALL_PYTHON_EXAMPLES=ON \
-D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib-3.3.0/modules \
-D BUILD_EXAMPLES=ON ..
CONF_SWAPSIZE=1024
sudo /etc/init.d/dphys-swapfile stop
sudo /etc/init.d/dphys-swapfile start
make -j4
sudo make install
sudo ldconfig
ls -l /usr/local/lib/python2.7/site-packages/
According to the tutorial I should find opencv + python findings after this list command but I am not finding it either in site-packages or in dist packages. Also when I use import cv or cv2 command in my program it says no module of this name found, suggesting that opencv is not installed. Considering that all the steps of the above code were executed without any error why is this happening. Kindly guide.

Building Python3.7.3 from source missing '_ctypes'

I'm trying to build Python-3.7.3 from source with ensurepip but i'm getting this error:
ModuleNotFoundError: No module named '_ctypes'
All of the answers online say that libffi-dev is needed but I have it installed and it still giving me this error.
root#4b6d672f1334:/Python-3.7.3# find / -name libffi.*
/usr/lib/pkgconfig/libffi.pc
/usr/lib/libffi.a
/usr/lib/libffi.so
/usr/lib/libffi.so.5.0.10
/usr/lib/libffi.so.5
/usr/share/info/libffi.info.gz
The build is in a container image from ubuntu:10.04.
It is that old on purpose because I'm using PyInstaller to compile the application and it needs to run on machines with an old glibc (2.11) and this image is the only one that I could find that have this old version.
I have done the same for Python-2.7.16 and it worked without any issues.
Update
Python-3.6.8 is working without any issues as well

I was able to find a solution here
The issue is probably with an old version of libffi-dev, the solution is to build and install libffi from source and then build Python3.7.3
Build libffi:
wget ftp://sourceware.org/pub/libffi/libffi-3.2.1.tar.gz
tar xzf libffi-3.2.1.tar.gz
cd libffi-3.2.1
./configure --disable-docs
make
make install
Build Python3.7.3:
wget https://www.python.org/ftp/python/3.7.2/Python-3.7.2.tgz
tar xzf Python-3.7.2.tgz &&
cd Python-3.7.2
export LD_LIBRARY_PATH=/usr/local/lib && \
export LD_RUN_PATH=/usr/local/lib && \
./configure --enable-optimizations --prefix=/usr/ --with-ensurepip=install --enable-shared LDFLAGS="-L/usr/local/lib" CPPFLAGS="-I /usr/local/lib/libffi-3.2.1/include"
make
make install

This is my solution on debian 6.0.60, according to Amir Rossert's solution, thanks a lot!
(1) Install libffi
tar zxf libffi-3.3.tar.gz
cd libffi-3.3
./configure
make
make install
(2) Install Python 3.8
tar zxf Python-3.8.5.tgz
cd Python-3.8.5
export LD_LIBRARY_PATH=/usr/local/lib && \
export LD_RUN_PATH=/usr/local/lib && \
./configure --prefix=/usr/local/python38 --with-openssl=/usr/local/openssl111 --enable-shared --enable-optimizations --with-system-ffi=/usr/local/lib/
make
make install
ln -s /usr/local/python38/bin/python3 /usr/local/bin/python3
ln -s /usr/local/python38/bin/pip3 /usr/local/bin/pip3
touch /etc/ld.so.conf/python38.conf
echo "/usr/local/python38/lib" > /etc/ld.so.conf/python38.conf
ldconfig
Ok, it works well.

The problem on Ubuntu 10.04 is the Cflags in libffi.pc from libffi-dev. The following fixes the issue without upgrading the libffi package :
$ sed -i 's/Cflags:.*/Cflags: -I${includedir}\/x86_64-linux-gnu/' /usr/lib/pkgconfig/libffi.pc
The following Dockerfile builds Python 3.9.7 on Ubuntu 10.04:
FROM ubuntu:10.04
ARG PYVER=3.9.7
# Change to old-releases
RUN sed -i 's/http:\/\/archive.ubuntu.com\//http:\/\/old-releases.ubuntu.com\//' /etc/apt/sources.list
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
libreadline-dev libssl-dev zlib1g-dev libffi-dev \
pkg-config build-essential
# Fix broken pkg-config for libffi (used by ctypes)
RUN sed -i 's/Cflags:.*/Cflags: -I${includedir}\/x86_64-linux-gnu/' /usr/lib/pkgconfig/libffi.pc
ADD https://www.python.org/ftp/python/$PYVER/Python-$PYVER.tgz /app/Python-$PYVER.tgz
RUN cd /app && tar zxvf Python-$PYVER.tgz && cd Python-$PYVER && \
./configure --prefix /opt/python3 && make && make install