I am trying to install python and pip & Ansible using Dockerfile but I get this error
/bin/sh: 1: python: not found
The command '/bin/sh -c curl -O https://bootstrap.pypa.io/pip/2.7/get-pip.py && python get-pip.py && python -m pip install --upgrade "pip < 21.0" && pip install ansible --upgrade' returned a non-zero code: 127
ERROR: Service 'jenkins' failed to build : Build failed
Here is my Dockerfile:
FROM jenkins/jenkins
USER root
RUN curl -O https://bootstrap.pypa.io/pip/2.7/get-pip.py && \
python get-pip.py && \
python -m pip install --upgrade "pip < 21.0" && \
pip install ansible --upgrade
USER jenkins
Note: I used the same instructions on another Dockerfile and it went without errors. Here is the Dockerfile from CentOS image:
FROM centos:7
RUN yum update -y && \
yum -y install openssh-server && \
yum install -y passwd
RUN useradd remote_user && \
echo "password" | passwd remote_user --stdin && \
mkdir /home/remote_user/.ssh && \
chmod 700 /home/remote_user/.ssh
COPY remote-key.pub /home/remote_user/.ssh/authorized_keys
RUN chown remote_user:remote_user -R /home/remote_user && \
chmod 600 /home/remote_user/.ssh/authorized_keys
RUN /usr/sbin/sshd-keygen
RUN yum -y install mysql
RUN curl -O https://bootstrap.pypa.io/pip/2.7/get-pip.py && \
python get-pip.py && \
python -m pip install --upgrade "pip < 21.0" && \
pip install awscli --upgrade
CMD /usr/sbin/sshd -D
Since I'm not entirely sure my comments were fully understandable, here is how I would install ansible in your current base image jenkins/jenkins.
Notes:
I fixed the tag to lts since building from latest is a bit on the edge. You can change that to whatever tag suits your needs.
That base image is itself based on Ubuntu and not CentOS as reported in your title (hence using apt and not yum/dnf)
I used two RUN directives (one for installing python, the other for ansible) but you can merge them in a single instruction if you want to further limit the number of layers.
FROM jenkins/jenkins:lts
USER root
RUN apt-get update && \
apt-get install -y python3-pip && \
rm -rf /var/lib/apt/lists/*
RUN pip install --upgrade pip && \
pip install ansible && \
pip cache purge
USER jenkins
I deleted RUN instructions and replaced it with :
RUN apt-get update
RUN apt-get install -y ansible
Worked like a charm.
Related
I tried to alias python3 to python3.8 in the Dockerfile. But It doesn't work for me. I am using ubuntu:18.04.
Step 25/41 : RUN apt-get update && apt-get install -y python3.8
---> Using cache
---> 9fa81ca14a53
Step 26/41 : RUN alias python3="python3.8" && python3 --version
---> Running in d7232d3c8b8f
Python 3.6.9
As you can see the python3 is still 3.6.9. How can I fix this issue?
Thanks.
EDIT
Just attach my Dockerfile:
##################################################################################################################
# Build
#################################################################################################################
#FROM openjdk:8
FROM ubuntu:18.04
############## Linux and perl packages ###############
RUN apt-get update && \
apt-get install -y openjdk-8-jdk && \
apt-get install -y ant && \
apt-get clean && \
rm -rf /var/lib/apt/lists/* && \
rm -rf /var/cache/oracle-jdk8-installer && \
apt-get update -y && \
apt-get install curl groff python-gdbm -y;
# Fix certificate issues, found as of
# https://bugs.launchpad.net/ubuntu/+source/ca-certificates-java/+bug/983302
RUN apt-get update && \
apt-get install -y ca-certificates-java && \
apt-get clean && \
update-ca-certificates -f && \
rm -rf /var/lib/apt/lists/* && \
rm -rf /var/cache/oracle-jdk8-installer;
# Setup JAVA_HOME, this is useful for docker commandline
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/
RUN export JAVA_HOME
# install git
RUN apt-get update && \
apt-get install -y mysql-server && \
apt-get install -y uuid-runtime git jq python python-dev python-pip python-virtualenv libdbd-mysql-perl && \
rm -rf /var/lib/apt/lists/* && \
apt-get install perl && \
perl -MCPAN -e 'CPAN::Shell->install("Inline")' && \
perl -MCPAN -e 'CPAN::Shell->install("DBI")' && \
perl -MCPAN -e 'CPAN::Shell->install("List::MoreUtils")' && \
perl -MCPAN -e 'CPAN::Shell->install("Inline::Python")' && \
perl -MCPAN -e 'CPAN::Shell->install("LWP::Simple")' && \
perl -MCPAN -e 'CPAN::Shell->install("JSON")' && \
perl -MCPAN -e 'CPAN::Shell->install("LWP::Protocol::https")';
RUN apt-get update && \
apt-get install --yes cpanminus
RUN cpanm \
CPAN::Meta \
YAML \
DBI \
Digest::SHA \
Module::Build \
Test::Most \
Test::Weaken \
Test::Memory::Cycle \
Clone
# Install perl modules for network and SSL (and their dependencies)
RUN apt-get install --yes \
openssl \
libssl-dev \
liblwp-protocol-https-perl
RUN cpanm \
LWP \
LWP::Protocol::https
# New module for v1.2 annotation
RUN perl -MCPAN -e 'CPAN::Shell->install("Text::NSP::Measures::2D::Fisher::twotailed")'
#############################################
############## python packages ###############
# python packages
RUN pip install pymysql==0.10.1 awscli boto3 pandas docopt fastnumbers tqdm pygr
############## python3 packages ###############
# python3 packages
RUN apt-get update && \
apt-get install -y python3-pip && \
python3 -m pip install numpy && \
python3 -m pip install pandas && \
python3 -m pip install sqlalchemy && \
python3 -m pip install boto3 && \
python3 -m pip install pymysql && \
python3 -m pip install pymongo;
RUN python3 -m pip install pyfaidx
#############################################
#############################################
############# expose tcp ports
EXPOSE 3306/tcp
EXPOSE 80/tcp
EXPOSE 8080
############# RUN entrypoint.sh
# commented out for testing
ENTRYPOINT ["./entrypoint.sh"]
© 2022 GitHub, Inc.
Terms
When I install the package pyfaidx with default python3.6, it raises an error. I found that python3.8 can install it. Thus, I want to switch to python3.8 to install all py3 packages.
Bash alias that you define in your RUN statement will be available only in the current shell session. When the current RUN statement finishes executing, you exit the session, effectively forgetting any aliases you set up there.
See also: How can I set Bash aliases for docker containers in Dockerfile?
Another option is to use update-alternatives, e.g.,
# update-alternatives --install `which python3` python3 `which python3.8` 20
update-alternatives: using /usr/bin/python3.8 to provide /usr/bin/python3 (python3) in auto mode
# python3 --version
Python 3.8.0
This may interfere with other container packages that do require 3.6 which was default on Ubuntu 18.04 back in the day. Furthermore, pip's authors do not recommend using pip to install system-wide packages like that. In fact, newer pip versions will emit a warning when attempting to use pip globally along the lines of your Dockerfile.
Therefore a better course of action is using a virtualenv:
# apt install -y python3-venv python3.8-venv
...
# python3.8 -m venv /usr/local/venv
# /usr/local/venv/bin/pip install -U pip setuptools wheel
# /usr/local/venv/bin/pip install -U pyfaidx
... (etc)
You can also "enter" your virtualenv by activating it:
root#a1d0210118a8:/# source /usr/local/venv/bin/activate
(venv) root#a1d0210118a8:/# python -V
Python 3.8.0
See also: Use different Python version with virtualenv.
I have the following Dockerfile:
FROM ubuntu:latest
RUN apt-get update \
&& apt-get install -y python3-pip python3-dev \
&& cd /usr/local/bin \
&& ln -s /usr/bin/python3 python \
&& pip3 install --upgrade pip
# Setup the Python's configs
RUN pip install --upgrade pip && \
pip install --no-cache-dir matplotlib==3.0.2 pandas==0.23.4 numpy==1.16.3 && \
pip install --no-cache-dir pybase64 && \
pip install --no-cache-dir scipy && \
pip install --no-cache-dir dask[complete] && \
pip install --no-cache-dir dash==1.6.1 dash-core-components==1.5.1 dash-bootstrap-components==0.7.1 dash-html-components==1.0.2 dash-table==4.5.1 dash-daq==0.2.2 && \
pip install --no-cache-dir plotly && \
pip install --no-cache-dir adjustText && \
pip install --no-cache-dir networkx && \
pip install --no-cache-dir scikit-learn && \
pip install --no-cache-dir tzlocal
# Setup the R configs
RUN apt-get update
RUN apt-get install -y software-properties-common
RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E298A3A825C0D65DFD57CBB651716619E084DAB9
RUN add-apt-repository 'deb https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/'
RUN apt update
ENV DEBIAN_FRONTEND=noninteractive
RUN apt install -y r-base
RUN pip install rpy2==2.9.4
RUN apt-get -y install libxml2 libxml2-dev libcurl4-gnutls-dev libssl-dev
RUN echo "r <- getOption('repos'); r['CRAN'] <- 'https://cran.r-project.org'; options(repos = r);" > ~/.Rprofile
RUN Rscript -e "install.packages('BiocManager')"
RUN Rscript -e "BiocManager::install('ggplot2')"
RUN Rscript -e "BiocManager::install('DESeq2')"
RUN Rscript -e "BiocManager::install('RColorBrewer')"
RUN Rscript -e "BiocManager::install('ggrepel')"
RUN Rscript -e "BiocManager::install('factoextra')"
RUN Rscript -e "BiocManager::install('FactoMineR')"
RUN Rscript -e "BiocManager::install('apeglm')"
WORKDIR /
# Copy all the necessary files of the app to the container
COPY ./ ./
# Install the slider-input component
WORKDIR /slider_input
RUN pip install --no-cache-dir slider_input-0.0.1.tar.gz
WORKDIR /
EXPOSE 8050
# Launch the app
CMD ["python", "./app.py"]
It's used for running dash app that using R commands, and it works fine.
The problem is the size of the image.
I want to minimize the size of the image as minimal as possible, but everything I tried was unsuccessful because of the combination of python and R.
Do you have any idea how can I minimize this image, and provide the same functionality?
Use docker-slim to minimize and secure your docker images. docker-slim will profile your docker image and throw away what you don't need.
It has been used with Node.js, Python, Ruby, Java, Golang, Rust, Elixir and PHP (some app types) running on Ubuntu, Debian, CentOS, Alpine and even Distroless.
docker-slim is production ready, but consider testing your container before deploying it to production. Minify docker images by up to 30x while making it secure too!
A multi-stage build will allow you to omit the compiler toolchain, headers, etc.. from the final image, only including the resulting code.
A three-part tutorial for Python specifically starts here: https://pythonspeed.com/articles/smaller-python-docker-images/
And the generic Docker docs: https://docs.docker.com/develop/develop-images/multistage-build/
FROM webdevops/base:ubuntu-16.04
RUN apt-get update && apt-get -y upgrade && apt-get install -y --no-install- recommends \
apache2 \
openssh-client \
python3 \
python3-dev \
python3-venv \
python3-psycopg2 \
python3-pip \
pyflakes3 \
pylint3 \
pep8 \
pep257 \
postgresql-client \
libapache2-mod-wsgi-py3 \
&& apt-get clean \
&& rm -fr /var/lib/apt/lists/*
RUN mkdir /var/www/html/hotels-project
RUN cd /var/www/html/hotels-project/ \
&& python3 -m venv hotels-venv \
&& /bin/bash -c "source hotels-venv/bin/activate"
RUN pip install 'django<2.0'
RUN pip install requests
RUN pip install psycopg2
show message:
ERROR: Service 'apache-python' failed to build: The command '/bin/sh
-c pip install 'django<2.0'' returned a non-zero code: 127
You have two issues in your docker file.
Using pip instead of pip3
Activating virtualenv in one step and running commands in another step
In Dockerfile for every RUN step, you get a fresh terminal. So any source command you executed in previous RUN statement is no more active.
So your code should be something like this
RUN cd /var/www/html/hotels-project/ \
&& python3 -m venv hotels-venv \
&& /bin/bash -c "source hotels-venv/bin/activate" \
&& pip3 install -r requirements.txt
And requirements.txt should have below content
django<2.0
requests==X.XX
psycopg2==y.yy
That's how you should do it
I am currently building a docker image with the latest version of Django but during the build I get this:
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-9LueOB/Django/
You are using pip version 8.1.2, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
The command '/bin/sh -c INSTALL_PKGS="python-pip oracle-instantclient-tnsnames.ora afs_tools_standalone nss_wrapper openssh-clients cx_Oracle libmemcached-devel python-ldap mod_wsgi httpd MySQL-python wassh-ssm-cern wassh" && INSTALL_PKGS_BUILD="gcc python-devel zlib-devel" && yum install -y centos-release-scl && yum update -y && yum install -y --setopt=tsflags=nodocs --enablerepo=centosplus $INSTALL_PKGS && yum install -y $INSTALL_PKGS_BUILD && rpm -e --nodeps centos-logos && pip install --upgrade && pip install -r requirements.txt && rm requirements.txt && yum remove -y $INSTALL_PKGS_BUILD && yum clean all -y && rm -rf /var/cache/yum && mkdir /var/django/ && mkdir -p /var/django/home/ && mkdir -p /usr/local/bin/ && mkdir -p /run/httpd/ && chown -R root:root /run/httpd && chmod a+rw /run/httpd' returned a non-zero code: 1
On my requirements.txt I am using this:
Django==2.0
djangorestframework==3.2.1
pyapi-gitlab==7.8.4
pylibmc==1.5.0
django-logtail
python-gitlab
Any idea of why this is happening?
I believe it's because Django 2.0 doesn't support Python 2.x anymore, try to replace "python" by "python3" in your command.
EDIT: or run "pip3" instead of "pip" when you install Django>=2.0
as mentioned by Piinthesky here you'll find the relevant Django documentation
I'm trying to install awscli using pip (as per Amazon's recommendations) in a custom Docker image that comes FROM library/node:6.11.2. Here's a repro:
FROM library/node:6.11.2
RUN apt-get update && \
apt-get install -y \
python \
python-pip \
python-setuptools \
groff \
less \
&& pip --no-cache-dir install --upgrade awscli \
&& apt-get clean
CMD ["/bin/bash"]
However, with the above I'm met with:
no such option: --no-cache-dir
Presumably because I've got incorrect versions of Python and/or Pip?
I'm installing Python, Pip, and awscli in a similar way with FROM maven:3.5.0-jdk-8 and there it works just fine. I'm unsure what the relevant differences between the two images are.
Removing said option from my Dockerfile doesn't do me much good either, because then I'm met with a big pile of different errors, an excerpt here:
Installing collected packages: awscli, PyYAML, docutils, rsa, colorama, botocore, s3transfer, pyasn1, jmespath, python-dateutil, futures, six
Running setup.py install for PyYAML
checking if libyaml is compilable
### ABBREVIATED ###
ext/_yaml.c:4:20: fatal error: Python.h: No such file or directory
#include "Python.h"
^
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
### ABBREVIATED ###
Bottom line: how do you properly install awscli in library/node:6.x based images?
Adding python-dev as per this other answer works, but throws an alarming number of compiler warnings (errors?), so I went with a variation of #SergeyKoralev's answer, which needed some tweaking before it worked.
Here's the changes I needed to make this work:
Change to python3 and pip3 everywhere.
Add a statement to upgrade pip itself.
Separate the awscli install in a separate RUN command.
Here's a full repro that does seem to work:
FROM library/node:6.11.2
RUN apt-get update && \
apt-get install -y \
python3 \
python3-pip \
python3-setuptools \
groff \
less \
&& pip3 install --upgrade pip \
&& apt-get clean
RUN pip3 --no-cache-dir install --upgrade awscli
CMD ["/bin/bash"]
You can probably also keep the aws install in the same RUN layer if you add a shell command before the install that refreshes things after upgrading pip. Not sure how though.
All the answers are about aws-cli version 1, If you want version 2 try the below
FROM node:lts-stretch-slim
RUN apt-get update && \
apt-get install -y \
unzip \
curl \
&& apt-get clean \
&& curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" \
&& unzip awscliv2.zip \
&& ./aws/install \
&& rm -rf \
awscliv2.zip \
&& apt-get -y purge curl \
&& apt-get -y purge unzip
CMD ["/bin/bash"]
As you have correctly stated, pip installing on the docker image you are using is an older one not supporting --no-cache-dir. You can try updating that or you can also fix the second problem which is about missing python source headers. This can be fixed by installing python-dev package. Just add that to the list of packages installed in the Dockerfile:
FROM library/node:6.11.2
RUN apt-get update && \
apt-get install -y \
python \
python-dev \
python-pip \
python-setuptools \
groff \
less \
&& pip install --upgrade awscli \
&& apt-get clean
CMD ["/bin/bash"]
You can then run aws which should be on your path.
Your image is based on Debian Jessie, so you are installing Python 2.7. Try using Python 3.x:
apt-get install -y python3-pip
pip3 install awscli
Install AWS CLI in docker container using below command:
apt upgrade -y;apt update;apt install python3 python3-pip python3-setuptools -y; python3 -m pip --no-cache-dir install --upgrade awscli
To check the assumed role or AWS identity run below command:
aws sts get-caller-identity