I have a Dockerfile that starts with
FROM python:3.7-slim-buster
and I want to install node.js and npm in it. How can I install them in this image?
This should work:
FROM python:3.7-slim-buster
# setup dependencies
RUN apt-get update
RUN apt-get install xz-utils
RUN apt-get -y install curl
# Download latest nodejs binary
RUN curl https://nodejs.org/dist/v14.15.4/node-v14.15.4-linux-x64.tar.xz -O
# Extract & install
RUN tar -xf node-v14.15.4-linux-x64.tar.xz
RUN ln -s /node-v14.15.4-linux-x64/bin/node /usr/local/bin/node
RUN ln -s /node-v14.15.4-linux-x64/bin/npm /usr/local/bin/npm
RUN ln -s /node-v14.15.4-linux-x64/bin/npx /usr/local/bin/npx
To run node start it with docker run -it <containerName> /bin/bash
Then node, npm and npx are available
npm globally packages need to add links if you want to use it as command.
FROM python:3.7-slim-buster
ENV NODE_VERSION 14.15.4
#if build in `china`, debian mirrors, npm registry change to china source
ARG AREA=london
RUN set -ex \
&& if [ 'china' = "$AREA" ] ; then \
sed -i "s#http://deb.debian.org#https://mirrors.aliyun.com#g" /etc/apt/sources.list; \
fi \
&& apt-get update \
&& apt-get install -y git xz-utils curl \
# install node
&& curl "https://nodejs.org/dist/v$NODE_VERSION/node-v$NODE_VERSION-linux-x64.tar.xz" -O \
&& tar -xf "node-v$NODE_VERSION-linux-x64.tar.xz" \
&& ln -s "/node-v$NODE_VERSION-linux-x64/bin/node" /usr/local/bin/node \
&& ln -s "/node-v$NODE_VERSION-linux-x64/bin/npm" /usr/local/bin/npm \
&& ln -s "/node-v$NODE_VERSION-linux-x64/bin/npx" /usr/local/bin/npx \
# npm install bump, openapi-generator
&& if [ 'china' = "$AREA" ] ; then \
npm config set registry https://registry.npm.taobao.org/; \
fi \
&& npm install -g bump-cli#2.1.0 \
&& ln -s "/node-v$NODE_VERSION-linux-x64/bin/bump" /usr/local/bin/bump \
# clear
&& npm cache clean --force \
&& rm -rf /var/lib/apt/lists/* \
&& rm -f "/node-v$NODE_VERSION-linux-x64.tar.xz" \
&& apt-get clean \
&& apt-get autoremove
Just use the official installation for Debian from here :
RUN curl -fsSL https://deb.nodesource.com/setup_19.x | bash - &&\
apt-get install -y nodejs
Related
I'm trying to deploy a python scraper to docker which requires selenium geckodriver and firefox.
I'm encountering a docker error 100 while trying to build a docker image.
Here's the docker code
FROM scrapinghub/scrapinghub-stack-scrapy:1.3-py3
#RUN apt-get install -y apt-transport-https unzip
RUN apt-get install unzip
RUN printf "deb http://archive.debian.org/debian/ jessie main\ndeb-src http://archive.debian.org/debian/ jessie main\ndeb http://security.debian.org jessie/updates main\ndeb-src http://security.debian.org jessie/updates main" > /etc/apt/sources.list
#============================================
# Firefox and Geckodriver
#============================================
RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates curl firefox-esr \
&& rm -fr /var/lib/apt/lists/* \
&& curl -L https://github.com/mozilla/geckodriver/releases/download/v0.24.0/geckodriver-v0.24.0-linux64.tar.gz | tar xz -C /usr/local/bin \
&& apt-get purge -y ca-certificates curl
ENV TERM xterm
ENV SCRAPY_SETTINGS_MODULE <my_project_name>.settings
RUN mkdir -p /app
WORKDIR /app
COPY ./requirements.txt /app/requirements.txt
RUN pip install -r requirements.txt
COPY . /app
RUN python setup.py install
Here's the log that eventually comes up on my terminal
Error The command '/bin/sh -c apt-get update && apt-get install -y --no-install-recommends ca-certificates curl firefox-esr && rm -fr /var/lib/apt/lists/*
&& curl -L https://github.com/mozilla/geckodriver/releases/download/v0.24.0/geckodriver-v0.24.0-linux64.tar.gz | tar xz -C /usr/local/bin && apt-get purge -y ca-certificates curl'
returned a non-zero code: 100:
{'code': 100, 'message': "The command '/bin/sh -c apt-get update && apt-get install -y --no-install-recommends ca-certificates curl firefox-esr && rm -fr /var/lib/apt/lists/*
&& curl -L https://github.com/mozilla/geckodriver/releases/download/v0.24.0/geckodriver-v0.24.0-linux64.tar.gz | tar xz -C /usr/local/bin
&& apt-get purge -y ca-certificates curl' returned a non-zero code: 100"}
I had the same issue, when running docker build . directly it gave me the following log line pointing to the issue:
#9 23.59 W: GPG error: http://archive.debian.org jessie Release: The following signatures were invalid: KEYEXPIRED 1587841717
If I remove the line:
RUN printf "deb http://archive.debian.org/debian/ jessie main\ndeb-src http://archive.debian.org/debian/ jessie main\ndeb http://security.debian.org jessie/updates main\ndeb-src http://security.debian.org jessie/updates main" > /etc/apt/sources.list
Then it appears to be behave.
I need to change the Dockerfile below, coming from pytorch_geometric (a popular PyTorch package), from CUDA9.0 to CUDA10.0.
FROM ubuntu:16.04
RUN apt-get update && apt-get install -y --no-install-recommends apt-utils ca-certificates apt-transport-https gnupg-curl && \
rm -rf /var/lib/apt/lists/* && \
NVIDIA_GPGKEY_SUM=d1be581509378368edeec8c1eb2958702feedf3bc3d17011adbf24efacce4ab5 && \
NVIDIA_GPGKEY_FPR=ae09fe4bbd223a84b2ccfce3f60f4b3d7fa2af80 && \
apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/7fa2af80.pub && \
apt-key adv --export --no-emit-version -a $NVIDIA_GPGKEY_FPR | tail -n +5 > cudasign.pub && \
echo "$NVIDIA_GPGKEY_SUM cudasign.pub" | sha256sum -c --strict - && rm cudasign.pub && \
echo "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64 /" > /etc/apt/sources.list.d/cuda.list && \
echo "deb https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64 /" > /etc/apt/sources.list.d/nvidia-ml.list
ENV CUDA_VERSION 9.0.176
ENV NCCL_VERSION 2.4.2
ENV CUDA_PKG_VERSION 9-0=$CUDA_VERSION-1
ENV CUDNN_VERSION 7.4.2.24
RUN apt-get update && apt-get install -y --no-install-recommends \
cuda-cudart-$CUDA_PKG_VERSION && \
ln -s cuda-9.0 /usr/local/cuda && \
rm -rf /var/lib/apt/lists/*
RUN apt-get update && apt-get install -y --allow-unauthenticated --no-install-recommends \
cuda-libraries-$CUDA_PKG_VERSION \
libnccl2=$NCCL_VERSION-1+cuda9.0 && \
apt-mark hold libnccl2 && \
rm -rf /var/lib/apt/lists/*
RUN apt-get update && apt-get install -y --allow-unauthenticated --no-install-recommends \
cuda-libraries-dev-$CUDA_PKG_VERSION \
cuda-nvml-dev-$CUDA_PKG_VERSION \
cuda-minimal-build-$CUDA_PKG_VERSION \
cuda-command-line-tools-$CUDA_PKG_VERSION \
cuda-core-9-0=9.0.176.3-1 \
cuda-cublas-dev-9-0=9.0.176.4-1 \
libnccl-dev=$NCCL_VERSION-1+cuda9.0 && \
rm -rf /var/lib/apt/lists/*
ENV LIBRARY_PATH /usr/local/cuda/lib64/stubs
# NVIDIA docker 1.0.
LABEL com.nvidia.volumes.needed="nvidia_driver"
LABEL com.nvidia.cuda.version="${CUDA_VERSION}"
RUN echo "/usr/local/nvidia/lib" >> /etc/ld.so.conf.d/nvidia.conf && \
echo "/usr/local/nvidia/lib64" >> /etc/ld.so.conf.d/nvidia.conf
ENV PATH /usr/local/nvidia/bin:/usr/local/cuda/bin:${PATH}
ENV LD_LIBRARY_PATH /usr/local/nvidia/lib:/usr/local/nvidia/lib64
# NVIDIA container runtime.
ENV NVIDIA_VISIBLE_DEVICES all
ENV NVIDIA_DRIVER_CAPABILITIES compute,utility
ENV NVIDIA_REQUIRE_CUDA "cuda>=9.0"
# PyTorch (Geometric) installation
RUN rm /etc/apt/sources.list.d/cuda.list && \
rm /etc/apt/sources.list.d/nvidia-ml.list
RUN apt-get update && apt-get install -y \
curl \
ca-certificates \
vim \
sudo \
git \
bzip2 \
libx11-6 \
&& rm -rf /var/lib/apt/lists/*
# Create a working directory.
RUN mkdir /app
WORKDIR /app
# Create a non-root user and switch to it.
RUN adduser --disabled-password --gecos '' --shell /bin/bash user \
&& chown -R user:user /app
RUN echo "user ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/90-user
USER user
# All users can use /home/user as their home directory.
ENV HOME=/home/user
RUN chmod 777 /home/user
# Install Miniconda.
RUN curl -so ~/miniconda.sh https://repo.continuum.io/miniconda/Miniconda3-4.5.12-Linux-x86_64.sh \
&& chmod +x ~/miniconda.sh \
&& ~/miniconda.sh -b -p ~/miniconda \
&& rm ~/miniconda.sh
ENV PATH=/home/user/miniconda/bin:$PATH
ENV CONDA_AUTO_UPDATE_CONDA=false
# Create a Python 3.6 environment.
RUN /home/user/miniconda/bin/conda install conda-build \
&& /home/user/miniconda/bin/conda create -y --name py36 python=3.6.5 \
&& /home/user/miniconda/bin/conda clean -ya
ENV CONDA_DEFAULT_ENV=py36
ENV CONDA_PREFIX=/home/user/miniconda/envs/$CONDA_DEFAULT_ENV
ENV PATH=$CONDA_PREFIX/bin:$PATH
# CUDA 9.0-specific steps.
RUN conda install -y -c pytorch \
cuda90=1.0 \
magma-cuda90=2.4.0 \
"pytorch=1.1.0=py3.6_cuda9.0.176_cudnn7.5.1_0" \
torchvision=0.2.1 \
&& conda clean -ya
# Install HDF5 Python bindings.
RUN conda install -y h5py=2.8.0 \
&& conda clean -ya
RUN pip install h5py-cache==1.0
# Install TorchNet, a high-level framework for PyTorch.
RUN pip install torchnet==0.0.4
# Install Requests, a Python library for making HTTP requests.
RUN conda install -y requests=2.19.1 \
&& conda clean -ya
# Install Graphviz.
RUN conda install -y graphviz=2.38.0 \
&& conda clean -ya
RUN pip install graphviz==0.8.4
# Install OpenCV3 Python bindings.
RUN sudo apt-get update && sudo apt-get install -y --no-install-recommends \
libgtk2.0-0 \
libcanberra-gtk-module \
&& sudo rm -rf /var/lib/apt/lists/*
RUN conda install -y -c menpo opencv3=3.1.0 \
&& conda clean -ya
# Install PyTorch Geometric.
RUN CPATH=/usr/local/cuda/include:$CPATH \
&& LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH \
&& DYLD_LIBRARY_PATH=/usr/local/cuda/lib:$DYLD_LIBRARY_PATH
RUN pip install --verbose --no-cache-dir torch-scatter \
&& pip install --verbose --no-cache-dir torch-sparse \
&& pip install --verbose --no-cache-dir torch-cluster \
&& pip install --verbose --no-cache-dir torch-spline-conv \
&& pip install torch-geometric
# Set the default command to python3.
CMD ["python3"]
I've tried starting it with FROM pytorch/pytorch:1.1.0-cuda10.0-cudnn7.5-runtime and commenting everything up to # PyTorch (Geometric) installation and the section on # CUDA 9.0-specific steps. for
RUN conda install -c pytorch pytorch
RUN conda install -c fragcolor cuda10.0 && conda clean -ya
and commenting out
# Install Graphviz.
RUN conda install -y graphviz=2.38.0 \
&& conda clean -ya
RUN pip install graphviz==0.8.4
which didn't seem to work even with CUDA9.0
This makes the docker work and load, pytorch to be able to be imported and cuda to work as well. However, when I try to import torch_geometric I get ModuleNotFoundError: No module named 'torch_scatter.scatter_cuda'
Since the package is well-maintained (4.5k stars, mentioned in the pytorch website), it seems to me it's likely my fault and something general about how to adapt from CUDA9.0 to CUDA10.0.
I'd appreciate any advice on what I could be doing wrong or a way of changing it without removing so many lines from the original Dockerfile, which is probably what's causing the issue.
Have you ever try with nvidia/cuda Docker base image?
try
FROM nvidia/cuda:10.0-cudnn7-devel-ubuntu16.04
RUN apt-get update && apt-get install -y --no-install-recommends \
apt-utils \
python3.6 \
python-dev \
python-pip \
python-setuptools \
&& \
rm -rf /var/lib/apt/lists/* && \
apt-get update
RUN pip install --upgrade pip==9.0.3 && \
pip --no-cache-dir install --upgrade torch==1.1.0 && \
pip --no-cache-dir install --upgrade torchvision==0.3.0
The versions of the packages I wrote are stable to use with Dockerfile.
I checked it and it worked well without any collision.
I tried to install pybgpstream without sudo rights on a machine.
Python installation is system wide. I had to install everything under ~/.local/lib...
After installing libbgstream binary I wanted to install pybgpstream with
pip install --global-option build_ext --global-option '--include-dir=/home/USER/.local/include' --global-option '--library-dir=/home/USER/.local/lib' pybgpstream
After successfull installation I tried running some script with pybgpstream. Doesn't work:
ImportError: /home/USER/.local/lib/python2.7/site-packages/_pybgpstream.so: undefined symbol: _Py_FalseStruct
I added /home/USER/.local/lib/ to LD_PATH...
I don't know what's wrong...
There's a docker solution to work with pybgpstream. Here is the docker file
FROM python:3.8
RUN apt update \
&& apt install -y build-essential curl zlib1g-dev libbz2-dev libcurl4-openssl-dev librdkafka-dev python3-gi-cairo nano \
&& mkdir ~/src && cd ~/src/ && \
curl -LO https://research.wand.net.nz/software/wandio/wandio-4.2.3.tar.gz && \
tar zxf wandio-4.2.3.tar.gz && \
cd wandio-4.2.3/ && ./configure && make install && ldconfig && \
cd ~/src/ && \
curl -LO https://github.com/CAIDA/libbgpstream/releases/download/v2.1.0/libbgpstream-2.1.0.tar.gz && \
tar zxf libbgpstream-2.1.0.tar.gz && \
cd libbgpstream-2.1.0/ && ./configure && make check install && ldconfig && \
pip install pybgpstream && \
pip install ipython && \
pip install statsmodels && \
pip install matplotlib && \
pip install ipykernel && \
pip install pylint && \
pip install autopep8
CMD ["/bin/bash"]
You can also use this container every time and run bgp streams
FROM webdevops/base:ubuntu-16.04
RUN apt-get update && apt-get -y upgrade && apt-get install -y --no-install- recommends \
apache2 \
openssh-client \
python3 \
python3-dev \
python3-venv \
python3-psycopg2 \
python3-pip \
pyflakes3 \
pylint3 \
pep8 \
pep257 \
postgresql-client \
libapache2-mod-wsgi-py3 \
&& apt-get clean \
&& rm -fr /var/lib/apt/lists/*
RUN mkdir /var/www/html/hotels-project
RUN cd /var/www/html/hotels-project/ \
&& python3 -m venv hotels-venv \
&& /bin/bash -c "source hotels-venv/bin/activate"
RUN pip install 'django<2.0'
RUN pip install requests
RUN pip install psycopg2
show message:
ERROR: Service 'apache-python' failed to build: The command '/bin/sh
-c pip install 'django<2.0'' returned a non-zero code: 127
You have two issues in your docker file.
Using pip instead of pip3
Activating virtualenv in one step and running commands in another step
In Dockerfile for every RUN step, you get a fresh terminal. So any source command you executed in previous RUN statement is no more active.
So your code should be something like this
RUN cd /var/www/html/hotels-project/ \
&& python3 -m venv hotels-venv \
&& /bin/bash -c "source hotels-venv/bin/activate" \
&& pip3 install -r requirements.txt
And requirements.txt should have below content
django<2.0
requests==X.XX
psycopg2==y.yy
That's how you should do it
How do I install R version 3.4.0 in my docker image. I've installed python using:
RUN yum -y install https://centos6.iuscommunity.org/ius-release.rpm \
&& yum -y install python36u \
&& yum -y install python36u-devel \
&& yum -y install python36u-pip \
&& yum -y install python36u-tkinter.x86_64
Similarly I need to install R:
I've specified following things in file so far for R:
ENV R_BASE_VERSION 3.4.0
RUN Rscript -e 'install.packages("devtools",dependencies=TRUE)' \
&&Rscript -e 'install.packages("methods",dependencies=TRUE)' \
&&Rscript -e 'install.packages("jsonlite",dependencies=TRUE)' \
Please suggest .I'm new to docker
I think I need to do something like below:
ENV R_BASE_VERSION 3.4.1
## Now install R and littler, and create a link for littler in /usr/local/bin
## Also set a default CRAN repo, and make sure littler knows about it too
RUN apt-get update \
&& apt-get install -t unstable -y --no-install-recommends \
littler \
r-cran-littler \
r-base=${R_BASE_VERSION}* \
r-base-dev=${R_BASE_VERSION}* \
r-recommended=${R_BASE_VERSION}* \
&& echo 'options(repos = c(CRAN = "https://cran.rstudio.com/"), download.file.method = "libcurl")' >> /etc/R/Rprofile.site \
&& echo 'source("/etc/R/Rprofile.site")' >> /etc/littler.r \
&& ln -s /usr/share/doc/littler/examples/install.r /usr/local/bin/install.r \
&& ln -s /usr/share/doc/littler/examples/install2.r /usr/local/bin/install2.r \
&& ln -s /usr/share/doc/littler/examples/installGithub.r /usr/local/bin/installGithub.r \
&& ln -s /usr/share/doc/littler/examples/testInstalled.r /usr/local/bin/testInstalled.r \
&& install.r docopt \
&& rm -rf /tmp/downloaded_packages/ /tmp/*.rds \
&& rm -rf /var/lib/apt/lists/*
But I do not know what is this litter and all. I just need R to be installed and then i will install required packages as I have specified above.
Edits : First line in my docker file installs node4.
Here are two DockerFile to install Python, R and NodeJS
The first one installs Python 3.4.2, R 3.1.1 and nodejs 4.8.4:
From node:4
RUN apt-get update && apt-get remove -y python && apt-get install -y python3 r-base
RUN cp /usr/bin/python3 /usr/bin/python
This second one installs Python 3.5.3, R 3.4.1 and nodejs 4.8.4:
From r-base:3.4.1
RUN apt-get update && apt-get install -y python3 nodejs
RUN cp /usr/bin/python3 /usr/bin/python
Choose the one that best fits your needs.
If your public base image (the base image of your own image) is really node:4, then it is not yum based but apt-get based to manage packages.
Thus you shoud install R the following way:
RUN apt-get update && apt-get install -y r-base
You should use some R images, like
https://hub.docker.com/_/r-base/
or some oher image in this list
https://hub.docker.com/search/?isAutomated=0&isOfficial=0&page=1&pullCount=0&q=R&starCount=0