Python docker build up error - python

I got this error when i run command
sudo docker-compose up
Docker File:
FROM alpine
ARG AWS_RDS_USER
ARG AWS_RDS_PASSWORD
ARG AWS_RDS_HOST
ARG AWS_RDS_DATABASE
ARG LOCALE_SERVICE_URL
ARG CRYPTO_KEY
ENV APP_DIR=/app
ENV APP_ENV=production
ENV DATABASE_CONNECTION_STRING=mysql://${AWS_RDS_USER}:${AWS_RDS_PASSWORD}#${AWS_RDS_HOST}/${AWS_RDS_DATABASE}
ENV LOCALE_SERVICE_URL=$LOCALE_SERVICE_URL
ENV CRYPTO_KEY=$CRYPTO_KEY
COPY build/requirements.txt build/app.ini ${APP_DIR}/
COPY build/nginx.conf /etc/nginx/nginx.conf
COPY api ${APP_DIR}/api
RUN apk add --no-cache curl python pkgconfig python-dev openssl-dev libffi-dev musl-dev make gcc
RUN curl -sS https://bootstrap.pypa.io/get-pip.py | python
RUN apk update && \
apk add --virtual .build-deps autoconf gcc make g++ python-dev && \
apk add nginx uwsgi uwsgi-python py2-pip py-mysqldb && \
chown -R nginx:nginx ${APP_DIR} && \
chmod 777 /run/ -R && \
chmod 777 /root/ -R && \
pip2 install --upgrade pip && \
pip2 install -r ${APP_DIR}/requirements.txt && \
apk del .build-deps && \
rm -fR tmp/* && \
pw_migrate migrate --database=$DATABASE_CONNECTION_STRING --directory=$APP_DIR/api/migrations -v
EXPOSE 80
CMD nginx && uwsgi --ini ${APP_DIR}/app.ini
For solution, I tried to install below packages
1) gcc package.
2) libffi packages.
3) pip openssl packages.
But still error is not resolved. Any help should be appreciated

Try the solution suggested here
This is because you need a working compiler, the easiest way around
this is too install the build-base package like so:
apk add --no-cache --virtual .pynacl_deps build-base python3-dev
libffi-dev This will install various tools that are required to
compile pynacl and pip install pynacl will now succeed.
Note it is optional to use the --virtual flag but it makes it easy to
trim the image because you can run apk del .pynacl_deps later in your
dockerfile as they are not needed any more and would reduce the
overall size of the image.

Related

Docker Invalid Reference Format Error while building in bamboo

In my dockerfile, I was previously using FROM python:3.9-alpineon top of which librdkafka 1.9.2 is built and this was successful. But today, with the same docker file, the build failed by throwing the below error:
#error "confluent-kafka-python requires librdkafka v2.0.2 or later. Install the latest version of librdkafka from the Confluent repositories, see http://docs.confluent.io/current/installation.html".
When I searched in the internet, alpine:edge seems to have the newest version of librdkafka package. So I changed the dockerfile to FROM python:3.9-alpine:edge. But on building, this threw my an error:
Step 1/41 : FROM python:3.9-alpine:edge build 25-Jan-2023 10:25:20 invalid reference format build 25-Jan-2023 10:25:20 [?1h=[41m[37;1mAn error occurred when executing task '
I am new to docker and I used https://www.docker.com/blog/how-to-use-the-alpine-docker-official-image/ for the format. Please do help me regarding this.
This is my dockerfile currently:
FROM python:3.9-alpine:edge
RUN adduser -D pythonwebapi
WORKDIR /home/pythonwebapi
COPY requirements.txt requirements.txt
COPY logger_config.py logger_config.py
# COPY kong.ini kong.ini
# COPY iot.ini iot.ini
# COPY project.ini project.ini
# COPY eom.ini eom.ini
# COPY notify.ini notify.ini
RUN echo 'http://dl-3.alpinelinux.org/alpine/v3.9/main' >> /etc/apk/repositories
RUN apk update \
&& apk upgrade \
&& apk add --no-cache build-base \
autoconf \
bash \
bison \
boost-dev \
cmake \
flex \
zlib-dev
RUN apk add make gcc g++
RUN apk add libffi-dev
RUN apk update && apk --no-cache add librdkafka-dev
RUN apk add postgresql-dev gcc python3-dev musl-dev
RUN pip install --upgrade pip && pip install -r requirements.txt && pip install gunicorn
RUN apk del gcc g++ make
RUN pip install --no-cache-dir six pytest numpy cython
RUN pip install --no-cache-dir pandas
RUN pip install --no-cache-dir confluent-kafka
ARG ARROW_VERSION=3.0.0
ARG ARROW_SHA1=c1fed962cddfab1966a0e03461376ebb28cf17d3
ARG ARROW_BUILD_TYPE=release
ENV ARROW_HOME=/usr/local \
PARQUET_HOME=/usr/local
#Download and build apache-arrow
RUN mkdir -p /arrow \
&& wget -q https://github.com/apache/arrow/archive/apache-arrow-${ARROW_VERSION}.tar.gz -O /tmp/apache-arrow.tar.gz \
&& echo "${ARROW_SHA1} *apache-arrow.tar.gz" | sha1sum /tmp/apache-arrow.tar.gz \
&& tar -xvf /tmp/apache-arrow.tar.gz -C /arrow --strip-components 1 \
&& mkdir -p /arrow/cpp/build \
&& cd /arrow/cpp/build \
&& cmake -DCMAKE_BUILD_TYPE=$ARROW_BUILD_TYPE \
-DOPENSSL_ROOT_DIR=/usr/local/ssl \
-DCMAKE_INSTALL_LIBDIR=lib \
-DCMAKE_INSTALL_PREFIX=$ARROW_HOME \
-DARROW_WITH_BZ2=ON \
-DARROW_WITH_ZLIB=ON \
-DARROW_WITH_ZSTD=ON \
-DARROW_WITH_LZ4=ON \
-DARROW_WITH_SNAPPY=ON \
-DARROW_PARQUET=ON \
-DARROW_PYTHON=ON \
-DARROW_PLASMA=ON \
-DARROW_BUILD_TESTS=OFF \
.. \
&& make -j$(nproc) \
&& make install \
&& cd /arrow/python \
&& python setup.py build_ext --build-type=$ARROW_BUILD_TYPE --with-parquet \
&& python setup.py install \
&& rm -rf /arrow /tmp/apache-arrow.tar.gz
COPY app app
COPY init_app.py ./
ENV FLASK_APP init_app.py
RUN chown -R pythonwebapi:pythonwebapi ./
RUN chown -R 777 ./
USER pythonwebapi
EXPOSE 8000 7000
ENTRYPOINT ["gunicorn","--timeout", "7000","init_app:app","-k","uvicorn.workers.UvicornWorker","-b","0.0.0.0"]```

Error while building confluent-kafka in docker for alphine image

I am trying to build a python application which require confluent-kafka package. But while building in bamboo, I got the below error
fatal error: librdkafka/rdkafka.h: No such file or directory
build 13-Dec-2022 11:46:59 23 | #include <librdkafka/rdkafka.h>
build 13-Dec-2022 11:46:59 | ^~~~~~~~~~~~~~~~~~~~~~
My dockerfile is as such:
FROM python:3.9-alpine
RUN adduser -D pythonwebapi
WORKDIR /home/pythonwebapi
COPY requirements.txt requirements.txt
COPY logger_config.py logger_config.py
RUN echo 'http://dl-3.alpinelinux.org/alpine/v3.9/main' >> /etc/apk/repositories
RUN apk update \
&& apk upgrade \
&& apk add --no-cache build-base \
autoconf \
bash \
bison \
boost-dev \
cmake \
flex \
# libressl-dev \
zlib-dev
RUN apk add make gcc g++
RUN apk add libffi-dev
RUN apk update && apk --no-cache add librdkafka-dev
RUN apk add postgresql-dev gcc python3-dev musl-dev
RUN pip install --upgrade pip && pip install -r requirements.txt && pip install gunicorn
RUN apk del gcc g++ make
RUN pip install --no-cache-dir six pytest numpy cython
RUN pip install --no-cache-dir pandas
RUN pip install --no-cache-dir confluent-kafka
ARG ARROW_VERSION=3.0.0
ARG ARROW_SHA1=c1fed962cddfab1966a0e03461376ebb28cf17d3
ARG ARROW_BUILD_TYPE=release
ENV ARROW_HOME=/usr/local \
PARQUET_HOME=/usr/local
#Download and build apache-arrow
RUN mkdir -p /arrow \
&& wget -q https://github.com/apache/arrow/archive/apache-arrow-${ARROW_VERSION}.tar.gz -O /tmp/apache-arrow.tar.gz \
&& echo "${ARROW_SHA1} *apache-arrow.tar.gz" | sha1sum /tmp/apache-arrow.tar.gz \
&& tar -xvf /tmp/apache-arrow.tar.gz -C /arrow --strip-components 1 \
&& mkdir -p /arrow/cpp/build \
&& cd /arrow/cpp/build \
&& cmake -DCMAKE_BUILD_TYPE=$ARROW_BUILD_TYPE \
-DOPENSSL_ROOT_DIR=/usr/local/ssl \
-DCMAKE_INSTALL_LIBDIR=lib \
-DCMAKE_INSTALL_PREFIX=$ARROW_HOME \
-DARROW_WITH_BZ2=ON \
-DARROW_WITH_ZLIB=ON \
-DARROW_WITH_ZSTD=ON \
-DARROW_WITH_LZ4=ON \
-DARROW_WITH_SNAPPY=ON \
-DARROW_PARQUET=ON \
-DARROW_PYTHON=ON \
-DARROW_PLASMA=ON \
-DARROW_BUILD_TESTS=OFF \
.. \
&& make -j$(nproc) \
&& make install \
&& cd /arrow/python \
&& python setup.py build_ext --build-type=$ARROW_BUILD_TYPE --with-parquet \
&& python setup.py install \
&& rm -rf /arrow /tmp/apache-arrow.tar.gz
COPY app app
COPY init_app.py ./
ENV FLASK_APP init_app.py
RUN chown -R pythonwebapi:pythonwebapi ./
RUN chown -R 777 ./
USER pythonwebapi
EXPOSE 8000 7000
ENTRYPOINT ["gunicorn","--timeout", "7000","init_app:app","-k","uvicorn.workers.UvicornWorker","-b","0.0.0.0"]
I am unable to gauge why the error is coming since librdkafka is already installed.My requirement is to use alpine image. Can anyone please help me regarding this?

Build and install aerospike 4.0.0 in python 3.8 alpine 3.16

I am trying to install aerospike for python 3.8.14 on alpine 1.16.2.
Following offical documanetion I've already built aerospike-client-c 4.6.18 (release), successfully installed aerospike using pip.
Problem is that I got following error:
uv_async_send: symbol not found
ImportError: Error relocating /usr/local/lib/python3.8/site-packages/aerospike.cpython-38-x86_64-linux-gnu.so: uv_async_send: symbol not found
I've tried use ./install_libuv in aearospike-c-client but it didn't help.
I am not familair with all these building stuff, so maybe I am missing something.
Any ideas I can work out?
Command that failed:
python3 -c 'import aerospike'
My current Dockerfile:
ARG PYTHON_VERSION=3.8
FROM python:${PYTHON_VERSION}-alpine as aerospike-c-builder
ENV AS_C_VERSION=4.6.18
WORKDIR /src
RUN apk update
RUN apk add --no-cache \
build-base \
bash linux-headers \
libuv libuv-dev \
openssl openssl-dev \
lua5.1 lua5.1-dev zlib-dev
RUN wget https://artifacts.aerospike.com/aerospike-client-c/${AS_C_VERSION}/aerospike-client-c-src-${AS_C_VERSION}.zip \
&& unzip aerospike-client-c-src-${AS_C_VERSION}.zip \
&& mv aerospike-client-c-src-${AS_C_VERSION} aerospike-client-c \
&& cd aerospike-client-c \
&& make EVENT_LIB=libuv
# Stage 2: Build Aerospike Python client
FROM python:${PYTHON_VERSION}-alpine as aerospike-python-builder
ENV AS_PYTHON_VERSION=4.0.0
WORKDIR /src
COPY --from=aerospike-c-builder /src/aerospike-client-c/target/Linux-x86_64/ aerospike-client-c/
ENV PREFIX=/src/aerospike-client-c/
RUN apk update
RUN apk add --no-cache \
build-base \
bash \
linux-headers \
zlib-dev \
openssl-dev \
git
ENV DOWNLOAD_C_CLIENT 0
RUN pip --no-cache-dir install aerospike==${AS_PYTHON_VERSION}

ERROR: unsatisfiable constraints - Docker

When I install Python project, I get this error with Docker. I did not find a solution despite researching on the google. By the way When I run project on Docker, how to display on browser ?
test#test-VirtualBox:~/backend$ sudo docker build -t test .
Sending build context to Docker daemon 489kB
Step 1/11 : FROM python:3.6-alpine
---> 267db919e15e
Step 2/11 : RUN addgroup -S app && adduser -S -g app app
---> Using cache
---> ce1632a22469
Step 3/11 : WORKDIR /usr/src/app
---> Using cache
---> 01b36ea9b7c1
Step 4/11 : RUN apk --update --upgrade add --virtual deps gcc python3-dev linux-headers musl-dev alpine-sdk openssl-dev gmp-dev libffi-dev postgresql-dev && apk --update --upgrade add --no-cache libpq gmp
---> Running in d962dfc4a26a
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gz
ERROR: unsatisfiable constraints:
openssl-dev-1.0.2o-r2:
conflicts:
libressl-dev-2.7.4-r0[pc:libcrypto=1.0.2o]
libressl-dev-2.7.4-r0[pc:libssl=1.0.2o]
libressl-dev-2.7.4-r0[pc:openssl=1.0.2o]
libressl-dev-2.7.4-r0:
conflicts:
openssl-dev-1.0.2o-r2[pc:libcrypto=2.7.4]
openssl-dev-1.0.2o-r2[pc:libssl=2.7.4]
openssl-dev-1.0.2o-r2[pc:openssl=2.7.4]
satisfies:
postgresql-dev-10.5-r0[libressl-dev]
deps-0:
masked in: cache
satisfies: world[deps]
The command '/bin/sh -c apk --update --upgrade add --virtual deps gcc python3-dev linux-headers musl-dev alpine-sdk openssl-dev gmp-dev libffi-dev postgresql-dev && apk --update --upgrade add --no-cache libpq gmp' returned a non-zero code: 4
Dockerfile:
FROM python:3.6-alpine
RUN addgroup -S app && adduser -S -g app app
WORKDIR /usr/src/app
RUN apk --update --upgrade add --virtual deps \
gcc python3-dev linux-headers musl-dev \
alpine-sdk gmp-dev libffi-dev \
postgresql-dev && \
apk --update --upgrade add --no-cache libpq gmp
COPY ./requirements.txt requirements.txt
RUN pip install --no-cache-dir -r requirements.txt
RUN apk del deps
COPY . /usr/src/app
RUN chown -R app:app /usr/src/app
USER app
EXPOSE 8080
Why do you need to install openssl-dev package? Alpine packages are compiled with libressl. You can't install both openssl-dev and libressl-dev in container with alphine 3.6.
I've met the same problem. Our company build a base image based on python:2.7-alpine, but maybe it is too old. So don't install openssl-dev is not enough.
Look this :
satisfies:
postgresql-dev-10.5-r0[libressl-dev]
postgresql-dev rely on libressl-dev, so I delete openssl-dev firstly:
RUN apk del openssl-dev
Then everything is ok.

psycopg2 installation for python:2.7-alpine in Docker

To use PostgreSql in python I need to
pip install psycopg2
However, it has dependency on libpq-dev and python-dev. I wonder how can I install the dependencies in alpine? Thanks.
Here is a Dockerfile:
FROM python:2.7-alpine
RUN apk add python-dev libpq-dev
RUN pip install psycopg2
and the output is:
Step 3 : RUN apk add python-dev libpq-dev ---> Running in
3223b1bf7cde WARNING: Ignoring APKINDEX.167438ca.tar.gz: No such file
or directory WARNING: Ignoring APKINDEX.a2e6dac0.tar.gz: No such file
or directory ERROR: unsatisfiable constraints: libpq-dev (missing):
required by: world[libpq-dev] python-dev (missing):
required by: world[python-dev] ERROR: Service 'service' failed to build: The command '/bin/sh -c apk add python-dev libpq-dev' returned
a non-zero code: 2
If you only need to install psycopg2 for python 2.7 on Docker image based on python:2.7-alpine then following code for Dockerfile will be nice for you:
FROM python:2.7-alpine
RUN apk update && \
apk add --virtual build-deps gcc python-dev musl-dev && \
apk add postgresql-dev
RUN pip install psycopg2
An explanation before compile/install psycopg2
libpq is the client library for PostgreSQL
postgresql-dev are the package with the source headers to link libpq in a library/binary in a compilation, in this case when pip compiles psycopg .
I use the following configuration in alpine 3.7, I add some comments to explain it.
# Installing client libraries and any other package you need
RUN apk update && apk add libpq
# Installing build dependencies
# For python3 you need to add python3-dev *please upvote the comment
# of #its30 below if you use this*
RUN apk add --virtual .build-deps gcc python-dev musl-dev postgresql-dev
# Installing and build python module
RUN pip install psycopg2
# Delete build dependencies
RUN apk del .build-deps
Had problems with running Python 3.7 and PostgreSQL under Alpine Linux in Docker.
This article helped https://www.rockyourcode.com/install-psycopg2-binary-with-docker/
The main thing is to reference psypcopg2-binary in your requirements file and install the following packages (in Dockerfile):
RUN apk update && \
apk add --no-cache --virtual build-deps gcc python3-dev musl-dev && \
apk add postgresql-dev
I couldn't get it to install from python:2.7.13-alpine. Ended up with this:
FROM gliderlabs/alpine:3.3
RUN apk add --no-cache --update \
python \
python-dev \
py-pip \
build-base
RUN apk add --virtual build-deps gcc python-dev musl-dev && \
apk add --no-cache --update postgresql-dev && \
pip install psycopg2==2.7.1
Seems like the package you need is libpq not libpq-dev:
https://pkgs.alpinelinux.org/package/edge/main/x86/py2-psycopg2
Have a look at the dependencies at the right
add it in dockerfile
RUN apk update && apk add --no-cache --virtual .build-deps\
postgresql-dev gcc libpq python3-dev musl-dev linux-headers\
&& pip install --no-cache-dir -r requirements.txt\
&& apk del .build-deps\
&& rm -rf /var/cache/apk/*
What helped me was:
RUN apk add --no-cache python3 \
&& python3 -m ensurepip \
&& pip3 install --upgrade pip setuptools \
&& apk add build-base \
&& apk add gcc musl-dev libffi-dev openssl-dev python3-dev \
&& apk add postgresql-dev \
&& rm -r /usr/lib/python*/ensurepip && \
if [ ! -e /usr/bin/pip ]; then ln -s pip3 /usr/bin/pip ; fi && \
if [[ ! -e /usr/bin/python ]]; then ln -sf /usr/bin/python3 /usr/bin/python; fi && \
rm -r /root/.cache
RUN pip install --trusted-host pypi.python.org psycopg2
Especially apk add gcc musl-dev libffi-dev openssl-dev python3-dev as indicated in Docker: Installing python cryptography on alpine linux distribution
This may have to something with blockage of some countries by docker so you may sure you are using vpn

Categories