I am trying to dockerize this repo. After building it like so:
docker build -t layoutlm-v2 .
I try to run it like so:
docker run -d -p 5001:5000 layoutlm-v2
It downloads the necessary libraries and packages:
And then nothing... No errors, no endpoints generated, just radio silence.
What's wrong? And how do I fix it?
You appear to be expecting your application to offer a service on port 5000, but it doesn't appear as if that's how your code behaves.
Looking at your code, you seem to be launching a service using gradio. According the quickstart, calling gr.Interface(...).launch() will launch a service on localhost:7860, and indeed, if you inspect a container booted from your image, we see:
root#74cf8b2463ab:/app# ss -tln
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0 2048 127.0.0.1:7860 0.0.0.0:*
There's no way to access a service listening on localhost from outside the container, so we need to figure out how to fix that.
Looking at these docs, it looks like you can control the listen address using the server_name parameter:
server_name
to make app accessible on local network, set this to "0.0.0.0". Can be set by environment variable GRADIO_SERVER_NAME. If None, will use "127.0.0.1".
So if we run your image like this:
docker run -p 7860:7860 -e GRADIO_SERVER_NAME=0.0.0.0 layoutlm-v2
Then we should be able to access the interface on the host at http://localhost:7860/...and indeed, that seems to work:
Unrelated to your question:
You're setting up a virtual environment in your Dockerfile, but you're not using it, primarily because of a typo here:
ENV PATH="VIRTUAL_ENV/bin:$PATH"
You're missing a $ on $VIRTUAL_ENV.
You could optimize the order of operations in your Dockerfile. Right now, making a simple change to your Dockerfile (e.g, editing the CMD setting) will cause much of your image to be rebuilt. You could avoid that by restructuring the Dockerfile like this:
FROM python:3.9
# Install dependencies
RUN apt-get update && apt-get install -y tesseract-ocr
RUN pip install virtualenv && virtualenv venv -p python3
ENV VIRTUAL_ENV=/venv
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
WORKDIR /app
COPY requirements.txt ./
RUN pip install -r requirements.txt
RUN git clone https://github.com/facebookresearch/detectron2.git
RUN python -m pip install -e detectron2
COPY . /app
# Run the application:
CMD ["python", "-u", "app.py"]
Related
Actually, I have a little python server (using fastapi but it's not important) that start a program like that:
#app.put("/start_simulation/")
async def start_simulation():
try:
Process = subprocess.Popen("Aimsun_Next.exe")
except Exception as e:
raise HTTPException(status_code=500, detail="Simulation process failed")
I put my little server in a python docker like that:
FROM python:3.9
WORKDIR /code
COPY ./requirements.txt /code/requirements.txt
COPY ./app /code/app
RUN pip install --no-cache-dir --upgrade -r ./requirements.txt
WORKDIR /code/app
CMD ["uvicorn", "server_main:app", "--reload", "--proxy-headers", "--host", "0.0.0.0", "--port", "8000"]
and it seems to work fine!
But when the request "start_simulation" is called, it don't work because we are now in a docker.
PS: My "put" query doesn't look good but I shortened it to have a simple example
I would like my server in my docker have access to the path of my host machine to call the command "Aimsun_Next.exe". It's possible ?
In your dockerfile you should indicate that you want to expose the FastAPI port. Something like EXPOSE 8000. See documentation.
When you start the container you have to publish the port to localhost docker run -p 8000:8080.
It's possible to access a file on your local filesystem from your container, by "mounting" a volume. See documentation.
But I'm not sure you can launch it and if I wouldn't recommend it at all.
The idea behind containerization is that a container can run everywhere and is secure as it's isolated from whatever underlying OS.
Your container is an isolated environment. Furthermore the python3.9 image is based on a specific Linux version and hasn't to ensure any kind of compatibility with whatever OS is used on the host.
I m not able to launch new jupyter-notebook from my project.
Below my dockerfile.
FROM python:3.9.0
ARG WORK_DIR=/opt/dir1
RUN apt-get update && apt-get install cron -y && apt-get install -y default-jre
# Install python libraries
COPY requirements.txt /tmp/requirements.txt
RUN pip install --upgrade pip && pip install -r /tmp/requirements.txt
WORKDIR $WORK_DIR
EXPOSE 8888
# Copy etl code
# copy code on container under your workdir "/opt/dir1"
COPY . .
ENTRYPOINT ["sh", "-c"]
CMD ["jupyter-notebook --ip 0.0.0.0 --no-browser --allow-root]
VOLUME /home/data/dir1/
then in my terminal i did
#build
docker build -t my-python-app .
#run
docker run -it -p 8888:8888 my-python-app
#in container i did
jupyter notebook --ip 0.0.0.0 --no-browser --allow-root
I think that my VOLUME doesn't work because when i did modifications in file of container nothing happens in the host /home/data/dir1/.
Does anyone knows why and how to solve it?
You can use, docker run -it /bin/bash (image name)
and try to navigate to the folder you have set the volume to, and see if an error occurs and check permissions.
When using volumes, check on the host system you can access the folder. Afterwards check which user you are, Docker allows to parse your USER_ID and GROUP_ID to the container.
From there you can use the same user and group as you are on the host system. If you wanted to access the same folder on the host system, you can enter into permissions problems.
More information on this on the following webpage.
https://jtreminio.com/blog/running-docker-containers-as-current-host-user/
Maybe it's could be help you : try another way to start the volume inside container, for example add it within "run" command container creation. i've working with docker and i've never added volumes in this way (this doesn't mean you way is wrong)
Here two examples to working with volumens,I recommend you second link.
docker official docs , working with volumes example
I am running a python application that reads two paths from Windows env vars and proceeds to use the executables in those paths to do OCR on some documents. Since POPPLER, TESSERACT env vars are already set in Windows, this Python snippet works for me:
popplerPath = os.environ.get('POPPLER')
tesseractPath = os.environ.get('TESSERACT')
Now I am trying to dockerize the app, and, to my understanding, since my container will need access to those paths, I need to mount them using VOLUME during run. My dockerfile looks like this:
FROM python:3.7.7-slim
WORKDIR ./
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY documents/ .
COPY src/ ./src
CMD [ "python", "./src/run.py" ]
I build the image using:
docker build -t ocr .
And I try to run my container using:
docker run -v %POPPLER%:%POPPLER% -v %TESSERACT%:%TESSERACT% ocr
... but my app still gets a None value for these paths and can't use the executable files. Is my approach correct and beyond that, is it a good dev practice?
See the doc, the switch for environment variable is -e:
$ docker run -e MYVAR1 --env MYVAR2=foo --env-file ./env.list ubuntu bash
and in dockerfile, you can use
ENV FOO=/bar
If I understand your statement correctly, your paths are mounted in the container in the same path as the host. The only problem is your Python script, which expects the paths to be provided by the environment variable. This will not exist unless you pass on them from your host system to your container system.
Once you verified your mounted volume with -v is there correctly, you can try with
docker run -v %POPPLER%:%POPPLER% -v %TESSERACT%:%TESSERACT% --env POPPLER=%POPPLER% --env TESSERACT=%TESSERACT% ocr
or, if you always run this, you can consider to put them in your dockerfile to save some keystroke.
Any executable you call must be built into the image. Containers can't usually call executables on the host or in other containers. In the specific example you show, a Linux container can't run a Windows executable, even if you do use a bind mount to inject it into the container.
The "slim" python images are built on Debian GNU/Linux, and you need to use its APT tool to install these executable dependencies in your Dockerfile. (https://www.debian.org/distrib/packages has a search box to help you find the right package name; Ubuntu Linux also uses Debian packages.)
FROM python:3.7-slim
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive \
apt-get install -y \
popper-utils \
tesseract-ocr-all
COPY requirements.txt .
...
I'd suggest putting reasonable defaults in your code if these environment variables aren't set. The apt-get install command will put them in the system path inside the image.
popplerPath = os.environ.get('POPPLER', 'poppler')
tesseractPath = os.environ.get('TESSERACT', 'tesseract')
If you really need them as environment variables you could use the Dockerfile ENV directive
ENV POPPLER=poppler TESSERACT=tesseract
Environment variables from the host don't automatically get passed through to the container; you need a Dockerfile ENV or docker run -e option. Also remember that the container has an isolated filesystem (and Windows-syntax paths don't make sense in Linux containers) so these environment variables would need to be container paths, the second half of your proposed docker run -v option.
So I have a docker file, that I intend to build and push to Google Cloud Run and it looks like this:
# pull official base image
FROM python:3.7-slim
# set work directory
WORKDIR /usr/src/app
# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# install dependencies
RUN pip install --upgrade pip
COPY ./requirements.txt /usr/src/app/requirements.txt
RUN pip install -r requirements.txt
# copy project
COPY . /usr/src/app/
CMD python manage.py runserver 0.0.0.0:$PORT
The idea is that once I push it to Cloud Run, my Django project will run on 0.0.0.0:$PORT, where the value of the environment variable $PORT is set by Google Cloud Run automatically.
I tried to run a container of the following image locally to see if this works. I set $PORT to 80, and then when I run a container of the docker image I get the following:
"CommandError: "0.0.0.0:" is not a valid port number or address:port pair."
Looking at other answers, such as this.
I understand that 0.0.0.0 is a placeholder for the public IP address of a given machine. My question is, why doesn't do I get the "CommandError" when I run docker run [DockerImage] locally??
If there are any other questions, please let me know I will clarify it.
Edit:
I also want to point out that I am following this tutorial
Setting the environment variable on host machine doesn't set it inside the container.
The command python manage.py runserver 0.0.0.0:$PORT is being run inside the container where $PORT is not set and so it expands to python manage.py runserver 0.0.0.0:.
Try docker run -e PORT=$PORT <your_image> to pass set the PORT env variable inside the container with value from the host machine.
You have not set the PORT env variable in the Dockerfile and that's the reason its failing to resolve $PORT during CMD execution.
you can update your Dockerfile as follow (I have used 8080 as an example, update this to required port):
# pull official base image
FROM python:3.7-slim
# set work directory
WORKDIR /usr/src/app
# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
ENV PORT 8080
# install dependencies
RUN pip install --upgrade pip
COPY ./requirements.txt /usr/src/app/requirements.txt
RUN pip install -r requirements.txt
# copy project
COPY . /usr/src/app/
CMD python manage.py runserver 0.0.0.0:$PORT
Hope this helps
I am trying to run a Flask app using Docker.
Normally, to execute the Flask app, I run this inside of my Terminal:
python myapp.py images/*
I am unsure of how to convert that to Docker CMD syntax (or if I need to edit ENTRYPOINT).
Here is my docker file:
RUN apt-get update -y
RUN apt-get install -y python-pip python-dev build-essential hdf5-tools
COPY . ~/myapp/
WORKDIR ~/myapp/
RUN pip install -r requirements.txt
ENTRYPOINT ["python"]
CMD ["myapp.py"]
Inside of requirements.txt:
flask
numpy
h5py
tensorflow
keras
When I run the docker image:
person#person:~/Projects/$ docker run -d -p 5001:5000 myapp
19645b69b68284255940467ffe81adf0e32a8027f3a8d882b7c024a10e60de46
docker ps:
Up 24 seconds 0.0.0.0:5001->5000/tcp hardcore_edison
When I got to localhost:5001 I get no response.
Is it an issue with my CMD parameter?
EDIT:
New Dockerfile:
RUN apt-get update -y
RUN apt-get install -y python-pip python-dev build-essential hdf5-tools
COPY . ~/myapp/
WORKDIR ~/myapp/
EXPOSE 5000
RUN pip install -r requirements.txt
CMD ["python myapp.py images/*.jpg "]
With this new configuration, when I run:
docker run -d -p 5001:5000 myapp
I get:
docker: Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "exec: \"python myapp.py images/*.jpg \": stat python myapp.py images/*.jpg : no such file or directory": unknown.
When I run:
docker run -d -p 5001:5000 myapp python myapp.py images/*.jpg
I get the Docker image to run, but now when I go to localhost:5001, it complains that the connection was reset.
I'm glad you've already solved this issue. I put up this answer just for those who still have the same confusions like you do about ENTRYPOINT and CMD executives.
In a Dockerfile, ENTRYPOINT and CMD are two similar executives, but still have strong difference between them. The most important one(only seems to me) is that CMD could be overwritten but ENTRYPOINT not.
To explain this, I may offer you guys the command blow:
docker run -tid --name=container_name image_name [command]
As we can see, command is optional, and it(if exists) could overwrite CMD defined in Dockerfile.
Let's back to your issue. You may have two ways to achieve your purpose-->
ENTRYPOINT ["python"] and CMD ["/path/to/myapp.py", "/path/to/images/*.jpg"].
CMD python /path/to/myapp.py /path/to/images/*.jpg. This is mentioned by #David Maze above.
To understand the first one, you may take CMD as arguments for ENTRYPOINT.
A simple example below.
Dockerfile-->
FROM ubuntu:18.04
ENTRYPOINT ["cat"]
CMD ["/etc/hosts"]
Build image named test-cmd-show and start a container from it.
docker run test-cmd-show
This would show the content in /etc/hosts file. And go on...
docker run test-cmd-show /etc/resolv.conf
And this would show us the content of /etc/resolv.conf file. And go on ...
docker run test-cmd-show --help
This would show the help information for command cat.
Fantastic, right?
Somehow, we could do more research though this functionality.
Add a relevant question: What's the difference between CMD and ENTRYPOINT?
The important thing is that you need a shell to expand your command line, so I’d write
CMD python myapp.py images/*
When you just write CMD like this (without the not-really-JSON brackets and quotes) Docker will implicitly feed the command line through a shell for you.
(You also might consider changing your application to support taking a directory name as configuration in some form and “baking it in” to your application, if these images will be in a fixed place in the container filesystem.)
I would only set ENTRYPOINT when (a) you are setting it to a wrapper shell script that does some first-time setup and then exec "$#"; or (b) when you have a FROM scratch image with a static binary and you literally cannot do anything with the container besides run the one binary in it.
One issue I found was that the app wasn't accessible to Docker. I added this to app.run:
host='0.0.0.0'
According to this:
Deploying a minimal flask app in docker - server connection issues
Next, Docker panics when you add a directory to the CMD parameters.
So, I removed ENTRYPOINT and CMD and manually added the command to the Docker run:
docker docker run -d -p 5001:5000 myappdocker python myapp.py images/*.jpg