I'm trying to build a docker image from inside a container using the Python docker SDK. The build command
client.build(dockerfile="my.Dockerfile", path=".", tag="my-tag")
fails with
OSError: Can not read file in context: /proc/1/mem
The issue was that docker cannot build from the container's root directory, which was implicit due to the build context path='.'. This can easily be fixed by using a working directory in the Dockerfile of the container performing the build operation, e.g.
FROM python:3.9-slim
RUN apt-get update -y
WORKDIR my-workdir <-- ADD TO FIX
COPY . .
CMD python -m my-script
I'm trying to create a Docker image from a Dockerfile, and while doing this, I encounter following error with the COPY steps:
failed to compute cache key: not found: not found when using relative paths, and
error from sender: Create file .......\Temp\empty-dir347165903\C:: The filename, directory name, or volume label syntax is incorrect when using absolute ones
The exact command I'm trying is COPY main.py ./
Important notes would be there is no .dockerignore file whatsoever, no container is set and both main.py and Dockerfile are located in the same directory
Here's what the Dockerfile itself looks like:
From public.ecr.aws/lambda/python:3.8
COPY requirements.txt .
RUN pip3 install -r requirements.txt
COPY main.py ./
RUN mkdir chrome
RUN curl -SL (chromedriver link here) > chromedriver.zip
RUN unzip chromedriver.zip -d chrome/
RUN rm chromedriver.zip
The command I'm running is docker build - < Dockerfile
This syntax is only valid if your build doesn't use the context. The docker build command expects one argument, and that's not the Dockerfile, rather it's the build context. Typically it's a directory, could be a remote git repo, or you can pass a tar file of the directory on stdin with the - syntax. There is an exception for passing a Dockerfile instead of the build context, but when this is done, you can't have any COPY or ADD steps that pull files from the build context. Instead, you almost certainly want:
docker build .
To perform the build using the current directory as your build context, which also contains the Dockerfile. And after that, you'll likely want to add a tag to your resulting image:
docker build -t your-image:latest .
(Thanks to to David for the pointer to the Dockerfile as input syntax.)
So I have a docker project, which is some kind of Python pytest that runs subprocess on an executable file as a blackbox test. I would like to build the container, and then run it each time by copying the executable file to a dedicated folder inside the container WORKDIR (e.g. exec/). But I am not sure how to do it.
Currently, I have to first include the executable in the folder then build the container.
The structure is currently like this:
my_test_repo
| |-- exec
| | |-- my_executable
| |-- tests
| | |-- test_the_executable.py
| |-- Dockerfile
I skipped over some other such as setup.
In the Dockerfile, I do the following:
FROM python:3.7.7-buster
WORKDIR /app
COPY . /app
RUN pip install --trusted-host pypi.python.org .
RUN pip install pytest
ENV NAME "Docker"
RUN pytest ./tests/ --executable=./exec/my_executable
For the last time, I setup a pytest fixture to accept the path of the executable.
I can run the test by building it:
docker build --tag testproject:1.0 .
How can I edit it so that the containers only consists of all the tests file. And it interacts with users so that I can cp my executable from my local dir to the container then run the test?
Many thanks.
What do you mean by edit container?
you can copy the executable files from local dir to the container using docker cp command. but only one file can be copied at a time.
docker cp path_of_executable/file_name docker_container_name:path_to_be_copy/
If I understand your question correctly, change your Dockerfile to look like this:
FROM python:3.7.7-buster
WORKDIR /app
COPY tests/ /app
RUN pip install --trusted-host pypi.python.org .
RUN pip install pytest
ENV NAME "Docker"
ENTRYPOINT ["pytest", "./tests/"]
CMD []
Then entrypoint will be executed at runtime (not buildtime) along with any arguments passed in (this is handled by CMD.)
You can run it like this (after building it like you indicated):
docker testproject --executable=<path to executable>
The documentation for ENTRYPOINT and CMD can be found here.
My docker-compose.yml file:
version: '3'
services:
dash:
build: ./docker
environment:
- COMPOSE_CONVERT_WINDOWS_PATHS=false
ports:
- "5000:5000"
volumes:
- c:/Users:/data
Dockerfile
FROM python:3
WORKDIR /data
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
COPY app.py ./
CMD [ "python", "./app.py" ]
doing a simple COPY command in Dockerfile is throwing this error when the file is in a folder (not same level as Dockerfile file)
My folder structure:
- docker
- Dockerfile
- requirements.txt
- app
- app.py
- docker-compose.yml
You got the error because the docker build context directory ./docker on your host does not contains app.py.
Make sure ./docker folder contains app.py file.
If you know the correct directory containing the build context and the app.py file then specify that directory as build context.
build: /path/to/build/context
More info about build context here.
To know what exactly is docker build context, check this.
Hope this helps.
Update:
After checking your folder structure it seems app/app.py is outside of ./docker directory which is your build context.
Bring the app directory inside docker folder and change copy command to COPY app/app.py ./. Also change CMD to CMD [ "python", "/data/app.py" ].
Using COPY and ADD, you can only use source files that are in the same folder as the Dockerfile, or in sub-folders:
COPY obeys the following rules:
The path must be inside the context of the build; you cannot COPY ../something /something, because the first step of a docker build
is to send the context directory (and subdirectories) to the docker
daemon.
(https://docs.docker.com/engine/reference/builder/#copy)
In your case, app.py is in a sibling folder of docker, which is the base directory of your build context. You'll need to move app.py somewhere within the docker folder. For example:
- docker
- Dockerfile
- requirements.txt
- app
- app.py
- docker-compose.yml
And adjust your Dockerfile:
WORKDIR /data
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
COPY app/app.py ./
CMD [ "python", "./app.py" ]
I want to create a docker image. This is my work directory:
Dockerfile.in test.json test.py
And this is my Dockerfile:
COPY ./test.json /home/test.json
COPY ./test.py /home/test.py
RUN python test.py
When i launch this command:
docker build -f Dockerfile.in -t 637268723/test:1.0 .
It gives me this error:
`Step 1/5 : COPY ./test.json /home/test.json
---> Using cache
---> 6774cd225d60
Step 2/5 : COPY ./test.py /home/test.py
COPY failed: stat /var/lib/docker/tmp/docker-builder428014112/test.py:
no such file or directory`
Can anyone help me?
You should put those files into the same directory with Dockerfile.
Check if there's a .dockerignore file, if so, add:
!mydir/test.json
!mydir/test.py
Q1: Check your .dockerignore file in build path, the files or dir you want to copy may be in the ignore file list!
Q2: The COPY directive is based on the context in which you are building the image, so be aware of any problems with the directory where you are currently building the image! See: https://docs.docker.com/engine/reference/builder/#copy
I had to use the following command to start the build:
docker build .
Removing ./ from source path should resolve your issue:
COPY test.json /home/test.json
COPY test.py /home/test.py
I was also facing the same, I moved my docker file to root of the project. then it worked
Make sure the context you build your image with is set correctly. You can set the context when building as an argument.
Example:
docker build -f ./Dockerfile .. where '..' is the context in this example.
In your case removing ./ should solve the issue. I had another case wherein I was using a directory from the parent directory and docker can only access files present below the directory where Dockerfile is present
so if I have a directory structure /root/dir and Dockerfile /root/dir/Dockerfile
I cannot copy do the following
COPY root/src /opt/src
In my case, it was the comment line that was messing up the COPY command
I removed the comment after the COPY command and placed it to a dedicated line above the command. Surprisingly it resolved the issue.
Faulty Dockerfile command
COPY qt-downloader . # https://github.com/engnr/qt-downloader -> contains the script to auto download qt for different architectures and versions
Working Dockerfile command
# https://github.com/engnr/qt-downloader -> contains the script to auto download qt for different architectures and versions
COPY qt-downloader .
Hope it helps someone.
This may help someone else facing similar issue.
Instead of putting the file floating in the same directory as the Dockerfile, create a dir and place the file to copy and then try.
COPY mydir/test.json /home/test.json
COPY mydir/test.json /home/test.json
Another potential cause is that docker will not follow symbolic links by default (i.e don't use ln -s).
The following structure in docker-compose.yaml will allow you to have the Dockerfile in a subfolder from the root:
version: '3'
services:
db:
image: postgres:11
environment:
- PGDATA=/var/lib/postgresql/data/pgdata
volumes:
- postgres-data:/var/lib/postgresql/data
ports:
- 127.0.0.1:5432:5432
**web:
build:
context: ".."
dockerfile: dockerfiles/Dockerfile**
command: ...
...
Then, in your Dockerfile, which is in the same directory as docker-compose.yaml, you can do the following:
ENV APP_HOME /home
RUN mkdir -p ${APP_HOME}
# Copy the file to the directory in the container
COPY test.json ${APP_HOME}/test.json
COPY test.py ${APP_HOME}/test.py
# Browse to that directory created above
WORKDIR ${APP_HOME}
You can then run docker-compose from the parent directory like:
docker-compose -f .\dockerfiles\docker-compose.yaml build --no-cache
In my case, I had to put all my project files into a subdirectory
app -|inside app directory we have the following
| package.js
| src
| assets
Dockerfile
Then I copied files in his way
COPY app ./
I had such error while trying to build a docker image and push to the container registry. Inside my docker file I tried to copy a jar file from target folder and try to execute it with java -jar command.
I was solving the issue by removing .jar file and target folder from .gitignore file.
When using the Docker compose files, publish, publishes to obj/Docker/Publish. When I copied my files there and pointed my Dockerfile to this directory (as generated), it works…
The way docker look for file is from the current directory
i.e. if your command is
COPY target/xyz.jar app.jar
ADD target/xyz.jar app.jar
The xyz jar should be in the current/target directory - here current is the place where you have your docker file.
So if you have docker in a different dir. its better bring to main project directory and have a straight path to the jar being added or copied to the image.
I had the same issue with a .tgz file .
It was just about the location of the file. Ensure the file is in the same directory of the Dockerfile.
Also ensure the .dockerignore file directory doesn't exclude the file regex pattern.
In my case the solution was to place file in a directory and copy whole directory content with one command, instead of copying a single file:
COPY --chown=1016:1016 myfiles /home/myapp/myfiles
Make sure your path names are the same (case sensitive), folder name /dist/inventory
COPY /Dist/Inventory ... -- was throwing the error
COPY /dist/inventory ... -- working smoothly
Using nodejs/express/javascript!
In my case I had multiple CMD ["npm" "run"...] on the same Dockerfile, where you can only have 1. Hence, the first CMD ["npm" "run" "build"] was not being run while the /build folder was not created. Therefore the cmd to copy the build folder COPY --from=build /usr/src/app/build ./build failed!
Change from a CMD to a RUN npm run build to fix the issue.
My Dockerfile:
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
# copy everything except content from .dockerignore
COPY . ./
#CMD ["npm", "run", "build"]
RUN npm run build
RUN ls -la | grep build
FROM node:lts-alpine3.17 as production
ARG NODE_ENV=production
ENV NODE_ENV=${NODE_ENV}
WORKDIR /usr/src/app
RUN pwd
COPY package*.json ./
RUN npm ci --only=production
COPY --from=build /usr/src/app/build ./build
CMD ["node", "build/index.js"]```
Here is the reason why it happens, i.e. your local directory in the host OS where you are running the docker should have the file, otherwise you get this error
One solution is to :
use RUN cp <src> <dst> instead of
COPY <src> <dst>
then run the command it works!
<plugin>
<groupId>io.fabric8</groupId>
<artifactId>docker-maven-plugin</artifactId>
<configuration>
<images>
<image>
<name>imagenam</name>
<alias>dockerfile</alias>
<build>
<!-- filter>#</filter-->
<dockerFileDir>dockerfile loaction</dockerFileDir>
<tags>
<tag>latest</tag>
<tag>0.0.1</tag>
</tags>
</build>
<run>
<ports>
<port>8080:8080</port>
</ports>
</run>
</image>
</images>
</configuration>
</plugin>