I have a Keras model, written in Python, and I want to run it in a Docker container. The Python script outputs a set of CSV-files (which are the predictions). I have tested the Python script locally on my PC and everything looks fine. When I run my Docker container, after building it, I write the following in the terminal:
docker run -it username/file bash
After this, I run my prediction which creates some CSV-files. I can see that the file is there, but I don't know how to see the content of them
You can always use the vim editor to open files inside a linux container.
Also there is a csv.vim plugin which you can use
Also you can use the -v tag with docker command to mention your link your local OS directory with the container directory. Any file change added or removed will be visible in the OS directory then
Related
I've been experimenting with using pack to build a docker image. I'm quite happy with using a Dockerfile if that is the better approach.
I have a single python script that uses click to parse arguments. In a Dockerfile, I'd specify as the ENTRYPOINT something like ["python", "myscript.py"] and then I could run:
docker run MYCONTAINER myarg1 myarg2
and replace my script with the docker run.
I am attempting to get the same functionality using buildpacks and a Procfile. Here is what I have, roughly, in the profile:
web: python myscript.py
However, when I try running the resulting successfully built image using docker run (eg., docker run MYCONTAINER myarg1 myarg2), I get errors like:
python myscript.py: python myscript.py: command not found
Running the same docker run without the arguments works as expected and prints the help for my command-line script and exits.
What am I missing? My google-foo has failed me...
I'm using the Docker image jupyter/scipy-notebook (which includes lot of packages and launches and jupyter notebook with them).
Problem: When I want to use the notebook and, with the same packages, run files with the terminal., I can't, because I cant kill the notebook without killing the container.
How can I modify the image in order to delete the auto-run of the notebook?
The entrypoint and command for the image are defined here: https://github.com/jupyter/docker-stacks/blob/6c85e4b4/base-notebook/Dockerfile#L108-L109
ENTRYPOINT ["tini", "-g", "--"]
CMD ["start-notebook.sh"]
You can edit these to run the container without the notebook server. You can either define new ones in your own dockerfile, or you could overwrite them on the command line. eg:
docker run --it --rm --entrypoint=bash jupyter/scipy-notebook echo hi
echo hi is the command in this place. With this image specifically you'll need to overwrite both to prevent the notebook server from starting.
I have a python code for which I want to create a docker image. Now as per my understanding we need a Dockerfile and our python code code.py. Inside a Dockerfile we need to write:
FROM python:3
ADD code.py /
RUN pip3 install imapclient
CMD [ "python", "./code.py" ]
My first question is about this Dockerfile. First we have mentioned FROM python:3 because we want to use python3. Next we have added our code. In RUN we can write a dependency of our code. So for example if our code need python package imapclient we can mention it here so that it will be installed before docker file is build. But what if our code do not have any requirements.? Is this line RUN important. Can we exclude it when we don't need it.?
So now let's say we have finally created our docker image python-hello-world by using command docker build -t python-hello-world .. I can see it using command docker images -a. Now when I do docker ps, it is not listed there because the container is not running. Now to start it, I'll have to do docker run python-hello-world. This will start the code. But I want it to be running always in the background like a Linux service. How to do that.?
Is this line RUN important? Can we exclude it when we don't need it?
Yes if your code doesn't need the packages then you can exclude it.
But I want it to be running always in the background like a Linux service. How to do that?
If you want to run it as background then use below command.
docker run -d --restart=always python-hello-world
This will start container in background and will start automatically when system reboots.
How does one iteratively develop their app using Docker? I have only just started using it and my workflow is very slow, so I'm pretty sure I'm using it wrong.
I'm following along with a python machine learning course on Youtube, and so I am using Docker to work with python 3. I know I can use virtualenv or a VM, but I want to learn Docker as well so bear with me.
My root directory looks like so:
Dockerfile main.py*
My docker file:
FROM python
COPY . /src
RUN pip install quandl
RUN pip install pandas
CMD ["python", "/src/main.py"]
And the Python file:
#!/usr/bin/env python
import pandas as pd
import quandl
print("Hello world from main.py")
df = quandl.get("WIKI/GOOGL")
print("getting data frame for WIKI/GOOGL")
print(df.head())
My workflow has been:
Learn something new from the tutorial
Update python file
Build the docker image: docker build -t myapp .
Run the app: docker run my app python /src/main.py
Questions:
How can I speed this all up? For every change I want to try, I end up rebuilding. This causes pip to get dependencies each time which takes way too long.
Instead of editing a python file and running it, how might a get an interactive shell from the python version running in the container?
If I wanted my program to write out a file, how could I get this file back to my local system from the container after the program has finished?
Thanks for the help!
Edit:
I should add, this was the tutorial I was following in general to run some python code in Docker: https://www.civisanalytics.com/blog/using-docker-to-run-python/
Speeding up the rebuild process
The simplest thing you can do is reorder your Dockerfile.
FROM python
RUN pip install quandl
RUN pip install pandas
COPY . /src
CMD ["python", "/src/main.py"]
The reason this helps is that Docker will re-use the cached build for commands it has already run. Now when you rebuild after modifying your source code, it will re-use the build results for the pip commands, as they do not need to be run again. It will only run the COPY step.
Getting a python shell
You can exec a shell in the running container and run your python command.
docker exec -it <container-id> bash
python <...>
Or, you can run a container with just a shell, and skip running your app entirely (then, run it however you want).
docker run -it <image> bash
python <...>
Writing outside the container
Mount an external directory into the container. Then write to the mounted path.
docker run -v /local/path:/path <.. rest of command ..>
Then when you write in the container to /path/file, the file will show up outside the container at /local/path/file.
So, I'm in a bit of a particular situation and i'm trying to find a clean solution.
Currently we've got 18 different repos, all with python deployment utilities copy and pasted 18 times with venv... to me this is disgusting.
I'd like to bake those utilities into some kind of "tools" docker image, and just execute them wherever i need, instead of having to have each folder install all the dependencies 18 times.
/devtools/venv
/user-service/code
/data-service/code
/proxy-service/code
/admin-service/code
Ultimately I'd like to CD into user-service, and run a command similar to docker run tools version_update.py -- and have the docker image mount user-service's code and run the script against it.
How would I do this, and is there a better way i'm not seeing?
Why use docker?
I would recommend placing your scripts into a "tools" directory alongside your services (or wherever you see fit), then you can cd into one of your service directories and run python ../tools/version_update.py.
It would depend on your docker image, but here is the basic concept.
In your docker image, lets say we have a /code directory where we will mount the source code that we want to do work on, and a /tools directory with all of our scripts.
We can then mount what ever directory we want into the /code directory in the docker image and run what ever script we want. The working directory inside of the container would be set to /code and the path would also have /tools in it. So using your example, the docker run commands would look like this.
docker run -v /user-service/code:/code tools version_update.py
This would run the tools docker image, mount the local /user-service/code directory to /code directory in the container, and then run the version_update.py script on that code. and then exit.
The same image can be used for all the other projects as well, just change the mount point. (assuming they all have the same structure)
docker run -v /data-service/code:/code tools version_update.py
docker run -v /proxy-service/code:/code tools version_update.py
docker run -v /admin-service/code:/code tools version_update.py
And if you want to run a different tool, just change the command that you pass in.
docker run -v /user-service/code:/code tools other_command.py