My Docker knowledge is very poor, I have Docker installed only because I would use freqtrade, so I followed this simple HOWTO
https://www.freqtrade.io/en/stable/docker_quickstart/
Now , all freqtrade commands run using docker , for example this
D:\ft_userdata\user_data>docker-compose run --rm freqtrade backtesting --config user_data/cryptofrog.config.json --datadir user_data/data/binance --export trades --stake-amount 70 --strategy CryptoFrog -i 5m
Well , I started to have problems when I would had try this strategy
https://github.com/froggleston/cryptofrog-strategies
for freqtrade . This strategy requires Python module finta .
I understood that the Python module finta should be installed in my Docker container
and NOT in my Windows system (it was easy "pip install finta" from console!).
Even if I tried to find a solution over stackoverflow and google, I do no understand how to do this step (install finta python module in freqtrade container).
after several hours I am really lost.
Someone can explain me in easy steps how to do this ?
Freqtrade mount point is
D:\ft_userdata\user_data
You can get bash from your container with this command:
docker-compose exec freqtrade bash
and then:
pip install finta
OR
run only one command:
docker-compose exec freqtrade pip install finta
If the above solutions didn't work, You can run docker ps command and get container id of your container. Then
docker exec -it CONTAINER_ID bash
pip install finta
You need to make your own docker image that has finta installed. Luckily you can build on top of the standard freqtrade docker image.
First make a Dockerfile with these two lines in it
FROM freqtradeorg/freqtrade:stable
RUN pip install finta
Then build the image (calling the new image myfreqtrade) by running the command
docker build -t myfreqtrade .
Finally change the docker-compose.yml file to run your image by changing the line
image: freqtradeorg/freqtrade:stable
to
image: myfreqtrade
And that should be that.
The way to get our Python code running in a container is to pack it as a Docker image and then run a container based on it.
To generate a Docker image we need to create a Dockerfile that contains instructions needed to build the image. The Dockerfile is then processed by the Docker builder which generates the Docker image. Then, with a simple docker run command, we create and run a container with the Python service.
An example of a Dockerfile containing instructions for assembling a Docker image for Python service installing finta is the following
# set base image (host OS)
FROM python:3.8
# install dependencies
RUN pip install finta
# command to run on container start
CMD [ "python", "-V" ]
For each instruction or command from the Dockerfile, the Docker builder generates an image layer and stacks it upon the previous ones. Therefore, the Docker image resulting from the process is simply a read-only stack of different layers.
docker build -t myimage .
Then, we can check the image is in the local image store:
docker images
Please refer to the freqtrade DockerFile https://github.com/freqtrade/freqtrade/blob/develop/Dockerfile
Related
I have crated a chat-bot using python 3.6 and TensorFlow 1.15. And created the Command line utility for testing in local environment.
The command line utility works fine without docker as shown in the below image.
The problem arrived when i dockerized or containerized the application with dependencies.
The Command line utility is automatically closing after running the docker image.
The dockerfile for the application as below.
FROM python:3.6-buster
WORKDIR /usr/app
COPY ./req.txt ./
RUN pip install -r req.txt
COPY ./ ./
RUN python -m nltk.downloader punkt
CMD ["python","botui.py"]
And after running the docker image it automatically shuts down the Command line utility.
Please help me in finding the solution.
should i need to add something in dockerfile ?
Your dockerfile seems fine.
for the interactive mode for your chatbot conversation you need to add "-i" flag in your docker run command.
docker run -i <image_name>
I've a RHEL host with docker installed, it has default Py 2.7. My python scripts needs a bit more modules which
I can't install due to lack of sudo access & moreover, I dont want to screw up with the default Py which is needed for host to function.
Now, I am trying to get a python in docker container where I get to add few modules do the needfull.
Issue - docker installed RHEL is not connected to internet and cant be connected as well
The laptop i have doesnt have the docker either and I can't install docker here (no admin acccess) to create the docker image and copy them to RHEL host
I was hoping if docker image with python can be downloaded from Internet I might be able to use that as is!,
Any pointers in any approprite direction would be appreciated.
what have I done - tried searching for the python images, been through the dockers documentation to create the image.
Apologies if the above question sounds silly, I am getting better with time on docker :)
If your environment is restricted enough that you can't use sudo to install packages, you won't be able to use Docker: if you can run any docker run command at all you can trivially get unrestricted root access on the host.
My python scripts needs a bit more modules which I can't install due to lack of sudo access & moreover, I dont want to screw up with the default Py which is needed for host to function.
That sounds like a perfect use for a virtual environment: it gives you an isolated local package tree that you can install into as an unprivileged user and doesn't interfere with the system Python. For Python 2 you need a separate tool for it, with a couple of steps to install:
export PYTHONUSERBASE=$HOME
pip install --user virtualenv
~/bin/virtualenv vpy
. vpy/bin/activate
pip install ... # installs into vpy/lib/python2.7/site-packages
you can create a docker image on any standalone machine and push the final required image to docker registry ( docker hub ). Then in your laptop you can pull that image and start working :)
Below are some key commands that will be required for the same.
To create a image, you will need to create a Dockerfile with all the packages installed
Or you can also do sudo docker run -it ubuntu:16.04 then install python and other packages as required.
then sudo docker commit container_id name
sudo docker tag SOURCE_IMAGE[:TAG] TARGET_IMAGE[:TAG]
sudo docker push IMAGE_NAME
Then you pull this image in your laptop and start working.
You can refer to this link for more docker commands https://github.com/akasranjan005/docker-k8s/blob/master/docker/basic-commands.md
Hope this helps. Thanks
I have a python code for which I want to create a docker image. Now as per my understanding we need a Dockerfile and our python code code.py. Inside a Dockerfile we need to write:
FROM python:3
ADD code.py /
RUN pip3 install imapclient
CMD [ "python", "./code.py" ]
My first question is about this Dockerfile. First we have mentioned FROM python:3 because we want to use python3. Next we have added our code. In RUN we can write a dependency of our code. So for example if our code need python package imapclient we can mention it here so that it will be installed before docker file is build. But what if our code do not have any requirements.? Is this line RUN important. Can we exclude it when we don't need it.?
So now let's say we have finally created our docker image python-hello-world by using command docker build -t python-hello-world .. I can see it using command docker images -a. Now when I do docker ps, it is not listed there because the container is not running. Now to start it, I'll have to do docker run python-hello-world. This will start the code. But I want it to be running always in the background like a Linux service. How to do that.?
Is this line RUN important? Can we exclude it when we don't need it?
Yes if your code doesn't need the packages then you can exclude it.
But I want it to be running always in the background like a Linux service. How to do that?
If you want to run it as background then use below command.
docker run -d --restart=always python-hello-world
This will start container in background and will start automatically when system reboots.
How does one iteratively develop their app using Docker? I have only just started using it and my workflow is very slow, so I'm pretty sure I'm using it wrong.
I'm following along with a python machine learning course on Youtube, and so I am using Docker to work with python 3. I know I can use virtualenv or a VM, but I want to learn Docker as well so bear with me.
My root directory looks like so:
Dockerfile main.py*
My docker file:
FROM python
COPY . /src
RUN pip install quandl
RUN pip install pandas
CMD ["python", "/src/main.py"]
And the Python file:
#!/usr/bin/env python
import pandas as pd
import quandl
print("Hello world from main.py")
df = quandl.get("WIKI/GOOGL")
print("getting data frame for WIKI/GOOGL")
print(df.head())
My workflow has been:
Learn something new from the tutorial
Update python file
Build the docker image: docker build -t myapp .
Run the app: docker run my app python /src/main.py
Questions:
How can I speed this all up? For every change I want to try, I end up rebuilding. This causes pip to get dependencies each time which takes way too long.
Instead of editing a python file and running it, how might a get an interactive shell from the python version running in the container?
If I wanted my program to write out a file, how could I get this file back to my local system from the container after the program has finished?
Thanks for the help!
Edit:
I should add, this was the tutorial I was following in general to run some python code in Docker: https://www.civisanalytics.com/blog/using-docker-to-run-python/
Speeding up the rebuild process
The simplest thing you can do is reorder your Dockerfile.
FROM python
RUN pip install quandl
RUN pip install pandas
COPY . /src
CMD ["python", "/src/main.py"]
The reason this helps is that Docker will re-use the cached build for commands it has already run. Now when you rebuild after modifying your source code, it will re-use the build results for the pip commands, as they do not need to be run again. It will only run the COPY step.
Getting a python shell
You can exec a shell in the running container and run your python command.
docker exec -it <container-id> bash
python <...>
Or, you can run a container with just a shell, and skip running your app entirely (then, run it however you want).
docker run -it <image> bash
python <...>
Writing outside the container
Mount an external directory into the container. Then write to the mounted path.
docker run -v /local/path:/path <.. rest of command ..>
Then when you write in the container to /path/file, the file will show up outside the container at /local/path/file.
I am using Windows and learning to use tensorflow, so I need to run it under Docker (Toolbox).
Following the usual instruction:
$ docker run -it gcr.io/tensorflow/tensorflow
I can launch a Jupyter notebook on my browser at 192.168.99.100:8888 and run the tutorial notebooks without problems.
Now when I try to import pandas as pd , which is installed in my computer with pip, on Juypter it just said ImportError: No module named pandas
Any idea how I can get this library to work inside the tensorflow images launched from docker?
Screenshot
The Docker image should be built on a linux operating system. You should launch a shell inside the Docker image grc.io/tensorflow/tensorflow to install the requisite python dependencies.
See Docker quickstart for using
docker run -it grc.io/tensorflow/tensorflow /bin/bash
and then
sudo apt-get install python-pandas
according to pandas docs.
To avoid doing this every time you launch the image, you need to commit the change to create a new image.
To commit the change, you need to get the container id (after run and installation steps above):
sudo docker ps –a # Get list of all containers previously started with run command
Then, commit your changes git style using the container_id displayed in the container list you just got and giving it an image_name of your choosing:
sudo docker commit container_id image_name
The new image will now show up in the list displayed by sudo docker ps –a.
If you get a free docker account you can push and pull your updated image to your docker repo, or just keep it locally.
See docs under 'Updating and Committing your image'.
For windows users:
docker run -d -p 8888:8888 -v /c/Users/YOUR_WIN_FOLDER:/home/ds/notebooks gcr.io/tensorflow/tensorflow
Then use the following command to see the name of your container for easy execution commands later on (the last column will be the name):
docker ps
Then run:
docker exec <NAME OF CONTAINER> apt-get update
And finally to install pandas:
docker exec <NAME OF CONTAINER> apt-get install -y python-pandas
(the -y is an automatic 'yes' to stop a prompt from appearing for you to agree to the installation taking up additional disk space)
Here is an image with the pandas installed -
https://hub.docker.com/r/zavolokas/tensorflow-udacity/
Or pull it docker pull zavolokas/tensorflow-udacity:pandas