I am trying to incorporate a python container and a dynamodb container into one stack file to experiment with Docker swarm. I have done tutorials on docker swarm seeing web apps running across multiple nodes before but never built anything independently. I am able to run docker-compose up with no issues, but struggling with swarm.
My docker-compose.yml looks like
version: '3.3'
services:
dynamodb:
image: "amazon/dynamodb-local"
ports:
- "8000:8000"
track-count:
image: "my-app"
links:
- "dynamodb:localhost"
Running docker stack deploy -c docker-compose.yml trial_stack brings up no errors however printing 'hello world' as the first line of python code is not displayed in the terminal. I get the following as CMD line output
Ignoring unsupported options: links
Creating network trial_stack_default
Creating service trial_stack_dynamodb
Creating service trial_stack_track-count
My question is:
1) Why is the deploy service ignoring the links? I have noticed this is repeated in the docs https://docs.docker.com/engine/reference/commandline/stack_deploy/ but unsure if this will cause my stack to fail.
2) Assuming the links issue is fixed, where will any command line output be shown, to confirm the system is running? Currently I only have one node, my local machine, which is the manager.
For reference, my python image is being built by the following Dockerfile:
FROM python:3.8-slim-buster
RUN mkdir /app
WORKDIR /app
RUN pip install --upgrade pip
COPY ./requirements.txt ./
RUN pip install -r ./requirements.txt
COPY / /
COPY /resources/secrets.py /resources/secrets.py
CMD [ "python", "/main.py" ]
You can update docker-compose.yaml to enable tty for the services for which you want to see the stdout on console.
Updated docker-compose.yaml should look like this:
version: '3.3'
services:
dynamodb:
image: "amazon/dynamodb-local"
ports:
- "8000:8000"
track-count:
image: "my-app"
tty: true
links:
- "dynamodb:localhost"
and then when once you have the task deployed, to check service logs you can run:
# get the service name
docker stack services <STACK_NAME>
# display the service logs, edited based on user's suggestion
docker service logs --follow --raw <SERVICE_NAME>
Related
My current organization is migrating to DataDog for Application Performance Monitoring. I am deploying a Python Flask web application using docker to Azure Container Registry. After the deployment to Azure the app should be listed/available on Datadog portal.
Please note I just started learning Docker containers. There is a high chance I could do completely wrong. Please bear with me
Steps followed
Option 1: Create a docker container on local machine and push to ACR
Added dd-trace python library to the docker image
Added dd-trace run command the docker file
build the image
run the container on local
Getting OSError: [Errno 99] Cannot assign requested address
FROM python:3.7
ENV VIRTUAL_ENV=/opt/venv
RUN python -m venv $VIRTUAL_ENV
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
ENV DD_API_KEY=apikeyfromdatadoghq
ENV DD_ENV=safhire-dev
ENV DD_LOGS_ENABLED=true
ENV DD_LOGS_INJECTION=true
ENV DD_SERVICE=dev-az1-pythonbusinessservice
ENV DD_TAGS=products:myprojects
ENV DD_TRACE_DEBUG=true
ENV DD_TRACE_ENABLED=true
ENV DOCKER_ENABLE_CI=true
COPY /app /app
COPY requirements.txt /
RUN pip install --no-cache-dir -U pip
RUN pip install --no-cache-dir -r /requirements.txt
CMD ddtrace-run python app/main.py runserver 127.0.0.1:3000
Option 2: Forward logs to Azure Blob Storage but a heavy process
Deploy Python using Code base Linux
Forward the logs to a Azure Blob storage
Create a BlobTrigger Azure Function to forward the logs to DataDogAPI
I believe with this approach we can not capture APM logs but, we can capture application and console logs
Option 3: using Serilog but, my organization does not want to use third party logging framework, we have our own logging framework
Any help is highly appreciated, I am looking for a solution using Option 1. I went through the Microsoft articles, Datadog documentation but, no luck.
I setup app registrations, Manage reader permissions on Subscription, created ClientID and app secrets on Azure portal. none of them helped
Could you confirm whether is there a way to collect the APM logs on datadog with out installing agent on Azure.
Thank you in advance.
After few days of research and follow up with datadog support team, I am able to get the APM logs on datadog portal.
Below is my docker-compose.yml file configuration, I believe it helps someone in future
version: "3"
services:
web:
build: web
command: ddtrace-run python standalone_api.py
volumes:
- .:/usr/src/app
depends_on:
datadog-agent:
condition: service_healthy
image: pythonbusinessservice:ICDNew
ports:
- 5000:5000
environment:
- DATADOG_HOST=datadog-agent
- DD_TRACE_AGENT_PORT=8126
- DD_AGENT_HOST=datadog-agent
datadog-agent:
build: datadog
image: gcr.io/datadoghq/agent:latest
ports:
- 8126:8126
environment:
- DD_API_KEY=9e3rfg*****************adf3
- DD_SITE=datadoghq.com
- DD_HOSTNAME=pythonbusinessservice
- DD_TAGS=env:dev
- DD_APM_ENABLED=true
- DD_APM_NON_LOCAL_TRAFFIC=true
- DD_DOGSTATSD_NON_LOCAL_TRAFFIC=true
- DD_SERVICE=pythonbusinessservice
- DD_LOGS_CONFIG_CONTAINER_COLLECT_ALL=true
- DD_CONTAINER_EXCLUDE="name:datadog-agent"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /proc/:/host/proc/:ro
- /opt/datadog-agent/run:/opt/datadog-agent/run:rw
- /sys/fs/cgroup:/host/sys/fs/cgroup:ro
The Dockerfile for my python long running application
FROM python:3.7
COPY . /app
WORKDIR /app
RUN pip install -r requirements.txt
CMD ["ddtrace-run python", "/app/standalone_api.py"]
Please note, on the requirements.txt file I have ddtrace package listed
I am running FastApi via docker by creating a sevice called ingestion-data in docker-compose. My Dockerfile :
FROM tiangolo/uvicorn-gunicorn-fastapi:python3.7
# Environment variable for directory containing our app
ENV APP /var/www/app
ENV PYTHONUNBUFFERED 1
# Define working directory
RUN mkdir -p $APP
WORKDIR $APP
COPY . $APP
# Install missing dependencies
RUN pip install -r requirements.txt
AND my docker-compose.yml file
version: '3.8'
services:
ingestion-service:
build:
context: ./app
dockerfile: Dockerfile
ports:
- "80:80"
volumes:
- .:/app
restart: always
I am not sure why this is not picking up any change automatically when I make any change in any endpoint of my application. I have to rebuild my images and container every time.
Quick answer: Yes :)
In the Dockerfile, you copying your app into /var/www/app.
The instructions form the Dockerfile are executed when you build your image (docker build -t <imgName>:<tag>)
If you change the code later on, how could the image be aware of that?
However, you can mount a volume(a directory) from your host machine, into the container when you execute the docker run / docker-compose up command, right under /var/www/app. You'll then be able to change the code in your local directory and the changes will automatically be seen in the container as well.
Perhaps you want to mount the current working directory(the one containing your app) at /var/www/app?
volumes:
- .:/var/www/app
I've been scratching my head for a while with this. I have the following Dockerfile for my python application:
# Use an official Python runtime as a parent image
FROM frankwolf/rpi-python3
# Set the working directory to /app
WORKDIR /app
# Copy the current directory contents into the container at /app
COPY . /app
RUN chmod 777 docker-entrypoint.sh
# Install any needed packages specified in requirements.txt
RUN pip3 install --trusted-host pypi.python.org -r requirements.txt
# Run __main__.py when the container launches
CMD ["sudo", "python3", "__main__.py", "-debug"] # Not sure if I need sudo here
docker-compose file:
version: "3"
services:
mongoDB:
restart: unless-stopped
volumes:
- "/data/db:/data/db"
ports:
- "27017:27017"
- "28017:28017"
image: "andresvidal/rpi3-mongodb3:latest"
mosquitto:
restart: unless-stopped
ports:
- "1883:1883"
image: "mjenz/rpi-mosquitto"
FG:
privileged: true
network_mode: "host"
depends_on:
- "mosquitto"
- "mongoDB"
volumes:
- "/home/pi:/home/pi"
#image: "arkfreestyle/fg:v1.8"
image: "test:latest"
entrypoint: /app/docker-entrypoint.sh
restart: unless-stopped
And this what docker-entrypoint.sh looks like:
#!/bin/sh
if [ ! -f /home/pi/.initialized ]; then
echo "Initializing..."
echo "Creating .initialized"
# Create .initialized hidden file
touch /home/pi/.initialized
else
echo "Initialized already!"
sudo python3 __main__.py -debug
fi
Here's what I am trying to do:
(This stuff already works)
1) I need a docker image which runs my python application when I run it in a container. (this works)
2) I need a docker-compose file which runs 2 services + my python application, BUT before running my python application I need to do some initialization work, for this I created a shell script which is docker-entrypoint.sh. I want to do this initialization work ONLY ONCE when I deploy my application on a machine for the first time. So I'm creating a .initialized hidden file which I'm using as a check in my shell script.
I read that using entrypoint in a docker-compose file overwrites any old entrypoint/cmd given to the Dockerfile. So that's why in the else portion of my shell script I'm manually running my code using "sudo python3 main.py -debug", this else portion works fine.
(This is the main question)
In the if portion, I do not run my application in the shell script. I've tested the shell script itself separately, both if and else statements work as I expect, but when I run "sudo docker-compose up", the first time when my shell script hits the if portion it echoes the two statements, creates the hidden file and THEN RUNS MY APPLICATION. The console output appears in purple/pink/mauve for the application, while the other two services print their logs out in yellow and cyan. I'm not sure if the colors matter, but in the normal condition my application logs are always green, in fact the first two echoes "Initializing" and "Creating .initialized" are also green! so I thought I'd mention this detail. After those two echoes, my application mysteriously begins and logs console output in purple...
Why/how is my application being invoked in the if statement of the shell script?
(This is only happens if I run through docker-compose, not if I just run the shell script with sh docker-entrypoint.sh)
Problem 1
Using ENTRYPOINT and CMD at the same time has some strange effects.
Problem 2
This happens to your container:
It is started the first time. The .initialized file does not exist.
The if case is executed. The file is created.
The script and therefore the container ends.
The restart: unless-stopped option restarts the container.
The .initialized file exists now, the else case is run.
python3 __main__.py -debug is executed.
BTW the USER command in the Dockerfile or the user option in Docker Compose are better options than sudo.
I have been reading this tutorial:
https://prakhar.me/docker-curriculum/
along with other tutorials, and Docker docks and I am still not completely clear on how to do this task.
The problem
My local machine is running Mac OS X, and I would like to set up a development environment for a python project. In this project I need to run call an api from a docker repo bamos/openface. The project also has some dependencies such as yaml, etc. If I just mount my local to openface, ie:
docker run -v path/to/project:/root/project -p 9000:9000 -p 8000:8000 -t -i bamos/openface /bin/bash
Then I need to install yaml and other dependencies, and every time I exit the container the installations would be lost. Additionally, it is also much slower for some reason. So the right way to do this is using Docker compose, but I am not sure how to proceed from here.
UPDATE
In response to the comments, I will now update the problem:
Right now my Dockerfile looks like this:
FROM continuumio/anaconda
ADD . /face-off
WORKDIR /face-off
RUN pip install -r requirements.txt
EXPOSE 5000
CMD [ "python", "app.py" ]
It is important that I build from anaconda since a lot of my code will use numpy and scipy. Now I also need bamos/openface, so I tried adding that to my docker-compose.yml file:
version: '2'
services:
web:
build: .
command: python app.py
ports:
- "5000:5000"
volumes:
- .:/face-off
openface:
build: bamos/openface
However, I am getting error:
build path path/to/face-off/bamos/openface either does not exist, is not accessible, or is not a valid URL
So I need to pass bamos/openface the right way so I can build a container with it. Right now bamos/openface is listed when I do docker images.
I have looked through the questions on this site, but I have not been able to fix this problem.
I created and ran an image of my django app, but when I try to view the app from the browser, the page does not load (can't establish a connection to the server)
I am using docker toolbox, I am using OS X El Capitan and the Macbook is from 2009.
The container IP is: 192.168.99.100
The django project root is called "Web app" and is the directory containing manage.py. My Dockerfile and my requirements.txt files are in this directory.
My dockerfile is:
FROM python:3.5
WORKDIR /usr/src/app
COPY requirements.txt ./
RUN pip install -r requirements.txt
COPY . .
EXPOSE 8000
CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"]
My requirements.txt has django and mysqlclient
My django app uses Mysql, and I tried to view the dockerized django app in the browser with and without linking it to the standard mysql image. In both cases, I only see the following error:
problem loading page couldn't establish connection to server
When I did try linking the django container to the mysql container I used:
docker run --link mysqlapp:mysql -d app
Where mysqlapp is my mysql image and 'app' is my django image.
In my django settings.py, the allowed hosts are:
ALLOWED_HOSTS: ['localhost', '127.0.0.1', '0.0.0.0', '192.168.99.100']
Again, the image is successfully created when I used docker build, and it is successfully run as a container. Why is the page not loading in the browser?
I suggest to use yml file and docker compose. Below is a template to get you started:
[Dockerfile]
FROM python:2.7
RUN pip install Django
RUN mkdir /code
WORKDIR /code
COPY code/ /code/
where your files are located in code directory.
[docker-compose.yml]
version: '2'
services:
db:
image: mysql
web0:
build: .
command: python manage.py runserver 0.0.0.0:8000
ports:
- "8000:8000"
depends_on:
- db
There might be a problem with your working directory path defined in Dockerfile. Hope above helps.
Solution provided by salehinejad seems to be good enough ,although i have not tested it personally but if you do not want to use yml file and want to go your way then you should expose the port by adding
-p 0:8000
in your run command
So your should look like this :
docker run -p 0:8000 --link mysqlapp:mysql -d app
I suspect you have not told Docker to talk to your VM, and that your containers are running on your host machine (if you can access at localhost, this is the issue).
Please see this post for resolution:
Connect to docker container using IP