I've been working on this all week, and I don't seem to understand what I'm missing. The problem is simple, I've a container running a platform on Django, and I need to create a Cronjob for an smaller task, I created a test ask that executes every minute and just print a log just to test, but it is not working, while it installs cron, add the cronjobs, start the cron service, and I can see them in the crontab, they are just never triggered.
When I first started, I had the Cron running in the same instance, but after reading this Question I found that I had to separate it in 2 instances, since apparently having Django running was afecting the cron service, so following that, this is how I have my files:
docker-compose.yml
version: '3'
services:
auth:
build:
context: ./
dockerfile: ./devops/Dockerfile
args:
[Bunch of parameters]
container_name: auth
volumes:
- ./project:/app
ports:
- 8000:8000
environment:
[Bunch of parameters]
command: python manage.py runserver 0.0.0.0:8000
cron:
build:
context: ./
dockerfile: ./devops/Dockerfile
args:
[Bunch of parameters]
container_name: cron
volumes:
- ./project:/app
environment:
[Bunch of parameters]
command: cron -f
Dockerfile
FROM python:3.8
ENV PYTHONUNBUFFERED 1
WORKDIR /app
COPY ./devops/requirements.txt .
COPY ./project .
# COPY ./.env .
RUN apt-get update
RUN apt-get -y install cron
RUN cp ./.env . || echo "file not found"
RUN pip install -r requirements.txt
#Set permission to entrypoint.sh
RUN chmod +x entrypoint.sh
# start web server
ENTRYPOINT ["./entrypoint.sh"]
CMD ["gunicorn", "-b", "0.0.0.0:8000", "project.wsgi:application", "--workers=5"]
entrypoint.sh
#!/bin/sh
# Set up scheduled jobs, if this is the cron container.
if [ "$1" = cron ]; then
service cron start
python ./manage.py crontab add
service cron stop
fi
# Run whatever command we got passed.
exec "$#"
settings.py
CRONJOBS = [
('*/1 * * * *', 'apps.coupons.cron.test'),
]
cron.py
import logging
logger = logging.getLogger(__name__)
def test():
logger.warning('Hello World')
logger.debug('Hello World')
logger.info('Hello World')
logger.error('Hello World')
logger.critical('Hello World')
print("Hello World")
return "Finished"
Here you can see that the cron were added, and that cron is running, and that executing the cronjob manually works.
Still, doesn't matter how long I wait, it doesn't seem like the Cronjob runs automatically every minute (I check this by using the file.log that I setted in the settings.py logger's options). What am I doing wrong or what may be missing to make the cron work?
Related
Trying to run the following docker compose file
version: '3'
services:
database:
image: postgres
container_name: pg_container
environment:
POSTGRES_USER: partman
POSTGRES_PASSWORD: partman
POSTGRES_DB: partman
app:
build: .
container_name: partman_container
links:
- database
environment:
- DB_NAME=partman
- DB_USER=partman
- DB_PASSWORD=partman
- DB_HOST=database
- DB_PORT=5432
- SECRET_KEY='=321t+92_)#%_4b+f-&0ym(fs2p5-0-_nz5mhb_cak9zlo!bv#'
depends_on:
- database
expose:
- "8000"
- "8020"
ports:
- "127.0.0.1:8020:8020"
volumes:
pgdata: {}
when running docker-compose up-build with the following docker file
# Dockerfile
# FROM directive instructing base image to build upon
FROM python:3.7-buster
RUN apt-get update && apt-get install nginx vim -y --no-install-recommends
COPY nginx.default /etc/nginx/sites-available/default
RUN ln -sf /dev/stdout /var/log/nginx/access.log \
&& ln -sf /dev/stderr /var/log/nginx/error.log
RUN mkdir .pip_cache \
mkdir -p /opt/app \
mkdir -p /opt/app/pip_cache \
mkdir -p /opt/app/py-partman
COPY start-server.sh /opt/app/
COPY requirements.txt start-server.sh /opt/app/
COPY .pip_cache /opt/app/pip_cache/
COPY partman /opt/app/py-partman/
WORKDIR /opt/app
RUN pip install -r requirements.txt --cache-dir /opt/app/pip_cache
RUN chown -R www-data:www-data /opt/app
RUN /bin/bash -c 'ls -la; chmod +x /opt/app/start-server.sh; ls -la'
EXPOSE 8020
STOPSIGNAL SIGTERM
CMD ["/opt/app/start-server.sh"]
/opt/app/start-server.sh :
#!/usr/bin/env bash
# start-server.sh
ls
pwd
cd py-partman
ls
pwd
python manage.py createsuperuser --no-input
python manage.py makemigrations
python manage.py migrate
python manage.py initialize_entities
the database image keeps on running, i want to stop it because otherwise the jenkins job will keep on waiting for the image to terminate.
Any good ideas / better ideas how to do so ?
Maybe with -> docker stop <"container id or container name">
Use -f to force it, if it can't be stopped.
Try it.
Docker Compose is generally oriented around long-running server-type processes, and where database containers can frequently take 30-60 seconds to start up, it's usually beneficial to not repeat them. (In fact, the artifacts you show look a little odd for not including a python manage.py runserver command.)
It looks like there is a docker-compose up option for what you're looking for
docker-compose up --build --abort-on-container-exit
If you wanted to do this more manually, and especially if your app container's normal behavior is to actually start a server, you can docker-compose run the initialization command. This will start up the container and its dependencies, but it also expects its command to return, and then you can clean up yourself.
docker-compose build
docker-compose run app /opt/app/initialize-only.sh
docker-compose down -v
I'm trying to make my first django container with uwsgi. It works as follows:
FROM python:3.5
RUN apt-get update && \
apt-get install -y && \
pip3 install uwsgi
COPY ./projects.thux.it/requirements.txt /opt/app/requirements.txt
RUN pip3 install -r /opt/app/requirements.txt
COPY ./projects.thux.it /opt/app
COPY ./uwsgi.ini /opt/app
COPY ./entrypoint /usr/local/bin/entrypoint
ENV PYTHONPATH=/opt/app:/opt/app/apps
WORKDIR /opt/app
ENTRYPOINT ["entrypoint"]
EXPOSE 8000
#CMD ["--ini", "/opt/app/uwsgi.ini"]
entrypoint here is a script that detects whether to call uwsgi (in case there are no args) or python manage in all other cases.
I'd like to use this container both as an executable (dj migrate, dj shell, ... - dj here is python manage.py the handler for django interaction) and as a long-term container (uwsgi --ini uwsgi.ini). I use docker-compose as follows:
web:
image: thux-projects:3.5
build: .
ports:
- "8001:8000"
volumes:
- ./projects.thux.it/web/settings:/opt/app/web/settings
- ./manage.py:/opt/app/manage.py
- ./uwsgi.ini:/opt/app/uwsgi.ini
- ./logs:/var/log/django
And I manage in fact to serve the project correctly but to interact with django to "check" I need to issue:
docker-compose exec web entrypoint check
while reading the docs I would have imagined I just needed the arguments (without entrypoint)
Command line arguments to docker run will be appended after
all elements in an exec form ENTRYPOINT, and will override all
elements specified using CMD. This allows arguments to be passed to
the entry point, i.e., docker run -d will pass the -d argument
to the entry point.
The working situation with "repeated" entrypoint:
$ docker-compose exec web entrypoint check
System check identified no issues (0 silenced).
The failing one if I avoid 'entrypoint':
$ docker-compose exec web check
OCI runtime exec failed: exec failed: container_linux.go:346: starting container process caused "exec: \"check\": executable file not found in $PATH": unknown
docker exec never uses a container's entrypoint; it just directly runs the command you give it.
When you docker run a container, the entrypoint and command you give to start it are combined to produce a single command line, and that command becomes the main container process. On the other hand, when you docker exec a command in a running container, it's interpreted literally; there aren't two parts of the command line to assemble, and the container's entrypoint isn't considered at all.
For the use case you describe, you don't need an entrypoint script to process the command in an unusual way. You can create a symlink to the manage.py script to give a shorter alias to run it, but make the default command be the uwsgi runner.
RUN chmod +x manage.py
RUN ln -s /opt/app/manage.py /usr/local/bin/dj
CMD ["uwsgi", "--ini", "/opt/app/uwsgi.ini"]
# Runs uwsgi:
docker run -p 8000:8000 myimage
# Manually trigger database migrations:
docker run --rm myimage dj migrate
I've been scratching my head for a while with this. I have the following Dockerfile for my python application:
# Use an official Python runtime as a parent image
FROM frankwolf/rpi-python3
# Set the working directory to /app
WORKDIR /app
# Copy the current directory contents into the container at /app
COPY . /app
RUN chmod 777 docker-entrypoint.sh
# Install any needed packages specified in requirements.txt
RUN pip3 install --trusted-host pypi.python.org -r requirements.txt
# Run __main__.py when the container launches
CMD ["sudo", "python3", "__main__.py", "-debug"] # Not sure if I need sudo here
docker-compose file:
version: "3"
services:
mongoDB:
restart: unless-stopped
volumes:
- "/data/db:/data/db"
ports:
- "27017:27017"
- "28017:28017"
image: "andresvidal/rpi3-mongodb3:latest"
mosquitto:
restart: unless-stopped
ports:
- "1883:1883"
image: "mjenz/rpi-mosquitto"
FG:
privileged: true
network_mode: "host"
depends_on:
- "mosquitto"
- "mongoDB"
volumes:
- "/home/pi:/home/pi"
#image: "arkfreestyle/fg:v1.8"
image: "test:latest"
entrypoint: /app/docker-entrypoint.sh
restart: unless-stopped
And this what docker-entrypoint.sh looks like:
#!/bin/sh
if [ ! -f /home/pi/.initialized ]; then
echo "Initializing..."
echo "Creating .initialized"
# Create .initialized hidden file
touch /home/pi/.initialized
else
echo "Initialized already!"
sudo python3 __main__.py -debug
fi
Here's what I am trying to do:
(This stuff already works)
1) I need a docker image which runs my python application when I run it in a container. (this works)
2) I need a docker-compose file which runs 2 services + my python application, BUT before running my python application I need to do some initialization work, for this I created a shell script which is docker-entrypoint.sh. I want to do this initialization work ONLY ONCE when I deploy my application on a machine for the first time. So I'm creating a .initialized hidden file which I'm using as a check in my shell script.
I read that using entrypoint in a docker-compose file overwrites any old entrypoint/cmd given to the Dockerfile. So that's why in the else portion of my shell script I'm manually running my code using "sudo python3 main.py -debug", this else portion works fine.
(This is the main question)
In the if portion, I do not run my application in the shell script. I've tested the shell script itself separately, both if and else statements work as I expect, but when I run "sudo docker-compose up", the first time when my shell script hits the if portion it echoes the two statements, creates the hidden file and THEN RUNS MY APPLICATION. The console output appears in purple/pink/mauve for the application, while the other two services print their logs out in yellow and cyan. I'm not sure if the colors matter, but in the normal condition my application logs are always green, in fact the first two echoes "Initializing" and "Creating .initialized" are also green! so I thought I'd mention this detail. After those two echoes, my application mysteriously begins and logs console output in purple...
Why/how is my application being invoked in the if statement of the shell script?
(This is only happens if I run through docker-compose, not if I just run the shell script with sh docker-entrypoint.sh)
Problem 1
Using ENTRYPOINT and CMD at the same time has some strange effects.
Problem 2
This happens to your container:
It is started the first time. The .initialized file does not exist.
The if case is executed. The file is created.
The script and therefore the container ends.
The restart: unless-stopped option restarts the container.
The .initialized file exists now, the else case is run.
python3 __main__.py -debug is executed.
BTW the USER command in the Dockerfile or the user option in Docker Compose are better options than sudo.
I am trying to add docker support to an already existing django project. I have a Dockerfile, a docker-compose, and a gunicorn.sh which I use as a script to launch the whole things. That script works fine when I run it from my shell.
When I run:
docker-compose -f docker-compose.yml up
I get this error:
ERROR: for intranet_django_1 Cannot start service django: oci runtime error: container_linux.go:247: starting container process caused "exec: \"/srv/gunicorn.sh\": stat /srv/gunicorn.sh: no such file or directory"
What the hell am I doing wrong?
I am very much a docker n00b so any explanation would be most welcome.
The Dockerfile looks like so:
FROM python:3
ENV PYTHONUNBUFFERED 1
ENV DB_NAME unstable_intranet_django
ENV DB_USER django
ENV DB_PASSWORD ookookEEK
ENV DB_HOST db
ENV DB_PORT 3306
RUN groupadd -r django
RUN useradd -r -g django django
COPY ./requirements/requierments.txt /srv/
RUN pip install -U pip
RUN pip install -r /srv/requierments.txt
COPY ./intranet_site/ /srv
RUN chmod a+rx /srv/gunicorn.sh
RUN chown -R django:django /srv/
USER django
WORKDIR /srv
I am well aware that the passwords should not be set here and that a permanent volume with a file containing them is probably the best way to deal with it. However, I kinda want something working instead of spending hours fiddling with things and not being able to see anything run…
The docker-compose.yml looks like:
version: '3'
services:
db:
image: mariadb
environment:
- MYSQL_ROOT_PASSWORD=fubar
- MYSQL_USER=django
- MYSQL_PASSWORD=ookookEEK
- MYSQL_DATABASE=unstable_intranet_django
django:
build: .
command: /srv/gunicorn.sh
volumes:
- .:/srv
ports:
- "8000:8000"
depends_on:
- db
Finally, the gunicorn.sh file is:
#!/bin/bash
# -*- coding: utf-8 -*-
# Check if the database is alive or not.
python << END
from MySQLdb import Error
from MySQLdb import connect
from sys import exit
from time import sleep
retry=0
while True:
try:
conn = connect(db="$DB_NAME",
user="$DB_USER",
password="$DB_PASSWORD",
host="$DB_HOST",
port=$DB_PORT)
print("✔ DB $DB_NAME on $DB_HOST:$DB_PORT is up.")
break
except Error as err:
snooze = retry / 10.0
print("✖ DB $DB_NAME on $DB_HOST:$DB_PORT is unavailable "
"→ sleeping for {}…".format(snooze))
sleep(snooze)
retry += 1
exit(0)
END
# Set up log file.
log="./gunicorn.log"
date > ${log}
# Collectstatic
echo "Collecting static files." | tee -a ${log}
python manage.py collectstatic -v 3 --noinput >> ${log}
# Migrate database
echo "Doing database migration." | tee -a ${log}
python manage.py migrate -v 3 >> ${log}
# New shiny modern hip way:
echo "Running Gunicorn on ${HOSTNAME} …" | tee -a ${log}
gunicorn -b ${HOSTNAME}:8000 -w 2 intranet_site.wsgi | tee -a ${log}
To make things stranger:
; docker run -it intranet_web /bin/bash
django#ce7f641cc1c7:/srv$ ls -l gunicorn.sh
-rwxrwxr-x. 1 django django 1677 Jun 2 07:51 gunicorn.sh
django#ce7f641cc1c7:/srv$ ./gunicorn.sh
✖ DB unstable_intranet_django on 127.0.0.1:3306 is unavailable → sleeping for 0.0…
So running the script from the containers seems to work just fine…
I think you should have:
ADD . /srv/ instead of COPY ./intranet_site/ /srv
because ADD . /srv/ adds all the content of the directory in which you have the Dockerfile to the srv folder from container. So the COPY/ADD command should be used in the folder that contains the Dockerfile. And I suppose your Dockerfile is in this root directory of the project (alongside docker-compose.yml and gunicorn.sh).
You could also use COPY . /srv/ with the same effect.
Suspect the path shouldn't have a leading .:
command: /srv/gunicorn.sh
I'm newbie in docker-compose and I have a docker with my django instance and a mysql database. I would like to create a self autoconfigured container which run a command only on the first docker run. In this command I would like to do the following tasks:
make initial database migrations
create the admin superuser
import a mysql backup into the database
After this the system should continue launching the django test webserver.
Are there any way to tell docker-compose to run a command just on it first run or are there any alternative in django to control if the system is already configured and updated?
In order to clarify here are my dockfile and docker-compose.yml:
FROM python:3.4
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
ADD requirements.txt /code/
RUN pip install -r requirements.txt
ADD . /code/
####################
version: '2'
services:
db:
image: "mysql:5.6"
ports:
- 3306:3306
environment:
MYSQL_ROOT_PASSWORD: xxxxxx
MYSQL_DATABASE: xxxxxx
MYSQL_USER: xxxxx
MYSQL_PASSWORD: xxxxxxx
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
Thanks.
Following the comments of #cricket_007, finally I have found a tricky solution to solve the problem. I have created a sh script for the database service and for my web service. Additionally I have created two version files in my folder, web_local.version and web_server.version.
The web_local.version has been added to my .gitignore because this file is used to storage the current app version.
The start_web.sh script is a simple script that compare if the folder contains a web_local.version file. In that case the project has been configured in the past and the script checks if the current app version is updated compared with the server version. In the case all is up to date simply run a webserver otherwise run a migrate to update the models and then run the webserver.
Here is the web_start.sh script for references:
#!/bin/bash
FILE="web_local.version"
if [ -f "$FILE" ];
then
echo "File $FILE exist."
if diff ./web_server.version ./web_local.version > /dev/null;
then
echo "model version up to date :)"
else
echo "model updated!!"
python manage.py migrate
cp ./web_server.version ./$FILE
fi
else
echo "File $FILE does not exist"
sleep 10 #added because the first time db take a long time to init and the script doesn't wait until db is finished
cp ./web_server.version ./$FILE
python manage.py migrate
fi
python manage.py runserver 0.0.0.0:8000
I suppose that there are more formal solutions but this solutions is functional for my case because it allows our team to maintain the same mock database and same models synced through git and we have a zero time configuration environment running just with one command.