I just want some help here, I'm kinda stuck here in Docker and can't find a way out. First, I'm using Windows for a Django APP and Docker
I'm using PgAdmin4 with PostgreSQL 14 and created a new server for docker
The log for the Postgres Image:
2022-07-16 19:39:23.655 UTC [1] LOG: starting PostgreSQL 14.4 (Debian 14.4-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2022-07-16 19:39:23.673 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2022-07-16 19:39:23.673 UTC [1] LOG: listening on IPv6 address "::", port 5432
2022-07-16 19:39:23.716 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2022-07-16 19:39:23.854 UTC [26] LOG: database system was shut down at 2022-07-16 16:50:47 UTC
2022-07-16 19:39:23.952 UTC [1] LOG: database system is ready to accept connections
PostgreSQL Database directory appears to contain a database; Skipping initialization
Log from my image: (you can see that doesn't have migrations)
0 static files copied to '/app/static', 9704 unmodified.
Operations to perform:
Apply all migrations: admin, auth, contenttypes, controle, sessions
Running migrations:
No migrations to apply.
Performing system checks...
System check identified no issues (0 silenced).
July 16, 2022 - 16:40:38
Django version 4.0.6, using settings 'setup.settings'
Starting development server at http://0.0.0.0:8000/
Quit the server with CONTROL-C.
My docker-compose (updated):
services:
db:
image: postgres
volumes:
- ./data/db:/var/lib/postgresql/data
networks:
- django_net
environment:
- POSTGRES_DB=${POSTGRES_DB}
- POSTGRES_USER = ${POSTGRES_USER}
- POSTGRES_PASSWORD = ${POSTGRES_PASSWORD}
ports:
- "5432:5432"
web:
build: .
command: >
sh -c "python manage.py collectstatic --noinput &&
python manage.py migrate &&
python manage.py runserver 0.0.0.0:8000"
volumes:
- .:/app
ports:
- "8000:8000"
depends_on:
- db
links:
- db
environment:
- POSTGRES_NAME=${POSTGRES_NAME:-djangodb}
- POSTGRES_USER=${POSTGRES_USER:-postgre}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-postgre}
networks:
- django_net
networks:
django_net:
driver: bridge
And my .env file (updated):
SECRET_KEY='django-insecure-1l2oh_bda$#s0w%d!#qyq8-09sn*8)6u-^wb(hx03==(vjk16h'
POSTGRES_NAME=postgres
POSTGRES_USER=postgres
POSTGRES_PASSWORD=mypass
POSTGRES_DB=mydb
DJANGO_ALLOWED_HOSTS=localhost 127.0.0.1 [::1]
So, analyzing the logs from Postgres last line, he found my local DB (is that right ?) and didn't initialize, but my superuser is gone and so my data.
Is there something that I'm missing ? Maybe it's like that, and I don't know... Just to be sure, I printed some lines from PGAdmin and the APP Screen
DB:
My APP:
My settings.py:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': os.environ.get('POSTGRES_NAME'),
'USER': os.environ.get('POSTGRES_USER'),
'PASSWORD': os.environ.get('POSTGRES_PASSWORD'),
'HOST': 'db',
'PORT': 5432,
}
}
If I correct understood your question, you can't connect to created database.
If you want to connect to your containerized docker database from outside, you should define ports parameter in your db service in docker-compose file.
services:
db:
image: postgres
volumes:
- ./data/db:/var/lib/postgresql/data
networks:
- django_net
environment:
- POSTGRES_DB=${POSTGRES_DB}
- POSTGRES_USER=${POSTGRES_USER}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
ports:
- "5432:5432"
I hope I correct understood your question about you can't connect to new database and I hope my answer will help you.
In this setup, I see two things:
You've configured DATABASE_URL to point to host.docker.internal, so your container is calling out of Docker space, to whatever's listening on port 5432.
In your Compose file, the db container does not have ports:, so you're not connecting to the database your Compose setup starts.
This implies to me that you're running another copy of the PostgreSQL server on your host, and your application is actually connecting to that. (Maybe you're on a MacOS host, and you installed it via Homebrew?)
You don't need to do any special setup to connect between containers; just use the Compose service name db as a host name. You in particular do not need the special host.docker.internal name here. (You can also delete the networks: from the file, so long as you delete all of them; Compose creates a network named default for you and attaches containers to it automatically.) I might configure this in the Compose file, overriding the .env file
version: '3.8'
services:
db: { ... }
web:
environment:
- DATABASE_URL=postgres://$(POSTGRES_USER):$(POSTGRES_PASSWORD)#db/$(POSTGRES_DB)
I hope my answer to help you solve the problem. Please change the config as follow:
version: "3.9"
services:
db:
image: postgres
volumes:
- ./data/db:/var/lib/postgresql/data
networks:
- django_net
environment:
- POSTGRES_DB=${POSTGRES_DB:-djangodb}
- POSTGRES_USER = ${POSTGRES_USER:-postgres}
- POSTGRES_PASSWORD = ${POSTGRES_PASSWORD:-changeme}
ports:
- "5432:5432"
web:
build: .
command: >
sh -c "python manage.py collectstatic --noinput &&
python manage.py migrate &&
python manage.py runserver 0.0.0.0:8000"
volumes:
- .:/app
ports:
- "8000:8000"
depends_on:
- db
links:
- db
environment:
- POSTGRES_NAME=${POSTGRES_NAME:-djangodb}
- POSTGRES_USER=${POSTGRES_USER:-postgre}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-postgre}
networks:
- django_net
networks:
django_net:
driver: bridge
I'm building an API that fetches data from a MySQL database using Docker. I've tried everything and I always get this error: 2005 (HY000): Unknown MySQL server host 'db' (-3). Here is my docker compose file:
version: '3'
services:
web:
container_name: nginx
image: nginx
volumes:
- ./nginx/nginx.conf:/tmp/nginx.conf
environment:
- FLASK_SERVER_ADDR=backend:9091
- DB_PASSWORD=password
- DB_USER=user
- DB_HOST=db
command: /bin/bash -c "envsubst < /tmp/nginx.conf > /etc/nginx/conf.d/default.conf && nginx -g 'daemon off;'"
ports:
- 80:80
networks:
- local
depends_on:
- backend
backend:
container_name: app
build: flask
environment:
- FLASK_SERVER_PORT=9091
- DB_PASSWORD=password
volumes:
- flask:/tmp/app_data
restart: unless-stopped
networks:
- local
depends_on:
- db
links:
- db
db:
container_name: db
image: mysql
restart: unless-stopped
volumes:
- ./mysql:/docker-entrypoint-initdb.d
environment:
- MYSQL_ROOT_PASSWORD=password
- MYSQL_DATABASE=database
- MYSQL_USER=user
- MYSQL_PASSWORD=password
ports:
- 3306:3306
networks:
local:
volumes:
flask:
driver: local
db:
driver: local
Inside the flask directory I have my Dockerfile like so:
FROM ubuntu:latest
WORKDIR /src
RUN apt -y update
RUN apt -y upgrade
RUN apt install -y python3
RUN apt install -y python3-pip
COPY . .
RUN chmod +x -R .
RUN pip install -r requirements.txt --no-cache-dir
CMD ["python3","app.py"]
Finally, on my app.py file I try to connect to the database with the name of the Docker container. I have tried using localhost and it still gives me the same error. This is the part of the code I use to access it:
conn = mysql.connector.connect(
host="db",
port=3306,
user="user",
password="password",
database="database")
What is it that I'm doing wrong?
The containers aren't on the same networks:, which could be why you're having trouble.
I'd recommend deleting all of the networks: blocks in the file, both the blocks at the top level and the blocks in the web and backend containers. Compose will create a network named default for you and attach all of the containers to that network. Networking in Compose in the Docker documentation has more details on this setup.
The links: block is related to an obsolete Docker networking mode, and I've seen it implicated in problems in other SO questions. You should remove it as well.
You also do not need to manually specify container_name: in most cases. For the Nginx container, the Docker Hub nginx image already knows how to do the envsubst processing so you do not need to override its command:.
This should leave you with:
version: '3.8'
services:
web:
image: nginx
volumes:
- ./nginx/nginx.conf:/etc/nginx/templates/default.conf.template
environment: { ... }
ports:
- 80:80
depends_on:
- backend
backend:
build: flask
environment: { ... }
volumes:
- flask:/tmp/app_data
restart: unless-stopped
depends_on:
- db
db:
image: mysql
restart: unless-stopped
volumes:
- ./mysql:/docker-entrypoint-initdb.d
- db:/var/lib/mysql
environment: { ... }
ports:
- 3306:3306
volumes:
flask:
db:
I am trying to connect to a postgreQSL-database initialized within a Dockerized Django project. I am currently using the python package psycopg2 inside a Notebook in Jupyter to connect and add/manipulate data inside the db.
With the code:
connector = psycopg2 .connect(
database="postgres",
user="postgres",
password="postgres",
host="postgres",
port="5432")
It raises the following error:
OperationalError: could not translate host name "postgres" to address:
Unknown host
Meanwhile, It connects correctly to the local db named postgres with host as localhost or 127.0.0.1, but it is not the db I want to access. How can I connect from Python to the db? Should I change something in the project setup?
You can find the Github repository here. Many thanks!
docker-compose.yml:
version: '3.8'
services:
web:
restart: always
build: ./web
expose:
- "8000"
links:
- postgres:postgres
- redis:redis
volumes:
- web-django:/usr/src/app
- web-static:/usr/src/app/static
env_file: .env
environment:
DEBUG: 'true'
command: /usr/local/bin/gunicorn docker_django.wsgi:application -w 2 -b :8000
nginx:
restart: always
build: ./nginx/
ports:
- "80:80"
volumes:
- web-static:/www/static
links:
- web:web
postgres:
restart: always
image: postgres:latest
hostname: postgres
ports:
- "5432:5432"
volumes:
- pgdata:/var/lib/postgresql/data/
environment:
POSTGRES_DB: postgres
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
pgadmin:
image: dpage/pgadmin4
depends_on:
- postgres
ports:
- "5050:80"
environment:
PGADMIN_DEFAULT_EMAIL: pgadmin4#pgadmin.org
PGADMIN_DEFAULT_PASSWORD: admin
restart: unless-stopped
redis:
restart: always
image: redis:latest
ports:
- "6379:6379"
volumes:
- redisdata:/data
volumes:
web-django:
web-static:
pgdata:
redisdata:
Dockefile:
FROM python:3.7-slim
RUN python -m pip install --upgrade pip
COPY requirements.txt requirements.txt
RUN python -m pip install -r requirements.txt
COPY . .
Edit
To verify that localhost is not the correct hostname I tried to visualize the tables inside PgAdmin (which connects to the correct host), and psycopg2:
The (correct) tables of pgadmin:
The (incorrect) tables of psycopg2:
I've set my django project and now I'm trying to test it with pytest. What is issue running pytest withing my containers doesn't kill it at the end of the process. So at the end of day I'm stuck with multiple running containers from pytest and often postgreSql connection problems.
My docker-compose file:
version: '3'
services:
license_server:
build: .
command: bash -c "python manage.py migrate && gunicorn LicenseServer.wsgi --reload --bind 0.0.0.0:8000"
depends_on:
- postgres
volumes:
- .:/code
environment:
DATABASE_NAME: "${DATABASE_NAME}"
DATABASE_USER: "${DATABASE_USER}"
DATABASE_PASSWORD: "${DATABASE_PASSWORD}"
DATABASE_PORT: "${DATABASE_PORT}"
DATABASE_HOST: "${DATABASE_HOST}"
env_file: .env
ports:
- "8000:8000"
restart: always
postgres:
build: ./postgres
volumes:
- ./postgres/postgres_data:/var/lib/postgresql/data/
environment:
POSTGRES_PASSWORD: postgres
DATABASE_NAME: "${DATABASE_NAME}"
DATABASE_USER: "${DATABASE_USER}"
DATABASE_PASSWORD: "${DATABASE_PASSWORD}"
DATABASE_PORT: "${DATABASE_PORT}"
DATABASE_HOST: "${DATABASE_HOST}"
command: "-p 8005"
env_file: .env
ports:
- "127.0.0.1:8005:8005"
restart: always
nginx:
image: nginx:latest
container_name: nginx1
ports:
- "8001:80"
volumes:
- .:/code
- ./config/nginx:/etc/nginx/conf.d
depends_on:
- license_server
What I want to achieve is automatically closing containers after the testing process is finished.
When you have restart: always they will just keep restarting when all the processes spawned by the command have exited. Even when you try to kill the running containers yourself they will tend to restart (which can be a nuisance). Try removing restart: always from your service descriptions.
For more info, check the docker-compose.yml reference
I'm trying to create a django/nginx/gunicorn/postgres docker-compose configuration.
Every time I call docker-compose down, I noticed that my postgres db was getting wiped. I did a little digging, and when I call docker-compose up, my named volume is not being created like i've seen in other tutorials.
What am I doing wrong?
Here is my yml file (if it helps, I'm using macOS to run my project)
version: "3"
volumes:
postgres:
driver: local
services:
database:
image: "postgres:latest" # use latest postgres
container_name: database
environment:
- POSTGRES_USER=REDACTED
- POSTGRES_PASSWORD=REDACTED
- POSTGRES_DB=REDACTED
volumes:
- postgres:/postgres
ports:
- 5432:5432
nginx:
image: nginx:latest
container_name: nginx
ports:
- "8000:8000"
volumes:
- ./src:/src
- ./config/nginx:/etc/nginx/conf.d
- ./src/static:/static
depends_on:
- web
migrate:
build: .
container_name: migrate
depends_on:
- database
command: bash -c "python manage.py makemigrations && python manage.py migrate"
volumes:
- ./src:/src
web:
build: .
container_name: django
command: gunicorn Project.wsgi:application --bind 0.0.0.0:8000
depends_on:
- migrate
- database
volumes:
- ./src:/src
- ./src/static:/static
expose:
- "8000"
You need to mount the data directory at /var/lib/postgresql/data
volumes:
- postgres:/var/lib/postgresql/data