We Keep Coding

Dockerize PostgreSQL database with existing data

I'm trying to dockerize my Django project, but my database container seems to not be working. When I initialize docker-compose up the server starts but there is a message telling me I have 53 unapplied migrations (all), and run python manage.py migrate to migrate (like it is a blank database). My docker-compose looks like this:
services:
db:
image: postgres
volumes:
- ./data/db:/var/lib/postgresql/data
environment:
- POSTGRES_DB=database_name
- POSTGRES_USER=database_admin
- POSTGRES_PASSWORD=pskfnbiu42834h
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
environment:
- POSTGRES_DB=database_name
- POSTGRES_USER=database_admin
- POSTGRES_PASSWORD=pskfnbiu42834h
depends_on:
- db
And I changed my settings.py DATABASE to this:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': os.environ.get('POSTGRES_NAME'),
'USER': os.environ.get('POSTGRES_USER'),
'PASSWORD': os.environ.get('POSTGRES_PASSWORD'),
'HOST': 'db',
'PORT': '5432',
}
}
I think the problem might have to do with the volumes part. Docker created a data folder in my project directory with PostgreSQL related files. Where do those /var/lib/ folder live? Anyone can give any orientation?

Authentication failed django

I get an error password authentication failed for user "usertodoproject". Really losing my patience, so far I have done couple of new databases and used fpr example this ALTER USER todouser WITH PASSWORD 'todo'; it did not help either. Any ideas?
*EDIT IT IS DOCKER-COMPOSE FAULT NOT SETTINGS. App runs without docker.
settings.py
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'todoproject',
'USER': 'usertodoproject',
'PASSWORD': 'todoprojectpass',
'HOST': 'db',
'PORT': '5432',
}
}
docker-compose
version: "3"
services:
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
ports:
- 8000:8000
volumes:
- .:/code
depends_on:
- db
restart: always
db:
image: postgres
environment:
- POSTGRES_DB=todoproject
- POSTGRES_USER=usertodoproject
- POSTGRES_PASSWORD=todoprojectpass
ports:
- 5432:5432

You should verify few point that outscope the python programming:
Status of postgresql, is it running with systemctl status postgresql command
verify the listening port 5432 with netstat -plantu command (db is the host ?)
is user usertodoproject is created ?

Django and Postgres problems with dockerizing

Hello I can not dockerize my django app because I got an error -
listen tcp4 0.0.0.0:5433: bind: address already in use
From the other hand, when I "kill" 5433 port in ubuntu terminal I get this error
Is the server running on host "localhost" (::1) and accepting
web_1 | TCP/IP connections on port 5433?
What can I do to solve this problem and dockerize successfully my app?
Dockerfile
FROM python:3
RUN adduser --system --no-create-home django
ENV PYTHONUNBUFFERED=1
WORKDIR /code
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
ENV PYTHONPATH /code
EXPOSE 8000
USER django
CMD ["./main.py"]
docker-compose
version: "3"
services:
db:
image: postgres
volumes:
- ./data/db:/var/lib/postgresql/data
environment:
- POSTGRES_DB=fitshop
- POSTGRES_USER=fituser
- POSTGRES_PASSWORD=fitpass
ports:
- "5433:5433"
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
restart: always
ports:
- "8000:8000"
depends_on:
- db
settings.py
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'fitshop',
'USER': 'fituser',
'PASSWORD': 'fitpass',
'HOST': 'localhost',
'PORT': '5433',
}
}

You should use "db" as host here. In a docker-compose setup each service is started in a separate Container and the /etc/hosts config of all containers is modified so, that each container can find the other containers via the service names.

I`ve got it!. In 'localhost" port of postgres was 5433 but for 'db' as HOST it was default, 5432.

how to connect postgresql to aws RDS using Django

I am unable to connect my postgreSQL database to AWS RDS while using Django.
I am not sure what i am doing wrong but i have the configuration below:
Django
docker-compose
postgres
#DJANGO Settings
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': "dbtest",
'USER': "mastername",
'PASSWORD': "somepassword",
'HOST': 'dbtest.zmrHnee2mhLunL.us-east-2.rds.amazonaws.com',
'PORT': 5432
}
}
docker-compose
version: "3.7"
services:
backend:
build: ./backend
container_name: backend
stdin_open: true
tty: true
restart: "on-failure"
env_file:
.env
volumes:
- ./backend:/backend
- ./backend/static:/backend/static
ports:
- 8000:8000
depends_on:
- db
links:
- db:db
networks:
- db_network
db:
image: postgres:11.5-alpine
container_name: db
restart: always
environment:
- POSTGRES_DB=dbtest
- POSTGRES_USER=mastername
- POSTGRES_PASSWORD=somepassword
volumes:
- postgres_data:/var/lib/postgresql/data/
ports:
- 5432:5432
networks:
- db_network
networks:
db_network:
driver: bridge
volumes:
postgres_data:
after starting the container docker-compose up -d --build
Now when i check docker-compose logs
I see the traceback below:
File "/usr/local/lib/python3.8/site-packages/psycopg2/__init__.py", line 127, in connect
backend | conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
backend | django.db.utils.OperationalError: could not translate host name "dbtest.zmrHnee2mhLunL.us-east-2.rds.amazonaws.com" to address: Name or service not known
and after creating another db instance and allowing it to be publicly accessible
File "/usr/local/lib/python3.8/site-packages/psycopg2/__init__.py", line 127, in connect
backend | conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
backend | django.db.utils.OperationalError: FATAL: database "dbtest" does not exist
security group
I have been trying to debug for hours, any help would be really helpful

Dockerizing Django App

I have a Django application that uses Python 2.7.10 and Django 1.10.3. My Dockerfile contains the following:
FROM python:2.7.10
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
ADD requirements.txt /code/
RUN pip install -r requirements.txt
ADD . /code/
My requirements.txt contains the following:
Django>=1.10,<2.0
Mysql-python
My docker-compose.yml contains the following:
version: '2'
services:
localhost:
image: mysql
restart: always
environment:
MYSQL_DATABASE: 'test'
MYSQL_USER: 'test'
MYSQL_PASSWORD: 'password'
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- localhost
settings.py contains the following for the database:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'test',
'USER': 'test',
'PASSWORD': 'password',
'HOST': 'localhost',
'PORT': '3306',
}
I first do a "% docker-compose build", then I do a "% docker-compose up" and get the following error:
web_1 | django.db.utils.OperationalError: (2002, "Can't connect
to local MySQL server through socket '/var/run/mysqld/mysqld.sock'
I've looked and tried the solutions available on Stackoverflow with no luck. Any help would be much appreciated.

Any help would be much appreciated.
In regards to MySql and Django App relation you have three possible cases:
MySql is residing on same container as Django App (localhost)
MySql is residing on separate container from Django App (hostname is different, can be handled on different topology - same subnet or different etc)
MySql is residing outside of docker ecosystem (bare metal, cloud, whatever...)
Judging from your settings.py and docker-compose.yml you are mixing first two approaches. Now, since you can reference external MySql server from settings.py I'll consider that case trivial, next, cramming MySql on same container as Django App is usually bad idea (if you ever want to scale them separately) so this answer will focus on second case (MySql in separate container) and, for simplicity, on same docker-compose file (you can separate them as well).
Important excerpt from docker-compose.yml is:
version: '2'
services:
# The Database
database:
image: mysql:5.6
volumes:
- dbdata:/var/lib/mysql
environment:
- "MYSQL_DATABASE=test_db"
- "MYSQL_USER=test_user"
- "MYSQL_PASSWORD=some_password"
- "MYSQL_ROOT_PASSWORD=some_root_password"
ports:
- "33061:3306"
container_name: "mysql_database"
# Django App (taken from your example, added env vars as hint)
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
environment:
- "DB_PORT=3306"
- "DB_HOST=database"
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- database
volumes:
dbdata:
With such docker-compose you would have following setting.py config (note that HOST is taking value of docker-compose service entry for database container, port is internal to container not external to docker-host):
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'test_db',
'USER': 'test_user',
'PASSWORD': 'some_password',
'HOST': 'database',
'PORT': '3306',
}
As a sidenote, we usually run database with separate docker-compose.yml from Django App file since in usual dev cycle it is far more frequent to tweak/restart/crash something on Django App container than on database one and we can handle it separately that way, plus such a separately defined database can handle several different DjangoApps each with its own container.