We Keep Coding

Dockerize PostgreSQL database with existing data

I'm trying to dockerize my Django project, but my database container seems to not be working. When I initialize docker-compose up the server starts but there is a message telling me I have 53 unapplied migrations (all), and run python manage.py migrate to migrate (like it is a blank database). My docker-compose looks like this:
services:
db:
image: postgres
volumes:
- ./data/db:/var/lib/postgresql/data
environment:
- POSTGRES_DB=database_name
- POSTGRES_USER=database_admin
- POSTGRES_PASSWORD=pskfnbiu42834h
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
environment:
- POSTGRES_DB=database_name
- POSTGRES_USER=database_admin
- POSTGRES_PASSWORD=pskfnbiu42834h
depends_on:
- db
And I changed my settings.py DATABASE to this:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': os.environ.get('POSTGRES_NAME'),
'USER': os.environ.get('POSTGRES_USER'),
'PASSWORD': os.environ.get('POSTGRES_PASSWORD'),
'HOST': 'db',
'PORT': '5432',
}
}
I think the problem might have to do with the volumes part. Docker created a data folder in my project directory with PostgreSQL related files. Where do those /var/lib/ folder live? Anyone can give any orientation?

Authentication failed django

I get an error password authentication failed for user "usertodoproject". Really losing my patience, so far I have done couple of new databases and used fpr example this ALTER USER todouser WITH PASSWORD 'todo'; it did not help either. Any ideas?
*EDIT IT IS DOCKER-COMPOSE FAULT NOT SETTINGS. App runs without docker.
settings.py
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'todoproject',
'USER': 'usertodoproject',
'PASSWORD': 'todoprojectpass',
'HOST': 'db',
'PORT': '5432',
}
}
docker-compose
version: "3"
services:
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
ports:
- 8000:8000
volumes:
- .:/code
depends_on:
- db
restart: always
db:
image: postgres
environment:
- POSTGRES_DB=todoproject
- POSTGRES_USER=usertodoproject
- POSTGRES_PASSWORD=todoprojectpass
ports:
- 5432:5432

You should verify few point that outscope the python programming:
Status of postgresql, is it running with systemctl status postgresql command
verify the listening port 5432 with netstat -plantu command (db is the host ?)
is user usertodoproject is created ?

Error with Docker Compose Django and PostgreSql

I have a problem when I create any object in my django admin. It shows me this error:
LINE 1: SELECT (1) AS "a" FROM "license_portal_client" WHERE
Here is my Dockerfile:
FROM python:3
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
COPY requirements.txt /code/
RUN pip install -r requirements.txt
COPY . /code/
Here is my docker-compose.yml:
version: '3'
services:
db:
image: postgres
environment:
- POSTGRES_DB=postgres
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
web:
build: .
command: python license_portal/manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
My database settings.py and I have added my app on installed apps:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'postgres',
'USER': 'postgres',
'PASSWORD': 'postgres',
'HOST': 'db',
'PORT': 5432,
}
}
And here is my models.py:
from django.db import models
class Person(models.Model):
first_name = models.CharField(max_length=30)
last_name = models.CharField(max_length=30)
And my admin.py:
from django.contrib import admin
from license_portal.models import Person
admin.site.register(Person)

This is likely due to your database not being migrated. Try running the django migrate command from within a django container and stop any existing database connections you may have -- like the web container running the development server.
# Stop all of the running containers to release all db connections
docker-compose stop
# Run the migrate command inside of the django container
docker-compose run web python license_portal/manage.py migrate
# Start all of the containers again to access your dev server
docker-compose up web

How to browse django psql backend from command line?

I'm developing a website with Django 3 (in a docker container) using postgres sql as the backend; i.e. in the project settings file I have:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'postgres',
'USER': 'postgres',
'PASSWORD': 'postgres',
'HOST': 'db',
'PORT': 5432
}
}
I've populated the backend database and can browse the data using the admin. However, Id like to connect to the database via the command line so I can more easily test queries. I have tried connecting to the database the normal way from the command line:
sudo -u postgres psql
postgres=# \c postgres
The problem is that there is no data found:
postgres=# \dt
Did not find any relations.
Since I'm new to docker I thought to try connecting other ways as well; specifically, based on another post I tried:
sudo docker run -d -p 5432 -t postgres/postgresql /bin/su postgres -c '/usr/lib/postgresql/10/bin/postgres -D /var/lib/postgresql/10/main -c config_file=/etc/postgresql/10/main/postgresql.conf'
This throws an error:
pull access denied for psql/postgresql, repository does not exist or may require 'docker login'
Again, I'd like to connect to the database via the command line so I can more easily test queries. Perhaps Im on the right track but assistance would be appreciated.

It is a bad idea to use postgres for the database name, as there is a postgres database used for maintenance by default by PostgreSQL itself. I'd recommend calling the database something like my_project, then creating a service account user my_project_user, and assign a password:
sudo -u postgres psql
postgres=# CREATE USER my_project_user WITH PASSWORD 'blahblah';
postgres=# CREATE DATABASE my_project WITH OWNER my_project_user;
postgres=# \q
Update your Django DATABASES["default"] settings accordingly, and run migrations. After running migrations, you can access your Django database using the following management command:
python manage.py dbshell
You may be able to issue the command above with your current setup, but you may run into problems using postgres as your database name. Good luck!

UPDATE, my docker-compose.yml file had the following service:
db:
image: postgres:11
volumes:
- postgres_data:/var/lib/postgresql/data/
I am able to connect to the Django backend using the following command:
sudo docker-compose exec db psql -U postgres
postgres=# \c postgres
As suggested by #FlipperPA I will start useing a different username and database name

This command below runs(opens) the command-line client for PostgreSQL so that you can test queries:
python manage.py dbshell

Dockerizing Django App

I have a Django application that uses Python 2.7.10 and Django 1.10.3. My Dockerfile contains the following:
FROM python:2.7.10
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
ADD requirements.txt /code/
RUN pip install -r requirements.txt
ADD . /code/
My requirements.txt contains the following:
Django>=1.10,<2.0
Mysql-python
My docker-compose.yml contains the following:
version: '2'
services:
localhost:
image: mysql
restart: always
environment:
MYSQL_DATABASE: 'test'
MYSQL_USER: 'test'
MYSQL_PASSWORD: 'password'
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- localhost
settings.py contains the following for the database:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'test',
'USER': 'test',
'PASSWORD': 'password',
'HOST': 'localhost',
'PORT': '3306',
}
I first do a "% docker-compose build", then I do a "% docker-compose up" and get the following error:
web_1 | django.db.utils.OperationalError: (2002, "Can't connect
to local MySQL server through socket '/var/run/mysqld/mysqld.sock'
I've looked and tried the solutions available on Stackoverflow with no luck. Any help would be much appreciated.

Any help would be much appreciated.
In regards to MySql and Django App relation you have three possible cases:
MySql is residing on same container as Django App (localhost)
MySql is residing on separate container from Django App (hostname is different, can be handled on different topology - same subnet or different etc)
MySql is residing outside of docker ecosystem (bare metal, cloud, whatever...)
Judging from your settings.py and docker-compose.yml you are mixing first two approaches. Now, since you can reference external MySql server from settings.py I'll consider that case trivial, next, cramming MySql on same container as Django App is usually bad idea (if you ever want to scale them separately) so this answer will focus on second case (MySql in separate container) and, for simplicity, on same docker-compose file (you can separate them as well).
Important excerpt from docker-compose.yml is:
version: '2'
services:
# The Database
database:
image: mysql:5.6
volumes:
- dbdata:/var/lib/mysql
environment:
- "MYSQL_DATABASE=test_db"
- "MYSQL_USER=test_user"
- "MYSQL_PASSWORD=some_password"
- "MYSQL_ROOT_PASSWORD=some_root_password"
ports:
- "33061:3306"
container_name: "mysql_database"
# Django App (taken from your example, added env vars as hint)
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
environment:
- "DB_PORT=3306"
- "DB_HOST=database"
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- database
volumes:
dbdata:
With such docker-compose you would have following setting.py config (note that HOST is taking value of docker-compose service entry for database container, port is internal to container not external to docker-host):
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'test_db',
'USER': 'test_user',
'PASSWORD': 'some_password',
'HOST': 'database',
'PORT': '3306',
}
As a sidenote, we usually run database with separate docker-compose.yml from Django App file since in usual dev cycle it is far more frequent to tweak/restart/crash something on Django App container than on database one and we can handle it separately that way, plus such a separately defined database can handle several different DjangoApps each with its own container.