I have a Django app that runs locally with no problems.
Now, I want to get a docker image of my app but when I try to build it, it gives me the following error:
django.db.utils.OperationalError: could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
could not connect to server: Cannot assign requested address
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 5432?
I am new on Django and Docker development and i was looking for similar questions but none answer solved my problem.
I'll show you my Dockerfile, I have copied the Postgres commands from another project that does work:
FROM python:3.8
RUN apt-get update
RUN mkdir /project
WORKDIR /project
RUN apt-get install -y vim
COPY requirements.txt /project/
RUN pip install -r requirements.txt
COPY . /project/
# Install postgresql
RUN apt install -y postgresql postgresql-contrib
RUN service postgresql start
# Run the rest of the commands as the ``postgres`` user created by the ``postgres-9.3`` package when it was ``apt-get installed``
USER postgres
# Create a PostgreSQL role
RUN /etc/init.d/postgresql start &&\
psql --command "CREATE USER admin WITH SUPERUSER PASSWORD 'passwd';" &&\
createdb -O admin plataforma
USER root
# setup postgresql
RUN sed -i "/^#listen_addresses/i listen_addresses='*'" /etc/postgresql/13/main/postgresql.conf
RUN sed -i "/^# DO NOT DISABLE\!/i # Allow access from any IP address" /etc/postgresql/13/main/pg_hba.conf
RUN sed -i "/^# DO NOT DISABLE\!/i host all all 0.0.0.0/0 md5\n\n\n" /etc/postgresql/13/main/pg_hba.conf
# running commands of my app
RUN python manage.py makemigrations accounts
RUN python manage.py sqlmigrate accounts 0001
RUN python manage.py migrate
RUN python manage.py inicializar
# Expose some ports
EXPOSE 22 5432 8080 8009 8000
# volumes
VOLUME ["/var/lib/postgresql/12/main"]
# Default command
CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"]
Here is my requirements.txt file:
#requirements.txt
Django==3.2.8
djangorestframework==3.9.1
gunicorn==19.9.0
pandas==1.3.3
path==16.2.0
six==1.14.0
Pillow==7.0.0
psycopg2>=2.8
django-environ==0.8.1
Here is my settings.py file (the databases fragment):
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': env('POSTGRESQL_NAME'),
'USER': env('POSTGRESQL_USER'),
'PASSWORD': env('POSTGRESQL_PASS'),
'HOST': env('POSTGRESQL_HOST'),
'PORT': env('POSTGRESQL_PORT'),
}
}
And here is my .env file:
POSTGRESQL_NAME=plataforma
POSTGRESQL_USER=admin
POSTGRESQL_PASS=passwd
POSTGRESQL_HOST=localhost
POSTGRESQL_PORT=5432
DEBUG=True
Some people told me that I could make a docker-compose file so I erased the postgres install commands from Dockerfile and made this:
version: "3.3"
services:
db:
image: postgres
volumes:
- ./data/db:/var/lib/postgresql/data
ports:
- "5432:5432"
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
environment:
- POSTGRES_NAME=plataforma
- POSTGRES_USER=admin
- POSTGRES_PASSWORD=passwd
depends_on:
- db
But this doesn't work either.
Footnote: If I use the Django's default database (SQLite) and (obviously) erase postgress commands from Dockerfile, I can build the Docker image without problems, and, again, this app with postgres works good if I run it locally. So, something is happening with Docker + postgres and I don't know what to do.
Somebody can help me? Thank you!
Edit: I erased the migrations commands from Dockerfile and replaced environ var of POSTGRESQL_HOST to db and when I run $ sudo docker-compose run web python manage.py runserver . the image is created but the container not, and when I try to run a container with that image i get the following error:
django.db.utils.OperationalError: could not translate host name "db" to address: Name or service not known
Sorry my english, I still learning it too.
EDIT AGAIN: I finally solved this issue in this question
Thanks for the help!
While creating a superuser your command is
CREATE USER admin WITH SUPERUSER PASSWORD 'administrador'
And in your env file you are using passwd as password.
Change your env from
POSTGRESQL_PASS=passwd
To this
POSTGRESQL_PASS=administrador
Related
This question already has an answer here:
Docker-compose: App container can't connect to Postgres
(1 answer)
Closed yesterday.
I am trying to run my django/postgres application with docker compose. When I run docker compose up -d I get the the following logs on my postgres container running on port 5432:
2023-02-18 00:10:25.049 UTC [1] LOG: starting PostgreSQL 13.8 on aarch64-unknown-linux-musl, compiled by gcc (Alpine 11.2.1_git20220219) 11.2.1 20220219, 64-bit
2023-02-18 00:10:25.049 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2023-02-18 00:10:25.049 UTC [1] LOG: listening on IPv6 address "::", port 5432
2023-02-18 00:10:25.052 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2023-02-18 00:10:25.054 UTC [22] LOG: database system was shut down at 2023-02-18 00:10:06 UTC
2023-02-18 00:10:25.056 UTC [1] LOG: database system is ready to accept connections
It appears my postgres container is working properly. However my python/django container has the following logs:
django.db.utils.OperationalError: could not connect to server: Connection refused
Is the server running on host "127.0.0.1" and accepting
TCP/IP connections on port 5432?
docker-compose.yml
version: '3.8'
services:
web:
build: .
command: sh -c "python manage.py runserver 0.0.0.0:8000"
volumes:
- .:/code
ports:
- "8000:8000"
env_file:
- ./xi/.env
depends_on:
- db
db:
image: postgres:13-alpine
environment:
- POSTGRES_DB=***
- POSTGRES_USER=***
- POSTGRES_PASSWORD=***
volumes:
- dev-db-data:/var/lib/postgresql/data
ports:
- 5432:5432
volumes:
dev-db-data:
Dockerfile:
FROM python:3.8
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1
WORKDIR /code
# install psycopg2 dependencies
RUN apt-get update
RUN apt-get install nano python3-dev libpq-dev -y
COPY requirements-prod.txt /code/
RUN pip install -r requirements.txt
COPY . /code/
I must be missing something small that allows the python container to communicate with the postgres container.
Also a few additional questions:
What does it mean that the container is "listening on IPv4 address '0.0.0.0', port 5432"? To my understanding 0.0.0.0 encapsulates all ip addresses including 127.0.0.1. So, in this case that shouldn't be an issue (correct me if I'm wrong)
I have been struggling with this for a few days. I have followed the getting started docs as well as the python usage guides on the docker docs, and it appears that I feel that I understand everything, but I am unable to debug my containers efficiently. What additional supplemental knowledge can I acquire that helps me debug a container with the same level of comfortability as I would a python script?
I tried a few things:
swapping env_file with the credentials hard coded in
changing python with python3
removing sh -c
I tried building my database first with docker-compose up -d --build db and then building my web app with docker-compose up -d --build web and the issue persisted.
I tried everything with the environment variables and it appears improper credentials is not the issue. Running python manage.py runserver without docker it successfully connects to the database. There are some similar stack overflow questions, but I have tried their solutions and they do not work.
Part of my issue is I don't know what to try and how to efficiently debug docker containers yet (hence the question above).
What have you set as your HOST variable in DATABASES['default'] in settings.py? If it's '127.0.0.1', try changing to 'db' to match the container service name.
I'm getting this error when running docker-compose up and I don't know why, tried researching it but all the solutions that I found didn't work. If any knows it would be awesome if you can share it. Thanks!
ERROR
sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on 'mysql' ([Errno -2] Name or service not known)")
This is my docker-compose.yml file. It has the 2 images that it needs to build.
docker-compose.yml
version: "3.7"
services:
web:
build: .
depends_on:
- mysql
ports:
- 5000:5000
links:
- mysql
environment:
MYSQL_HOST: mysql
MYSQL_USER: root
MYSQL_PASSWORD: 12345678
MYSQL_DB: flaskmysql
mysql:
image: mysql:5.7
ports:
- "32000:3306"
volumes:
- ./mysql-data:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: 12345678
MYSQL_DATABASE: flaskmysql
volumes:
mysql-data:
This is my Dockerfile that has all the steps to run my application.
Dockerfile
FROM python:3.9-slim-buster
RUN apt-get update && apt-get install -y git python3-dev gcc gfortran libopenblas-dev liblapack-dev \
&& rm -rf /var/lib/apt/lists/*
COPY requirements.txt .
RUN pip install --upgrade -r requirements.txt
COPY app app/
RUN python app/server.py
EXPOSE 5000
CMD ["python", "app/server.py", "serve"]
Here I've got the lines of code that tries to make a connection to the service that docker-compose created with the image given.
Server.py
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI']= 'mysql+pymysql://root:12345678#mysql:3306/flaskmysql'
app.config['SQLALCHEMY_TRACK_MODIFICATIONS']= False
I believe you are running into a race condition here. Even though you've specified the depends_on dependency for your application, note that docker-compose will not 'wait' for the database to be available before proceeding to the next step. That is because docker-compose doesn't "know" what it means for this service to become "ready".
This mean, as long as the container is 'Running' (can be still initialising the database), docker-compose will move on to try and build the image for your application, and running the application, which attempts to connect with a database not ready. You can do two things here:
Add a waiting loop in your application to attempt retries. RECOMMENDED.
Add a solution like wait-for-it to your docker compose setup.
You can find more details in this docker documentation page on startup order setting.
I suggest add a simple retry loop in your application :)
I am trying to configure Django and MySql application with Docker containers.
For Django I am using python:3.7-slim image and for MySql mysql:5.6.
When I run docker-compose up it returns an error stated below -
ERROR: for app_mysql_db_1 Cannot start service mysql_db: driver failed programming external connectivity on endpoint app_mysql_db_1 (c647d4793a198af2c09cc52d08191fb2cd984025ad0a61434ad1577d9dcccebe): Error starting userland proxy: listen tcp 0.0.0.0:3306: bind: address already in use
I run command docker ps -a to check docker status and found that mysql container was created but the python container status was exited.
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7d91795e0bae mysql:5.6 "docker-entrypoint.s…" 15 seconds ago Created app_mysql_db_1
fa0419ad0f21 e0bf94710555 "/bin/sh -c 'adduser…" 2 minutes ago Exited (1) 2 minutes ago pedantic_faraday
can someone rewrite or suggest the modification for the configurations.
Dockerfile
FROM python:3.7-slim
ENV PYTHONUNBUFFERED 1
RUN apt-get update
RUN apt-get install python3-dev default-libmysqlclient-dev gcc -y
COPY ./requirements.txt /requirements.txt
RUN pip install -r /requirements.txt
RUN mkdir /app
WORKDIR /app
COPY . /app
docker-compose.yaml
version: "3"
services:
eitan-application:
restart: always
build:
context: .
ports:
- "8000:8000"
volumes:
- ./eitan:/app
command: >
sh -c "python3 manage.py runserver 0.0.0.0:8000
&& python3 manage.py makemigrations
&& python3 manage.py migrate"
depends_on:
- mysql_db
mysql_db:
image: mysql:5.6
command: mysqld --default-authentication-plugin=mysql_native_password
volumes:
- "./mysql:/var/lib/mysql"
ports:
- "3306:3306"
restart: always
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=root
- MYSQL_USER=root
- MYSQL_PASSWORD=root
settings.py
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'my-app-db',
'USER': 'root',
'PASSWORD': 'root',
'HOST': 'mysql_db',
'PORT': 3307,
}
}
bind: address already in use suggests that you have some local database running. If you don't need to access database outside of docker-compose network don't expose port 3306. So I'd try to test it without
...
ports:
- "3306:3306"
...
Also in settings.py you connect to port mysql_db:3307 so change it to default port 3306.
Even if you expose database port to some other port on localhost then settings.py connects using mysql_db network, so you shouldn't change this port in django settings.
I have an app developed in Django (2.2.7) with python (3.8.0), Docker (19.03.5) and docker-compose (1.25.2) running in Windows 10 pro. I want to Dockerize it with changing the sqlite3 database for a MySQL database. I've already write this Dockerfile:
FROM python:3.7
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
ADD . /code/
RUN pip install --upgrade pip && pip install -r requirements.txt
RUN pip install mysqlclient
COPY . /code/
And this docker-compose.yml file:
version: '3'
services:
db:
image: mysql:5.7
ports:
- '3306:3306'
environment:
MYSQL_DATABASE: 'my-app-db'
MYSQL_USER: 'root'
MYSQL_PASSWORD: 'password'
MYSQL_ROOT_PASSWORD: 'password'
volumes:
- .setup.sql:/docker-entrypoint-initbd.d/setup.sql
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
links:
- db
Also I have change the default database configurations in settings.py for this:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'my-app-db',
'USER': 'root',
'PASSWORD': 'password',
'HOST': 'db',
'PORT': 3306,
}
}
After all of this the docker compose works and the app starts, but the problem is that the tables in the database are not created. I've tried with these How do I add a table in MySQL using docker-compose, Seeding a MySQL DB for a Dockerized Django App or this Seeding a MySQL DB for a Dockerized Django App but I couldn't fix it yet.
How can I create the required tables in the MySQL db container while runing the docker-compose? Must I add every single table by hand or there is a way to do it from the django app automatically?
Thanks
Hi i think this answer helps you
##1.- Reset all your migrations
find . -path "*/migrations/*.py" -not -name "__init__.py" -delete
find . -path "*/migrations/*.pyc" -delete
##2.- See and apply your migrations again
python manage.py showmigrations
python manage.py makemigrations
python manage.py migrate
You should not reset your migrations unless you want to wipe all of the data completely and start over. Your migrations should exist beforehand. So if you dont mind about old migrations you can delete them and use python manage.py makemigrations and then proceed to following steps:
So if your applications starts at the moment, we need to update your docker-compose file in a way that it uses entrypoint.sh. An ENTRYPOINT allows you to configure a container that will run as an executable.
First things first, create your entrypoint.sh file on the same level as docker-compose.yaml.
Next, don't forget to add chmod +x entrypoint.sh so entrypoint can be executed
entrypoint.sh file:
#!/bin/bash
set -e
echo "${0}: running migrations."
python manage.py migrate --noinput
echo "${0}: collecting statics."
python manage.py collectstatic --noinput
python manage.py runserver 0.0.0.0:8000
Afterwards update your docker-compose.yaml file. Change your command line to:
command:
- /bin/sh
- '-c'
- '/code/entrypoint.sh'
Additionally you should store all of your pip requirements in requirements.txt file and in your Dockerfile you should run pip install -r requirements.txt
You can dump your pip requirements with a command pip freeze > requirements.txt
I was reading an article in here which is about setting up project using docker, django and mysql together.
these are my files in project:
Dockerfile
FROM python:3.7
MAINTAINER masoud masoumi moghadam
ENV PYTHONUNBUFFERED 1
RUN mkdir /app
WORKDIR /app
ADD . /app
ADD requirements.txt /app/requirements.txt
RUN pip install --upgrade pip && pip install -r requirements.txt
Docker-compose
version: "3"
services:
app:
build:
context: .
ports:
- "8000:8000"
volumes:
- ./app:/app
command: >
sh -c "python manage.py runserver 0.0.0.0:8000"
environment:
- DB_HOST=localhost
- DB_NAME=contact_list
- DB_USER=root
- DB_PASS=secretpassword
depends_on:
- db
db:
image: mysql:5.7
environment:
- MYSQL_DATABASE=contact_list
- MYSQL_USER=root
- MYSQL_PASSWORD=secretpassword
requirements
Django>=2.0,<3.0
djangorestframework<3.10.0
mysqlclient==1.3.13
django-mysql==2.2.0
and also this settings in my setting.py:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'HOST': os.environ.get('DB_HOST'),
'NAME': os.environ.get('DB_NAME'),
'USER': os.environ.get('DB_USER'),
'PASSWORD': os.environ.get('DB_PASS')
}
}
When I use docker-compose build I face no problem and everything is just fine. Then I run service mysql start. I can assure that mysql service is in run and works because I have access to datasets. The problem occurs when I do the migration using this command docker-compose run app sh -c "python manage.py makemigrations core" I get this error:
django.db.utils.OperationalError: (2002,
"Can't connect to local MySQL server through socket
'/var/run/mysqld/mysqld.sock' (2)")
When I change localhost to 127.0.0.1 I get this error:
django.db.utils.OperationalError:
(2002, "Can't connect to MySQL
server on '127.0.0.1' (115)")
I spent 20 hours looking for the best possible configuration for these technologies, But I still can't figure out anything. I also used python-alpine but I still could not find it useful for project because I had the same mysql dependencies problem when I was trying to do docker build. Does anybody have the same experience? I would appreciate if you could help me here.
Your problem is that you use localhost as the host of mysql in django's config.
But docker containers have their own IP, they are not localhost.
So first in your docker-compose file, name your containers :
db:
image: mysql:5.7
container_name: db
...
Then in your django settings, set your db HOST to your db container name : "db" :
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'HOST': 'db',
'PORT' : 3306, # (?)
...
Also you are missing the db 'PORT' in django settings, I think that for Mysql it is 3306 (I've added it above).
According to this beautifully organized article, I found out this configurations will have no issues running on a server(changed mysql service to postgress service):
Dockerfile
# pull official base image
FROM python:3.8.0-alpine
# set work directory
WORKDIR /usr/src/app
# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# install psycopg2 dependencies
RUN apk update \
&& apk add postgresql-dev gcc python3-dev musl-dev
# install dependencies
RUN pip install --upgrade pip
COPY requirements.txt /usr/src/app/requirements.txt
RUN pip install -r requirements.txt
# copy entrypoint.sh
COPY entrypoint.sh /usr/src/app/entrypoint.sh
# copy project
COPY . /usr/src/app/
# run entrypoint.sh
ENTRYPOINT ["/usr/src/app/entrypoint.sh"]
Docker-compose.yml
version: '3.7'
services:
web:
build: app
# container_name: web
command: python manage.py runserver 0.0.0.0:8000
restart: always
volumes:
- ./app/:/usr/src/app/
ports:
- 8000:8000
env_file:
- .env.dev
depends_on:
- db
db:
image: postgres:12.0-alpine
# container_name: postgres
restart: always
volumes:
- postgres_data:/var/lib/postgresql/data/
environment:
- POSTGRES_USER=user
- POSTGRES_PASSWORD=123456
- POSTGRES_DB=contact_list
requirements
Django==2.2.6
gunicorn==19.9.0
djangorestframework>=3.9.2,<3.10.0
psycopg2-binary==2.8.3
settings
DATABASES = {
'default': {
'ENGINE': os.environ.get('SQL_ENGINE', "django.db.backends.sqlite3"),
'NAME': os.environ.get("SQL_DB_NAME", os.path.join(BASE_DIR, "db.sqlite3")),
'USER': os.environ.get("SQL_USER", "user"),
'PASSWORD': os.environ.get("SQL_PASSWORD", "password"),
'HOST': os.environ.get("SQL_HOST", "localhost"),
'PORT': os.environ.get("SQL_PORT", "5432"),
}
}
I added a file named as .env.dev which is for better access to environment variables:
DEBUG=1
SECRET_KEY=123456
DJANGO_ALLOWED_HOSTS=localhost 127.0.0.1 [::1]
SQL_ENGINE=django.db.backends.postgresql
SQL_HOST=db
SQL_DB_NAME=contact_list
SQL_USER=user
SQL_PASSWORD=123456
SQL_PORT=5432
DATABASE=postgres
first try to login to mysql with provided credentials
docker exec -it <CONTAINER_ID> /bin/sh
Inside the docker please login to mysql via command line
mysql -uroot -psecretpassword