We Keep Coding

(pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on 'mysql' ([Errno -3] Temporary failure in name resolution

I am trying to Dockerize a FastAPI app that uses MYSQL and Seleniun.
I am having issues with connecting MYSQL with the FASTAPI app in the Docker.
I have tried to establish connection with MYSQL container using MYSQL Workbench which worked well using 'localhost' as the host. However, when I try to run the fastapi container which should connect with MySqL database, I am having this error:
sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on 'mysql' ([Errno -3] Temporary failure in name resolution
Here is docker-compose.yml:
version: '3'
services:
chrome:
build: .
image: selenium/node-chrome:3.141.59-20210929
ports:
- "4444:4444"
- "5900:5900"
volumes:
- "/dev/shm:/dev/shm"
networks:
- selenium
mysql:
image: mysql:8.0
command: --default-authentication-plugin=mysql_native_password
restart: always
environment:
- MYSQL_ROOT_PASSWORD=admin
- MYSQL_DATABASE=autojob
- MYSQL_USER=user
- MYSQL_PASSWORD=4444
healthcheck:
test: ["CMD", "mysqladmin" ,"ping", "-h", "localhost"]
timeout: 20s
retries: 10
volumes:
- ./init:/docker-entrypoint-initdb.d
- autojob:/var/lib/mysql
ports:
- "3307:3306"
expose:
- "3307"
app:
build: .
restart: on-failure
container_name: "autojobserve_container"
command:
uvicorn autojobserve.app:app --host 0.0.0.0 --port 8000 --reload
ports:
- 8000:8000
volumes:
- "./:/app"
networks:
- selenium
depends_on:
mysql:
condition: service_healthy
volumes:
autojob: {}
networks:
selenium:
Here is the line that connects with MYSQL in FastAPI:
engine = create_engine("mysql+pymysql://user:4444#mysql:3307/autojob")
In DockerDesktop, it shows that Mysql container is ready for connection too:
2022-11-08T11:49:26.334069Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Bind-address: '::' port: 33060, socket: /var/run/mysqld/mysqlx.sock
2022-11-08T11:49:26.334869Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.31' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server - GPL.
2022-11-08 11:49:14+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.31-1.el8 started.
2022-11-08 11:49:14+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2022-11-08 11:49:14+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.31-1.el8 started.
'/var/lib/mysql/mysql.sock' -> '/var/run/mysqld/mysqld.sock'
What possibly could be wrong?
Note: Everything works well before dockerizing.

Your app container declares networks: [selenium]. The mysql container doesn't have a networks: block at all, so Compose automatically inserts networks: [default]. Since the two containers aren't on the same Docker network they can't communicate with each other, and one of the ways you see that is with the DNS-resolution issue you're seeing.
The setup I'd recommend here is to delete all of the networks: blocks in the whole file. Compose will automatically create the default network and attach containers to it, and for most applications this is a correct setup.
(You also do not need the obsolete expose: option, or to manually specify container_name:. You should not need to use volumes: to inject code into your container or command: either, the code and its default command should generally be specified in the Dockerfile.)

psycopg2.OperationalError: could not translate host name to address using psycopg2 and docker

I've been trying for a few hours now but no solution from similar asked questions seem to work for me...
I am using docker-compose to setup a postgresql database and run a python webserver from where I want to connect to my postgressql database (so it's running inside the container)
version: '3.8'
services:
database:
container_name: database
hostname: database
image: postgres
restart: always
environment:
POSTGRES_DB: mydatabase
POSTGRES_USER: postgres
POSTGRES_PASSWORD: password
volumes:
- postgres:/pgdata
- ./application/ressources/fixtures.sql:/docker-entrypoint-initdb.d/fixtures.sql
ports:
- "5432:5432"
application:
container_name: application
build: .
ports:
- "5001:5001"
volumes:
- ./application:/application
restart: always
depends_on:
- database
volumes:
postgres:
I trying to connect as follows ( I have read that despite the depends on in my dockerfile the database needs some more time until it can accept connections so i added a retry logic):
retries = 0
while retries < 5:
retries = retries + 1
self.conn = psycopg2.connect(user='postgres', password='password',
host='database', port="5432", database='mydatabase')
if not self.conn:
logging.info("retry to connect")
sleep(5)
The weird thing is that when running it with docker-compose -f docker-compose.yml up everything works fine.
But when I built the image (docker build -t myapp:0.1) and run it (docker run myapp:0.1) it gives me the following error:
File "/application/libraries/database.py", line 18, in establishConnection
self.conn = psycopg2.connect(user=CONFIG.DATABASE_USER, password=CONFIG.DATABASE_PASSWORD,
File "/usr/local/lib/python3.9/site-packages/psycopg2/__init__.py", line 127, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: could not translate host name "database" to address: Name or service not known
I've read that when using docker-compose a single network is created, so this can't be the error here i guess Docker Compose doku
Thanks in advance,
Jacky

If you run docker run on an image, it does only what's on that command line and no more. In particular, the plain docker commands don't know about docker-compose.yml and any of the settings you might specify there.
The short answer is to always use docker-compose up to launch the containers.
In principle you could translate the docker-compose.yml file into explicit docker commands. At a minimum you'd need to manually create a Docker network and specify the container names:
docker network create app-net
docker run -d --net app-net --name database -p 5432:5432 postgres
docker run -d --net app-net -e PGHOST=database -p 5001:5001 myapp:0.1
This hasn't included the other options in the Compose setup, though, notably database persistence. There are enough settings that you'd want to write them down in a file, and at that point you've basically reconstructed the Compose environment.

unable to access mysql db using python (both running on different containers)

I am trying to connect to mysql db using a python program. When run locally it works.
But while dockerizing the application I created, one container is for the python code and the other for the mysql db, when ran i this manner it fails to connect.
Python_code:
db.bind(provider='mysql', user='docker_root', password='password', host='db', database=database, port = 3306)
docker-compose:
version: "3"
services:
app:
image: app:latest
links:
- db
ports:
- "8001:8081"
environment:
- DB_HOST= db
db:
image: mysql:5.7.26
restart: always
environment:
MYSQL_DATABASE: 'my_db'
MYSQL_USER: 'docker_root'
MYSQL_PASSWORD: 'password'
MYSQL_ROOT_PASSWORD: 'password'
ports:
- "3306:3306"
volumes:
- ./DB_config/:/etc/mysql/mysql.conf.d
And the docker-compose up fails with the eroor:
pony.orm.dbapiprovider.OperationalError: (2003, "Can't connect to
MySQL server on 'db' ([Errno 111] Connection refused)")
Where am I going wrong? Please advise!

I would recommend you to exit your application in case it cannot connect to MySQL and set the restart policy to always, because depends_on does not guarantee that MySQL will be totally up when app starts but it is good to have it there.
version: "3"
services:
app:
image: app:latest
restart: always
links:
- db
ports:
- "8001:8081"
environment:
- DB_HOST= db
depends_on:
- db
db:
image: mysql:5.7.26
restart: always
environment:
MYSQL_DATABASE: 'my_db'
MYSQL_USER: 'docker_root'
MYSQL_PASSWORD: 'password'
MYSQL_ROOT_PASSWORD: 'password'
ports:
- "3306:3306"
volumes:
- ./DB_config/:/etc/mysql/mysql.conf.d
And your application code should be something like:
try:
db.bind(provider='mysql', user='docker_root', password='password', host='db', database=database, port = 3306)
except:
# write some logs
exit(1)

Try using container port 3306 -
db.bind(provider='mysql', user='docker_root', password='password', host='db', database=database, port = 3306)
Also, add depends_on attribute, you can remove the links attribute -
depends_on:
- db

YOu can use depends_on flag as mentioned in accepted answer. If it does not solve your problem them use this approach.
After starting the container, your server will try to connect to the database server. sometimes database server may take some time to boot up and in this window, if the server tries to connect to database server it will face problems.
Try adding logic to reconnect database server after few seconds if the connection fails.
try{
connectToDatabase();
}catch(error){
waitForHalfMinute();
connectToDatabase();
}

Can not connect PostgreSQL database from docker to python

I am trying to use Postgresql with python. I have used the following docker compose the file.
version: '3.1'
services:
db:
image: postgres
restart: always
environment:
POSTGRES_PASSWORD: admin_123
POSTGRES_USER: admin
adminer:
image: adminer
restart: always
ports:
- 8080:8080
With the following code, I am trying to connect with the database.
conn = psycopg2.connect(
database = "db_test",
user ="admin",
password = "admin_123",
host = "db"
)
But I am getting this error.
OperationalError: could not translate host name "db" to address:
nodename nor servname provided, or not known
What I am doing wrong ?

You need to expose the BD port in the docker compose like this :
db:
image: postgres
restart: always
environment:
POSTGRES_PASSWORD: admin_123
POSTGRES_USER: admin
ports:
- "5432:5432"
And then connect with localhost:5432

Another possible scenario,
Check if ports have been used or not by other docker container.
Use command:
$ docker container ls --format "table {{.ID}}\t{{.Names}}\t{{.Ports}}" -a

Here is my docker-compose.yml
$ cat docker-compose.yml
version: '3.1' # specify docker-compose version
services:
dockerpgdb:
image: postgres
ports:
- "5432:5432"
restart: always
environment:
POSTGRES_PASSWORD: Password
POSTGRES_DB: dockerpgdb
POSTGRES_USER: abcUser
volumes:
- ./data:/var/lib/postgresql%
Now in PgAdmin4 you can setup a new server as below to test the connection:
host: localhost
port: 5432
maintenance database: postgres
username: abcUser
password: Password

Docker cannot connect application to MySQL

I am trying to run integration tests (in python) which depend on mysql. Currently they depend on SQL running locally, but I want them to depend on a MySQL running in docker.
Contents of Dockerfile:
FROM continuumio/anaconda3:4.3.1
WORKDIR /opt/workdir
ADD . /opt/workdir
RUN python setup.py install
Contents of Docker Compose:
version: '2'
services:
mysql:
image: mysql:5.6
container_name: test_mysql_container
environment:
- MYSQL_ROOT_PASSWORD=test
- MYSQL_DATABASE=My_Database
- MYSQL_USER=my_user
- MYSQL_PASSWORD=my_password
volumes:
- db_data:/var/lib/mysql
restart: always
expose:
- "3306"
my_common_package:
image: my_common_package
depends_on:
- mysql
restart: always
links:
- mysql
volumes:
db_data:
Now, I try to run the tests in my package using:
docker-compose run my_common_package python testsql.py
and I receive the error
pymysql.err.OperationalError: (2003, "Can't connect to MySQL server on
'localhost' ([Errno 99] Cannot assign requested address)")

docker-compose will by default create virtual network were all the containers/services in the compose file can reach each other by an IP address. By using links, depends_on or network aliases they can reach each other by host name. In your case the host name is the service name, but this can be overridden. (see: docs)
Your script in my_common_package container/service should then connect to mysql on port 3306 according to your setup. (not localhost on port 3306)
Also note that using expose is only necessary if the Dockerfile for the service don't have an EXPOSE statement. The standard mysql image already does this.
If you want to map a container port to localhost you need to use ports, but only do this if it's necessary.
services:
mysql:
image: mysql:5.6
container_name: test_mysql_container
environment:
- MYSQL_ROOT_PASSWORD=test
- MYSQL_DATABASE=My_Database
- MYSQL_USER=my_user
- MYSQL_PASSWORD=my_password
volumes:
- db_data:/var/lib/mysql
ports:
- "3306:3306"
Here we are saying that port 3306 in the mysql container should be mapped to localhost on port 3306.
Now you can connect to mysql using localhost:3306 outside of docker. For example you can try to run your testsql.py locally (NOT in a container).
Container to container communication will always happen using the host name of each container. Think of containers as virtual machines.
You can even find the network docker-compose created using docker network list:
1b1a54630639 myproject_default bridge local
82498fd930bb bridge bridge local
.. then use docker network inspect <id> to look at the details.
Assigned IP addresses to containers can be pretty random, so the only viable way for container to container communication is using hostnames.