I am trying to run 2 docker containers using docker-compose and connect mysql container to app container.Mysql container is running but app container is failing to start with the error Error:2003: Can't connect to MySQL server on '127.0.0.1:3306' (111 Connection refused)
It seems like my app container is trying to connect my host mysql instead of mysql container.
docker-compose.yml
version: '2'
services:
mysql:
image: mysql:5.7
container_name: database
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: malicious
MYSQL_USER: root
MYSQL_PASSWORD: root
app:
build: .
restart: unless-stopped
volumes:
- .:/Docker_compose_app #app directory
depends_on:
- "mysql"
command: [ "python", "database_update.py"]
restart: unless-restart
environment:
# Environment variables to configure the app on startup.
MYSQL_DATABASE: malicious
MYSQL_HOST: database
Dockerfile
FROM python:2.7
ADD . /Docker_compose_app
WORKDIR /Docker_compose_app
RUN apt-get update
RUN pip install --requirement requirement.txt
This is my database_update.py file.
def create_TB(cursor,connection):
query = '''CREATE TABLE {}(malicious VARCHAR(100) NOT NULL)'''.format("url_lookup")
cursor.execute(query)
connection.commit()
def connection():
try:
cnx = mysql.connector.connect(user="root",password = 'root',database=malicious)
cursor = cnx.cursor()
create_TB(cursor,cnx)
except mysql.connector.errors.Error as err:
data = {"There is an issue in connection to DB":"Error: {}".format(err)}
There are two issues I can see:
Try to add
links:
- mysql:mysql
to the app service in your Docker Compose file. This will make sure that you can reach the mysql container from app. It will set up a hostname mapping (DNS) in your app container, so when you ping mysql from app, it will resolve it to the mysql container's IP address.
In your .py file, where are you defining which host to connect to? Add host="mysql" to the connect call. By default, it will connect to 127.0.0.1, which is what you're seeing.
cnx = mysql.connector.connect(host="mysql", user="root", password = 'root', database=malicious)
Fixing both of these should solve your problem.
You might want to consider using Docker Networks.
I was having a similar problem while having two seperate Python container connecting to one mysql-Container, while those 2 were connected to a Vue-Frontend.
First I tried using links (which was not optimal, because the communication-flow is not entirely linear), just like you but the I ran across this great post:
https://www.cbtnuggets.com/blog/devops/how-to-share-a-mysql-db-with-multiple-docker-containers
Using Networks shift the port mapping off and lets you enhance your overall App-Architecture.
Therefore I think you should try something like:
services:
python-app:
networks:
- network_name
...
mysql:
networks:
- network_name
...
networks:
network_name:
Related
I've been trying for a few hours now but no solution from similar asked questions seem to work for me...
I am using docker-compose to setup a postgresql database and run a python webserver from where I want to connect to my postgressql database (so it's running inside the container)
version: '3.8'
services:
database:
container_name: database
hostname: database
image: postgres
restart: always
environment:
POSTGRES_DB: mydatabase
POSTGRES_USER: postgres
POSTGRES_PASSWORD: password
volumes:
- postgres:/pgdata
- ./application/ressources/fixtures.sql:/docker-entrypoint-initdb.d/fixtures.sql
ports:
- "5432:5432"
application:
container_name: application
build: .
ports:
- "5001:5001"
volumes:
- ./application:/application
restart: always
depends_on:
- database
volumes:
postgres:
I trying to connect as follows ( I have read that despite the depends on in my dockerfile the database needs some more time until it can accept connections so i added a retry logic):
retries = 0
while retries < 5:
retries = retries + 1
self.conn = psycopg2.connect(user='postgres', password='password',
host='database', port="5432", database='mydatabase')
if not self.conn:
logging.info("retry to connect")
sleep(5)
The weird thing is that when running it with docker-compose -f docker-compose.yml up everything works fine.
But when I built the image (docker build -t myapp:0.1) and run it (docker run myapp:0.1) it gives me the following error:
File "/application/libraries/database.py", line 18, in establishConnection
self.conn = psycopg2.connect(user=CONFIG.DATABASE_USER, password=CONFIG.DATABASE_PASSWORD,
File "/usr/local/lib/python3.9/site-packages/psycopg2/__init__.py", line 127, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: could not translate host name "database" to address: Name or service not known
I've read that when using docker-compose a single network is created, so this can't be the error here i guess Docker Compose doku
Thanks in advance,
Jacky
If you run docker run on an image, it does only what's on that command line and no more. In particular, the plain docker commands don't know about docker-compose.yml and any of the settings you might specify there.
The short answer is to always use docker-compose up to launch the containers.
In principle you could translate the docker-compose.yml file into explicit docker commands. At a minimum you'd need to manually create a Docker network and specify the container names:
docker network create app-net
docker run -d --net app-net --name database -p 5432:5432 postgres
docker run -d --net app-net -e PGHOST=database -p 5001:5001 myapp:0.1
This hasn't included the other options in the Compose setup, though, notably database persistence. There are enough settings that you'd want to write them down in a file, and at that point you've basically reconstructed the Compose environment.
I am trying to create 2 containers as per the following docker-compose.yml file. The issue is that if I start up the mongo database container and then run my code locally (hitting 127.0.0.1) then everything is fine but if I try and run my api container and hit that (see yml file) then I get connection refused i.e.
172.29.0.12:27117: [Errno 111] Connection refused, Timeout: 30s, Topology Description: <TopologyDescription id:
60437a460a3e0fa904650e35, topology_type: Single, servers: [<ServerDescription ('172.29.0.12', 27117) server_type:
Unknown, rtt: None, error=AutoReconnect('172.29.0.12:27117: [Errno 111] Connection refused')>]>
Please note: I have set mongo to use port 27117 rather than 27017
My app is a Python Flask app and I am using PyMongo in the following manner:
try:
myclient = pymongo.MongoClient('mongodb://%s:%s#%s:%s/%s' % (username, password, hostName, port, database))
mydb = myclient[database]
cursor = mydb["temperatures"]
app.logger.info('Database connected to: ' + database)
except:
app.logger.error('Error connecting to database')
What's driving me mad is it runs locally and successfully accesses mongo via the container, but as soon as I try the app in a container it fails.
docker-compose.yml as follows:
version: '3.7'
services:
hotbin-db:
image: mongo
container_name: hotbin-db
restart: always
ports:
# <Port exposed> : < MySQL Port running inside container>
- '27117:27017'
expose:
# Opens port 3306 on the container
- '27117'
command: [--auth]
environment:
MONGO_INITDB_ROOT_USERNAME: ***
MONGO_INITDB_ROOT_PASSWORD: ***
MONGO_INITDB_DATABASE: ***
MONGODB_DATA_DIR: /data/db
MONDODB_LOG_DIR: /dev/null
# Where our data will be persisted
volumes:
- /home/simon/mongodb/database/hotbin-db/:/data/db
#- my-db:/var/lib/mysql
# env_file:
# - .env
networks:
hotbin-net:
ipv4_address: 172.29.0.12
hotbin-api:
image: scsherlock/compost-api:latest
container_name: hotbin-api
environment:
MONGODB_DATABASE: ***
MONGODB_USERNAME: ***
MONGODB_PASSWORD: ***
MONGODB_HOSTNAME: 172.29.0.12
MONGODB_PORT: '27117'
depends_on:
- hotbin-db
restart: always
ports:
# <Port exposed> : < MySQL Port running inside container>
- '5050:5050'
expose:
- '5050'
networks:
hotbin-net:
ipv4_address: 172.29.0.13
# # Names our volume
volumes:
my-db:
networks:
hotbin-net:
driver: bridge
ipam:
driver: default
config:
- subnet: 172.29.0.0/16
Using the service name of the mongo container and the standard port of
27017 instead of 27117 (even though that's what is defined in the
docker-compose file) works. I'd like to understand why though
Your docker compose file does NOT configure MongoDB to run on port 27117. If you want to get it to run on 27117 you would have to change this line in the docker compose:
command: mongod --auth --port 27117
As you haven't specified a port, MongoDB will run on the default port 27017.
Your expose section exposes the container port 27117 to the host, but Mongo isn't running on that port, so that line is effectively doing nothing.
Your ports section maps a host port 27117 to a container port 27017. This means if you're connecting from the host, you can connect on port 27117, but that is connecting to port 27017 on the container.
Now to your python program. As this is running in the container network, to connect services within a docker-compose network, you reference them by their service name.
Putting this together, your connection string will be: mongodb://hotbin-db:27017/yourdb?<options>
As others have mentioned, you really don't need to create specific IP addresses unless you have a very good need to. You also don't even to define a network, as docker-compose creates it's own internal network.
Reference: https://docs.docker.com/compose/networking/
Are you using Windows to run the container?
If yes,localhost is identified as localhost of the container and not the localhost of your host machine.
Hence, instead of providing the IP address of your host, try modifying your mongodB string this way when running inside the docker container:
Try this:
mongodb://host.docker.internal:27017/
instead of:
mongodb://localhost:27017/
I am running locally this script. On postgres connection, I am facing "Exception has occurred: OperationalError
could not translate host name "db" to address: Unknown host". Database is up, I started it with
docker run --publish 8000:8000 --detach --name db REMOTEURL:latest
When I do the same using docker-compose, and exec to django app and run it as management command, it works. The code:
conn_string = "host='db' dbname='taras' user='postgres' password='postgres'"
with psycopg2.connect(conn_string) as connection:
with connection.cursor() as cursor:
... my stuff
I dont know why when accessing docker container from local script on my computer, docker name (url) is never recognized. Tried "localhost" instead of "db" also. When running python script inside docker container, it has no problem to recognize other docker container (db in my case). What am I missing?
EDIT: Just to clarify, I am running only database in docker. Python script is started locally, using in my windows command line
python myscript.py
Docker-compose file:
version: "2"
services:
db:
image: REMOTEURL:latest
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_HOST_AUTH_METHOD: trust
This probably happens because when you launch containers, they can resolve hostnames based the names you give them internally, but if you wanna access a container from the outside (I'm assuming you wanna access your postgres db), you also need to open the port for the postgres container by adding -p 5432:5432 to your postgres container. If you want to access postgres from another container on the same network, connect with db:5432, and from the outside with localhost:5432
Edit:
Try this as your docker-compose.yml
version: "2"
services:
db:
image: REMOTEURL:latest
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_HOST_AUTH_METHOD: trust
ports:
- 5432:5432
I am trying to connect to mysql db using a python program. When run locally it works.
But while dockerizing the application I created, one container is for the python code and the other for the mysql db, when ran i this manner it fails to connect.
Python_code:
db.bind(provider='mysql', user='docker_root', password='password', host='db', database=database, port = 3306)
docker-compose:
version: "3"
services:
app:
image: app:latest
links:
- db
ports:
- "8001:8081"
environment:
- DB_HOST= db
db:
image: mysql:5.7.26
restart: always
environment:
MYSQL_DATABASE: 'my_db'
MYSQL_USER: 'docker_root'
MYSQL_PASSWORD: 'password'
MYSQL_ROOT_PASSWORD: 'password'
ports:
- "3306:3306"
volumes:
- ./DB_config/:/etc/mysql/mysql.conf.d
And the docker-compose up fails with the eroor:
pony.orm.dbapiprovider.OperationalError: (2003, "Can't connect to
MySQL server on 'db' ([Errno 111] Connection refused)")
Where am I going wrong? Please advise!
I would recommend you to exit your application in case it cannot connect to MySQL and set the restart policy to always, because depends_on does not guarantee that MySQL will be totally up when app starts but it is good to have it there.
version: "3"
services:
app:
image: app:latest
restart: always
links:
- db
ports:
- "8001:8081"
environment:
- DB_HOST= db
depends_on:
- db
db:
image: mysql:5.7.26
restart: always
environment:
MYSQL_DATABASE: 'my_db'
MYSQL_USER: 'docker_root'
MYSQL_PASSWORD: 'password'
MYSQL_ROOT_PASSWORD: 'password'
ports:
- "3306:3306"
volumes:
- ./DB_config/:/etc/mysql/mysql.conf.d
And your application code should be something like:
try:
db.bind(provider='mysql', user='docker_root', password='password', host='db', database=database, port = 3306)
except:
# write some logs
exit(1)
Try using container port 3306 -
db.bind(provider='mysql', user='docker_root', password='password', host='db', database=database, port = 3306)
Also, add depends_on attribute, you can remove the links attribute -
depends_on:
- db
YOu can use depends_on flag as mentioned in accepted answer. If it does not solve your problem them use this approach.
After starting the container, your server will try to connect to the database server. sometimes database server may take some time to boot up and in this window, if the server tries to connect to database server it will face problems.
Try adding logic to reconnect database server after few seconds if the connection fails.
try{
connectToDatabase();
}catch(error){
waitForHalfMinute();
connectToDatabase();
}
I am trying to run integration tests (in python) which depend on mysql. Currently they depend on SQL running locally, but I want them to depend on a MySQL running in docker.
Contents of Dockerfile:
FROM continuumio/anaconda3:4.3.1
WORKDIR /opt/workdir
ADD . /opt/workdir
RUN python setup.py install
Contents of Docker Compose:
version: '2'
services:
mysql:
image: mysql:5.6
container_name: test_mysql_container
environment:
- MYSQL_ROOT_PASSWORD=test
- MYSQL_DATABASE=My_Database
- MYSQL_USER=my_user
- MYSQL_PASSWORD=my_password
volumes:
- db_data:/var/lib/mysql
restart: always
expose:
- "3306"
my_common_package:
image: my_common_package
depends_on:
- mysql
restart: always
links:
- mysql
volumes:
db_data:
Now, I try to run the tests in my package using:
docker-compose run my_common_package python testsql.py
and I receive the error
pymysql.err.OperationalError: (2003, "Can't connect to MySQL server on
'localhost' ([Errno 99] Cannot assign requested address)")
docker-compose will by default create virtual network were all the containers/services in the compose file can reach each other by an IP address. By using links, depends_on or network aliases they can reach each other by host name. In your case the host name is the service name, but this can be overridden. (see: docs)
Your script in my_common_package container/service should then connect to mysql on port 3306 according to your setup. (not localhost on port 3306)
Also note that using expose is only necessary if the Dockerfile for the service don't have an EXPOSE statement. The standard mysql image already does this.
If you want to map a container port to localhost you need to use ports, but only do this if it's necessary.
services:
mysql:
image: mysql:5.6
container_name: test_mysql_container
environment:
- MYSQL_ROOT_PASSWORD=test
- MYSQL_DATABASE=My_Database
- MYSQL_USER=my_user
- MYSQL_PASSWORD=my_password
volumes:
- db_data:/var/lib/mysql
ports:
- "3306:3306"
Here we are saying that port 3306 in the mysql container should be mapped to localhost on port 3306.
Now you can connect to mysql using localhost:3306 outside of docker. For example you can try to run your testsql.py locally (NOT in a container).
Container to container communication will always happen using the host name of each container. Think of containers as virtual machines.
You can even find the network docker-compose created using docker network list:
1b1a54630639 myproject_default bridge local
82498fd930bb bridge bridge local
.. then use docker network inspect <id> to look at the details.
Assigned IP addresses to containers can be pretty random, so the only viable way for container to container communication is using hostnames.