I am trying to run integration tests (in python) which depend on mysql. Currently they depend on SQL running locally, but I want them to depend on a MySQL running in docker.
Contents of Dockerfile:
FROM continuumio/anaconda3:4.3.1
WORKDIR /opt/workdir
ADD . /opt/workdir
RUN python setup.py install
Contents of Docker Compose:
version: '2'
services:
mysql:
image: mysql:5.6
container_name: test_mysql_container
environment:
- MYSQL_ROOT_PASSWORD=test
- MYSQL_DATABASE=My_Database
- MYSQL_USER=my_user
- MYSQL_PASSWORD=my_password
volumes:
- db_data:/var/lib/mysql
restart: always
expose:
- "3306"
my_common_package:
image: my_common_package
depends_on:
- mysql
restart: always
links:
- mysql
volumes:
db_data:
Now, I try to run the tests in my package using:
docker-compose run my_common_package python testsql.py
and I receive the error
pymysql.err.OperationalError: (2003, "Can't connect to MySQL server on
'localhost' ([Errno 99] Cannot assign requested address)")
docker-compose will by default create virtual network were all the containers/services in the compose file can reach each other by an IP address. By using links, depends_on or network aliases they can reach each other by host name. In your case the host name is the service name, but this can be overridden. (see: docs)
Your script in my_common_package container/service should then connect to mysql on port 3306 according to your setup. (not localhost on port 3306)
Also note that using expose is only necessary if the Dockerfile for the service don't have an EXPOSE statement. The standard mysql image already does this.
If you want to map a container port to localhost you need to use ports, but only do this if it's necessary.
services:
mysql:
image: mysql:5.6
container_name: test_mysql_container
environment:
- MYSQL_ROOT_PASSWORD=test
- MYSQL_DATABASE=My_Database
- MYSQL_USER=my_user
- MYSQL_PASSWORD=my_password
volumes:
- db_data:/var/lib/mysql
ports:
- "3306:3306"
Here we are saying that port 3306 in the mysql container should be mapped to localhost on port 3306.
Now you can connect to mysql using localhost:3306 outside of docker. For example you can try to run your testsql.py locally (NOT in a container).
Container to container communication will always happen using the host name of each container. Think of containers as virtual machines.
You can even find the network docker-compose created using docker network list:
1b1a54630639 myproject_default bridge local
82498fd930bb bridge bridge local
.. then use docker network inspect <id> to look at the details.
Assigned IP addresses to containers can be pretty random, so the only viable way for container to container communication is using hostnames.
Related
I am trying to create 2 containers as per the following docker-compose.yml file. The issue is that if I start up the mongo database container and then run my code locally (hitting 127.0.0.1) then everything is fine but if I try and run my api container and hit that (see yml file) then I get connection refused i.e.
172.29.0.12:27117: [Errno 111] Connection refused, Timeout: 30s, Topology Description: <TopologyDescription id:
60437a460a3e0fa904650e35, topology_type: Single, servers: [<ServerDescription ('172.29.0.12', 27117) server_type:
Unknown, rtt: None, error=AutoReconnect('172.29.0.12:27117: [Errno 111] Connection refused')>]>
Please note: I have set mongo to use port 27117 rather than 27017
My app is a Python Flask app and I am using PyMongo in the following manner:
try:
myclient = pymongo.MongoClient('mongodb://%s:%s#%s:%s/%s' % (username, password, hostName, port, database))
mydb = myclient[database]
cursor = mydb["temperatures"]
app.logger.info('Database connected to: ' + database)
except:
app.logger.error('Error connecting to database')
What's driving me mad is it runs locally and successfully accesses mongo via the container, but as soon as I try the app in a container it fails.
docker-compose.yml as follows:
version: '3.7'
services:
hotbin-db:
image: mongo
container_name: hotbin-db
restart: always
ports:
# <Port exposed> : < MySQL Port running inside container>
- '27117:27017'
expose:
# Opens port 3306 on the container
- '27117'
command: [--auth]
environment:
MONGO_INITDB_ROOT_USERNAME: ***
MONGO_INITDB_ROOT_PASSWORD: ***
MONGO_INITDB_DATABASE: ***
MONGODB_DATA_DIR: /data/db
MONDODB_LOG_DIR: /dev/null
# Where our data will be persisted
volumes:
- /home/simon/mongodb/database/hotbin-db/:/data/db
#- my-db:/var/lib/mysql
# env_file:
# - .env
networks:
hotbin-net:
ipv4_address: 172.29.0.12
hotbin-api:
image: scsherlock/compost-api:latest
container_name: hotbin-api
environment:
MONGODB_DATABASE: ***
MONGODB_USERNAME: ***
MONGODB_PASSWORD: ***
MONGODB_HOSTNAME: 172.29.0.12
MONGODB_PORT: '27117'
depends_on:
- hotbin-db
restart: always
ports:
# <Port exposed> : < MySQL Port running inside container>
- '5050:5050'
expose:
- '5050'
networks:
hotbin-net:
ipv4_address: 172.29.0.13
# # Names our volume
volumes:
my-db:
networks:
hotbin-net:
driver: bridge
ipam:
driver: default
config:
- subnet: 172.29.0.0/16
Using the service name of the mongo container and the standard port of
27017 instead of 27117 (even though that's what is defined in the
docker-compose file) works. I'd like to understand why though
Your docker compose file does NOT configure MongoDB to run on port 27117. If you want to get it to run on 27117 you would have to change this line in the docker compose:
command: mongod --auth --port 27117
As you haven't specified a port, MongoDB will run on the default port 27017.
Your expose section exposes the container port 27117 to the host, but Mongo isn't running on that port, so that line is effectively doing nothing.
Your ports section maps a host port 27117 to a container port 27017. This means if you're connecting from the host, you can connect on port 27117, but that is connecting to port 27017 on the container.
Now to your python program. As this is running in the container network, to connect services within a docker-compose network, you reference them by their service name.
Putting this together, your connection string will be: mongodb://hotbin-db:27017/yourdb?<options>
As others have mentioned, you really don't need to create specific IP addresses unless you have a very good need to. You also don't even to define a network, as docker-compose creates it's own internal network.
Reference: https://docs.docker.com/compose/networking/
Are you using Windows to run the container?
If yes,localhost is identified as localhost of the container and not the localhost of your host machine.
Hence, instead of providing the IP address of your host, try modifying your mongodB string this way when running inside the docker container:
Try this:
mongodb://host.docker.internal:27017/
instead of:
mongodb://localhost:27017/
I had created a docker-container of python application where the code in it tries to connect to remote HBase cluster hosted on Cloudera.
Docker is running fine,except that, it is not doing read/write operation on remote HBase.
Here is my part of docker-compose.yml file
version: '2'
services:
app:
build: .
command: python3 app.py
networks:
- default
ports:
- "8007:8007"
Suggestions are welcomed.
Solved this issue ,this is because at remote HBase-cluster, thrift server was not accessible by docker.
Whitelisting my docker IP at HBase-cluster solved the issue.
I am currently trying to run a docker Odoo container and expose it to my local network so my team can start testing it out, but I can't access the container from another computer on the same network. How can I host odoo on a windows docker machine that will let my co-workers access and work with Odoo?
You simply need to expose the port that your odoo web service is running at. From the official Odoo docker hub repository:
version: '2'
services:
web:
image: odoo:12.0
depends_on:
- db
ports:
- "8069:8069"
db:
image: postgres:10
environment:
- POSTGRES_DB=postgres
- POSTGRES_PASSWORD=odoo
- POSTGRES_USER=odoo
Or without docker-compose you could use e.g.
docker run -p 8069:8069 --name odoo --link db:db -t odoo -- --db-filter=odoo_db_.*
If you want to access the internal port 8069 from external port 80, you can simply change to port mapping to 80:8069.
Afterwards odoo can be accessed with a browser at [your-ip]:8069 or simply [your-ip] if you map the external port to 80.
I have an app in python that I want to run in a docker container and it has a line:
h2o.connect(ip='127.0.0.1', port='54321')
The h2o server is running in docker container and it always has different ip. One time it was started on 172.19.0.5, the other time 172.19.0.3, sometimes 172.17.0.3.
So it is always random, and I can't connect the python app.
I tried to expose the port of h2o server to localhost and then connect the python (the code above), but it is not working.
You dont connect two docker containers though ip addresses. Instead, you want to use docker internal network aliases:
version: '3'
services:
server:
...
depends_on:
- database
database:
...
expose:
- 54321:54321
then you can define your connectio in server as:
h2o.connect(ip='127.0.0.1', port='54321')
I am trying to run 2 docker containers using docker-compose and connect mysql container to app container.Mysql container is running but app container is failing to start with the error Error:2003: Can't connect to MySQL server on '127.0.0.1:3306' (111 Connection refused)
It seems like my app container is trying to connect my host mysql instead of mysql container.
docker-compose.yml
version: '2'
services:
mysql:
image: mysql:5.7
container_name: database
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: malicious
MYSQL_USER: root
MYSQL_PASSWORD: root
app:
build: .
restart: unless-stopped
volumes:
- .:/Docker_compose_app #app directory
depends_on:
- "mysql"
command: [ "python", "database_update.py"]
restart: unless-restart
environment:
# Environment variables to configure the app on startup.
MYSQL_DATABASE: malicious
MYSQL_HOST: database
Dockerfile
FROM python:2.7
ADD . /Docker_compose_app
WORKDIR /Docker_compose_app
RUN apt-get update
RUN pip install --requirement requirement.txt
This is my database_update.py file.
def create_TB(cursor,connection):
query = '''CREATE TABLE {}(malicious VARCHAR(100) NOT NULL)'''.format("url_lookup")
cursor.execute(query)
connection.commit()
def connection():
try:
cnx = mysql.connector.connect(user="root",password = 'root',database=malicious)
cursor = cnx.cursor()
create_TB(cursor,cnx)
except mysql.connector.errors.Error as err:
data = {"There is an issue in connection to DB":"Error: {}".format(err)}
There are two issues I can see:
Try to add
links:
- mysql:mysql
to the app service in your Docker Compose file. This will make sure that you can reach the mysql container from app. It will set up a hostname mapping (DNS) in your app container, so when you ping mysql from app, it will resolve it to the mysql container's IP address.
In your .py file, where are you defining which host to connect to? Add host="mysql" to the connect call. By default, it will connect to 127.0.0.1, which is what you're seeing.
cnx = mysql.connector.connect(host="mysql", user="root", password = 'root', database=malicious)
Fixing both of these should solve your problem.
You might want to consider using Docker Networks.
I was having a similar problem while having two seperate Python container connecting to one mysql-Container, while those 2 were connected to a Vue-Frontend.
First I tried using links (which was not optimal, because the communication-flow is not entirely linear), just like you but the I ran across this great post:
https://www.cbtnuggets.com/blog/devops/how-to-share-a-mysql-db-with-multiple-docker-containers
Using Networks shift the port mapping off and lets you enhance your overall App-Architecture.
Therefore I think you should try something like:
services:
python-app:
networks:
- network_name
...
mysql:
networks:
- network_name
...
networks:
network_name: