We Keep Coding

Can not connect to Postgres from Python [duplicate]

This question already has answers here:
Connecting to Postgresql in a docker container from outside
(17 answers)
Closed 1 year ago.
I created a docker image with Postgres via Dockerfile:
FROM postgres:9.6-alpine
After I started this docker container, I'm checking that it's up and running using a connection from the different docker container that has pre-installed psql:
docker run -it --rm --link ml-postgres:postgres postgres:12.2-alpine
psql --dbname mlpython -h postgres -U postgres
The result is that I'm able to connect to the first container with postgres and perform all regular operations with the postgres DB.
Troubles begin when I want to connect to the container with postgres DB from a Python script that I created locally:
import psycopg2
conn = psycopg2.connect(
host="127.0.0.1",
database="mlpython",
user="postgres",
password="test",
port="5431"
)
cur = conn.cursor()
cursor.execute('SELECT COUNT(*) FROM mytable LIMIT 10')
cur.close()
Here is an error which I get:
> psycopg2.OperationalError: server closed the connection unexpectedly
> This probably means the server terminated abnormally before or while
> processing the request.
What do I miss while trying to bootstrap this simple code sample where Python interacts with Postgres?

Please read through the README.md of a docker image you use. It should answer your questions.
I'm not sure, if you did, because:
I see, you started psql - the client. Why, if you're going to connect from python? And have you started the server?
I can't see if you exposed any container port to a host machine

Exception has occurred: OperationalError could not translate host name "db" to address: Unknown host

I am running locally this script. On postgres connection, I am facing "Exception has occurred: OperationalError
could not translate host name "db" to address: Unknown host". Database is up, I started it with
docker run --publish 8000:8000 --detach --name db REMOTEURL:latest
When I do the same using docker-compose, and exec to django app and run it as management command, it works. The code:
conn_string = "host='db' dbname='taras' user='postgres' password='postgres'"
with psycopg2.connect(conn_string) as connection:
with connection.cursor() as cursor:
... my stuff
I dont know why when accessing docker container from local script on my computer, docker name (url) is never recognized. Tried "localhost" instead of "db" also. When running python script inside docker container, it has no problem to recognize other docker container (db in my case). What am I missing?
EDIT: Just to clarify, I am running only database in docker. Python script is started locally, using in my windows command line
python myscript.py
Docker-compose file:
version: "2"
services:
db:
image: REMOTEURL:latest
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_HOST_AUTH_METHOD: trust

This probably happens because when you launch containers, they can resolve hostnames based the names you give them internally, but if you wanna access a container from the outside (I'm assuming you wanna access your postgres db), you also need to open the port for the postgres container by adding -p 5432:5432 to your postgres container. If you want to access postgres from another container on the same network, connect with db:5432, and from the outside with localhost:5432
Edit:
Try this as your docker-compose.yml
version: "2"
services:
db:
image: REMOTEURL:latest
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_HOST_AUTH_METHOD: trust
ports:
- 5432:5432

running python and mysql with docker error - cannot connect

I am using mysql-connector, when ever I run the container using docker I get this error:
mysql.connector.errors.InterfaceError: 2003: Can't connect to MySQL server on 'db:3306' (-5 No address associated with hostname)
but when I run the project using python only it executes with no errors
I want to use phpmyadmin only for the database please help.

To create a docker from your linux machine:
docker pull mysql:latest
To run it and mount a persistent folder with port access on 3306 (std port for mysql):
docker run --name=mysql_dockerdb --env="MYSQL_ROOT_PASSWORD=<your_password>" -p 3306:3306 -v /home/ubuntu/sql_db/<your_dbasename>:/var/lib/mysql -d mysql:latest
To connect to the docker instance so that you can create the database within the docker:
docker exec -it mysql_dockerdb mysql -uroot -p<your_password>
My SQL code to establish the database:
CREATE DATABASE dockerdb;
CREATE USER 'newuser'#'%' IDENTIFIED BY 'newpassword';
GRANT ALL PRIVILEGES ON dockerdb.* to 'newuser'#'%';
ALTER USER 'username'#'%' IDENTIFIED WITH mysql_native_password BY 'userpassword';
You will now have a docker running with a persistent SQL database. You connect to it from your Python code. I am running Flask mySql. You will want to keep your passwords in environment variables. I am using a Mac so therefore my ~/.bash_profile contains:
export RDS_LOGIN="mysql+pymysql://<username>:<userpassword>#<dockerhost_ip>/dockerdb"
Within Python:
import os
SQLALCHEMY_DATABASE_URI = os.environ.get('RDS_LOGIN')
And at that point you should be able to connect in your usual Python manner. Note that I've glossed over any security aspects on the presumption this is local behind a firewall.

how to connect python app in docker container with running docker container with url

I have an app in python that I want to run in a docker container and it has a line:
h2o.connect(ip='127.0.0.1', port='54321')
The h2o server is running in docker container and it always has different ip. One time it was started on 172.19.0.5, the other time 172.19.0.3, sometimes 172.17.0.3.
So it is always random, and I can't connect the python app.
I tried to expose the port of h2o server to localhost and then connect the python (the code above), but it is not working.

You dont connect two docker containers though ip addresses. Instead, you want to use docker internal network aliases:
version: '3'
services:
server:
...
depends_on:
- database
database:
...
expose:
- 54321:54321
then you can define your connectio in server as:
h2o.connect(ip='127.0.0.1', port='54321')

Docker compose mysql connection failing

I am trying to run 2 docker containers using docker-compose and connect mysql container to app container.Mysql container is running but app container is failing to start with the error Error:2003: Can't connect to MySQL server on '127.0.0.1:3306' (111 Connection refused)
It seems like my app container is trying to connect my host mysql instead of mysql container.
docker-compose.yml
version: '2'
services:
mysql:
image: mysql:5.7
container_name: database
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: malicious
MYSQL_USER: root
MYSQL_PASSWORD: root
app:
build: .
restart: unless-stopped
volumes:
- .:/Docker_compose_app #app directory
depends_on:
- "mysql"
command: [ "python", "database_update.py"]
restart: unless-restart
environment:
# Environment variables to configure the app on startup.
MYSQL_DATABASE: malicious
MYSQL_HOST: database
Dockerfile
FROM python:2.7
ADD . /Docker_compose_app
WORKDIR /Docker_compose_app
RUN apt-get update
RUN pip install --requirement requirement.txt
This is my database_update.py file.
def create_TB(cursor,connection):
query = '''CREATE TABLE {}(malicious VARCHAR(100) NOT NULL)'''.format("url_lookup")
cursor.execute(query)
connection.commit()
def connection():
try:
cnx = mysql.connector.connect(user="root",password = 'root',database=malicious)
cursor = cnx.cursor()
create_TB(cursor,cnx)
except mysql.connector.errors.Error as err:
data = {"There is an issue in connection to DB":"Error: {}".format(err)}

There are two issues I can see:
Try to add
links:
- mysql:mysql
to the app service in your Docker Compose file. This will make sure that you can reach the mysql container from app. It will set up a hostname mapping (DNS) in your app container, so when you ping mysql from app, it will resolve it to the mysql container's IP address.
In your .py file, where are you defining which host to connect to? Add host="mysql" to the connect call. By default, it will connect to 127.0.0.1, which is what you're seeing.
cnx = mysql.connector.connect(host="mysql", user="root", password = 'root', database=malicious)
Fixing both of these should solve your problem.

You might want to consider using Docker Networks.
I was having a similar problem while having two seperate Python container connecting to one mysql-Container, while those 2 were connected to a Vue-Frontend.
First I tried using links (which was not optimal, because the communication-flow is not entirely linear), just like you but the I ran across this great post:
https://www.cbtnuggets.com/blog/devops/how-to-share-a-mysql-db-with-multiple-docker-containers
Using Networks shift the port mapping off and lets you enhance your overall App-Architecture.
Therefore I think you should try something like:
services:
python-app:
networks:
- network_name
...
mysql:
networks:
- network_name
...
networks:
network_name: