I am using mysql-connector, when ever I run the container using docker I get this error:
mysql.connector.errors.InterfaceError: 2003: Can't connect to MySQL server on 'db:3306' (-5 No address associated with hostname)
but when I run the project using python only it executes with no errors
I want to use phpmyadmin only for the database please help.
To create a docker from your linux machine:
docker pull mysql:latest
To run it and mount a persistent folder with port access on 3306 (std port for mysql):
docker run --name=mysql_dockerdb --env="MYSQL_ROOT_PASSWORD=<your_password>" -p 3306:3306 -v /home/ubuntu/sql_db/<your_dbasename>:/var/lib/mysql -d mysql:latest
To connect to the docker instance so that you can create the database within the docker:
docker exec -it mysql_dockerdb mysql -uroot -p<your_password>
My SQL code to establish the database:
CREATE DATABASE dockerdb;
CREATE USER 'newuser'#'%' IDENTIFIED BY 'newpassword';
GRANT ALL PRIVILEGES ON dockerdb.* to 'newuser'#'%';
ALTER USER 'username'#'%' IDENTIFIED WITH mysql_native_password BY 'userpassword';
You will now have a docker running with a persistent SQL database. You connect to it from your Python code. I am running Flask mySql. You will want to keep your passwords in environment variables. I am using a Mac so therefore my ~/.bash_profile contains:
export RDS_LOGIN="mysql+pymysql://<username>:<userpassword>#<dockerhost_ip>/dockerdb"
Within Python:
import os
SQLALCHEMY_DATABASE_URI = os.environ.get('RDS_LOGIN')
And at that point you should be able to connect in your usual Python manner. Note that I've glossed over any security aspects on the presumption this is local behind a firewall.
Related
I have a django api that streams data from an Active directory source and processes it. My connection looks something like this
from ldap3 import Server, Connection
server = Server(url, get_info=ALL)
conn = Connection(server, username, password, auto_bind=True)
I put this app on a container, the dockerfile is simple and looks like this
FROM python:3.9
EXPOSE 8002
# Install Dependencies
ADD requirements.txt .
RUN pip install -r requirements.txt
ADD . .
CMD ./server.sh
Server.sh is also fairly simple:
#!/usr/bin/env bash
aws s3 cp s3://some_creds .
python manage.py makemigrations && python manage.py migrate && python manage.py runserver 0.0.0.0:80
Now, on local, and on gitpod this connection has no issues. I go on to do searches on the conn without problems. However, when I deploy the same container on ecs via ecr I was running on local, I get this error:
<class 'ldap3.core.exceptions.LDAPSocketOpenError'>, LDAPSocketOpenError('socket connection error while opening: [Errno 110] Connection timed out'), ('xxx.xxx.xxx.xxx', xxx))])
This may be a side effect of accessing the api from ssl, but if that is the case, I simply cannot replicate it on locale.
This error occurs only on POST requests; any other request goes through as expected.
The problem was caused by the Active Directory being only accessible via vpn/proxy. The gitpod instance I was using was company provided so it also had network access to the AD.
This was solved by using a VPC that had default access to this network.
I've been trying to configure my m1 to work with an older ruby on rails api and I think in the process I've broken my ability to connect any of my python apis to their database images in docker running locally.
When I run:
psql -U dev -h localhost database
Instead of the lovely psql blinking cursor allowing me to run any sql statement I'd like I get this error message instaad:
psql: error: connection to server at "localhost" (::1), port 5432 failed: FATAL: database "dev" does not exist
I've tried docker-compuse up and down and force recreating and brew uninstalling postgres and reinstalling postgres via brew. I've downloaded the postgres.app dmg and made sure to change it to a different port hoping that that would trigger the steps needed just for psycopg2 to connect to the docker image.
the docker-compose.yaml looks like this:
services:
db:
image: REDACTED
container_name: db_name
restart: always
environment:
POSTGRES_USER: dev
POSTGRES_HOST_AUTH_METHOD: trust
networks:
default:
aliases:
- postgres
ports:
- 5432:5432
What am I missing and what can I blame ruby on rails for (which works by the way) 🤣
I think it's just docker configuration you need to update
First of all check your existing services in your local machine if the port is used by any other services (Mostly likely ylocal postgres server).
next step is to change your yaml file as below
services:
db:
image: REDACTED
container_name: db_name
restart: always
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=test_db
ports:
- 5434:5432
after that you can connect with following command in your cmd
psql -U postgres -h localhost -p 5434
assuming that you have separate yaml file for your python application
if you merge your python code in same yaml file then your connection string will be your service name (db in your case) and port will be 5432
So the answer is pretty simple. What was happening was that I had a third instance of postgres running on my computer that I had not accounted for which was the brew version. Simply running brew services stop postgres and later brew uninstall postgres fixed all my problems with being able to have my ruby on rails api work which rely on "postgres native" on my mac (protip, I changed this one to use port 5431) and my python api work which use a containerized postgres on port 5432 without any headaches. During some intial confusion during my Ruby on Rails setup which required me getting Ruby 2.6.7 running on an m1 mac I must have installed postgres via brew in an attempt to get something like db:create to work.
I am using pony.orm to connect to mysqldb using a python code:
db.bind(provider='mysql', user=username, password=password, host='0.0.0.0', database=database)
And when I write the docker compose file:
db:
image: mariadb
ports:
- "3308:3306"
environment:
MYSQL_DATABASE: db
MYSQL_USER: root
MYSQL_ROOT_PASSWORD: ''
How can I pass the hostname to the python program by giving a value (in environment:) in the docker-compose.yml file ?
If I pass the value there can I access the value through os.environ['PARAM'] in the Python code?
Because you've named your service db in the docker-compose.yaml, you can use that as the host, provided you are on the same network:
db.bind(provider='mysql', user=username, password=password, host='db', database=database)
To ensure you are on that network, in your docker-compose.yaml, at the bottom, you'll want:
networks:
default:
external:
name: <your-network>
And you'll need to create that network before running docker-compose up
docker network create <your-network>
This avoids the need for an environment variable, as the container name will be added to the routing table of the network.
You don't need to define your own network, as docker-compose will handle that for you, but if you prefer to be a bit more explicit, it allows you the flexibility to do so. Normally, you would reserve this for multiple compose solutions that you wanted to join together on a single network, which is not the case here.
It's handled in docker-compose the same way you would do it in vanilla docker:
docker run -d -p 3308:3306 --network <your-network> --name db mariadb
docker run -it --network <your-network> ubuntu bash
# in the shell of the ubuntu container
apt-get update && apt-get install iputils-ping -y
ping -c 5 db
# here you will see the results of ping reaching container db
5 packets transmitted, 5 received, 0% packet loss, time 4093ms
Edit
As a note, per #DavidMaze's comment, the port you will be communicating with is 3306, since that's the port that the container is listening on, not 3308.
I have used the following commands to create a Docker image with Postgres running on it:
docker pull postgres
docker run --name test-db -e POSTGRES_PASSWORD=my_secret_password -d postgres
I then created a table called test and inserted some random data into a couple of rows.
I am now trying to make a connection to this database table through psycopg2 in Python on my local machine.
I used the command docker-machine ip default to find out the IP address of the machine as 192.168.99.100 and am using the following to try and connect:
conn = psycopg2.connect("dbname='test-db' user='postgres' host='192.168.99.100' password='my_secret_password' port='5432'")
This is not working with the error message of "OperationalError: could not connect to server: Connection refused (0x0000274D/10061)"
.
.
Everything seems to be in order so I can't think why this would be refused.
According to the documentation for this postgres image, (at https://hub.docker.com/_/postgres/) this image includes EXPOSE 5432 (the postgres port) and the default username is postgres.
I also tried to get the IP address of the image itself with docker inspect test-db | grep IPAddress | awk 'print{$2}' | tr -d '",' that I found on SA to a slightly related article, but that IP address didn't work either.
The EXPOSE instruction may not be doing what you expect. It is used for links and inter-container communication inside the Docker network. When connecting to a container from outside the Docker bridge network you need to publish to port with -p. Try adding -p 5432:5432 to your docker run command so that it looks like:
docker run --name test-db -e POSTGRES_PASSWORD=my_secret_password -d -p 5432:5432 postgres
Here is a decent explanation of the differences between publish and exposed ports: https://stackoverflow.com/a/22150099/684908. Hope this helps!
I am using windows8, for writing code I use IDLE. I tried to connect python to mongodb. But when trying to get collection name than it gives an error.
ServerSelectionTimeoutError: localhost:20101: [Errno 10061] No connection could be made because the target machine actively refused it
This is code for which i am getting an error.
from pymongo import MongoClient
connection = MongoClient('localhost',20101)
db = connection['Bhautik']
collection = db['Student']
db.collection_names(include_system_collections=True)
By the output message you probably didn't set your mongo bind_ip or didn't set the dbpath. Try this:
mongod --dbpath <database_path> --bind_ip 127.0.0.1 --port 20101
It would be more helpful to put alongside with your code some information regarding the mongodb configuration, like the server port, if you are using authentication or not, which dbpath you are using and so on.
So put in your question your mongodb.conf (if you are using one) or the command you are using to start the mongo server.
If you are starting to use mongoDB after installation, make C:/data/db because it is a default database directory which mongoDB uses.
To change the database directory, do type below:
C:\Program Files\MongoDB\Server\3.x\bin> mongod --dbpath "c:\custom_folder"
You can try
run mongo like that:
"C:\\Program Files\\MongoDB\\Server\\3.6\\bin\\mongod.exe" --dbpath E:\\data\\db --port 27017 --bind_ip 127.0.0.1
E:\data\db should be your location path
then in you code
it will lok like
client = MongoClient("127.0.0.1", 27017)
db = client['addsome']
datas = db.follow_up
and if you want to access from a distant machine make sure you open the port "27017" in the firewall
Some times it's gives this error when you forgot to run the local server (if it's run with local server).
To run it you need to write on your terminal:
mongod
or, if MongoDB not in PATH, you can find it via this link in your computer:
C:\Program Files\MongoDB\Server\4.0\bin\mongod.exe
In order to run MongoDB,
You should have installed MongoDB into your OS, download it from https://www.mongodb.com/download-center/community?tck=docs_server
Add the installation's bin folder to your system environment variables.
Openup the terminal and check 'mongod' and 'mongo' commands are working.
Then try to rerun your python script.