We Keep Coding

What does 'DPY-6001: cannot connect to database' mean with python-oracledb?

With Python code that uses the
python-oracledb driver:
import oracledb
import os
un = os.environ.get("PYTHON_USERNAME")
pw = os.environ.get("PYTHON_PASSWORD")
cs = "localhost/doesnotexist"
c = oracledb.connect(user=un, password=pw, dsn=cs)
what does this error message mean?
DPY-6001: cannot connect to database. Service "doesnotexist" is not registered with the listener at host "localhost" port 1521. (Similar to ORA-12514)

The error means that Python successfully reached a computer (in this case
"localhost" using the default port 1521) that is running a database. However
the database service you wanted ("doesnotexist") doesn't exist there.
Technically the error means the listener doesn't know about the service at the
moment. So you might also get this error if the DB is currently restarting.
This error is similar to the ORA-12514 error that you would see when connecting
with python-oracledb in Thick mode, or might see with some other Oracle tools.
The solution is to use a valid service name, for example:
cs = "localhost/xepdb1"
You can:
Check and fix any typos in the service name you used
Check the hostname and port are correct
Ask your DBA for the correct values
Wait a few moments and re-try in case the DB is restarting
Review the connection information in your cloud console or cloud wallet, if you are using a cloud DB
Run lsnrctl status on the database machine to find the known service names

Python - psycopg2 giving error after execution

I am getting this error when executing my code in Python.
Here is my Python - DataBaseHelper.py:
import psycopg2
#class
class DataBaseHelper:
database = "testdata";user = "test";password = "pass123"; host = "mtest.75tyey.us-east-1.rds.amazonaws.com"
#create and return the connection of database
def getConection(self):
self.conn = psycopg2.connect(database=self.database, user = self.user, password = self.password, host = self.host, port = "5432")
return self.conn
Then I am importing this file and using in another python file - MyScript.py:
import sys
import uuid
from DBHelper import DataBaseHelper
from ExecutionLogHelper import ExecutionLogHelper
from GotUtility import GotUtility
class MyScript:
def __init__(self,):
self.con = DataBaseHelper().getConection()
self.logHelper = ExecutionLogHelper()
self.uuid = self.logHelper.Get_TEST_Run_Id()
When I run my code concurrently, it gives me this error:
psycopg2.errors.AdminShutdown: terminating connection due to administrator command
SSL connection has been closed unexpectedly
I am not able to understand why am I getting this error. When I run the Python program again, it works. And I checked the Postgres server is in running, no restart, no signal to shutdown. This keeps on happening every few hours for me.

This is happening because psycopg2 is try to connect to AWS Postgresql over SSL and failing to do so.
Try connecting with sslmode = disable
def getConection(self):
self.conn = psycopg2.connect(database=self.database,
user = self.user,
password = self.password,
host = self.host,
port = "5432",
sslmode="disable")
return self.conn
Method 1 will not work if your AWS Postgresql is configured to force a ssl connection ie. parameter rds.force_ssl = 1. If you enable set rds.force_ssl all non-SSL connections are refused. In that case try connecting using something like this:
$ psql -h testpg.cdhmuqifdpib.us-east-1.rds.amazonaws.com -p 5432 "dbname=testpg user=testuser sslrootcert=rds-ca-2015-root.pem sslmode=verify-full"
For more on how to connect to AWS RDS over ssl using various drivers : AWS RDS SSL.

After a little digging, I found a few answers. According to this link:
This error message comes from intervention by a program external to
Postgres: http://www.postgresql.org/message-id/4564.1284559661#sss.pgh.pa.us
To elaborate, this link says:
If user stop postgresql server with "service postgresql stop" or if any SIGINT has been called on the postgresql PID then this error will occur.
Solution:
Since your code is running concurrently, multiple transactions at the same time, at the same row could be causing this error. So you've got to make sure that that doesn't happen... Look here for more details:
When you update rows in a transaction, these rows are locked until the transaction is committed.
If you are unable to do that, I suggest you enable query logging and look to see if something odd is in it.

Connecting to mysql db via ssh with python

I have tried solutions found on other SO questions but none of them have worked for me. I am attempting to pull data from a mysql db running on a remote server by setting up an ssh tunnel. My code is as follows:
server = sshtunnel.SSHTunnelForwarder(
('10.6.41.10', 22),
ssh_username= 'serveruser',
ssh_password= 'serverpw',
remote_bind_address=('127.0.0.1', 3306))
server.start()
print(server.local_bind_port)
cnx = mysql.connector.connect(user='root', password='mysqlpw',
host='127.0.0.1',
database='mydb',
charset='utf8',
use_unicode='FALSE',
port = 3306)
However, when I run this code I receive:
1045 (28000): Access denied for user 'root'#'localhost' (using password: YES)
I have also tried adding
local_bind_address = ('0.0.0.0', 3306)
to the sshtunnel setup and instead recieved
Problem setting SSH Forwarder up: Couldn't open tunnel 0.0.0.0:3306 <> 127.0.0.1:3306 might be in use or destination not reachable
I don't fully understand the remote_bind_address and local_bind_address, so my guess is that must be doing something wrong there. I know my username/pw/server info is correct, I am able to ssh into my server via terminal and then use
mysql -h 127.0.0.1 -u root -p
to successfully log into my mysql server. So what do I need to fix to get it running in python? Thanks.

If you don't specify local_bind_address in sshtunnel.SSHTunnelForwarder, the local port is allocated randomly. In that case set port=server.local_bind_port in mysql.connector.connect().
Instead, you can also set local_bind_address=('0.0.0.0', [some port which is not in use]) in sshtunnel.SSHTunnelForwarder. The sshtunnel.HandlerSSHTunnelForwarderError ("Problem setting...") tells you that you can't use local_bind_address=('0.0.0.0', 3306).

How to connect MySQL database using Python+SQLAlchemy remotely?

I am having difficulty accessing MySQL remotely. I use SSH tunnel and want to connect the database MySQL using Python+SQLALchemy.
When i use MySQL-client in my console and specify "ptotocol=TCP", then everything is fine!
I use command:
mysql -h localhost —protocol=TCP -u USER -p
I get access to remote database through SSH-tunnel.
However, when I want to connect to the database using the Python+SQLAchemy I can't find such option like —protocol=TCP
Otherwise, i have only connect to local MySQL Databases.
Tell me please, is there a way to do it using SQLAlchemy.

The classic answer to this issue is to use 127.0.0.1 or the IP of the host or the host name instead of the "special name" localhost. From the documentation:
[...] connections on Unix to localhost are made using a Unix socket file by default
And later:
On Unix, MySQL programs treat the host name localhost specially, in a way that is likely different from what you expect compared to other network-based programs. For connections to localhost, MySQL programs attempt to connect to the local server by using a Unix socket file. This occurs even if a --port or -P option is given to specify a port number. To ensure that the client makes a TCP/IP connection to the local server, use --host or -h to specify a host name value of 127.0.0.1, or the IP address or name of the local server.
However, this simple trick doesn't appear to work in your case, so you have to somehow force the use of a TCP socket. As you explained it yourself, when invoking mysql on the command line, you use the --protocol tcp option.
As explained here, from SQLAlchemy, you can pass the relevant options (if any) to your driver either as URL options or using the connect_args keyword argument.
For example using PyMySQL, on a test system I've setup for that purpose (MariaDB 10.0.12, SQLAlchemy 0.9.8 and PyMySQL 0.6.2) I got the following results:
>>> engine = create_engine(
"mysql+pymysql://sylvain:passwd#localhost/db?host=localhost?port=3306")
# ^^^^^^^^^^^^^^^^^^^^^^^^^^
# Force TCP socket. Notice the two uses of `?`
# Normally URL options should use `?` and `&`
# after that. But that doesn't work here (bug?)
>>> conn = engine.connect()
>>> conn.execute("SELECT host FROM INFORMATION_SCHEMA.PROCESSLIST WHERE ID = CONNECTION_ID()").fetchall()
[('localhost:54164',)]
# Same result by using 127.0.0.1 instead of localhost:
>>> engine = create_engine(
"mysql+pymysql://sylvain:passwd#127.0.0.1/db?host=localhost?port=3306")
>>> conn = engine.connect()
>>> conn.execute("SELECT host FROM INFORMATION_SCHEMA.PROCESSLIST WHERE ID = CONNECTION_ID()").fetchall()
[('localhost:54164',)]
# Alternatively, using connect_args:
>>> engine = create_engine("mysql+pymysql://sylvain:passwd#localhost/db",
connect_args= dict(host='localhost', port=3306))
>>> conn = engine.connect()
>>> conn.execute("SELECT host FROM INFORMATION_SCHEMA.PROCESSLIST WHERE ID = CONNECTION_ID()").fetchall()
[('localhost:54353',)]
As you noticed, both will use a TCP connection (I know that because of the port number after the hostname). On the other hand:
>>> engine = create_engine(
"mysql+pymysql://sylvain:passwd#localhost/db?unix_socket=/path/to/mysql.sock")
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# Specify the path to mysql.sock in
# the `unix_socket` option will force
# usage of a UNIX socket
>>> conn = engine.connect()
>>> conn.execute("SELECT host FROM INFORMATION_SCHEMA.PROCESSLIST WHERE ID = CONNECTION_ID()").fetchall()
[('localhost',)]
# Same result by using 127.0.0.1 instead of localhost:
>>> engine = create_engine(
"mysql+pymysql://sylvain:passwd#127.0.0.1/db?unix_socket=/path/to/mysql.sock")
>>> conn = engine.connect()
>>> conn.execute("SELECT host FROM INFORMATION_SCHEMA.PROCESSLIST WHERE ID = CONNECTION_ID()").fetchall()
[('localhost',)]
# Alternatively, using connect_args:
>>> engine = create_engine("mysql+pymysql://sylvain:passwd#localhost/db",
connect_args= dict(unix_socket="/path/to/mysql.sock"))
>>> conn = engine.connect()
>>> conn.execute("SELECT host FROM INFORMATION_SCHEMA.PROCESSLIST WHERE ID = CONNECTION_ID()").fetchall()
[('localhost',)]
No port after the hostname: this is an UNIX socket.

This worked for me:
import pandas as pd
import pymysql
from sqlalchemy import create_engine
cnx = create_engine('mysql+pymysql://<username>:<password>#<host>/<dbname>')
df = pd.read_sql('SELECT * FROM <table_name>', cnx) #read the entire table
Where credentials are added to mysql database like this:
CREATE USER '<username>' IDENTIFIED BY '<password>';
GRANT ALL PRIVILEGES ON *.* TO '<username>' WITH GRANT OPTION;
FLUSH PRIVILEGES;

In my setup (I'm using mysql-python) just using 127.0.0.1 instead of localhost in the MySQL SQLAlchemy url works. The complete url I'm using exactly for that scenario (tunnel with local port 3307) is:
mysql:/user:passwd#127.0.0.1:3307/
I'm using SQLAlchemy 1.0.5, but I guess that doesn't matter too much...

I Tried this to connect with mysql db of xampp server
from sqlalchemy import create_engine
engine = create_engine("mysql+pymysql://usrnme:passwd#hstnme/dbname")

If you are using Python 3.x you can use:
pip install mysql-connector-python
Then:
import sqlalchemy as db
engine = db.create_engine("mysql+mysqlconnector://username:password#hostname:port/dbname")

Problems in connecting to MusicBrainz database using psycopg2

I am trying to connect to the MusicBrainz database using the psycopg2 python's module. I have followed the instructions presented on http://musicbrainz.org/doc/MusicBrainz_Server/Setup, but I cannot succeed in connecting. In particular I am using the following little script:
import psycopg2
conn = psycopg2.connect( database = 'musicbrainz_db', user= 'musicbrainz', password = 'musicbrainz', port = 5000, host='10.16.65.250')
print "Connection Estabilished"
The problem is that when I launch it, it never reaches the print statement, and the console (I'm on linux) is block indefinitely. It does not even catches the ctrl-c kill, so I have to kill python itself in another console. What can cause this?

You seem to be mistaking MusicBrainz-Server to be only the database.
What's running on port 5000 is the Web Server.
You can access http://10.16.65.250:5000 in the browser.
Postgres is also running, but listens on localhost:5432.
This works:
import psycopg2
conn = psycopg2.connect(database="musicbrainz_db",
user="musicbrainz", password="musicbrainz",
port="5432", host="localhost")
print("Connection established")
In order to make postgres listen to more than localhost you need to change listen_addresses in /etc/postgresql/9.1/main/postgres.conf and make an entry for your (client) host or network in /etc/postgresql/9.1/main/pg_hba.conf.
My VM is running in a 192.168.1.0/24 network so I set listen_addresses='*' in postgres.conf and in pg_hab.conf:
host all all 192.168.1.0/24 trust
I can now connect from my local network to the DB in the VM.
Depending on what you actually need, you might not want to connect to the MusicBrainz Server via postgres. There is a MusicBrainz web service you can access in the VM.
Example:
http://10.16.65.250:5000/ws/2/artist/c5c2ea1c-4bde-4f4d-bd0b-47b200bf99d6.
In that case you might be interested in a library to process the data:
python-musicbrainzngs.
EDIT:
You need to set musicbrainzngs.set_hostname("10.16.65.250:5000") for musicbrainzngs to connect to your local VM.