We Keep Coding

Python will not connect to database

I'm trying to connect to a database via python. I keep getting this error when trying to run the code in python:
DatabaseError: ORA-12154: TNS:could not resolve the connect identifier specified
I know the tns settings are good because I can connect to the database via sql developer using the same computer. What's wrong with Python.
host = '205.218.7.153'
port = '1521'
sid= 'pdevl3'
username = 'uname'
password = 'pwd'
connect_str = username + '/' + password + '#' + host + ':' + port + '/' + sid
orcl = cx_Oracle.connect(connect_str)
curs = orcl.cursor()
curs.execute(query2)
rows = curs.fetchall()
curs.close()

Instead of building the string yourself, try using cx_Oracle to help build it for you:
import cx_Oracle
host = '205.218.7.153'
port = '1521'
sid= 'pdevl3'
username = r'uname' # make sure to use an r string if you have any special characters
password = r'pwd'
dsn_tns = cx_Oracle.makedsn(host, port, service_name=sid)
orcl = cx_Oracle.connect(user=username, password=password, dsn=dsn_tns)

Issues with connecting to Postgres DB via SSH in Python

I've been given a postgres DB in my uni project in which I have to SSH into the network from which I can then access the DB. I've set up the connection in DBeaver using the SSH tab and it works perfectly fine. However, using Python, I can connect to the SSH just fine, but cannot connect to the DB itself. I've checked with another DB that doesn't require SSH and that works just fine. Here is my code. Note: I've already tried using SSHTunnel, too, to no avail. Also, ignore my quick hack to anonymize my SSH login data, as I didn't find how to use a proper config file with paramiko late at night yesterday...
import os
from psycopg2 import connect, Error
from paramiko import SSHClient
from config import config
with open("ssh_config.txt", "r") as f:
lines = f.readlines()
hostname = lines[0].strip()
username = lines[1].strip()
password = lines[2].strip()
ssh = SSHClient()
ssh.load_host_keys(os.path.expanduser('~/.ssh/known_hosts'))
ssh.connect(hostname=hostname, username=username, password=password)
print("SSH connected.")
try:
params = config()
conn = connect(**params)
cursor = conn.cursor()
print("DB connected.")
# Print PostgreSQL connection properties.
print(conn.get_dsn_parameters(), "\n")
# Print PostgreSQL version.
cursor.execute("SELECT version();")
record = cursor.fetchone()
print("You are connected to - ", record, "\n")
except (Exception, Error) as error:
print("Error while connecting to PostgreSQL", error)
Any help would be appreciated. Thanks!

I've figured it out myself. Here is the updated code. Basically, I had to forward the remote address to localhost and then connect to localhost instead of the DB address.
from psycopg2 import connect, Error
from sshtunnel import SSHTunnelForwarder
from config import config
with open("ssh_config.txt", "r") as f:
lines = f.readlines()
hostname = lines[0].strip()
username = lines[1].strip()
password = lines[2].strip()
remote_bind_address = lines[3].strip()
try:
with SSHTunnelForwarder(
(hostname, 22),
ssh_username=username,
ssh_password=password,
remote_bind_address=(remote_bind_address, 5432),
local_bind_address=("localhost", 8080)) \
as tunnel:
tunnel.start()
print("SSH connected.")
params = config()
conn = connect(**params)
cursor = conn.cursor()
print("DB connected.")
# Print PostgreSQL connection properties.
print(conn.get_dsn_parameters(), "\n")
# Print PostgreSQL version.
cursor.execute("SELECT version();")
record = cursor.fetchone()
print("You are connected to - ", record, "\n")
cursor.close()
conn.close()
tunnel.close()
print("DB disconnected.")
except (Exception, Error) as error:
print("Error while connecting to DB", error)

Python.Dataframe to MySql: MySQL server has gone away

I am trying to write several dataframes in mysql. I use mysql.connector for the connection and sqlalchemy to create an engine.
Most dataframes are written correctly to the database. Unfortunately the application stops with the following error:
sqlalchemy.exc.DatabaseError: (mysql.connector.errors.DatabaseError) 2006 (HY000): MySQL server has gone away
...
During handling of the above exception, another exception occurred:
...
_mysql_connector.MySQLInterfaceError: MySQL server has gone away
Since the dataframe that breaks the connection is also the largest one (pickle file: 198MB) I assumed that it is due to the MySql-Server setting max_allowed_packet or timeout. (As described here)
So I have extended the config file within my SQLConnector (see below) by 'connect_timeout': 900.
Unfortunately without result.
I have also read that you should adjust the my.cnf of the MySql server. Unfortunately I did not know exactly where to find them. After a long search I found the following order. C:\Program Files\MySQL\MySQL Server 8.0\etc with the file mysqlrouter.conf.sample.
Here I have downloaded a my.cnf file and put it into the folder. Unfortunately without result, because I did not know how to set up MySql so that the server uses this file.
Does anyone know how I can fix this error? Or how I can configure the MySql Server preference settings max_allowed_packet or timeout?
Main:
mysqlConnector = MySQLConnector()
dbConnection = mysqlConnector.init_engine()
for df_tuple in df_all_tuple:
df_name = df_tuple[0]
df = pd.DataFrame(df_tuple[1])
df.to_sql(con=mydb, name=df_name, if_exists='fail', chunksize=20000)
SQLConnector:
import mysql.connector
from mysql.connector import errorcode
import sqlalchemy as db
config = {
'user': 'root',
'password': '',
'host': '127.0.0.1',
'database': 'test',
'raise_on_warnings': True,
'use_pure': False,
'autocommit': True,
'connect_timeout': 900
}
class MySQLConnector:
def __init__(self):
connectTest = self.connect()
print("Connection to database: " + str(connectTest))
def init_engine(self):
try:
connect_string = 'mysql+mysqlconnector://{}:{}#{}/{}?charset=utf8mb4'.format(config.get("user"),
config.get("password"),
config.get("host"),
config.get("database"),
pool_pre_ping=True)
print("Engine: " + connect_string)
sqlengine = db.create_engine(connect_string)
except:
print("SQLConnector: Sqlalchemy error. Can't create engine....")
else:
dbConnection = sqlengine.connect()
return dbConnection
def connect(self):
try:
con = mysql.connector.connect(**config)
except mysql.connector.Error as err:
if err.errno == errorcode.ER_ACCESS_DENIED_ERROR:
print("Something is wrong with your user name or password")
elif err.errno == errorcode.ER_BAD_DB_ERROR:
print("Database does not exist")
if self.createDB() is True:
return self.connect()
elif err.errno == errorcode.CR_SERVER_GONE_ERROR:
print("The client couldn't send a question to the server.")
print("err: " + err)
elif err.errno == errorcode.CR_SERVER_LOST:
print(
"The client didn't get an error when writing to the server, but it didn't get a full answer (or any answer) to the question.")
print("err: " + err)
else:
print(err)
return None
else:
if con.is_connected():
db_Info = con.get_server_info()
print("Connected to MySQL Server version ", db_Info)
sqlcursor = con.cursor()
return con
def createDB(self):
try:
mydb = mysql.connector.connect(
host=config.get("host"),
user=config.get("user"),
passwd=config.get("password")
)
except mysql.connector.Error as err:
print(err)
else:
sqlcursor = mydb.cursor()
try:
sqlcursor.execute('CREATE DATABASE {}'.format(config.get("database")))
except mysql.connector.Error as err:
print(err)
return False
else:
print("Database created!")
return True

Accessing a remote database through SSH tunnel

I want to access a remote database through SSH tunnel.
server = SSHTunnelForwarder(
('172.17.9.125', 22),
ssh_password="123456",
ssh_username="root",
remote_bind_address=('127.0.0.1', 3306))
server.start()
database = pymysql.connect(host='127.0.0.1',
port=3306,
user='root',
passwd='root')
try:
dbsql = "CREATE DATABASE TestDB" # Create database
except Exception as e:
print("Error: ", e)
database.cursor().execute(dbsql)
global db, cursor
db = pymysql.connect(host='127.0.0.1', port=3306, user='root', passwd='root', db='TestDB')
cursor = db.cursor()
print("Connected to MySQL database")
f = open("testdb.sql") # Execute .sql file, creating data tables
full_sql = f.read()
sql_commands = full_sql.split(';')[:-1]
try:
for sql_command in sql_commands:
if sql_command is not None:
cursor.execute(sql_command)
else:
print("Null")
print("Created database")
except Exception as e:
print("Error: ", e)
I want to create a new database named as "TESTDB" on this remote server, 172.17.9.125.
But I get this database created in localhost. What am I doing wrong here?

Since you already have a database running, your 3306 port is used in that so won't be able to bind.
Solution : bind it to some other port and try to connect to that.
You can bind to another address using local_bind_address=('0.0.0.0', 1234) (this will be your target local address/port to which it will be bound) in your arguments to SSHTunnelForwarder.
So your connection TunnelForwarder should be something like this
server = SSHTunnelForwarder(
('172.17.9.125', 22),
ssh_password="123456",
ssh_username="root",
local_bind_address=('0.0.0.0', 1234),
remote_bind_address=('127.0.0.1', 3306))
And now connection will be made to port 1234
database = pymysql.connect(host='127.0.0.1',
port=1234,
user='root',
passwd='root')
db = pymysql.connect(host='127.0.0.1', port=1234, user='root', passwd='root', db='TestDB')

How would one connect an SQL db securely to an external client?

I'm attempting to connect a database, located on a web server, to a robot but I do not know how to connect the database to the robot. I would like the robot to run SELECT and UPDATE queries from the robot. The other issue is that I do not intend on using C-languages or Java; I plan on using python in the main control system.
I do know:
PHP
VBScript
Batch
Python
If anyone knows how to connect the DB to a bot it would be a great help.

So basically how to connect to an SQL DB in python? I'm working on a virtual bot right now doing the same thing. Look into the module , SQL-connector! http://www.mysqltutorial.org/python-connecting-mysql-databases/
You would start with creating a config.ini with your credentials
[mysql]
host = localhost
database = python_mysql
user = root
password =
Read Config.ini and return a dictionary
from configparser import ConfigParser
def read_db_config(filename='config.ini', section='mysql'):
""" Read database configuration file and return a dictionary object
:param filename: name of the configuration file
:param section: section of database configuration
:return: a dictionary of database parameters
"""
# create parser and read ini configuration file
parser = ConfigParser()
parser.read(filename)
# get section, default to mysql
db = {}
if parser.has_section(section):
items = parser.items(section)
for item in items:
db[item[0]] = item[1]
else:
raise Exception('{0} not found in the {1} file'.format(section, filename))
return db
and connect to MYSQL database
from mysql.connector import MySQLConnection, Error
from python_mysql_dbconfig import read_db_config
def connect():
""" Connect to MySQL database """
db_config = read_db_config()
try:
print('Connecting to MySQL database...')
conn = MySQLConnection(**db_config)
if conn.is_connected():
print('connection established.')
else:
print('connection failed.')
except Error as error:
print(error)
finally:
conn.close()
print('Connection closed.')
if __name__ == '__main__':
connect()
and update statement would look like the following
def update_book(book_id, title):
# read database configuration
db_config = read_db_config()
# prepare query and data
query = """ UPDATE books
SET title = %s
WHERE id = %s """
data = (title, book_id)
try:
conn = MySQLConnection(**db_config)
# update book title
cursor = conn.cursor()
cursor.execute(query, data)
# accept the changes
conn.commit()
except Error as error:
print(error)
finally:
cursor.close()
conn.close()
if __name__ == '__main__':
update_book(37, 'The Giant on the Hill *** TEST ***')