We Keep Coding

Connect to MySQL database (over SSH) in python script

I am in a situation where I need to access a MySQL database running on another computer (Computer B). Unfortunately, the computers aren't on the same local network. The only way I found that I can access the SQL database on Computer B from my laptop (Computer A) is via SSH.
My question is, can I access the SQL database over SSH from python?

you can use this snippet:
import pymysql
import paramiko
import pandas as pd
from paramiko import SSHClient
from sshtunnel import SSHTunnelForwarder
from os.path import expanduser
home = expanduser('~')
mypkey = paramiko.RSAKey.from_private_key_file(home + pkeyfilepath)
# if you want to use ssh password use - ssh_password='your ssh password', bellow
sql_hostname = 'sql_hostname'
sql_username = 'sql_username'
sql_password = 'sql_password'
sql_main_database = 'db_name'
sql_port = 3306
ssh_host = 'ssh_hostname'
ssh_user = 'ssh_username'
ssh_port = 22
sql_ip = '1.1.1.1.1'
with SSHTunnelForwarder(
(ssh_host, ssh_port),
ssh_username=ssh_user,
ssh_pkey=mypkey,
remote_bind_address=(sql_hostname, sql_port)) as tunnel:
conn = pymysql.connect(host='127.0.0.1', user=sql_username,
passwd=sql_password, db=sql_main_database,
port=tunnel.local_bind_port)
query = '''SELECT VERSION();'''
data = pd.read_sql_query(query, conn)
conn.close()

SSH tunnel forwarding with jump host and remote database

I have a remote MySQL database hosted on Amazon RDS ("D"). For security purposes, it is only accessible through a remote server ("C"). C is accessible via ssh through a jump host "B". I need a double ssh tunnel to then access a remote SQL host.
[A: local host] -> [B: jump host] -> [C: target host] => [D: RDS MySQL host]
I would like to access D through Python, using paramiko and/or sshtunnel. All of the information I can find involves:
a single ssh tunnel and a remote SQL host (ex. A -> C => D, no jump host)
ssh first with mysqldb in python
python mysql connectivity via ssh
a double ssh tunnel to an SQL host (ex. A -> B -> C, D is hosted on C).
Connecting to remote Postgresql database over ssh tunnel using python
Paramiko: Port Forwarding Around A NAT Router
Nested SSH session with Paramiko
So far, I'm using paramiko with a proxy command to get from A to C. I can access D by executing a command on C, but not by connecting with mysqldb or sqlalchemy (my ultimate goal).
My current code:
import paramiko
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
proxy = paramiko.ProxyCommand("ssh -A B_username#B_host -W C_host:12345")
ssh.connect("C_host", username="C_username", sock=proxy)
stdin, stdout, stderr = ssh.exec_command("mysql -u D_username -p D_password -h D_host_rds")
print("STDOUT:\n{}\n\nSTDERR:\n{}\n".format(stdout.read(), stderr.read()))
# successfully prints out MySQL welcome screen
I'm looking for something like this (modified from example 2 in the sshtunnel docs):
import paramiko
from sshtunnel import SSHTunnelForwarder
with SSHTunnelForwarder(
intermediate = {
("B_host", 22),
ssh_username = "B_username",
ssh_password = "B_password")},
remote = {
("C_host", 12345),
ssh_username = "C_username",
ssh_password = "C_password")},
remote_bind_address=("D_host_rds", 3306),
local_bind_address=("0.0.0.0", 3307)) as server:
conn = MySQLdb.connect(
user = "D_username",
passwd = "D_password",
db = "my_database",
host = "127.0.0.1",
port = 3307)
tl;dr: How do I forward a port through two ssh jumps in Python?

I figured it out. It works with a combination of ssh config settings and the SSHTunnelForwarder context manager from the sshtunnel library.
Using the following model and naming conventions:
[A: local host] -> [B: jump host] -> [C: target host] => [D: RDS MySQL host]
I set up my ~/.ssh/config to get from A to C through B:
Host C_ssh_shortcut
HostName C_host
User C_user
Port 22
ForwardAgent yes
ProxyCommand ssh B_user#B_host -W %h:%p
I added the key/keys I used to log in to B and C to my ssh-agent:
ssh-add
And finally I set up SSHTunnelForwarder:
import sqlalchemy
from sshtunnel import SSHTunnelForwarder
with SSHTunnelForwarder(
"C_ssh_shortcut", # The SSHTunnelForwarder "ssh_address_or_host" argument, which takes care of bypassing B through the ProxyCommand set up in ~/.ssh/config
remote_bind_address=(D_host, 3306), # Points to your desired destination, ie. database host on 3306, which is the MySQL port
local_bind_address=('', 1111) # Gives a local way to access this host and port on your machine. '' is localhost / 127.0.0.1, 1111 is an unused port
) as server:
connection_string = "mysql+pymysql://D_user:D_password#localhost:1111/D_dbname" # note that D_host and D_port were replaced by the host and port defined in "local_bind_address"
engine = sqlalchemy.create_engine(connection_string)
# do your thing
From here, I am able to use my engine as usual to interact with my database.

This code work for me
import pymysql
import paramiko
from paramiko import SSHClient
from sshtunnel import SSHTunnelForwarder
from sqlalchemy import create_engine
#ssh config
mypkey = paramiko.RSAKey.from_private_key_file('your/user/location/.ssh/id_rsa')
ssh_host = 'your_ssh_host'
ssh_user = 'ssh_host_username'
ssh_port = 22
#mysql config
sql_hostname = 'your_mysql_host name'
sql_username = 'mysql_user'
sql_password = 'mysql_password'
sql_main_database = 'your_database_name'
sql_port = 3306
host = '127.0.0.1'
with SSHTunnelForwarder(
(ssh_host, ssh_port),
ssh_username=ssh_user,
ssh_pkey=mypkey,
remote_bind_address=(sql_hostname, sql_port)) as tunnel:
engine = create_engine('mysql+pymysql://'+sql_username+':'+sql_password+'#'+host+':'+str(tunnel.local_bind_port)+'/'+sql_main_database)
connection = engine.connect()
print('engine creating...')
sql = text(""" select * from nurse_profiles np limit 50""")
nurseData = connection.execute(sql)
connection.close()
nurseList = []
for row in nurseData:
nurseList.append(dict(row))
print('nurseList len: ', len(nurseList))
print('nurseList: ', nurseList)

I use this code for PostgreSQL database and it works. I am sure it will work too if use MySQL database. I change the PostgreSQL database part here to MySQL, here is the code:
import pymysql
import paramiko
import sqlalchemy
from sshtunnel import SSHTunnelForwarder
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
import pandas as pd
#SSH config
mypkey = paramiko.RSAKey.from_private_key_file('id_rsa_file', password = 'id_rsa_password')
ssh_host = 'your_ssh_host'
ssh_user = 'your_ssh_host_username'
ssh_port = 22
#SQL config
sql_hostname = 'your_sql_host_name'
sql_username = 'sql_user'
sql_password = 'sql_password'
sql_main_database = 'your_database_name'
sql_port = 3306
host = '127.0.0.1'
with SSHTunnelForwarder((ssh_host, ssh_port),
ssh_username=ssh_user,
ssh_pkey=mypkey,
remote_bind_address=(sql_hostname, sql_port)) as tunnel:
#Connect to SQL
local_port = str(tunnel.local_bind_port)
engine = create_engine(f'mysql+pymysql://{sql_username}:{sql_password}#127.0.0.1:' + local_port +f'/{sql_main_database}')
Session = sessionmaker(bind = engine)
session = Session()
print('Database session created!')
#To inspect the schemas and tables in your database
inspector = inspect(engine)
schemas = inspector.get_schema_names()
for schema in schemas:
print(f'schema:{schema}')
for table_name in inspector.get_table_names(schema = schema):
print(f'table: {table_name}')
query_code = "your query code from SQL here"
#Execute query code
exec_database = session.execute(query_code)
df = pd.DataFrame(exec_database.fetchall())
df.columns = exec_database.keys()
print('Dataframe created from database!')
session.close()
engine.dispose()
You can also change the part below:
#Execute query code
exec_database = session.execute(query_code)
df = pd.DataFrame(exec_database.fetchall())
df.columns = exec_database.keys()
to read SQL query directly using pandas using code below:
df = pd.read_sql_query(query_code, engine)
Additionally, part of code below:
#To inspect the schemas and tables in your database
inspector = inspect(engine)
schemas = inspector.get_schema_names()
for schema in schemas:
print(f'schema:{schema}')
for table_name in inspector.get_table_names(schema = schema):
print(f'table: {table_name}')
is only necessary when you don't have any idea what schemas and tables are in your database. You can use those codes above to inspect and show them.

Connecting to PostgreSQL database through SSH tunneling in Python

I am trying to connect to a server remotely and then access it's local database with Python. I am successfully connecting to the server, although I can't seem to connect to the database on the server. My code is below:
import psycopg2
from sshtunnel import SSHTunnelForwarder
try:
with SSHTunnelForwarder(
('<server ip address>', 22),
ssh_private_key="</path/to/private/ssh/key>",
ssh_username="<server username>",
remote_bind_address=('localhost', 5432)) as server:
print "server connected"
conn = psycopg2.connect(database="<dbname>",port=server.local_bind_port)
curs = conn.cursor()
print "database connected
except:
print "Connection Failed"
These are pieces of code I have found on the internet and pieced together. I have also tried the connection statements below in place of the code above:
params = {
'database': '<dbname>',
'user': '<dbusername>',
'password': '<dbuserpass>',
'host': 'localhost',
'port': 5432
}
conn = psycopg2.connect(**params)
I know I can connect to the database because on my machine; I am able to use sqlectron to tunnel in and connect appropriately.
Just in case it is not clear what I am trying to do from above, I need to ssh tunnel into my remote server using a private ssh key on my computer (working properly), and then I need to connect to a PostgreSQL database that is on localhost at port 5432.
I am currently getting the current error message for both ways of trying to connect:
2016-01-23 11:16:10,978 | ERROR | Tunnel: 0.0.0.0:49386 <> localhost:5432 error: (9, 'Bad file descriptor')

I don't know if this may be helpful, but I had to connect to a PostgreSQL database through SSH tunneling as well. I succeeded to connect using your code with some modifications:
import psycopg2
from sshtunnel import SSHTunnelForwarder
try:
with SSHTunnelForwarder(
('<server ip address>', 22),
#ssh_private_key="</path/to/private/ssh/key>",
### in my case, I used a password instead of a private key
ssh_username="<server username>",
ssh_password="<mypasswd>",
remote_bind_address=('localhost', 5432)) as server:
server.start()
print "server connected"
params = {
'database': '<dbname>',
'user': '<dbusername>',
'password': '<dbuserpass>',
'host': 'localhost',
'port': server.local_bind_port
}
conn = psycopg2.connect(**params)
curs = conn.cursor()
print "database connected"
except:
print "Connection Failed"
After adding server.start(), the code worked nicely. Furthermore, inverted commas were missing after 'database connected'.
I hope this might be helpful to you, thanks for sharing your code!

Both these examples were very helpful. I just needed to combine the good parts from both.
from sshtunnel import SSHTunnelForwarder #Run pip install sshtunnel
from sqlalchemy.orm import sessionmaker #Run pip install sqlalchemy
from sqlalchemy import create_engine
with SSHTunnelForwarder(
('<remote server ip>', 22), #Remote server IP and SSH port
ssh_username = "<username>",
ssh_password = "<password>",
remote_bind_address=('<local server ip>', 5432)) as server: #PostgreSQL server IP and sever port on remote machine
server.start() #start ssh sever
print 'Server connected via SSH'
#connect to PostgreSQL
local_port = str(server.local_bind_port)
engine = create_engine('postgresql://<username>:<password>#127.0.0.1:' + local_port +'/database_name')
Session = sessionmaker(bind=engine)
session = Session()
print 'Database session created'
#test data retrieval
test = session.execute("SELECT * FROM database_table")
for row in test:
print row['id']
session.close()

SQLAlchemy through Paramiko SSH

I have a database on a server which I need to access through SSH. Right now I deal with the DB by using the command line to get the data.
import paramiko
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(hostname='XX.XX.XX', username='user', password='pass', port = YYY)
query = "mysql -u " + username_sql + " -p" + password_sql +" dbb -e \"" + sql_query + "\""
ssh.exec_command(query.decode('string_escape'))
ssh.close()
Is there a way to do this with SQLAlchemy to be more efficient and so I can work with pandas DataFrames directly?
from sqlalchemy import create_engine
engine = create_engine(
"mysql://username_sql:password_sql#localhost/dbb")

In case there is anyone who's interested in connecting to a remote Postgresql database via SSH and wants to load data into a pandas DataFrame here is how to do it.
Suppose we have installed a postgresql database on a remote server, to which we can ssh by the following parameters.
SSH parameters:
Server's ip: 10.0.0.101
SSH port: 22 (default port for SSH)
Username: my_username
Password: my_password
Database parameters:
Port: 5432 (postgresql default port)
Database name: db
Database user: postgres_user (default username is postgres)
Database password: postgres_pswd (default password is an empty string)
Table with our data: MY_TABLE
Now, we want to connect to this database on our end and load data into a pandas DataFrame:
from sshtunnel import SSHTunnelForwarder
from sqlalchemy import create_engine
import pandas as pd
server = SSHTunnelForwarder(
('10.0.0.101', 22),
ssh_username="my_username",
ssh_password="my_password",
remote_bind_address=('127.0.0.1', 5432)
)
server.start()
local_port = str(server.local_bind_port)
engine = create_engine('postgresql://{}:{}#{}:{}/{}'.format("postgres_user", "postgres_pswd", "127.0.0.1", local_port, "db"))
dataDF = pd.read_sql("SELECT * FROM \"{}\";".format("MY_TABLE"), engine)
server.stop()

The easiest way to do this would be to run an SSH tunnel to the mysql port on the remote host. For example:
ssh -f user#XX.XX.XX.XX -L 3307:mysql1.example.com:3306 -N
Then connect locally with SQLAlchemy:
engine = create_engine("mysql://username_sql:password_sql#localhost:3307/dbb")
If you really want to use paramiko, try this demo code in the paramiko repo or the sshtunnel module. The ssh command might be the easiest method though.. and you can use autossh to restart the tunnel if it goes down.

You could use the SSHTunnel library as follows:
from sshtunnel import SSHTunnelForwarder #Run pip install sshtunnel
from sqlalchemy.orm import sessionmaker #Run pip install sqlalchemy
with SSHTunnelForwarder(
('10.160.1.24', 22), #Remote server IP and SSH port
ssh_username = "<usr>",
ssh_password = "<pwd>",
remote_bind_address=('127.0.0.1', 5432)
) as server:
server.start() #start ssh sever
print 'Server connected via SSH'
#connect to PostgreSQL
local_port = str(server.local_bind_port)
engine = create_engine('postgresql://<db_user>:<db_pwd>#127.0.0.1:' + local_port +'/<db_name>')
Session = sessionmaker(bind=engine)
session = Session()
print 'Database session created'
#test data retrieval
test = session.execute("SELECT * FROM <table_name>")

Just swap the (host, port) of the server with postgres:
from sshtunnel import SSHTunnelForwarder #Run pip install sshtunnel
server = SSHTunnelForwarder(
(<'your host'>, <host port>),
ssh_username=<"os remote username">,
ssh_pkey=<'path/to/key.pem'>, # or ssh_password.
remote_bind_address=(<'postgres db host'>, <'postgres db port'>))
server.start()
connection_data = 'postgresql://{user}:{password}#{host}:{port}/{db}'.format(user=<'postgres user'>,
password=<'postgres password'>,
host=server.local_bind_host,
port=server.local_bind_port,
db=<'postgres db name'>)
engine = create_engine(connection_data)
# Do your queries
server.stop()

I'm going to piggy back #Matin Kh with a non-postgresql db - MySQL and using Pythonanywhere.com.
This code will take a table and convert it to an excel file.
import sshtunnel
import sqlalchemy
import pymysql
import pandas as pd
from pandas import ExcelWriter
import datetime as dt
from sshtunnel import SSHTunnelForwarder
server = SSHTunnelForwarder(
('ssh.pythonanywhere.com'),
ssh_username='username',
ssh_password='password',
remote_bind_address=('username.mysql.pythonanywhere-services.com', 3306) )
server.start()
local_port = str(server.local_bind_port)
db = 'username$database'
engine = sqlalchemy.create_engine(f'mysql+pymysql://username:password#127.0.0.1:{local_port}/{db}')
print('Engine Created')
df_read = pd.read_sql_table('tablename',engine)
print('Grabbed Table')
writer = ExcelWriter('excelfile.xlsx')
print('writer created')
df_read.to_excel(writer,'8==D') # '8==D' specifies sheet
print('df to excel')
writer.save()
print('saved')
server.stop()

Connecting to MySQL database via SSH

I am trying to connect my python program to a remote MySQL Database via SSH.
I am using Paramiko for SSH and SQLAlchemy.
Here is what I have so far:
import paramiko
from sqlalchemy import create_engine
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect('host', port=port, username='user', password='pass')
engine = create_engine('mysql+mysqldb://user:pass#host/db')
I am getting an error:
sqlalchemy.exc.OperationalError: (_mysql_exceptions.OperationalError) (2003, "Can't connect to MySQL server on 'mcsdev.croft-it.com' (60)")

Sorry I posted a duplicated answer before. Here is a more elaborated answer tailored exactly to your question ;)
If you still in need of connecting to a remote MySQL db via SSH I have used a library named sshtunnel, that wraps ands simplifies the use of paramiko (a dependency of the sshtunnel).
With this code I think you will be good to go:
from sshtunnel import SSHTunnelForwarder
from sqlalchemy import create_engine
server = SSHTunnelForwarder(
('host', 22),
ssh_password="password",
ssh_username="username",
remote_bind_address=('127.0.0.1', 3306))
server.start()
engine = create_engine('mysql+mysqldb://user:pass#127.0.0.1:%s/db' % server.local_bind_port)
# DO YOUR THINGS
server.stop()

This code work for me
import pymysql
import paramiko
from paramiko import SSHClient
from sshtunnel import SSHTunnelForwarder
from sqlalchemy import create_engine
#ssh config
mypkey = paramiko.RSAKey.from_private_key_file('your/user/location/.ssh/id_rsa')
ssh_host = 'your_ssh_host'
ssh_user = 'ssh_host_username'
ssh_port = 22
#mysql config
sql_hostname = 'your_mysql_host name'
sql_username = 'mysql_user'
sql_password = 'mysql_password'
sql_main_database = 'your_database_name'
sql_port = 3306
host = '127.0.0.1'
with SSHTunnelForwarder(
(ssh_host, ssh_port),
ssh_username=ssh_user,
ssh_pkey=mypkey,
remote_bind_address=(sql_hostname, sql_port)) as tunnel:
engine = create_engine('mysql+pymysql://'+sql_username+':'+sql_password+'#'+host+':'+str(tunnel.local_bind_port)+'/'+sql_main_database)
connection = engine.connect()
print('engine creating...')
sql = text(""" select * from nurse_profiles np limit 50""")
nurseData = connection.execute(sql)
connection.close()
nurseList = []
for row in nurseData:
nurseList.append(dict(row))
print('nurseList len: ', len(nurseList))
print('nurseList: ', nurseList)

Using external ubuntu server
With ssh key on digital ocean
The accepted answer did not work for me, I had to specify the ssh_private_key, which is the path to your private key
from sqlalchemy import create_engine
from sshtunnel import SSHTunnelForwarder
server = SSHTunnelForwarder(
('133.22.166.19', 22),
ssh_password="123ABC123",
ssh_username="erfan",
ssh_private_key=r'C:\Users\Erfan\.ssh\id_rsa',
remote_bind_address=('127.0.0.1', 3306)
)
server.start()
engine = create_engine(
f'mysql+mysqldb://root:safepassword123#127.0.0.1:{server.local_bind_port}'
)
dbs = engine.execute('SHOW DATABASES;')
for db in dbs:
print(db)
server.stop()