Can I connect to GHTorrent MySQL/Mongodb database through ssh? - python

I am trying to connect to GHTorrent database through ssh in python so I can deal with the data.
There is the example of how this ssh works in command lines and it works.
http://ghtorrent.org/mysql.html
import pymysql
from sshtunnel import SSHTunnelForwarder
mypkey = paramiko.RSAKey.from_private_key_file("/Users/***/.ssh/id_rsa")
with SSHTunnelForwarder(
('web.ghtorrent.org', 3306),
ssh_username="ghtorrent",
ssh_pkey=mypkey,
ssh_private_key_password="*****",#my password for my pc
remote_bind_address=('web.ghtorrent.org', 3306)) as server:
conn = pymysql.connect(host='127.0.0.1',
port=server.local_bind_port,
user='ght',
passwd='',
db='ghtorrent')
From my code the ssh can't connect to the server. I am not really sure whether my connection information is correct.
Since it uses a website name rather than the IP address so I have no idea whether it works.
Thank you so much!

Firstly, the GHTorrent SSH server is listening on port 22, you are trying to connect to 3306, which is incorrect.
Also, have you tried just specifying the SSH private key path directly?
i.e. ssh_pkey="/Users/***/.ssh/id_rsa"
SSHTunnelForwarder(
('web.ghtorrent.org', 22),
ssh_username="ghtorrent",
ssh_pkey="/Users/***/.ssh/id_rsa",
ssh_private_key_password="*****",#my password for my pc
remote_bind_address=('web.ghtorrent.org', 3306))

Related

Python MySQL Connection with SSH

Hi I have a shared hosting i bought and it allows for remote MySQL connection only with SSH.
So far I know that it doesn't have any Public or Private Keys..
And here's my connection setup on the MySQL Workbench which works when I try to connect:
I have looked at another stackoverflow question: Here but none of the answers seems to work for me.. :/ I'm really at a dead end and I need to get this to work. Can someone help me out please?
So I figured it out with like a million trial and error:
import pymysql
import paramiko
import pandas as pd
from paramiko import SSHClient
from sshtunnel import SSHTunnelForwarder
ssh_host = '198.54.xx.xx'
ssh_host_port = 21098 #Ur SSH port
ssh_username = "sshuser123" #Change this
ssh_password = "sshpassword123" #Change this
db_user = 'db user' #change this
db_password = 'password123' #change this
db = 'main_db' #The db that the user is linked to
with SSHTunnelForwarder(
(ssh_host, ssh_host_port),
ssh_username=ssh_username,
ssh_password=ssh_password,
remote_bind_address=('127.0.0.1', 3306)) as tunnel:
conn = pymysql.connect(host='127.0.0.1', user=db_user,
passwd=db_password, db=db,
port=tunnel.local_bind_port)
query = '''SELECT * from tablename;'''
data = pd.read_sql_query(query, conn)
print(data)
conn.close()
This is the code you should use if your SSH on MySql doesn't have any Public / Private Key.
Hope this helps anyone facing the same issue!!
Connect to server 198.54.x.240:21098 via ssh with port-forwarding
like ssh -t -gL 33069:localhost:3306 198.54.x.240
in windows use PuTTY, i like KiTTy (fork putty )
add connection and SSH Tunnel look at the pictures
Connect to MySQL via localhost:33069 (answer you know)WorkBench do the same, but 3306 on 3306, if you need more than 1 remote connection best practice forward different porst.

PyGreSQL/pg hangs when connecting to DB over SSH tunnel

In my Python script, I want to be able to connect to a Postgres DB via an SSH tunnel.
I'm using sshtunnel package to create a tunnel, and using PyGreSQL to connect to the DB.
When I try to establish the database connection, the pg.connect call just hangs. I don't get any errors at all. When I use psql to connect to the DB using the tunnel created by sshtunnel, the connection is successful.
When I create the tunnel beforehand using ssh in shell, pg.connect call successfully connects to the database.
So to summarize:
Tunnel created in Python/sshtunnel -> pg.connect call hangs
Tunnel created in Python/sshtunnel -> psql works just fine
Tunnel created using ssh -> pg.connect call is successful
This seems to be a problem with PyGreSQL since psql can access the DB using tunnel by sshtunnel just fine. However, there could be something different about the tunnel by sshtunnel package that I'm not seeing.
This is the command I'm using to create the tunnel using SSH:
ssh -g -L <local_bind_port>:localhost:<remote_bind_port> -f -N root#myip
Following is my code to connect to the DB in Python using SSH Tunnel and pg.connect
from sshtunnel import SSHTunnelForwarder
dbasename = 'db'
username = 'admin'
password = 'admin'
portnum = 5432
tunnel = SSHTunnelForwarder(
<ip_address>,
ssh_username="admin",
ssh_password="admin",
remote_bind_address=('127.0.0.1', portnum)
)
tunnel.start()
# The line below hangs
db = pg.connect(host=tunnel.local_bind_host, port=tunnel.local_bind_port, dbname=dbasename, user=username, passwd=password)
Any ideas about what could cause this problem? Are there any logs etc I that might help identify the problem?
Thanks.
EDIT:
It turns out that if I open a tunnel using python/SSHTunnel in one python shell, but use pg.connect to connect to that tunnel in the 2nd python shell it connects successfully.
So if I copy paste the following in the 1st shell:
from sshtunnel import SSHTunnelForwarder
dbasename = 'db'
username = 'admin'
password = 'admin'
portnum = 5432
tunnel = SSHTunnelForwarder(
<ip_address>,
ssh_username="admin",
ssh_password="admin",
remote_bind_address=('127.0.0.1', portnum)
)
tunnel.start()
then open another shell and connect to the tunnel from the 1st shell
import pg
# This works for some reason
db = pg.connect(host='127.0.0.1', port=<local hind port from the 1st shell>, dbname=dbasename, user=username, passwd=password)
the connection is successful

Connecting to mysql db via ssh with python

I have tried solutions found on other SO questions but none of them have worked for me. I am attempting to pull data from a mysql db running on a remote server by setting up an ssh tunnel. My code is as follows:
server = sshtunnel.SSHTunnelForwarder(
('10.6.41.10', 22),
ssh_username= 'serveruser',
ssh_password= 'serverpw',
remote_bind_address=('127.0.0.1', 3306))
server.start()
print(server.local_bind_port)
cnx = mysql.connector.connect(user='root', password='mysqlpw',
host='127.0.0.1',
database='mydb',
charset='utf8',
use_unicode='FALSE',
port = 3306)
However, when I run this code I receive:
1045 (28000): Access denied for user 'root'#'localhost' (using password: YES)
I have also tried adding
local_bind_address = ('0.0.0.0', 3306)
to the sshtunnel setup and instead recieved
Problem setting SSH Forwarder up: Couldn't open tunnel 0.0.0.0:3306 <> 127.0.0.1:3306 might be in use or destination not reachable
I don't fully understand the remote_bind_address and local_bind_address, so my guess is that must be doing something wrong there. I know my username/pw/server info is correct, I am able to ssh into my server via terminal and then use
mysql -h 127.0.0.1 -u root -p
to successfully log into my mysql server. So what do I need to fix to get it running in python? Thanks.
If you don't specify local_bind_address in sshtunnel.SSHTunnelForwarder, the local port is allocated randomly. In that case set port=server.local_bind_port in mysql.connector.connect().
Instead, you can also set local_bind_address=('0.0.0.0', [some port which is not in use]) in sshtunnel.SSHTunnelForwarder. The sshtunnel.HandlerSSHTunnelForwarderError ("Problem setting...") tells you that you can't use local_bind_address=('0.0.0.0', 3306).

How can I use python get connection to MySQL via pubilic IP?

I am trying to use python to get a connection to MySQL,
It is python code is :
import MySQLdb
conn= MySQLdb.connect(
host='public ip',
port = 3306,
user='root',
passwd='123456',
db ='test_schema',
)
so it always give the errors:
_mysql_exceptions.OperationalError: (2003, "Can't connect to MySQL server on 'pubilic ip' (10060)")
Building off AK47's answer
You need to first find the public facing ip address of your machine.
Go to google and search What's my ip address you should get a set of numbers xx.xx.xxxx.xx for example 12.42.111.2
Then in your mysql you need to modify the conf file if linux, ini file if windows
/etc/mysql/my.cnf Unix/OSX systems.
C:\Program Files\MySQL\MySQL Server 5.5\ Windows system
change bind-address to your ip address from google.
Then in AK47's answer replace host='127.0.0.1' to host=<ip from google>
Update your connection parameters to have an actual IP address in the 'host' field
import MySQLdb
conn= MySQLdb.connect(
host='127.0.0.1',
port = 3306,
user='root',
passwd='123456',
db ='test_schema')

Connecting to remote PostgreSQL database over SSH tunnel using Python

I have a problem with connecting to a remote database using SSH tunnel (now I'm trying with Paramiko). Here is my code:
#!/usr/bin/env python3
import psycopg2
import paramiko
import time
#ssh = paramiko.SSHClient()
#ssh.load_system_host_keys()
#ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
#ssh.connect('pluton.kt.agh.edu.pl', 22, username='aburban', password='pass')
t = paramiko.Transport(('pluton.kt.agh.edu.pl', 22))
t.connect(username="aburban", password='pass')
c = paramiko.Channel(t)
conn = psycopg2.connect(database="dotest")
curs = conn.cursor()
sql = "select * from tabelka"
curs.execute(sql)
rows = curs.fetchall()
print(rows)
A problem is that the program always tries to connect to the local database. I tried with other SSH tunnels and there was the same situation. Database on remote server exists and works fine using "classical" SSH connection via terminal.
You can try using sshtunnel module that uses Paramiko and it's Python 3 compatible.
Hopefully it helps you... I scratched myself the head for a while too to do it within Python code and avoid SSH external tunnels, we need to thank developers that wrap complex libraries into simple code!
Will be simple, generate a tunnel to port 5432 in localhost of remote server from a local port then you use it to connect via localhost to the remote DB.
This will be a sample working code for your needs:
#!/usr/bin/env python3
import psycopg2
from sshtunnel import SSHTunnelForwarder
import time
with SSHTunnelForwarder(
('pluton.kt.agh.edu.pl', 22),
ssh_password="password",
ssh_username="aburban",
remote_bind_address=('127.0.0.1', 5432)) as server:
conn = psycopg2.connect(database="dotest",port=server.local_bind_port)
curs = conn.cursor()
sql = "select * from tabelka"
curs.execute(sql)
rows = curs.fetchall()
print(rows)

Categories