I am having difficulty accessing MySQL remotely. I use SSH tunnel and want to connect the database MySQL using Python+SQLALchemy.
When i use MySQL-client in my console and specify "ptotocol=TCP", then everything is fine!
I use command:
mysql -h localhost —protocol=TCP -u USER -p
I get access to remote database through SSH-tunnel.
However, when I want to connect to the database using the Python+SQLAchemy I can't find such option like —protocol=TCP
Otherwise, i have only connect to local MySQL Databases.
Tell me please, is there a way to do it using SQLAlchemy.
The classic answer to this issue is to use 127.0.0.1 or the IP of the host or the host name instead of the "special name" localhost. From the documentation:
[...] connections on Unix to localhost are made using a Unix socket file by default
And later:
On Unix, MySQL programs treat the host name localhost specially, in a way that is likely different from what you expect compared to other network-based programs. For connections to localhost, MySQL programs attempt to connect to the local server by using a Unix socket file. This occurs even if a --port or -P option is given to specify a port number. To ensure that the client makes a TCP/IP connection to the local server, use --host or -h to specify a host name value of 127.0.0.1, or the IP address or name of the local server.
However, this simple trick doesn't appear to work in your case, so you have to somehow force the use of a TCP socket. As you explained it yourself, when invoking mysql on the command line, you use the --protocol tcp option.
As explained here, from SQLAlchemy, you can pass the relevant options (if any) to your driver either as URL options or using the connect_args keyword argument.
For example using PyMySQL, on a test system I've setup for that purpose (MariaDB 10.0.12, SQLAlchemy 0.9.8 and PyMySQL 0.6.2) I got the following results:
>>> engine = create_engine(
"mysql+pymysql://sylvain:passwd#localhost/db?host=localhost?port=3306")
# ^^^^^^^^^^^^^^^^^^^^^^^^^^
# Force TCP socket. Notice the two uses of `?`
# Normally URL options should use `?` and `&`
# after that. But that doesn't work here (bug?)
>>> conn = engine.connect()
>>> conn.execute("SELECT host FROM INFORMATION_SCHEMA.PROCESSLIST WHERE ID = CONNECTION_ID()").fetchall()
[('localhost:54164',)]
# Same result by using 127.0.0.1 instead of localhost:
>>> engine = create_engine(
"mysql+pymysql://sylvain:passwd#127.0.0.1/db?host=localhost?port=3306")
>>> conn = engine.connect()
>>> conn.execute("SELECT host FROM INFORMATION_SCHEMA.PROCESSLIST WHERE ID = CONNECTION_ID()").fetchall()
[('localhost:54164',)]
# Alternatively, using connect_args:
>>> engine = create_engine("mysql+pymysql://sylvain:passwd#localhost/db",
connect_args= dict(host='localhost', port=3306))
>>> conn = engine.connect()
>>> conn.execute("SELECT host FROM INFORMATION_SCHEMA.PROCESSLIST WHERE ID = CONNECTION_ID()").fetchall()
[('localhost:54353',)]
As you noticed, both will use a TCP connection (I know that because of the port number after the hostname). On the other hand:
>>> engine = create_engine(
"mysql+pymysql://sylvain:passwd#localhost/db?unix_socket=/path/to/mysql.sock")
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# Specify the path to mysql.sock in
# the `unix_socket` option will force
# usage of a UNIX socket
>>> conn = engine.connect()
>>> conn.execute("SELECT host FROM INFORMATION_SCHEMA.PROCESSLIST WHERE ID = CONNECTION_ID()").fetchall()
[('localhost',)]
# Same result by using 127.0.0.1 instead of localhost:
>>> engine = create_engine(
"mysql+pymysql://sylvain:passwd#127.0.0.1/db?unix_socket=/path/to/mysql.sock")
>>> conn = engine.connect()
>>> conn.execute("SELECT host FROM INFORMATION_SCHEMA.PROCESSLIST WHERE ID = CONNECTION_ID()").fetchall()
[('localhost',)]
# Alternatively, using connect_args:
>>> engine = create_engine("mysql+pymysql://sylvain:passwd#localhost/db",
connect_args= dict(unix_socket="/path/to/mysql.sock"))
>>> conn = engine.connect()
>>> conn.execute("SELECT host FROM INFORMATION_SCHEMA.PROCESSLIST WHERE ID = CONNECTION_ID()").fetchall()
[('localhost',)]
No port after the hostname: this is an UNIX socket.
This worked for me:
import pandas as pd
import pymysql
from sqlalchemy import create_engine
cnx = create_engine('mysql+pymysql://<username>:<password>#<host>/<dbname>')
df = pd.read_sql('SELECT * FROM <table_name>', cnx) #read the entire table
Where credentials are added to mysql database like this:
CREATE USER '<username>' IDENTIFIED BY '<password>';
GRANT ALL PRIVILEGES ON *.* TO '<username>' WITH GRANT OPTION;
FLUSH PRIVILEGES;
In my setup (I'm using mysql-python) just using 127.0.0.1 instead of localhost in the MySQL SQLAlchemy url works. The complete url I'm using exactly for that scenario (tunnel with local port 3307) is:
mysql:/user:passwd#127.0.0.1:3307/
I'm using SQLAlchemy 1.0.5, but I guess that doesn't matter too much...
I Tried this to connect with mysql db of xampp server
from sqlalchemy import create_engine
engine = create_engine("mysql+pymysql://usrnme:passwd#hstnme/dbname")
If you are using Python 3.x you can use:
pip install mysql-connector-python
Then:
import sqlalchemy as db
engine = db.create_engine("mysql+mysqlconnector://username:password#hostname:port/dbname")
Related
Hi I have a shared hosting i bought and it allows for remote MySQL connection only with SSH.
So far I know that it doesn't have any Public or Private Keys..
And here's my connection setup on the MySQL Workbench which works when I try to connect:
I have looked at another stackoverflow question: Here but none of the answers seems to work for me.. :/ I'm really at a dead end and I need to get this to work. Can someone help me out please?
So I figured it out with like a million trial and error:
import pymysql
import paramiko
import pandas as pd
from paramiko import SSHClient
from sshtunnel import SSHTunnelForwarder
ssh_host = '198.54.xx.xx'
ssh_host_port = 21098 #Ur SSH port
ssh_username = "sshuser123" #Change this
ssh_password = "sshpassword123" #Change this
db_user = 'db user' #change this
db_password = 'password123' #change this
db = 'main_db' #The db that the user is linked to
with SSHTunnelForwarder(
(ssh_host, ssh_host_port),
ssh_username=ssh_username,
ssh_password=ssh_password,
remote_bind_address=('127.0.0.1', 3306)) as tunnel:
conn = pymysql.connect(host='127.0.0.1', user=db_user,
passwd=db_password, db=db,
port=tunnel.local_bind_port)
query = '''SELECT * from tablename;'''
data = pd.read_sql_query(query, conn)
print(data)
conn.close()
This is the code you should use if your SSH on MySql doesn't have any Public / Private Key.
Hope this helps anyone facing the same issue!!
Connect to server 198.54.x.240:21098 via ssh with port-forwarding
like ssh -t -gL 33069:localhost:3306 198.54.x.240
in windows use PuTTY, i like KiTTy (fork putty )
add connection and SSH Tunnel look at the pictures
Connect to MySQL via localhost:33069 (answer you know)WorkBench do the same, but 3306 on 3306, if you need more than 1 remote connection best practice forward different porst.
I'm unable to connect to SQL Server from Python (3.4.4 64 bit) on Windows 10. This is what I did:
I found this nice library.
Then, I followed this page and installed FreeTDS
After that I installed pymssql with this command: easy_install pymssql
In SQL Server Network Configuration I enabled Named Pipes and TCP/IP for my SQLEXPRESS instance
So, at this moment I can run SQL Server and connect to my database, using SQL Server Management Studio. When I log in, I use DESKTOP-1JA5E9F\SQLEXPRESS as Server Name, sa as Login and 123 as Password. Besides, in Python shell I can import pymssql like:
>>> import pymssql
It does not raise any error. However, I can not connect to my database instance. I tried dozens attempts like:
conn = pymssql.connect(host=r'DESKTOP-1JA5E9F\SQLEXPRESS',
user=r'sa', password=r'123', database=r'reestr')
The code above ^^^ never completes (I see just blinking _ in the shell, that is blinking for ever). I also tried this:
conn = pymssql.connect(host=r'SQLEXPRESS', user=r'sa', password=r'123', database=r'reestr')
This results in pymssql.InterfaceError: Connection to the database failed for an unknown reason.. I tried this:
conn=pymssql.connect(host=r'SQLEXPRESS:1433',user=r'sa',password=r'123', database=r'reestr')
It also results in the same error message. So, I if anybody knows those magic voodoo spells that can do the trick, you are welcome.
Acording to the docs, there's no host keyword arg anymore, but server. Also it should be server name, not instance name, or full instance name (with server name). See connect() description , and examples of name construction for Connection class.
In your case server name is DESKTOP-1JA5E9F, also . and (local) should work since you do it all on local machine; your named instance name is SQLEXPRESS.
Try like these:
import pymssql
#for instance with known name 'SQLEXPRESS'
conn = pymssql.connect(server=r'DESKTOP-1JA5E9F\SQLEXPRESS',
user=r'sa', password=r'123', database=r'reestr')
#on localhost this should work too
conn = pymssql.connect(server=r'.\SQLEXPRESS',
user=r'sa', password=r'123', database=r'reestr')
#for default instance with port taken from freetds.conf
#(this probably won't work for your case, because you use named instance
#instead of default instance, which is named MSSQLSERVER)
conn = pymssql.connect(server=r'DESKTOP-1JA5E9F', user=r'sa', password=r'123',
database=r'reestr')
#for instance on known port '1433'
conn = pymssql.connect(server=r'DESKTOP-1JA5E9F:1433', user=r'sa', password=r'123',
database=r'reestr')
If this won't help, test the connection with tsql as described in the docs, e.g.:
tsql -H DESKTOP-1JA5E9F -p 1433 -U sa -P 123 -D reestr
or if you have freetds.conf:
tsql -S 'DESKTOP-1JA5E9F\SQLEXPRESS' -U sa -P 123 -D reestr
I have a problem with connecting to a remote database using SSH tunnel (now I'm trying with Paramiko). Here is my code:
#!/usr/bin/env python3
import psycopg2
import paramiko
import time
#ssh = paramiko.SSHClient()
#ssh.load_system_host_keys()
#ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
#ssh.connect('pluton.kt.agh.edu.pl', 22, username='aburban', password='pass')
t = paramiko.Transport(('pluton.kt.agh.edu.pl', 22))
t.connect(username="aburban", password='pass')
c = paramiko.Channel(t)
conn = psycopg2.connect(database="dotest")
curs = conn.cursor()
sql = "select * from tabelka"
curs.execute(sql)
rows = curs.fetchall()
print(rows)
A problem is that the program always tries to connect to the local database. I tried with other SSH tunnels and there was the same situation. Database on remote server exists and works fine using "classical" SSH connection via terminal.
You can try using sshtunnel module that uses Paramiko and it's Python 3 compatible.
Hopefully it helps you... I scratched myself the head for a while too to do it within Python code and avoid SSH external tunnels, we need to thank developers that wrap complex libraries into simple code!
Will be simple, generate a tunnel to port 5432 in localhost of remote server from a local port then you use it to connect via localhost to the remote DB.
This will be a sample working code for your needs:
#!/usr/bin/env python3
import psycopg2
from sshtunnel import SSHTunnelForwarder
import time
with SSHTunnelForwarder(
('pluton.kt.agh.edu.pl', 22),
ssh_password="password",
ssh_username="aburban",
remote_bind_address=('127.0.0.1', 5432)) as server:
conn = psycopg2.connect(database="dotest",port=server.local_bind_port)
curs = conn.cursor()
sql = "select * from tabelka"
curs.execute(sql)
rows = curs.fetchall()
print(rows)
I had installed mysql and on python shell import MySQLdb does work. I also changed the settings.py as:
**DATABASE_ENGINE = 'mysql'
DATABASE_NAME = '/home/database/my_db.db'
DATABASE_USER = ''
DATABASE_PASSWORD = ''
DATABASE_HOST = ''
DATABASE_PORT = ''**
but when I test for the connection it fails.
terminal:
>>> from django.db import connection
>>> cursor = connection.cursor()
It gives the error as :
OperationalError: (2002, "Can't connect to local MySQL server through socket '/var/run/mysql' (2)")
Django say its engine should be https://docs.djangoproject.com/en/dev/ref/settings/#engine 'django.db.backends.mysql'
I think database name is the name of the database inside mysql not a path. A path is just for sqlite. Do you have mysql server running?
USER, PASSWORD are required. host defaults to localhost port defualts to 3306.
Are you running a MySQL server? is it on:
localhost (unix domain socket), or
localhost (tcp), or
localhost but not accessible, or
some other host?
also, database name is name as mysql understands it, not a file path.
The location for mysql.sock on my system is /usr/local/mysql5/mysqld.sock
thrilllap-2:tmp reuven$ mysqld --print-defaults
mysqld would have been started with the following arguments:
--socket=/usr/local/mysql5/mysqld.sock --port=3306
When I try to use mysql via sqlalchemy from flask, I get:
File "build/bdist.macosx-10.6-intel/egg/MySQLdb/connections.py", line 187, in __init__
sqlalchemy.exc.OperationalError: (OperationalError) (2002, "Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2)") None None
The mysql program connects correctly to the database, as does every other mysql client on my system.
My my.cnf has the correct location for the socket
[client]
port = 3306
socket = /usr/local/mysql5/mysqld.sock
[safe_mysqld]
socket = /usr/local/mysql5/mysqld.sock
[mysqld_safe]
socket = /usr/local/mysql5/mysqld.sock
[mysqld]
socket = /usr/local/mysql5/mysqld.sock
port = 3306
The base "SQLAlchemy" library has an option where you can specify the location of the mysql.sock, but this isn't exposed through the sqlalchemy / flask library
http://packages.python.org/Flask-SQLAlchemy/config.html
My questions:
Where does sqlalchemy get the idea that /tmp/mysql.sock is the correct location?
Is there a way to change the default via the Flash-SQLAlchemy connector
You'll have to dig up the exact syntax, but for MySQL I think they use a unix_socket query opt. Something like:
mysql:///dbname?unix_socket=/opt/mysql/mysql.sock'
Should be your connect URI for SQLAlchemy.
Yes! Sean was right
app.config['SQLALCHEMY_DATABASE_URI'] = ''mysql://dayenu:secret.word#localhost/dayenu?unix_socket=/usr/local/mysql5/mysqld.sock
db = SQLAlchemy(app)
works fine! I think this parameter is used by pyodbc, which is what SQLAlchemy uses to talk to mysql, but I couldn't find this parameter anywhere in the pyodbc documentation.
You could create the conexión using
sqlalchemy.create_engine(
mysql_str = sqlalchemy.engine.url.URL(
drivername='mysql+pymysql',
username="db_user",
password="db_pass",
database=db_name,
query={
'unix_socket': '/usr/local/mysql5/mysqld.sock'
}
)
)