I'm trying to connect to a SQL Server 2012 database using SQLAlchemy (with pyodbc) on Python 3.3 (Windows 7-64-bit). I am able to connect using straight pyodbc but have been unsuccessful at connecting using SQLAlchemy. I have dsn file setup for the database access.
I successfully connect using straight pyodbc like this:
con = pyodbc.connect('FILEDSN=c:\\users\\me\\mydbserver.dsn')
For sqlalchemy I have tried:
import sqlalchemy as sa
engine = sa.create_engine('mssql+pyodbc://c/users/me/mydbserver.dsn/mydbname')
The create_engine method doesn't actually set up the connection and succeeds, but
iIf I try something that causes sqlalchemy to actually setup the connection (like engine.table_names()), it takes a while but then returns this error:
DBAPIError: (Error) ('08001', '[08001] [Microsoft][ODBC SQL Server Driver][DBNETLIB]SQL Server does not exist or access denied. (17) (SQLDriverConnect)') None None
I'm not sure where thing are going wrong are how to see what connection string is actually being passed to pyodbc by sqlalchemy. I have successfully using the same sqlalchemy classes with SQLite and MySQL.
The file-based DSN string is being interpreted by SQLAlchemy as server name = c, database name = users.
I prefer connecting without using DSNs, it's one less configuration task to deal with during code migrations.
This syntax works using Windows Authentication:
engine = sa.create_engine('mssql+pyodbc://server/database')
Or with SQL Authentication:
engine = sa.create_engine('mssql+pyodbc://user:password#server/database')
SQLAlchemy has a thorough explanation of the different connection string options here.
In Python 3 you can use function quote_plus from module urllib.parse to create parameters for connection:
import urllib
params = urllib.parse.quote_plus("DRIVER={SQL Server Native Client 11.0};"
"SERVER=dagger;"
"DATABASE=test;"
"UID=user;"
"PWD=password")
engine = sa.create_engine("mssql+pyodbc:///?odbc_connect={}".format(params))
In order to use Windows Authentication, you want to use Trusted_Connection as parameter:
params = urllib.parse.quote_plus("DRIVER={SQL Server Native Client 11.0};"
"SERVER=dagger;"
"DATABASE=test;"
"Trusted_Connection=yes")
In Python 2 you should use function quote_plus from library urllib instead:
params = urllib.quote_plus("DRIVER={SQL Server Native Client 11.0};"
"SERVER=dagger;"
"DATABASE=test;"
"UID=user;"
"PWD=password")
I have an update info about the connection to MSSQL Server without using DSNs and using Windows Authentication. In my example I have next options:
My local server name is "(localdb)\ProjectsV12". Local server name I see from database properties (I am using Windows 10 / Visual Studio 2015).
My db name is "MainTest1"
engine = create_engine('mssql+pyodbc://(localdb)\ProjectsV12/MainTest1?driver=SQL+Server+Native+Client+11.0', echo=True)
It is needed to specify driver in connection.
You may find your client version in:
control panel>Systems and Security>Administrative Tools.>ODBC Data
Sources>System DSN tab>Add
Look on SQL Native client version from the list.
Just want to add some latest information here:
If you are connecting using DSN connections:
engine = create_engine("mssql+pyodbc://USERNAME:PASSWORD#SOME_DSN")
If you are connecting using Hostname connections:
engine = create_engine("mssql+pyodbc://USERNAME:PASSWORD#HOST_IP:PORT/DATABASENAME?driver=SQL+Server+Native+Client+11.0")
For more details, please refer to the "Official Document"
import pyodbc
import sqlalchemy as sa
engine = sa.create_engine('mssql+pyodbc://ServerName/DatabaseName?driver=SQL+Server+Native+Client+11.0',echo = True)
This works with Windows Authentication.
I did different and worked like a charm.
First you import the library:
import pandas as pd
from sqlalchemy import create_engine
import pyodbc
Create a function to create the engine
def mssql_engine(user = os.getenv('user'), password = os.getenv('password')
,host = os.getenv('SERVER_ADDRESS'),db = os.getenv('DATABASE')):
engine = create_engine(f'mssql+pyodbc://{user}:{password}#{host}/{db}?driver=SQL+Server')
return engine
Create a variable with your query
query = 'SELECT * FROM [Orders]'
Execute the Pandas command to create a Dataframe from a MSSQL Table
df = pd.read_sql(query, mssql_engine())
Related
Im trying to connect Sybase database with flask SQLalchemy using a ODBC connection .
My connection string :
'SQLALCHEMY_DATABASE_URI' = "sybase+pyodbc://username:passw#rd#host:port/dbname?driver=Adaptive+Server+Enterprise"
Getting this Error :
But I'm pretty sure i'm using the right port in the connection string. But when i try connect to connect to the same instance using this method .
Second method :
con = pyodbc.connect(server=server ,port=port ,username=username ,password=password ,driver=driver)
The connection works perfectly fine now with the same connection details.
Can anyone help me in building the connection string URL and help me fix this. Cause I want to use the "db Object" instead of "cursor Object ".
As noted in the Getting Connected wiki page, "Hostname Connections" are not supported. You can either create an ODBC DSN, or use your pyodbc connection string with an ODBC direct pass-through connection:
import urllib
from sqlalchemy import create_engine
connection_string = (
'DRIVER=SAP ASE ODBC driver;'
'SERVER=centos7-vm01;'
'PORT=5000;'
'UID=scott;PWD=tiger;'
'DATABASE=mydatabase;'
'charset=utf8;'
)
connection_uri = f"sybase+pyodbc:///?odbc_connect={urllib.parse.quote_plus(connection_string)}"
engine = create_engine(connection_uri)
After so much research I found that, this error is due incorrect parameter in the connection string.
Invalid port number error will be also fixed with this block of code .
Code here :
import urllib
connection_string = (
'DRIVER=Adaptive Server Enterprise;'
'SERVER=server;'
'PORT=port;'
'UID=username;PWD=password;'
'DATABASE=dbname;'
)
connection_uri = f"sybase+pyodbc:///?odbc_connect={urllib.parse.quote_plus(connection_string)}"
SQLALCHEMY_DATABASE_URI = connection_uri # connection string for SQLALchemy
This code is best solution .
Normally, when trying to connect to a SQL sever DB in Python, I use the pyodbc package like this:
import pyodbc
conn = pyodbc.connect("Driver={SQL Server};"
"Server=<server-ip>;"
"Database=<DB-name>;"
"UID=<user-name>;"
"PWD=<password>;"
"Trusted_Connection=yes;"
)
However, I don't know how to connect to a linked server in Python. If my linked server is called linked-server and has a DB called linked-DB for example; I have tried the same connection string as above, and changing the database name like this: "Database=<linked-server>.<linked-DB>;", since that's how I query the linked server DB in SSMS. But this doesn't work in Python.
Thank you very much for your help.
I've recently changed my project to use SQLAlchemy and my project runs fine, it used an external MySQL server.
Now I'm trying to work with a different MySQL server with SSL CA, and it doesn't connect.
(It did connect using MySQL Workbench, so the certificate should be fine)
I'm using the following code:
ssl_args = {'ssl': {'ca': ca_path}}
engine = create_engine("mysql+pymysql://<user>:<pass>#<addr>/<schema>",
connect_args=ssl_args)
and I get the following error:
Can't connect to MySQL server on '\addr\' ([WinError 10054] An existing connection was forcibly closed by the remote host)
Any suggestions?
I changed the DBAPI to MySQL-Connector, and used the following code:
ssl_args = {'ssl_ca': ca_path}
engine = create_engine("mysql+mysqlconnector://<user>:<pass>#<addr>/<schema>",
connect_args=ssl_args)
And now it works.
If you just connect from a client machine with an ssl connection (so you don't have access to the cert and key), you could simple add ssl=true to your uri.
Edit:
For example:
mysql_db = "mysql+mysqlconnector://<user>:<pass>#<addr>/<schema>?ssl=true"
The official doc is well documented:
engine = create_engine(
db_url,
connect_args={
"ssl": {
"ssl_ca": "ca.pem",
"ssl_cert": "client-cert.pem",
"ssl_key": "client-key.pem"
}
}
)
Another solution is to use sqlalchemy.engine.url.URL to define the URL and pass it to create_engine.
sqlUrl = sqlalchemy.engine.url.URL(
drivername="mysql+pymysql",
username=db_user,
password=db_pass,
host=db_host,
port=3306,
database=db_name,
query={"ssl_ca": "main_app/certs/BaltimoreCyberTrustRoot.crt.pem"},
)
create_engine(sqlUrl)
You can include SSL parameters as a dictionary in the query argument.
This approach is useful if you are using Flask to initialize the SqlAlchemy engine with a config parameter like SQLALCHEMY_DATABASE_URI rather than directly using create_engine.
I have a python code through which I am getting a pandas dataframe "df". I am trying to write this dataframe to Microsoft SQL server. I am trying to connect through the following code by I am getting an error
import pyodbc
from sqlalchemy import create_engine
engine = create_engine('mssql+pyodbc:///?odbc_connect=DRIVER={SQL Server};SERVER=bidept;DATABASE=BIDB;UID=sdcc\neils;PWD=neil!pass')
engine.connect()
df.to_sql(name='[BIDB].[dbo].[Test]',con=engine, if_exists='append')
However at the engine.connect() line I am getting the following error
sqlalchemy.exc.DBAPIError: (pyodbc.Error) ('08001', '[08001] [Microsoft][ODBC SQL Server Driver]Neither DSN nor SERVER keyword supplied (0) (SQLDriverConnect)')
Can anyone tell me what I am missing. I am using Microsoft SQL Server Management Studio - 14.0.17177.0
I connect to the SQL server through the following
Server type: Database Engine
Server name: bidept
Authentication: Windows Authentication
for which I log into my windows using username : sdcc\neils
and password : neil!pass
I am new to databases and python. Kindly let me know if you need any additional details. Any help will be greatly appreciated. Thank you in advance.
I was finally able to make it run.
import pyodbc
from sqlalchemy import create_engine
import urllib
params = urllib.quote_plus(r'DRIVER={SQL Server};SERVER=bidept;DATABASE=BIDB;Trusted_Connection=yes')
### For python 3.5: urllib.parse.quote_plus
conn_str = 'mssql+pyodbc:///?odbc_connect={}'.format(params)
engine = create_engine(conn_str)
reload(sys)
sys.setdefaultencoding('utf8')
df.to_sql(name='Test',con=engine, if_exists='append',index=False)
Thanks to #gord-thompson who answered Here
Although my in my sql server, all the tables are under the 'dbo' schema (i.e. dbo.Test1, dbo.Other_Tables) and this query puts my table in 'sdcc\neils' schema (i.e. sdcc\neils.Test1, sdcc\neils.Other_Tables) any solution to this?
Update: I've confirmed this is only a problem when using an Azure SQL instance. I can use the same conn string to connect to local, network, and remote SQL (AWS) instances - it is only failing when connecting to Azure. I can connect to the Azure instance with other tools, like Management Studio.
I am building a small Python(3.4.x)/Flask application. I'm a complete noob here so forgive me if I break any rules in posting.
I have created the database engine with:
from sqlalchemy import create_engine
from sqlalchemy.orm import scoped_session, sessionmaker
from sqlalchemy.ext.declarative import declarative_base
engine = create_engine('mssql+pymssql://dbadmin:dbadminpass#somedomain.server.net/databasename?charset=utf8')
db_session = scoped_session(sessionmaker(autocommit = False, autoflush = False, bind = engine))
Base = declarative_base()
Base.query = db_session.query_property()
def init_db():
import models
Base.metadata.creat_all(bind=engine)
Everything builds/interprets correctly at runtime but I get an error on running the query:
usr = User.query.filter_by(username=form.user.data).first()
The error is:
sqlalchemy.exc.OperationalError: (OperationalError) (20002, b'DB-Lib error message 20002, severity 9:\nAdaptive Server connection failed\n') None None
packages are: Flask==0.10.1, pymssql==2.1.1, SQLAlchemy==0.9.8
Thanks in advance.
I had similar problem and solved it by explicitly setting tds version = 7.0. FreeTDS reads the user's ${HOME}/.freetds.conf before resorting to the system-wide sysconfdir/freetds.conf. So, I created ~/.freetds.conf with [global] section as:
[global]
tds version = 7.0
You can find more information on freetds.con: http://www.freetds.org/userguide/freetdsconf.htm
As I just had the same problem.
Since I could get pymssql to connect bypassing sqlalchemy, I figured everything else should be fine, so I used the create_engine parameter connect_args to pass everything straight to pymssql.connect.
server_name = "sql_server_name"
server_addres = server_name + ".database.windows.net"
database = "database_name"
username = "{}#{}".format("my_username", server_name)
password = "strong_password"
arguments = dict(server=server_addres, user=username,
password=password, database=database, charset="utf8")
AZURE_ENGINE = create_engine('mssql+pymssql:///', connect_args=arguments)
This works fine and does not require one to meddle with the .freetds.conf file at all.
Also, note that pymssql requires usernname to be in the form username#servername. For more information see the linked documentation.