I imported some SQL libraries to be used on jupyter notebook, the SQL Server Management Studio credential is a Window Authentication (i.e i do not need a password to use SQL SSMS).
Below is the code snippet and an image that shows the output.
import sqlalchemy
import pyodbc
SERVER = 'SERVERNAME'
DATABASE = 'DATEBASENAME'
DRIVER = 'SQL Server Native Client 11.0'
USERNAME = 'MyUserName'
PASSWORD = ''
engine = sqlalchemy.create_engine('mssql+pyodbc://#' + SERVER + '/' + DATABASE + '?trusted_connection=yes&driver=ODBC+Driver+13+for+SQL+Server')
connection = engine.connect()
%reload_ext sql
%sql mssql+pyodbc://#SERVERNAME/DATABASENAME?driver=ODBC+Driver+13+for+SQL+Server&trusted_connection=yes
team_query = """
SQL_QUERY
"""
team = %sql $team_query
team = team.DataFrame()
How do i hide the output generated from the image above.
Maybe you could use the cell magic function %%capture --no-display:
how do you connect sql-server from jupyter and read data into tensorflow?
Solution:
here is a link for the driver-installation on possible linux distros.
and another link with installation und connection examples.
and solution code looks like as following,
import pandas as pd # data processing
import pyodbc
pyodbc.drivers()
server = 'MY_SERVER'
database = 'MY_DB'
username = 'DB_USER'
password = 'DB_PASS'
cnxn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};'
'SERVER='+server+
';DATABASE='+database+';UID='+username+';PWD='+ password)
cursor = cnxn.cursor()
query = 'SELECT * FROM XXX'
sales = pd.read_sql(query, cnxn)
print(sales.head(26))
print(sales.info())
I am trying to connect to the SQL Server database using Python3.4
This is the code that works for me
cnxn = pyodbc.connect('DRIVER={ODBC Driver 13 for SQL Server};SERVER=DESKTOP-GDM2HQ17\SQLEXPRESS;DATABASE=pyconnect;Trusted_Connection=yes')
and I login into my Management studio - database using Windows connection.
Here is the code, which is not working for me :
cnxn = pyodbc.connect('DRIVER={ODBC Driver 13 for SQL Server};SERVER=DESKTOP-GDM2HQ17\SQLEXPRESS;DATABASE=pyconnect;UID=DESKTOP-GDM2HQ17\sid;PWD=123')
Kindly share your thoughts on where I am going wrong.
There are two SQL Server Authentication modes:
1, Connecting Through Windows Authentication:
When a user connects through a Windows user account, SQL Server validates the account name and password using the Windows principal token in the operating system.
2, Connecting Through SQL Server Authentication:
When using SQL Server Authentication, logins are created in SQL Server that are not based on Windows user accounts. Both the user name and the password are created by using SQL Server and stored in SQL Server.
Your first code is working as it is Connecting Through Windows Authentication.
Your second code is not working as it is trying to find the credentials (login and password) which are stored in SQL Server, but the credentials is not created in SQL server.
Moreover, you could refer official doc to know how to Change Server Authentication Mode.
Hope it will help you.
DRIVER='{SQL Server}' works
below code is tested.....
import pyodbc
con = pyodbc.connect('Driver={SQL Server};'
'Server=LAPTOP-PPDS6BPG;'
'Database=training;'
'Trusted_Connection=yes;')
cursor = con.cursor()
sql_query = 'SELECT * FROM Students'
cursor.execute(sql_query)
for row in cursor:
print(row)
This works fine for me better than any other I could find
import pyodbc
import pandas as pd
conn = pyodbc.connect('Driver={SQL Server};'
'Server=10.****;'
'Database=Ma**;'
'UID=sql**;'
'PWD=sql**;')
cursor = conn.cursor()
sql = """\
EXEC [dbo].[GetNewPayment] #Login=?, #PasswordMD5=?, #RevokeTimeFrom=?, #RevokeTimeTo=? Status=?
"""
params = ('a***', 'c2ca***', '2021-05-01','2021-05-02', '3')
cursor.execute(sql, params)
I am trying to connect to teradata server and load a dataframe into a table using python. Here is my code -
import sqlalchemy
engine = sqlalchemy.create_engine("teradata://username:passwor#hostname:port/")
f3.to_sql(con=engine, name='sample', if_exists='replace', schema = 'schema_name')
But I am getting the following error -
InterfaceError: (teradata.api.InterfaceError) ('DRIVER_NOT_FOUND', "No driver found for 'Teradata'. Available drivers: SQL Server,SQL Server Native Client 11.0,ODBC Driver 13 for SQL Server")
Can anybody help me to figure out whats wrong in my approach?
There's is different ways to connect to Teradata in Python. The following list is not exhaustive.
SQLAlchemy
If you wish to use SQLAlchemy, you will also need to install the package SQLAlchemy-Teradata. Here is how you can connect:
from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base, DeferredReflection
from sqlalchemy.orm import scoped_session, sessionmaker
[...]
# Connect
engine = create_engine('teradata://' + user + ':' + password + '#' + host + ':22/' + database)
db_session = scoped_session(sessionmaker(autocommit=False, autoflush=False, bind=engine))
db_session.execute('SET SESSION CHARACTERISTICS AS TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;') # To avoid locking tables when doing select on tables
db_session.commit()
Base = declarative_base(cls=DeferredReflection)
Base.query = db_session.query_property()
Then you can use db_session to make queries. See SQLAlchemy Session API
Pyodbc
If you wish to use Pyodbc you will first need to install Teradata driver on your machine. Example on mine, after installing Teradata driver I have the following entry in /etc/odbcinst.ini
[Teradata]
Driver=/opt/teradata/client/16.00/odbc_64/lib/tdata.so
APILevel=CORE
ConnectFunctions=YYY
DriverODBCVer=3.51
SQLLevel=1
Then I can connect with the following:
import pyodbc
[...]
#Teradata Connection
connection= pyodbc.connect("driver={Teradata};dbcname=" + host + ";uid=" + user + ";pwd=" + pwd + ";charset=utf8;", autocommit=True)
connection.setdecoding(pyodbc.SQL_CHAR, encoding='utf-8')
connection.setdecoding(pyodbc.SQL_WCHAR, encoding='utf-8')
connection.setdecoding(pyodbc.SQL_WMETADATA, encoding='utf-8')
connection.setencoding(encoding='utf-8')
cursor= n.cursor()
cursor.execute("Select 'Hello World'")
for row in cursor:
print (row)
To connect to a teradata database, you need pyodbc, i also have problems with teradata dialect.
Example:
import pyodbc
user = 'user'
pasw = 'pass'
host = 'host'
connection = pyodbc.connect('DRIVER=Teradata;DBCNAME=' + host +';UID=' + user + ';PWD=' + pasw +';QUIETMODE=YES', autocommit=True,unicode_results=True)
I am not sure why your are using sqlalchemy. But you could explore using Teradata module to connect to Teradata as explained in the other link:
Connecting Python with Teradata using Teradata module
I met a similar problem in airflow, I used jars and jaydebeapi to connect teradata database and execute sql:
[root#myhost transfer]# cat test_conn.py
import jaydebeapi
from contextlib import closing
jclassname='com.teradata.jdbc.TeraDriver'
jdbc_driver_loc = '/opt/spark-2.3.1/jars/terajdbc4-16.20.00.06.jar,/opt/spark-2.3.1/jars/tdgssconfig-16.20.00.06.jar'
jdbc_driver_name = 'com.teradata.jdbc.TeraDriver'
host='my_teradata.address'
url='jdbc:teradata://' + host + '/TMODE=TERA'
login="teradata_user_name"
psw="teradata_passwd"
sql = "SELECT COUNT(*) FROM A_TERADATA_TABLE_NAME where month_key='202009'"
conn = jaydebeapi.connect(jclassname=jdbc_driver_name,
url=url,
driver_args=[login, psw],
jars=jdbc_driver_loc.split(","))
with closing(conn) as conn:
with closing(conn.cursor()) as cur:
cur.execute(sql)
print(cur.fetchall())
[root#myhost transfer]# python test_conn.py
[(7734133,)]
[root#myhost transfer]#
I am trying to connect to SQL through python to run some queries on some SQL databases on Microsoft SQL server. From my research online and on this forum the most promising library seems to be pyodbc. So I have made the following code
import pyodbc
conn = pyodbc.connect(init_string="driver={SQLOLEDB}; server=+ServerName+;
database=+MSQLDatabase+; trusted_connection=true")
cursor = conn.cursor()
and get the following error
Traceback (most recent call last):
File "C:\Users...\scrap.py", line 3, in <module>
conn = pyodbc.connect(init_string="driver={SQLOLEDB}; server=+ServerName+; database=+MSQLDatabase+; trusted_connection=true")
pyodbc.Error: ('IM002', '[IM002] [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified (0) (SQLDriverConnect)')
I have looked at the folowing posts and tried changing my driver to {sql server} and have connected using ODBC links before in SAS, which is partially what my above code is based on, so don't think I need to install anything else.
pyodbc.Error: ('IM002', '[IM002] [unixODBC][Driver Manager]Data source name not found, and no default driver specified (0) (SQLDriverConnect)')
Pyodbc - "Data source name not found, and no default driver specified"
Thanks
This is how I do it...
import pyodbc
cnxn = pyodbc.connect("Driver={SQL Server Native Client 11.0};"
"Server=server_name;"
"Database=db_name;"
"Trusted_Connection=yes;")
cursor = cnxn.cursor()
cursor.execute('SELECT * FROM Table')
for row in cursor:
print('row = %r' % (row,))
Relevant resources:
https://github.com/mkleehammer/pyodbc/wiki/Connecting-to-SQL-Server-from-Windows
http://blogs.msdn.com/b/cdndevs/archive/2015/03/11/python-and-data-sql-server-as-a-data-source-for-python-applications.aspx
Minor addition to what has been said before. You likely want to return a dataframe. This would be done as
import pypyodbc
import pandas as pd
cnxn = pypyodbc.connect("Driver={SQL Server Native Client 11.0};"
"Server=server_name;"
"Database=db_name;"
"uid=User;pwd=password")
df = pd.read_sql_query('select * from table', cnxn)
In data source connections between a client and server there are two general types: ODBC which uses a DRIVER and OLEDB which uses a PROVIDER. And in the programming world, it is a regular debate as to which route to go in connecting to data sources.
You are using a provider, SQLOLEDB, but specifying it as a driver. As far as I know, neither the pyodbc nor pypyodbc modules support Window OLEDB connections. However, the adodbapi does which uses the Microsoft ADO as an underlying component.
Below are both approaches for your connection parameters. Also, I string format your variables as your concatenation did not properly break quotes within string. You'll notice I double the curly braces since it is needed in connection string and string.format() also uses it.
# PROVIDER
import adodbapi
conn = adodbapi.connect("PROVIDER=SQLOLEDB;Data Source={0};Database={1}; \
trusted_connection=yes;UID={2};PWD={3};".format(ServerName,MSQLDatabase,username,password))
cursor = conn.cursor()
# DRIVER
import pyodbc
conn = pyodbc.connect("DRIVER={{SQL Server}};SERVER={0}; database={1}; \
trusted_connection=yes;UID={2};PWD={3}".format(ServerName,MSQLDatabase,username,password))
cursor = conn.cursor()
I Prefer this way ... it was much easier
http://www.pymssql.org/en/stable/pymssql_examples.html
conn = pymssql.connect("192.168.10.198", "odoo", "secret", "EFACTURA")
cursor = conn.cursor()
cursor.execute('SELECT * FROM usuario')
Following Python code worked for me. To check the ODBC connection, I first created a 4 line C# console application as listed below.
Python Code
import pandas as pd
import pyodbc
cnxn = pyodbc.connect("Driver={SQL Server};Server=serverName;UID=UserName;PWD=Password;Database=My_DW;")
df = pd.read_sql_query('select TOP 10 * from dbo.Table WHERE Patient_Key > 1000', cnxn)
df.head()
Calling a Stored Procedure
dfProcResult = pd.read_sql_query('exec dbo.usp_GetPatientProfile ?', cnxn, params=['MyParam'] )
C# Program to Check ODBC Connection
static void Main(string[] args)
{
string connectionString = "Driver={SQL Server};Server=serverName;UID=UserName;PWD=Password;Database=My_DW;";
OdbcConnection cn = new OdbcConnection(connectionString);
cn.Open();
cn.Close();
}
Try using pytds, it works throughout more complexity environment than pyodbc and more easier to setup.
I made it work on Ubuntu 18.04
Ref: https://github.com/denisenkom/pytds
Example code in documentation:
import pytds
with pytds.connect('server', 'database', 'user', 'password') as conn:
with conn.cursor() as cur:
cur.execute("select 1")
cur.fetchall()
Try with pymssql: pip install pymssql
import pymssql
try:
conn = pymssql.connect(server="host_or_ip", user="your_username", password="your_password", database="your_db")
cursor = conn.cursor()
cursor.execute ("SELECT ##VERSION")
row = cursor.fetchone()
print(f"\n\nSERVER VERSION:\n\n{row[0]}")
cursor.close()
conn.close()
except Exception:
print("\nERROR: Unable to connect to the server.")
exit(-1)
Output:
SERVER VERSION:
Microsoft SQL Server 2016 (SP2-CU14) (KB4564903) - 13.0.5830.85 (X64)
Jul 31 2020 18:47:07
Copyright (c) Microsoft Corporation
Standard Edition (64-bit) on Windows Server 2012 R2 Standard 6.3 <X64> (Build 9600: ) (Hypervisor)
The connection can also be checked from the terminal, with a single line of code with sqlcmd. See syntax.
╔═════════╦═════════════════════════════════════════╗
║ Command ║ Description ║
╠═════════╬═════════════════════════════════════════╣
║ -S ║ [protocol:]server[instance_name][,port] ║
║ -U ║ login_id ║
║ -p ║ password ║
║ -Q ║ "cmdline query" (and exit) ║
╚═════════╩═════════════════════════════════════════╝
sqlcmd -S "host_or_ip" -U "your_username" -p -Q "SELECT ##VERSION"
output:
Password: your_password
--------------------------------------------------------------------------------------------------------------------------------------------------------
Microsoft SQL Server 2016 (SP2-CU14) (KB4564903) - 13.0.5830.85 (X64)
Jul 31 2020 18:47:07
Copyright (c) Microsoft Corporation
Standard Edition (64-bit) on Windows Server 2012 R2 Standard 6.3 <X64> (Build 9600: ) (Hypervisor)
(1 rows affected)
Network packet size (bytes): 4096
1 xact[s]:
Clock Time (ms.): total 1 avg 1.00 (1000.00 xacts per sec.)
here's the one that works for me:
from sqlalchemy import create_engine
import urllib
import pandas
conn_str = (
r'Driver=ODBC Driver 13 for SQL Server;'
r'Server=DefinitelyNotProd;'
r'Database=PlayPen;'
r'Trusted_Connection=Yes;')
quoted_conn_str = urllib.parse.quote_plus(conn_str)
engine = create_engine('mssql+pyodbc:///?odbc_connect={}'.format(quoted_conn_str))
sqlcmd = """select * from information_schema.tables"""
df = pd.read_sql(sqlcmd, engine)
I tried to connect sql server in following ways and those worked for me.
To connect using windows authentication
import pyodbc
conn = pyodbc.connect('Driver={SQL Server};Server='+servername+';Trusted_Connection=yes;Database='+databasename+';')
cursor = conn.cursor()
cursor.execute("Select 1 as Data")
To use sql server authentication I used following code.
import pyodbc
conn = pyodbc.connect('Driver={SQL Server};Server='+servername+ ';UID='+userid+';PWD='+password+';Database='+databasename)
cursor1 = conn.cursor()
cursor1.execute("SELECT 1 AS DATA")
My version. Hope it helps.
import pandas.io.sql
import pyodbc
import sys
server = 'example'
db = 'NORTHWND'
db2 = 'example'
#Crear la conexión
conn = pyodbc.connect('DRIVER={SQL Server};SERVER=' + server +
';DATABASE=' + db +
';DATABASE=' + db2 +
';Trusted_Connection=yes')
#Query db
sql = """SELECT [EmployeeID]
,[LastName]
,[FirstName]
,[Title]
,[TitleOfCourtesy]
,[BirthDate]
,[HireDate]
,[Address]
,[City]
,[Region]
,[PostalCode]
,[Country]
,[HomePhone]
,[Extension]
,[Photo]
,[Notes]
,[ReportsTo]
,[PhotoPath]
FROM [NORTHWND].[dbo].[Employees] """
data_frame = pd.read_sql(sql, conn)
data_frame
This is how I had done it.
import pyodbc
connection = pyodbc.connect("DRIVER={SQL Server Native Client 10.0};"
"SERVER=server_name;"
"DATABASE=database_name;"
"UID=user_id_of_database;"
"PWD=password_of_database;")
cursor = connection.cursor()
cursor.execute('SELECT * FROM Table')
Always make sure you had specified the correct Driver. You can check your Driver by following the steps given below.
Open the Windows Control Panel.
Open the Administrative Tools folder.
Double-click Data Sources (ODBC) to open the ODBC Data Source Administrator window.
Click the Drivers tab
An alternative approach would be installing Microsoft ODBC Driver 13, then replace SQLOLEDB with ODBC Driver 13 for SQL Server
Regards.
I found up-to-date resources here:
Microsoft | SQL Docs | Python SQL Driver
There are these two options explained including all the prerequisites needed and code examples:
Python SQL driver - pyodbc (tested & working)
Python SQL driver - pymssql