I imported some SQL libraries to be used on jupyter notebook, the SQL Server Management Studio credential is a Window Authentication (i.e i do not need a password to use SQL SSMS).
Below is the code snippet and an image that shows the output.
import sqlalchemy
import pyodbc
SERVER = 'SERVERNAME'
DATABASE = 'DATEBASENAME'
DRIVER = 'SQL Server Native Client 11.0'
USERNAME = 'MyUserName'
PASSWORD = ''
engine = sqlalchemy.create_engine('mssql+pyodbc://#' + SERVER + '/' + DATABASE + '?trusted_connection=yes&driver=ODBC+Driver+13+for+SQL+Server')
connection = engine.connect()
%reload_ext sql
%sql mssql+pyodbc://#SERVERNAME/DATABASENAME?driver=ODBC+Driver+13+for+SQL+Server&trusted_connection=yes
team_query = """
SQL_QUERY
"""
team = %sql $team_query
team = team.DataFrame()
How do i hide the output generated from the image above.
Maybe you could use the cell magic function %%capture --no-display:
Related
in my jupyter notebook I connect to snowflake with an externalbrowser auth like so:
conn = snowflake.connector.connect(
user='<my user>',
authenticator='externalbrowser',
account='<my account>',
warehouse='<the warehouse>')
this opens an external browser to auth and after that works fine with pandas read sql:
pd.read_sql('<a query>', conn)
want to use it with ipython sql, but when I try:
%sql snowflake://conn.user#conn.account
I get:
snowflake.connector.errors.ProgrammingError) Password is empty
well I don't have one :)
any ideas how to pass this?
IPython-sql connection strings are SQLAlchemy URL standard, therefore you can do the following:
%load_ext sql
from sqlalchemy import create_engine
from snowflake.sqlalchemy import URL
engine = create_engine(URL(
account = '<account>',
user = '<user>',
database = 'testdb',
schema = 'public',
warehouse = '<wh>',
role='public',
authenticator='externalbrowser'
))
connection = engine.connect()
This would open the external browser for authentication.
I am using a Python script to connect to a SQL Server database:
import pyodbc
import pandas
server = 'SQL'
database = 'DB_TEST'
username = 'USER'
password = 'My password'
sql='''
SELECT *
FROM [DB_TEST].[dbo].[test]
'''
cnxn = pyodbc.connect('DRIVER=SQL Server;SERVER='+server+';DATABASE='+database+';UID='+username+';PWD='+ password)
data = pandas.read_sql(sql,cnxn)
cnxn.close()
The script is launched everyday by an automatisation tools so there is no physical user.
The issue is how to replace the password field by a secure method?
The automated script is still ran by a windows user. Add this windows user to the SQL-Server users and give it the appropriate permissions, so you can use:
import pyodbc
import pandas
server = 'SQL'
database = 'DB_TEST'
sql='''
SELECT *
FROM [DB_TEST].[dbo].[test]
'''
cnxn = pyodbc.connect(
f'DRIVER=SQL Server;SERVER={server};DATABASE={database};Trusted_Connection=True;')
data = pandas.read_sql(sql,cnxn)
cnxn.close()
I am also interested in secure coding using Python .I did my own research to figure out available options, I would recommend reviewing this post as it summarize it all. Check on the listed options, and apply the one suits you better.
Not able to connect to Azure DB. I get the following error while connecting via Python.
I'm able to connect to my usual SQL environment
import pandas as pd
import pymssql
connPDW = pymssql.connect(host=r'dwprd01.database.windows.net', user=r'internal\admaaron',password='',database='')
connPDW.autocommit(True)
cursor = connPDW.cursor()
conn.autocommit(True)
cursor = conn.cursor()
sql = """
select Top (10) * from TableName
"""
cursor.execute(sql);
Run without errors.
Just according to your code, there is an obvious issue of connecting Azure SQL Database by pymssql package in Python which use the incorrect user format and lack of the values of password and database parameters.
Please follow the offical document Step 3: Proof of concept connecting to SQL using pymssql carefully to change your code correctly.
If you have an instance of Azure SQL Database with the connection string of ODBC, such as Driver={ODBC Driver 13 for SQL Server};Server=tcp:<your hostname>.database.windows.net,1433;Database=<your database name>;Uid=<username>#<host>;Pwd=<your_password>;Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30; show in the Connection strings tab of your SQL Database on Azure portal.
Then, your code should be like below
hostname = '<your hostname>'
server = f"{hostname}.database.windows.net"
username = '<your username>'
user = f"{username}#{hostname}"
password = '<your password>'
database = '<your database name>'
import pymssql
conn = pymssql.connect(server=server, user=user, password=password, database=database)
Meanwhile, just additional note for the version of Azure SQL Database and MS SQL Server are 2008+ like the latest Azure SQL Database, you should use the ODBC Driver connection string which be started with DRIVER={ODBC Driver 17 for SQL Server};, not 13 show in the connection string of Azure portal if using ODBC driver for Python with pyodbc, please refer to the offical document Step 3: Proof of concept connecting to SQL using pyodbc.
I am trying to connect to the SQL Server database using Python3.4
This is the code that works for me
cnxn = pyodbc.connect('DRIVER={ODBC Driver 13 for SQL Server};SERVER=DESKTOP-GDM2HQ17\SQLEXPRESS;DATABASE=pyconnect;Trusted_Connection=yes')
and I login into my Management studio - database using Windows connection.
Here is the code, which is not working for me :
cnxn = pyodbc.connect('DRIVER={ODBC Driver 13 for SQL Server};SERVER=DESKTOP-GDM2HQ17\SQLEXPRESS;DATABASE=pyconnect;UID=DESKTOP-GDM2HQ17\sid;PWD=123')
Kindly share your thoughts on where I am going wrong.
There are two SQL Server Authentication modes:
1, Connecting Through Windows Authentication:
When a user connects through a Windows user account, SQL Server validates the account name and password using the Windows principal token in the operating system.
2, Connecting Through SQL Server Authentication:
When using SQL Server Authentication, logins are created in SQL Server that are not based on Windows user accounts. Both the user name and the password are created by using SQL Server and stored in SQL Server.
Your first code is working as it is Connecting Through Windows Authentication.
Your second code is not working as it is trying to find the credentials (login and password) which are stored in SQL Server, but the credentials is not created in SQL server.
Moreover, you could refer official doc to know how to Change Server Authentication Mode.
Hope it will help you.
DRIVER='{SQL Server}' works
below code is tested.....
import pyodbc
con = pyodbc.connect('Driver={SQL Server};'
'Server=LAPTOP-PPDS6BPG;'
'Database=training;'
'Trusted_Connection=yes;')
cursor = con.cursor()
sql_query = 'SELECT * FROM Students'
cursor.execute(sql_query)
for row in cursor:
print(row)
This works fine for me better than any other I could find
import pyodbc
import pandas as pd
conn = pyodbc.connect('Driver={SQL Server};'
'Server=10.****;'
'Database=Ma**;'
'UID=sql**;'
'PWD=sql**;')
cursor = conn.cursor()
sql = """\
EXEC [dbo].[GetNewPayment] #Login=?, #PasswordMD5=?, #RevokeTimeFrom=?, #RevokeTimeTo=? Status=?
"""
params = ('a***', 'c2ca***', '2021-05-01','2021-05-02', '3')
cursor.execute(sql, params)
When I am trying to connect python with SQL Server, following error occurred.
"pyodbc.Error: ('08001', '[08001] [Microsoft][ODBC SQL Server
Driver][DBNETLIB]SQL Server does not exist or access denied. (17)
(SQLDriverConnect)')"
Following is the my code.
import pyodbc
connection = pyodbc.connect("Driver={SQL Server}; Server=localhost;
Database=emotionDetection; uid=uname ;pwd=pw;Trusted_Connection=yes")
cursor = connection.cursor()
SQLCommand = ("INSERT INTO emotion" "(happy, sad, angry) "
"VALUES (?,?,?)")
Values = ['smile','cry','blame']
cursor.execute(SQLCommand,Values)
connection.commit()
connection.close()
This is my first attempt to connect Python with sql server. I don't have an idea what would be the driver name, server name, username and password.Do you have any idea of what should be my configuration. Please help me.
CONNECTION FROM WINDOWS TO MS SQL SERVER DATABASE:
Here you have an example I use myself to connect to MS SQL database table with a Python script:
import pyodbc
server = 'ip_database_server'
database = 'database_name'
username = 'user_name'
password = 'user_password'
driver = '{SQL Server}' # Driver you need to connect to the database
port = '1433'
cnn = pyodbc.connect('DRIVER='+driver+';PORT=port;SERVER='+server+';PORT=1443;DATABASE='+database+';UID='+username+
';PWD='+password)
cursor = cnn.cursor()
'User' and 'password' and 'table_name' are attibutes defined by the DB administrator, and he should give them to you. The port to connect to is also defined by the admin. If you are trying to connect from a Windows device to the DB, go to ODBC Data Source Administrator from Windows, and check if you have installed the driver:
Where is the ODBC data source administrator in a Windows machine.
The image is in spanish, but you only have to click on 'Drivers' tab, and check if the driver is there as in the image.
CONNECTION FROM LINUX/UNIX TO MS SQL SERVER DATABASE:
If you are working in Linux/Unix, then you shoud install a ODBC manager like 'FreeTDS' and 'unixODBC'. To configure them, you have some examples in the following links:
Example: Connecting to Microsoft SQL Server from Linux/Unix
Example: Installing and Configuring ODBC
I think you should check out this.
stackoverflow answer about odbc
Also, what sql server do you use?
The library pymssql doesnot require any drivers and works on both Windows as well as Ubunutu.
import pymssql
import pandas as pd
server = 'yourusername'
username = 'yourusername'
password = 'yourpassword'
database = 'yourdatabase'
table_name = 'yourtablename'
conn = pymssql.connect(host=server,user=username,password=password,database=database)
dat = pd.read_sql("select * from table_name,conn)
Try pyodbc with SQLalchemy
try this:
import sqlalchemy
import pyodbc
from sqlalchemy import create_engine
engine = create_engine("mssql+pyodbc://user:password#host:port/databasename?driver=ODBC+Driver+17+for+SQL+Server")
cnxn = engine.connect()
Use your corresponding driver
It works for me
Luck!
Working Examples Work Best For Me:
Need Mac ODBC Drivers?
If you need the mac driver I used homebrew and found the commands here
Detail
I personally learn best by reverse enginerring, with that said I am sharing one of my examples, it may be a bit crude but I'm growing my Python skills.
My script I created allows me to connect my Mac OS to a AWS RDS instance.
The whole script is a copy paste with a little modification for you about your server info, and you are off and running.
just modify these lines to connect.
server = 'yourusername'
username = 'yourusername'
password = 'yourforgottencomplicatedpassword'
database = 'yourdatabase'
Then Run the file: python3 ~/Your/path/pyodbc_mssqldbtest.py and you should be set.
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
# =============================================================================
# Created By : Jeromie Kirchoff
# Created Date: Mon July 31 22:32:00 PDT 2018
# FILENAME: pyodbc_mssqldbtest.py
# =============================================================================
"""The Module Has Been Build for Interaction with MSSQL DBs To Test the con."""
# =============================================================================
# Thanks to this post for headers https://stackoverflow.com/q/12704305/1896134
# Answer to an SO question: https://stackoverflow.com/q/42433408/1896134
# =============================================================================
import pyodbc
def runningwithqueries(query):
"""The Module Has Been Build to {Open, Run & Close} query connection."""
print("\nRunning Query: " + str(query) + "\nResult :\n")
crsr = cnxn.execute(query)
columns = [column[0] for column in crsr.description]
print(columns)
for row in crsr.fetchall():
print(row)
crsr.close()
# =============================================================================
# SET VARIABLES NEEDED FOR SERVER CONNECTION
# =============================================================================
server = 'yourusername'
username = 'yourusername'
password = 'yourforgottencomplicatedpassword'
database = 'yourdatabase'
connStr = (r'DRIVER={ODBC Driver 17 for SQL Server};' +
r"Integrated Security=True;" +
r'SERVER=' + server +
r';UID=' + username +
r';PWD=' + password +
r';DSN=MSSQL-PYTHON' +
r';DATABASE=' + database + ';'
)
print("Your Connection String:\n" + str(connStr) + "\n\n")
# =============================================================================
# CONNECT TO THE DB
# =============================================================================
cnxn = pyodbc.connect(connStr, autocommit=True)
# =============================================================================
# SET QUERIES TO VARIABLES
# =============================================================================
SQLQUERY1 = ("SELECT ##VERSION;")
SQLQUERY2 = ("SELECT * FROM sys.schemas;")
SQLQUERY3 = ("SELECT * FROM INFORMATION_SCHEMA.TABLES;")
SQLQUERY4 = ("SELECT * FROM INFORMATION_SCHEMA.COLUMNS;")
SQLQUERY5 = ("SELECT * FROM INFORMATION_SCHEMA.CHECK_CONSTRAINTS;")
SQLQUERY6 = ("EXEC sp_databases;")
SQLQUERY7 = ("EXEC sp_who2 'active';")
# =============================================================================
# RUN QUERIES
# YOU CAN RUN AS MANY QUERIES AS LONG AS THE CONNECTION IS OPEN TO THE DB
# =============================================================================
runningwithqueries(SQLQUERY1)
runningwithqueries(SQLQUERY2)
runningwithqueries(SQLQUERY3)
runningwithqueries(SQLQUERY4)
runningwithqueries(SQLQUERY5)
runningwithqueries(SQLQUERY6)
runningwithqueries(SQLQUERY7)
# =============================================================================
# CLOSE THE CONNECTION TO THE DB
# =============================================================================
cnxn.close()
import pyodbc
conn = pyodbc.connect('Driver={SQL Server};' 'Server=**SERVER NAME**;' 'Database=**DATABASE NAME**;' 'Trusted_Connection=yes;')
cursor = conn.cursor()
cursor.execute('SELECT * FROM Output3')
This works just check you specify the Respective Driver, Server and the Database names correctively!