How do I edit Azure SQL Database using pyodbc/odbc? - python

I wrote a Python program that web scraped a website and added the results to a Microsoft Access database. I now want to run the script again, with it adding the data to an Azure SQL database. I keep getting this error.
A network-related or instance-specific error has occurred while establishing a connection to SQL Server. Server is not found or not accessible. Check if instance name is correct and if SQL Server is configured to allow remote connections.
I have tried t edit the settings of the database to no avail. Could someone tell me what settings to apply to the database? I also tried to see if there was a way to run the Python script inside azure to try to avoid the problem. Is this possible?
cnxn = pyodbc.connect(r'Driver={ODBC Driver 18 for SQL Server};Server=tcp:servername.database.windows.net,1433;Database=sizedb3;Uid={your_user_name};Pwd={your_password_here};Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30;Authentication=ActiveDirectoryPassword')
I tried this driver. I have downloaded the driver from Microsoft's website. This driver is a connection string in the ODBC section of the Azure SQL database in the Azure portal.

I tried running the below python code to connect to Azure SQL DB with Azure AD authentication.
Code:-
import pyodbc
conn = pyodbc.connect('Driver={ODBC Driver 17 for SQL Server};'
'Server=tcp:siliconserver.database.windows.net,1433;'
'Database=silicondb;'
'Uid=xxxser#sid24desaioutlook.onmicrosoft.com;'
'Pwd=xxxxxxxxxx#123;'
'authentication=ActiveDirectoryPassword')
cursor = conn.cursor()
cursor.execute('SELECT * FROM StudentReviews')
for i in cursor:
print(i)
cursor.close()
conn.close()
Output:-
Make sure you have allowed your Client IP in your Azure SQL server Networking tab like below:-
I tried to remove one syntax/spelling from my Azure SQL connection string and got the same error code as yours like below:-
You can validate your connection string server spelling and syntax from your Azure SQL server DB > Connection String > like below:-
Also, Make sure you have added your client IP and allowed it in your Azure SQL Server like below:-

Related

connection to microsoft azure sql database works in local enviornment but not in production

I am trying to open a web service app that will allow me to connect to the database on azure with python code, Ive tried using sqlalchemy and pyodbc and i am successfully able to connect to the database on my machine, in the local host, i can perform all necessary actions i want to there. but i want to be able to set this code up to be able to hit specific routes in an ajax call that can perform certain actions on my database, like flipping a users active flag to false. however the problem is that when i upload the python code to azure, using this guide here (https://learn.microsoft.com/en-us/azure/app-service/quickstart-python?tabs=bash&pivots=python-framework-flask) it just returns a 500 server error, i cant find anything in the trace as to why its not working, I thought that it might just be that my local machine is whitelisted in the IP address's to the database, but even still if i add that app services IP address to the allowed IP's it still returns a server error. here is the setup of the code:
from flask import Flask
from sqlalchemy import create_engine
import pyodbc
app = Flask(__name__)
#app.route('/')
def connection():
Driver = "{ODBC Driver 13 for SQL Server}"
Server = "server string"
Port = 1433
Database = "dbname"
Uid = "user"
Pwd = "pass"
try:
cnxn = pyodbc.connect(f'DRIVER={Driver};SERVER={Server};DATABASE={Database};Uid={Uid};Pwd={Pwd};Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30;')
cursor = cnxn.cursor()
cursor.execute(f"SELECT * FROM (Database Table) where id = 999999999;")
for row in cursor:
print('row = %r' % (row,))
return "Connection to database successful"
I ommitted certain details but the syntax should remain intact, Again, in my local machine I can connect to the database and return data. but once it compiles the code in azure, it no longer works.
excuse the try statement. I was trying to catch the error and send it to the client in hopes of gathering more information from the 500 error, but it didnt work.
**Edit: its worth mentioning that if i remove the actual connection string and anything to do with connecting to the database, the code will return "connection to database successful" this leads me to believe that it isnt fully making a connection to the database at all. and thats why its erroring out, however the question remains, why can i connect in my local enviornment but not in azure?
have you heard about remote debugging in azure. This is the best tool to trace out errors using your visual studio code or VS where you web or app is initially working well. You can use multiple methods to debug line by line or cluster by cluster. Kindly check it out here
https://learn.microsoft.com/en-us/visualstudio/debugger/remote-debugging?view=vs-2019
So the short answer to this is the App service was missing the driver {ODBC Driver 13 for SQL Server} it wasn't erroring out locally because I have that driver installed locally, but this driver isn't available in the app. To get this to work I would have to install the App to a Virtual Machine to be able to download the driver appropriately.
fortunately however Azure web apps built on a linux base already come with {ODBC Driver 17 for SQL Server} So flipping the driver from 13 to 17 allowed the app to successfully connect to my Database.

SQL Connect error in Python(Windows): severity 9:\nAdaptive Server connection failed

Not able to connect to Azure DB. I get the following error while connecting via Python.
I'm able to connect to my usual SQL environment
import pandas as pd
import pymssql
connPDW = pymssql.connect(host=r'dwprd01.database.windows.net', user=r'internal\admaaron',password='',database='')
connPDW.autocommit(True)
cursor = connPDW.cursor()
conn.autocommit(True)
cursor = conn.cursor()
sql = """
select Top (10) * from TableName
"""
cursor.execute(sql);
Run without errors.
Just according to your code, there is an obvious issue of connecting Azure SQL Database by pymssql package in Python which use the incorrect user format and lack of the values of password and database parameters.
Please follow the offical document Step 3: Proof of concept connecting to SQL using pymssql carefully to change your code correctly.
If you have an instance of Azure SQL Database with the connection string of ODBC, such as Driver={ODBC Driver 13 for SQL Server};Server=tcp:<your hostname>.database.windows.net,1433;Database=<your database name>;Uid=<username>#<host>;Pwd=<your_password>;Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30; show in the Connection strings tab of your SQL Database on Azure portal.
Then, your code should be like below
hostname = '<your hostname>'
server = f"{hostname}.database.windows.net"
username = '<your username>'
user = f"{username}#{hostname}"
password = '<your password>'
database = '<your database name>'
import pymssql
conn = pymssql.connect(server=server, user=user, password=password, database=database)
Meanwhile, just additional note for the version of Azure SQL Database and MS SQL Server are 2008+ like the latest Azure SQL Database, you should use the ODBC Driver connection string which be started with DRIVER={ODBC Driver 17 for SQL Server};, not 13 show in the connection string of Azure portal if using ODBC driver for Python with pyodbc, please refer to the offical document Step 3: Proof of concept connecting to SQL using pyodbc.

How to connect to a SQL server linked server in Python

Normally, when trying to connect to a SQL sever DB in Python, I use the pyodbc package like this:
import pyodbc
conn = pyodbc.connect("Driver={SQL Server};"
"Server=<server-ip>;"
"Database=<DB-name>;"
"UID=<user-name>;"
"PWD=<password>;"
"Trusted_Connection=yes;"
)
However, I don't know how to connect to a linked server in Python. If my linked server is called linked-server and has a DB called linked-DB for example; I have tried the same connection string as above, and changing the database name like this: "Database=<linked-server>.<linked-DB>;", since that's how I query the linked server DB in SSMS. But this doesn't work in Python.
Thank you very much for your help.

Connect to Database on local host with sqlite3 in python

I have a database that I am running on my local machine which I can access through Microsoft SQL Server Manager Studio. I connect to this server "JIMS-LAPTOP\SQLEXPRESS" and then I can run queries through the manager. However I need to be able to connect to this database and work with it through python.
When I try to connect using sqlite3 like
conn = sqlite3.connect("JIMS-LAPTOP\SQLEXPRESS")
I get an unable to open database file error
I tried accessing the temporary file directly like this
conn = sqlite3.connect("C:\Users\Jim Notaro\AppData\Local\Temp\~vs13A7.sql")
c = conn.cursor()
c.execute("SELECT name FROM sqlite_master WHERE type = \"table\"")
print c.fetchall()
Which allows me to access a database but it is completely empty (No tables are displayed)
I also tried connecting like this
conn = sqlite3.connect("SQL SERVER (SQLEXPRESS)")
Which is what the name is in the sql server configuration manager but that also returns a blank database.
I'm not sure how I am suppose to be connecting to the database using python
You can't use sqlite3 to connect to SQL server, only to Sqlite databases.
You need to use a driver that can talk to MS SQL, like pyodbc.

SQL server bulkinsert errors

When I run this query to bulkinsert a file on a shared drive to SQL server 2008 with username and password (not Windows authentication), I get these errors. DBA, system admins and network guys are all denying these errors are related to their teams and I am lost... Can anyone please help me to identify where the issue is? When I run bulkinsert with database username and password, what authentication does SQL server use to open the file?
Run this on MS Management Studio
BULK INSERT DatabaseName.dbo.TableName
FROM '\\shared_server\parent\child\file_name.txt'
WITH(FIRE_TRIGGERS, DATAFILETYPE='char', FIELDTERMINATOR='\t',ROWTERMINATOR='\n', FIRSTROW=2);
and I get
Cannot bulk load because the file "\\shared_server\parent\child\file_name.txt" could not be opened. Operating system error code 5(Access is denied.).
Run this on python
import pyodbc
database = 'DatabaseName'
username = 'username'
password = 'password'
server = 'server_name'
failover = 'failover_server_name'
cnxn_string = 'DRIVER={SQL Server Native Client 10.0};SERVER=%s;FAILOVER_PARTNER=%s;DATABASE=%s;UID=%s;PWD=%s;CHARSET=UTF8' % (server, failover, database, username, password)
cnxn = pyodbc.connect(cnxn_string)
cursor = cnxn.cursor()
query = r"""
BULK INSERT Estimates.dbo.FundamentalsIS
FROM '\\shared_server\parent\child\file_name.txt'
WITH(FIRE_TRIGGERS, DATAFILETYPE='char', FIELDTERMINATOR='\t',ROWTERMINATOR='\n', FIRSTROW=2);
"""
cursor.execute(query)
cursor.commit()"
and I get
ProgrammingError: ('42000', '[42000] [Microsoft][SQL Server Native Client 10.0][SQL Server]Cannot bulk load because the file "\\shared_server\parent\child\file_name.txt" could not be opened. Operating system error code 1326(Logon failure: unknown user name or bad password.). (4861) (SQLExecDirectW)')
Could the MS SQL server 2008 possibly be on a different security group (or have different settings) than the shared drives, where the file is located?
Because the bulk insert operation is run on the MS Management studio server side, it might not have access to the file, the 'access denied' leads me to believe DB server cannot get to shared file drive, and possibly does not have permission to access it. Likewise, even if using python to execute the BULK INSERT statement, the DB server still needs to have access to where ever the file is located.
I had a similar issue in the past, because the DB server could not get to the shared file, located elsewhere. My workaround was to use local computer to read in the file and run the insert queries using python. It sounds like the local environment has access to both and can be used as the central communication hub. You might have to do something similar to
https://stackoverflow.com/a/6482610/3761363
https://stackoverflow.com/a/11219626/3761363

Categories