We Keep Coding

Python Script in SQL Server / sp_execute_external_script / pyodbc Error

I need to build a model using a Python script in SQL Server. But it needs to have multiple input data sets. And I gather you can only have one. I read that a way to do it is to use pyodbc inside sp_execute_external_script.
But it won't connect. The exact same code does work in Spyder however so it's not a general configuration issue (I can also connect from Tableau etc.). As it's the local instance I tried using a dot (.) for the server name but that didn't work so I also tried just putting the server and instance name in full (hence the two different connection strings below).
The code:
EXEC sp_execute_external_script
#language =N'Python',
#script=N'
import pyodbc
sql_conn = pyodbc.connect("DRIVER={ODBC Driver 17 for SQL Server};SERVER=XXX\YYY;DATABASE=MY_PYTHON;trusted_connection=Yes")
sql_conn = pyodbc.connect("Driver={ODBC Driver 17 for SQL Server};Server=.;Database=MY_PYTHON;Trusted_Connection=Yes;")
'
The error message:
Msg 39004, Level 16, State 20, Line 52
A 'Python' script error occurred during execution of 'sp_execute_external_script' with HRESULT 0x80004004.
Msg 39019, Level 16, State 2, Line 52
An external script error occurred:
Error in execution. Check the output for more information.
Traceback (most recent call last):
File "", line 5, in
File "C:\ProgramData\GTMDEV\Temp-PY\Appcontainer1\A0F0D72C-4B70-46BD-8307-D51C36E9A8F1\sqlindb_0.py", line 39, in transform
sql_conn = pyodbc.connect("Driver={ODBC Driver 17 for SQL Server};Server=.;Database=MY_PYTHON;Trusted_Connection=Yes;")
Msg 39019, Level 16, State 2, Line 52
An external script error occurred:
pyodbc.OperationalError: ('08001', '[08001] [Microsoft][ODBC Driver 17 for SQL Server]Named Pipes Provider: Could not open a connection to SQL Server [2]. (2) (SQLDriverConnect); [08001] [Microsoft][ODBC Driver 17 for SQL Server]Login timeout expired (0); [08001] [Microsoft][ODBC Driver 17 for SQL Server]A network-related or instance-specific error has occurred while establishing a connection to SQL Server. Server is not found or not accessible. Check if instance name is correct and if SQL Server is configured to allow remote connections. For more information see SQL Server Books Online. (2)')
SqlSatelliteCall error: Error in execution. Check the output for more information.
STDOUT message(s) from external script:
SqlSatelliteCall function failed. Please see the console output for more information.
Traceback (most recent call last):
File "C:\Program Files\Microsoft SQL Server\MSSQL15.GTMDEV\PYTHON_SERVICES\lib\site-packages\revoscalepy\computecontext\RxInSqlServer.py", line 605, in rx_sql_satellite_call
rx_native_call("SqlSatelliteCall", params)
File "C:\Program Files\Microsoft SQL Server\MSSQL15.GTMDEV\PYTHON_SERVICES\lib\site-packages\revoscalepy\RxSerializable.py", line 375, in rx_native_call
ret = px_call(functionname, params)
RuntimeError: revoscalepy function failed.
Internet research has led me to create a login for SQLRUserGroupYYY in SQL Server and add it to "Allow log on locally" in Local Security Policy. But that has not changed anything. And I can't find anything else that works.
In case it is relevant the server is on my PC and it's name is MYPCNAME\MYNAMEDINSTANCE. (And the name is correct and it is running (suggested in the error message) because the connection works in Spyder, Tableau etc.).
Any ideas what else I can try?
Thanks.
For reference, this is one of the articles that suggested the method:
https://medium.com/wandering-in-advanced-analytics/passing-multiple-t-sql-queries-to-sp-execute-external-script-and-loop-back-requests-d153ee582235
The example given is for R but the principle should be the same for Python. I did try it in R and got the same error message.
This is another post with the same sort of issue (that had a link for the SQLRUserGroup login suggestion):
pyodbc.OperationalError when used inside sp_execute_external_script

Problem opening connection to SQL Server DB using pyodbc and machine learning services

I have Anaconda installed on one of our servers, and some code which successfully opens a connection to SQL (2016) on another server:
import pyodbc
conn_string = 'DRIVER={SQL Server Native Client 11.0};SERVER=wpic-smir;Trusted_Connection=yes'
conn = pyodbc.connect(conn_string)
cursor=conn.cursor()
qstring = 'select UserID from Diler_BW.Beckwith.V_Thermovals'
cursor.execute(qstring)
row=cursor.fetchone()
print row[0]
del cursor
del conn
Now, on the same server as Anaconda, we have SQL Server (2017) installed, and Machine Learning Service for Python is also installed. I'm trying to do run essentially the same code above, but from within SQL:
exec sp_execute_external_script
#LANGUAGE = N'Python',
#script = N'
import pyodbc
conn_string = ''DRIVER={SQL Server Native Client 11.0};SERVER=wpic-smir;Trusted_Connection=yes''
conn = pyodbc.connect(conn_string)
cursor=conn.cursor()
qstring = ''select UserID from Diler_BW.Beckwith.V_Thermovals''
cursor.execute(qstring)
row=cursor.fetchone()
print row[0]
del cursor
del conn
'
The code runs successfully from standalone Anaconda, but, from a SQL query connected to the 2017 server, it does not:
Msg 39004, Level 16, State 20, Line 0
A 'Python' script error occurred during execution of 'sp_execute_external_script' with HRESULT 0x80004004.
Msg 39019, Level 16, State 2, Line 0
An external script error occurred:
Error in execution. Check the output for more information.
Traceback (most recent call last):
File "<string>", line 5, in <module>
File "C:\ProgramData\MSSQLSERVER\Temp-PY\Appcontainer1\41EF254D-1B14-4D9D-99AF-8AD356A84BDC\sqlindb_0.py", line 39, in transform
conn = pyodbc.connect(conn_string)
pyodbc.OperationalError: ('08001', '[08001] [Microsoft][SQL Server Native Client 11.0]Named Pipes Provider: Could not open a connection to SQL Server [65]. (65) (SQLDriverConnect); [08001] [Microsoft][SQL Server Native Client 11.0]Login timeout expired (0); [08001] [Microsoft][SQL Server Native Client 11.0]A network-related or instance-specific error has occurred while establishing a connection to SQL Server. Server is not found or not accessible. Check if instance name is correct and if SQL Server is configured to allow remote connections. For more information see SQL Server Books Online. (65)')
Msg 39019, Level 16, State 2, Line 0
An external script error occurred:
SqlSatelliteCall error: Error in execution. Check the output for more information.
STDOUT message(s) from external script:
Express Edition will continue to be enforced.
SqlSatelliteCall function failed. Please see the console output for more information.
Traceback (most recent call last):
File "C:\Program Files\Microsoft SQL Server\MSSQL15.MSSQLSERVER\PYTHON_SERVICES\lib\site-packages\revoscalepy\computecontext\RxInSqlServer.py", line 605, in rx_sql_satellite_call
rx_native_call("SqlSatelliteCall", params)
File "C:\Program Files\Microsoft SQL Server\MSSQL15.MSSQLSERVER\PYTHON_SERVICES\lib\site-packages\revoscalepy\RxSerializable.py", line 375, in rx_native_call
ret = px_call(functionname, params)
RuntimeError: revoscalepy function failed.
The account I'm using for development is in the sysadmin role on both servers. We use Windows authentication only on both servers. I assume that something significant is different about the instance of python that SQL/MLS uses than the standalone installation.
Thank you.

Can no longer access files from shared folder or SQL server using Python after updating password on Windows 10?

I have a script that I run in Jupyter Notebook every week that requires connecting to MSSQL and accessing files from a shared folder. I have never had problems doing this before, however, I recently had to update my Windows password. Since then, I've been having these issues.
When trying to connect to the MSSQL server I get the error:
Error: ('HY000', '[HY000] [Microsoft][ODBC Driver 17 for SQL Server]SQL
Server Network Interfaces: The logon attempt failed\r\n (-2146893044)
(SQLDriverConnect); [HY000] [Microsoft][ODBC Driver 17 for SQL Server]Cannot
generate SSPI context (-2146893044); [HY000] [Microsoft][ODBC Driver 17 for
SQL Server]SQL Server Network Interfaces: The logon attempt failed\r\n
(-2146893044); [HY000] [Microsoft][ODBC Driver 17 for SQL Server]Cannot
generate SSPI context (-2146893044)')
When trying to access a file on the shared folder I get:
com_error: (-2147352567, 'Exception occurred.', (0, 'Microsoft Excel',
"Sorry, we couldn't find {file} Is it possible it was moved, renamed or
deleted?", 'xlmain11.chm', 0, -2146827284), None)
or
OSError: [Errno 22] Invalid argument: {file}
However, if I manually navigate to these folders/files I don't have any issues and if I sign into the MSSQL server using Windows Authentication I don't have any issues either. Any suggestions on how to solve this?
EDIT:
This is the command I use to connect:
con = pyodbc.connect('Trusted_Connection=yes', driver = '{ODBC Driver 17 for SQL Server}',server = '{server}', database = '{database}')
cursor = con.cursor()
and I just use
pd.read_excel({file_path})
to get the file.

Python ADO + ODBC function

I am writing a small module to help a transfer from M$-Access to SQLite (database needs to be portable), but I'm struggling in interpreting the error message that follows from this code (and of course to get it to work).
import pyodbc
import win32com.client
def ado(db, sqlstring='select * from table', user='admin', password=''):
conn = win32com.client.Dispatch(r'ADODB.Connection')
DSN = ('PROVIDER = Microsoft.Jet.OLEDB.4.0;DATA SOURCE = ' + db + ';')
conn.Open(DSN)
rs = win32com.client.Dispatch(r'ADODB.Recordset')
rs.Open(strsql, conn, 1, 3)
data = rs.GetRows()
conn.Close()
return data
def odbc(db, sqlstring='select * from table', user= 'admin', password=''):
"""Create function for connecting to Access databases."""
odbc_conn_str = 'DRIVER={Microsoft Access Driver (*.mdb)};DBQ=%s;UID=%s;PWD=%s' % (db, user, password)
conn = pyodbc.connect(odbc_conn_str)
cur = conn.cursor()
cur.execute(strsql)
data = list(cur)
conn.close()
return data
if __name__ == '__main__': # Unit test
db = r'C:\pyodbc_access2007_sample.accdb'
sql="select * from Customer Orders" ## tables: 'Customer Orders', 'Physical Stoks','Prodplans'
data1 = ado(db,sql)
data2 = odbc(db,sql)
From the ado function I get the error:
Traceback (most recent call last):
File "C:/pyodbc_access2007_example.py", line 27, in <module>
data1 = ado(db,sql)
File "C:/pyodbc_access2007_example.py", line 7, in ado
conn.Open(DSN)
File "<COMObject ADODB.Connection>", line 3, in Open
File "C:\Python27\lib\site-packages\win32com\client\dynamic.py", line 282, in _ApplyTypes_
result = self._oleobj_.InvokeTypes(*(dispid, LCID, wFlags, retType, argTypes) + args)
com_error: (-2147352567, 'Exception occurred.', (0, u'Microsoft JET Database Engine', u"Unrecognized database format 'C:\\pyodbc_access2007_sample.accdb'.", None, 5003049, -2147467259), None)
and from the odbc function I get the error:
Traceback (most recent call last):
File "C:/pyodbc_access2007_example.py", line 28, in <module>
data2 = odbc(db,sql)
File "C:/pyodbc_access2007_example.py", line 17, in odbc
conn = pyodbc.connect(odbc_conn_str)
Error: ('HY000', "[HY000] [Microsoft][ODBC Microsoft Access Driver] Cannot open database '(unknown)'. It may not be a database that your application recognizes, or the file may be corrupt. (-1028) (SQLDriverConnect); [01000] [Microsoft][ODBC Microsoft Access Driver]General Warning Unable to open registry key 'Temporary (volatile) Jet DSN for process 0x18c0 Thread 0xe70 DBC 0x379fe4 Jet'. (1); [01000] [Microsoft][ODBC Microsoft Access Driver]General Warning Unable to open registry key 'Temporary (volatile) Jet DSN for process 0x18c0 Thread 0xe70 DBC 0x379fe4 Jet'. (1); [01000] [Microsoft][ODBC Microsoft Access Driver]General Warning Unable to open registry key 'Temporary (volatile) Jet DSN for process 0x18c0 Thread 0xe70 DBC 0x379fe4 Jet'. (1); [01000] [Microsoft][ODBC Microsoft Access Driver]General Warning Unable to open registry key 'Temporary (volatile) Jet DSN for process 0x18c0 Thread 0xe70 DBC 0x379fe4 Jet'. (1); [HY000] [Microsoft][ODBC Microsoft Access Driver] Cannot open database '(unknown)'. It may not be a database that your application recognizes, or the file may be corrupt. (-1028)")
Any good idea's on how to read this?

Your connection string only recognizes mdb access files. There is a connection string that will do mdb and accdb files in pyodbc.

select * from Customer Orders
----------------------^
Having spaces in table names like that, does that really work in Access?
For MSSQL Server I'd quote it like in
[Customer Orders]

For accessing accdb files, you need to install AccessDatabaseEngine.
Also, be aware if you need 32 or 64 bits!

Error in opening an Access database in python

I am a new to python programming and i want to write a python program to read and write data to and from the database.
The connection code is as follows:
DNS='catalog'
DRV = '{Microsoft Access Driver (*.mdb)}'
conn = pyodbc.connect('DRIVER=%s;DSN=%s;' % (DRV,DNS))
catalog is the DSN name.
I am am getting the following error:
Traceback (most recent call last):
File "C:\Python27\exampes\xxx.py", line 8, in <module>
conn = pyodbc.connect('DRIVER=%s;DSN=%s;' % (DRV,DNS))
Error: ('01000', "[01000] [Microsoft][ODBC Microsoft Access Driver]General Warning Unable to open registry key 'Temporary (volatile) Jet DSN for process 0x12b4 Thread 0x1544 DBC 0x567ea4 Jet'. (1) (SQLDriverConnect);
[01000] [Microsoft][ODBC Microsoft Access Driver]General Warning Unable to open registry key 'Temporary (volatile) Jet DSN for process 0x12b4 Thread 0x1544 DBC 0x567ea4 Jet'. (1)"
Can anyone please help me..?

The account under which you are running is not an administrator. It needs registry access as per the message (also described here)
Registry access is needed to find the ODBC driver for MS Access.

try to uncheck Attributes: Read-only box in file properties.