Please note that I have been attempting to create a new database using SQLAlchemy, but have gotten many errors where I am attempting to work with laravel and vagrant. Thankfully I have been able to create a new database using pyodbc, but ultimately I would like to do everything through SQLAlchemy.
This is the code that works for creating a new database at this point using pyodbc:
for tagList in myTagList:
conn = pyodbc.connect("driver={SQL Server Native Client 11.0};server=localhost,2433; database=master; UID=myUser;PWD=myUser2", autocommit=True)
conn.execute('CREATE DATABASE ' + tagList["job"].jobNumber)
How would I change this to work with SQLAlchemy? Presently I have looked at many links like How to create a new database using SQLAlchemy? and sqlalchemy,creating an sqlite database if it doesn't exist but and https://sqlalchemy-utils.readthedocs.io/en/latest/database_helpers.html it appears to work with postgresql but not SQL Server:
Add support for creating databases (see How to create a new database using SQLAlchemy?):
Projects>pip install sqlalchemy-utils
pip install sqlalchemy-utils
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won`t be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Collecting sqlalchemy-utils
Downloading https://files.pythonhosted.org/packages/bf/7e/3211ad9b3983b216d1b1863fd7734f80bacd1a62a5de8ff6844fb5ed1498/SQLAlchemy-Utils-0.35.0.tar.gz (129kB)
Requirement already satisfied: six in c:\users\myUser\appdata\roaming\python\python27\site-packages (from sqlalchemy-utils) (1.13.0)
Requirement already satisfied: SQLAlchemy>=1.0 in c:\python27\lib\site-packages (from sqlalchemy-utils) (1.3.11)
Installing collected packages: sqlalchemy-utils
Running setup.py install for sqlalchemy-utils: started
Running setup.py install for sqlalchemy-utils: finished with status 'done'
Successfully installed sqlalchemy-utils-0.35.0
--------------------------------------------------------------------------------
#Save historical data into new MyDB000N database
for tagList in myTagList:
fsDatabaseEngine = create_engine('mssql+pyodbc://myUser:myUser2#localhost:2433/' + tagList["job"].jobNumber + '?driver=SQL+Server+Native+Client+11.0?trusted_connection=yes')
if not database_exists(fsDatabaseEngine.url):
create_database(fsDatabaseEngine.url)
print(database_exists(fsDatabaseEngine.url))
-----
On this line if not database_exists(fsDatabaseEngine.url): receiving:
self.__connect(first_connect_check=True)
File "C:\Python27\lib\site-packages\sqlalchemy\pool\base.py", line 639, in __connect
connection = pool._invoke_creator(self)
File "C:\Python27\lib\site-packages\sqlalchemy\engine\strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "C:\Python27\lib\site-packages\sqlalchemy\engine\default.py", line 482, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlalchemy.exc.InterfaceError: (pyodbc.InterfaceError) ('28000', u'[28000] [Microsoft][SQL Server Native Client 11.0][SQL Server]Login failed for user \'myUser\'. (18456) (SQLDriverConnect); [28000] [Microsoft][SQL Server Native Client 11.0][SQL Server]Cannot open database "MyDB00001" requested by the login. The login failed. (4060); [28000] [Microsoft][SQL Server Native Client 11.0][SQL Server]Login failed for user \'myUser\'. (18456); [28000] [Microsoft][SQL Server Native Client 11.0][SQL Server]Cannot open database "MyDB00001" requested by the login. The login failed. (4060)')
(Background on this error at: http://sqlalche.me/e/rvf5)
-----
C:\Python27\lib\site-packages\sqlalchemy\connectors\pyodbc.py:79: SAWarning: No driver name specified; this is expected by PyODBC when using DSN-less connections
"No driver name specified; "
-----
#Save historical data into new MyDB000N database
for tagList in myTagList:
# fsDatabaseEngine = create_engine('mssql://mssql#localhost:2433/' + tagList["job"].jobNumber + '?driver=SQL+Server+Native+Client+11.0?trusted_connection=yes')
# if not database_exists(fsDatabaseEngine.url):
create_database('mssql://mssql#localhost:2433/' + tagList["job"].jobNumber)
#print(database_exists(fsDatabaseEngine.url))
File "C:\Python27\lib\site-packages\sqlalchemy\engine\default.py", line 482, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlalchemy.exc.InterfaceError: (pyodbc.InterfaceError) ('IM002', u'[IM002] [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified (0) (SQLDriverConnect)')
(Background on this error at: http://sqlalche.me/e/rvf5)
PS Projects>
-----
#Save historical data into new MyDB000N database
for tagList in myTagList:
# fsDatabaseEngine = create_engine('mssql://mssql#localhost:2433/' + tagList["job"].jobNumber + '?driver=SQL+Server+Native+Client+11.0?trusted_connection=yes')
# if not database_exists(fsDatabaseEngine.url):
create_database('mssql://myUser:myUser2#localhost:2433/' + tagList["job"].jobNumber+ '?driver=SQL+Server+Native+Client+11.0?trusted_connection=yes')
#print(database_exists(fsDatabaseEngine.url))
File "C:\Python27\lib\site-packages\sqlalchemy\engine\base.py", line 1246, in _execute_context
cursor, statement, parameters, context
File "C:\Python27\lib\site-packages\sqlalchemy\engine\default.py", line 581, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (pyodbc.ProgrammingError) ('42000', u'[42000] [Microsoft][SQL Server Native Client 11.0][SQL Server]CREATE DATABASE statement not allowed within multi-statement transaction. (226) (SQLExecDirectW)')
[SQL: CREATE DATABASE [MyDB00001]]
(Background on this error at: http://sqlalche.me/e/f405)
PS Projects>
-----
#Save historical data into new MyDB000N database
for tagList in myTagList:
with create_engine(
'mssql://mssql',
isolation_level="AUTOCOMMIT"
).connect() as connection:
connection.execute('CREATE DATABASE ' + tagList["job"].jobNumber)
# if not database_exists(fsDatabaseEngine.url):
#create_database('mssql://myUser:myUser2#localhost:2433/' + tagList["job"].jobNumber)
#print(database_exists(fsDatabaseEngine.url))
File "C:\Python27\lib\site-packages\sqlalchemy\engine\default.py", line 482, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlalchemy.exc.InterfaceError: (pyodbc.InterfaceError) ('IM002', u'[IM002] [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified (0) (SQLDriverConnect)')
(Background on this error at: http://sqlalche.me/e/rvf5)
PS Projects>
-----
#Save historical data into new MyDB000N database
for tagList in myTagList:
session.execute('CREATE DATABASE ' + tagList["job"].jobNumber)
File "C:\Python27\lib\site-packages\sqlalchemy\engine\default.py", line 581, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (pyodbc.ProgrammingError) ('42000', u'[42000] [Microsoft][SQL Server Native Client 11.0][SQL Server]CREATE DATABASE statement not allowed within multi-statement transaction. (226) (SQLExecDirectW)')
[SQL: CREATE DATABASE MyDB00001]
(Background on this error at: http://sqlalche.me/e/f405)
PS Projects>
-----
#Save historical data into new MyDB000N database
for tagList in myTagList:
conn = pyodbc.connect("driver={SQL Server};server=ubuntu-bionic; database=master; trusted_connection=true",
autocommit=True)
Exception has occurred: OperationalError
('08001', u'[08001] [Microsoft][ODBC SQL Server Driver][DBNETLIB]SQL Server does not exist or access denied. (17) (SQLDriverConnect); [08001] [Microsoft][ODBC SQL Server Driver][DBNETLIB]ConnectionOpen (Connect()). (53)')
File "Projects\interesting_data.py", line 81, in <module>
autocommit=True)
-----
#Save historical data into new MyDB000N database
for tagList in myTagList:
conn = pyodbc.connect("driver={SQL Server Native Client 10.0};server=localhost,2433; database=master; trusted_connection=true", autocommit=True)
Exception has occurred: InterfaceError
('IM002', u'[IM002] [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified (0) (SQLDriverConnect)')
File "Projects\interesting_data.py", line 80, in <module>
conn = pyodbc.connect("driver={SQL Server Native Client 10.0};server=localhost,2433; database=master; trusted_connection=true", autocommit=True)
-----
#Save historical data into new MyDB000N database
for tagList in myTagList:
conn = pyodbc.connect("driver={SQL Server Native Client 11.0};server=localhost,2433; database=master; trusted_connection=true", autocommit=True)
Exception has occurred: OperationalError
('08001', u"[08001] [Microsoft][SQL Server Native Client 11.0]Invalid value specified for connection string attribute 'trusted_connection' (0) (SQLDriverConnect)")
File "Projects\interesting_data.py", line 80, in <module>
conn = pyodbc.connect("driver={SQL Server Native Client 11.0};server=localhost,2433; database=master; trusted_connection=true", autocommit=True)
-----
#Save historical data into new MyDB000N database
for tagList in myTagList:
conn = pyodbc.connect("driver={SQL Server Native Client 11.0};server=localhost,2433; database=master; trusted_connection=yes", autocommit=True)
Exception has occurred: Error
('HY000', u'[HY000] [Microsoft][SQL Server Native Client 11.0]SQL Server Network Interfaces: No credentials are available in the security package\r\n (-2146893042) (SQLDriverConnect); [HY000] [Microsoft][SQL Server Native Client 11.0]Cannot generate SSPI context (-2146893042); [HY000] [Microsoft][SQL Server Native Client 11.0]SQL Server Network Interfaces: No credentials are available in the security package\r\n (-2146893042); [HY000] [Microsoft][SQL Server Native Client 11.0]Cannot generate SSPI context (-2146893042)')
File "Projects\interesting_data.py", line 80, in <module>
conn = pyodbc.connect("driver={SQL Server Native Client 11.0};server=localhost,2433; database=master; trusted_connection=yes", autocommit=True)
import urllib
#Save historical data into new MyDB000N database
for tagList in rdTagList:
# if not database_exists(fsDatabaseEngine.url):
params = urllib.parse.quote_plus("driver={SQL Server Native Client 11.0};server=localhost,2433; database=master; UID=phpUser;PWD=phpUser2")
tagListEngine = sqlalchemy.create_engine("mssql+pyodbc:///?odbc_connect=%s" % params)
tagListEngine.connect()
Which gives:
Exception has occurred: AttributeError
'module' object has no attribute 'parse'
File "C:\Projects\interesting_data.py", line 82, in <module>
params = urllib.parse.quote_plus("driver={SQL Server Native Client 11.0};server=localhost,2433; database=master; UID=myUser;PWD=myUser2")
What do you suggest? TIA.
UPDATE:
Not sure how I missed it before but ultimately based on #snakecharmerb's suggestion it is now working without requiring urllib:
#Save historical data into new MyDB000N database
for tagList in rdTagList:
fsDatabaseEngine = create_engine('mssql+pyodbc://myUser:myUser2#localhost:2433/' + tagList["job"].jobNumber + '?driver=SQL+Server+Native+Client+11.0?trusted_connection=yes')
if not database_exists(fsDatabaseEngine.url):
create_database(fsDatabaseEngine.url)
print(database_exists(fsDatabaseEngine.url))
Thank you!
UPDATE 2:
To zero in on the solution, in my research I had found explanations using postgres://postgres ..., but ultimately in my case I needed to change mssql://myUser to mssql+pyodbc://myUser...
Based on #shakecharmerb's comment, this is the solution:
#Save historical data into new MyDB000N database
for tagList in rdTagList:
fsDatabaseEngine = create_engine('mssql+pyodbc://myUser:myUser2#localhost:2433/' + tagList["job"].jobNumber + '?driver=SQL+Server+Native+Client+11.0?trusted_connection=yes')
if not database_exists(fsDatabaseEngine.url):
create_database(fsDatabaseEngine.url)
print(database_exists(fsDatabaseEngine.url))
Related
I have DataGrip program, I tried to connect with it to azure sql database and it connect successfully.
I try to connect to azure sql database using sqlalchemy but it fail, here is the code:
from sqlalchemy import create_engine
try:
connection = create_engine("mysql+pymysql://<username>:<password>#<dbserver>.database.windows.net:1433/<dbname>").connect()
with connection:
with connection.begin():
print("OK")
except Exception as err:
print(err)
then the result was:
(pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query ([WinError 10054] An existing connection was forcibly closed by the remote host)')
(Background on this error at: https://sqlalche.me/e/14/e3q8)
I tried to use pyodbc instead of pymysql but the result was:
(pyodbc.InterfaceError) ('IM002', '[IM002] [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified (0) (SQLDriverConnect)')
(Background on this error at: https://sqlalche.me/e/14/rvf5)
the final goal is deploy my fastapi project into azure as webapp then use azure sql.
how can I make this working ?
I need to connect to a SQL server database from Python. In this case, I am a non-administrator Windows user.
This is the code that I used:
import pyodbc
conn = pyodbc.connect('Driver={SQL Server};'
'Server=XXXXXXXX;'
'Database=XXX;'
'Trusted_Connection=yes;'
'port = 1433;')
But this error message appear:
Traceback (most recent call last):
File "<ipython-input-1-c018c40360dc>", line 3, in <module>
conn = pyodbc.connect('Driver={SQL Server};'
ProgrammingError: ('42000', '[42000] [Microsoft][ODBC SQL Server Driver][SQL Server]Cannot open database "XXX" requested by login. Login failed. (4060) (SQLDriverConnect); [42000 ] [Microsoft][ODBC SQL Server Driver]Invalid connection string attribute (0);[42000] [Microsoft][ODBC SQL Server Driver][SQL Server]Cannot open database "XXX" requested by login . Login failed. (4060); [42000] [Microsoft][ODBC SQL Server Driver]Invalid connection string attribute (0)')
This is the Conection Section in my SQL Server Manangement Studio:
The image description:
Server Type: Database Engine
Server Name: XXXXX
Authentication: Windows Authentication
Username: SERVERNAME / username
Password: (empty)
What am i making wrong?
I'm trying to write to MSSQL using SQL Alchemy but can't get the connection to work. I've got
engine = create_engine('mssql+pyodbc://<Username>#localhost:5432/<DBName>?driver=SQL+Server+Native+Client+11.0')
df.to_sql('<TableName>', con=engine, if_exists='append')
This gives:
OperationalError: (pyodbc.OperationalError) ('08001', '[08001] [Microsoft][SQL Server Native Client 11.0]TCP Provider: No connection could be made because the target machine actively refused it.\r\n (10061) (SQLDriverConnect); [08001] [Microsoft][SQL Server Native Client 11.0]Login timeout expired (0); [08001] [Microsoft][SQL Server Native Client 11.0]A network-related or instance-specific error has occurred while establishing a connection to SQL Server. Server is not found or not accessible. Check if instance name is correct and if SQL Server is configured to allow remote connections. For more information see SQL Server Books Online. (10061)')
(Background on this error at: http://sqlalche.me/e/13/e3q8)
I've looked at the error documentation and tried several different things with the connection string but nothing seems to work. Reading from the same database using pyodbc works fine:
con = pyodbc.connect('Driver={SQL Server};'
'Server=<Server>;'
'Database=<DBName>;'
'Trusted_Connection=yes;')
df = pd.read_sql_query('select * from <DBName>.dbo.<TableName>', con)
I have Anaconda installed on one of our servers, and some code which successfully opens a connection to SQL (2016) on another server:
import pyodbc
conn_string = 'DRIVER={SQL Server Native Client 11.0};SERVER=wpic-smir;Trusted_Connection=yes'
conn = pyodbc.connect(conn_string)
cursor=conn.cursor()
qstring = 'select UserID from Diler_BW.Beckwith.V_Thermovals'
cursor.execute(qstring)
row=cursor.fetchone()
print row[0]
del cursor
del conn
Now, on the same server as Anaconda, we have SQL Server (2017) installed, and Machine Learning Service for Python is also installed. I'm trying to do run essentially the same code above, but from within SQL:
exec sp_execute_external_script
#LANGUAGE = N'Python',
#script = N'
import pyodbc
conn_string = ''DRIVER={SQL Server Native Client 11.0};SERVER=wpic-smir;Trusted_Connection=yes''
conn = pyodbc.connect(conn_string)
cursor=conn.cursor()
qstring = ''select UserID from Diler_BW.Beckwith.V_Thermovals''
cursor.execute(qstring)
row=cursor.fetchone()
print row[0]
del cursor
del conn
'
The code runs successfully from standalone Anaconda, but, from a SQL query connected to the 2017 server, it does not:
Msg 39004, Level 16, State 20, Line 0
A 'Python' script error occurred during execution of 'sp_execute_external_script' with HRESULT 0x80004004.
Msg 39019, Level 16, State 2, Line 0
An external script error occurred:
Error in execution. Check the output for more information.
Traceback (most recent call last):
File "<string>", line 5, in <module>
File "C:\ProgramData\MSSQLSERVER\Temp-PY\Appcontainer1\41EF254D-1B14-4D9D-99AF-8AD356A84BDC\sqlindb_0.py", line 39, in transform
conn = pyodbc.connect(conn_string)
pyodbc.OperationalError: ('08001', '[08001] [Microsoft][SQL Server Native Client 11.0]Named Pipes Provider: Could not open a connection to SQL Server [65]. (65) (SQLDriverConnect); [08001] [Microsoft][SQL Server Native Client 11.0]Login timeout expired (0); [08001] [Microsoft][SQL Server Native Client 11.0]A network-related or instance-specific error has occurred while establishing a connection to SQL Server. Server is not found or not accessible. Check if instance name is correct and if SQL Server is configured to allow remote connections. For more information see SQL Server Books Online. (65)')
Msg 39019, Level 16, State 2, Line 0
An external script error occurred:
SqlSatelliteCall error: Error in execution. Check the output for more information.
STDOUT message(s) from external script:
Express Edition will continue to be enforced.
SqlSatelliteCall function failed. Please see the console output for more information.
Traceback (most recent call last):
File "C:\Program Files\Microsoft SQL Server\MSSQL15.MSSQLSERVER\PYTHON_SERVICES\lib\site-packages\revoscalepy\computecontext\RxInSqlServer.py", line 605, in rx_sql_satellite_call
rx_native_call("SqlSatelliteCall", params)
File "C:\Program Files\Microsoft SQL Server\MSSQL15.MSSQLSERVER\PYTHON_SERVICES\lib\site-packages\revoscalepy\RxSerializable.py", line 375, in rx_native_call
ret = px_call(functionname, params)
RuntimeError: revoscalepy function failed.
The account I'm using for development is in the sysadmin role on both servers. We use Windows authentication only on both servers. I assume that something significant is different about the instance of python that SQL/MLS uses than the standalone installation.
Thank you.
I'm trying to connect to SQL Server and run a query using the following code in Python:
import pyodbc
cnxn = pyodbc.connect("Driver={SQL Server Native Client 11.0};"
"Server = server_name;"
"Database = database_name;"
"UID = user;"
"PWD = password;")
cursor = cnxn.cursor()
cursor.execute('SELECT TOP 10 [column] FROM [table]')
for row in cursor:
print('row = %r' % (row,))
I'm getting the following error:
Traceback (most recent call last):
File "filename", line 3, in <module>
cnxn = pyodbc.connect("Driver={SQL Server Native Client 11.0};"
pyodbc.Error: ('28000', "[28000] [Microsoft][SQL Server Native Client 11.0][SQL Server]Login failed for user 'username'. (18456) (SQLDriverConnect)")
("filename" and "username" inserted above as placeholders)
This is the same error, regardless of what I change the SQL Server username and password to in the code, and the user in the error is my windows login username.
I've also tried replacing UID and PWD with:
"Trusted_connection=yes"
... to no avail. I get the exact same error back.
I've tried several solutions for similar posts on Stackoverflow and elsewhere but no luck. Ideas to the problem or an alternative means of connecting to the database would be appreciated.
Thanks so much