The problem is that I can't create a field named Date (I think because its a type)
Any ideas how to do that?
from pyodbc import connect
# database connect
conn = connect('DRIVER={Microsoft Access Driver (*.mdb)};DBQ=test.mdb')
cursor = conn.cursor()
# sql query execute
query = "create table MyTable(name varchar(30), age integer , Date date)"
cursor.execute(query)
# commit changes
conn.commit()
conn.close()
Error:
Traceback (most recent call last):
File "\Path\to\myscript\test.py", line 9, in <module>
cursor.execute(query)
ProgrammingError: ('42000', '[42000] [Microsoft][ODBC Microsoft Access Driver] Syntax error in field definition. (-3553) (SQLExecDirectW)')
Environment: Windows 7 64bit, Python 2.7 pyodbc-3.0.6.win-amd64-py2.7
DATE is a reserved word in Access (and other software).
Try wrapping the Date column name with square brackets [], or better yet, come up with a different column name.
...
query = "create table MyTable(name varchar(30), age integer , [Date] date)"
...
Surround the name in backticks the ` symbol. But, I would highly suggest changing the name to something else to prevent typos from happening.
You're right, you can't create a column with the same name as type. Some RDBMS's will allow you to do this if you quote it, i.e. "..age integer, "Date" date)" (or as others have said, backticks or square brackets) but then you have to quote it in all your queries as well. Better to avoid that.
Note that this isn't a python problem, it's a problem with the database server (which appears to be MS Access). (And to be accurate, it's not a problem with the database server, that's just how it works. It just has nothing to do with python.)
Related
I am using pyodbc to connect to a local MS SQL Server 2019 instance on Windows. There is a table in a database on that instance called 'Media', which has a column called 'mediaId', which has a bigint datatype. For reference, it is a column that is uniquely constrained, but is neither a primary nor a foreign key. I am trying to update specific records in this column with pyodbc.
For reference, the following code executes successfully in Python using pyodbc.
db = DB() # db contains pyodbc context
cursor = db.cursor
command = "UPDATE Media SET {}={} WHERE id=476506567332605952".format('mediaId', 476506524101914624)
cursor.execute(command)
What does not execute cleanly is the following snippet.
db = DB()
cursor = db.cursor
command = "UPDATE Media SET ?=? WHERE id=476506567332605952"
params = ('mediaId', 476506524101914624)
cursor.execute(command, params)
As far as I am aware, the parameters should be inserted where the question marks are in the same way as the statement above. However, this yields the following error message.
[22003] [Microsoft][ODBC Driver 17 for SQL Server][SQL
Server]Arithmetic overflow error converting expression to data type
nvarchar. (8115) (SQLExecDirectW); [22003] [Microsoft][ODBC Driver 17
for SQL Server][SQL Server]The statement has been terminated. (3621)
No nvarchar types are used in this table, and the integer that is to be set falls under the bigint size limit and under the int limit for a 64-bit system in Python.
I am at a loss as to why I might be getting this error.
The issue is that you are using ? substitution for the column name. The database adapter will quote the value as if it were a data column, so the expression becomes UPDATE Media SET (string)=(integer), and I'm guessing the parser tries to coerce the integer to a string to compare it.
Don't use ? substitution for column and table names.
I'm using Python + MySQL and want to use parameterized query. I'm stuck. I've encountered an error and can't figure out how to solve it. I've spent a day, checked dozens of articles, used various options (sinle quotes, double quotes, prepared statements) and still no luck.
Requirements: use Parameterized Query
Here is basic demo of the issue:
#!/usr/bin/python3
import mysql.connector as mysql
conn = mysql.connect(host=server, user=username, passwd=password, autocommit=True)
try:
create_database_query = "CREATE DATABASE %s;"
db_name = "BOOKS"
cursor = conn.cursor()
print(f"Creating {db_name} database... ", end='')
cursor.execute(create_database_query, (db_name,))
print("Success")
except mysql.Error as error:
print("Parameterized query failed {}".format(error))
Output:
Creating BOOKS database... Parameterized query failed 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''BOOKS'' at line 1
So it looks like it uses too many quotes (2 single quotes on each side). The code above works fine if I change the following line:
create_database_query = "CREATE DATABASE %s;"
and put backtick around %s
The problem that now it creates a database but with invalid chars - 'BOOKS' (quotes are now part of db name). Duh...
If I use prepared statements then the same issue occurs but slightly different error message:
1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '?' at line 1
Environment:
MacOS Catalina
Python 3.8
PyCharm 2019.3 IDE
MySQL 8.0.19
mysql-connector-python module 8.0.19
What is going on? Any ideas?
Thanks
You can't use query parameters for identifiers (like a database name or table name or column name).
Query parameters can be used only in place of a constant value — a quoted string, quoted date/time, or a numeric value. Not identifiers, expressions, SQL keywords, etc.
To combine a database name with your CREATE DATABASE statement, you have to format it into the string in a way that forms the full statement before it is sent to MySQL.
db_name = "BOOKS"
create_database_query = "CREATE DATABASE %s;" % db_name
cursor.execute(create_database_query)
Because this creates a risk of SQL injection when you format variables into your string, it's up to you to make sure the db_name is safe.
Update: Thanks to #Parfait for the reminder about current best practices of string-formatting.
Prefer:
db_name = "BOOKS"
create_database_query = "CREATE DATABASE {};".format(db_name)
Or F-strings:
db_name = "BOOKS"
create_database_query = f"CREATE DATABASE {db_name};"
(In other words, Python has become Ruby ;-)
cnxn = pyobdc.connect(...)
cursor = cnxn.cursor()
sql = 'create table if not exists tab1 (id integer primary key, entry varchar (200), exit varchar (200))'
cursor.execute(sql)
cnxn.commit()
in the above code, the query "sql" has the keyword "exit" as a column name which throws up an error. I therefore made the change of enclosing the keyword in quotes like
sql = 'create table if not exists tab1 (id integer primary key, entry varchar (200), "exit" varchar (200))'
and even tried
sql = 'create table if not exists tab1 (id integer primary key, entry varchar (200), `exit` varchar (200))'
Which still gives me the error:
ProgrammingError: ('42000', "[42000] [Microsoft][ODBC SQL Serveer Driver][SQLServer]Incorrect syntax near '"'. (102) (SQLExecDirectW)")
or accordingly,
ProgrammingError: ('42000', "[42000] [Microsoft][ODBC SQL Serveer Driver][SQLServer]Incorrect syntax near '`'. (102) (SQLExecDirectW)")
Please help me understand how to overcome this problem without having to change the column name, thanks!
As I said in the comments, firstly SQL Server does not support CREATE TABLE IF NOT EXISTS. See this question for specific answers on that.
Next your second statement. This won't generate the error you describe. In fact, if you do try to run it, you get the errors:
Incorrect syntax near the keyword 'if'.
Incorrect syntax near 'tab1'.
Again, this is because of comment 1. If you try the latter statement, with the backticks (`) you get the same identical errors, you don't get the error about the backtick. Though, as I also mentioned, the backticks are not a valid delimit identifier in T-SQL. Either use T-SQL own, brackets ([]), or ANSI SQLs, double quotes (").
Finally, to fix the problem, just don't use a reserved keyword for your object name. Perhaps something like this:
IF OBJECT_ID(N'dbo.tab1') IS NULL CREATE TABLE tab1 (id integer PRIMARY KEY, UserEntry varchar (200), UserExit varchar (200));
I get error using python 3.8.5
( cursor.execute("INSERT INTO dbo.sftpserverlist(FileName,FileSize) VALUES ("+files[0]+","+str(sizes[0])+")")
pyodbc.ProgrammingError: ('42000', '[42000] [Microsoft][ODBC SQL Server Driver][SQL Server]The multi-part identifier "DidosSupply.zip" could not be bound. (4104) (SQLExecDirectW)')
while I am trying call following function and insert into table dbo.sftpserverlist.
def getfile(sftp):
FileNames=[]
FileName = sftp.listdir_attr()
for i in FileName:
FileNames.append(i.filename)
FileSizes=[]
FileSize = sftp.listdir_attr()
for i in FileSize:
FileSizes.append(i.st_size)
return FileNames,FileSizes
-----------------------------------------------------------
cursor.execute("INSERT INTO dbo.sftpserverlist(FileName,FileSize) VALUES ("+files[0]+","+str(sizes[0])+")")
conn.commit()
Parametrise your query, don't inject the values, which is a huge security flaw.
cursor.execute("INSERT INTO dbo.sftpserverlist(FileName,FileSize) VALUES (?,?)",files[0],str(sizes[0]))
conn.commit()
cursor.close()
The reason you were getting the error was because the values you were (insecurely) injecting weren't quoted; thus the value of files[0] (which in this case had the value "DidosSupply.zip") was being interpreted as a column's name; which it can't be as you're within a VALUES clause with no FROM.
Of course, just wrapping quotes around the value isn't the solution, as quotes can be escaped.
I'm not an expert with Python, by any means, but I think this is the old-school way of doing it (INSERT INTO). I recently stumbled upon a super-easy, scalable, and controllable, way of pushing data from Python to SQL Server. Try the sample code and post back if you have additional questions.
import pyodbc
import pandas as pd
engine = "mssql+pyodbc://your_server_name/your_database_name?driver=SQL Server Native Client 11.0?trusted_connection=yes"
... dataframe here...
dataframe.to_sql(x, engine, if_exists='append', index=True)
dataframe is pretty self explanatory.
x = the name yo uwant your table to be in SQL Server.
I have
cursor.execute("select RM_ID FROM Sales.dbo.MARKETING where VERSION = 'SomeVersion'")
which gives me the traceback error:
pyodbc.ProgrammingError: ('42S02', "[42S02] [Microsoft][ODBC SQL Server Driver][SQL Server]Invalid object name 'Sales.dbo.MARKETING'. (208) (SQLExecDirectW)")
I have several other lines of code:
cursor.execute("select RUNDATEEST FROM Sales.dbo.MARKETING where VERSION = 'SomeVersion'")
that are exactly the same except for the column name that give me no error. I'm not sure what my mistake is.
Of note:
I have already checked the table for the column name to make sure it exists.
I have noticed that this column is the key for this table. Perhaps a different syntax is required for keys?
When I execute the query in SQL Server, it runs just fine.
Try to surround schema and table names with brackets:
[Sales].[dbo].[MARKETING]
Perhaps you need to surround column names in the same way.