I get error using python 3.8.5
( cursor.execute("INSERT INTO dbo.sftpserverlist(FileName,FileSize) VALUES ("+files[0]+","+str(sizes[0])+")")
pyodbc.ProgrammingError: ('42000', '[42000] [Microsoft][ODBC SQL Server Driver][SQL Server]The multi-part identifier "DidosSupply.zip" could not be bound. (4104) (SQLExecDirectW)')
while I am trying call following function and insert into table dbo.sftpserverlist.
def getfile(sftp):
FileNames=[]
FileName = sftp.listdir_attr()
for i in FileName:
FileNames.append(i.filename)
FileSizes=[]
FileSize = sftp.listdir_attr()
for i in FileSize:
FileSizes.append(i.st_size)
return FileNames,FileSizes
-----------------------------------------------------------
cursor.execute("INSERT INTO dbo.sftpserverlist(FileName,FileSize) VALUES ("+files[0]+","+str(sizes[0])+")")
conn.commit()
Parametrise your query, don't inject the values, which is a huge security flaw.
cursor.execute("INSERT INTO dbo.sftpserverlist(FileName,FileSize) VALUES (?,?)",files[0],str(sizes[0]))
conn.commit()
cursor.close()
The reason you were getting the error was because the values you were (insecurely) injecting weren't quoted; thus the value of files[0] (which in this case had the value "DidosSupply.zip") was being interpreted as a column's name; which it can't be as you're within a VALUES clause with no FROM.
Of course, just wrapping quotes around the value isn't the solution, as quotes can be escaped.
I'm not an expert with Python, by any means, but I think this is the old-school way of doing it (INSERT INTO). I recently stumbled upon a super-easy, scalable, and controllable, way of pushing data from Python to SQL Server. Try the sample code and post back if you have additional questions.
import pyodbc
import pandas as pd
engine = "mssql+pyodbc://your_server_name/your_database_name?driver=SQL Server Native Client 11.0?trusted_connection=yes"
... dataframe here...
dataframe.to_sql(x, engine, if_exists='append', index=True)
dataframe is pretty self explanatory.
x = the name yo uwant your table to be in SQL Server.
Related
I am using pyodbc to connect to a local MS SQL Server 2019 instance on Windows. There is a table in a database on that instance called 'Media', which has a column called 'mediaId', which has a bigint datatype. For reference, it is a column that is uniquely constrained, but is neither a primary nor a foreign key. I am trying to update specific records in this column with pyodbc.
For reference, the following code executes successfully in Python using pyodbc.
db = DB() # db contains pyodbc context
cursor = db.cursor
command = "UPDATE Media SET {}={} WHERE id=476506567332605952".format('mediaId', 476506524101914624)
cursor.execute(command)
What does not execute cleanly is the following snippet.
db = DB()
cursor = db.cursor
command = "UPDATE Media SET ?=? WHERE id=476506567332605952"
params = ('mediaId', 476506524101914624)
cursor.execute(command, params)
As far as I am aware, the parameters should be inserted where the question marks are in the same way as the statement above. However, this yields the following error message.
[22003] [Microsoft][ODBC Driver 17 for SQL Server][SQL
Server]Arithmetic overflow error converting expression to data type
nvarchar. (8115) (SQLExecDirectW); [22003] [Microsoft][ODBC Driver 17
for SQL Server][SQL Server]The statement has been terminated. (3621)
No nvarchar types are used in this table, and the integer that is to be set falls under the bigint size limit and under the int limit for a 64-bit system in Python.
I am at a loss as to why I might be getting this error.
The issue is that you are using ? substitution for the column name. The database adapter will quote the value as if it were a data column, so the expression becomes UPDATE Media SET (string)=(integer), and I'm guessing the parser tries to coerce the integer to a string to compare it.
Don't use ? substitution for column and table names.
I'm trying to debug a problem inserting data into an a Microsoft SQL database. The error message I keep getting is:
ProgrammingError: ('42000', "[42000] [Microsoft][ODBC SQL Server
Driver][SQL Server]The incoming Tabular Data Stream (TDS) for the
Remote Procedure Call (RPC) protocol is not correct. Parameter 16
(''): The specified value is not a valid instance of the float data
type. Check the source data for invalid values. An example of an
invalid value is data of a numeric type whose decimal places exceed
the precision value. (8023) (SQLExecDirectW)")
I (believe) I understand the problem but I can't isolate which values are tripping it. I made sure all the columns in the dataframe are exactly the same name and order of the database, but obviously something still isn't jiving. My question is: Where exactly is this error message pointing too when it says 'Parameter 16'. Is is a row? Is it a column? Is it zero-indexing?
I unfortunately can't easily provide data or examples but I can at least provide my sql insert code for more context:
with engine.begin() as conn:
stmt = insert(reflection)
values_list = df.to_dict(orient='records')
conn.execute(stmt, values_list)
Any tips?
I have a python script that makes a call to an API, submits a request, and then is supposed to insert the result into a Sql Server 2012 table. When it goes to execute the insert into SQL, it breaks. I am currently importing json, requests, and pyodbc into the file. Here is the location where it is breaking:
conn = pyodbc.connect('DRIVER={SQL Server};SERVER={localServer};DATABASE={localDB}')
cursor = conn.cursor()
for record in response:
print(json.dumps(record))
cursor.execute("Insert Into Ticket_Info values ?", json.dumps(record))
cursor.commit()
cursor.close()
conn.close()
It is at the cursor.execute() line where the breakage occurs. This is the error I got when I attempted to run this.
pyodbc.ProgrammingError: ('42000', "[42000] [Microsoft][ODBC SQL
Server Driver][SQL Server]Incorrect syntax near '#P1'. (102)
(SQLExecDirectW); [42000] [Microsoft][ODBC SQL Server Driver][SQL
Server]Statement(s) could not be prepared. (8180)"
Any help I could get I would appreciate. I have searched and tried several different methods at this point, the only thing that changes is the error.
The second argument to cursor.execute() must be a sequence of values to interpolate, one for each SQL parameter in your statement.
You gave ODBC a string instead, which is also a sequence, but one that contains (many) more elements (characters) than your query requires.
Use a single-element tuple here:
cursor.execute("Insert Into Ticket_Info values (?)", (json.dumps(record),))
I also put parenthesis around the values section, as per the SQL Server INSERT syntax:
VALUES
Introduces the list or lists of data values to be inserted. There must be one data value for each column in column_list, if specified, or in the table. The value list must be enclosed in parentheses.
Unless Ticket_Info has only one column per row (unlikely, you'd have a primary key column at least), you probably need to specify what column you are inserting your value into:
cursor.execute("Insert Into Ticket_Info (<columnname>) values (?)", (json.dumps(record),))
where you need to replace <columnname> with the actual column name in your table.
I'm a postgres newbie and am having some issues querying a text field in postgresql using Python. What is the correct syntax that will allow me to search the content of column "body" from table "jivemessage" out of database "postgres"?
try:
conn = psycopg2.connect("dbname='postgres' user='postgres' host='localhost' password='<password>'")
except:
print "cannot connect"
i = 'test'
cur = conn.cursor()
cur.execute('SELECT * from jivemessage WHERE body LIKE "%'+i+'%"')
Keep getting the following error:
ProgrammingError: column "%test%" does not exist
Thanks for any help.
You are not quoting the query properly. Don't use string concatenation here, use SQL parameters instead:
cur.execute('SELECT * from jivemessage WHERE body LIKE %s', ("%{}%".format(i),))
Here, the %s placeholder signals to the database driver that the first value of the second argument should be placed there when querying.
This leaves the interpolation up to the database driver, giving the database the opportunity to optimize for the query once, even if you were to reuse the same query.
It also prevents SQL injection attacks better than you could yourself, and most of all, guarantees that the correct quoting rules are followed.
The problem is that I can't create a field named Date (I think because its a type)
Any ideas how to do that?
from pyodbc import connect
# database connect
conn = connect('DRIVER={Microsoft Access Driver (*.mdb)};DBQ=test.mdb')
cursor = conn.cursor()
# sql query execute
query = "create table MyTable(name varchar(30), age integer , Date date)"
cursor.execute(query)
# commit changes
conn.commit()
conn.close()
Error:
Traceback (most recent call last):
File "\Path\to\myscript\test.py", line 9, in <module>
cursor.execute(query)
ProgrammingError: ('42000', '[42000] [Microsoft][ODBC Microsoft Access Driver] Syntax error in field definition. (-3553) (SQLExecDirectW)')
Environment: Windows 7 64bit, Python 2.7 pyodbc-3.0.6.win-amd64-py2.7
DATE is a reserved word in Access (and other software).
Try wrapping the Date column name with square brackets [], or better yet, come up with a different column name.
...
query = "create table MyTable(name varchar(30), age integer , [Date] date)"
...
Surround the name in backticks the ` symbol. But, I would highly suggest changing the name to something else to prevent typos from happening.
You're right, you can't create a column with the same name as type. Some RDBMS's will allow you to do this if you quote it, i.e. "..age integer, "Date" date)" (or as others have said, backticks or square brackets) but then you have to quote it in all your queries as well. Better to avoid that.
Note that this isn't a python problem, it's a problem with the database server (which appears to be MS Access). (And to be accurate, it's not a problem with the database server, that's just how it works. It just has nothing to do with python.)