I am working with mySQL in Python with the PyMySQL module handling the connection. dbCursor is the cursor object.
I have the following statement:
statement = f"INSERT INTO Machine_status (timestamp, num_of_char, STATUS, emergency_status) VALUES ({timestamp},{num_of_char},{status},{emergency_status})"
dbCursor.execute(statement)
However, whenever I try to execute this statement, the program hangs. This is perplexing because this statement similar to it:
statement = f"INSERT INTO Speed (timestamp, meters_per_second) VALUES ({timestamp}, {meters})"
dbCursor.execute(statement)
does not give any issue. What is wrong with the first statement? I'm truly at a loss as to what it could be. I checked and made sure that the variable names match, I checked the structure in the database to make sure the variables matched what the columns could accept, made sure there were no extra parentheses and that syntax was correct but it continues to hang when trying to execute the statement. The frustrating part is that trying the run the statement directly in the database (using phpMyAdmin) gives no issues. What's going on here?
EDIT1: Added more relevant code.
EDIT2: I've already tried reformatting it to using placeholders instead (i.e. VALUES (%d, %d, %s, %d)). It doesn't resolve the issue.
EDIT3: CREATE TABLE statement
"CREATE TABLE Machine_status (id AUTO_INCREMENT PRIMARY KEY, timestamp int, num_of_char int, status varchar(255), emergency_status int)"
Complete insert string
payload = item[0].split("{")[1].split("}")[0].split(" ") #returns a list containing everything between brackets of statement
timestamp = int(payload[0])
num_of_char = int(payload[1])
status = payload[2]
emergency_status = int(payload[3])
statement = "INSERT INTO Machine_STATUS (timestamp, num_of_char, STATUS, emergency_status) VALUES (%d, %d, %s, %d)"
try:
dbCursor.execute(statement, (timestamp, num_of_char, status, emergency_status))
print("Complete.")
except:
print("something went wrong")
EDIT4: I figured out what the issue was. pymysql was giving me issues with trying to extract the error code, so I switched to mysql.connector. It finally gave me an error of "table not found". Apparently the table name I was using in the code didn't match exactly what I was using in the database, and therefore it was causing the table to not be found. I changed it, and now the issue is gone.
As Klaus already wrote don't use stirng insertion use place holders.
Like
statement = "INSERT INTO Machine_status (timestamp, num_of_char, STATUS, emergency_status) VALUES (%s,%s,%s,%s)"
dbCursor.execute(statement,(timestamp,num_of_char,status,emergency_status))
Related
I want to do the face recognition for attendance program using python in which the program would detect a face, save the information of the user like ID and Name into people table in MySQL database, and then the program will recognize the face and insert the information of the saved user previously into another table, the attendance table. However, I have encountered an error "TypeError: cannot unpack non-iterable int object" on the coding provided below:
db = database.open()
cursor_person = db.cursor()
cursor_att = db.cursor()
cursor_person.execute("SELECT pID, name_reg FROM people")
for row in cursor_person.fetchone():
pID, name_reg = row
cursor_att.execute("UPDATE attendance(ppID, name_att) SET ('{}', '{}') WHERE" .format (pID, name_reg))
db.commit()
The part where the error occurred is as shown below:
pID, name_reg = row <--- TypeError: cannot unpack non-iterable int object
Does anyone know how to solve this? I hope the information that I have provided is helpful. Thank you in advance.
I believe you would like to use fetchall
Note that your update sql phrase is broken.
for row in cursor_person.fetchall():
cursor_att.execute("UPDATE attendance SET ('{}', '{}') WHERE" .format (row[0], row[1]))
The way to debug this is to print(row) and see what it is.
But I'm guessing your real problem is that those two cursors aren't isolated since they share the same session. The cursor_att.execute() line is probably causing cursor_person.fetchone() to return the number of updated rows, an int 1.
Lastly, you shouldn't use .format to set the UPDATE clause values. Otherwise you're at risk of SQL injection attacks, or just accidental bugs on your own part when values contain characters like '. There's an example on how to properly pass in parameters like so:
cursor.execute("UPDATE attendance SET (%s, %s)", (pID, name_reg))
And the mysql-connector library will automatically escape the strings for you.
I work with the Python mysql.connector for the first time and I am not able to create a working insert statement.
This is the table:
'CREATE TABLE IF NOT EXISTS products (id INT AUTO_INCREMENT PRIMARY KEY, title VARCHAR(255));'
I am trying to insert a variable as title while the id should be auto incremented. I have tried multiple solutions but it simply won't work.
def insert_product(title: str):
insert_product_query = 'INSERT INTO products (title) VALUES (%s);'
cursor.execute(insert_product_query, (title,))
This runs without any error, but the insert is not working. It does nothing. I tried multiple versions of this, with '?' instead of '%s' and without a tuple but it won't work.
Another solution I tried is this:
def insert_product(title: str):
insert_product_query = f'INSERT INTO products (title) VALUES (\'{title}\')'
print(insert_product_query)
cursor.execute(insert_product_query)
I printed the insert statement and when I copy paste it directly into the database it works perfectly, so I don't have any idea why it is not working out of the python code as it is not producing any errors.
I found many similar problems but none of the solution worked for me.
I hope someone can help me as I might overlook something obvious.
Thanks in advance!
Python's connector disables autocommit by default (as a reasonable library would do!). You need to explicitly commit after you perform a DML statement:
con.commit() # Assuming con is the name of the connection variable
i'm using sqlite to organize data fetched with a scraper script, having trouble executing an 'insert into' command.
I'm making a scraper for an electronics website as a newbie Python enthusiast.
I already had a working script that would scrape all the pages up until when i decided to modify the code to create a new column with prices and name that column using today's date.
Now for some reason the SQL command that inserts the data to the table refuses to execute with the new column i add.
Tried adding the new column to the SQL command with ? method and .format() method with no success.
Tried all kinds for ' locations around the ?s and the {}s.
This is the Code:
class Product:
def __init__(self, prodId, title, price=None):
self.prodId = prodId
self.title = title
self.price = price
self.currDatePriceCloumnName = date + 'Price'
def insertToTable(self):
self.addColumn()
conn = sqlite3.connect(databaseName)
c = conn.cursor()
c.execute("insert into {} (?,?,?) values (?,?,?)".format(table),('Product_ID','Title',str(self.currDatePriceCloumnName),str(self.prodId),str(self.title),str(self.price)))
conn.commit()
conn.close()
def addColumn(self):
conn = sqlite3.connect(databaseName)
c = conn.cursor()
try:
c.execute("alter table {} add column '?'".format(table),(str(self.currDatePriceCloumnName),))
conn.commit()
conn.close()
except:
pass
i expected the c.execute in the insertToTable to insert the data to the table, but what i get is this error:
File "/home/sergio/Desktop/test/scraper.py", line 67, in insertToTable
c.execute("insert into {} (?,?,?) values (?,?,?)".format(table),('Product_ID','Title',str(self.currDatePriceCloumnName),str(self.prodId),str(self.title),str(self.price)))
sqlite3.OperationalError: near "?": syntax error
The weird thing is that the the column is created but not filled.
When i used the .format() method, the error had the desired column name instead of the ?, this tells me that the issue might be with the fact im using self.currDatePriceCloumnName but i'm stuck from here.
Please help..
Thanks in advance! =]
You have a typo in:
c.execute("insert into {} (?,?,?) values (?,?,?)".format(table),('Product_ID','Title',str(self.currDatePriceCloumnName),str(self.prodId),str(self.title),str(self.price)))
At:
values (?,?,?)".format(table),
The end of .format(table) is all you have inserted into the string. The extra ) is causing the .format() to end throwing a syntax error because it's not expecting a ,. You're not passing any other values.
That being said don't use string formatting with SQL statements in the future (as a general side note) as it is bad practice for security purposes:
https://www.w3schools.com/sql/sql_injection.asp
I am trying to write a simple Python script that gets data from an API, stores it in a MySQL database, and performs some calculations on that data. I try fetch all data from a table where I just inserted some, but that query keeps returning None.
Part that doesn't work:
import MySQLdb
db = MySQLdb.connect("localhost", "stijn", "password", "GW2")
curs = db.cursor()
curs.execute("select gw2_id, naam from PrijzenMats")
for record in curs.fetchall():
curs2 = db.cursor()
curs2.execute("insert into MaterialPrijzenLogs(mat,prijs,tijd) values(%s, %s, %s)", (record[1], prijs, tijd))
db.commit()
curs2.execute("select prijs from MaterialPrijzenLogs")
top10 = len(curs2.fetchall())/10
print(str(len(curs2.fetchall())))
That last print keeps giving 0, even when I populate the table before running the script.
Full code
I solved the problem. Apparently when you call fetchall() it doesn't just get the data from the cursor like a normal getter in Java would do, but it also deletes the data from the cursor. In my code I called fetchall() first to initialize a variable, and after that I tried to print the length of curs2.fetchall(), which had become 0 at that point. This can be easily solved by adding something like myList = curs2.fetchall() directly after curs2.execute("select prijs from MaterialPrijzenLogs") and using the myList variable in the rest of the code instead of curs2.fetchall(). I did not include the declaration of that top10 variable in the code example in my original question because I thought it had nothing to do with the problem. I edited the question so future readers can easily understand the problem.
I am receiving an error when trying to write data to a database table when using a variable for the table name that I do not get when using a static name. For some reason on the line where I insert, if I insert an integer as the column values the code runs and the table is filled, however, if I try to use a string I get a SQL syntax error
cursor = db.cursor()
cursor.execute('DROP TABLE IF EXISTS %s' %data[1])
sql ="""CREATE TABLE %s (IP TEXT, AVAILIBILITY INT)""" %data[1]
cursor.execute(sql)
for key in data[0]:
cur_ip = key.split(".")[3]
cursor.execute("""INSERT INTO %s VALUES (%s,%s)""" %(data[1],key,data[0][key]))
db.commit()
the problem is where I have %(data[1], key, data[0][key]) any ideas?
It's a little hard to analyse your problem when you don't post the actual error, and since we have to guess what your data actually is. But some general points as advise:
Using a dynamic table name is often not way DB-systems want to be used. Try thinking if the problem could be used by using a static table name and adding an additional key column to your table. Into that field you can put what you did now as a dynamic table name. This way the DB might be able to better optimize your queries, and your queries are less likely to get errors (no need to create extra tables on the fly for once, which is not a cheap thing to do. Also you would not have a need for dynamic DROP TABLE queries, which could be a security risk.
So my advice to solve your problem would be to actually work around it by trying to get rid of dynamic table names altogether.
Another problem you have is that you are using python string formatting and not parameters to the query itself. That is a security problem in itself (SQL-Injections), but also is the problem of your syntax error. When you use numbers, your expression evaluates to
INSERT INTO table_name VALUES (100, 200)
Which is valid SQL. But with strings you get
INSERT INTO table_name VALUES (Some Text, some more text)
which is not valid (since you have no quotes ' around the strings.
To get rid of your syntax problem and of the sql-injection-problem, don't add the values to the string, pass them as a list to execute():
cursor.execute("INSERT INTO table_name VALUES (%s,%s)", (key, data[0][key]))
If you must have a dynamic table name, put that in your query string first (e.g. with % formatting), and give the actual values for your query as parameters as above (since I cannot imagine that execute will accept the table name as a parameter).
To put it in some simple sample code. Right now you are trying to do it like this:
# don't do this, this won't even work!
table_name = 'some_table'
user_name = 'Peter Smith'
user_age = 47
query = "INSERT INTO %s VALUES (%s, %s)" % (table_name, user_name, user_age)
cursor.execute(query)
That creates query
INSERT INTO some_table VALUES (Peter Smith, 100)
Which cannot work, because of the unquoted string. So you needed to do:
# DON'T DO THIS, it's bad!
query = "INSERT INTO %s VALUES ('%s', %s)" % (table_name, user_name, user_age)
That's not a good idea, because you need to know where to put quotes and where not (which you will mess up at some point). Even worse, imagine a user named named Connor O'Neal. You would get a syntax error:
INSERT INTO some_table VALUES ('Connor O'Neal', 100)
(This is also the way sql-injections are used to crush your system / steal your data). So you would also need to take care of escaping the values that are strings. Getting more complicated.
Leave those problems to python and mysql, by passing the date (not the table name) as arguments to execute!
table_name = 'some_table'
user_name = 'Peter Smith'
user_age = 47
query = "INSERT INTO " + table_name + " VALUES (%s, %s)"
cursor.execute(query, (user_name, user_age))
This way you can even pass datetime objects directly. There are other ways to put the data than using %s, take a look at this examples http://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursor-execute.html (that is python3 used there, I don't know which you use - but except of the print statements it should work with python2 as well, I think).