This question already has answers here:
Does Python support MySQL prepared statements?
(7 answers)
How to use variables in SQL statement in Python?
(5 answers)
Closed 10 months ago.
I am new to python and i am facing problem while fetching data from mysql db while i am passing parameters in mysql query i think my mysql syntax is incorrect .
Here is the Error displayed on Screen Like this.
Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator at webmaster#localhost to inform them of the time this error occurred, and the actions you performed just before this error.
More information about this error may be available in the server error log.
Apache/2.4.6 (Ubuntu) Server at localhost Port 80
Here Is My Code For Select query in that I want to fetch data from get parameter of Url.
#!/usr/bin/python2.7
import cgi;
import cgitb; cgitb.enable();
import time, calendar;
print "Content-type: text/html\n\n";
print "<h1>Hello Python</h1>";
#!/usr/bin/python
import MySQLdb
# Create instance of FieldStorage
form = cgi.FieldStorage()
# Get data from fields
first_name = form.getvalue('first_name')
last_name = form.getvalue('last_name')
# Open database connection
db = MySQLdb.connect("localhost","root","123456789","testdrive" )
# prepare a cursor object using cursor() method
cursor = db.cursor()
# Prepare SQL query to INSERT a record into the database
sqlstmt = "SELECT * FROM EMPLOYEE WHERE FIRST_NAME = %(first_name)s AND LAST_NAME = %(last_name)s"
try:
# Execute the SQL command
cursor.execute(sqlstmt, {'first_name': first_name, 'last_name': last_name})
# Fetch all the rows in a list of lists.
results = cursor.fetchall()
for row in results:
fname = row[0]
lname = row[1]
age = row[2]
sex = row[3]
income = row[4]
# Now print fetched result
print "fname=%s,lname=%s,age=%d,sex=%s,income=%d" % \
(fname, lname, age, sex, income )
except:
print "Error: unable to fecth data"
# disconnect from server
db.close()
You have an error in this line
sql = "SELECT * FROM EMPLOYEE WHERE FIRST_NAME = '".first_name."' AND LAST_NAME = '".last_name."'"
This isn't PHP, meaning you can't concat strings and variables like this. I assume you wan't to make prepared statements. In that case, you should read following reply.
Relevant part of your code would look like this:
cursor.execute("SELECT * FROM EMPLOYEE WHERE FIRST_NAME = %s AND LAST_NAME = %s", [first_name, last_name])
First off, you should try abstracting all that into a single function you can call outside of CGI, but that's a whole other exercise now. Anyway, if you had done that you can get the stacktrace much easier to see what you did wrong, however, I can see you have a syntax error in the code you helpfully included
sql = "SELECT * FROM EMPLOYEE WHERE FIRST_NAME = '".first_name."' AND LAST_NAME = '".last_name."'"
Python string concatenation uses the + operator, not . like it is in PHP.
Second, this code is not secure. See http://xkcd.com/327/
To fix this, the cursor.execute method provides a second argument to fill out the tokens, this is what you should do
sqlstmt = "SELECT * FROM EMPLOYEE WHERE FIRST_NAME = %(first_name)s AND LAST_NAME = %(last_name)s"
try:
cursor.execute(sqlstmt, {'first_name': first_name, 'last_name': last_name})
...
I think we don't need fetchall() here, which maybe legacy from sqlite code. Just do something like:
for row in cursor:
x = row[0]
y = row[1]
...
Try this code
import mysql.connector
from mysql.connector import Error
try:
mySQLconnection = mysql.connector.connect(host='localhost',
database='python_database',
user='username',
password='passw0rd')
sql_select_Query = "select * from tablename"
cursor = mySQLconnection .cursor()
cursor.execute(sql_select_Query)
records = cursor.fetchall()
for row in records:
print("Sr No = ", row[0], )
print("Name = ", row[1])
print("Age = ", row[2])
print("Gender = ", row[3], "\n")
cursor.close()
except Error as e :
print ("Error connecting MySQL", e)
finally:
#closing database connection.
if(mySQLconnection .is_connected()):
connection.close()
print("MySQL connection is closed Now"
Ref
Related
I am inserting JSON data into a MySQL database
I am parsing the JSON and then inserting it into a MySQL db using the python connector
Through trial, I can see the error is associated with this piece of code
for steps in result['routes'][0]['legs'][0]['steps']:
query = ('SELECT leg_no FROM leg_data WHERE travel_mode = %s AND Orig_lat = %s AND Orig_lng = %s AND Dest_lat = %s AND Dest_lng = %s AND time_stamp = %s')
if steps['travel_mode'] == "pub_tran":
travel_mode = steps['travel_mode']
Orig_lat = steps['var_1']['dep']['lat']
Orig_lng = steps['var_1']['dep']['lng']
Dest_lat = steps['var_1']['arr']['lat']
Dest_lng = steps['var_1']['arr']['lng']
time_stamp = leg['_sent_time_stamp']
if steps['travel_mode'] =="a_pied":
query = ('SELECT leg_no FROM leg_data WHERE travel_mode = %s AND Orig_lat = %s AND Orig_lng = %s AND Dest_lat = %s AND Dest_lng = %s AND time_stamp = %s')
travel_mode = steps['travel_mode']
Orig_lat = steps['var_2']['lat']
Orig_lng = steps['var_2']['lng']
Dest_lat = steps['var_2']['lat']
Dest_lng = steps['var_2']['lng']
time_stamp = leg['_sent_time_stamp']
cursor.execute(query,(travel_mode, Orig_lat, Orig_lng, Dest_lat, Dest_lng, time_stamp))
leg_no = cursor.fetchone()[0]
print(leg_no)
I have inserted higher level details and am now searching the database to associate this lower level information with its parent. The only way to find this unique value is to search via the origin and destination coordinates with the time_stamp. I believe the logic is sound and by printing the leg_no immediately after this section, I can see values which appear at first inspection to be correct
However, when added to the rest of the code, it causes subsequent sections where more data is inserted using the cursor to fail with this error -
raise errors.InternalError("Unread result found.")
mysql.connector.errors.InternalError: Unread result found.
The issue seems similar to MySQL Unread Result with Python
Is the query too complex and needs splitting or is there another issue?
If the query is indeed too complex, can anyone advise how best to split this?
EDIT As per #Gord's help, Ive tried to dump any unread results
cursor.execute(query,(leg_travel_mode, leg_Orig_lat, leg_Orig_lng, leg_Dest_lat, leg_Dest_lng))
leg_no = cursor.fetchone()[0]
try:
cursor.fetchall()
except mysql.connector.errors.InterfaceError as ie:
if ie.msg == 'No result set to fetch from.':
pass
else:
raise
cursor.execute(query,(leg_travel_mode, leg_Orig_lat, leg_Orig_lng, leg_Dest_lat, leg_Dest_lng, time_stamp))
But, I still get
raise errors.InternalError("Unread result found.")
mysql.connector.errors.InternalError: Unread result found.
[Finished in 3.3s with exit code 1]
scratches head
EDIT 2 - when I print the ie.msg, I get -
No result set to fetch from
All that was required was for buffered to be set to true!
cursor = cnx.cursor(buffered=True)
The reason is that without a buffered cursor, the results are "lazily" loaded, meaning that "fetchone" actually only fetches one row from the full result set of the query. When you will use the same cursor again, it will complain that you still have n-1 results (where n is the result set amount) waiting to be fetched. However, when you use a buffered cursor the connector fetches ALL rows behind the scenes and you just take one from the connector so the mysql db won't complain.
I was able to recreate your issue. MySQL Connector/Python apparently doesn't like it if you retrieve multiple rows and don't fetch them all before closing the cursor or using it to retrieve some other stuff. For example
import mysql.connector
cnxn = mysql.connector.connect(
host='127.0.0.1',
user='root',
password='whatever',
database='mydb')
crsr = cnxn.cursor()
crsr.execute("DROP TABLE IF EXISTS pytest")
crsr.execute("""
CREATE TABLE pytest (
id INT(11) NOT NULL AUTO_INCREMENT,
firstname VARCHAR(20),
PRIMARY KEY (id)
)
""")
crsr.execute("INSERT INTO pytest (firstname) VALUES ('Gord')")
crsr.execute("INSERT INTO pytest (firstname) VALUES ('Anne')")
cnxn.commit()
crsr.execute("SELECT firstname FROM pytest")
fname = crsr.fetchone()[0]
print(fname)
crsr.execute("SELECT firstname FROM pytest") # InternalError: Unread result found.
If you only expect (or care about) one row then you can put a LIMIT on your query
crsr.execute("SELECT firstname FROM pytest LIMIT 0, 1")
fname = crsr.fetchone()[0]
print(fname)
crsr.execute("SELECT firstname FROM pytest") # OK now
or you can use fetchall() to get rid of any unread results after you have finished working with the rows you retrieved.
crsr.execute("SELECT firstname FROM pytest")
fname = crsr.fetchone()[0]
print(fname)
try:
crsr.fetchall() # fetch (and discard) remaining rows
except mysql.connector.errors.InterfaceError as ie:
if ie.msg == 'No result set to fetch from.':
# no problem, we were just at the end of the result set
pass
else:
raise
crsr.execute("SELECT firstname FROM pytest") # OK now
cursor.reset() is really what you want.
fetchall() is not good because you may end up moving unnecessary data from the database to your client.
The problem is about the buffer, maybe you disconnected from the previous MySQL connection and now it cannot perform the next statement. There are two ways to give the buffer to the cursor. First, only to the particular cursor using the following command:
import mysql.connector
cnx = mysql.connector.connect()
# Only this particular cursor will buffer results
cursor = cnx.cursor(buffered=True)
Alternatively, you could enable buffer for any cursor you use:
import mysql.connector
# All cursors created from cnx2 will be buffered by default
cnx2 = mysql.connector.connect(buffered=True)
cursor = cnx.cursor()
In case you disconnected from MySQL, the latter works for you.
Enjoy coding
If you want to get only one result from a request, and want after to reuse the same connexion for other requests, limit your sql select request to 1 using "limit 1" at the end of your request.
ex "Select field from table where x=1 limit 1;"
This method is faster using "buffered=True"
Set the consume_results argument on the connect() method to True.
cnx = mysql.connector.connect(
host="localhost",
user="user",
password="password",
database="database",
consume_results=True
)
Now instead of throwing an exception, it basically does fetchall().
Unfortunately this still makes it slow, if you have a lot of unread rows.
There is also a possibility that your connection to MySQL Workbench is disconnected. Establish the connection again. This solved the problem for me.
cursor.reset()
and then create tables and load entries
Would setting the cursor within the for loop, executing it, and then closing it again in the loop help?
Like:
for steps in result['routes'][0]['legs'][0]['steps']:
cursor = cnx.cursor()
....
leg_no = cursor.fetchone()[0]
cursor.close()
print(leg_no)
I am trying to get data from an excel spreadsheet into a MySQL database using Python. I can do it fine if add my records to a list of tuples manually in my SQL query, but trying to loop through the records I get an error stating the tuple index out of range. Not sure what the issue is here and how to remedy it. Any help would be appreciated. Thank you.
Picture is attached for code and error messages. ErrorMSG PythonCode
import mysql.connector
import xlrd
mydb = mysql.connector.connect(
host = "localhost",
user = "root",
passwd = "*******",
database = "testdb",
)
mycursor = mydb.cursor()
loc=("E:\\SOX Reports\\2021 Reports\\populateData.xlsx")
l=list()
a=xlrd.open_workbook(loc)
sheet=a.sheet_by_index(0)
sheet.cell_value(0,0)
for i in range (1,418):
l.append(tuple(sheet.row_values(i)))
addRows = "insert into emplist (ID,SITE, OU, firstName) values(%s,%s,%s,%s)"
mycursor.executemany(addRows,l)
mydb.commit()
mydb.close()
I have a mysql database in which 'user' table having f_name,l_name,password email(pk) by which session is created and table 'friendgroup' having fg_name(pk), email((pk),users.email(FK)) and table 'member' having email(pk,user.email(fk)), owner_email(pk,friendgroup.email(fk)), fg_name(pk,friendgroup.fg_name(fk)), and a python flask file below.
After login account, I wish to add a friend in chat. I tried to fix it from session['email']
def add_friend():
user = session['email']
friendgroups = _get_own_friendgroups(user) return
render_template('addFriend.html', friendgroups=friendgroups)
def _get_own_friendgroups(user):
cursor = mysql.connection.cursor()
#find all friendgroups that the user owns
find_owned_friendgroups = 'SELECT fg_name, description FROM friendgroup WHERE owner_email = %s ORDER BY fg_name ASC'
cursor.execute(find_owned_friendgroups, (user))
owned_friendgroups = cursor.fetchall()
cursor.close()
return owned_friendgroups
I expect output will be an open window and actively use of add friend when needed but showing error:
MySQLdb._exceptions.ProgrammingError: not all arguments converted during bytes formatting
A common error in python is to use (bar) instead of (bar,) the former not being a tuple.
Try with:
cursor.execute(find_owned_friendgroups, (user,))
I am trying to create a function that will query a database using a string (or part of a string) inserted via raw_input.
The database PManDB has only one table ( accounts ) with 3 columns ( website, username, password ).
Example: if in the column website I have the string 'www.website.com' I want to obtain it inserting the values 'w' or 'web' or 'com' in the raw_input.
The full code of the project is here.
Below an extract without the function to print the database:
def SearchRecords(self):
search = raw_input("Insert value to be searched in the DB:")
db = MySQLdb.connect("localhost", "root", "pass", "PManDB")
cursor = db.cursor()
sql = """SELECT * FROM accounts WHERE website LIKE %s"""
data = (search)
try:
# Execute the SQL command
cursor.execute(sql, data)
print "The research has been made with success"
except:
# Rollback in case there is any error
print "There is an error"
db.rollback()
# disconnect from server
The idea is to add a cursor.fetchall() after the print statement "The research has been made with success" using the following sintax
result = cursor.fetchall()
print result
Or to add a for loop
result = cursor.fetchall()
for row in result:
print row[0]
But in both ways the code is not working.
Could you please help me understanding where am I wrong and why?
EDIT1: According to mata comment, I have added the error.
When I launch the application in PyCharm, introducing the modification proposed by mata, I receive the following error
Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)
The code updated is the following (still not working with the error above):
def SearchRecords(self):
search = raw_input("Insert value to be searched in the DB:")
db = MySQLdb.connect("localhost", "root", "studio", "PManDB")
cursor = db.cursor()
sql = """SELECT * FROM accounts WHERE website LIKE %s"""
data = ('%' + search + '%',)
cursor.execute(sql, data)
result = cursor.fetchall()
print result
Solved removing the db.close and the db.commit at the end of the function.
This question already has answers here:
How do I connect to a MySQL Database in Python?
(26 answers)
Closed 5 years ago.
i am familiar with python and also familiar with mysql and SQL. I am also clear on con, cur and commit, but my problem here is that im trying to make a small program in python (no need for a gui) to insert data into a mysql database, and bring it on my console or file. I'm confused where to start, i tried seeking google but i couldn't find any toturials about my issue, any help? a link or where to start. Also:
i know the python program can be written in an IDE or a text file, but how does it connect to mysql database? if im wrong please correct me.
SQLAlchemy is good: https://www.sqlalchemy.org/
Otherwise, using the conn/cur as you described is easy: https://www.tutorialspoint.com/python/python_database_access.htm
Go though the documentation to get yourself familiar with python, mysql and how to work with them together.
Although, the minimal code would look something like this :
import MySQLdb
query = "insert into DB_NAME values (1,2)"
try :
conn = MySQLdb.connect(host="",
user="",
passwd="",
db="")
cursor = conn.cursor()
cursor.execute(query)
conn.commit()
cursor.close()
conn.close()
except (MySQLdb.Error, Exception) as error :
print error
print "Insert data unsuccessful"
See the code below
import mysql.connector
from mysql.connector import MySQLConnection, Error
class SQL_Connect:
def __init__(self):
#-------------------------------------------------------
# Database Connection Param's
self.host_Address = 'Host Here'
self.database_Name = 'Database Name'
self.userName = 'User Name'
self.db_Password = 'Password'
#-------------------------------------------------------
def insert_IntoDB(self, Manufacturer, partNum, formFactor, socket, chipSet, memSlots, memType, maxMem, raidSup, onboardVid, crosFire_Sup, sli_Sup, sata6GBS, sataExpress, onboard_Ether):
test_Query = 'INSERT INTO motherboards (Manufacturer, modelNum, formFactor, socket, chipset, memSlots, memType, maxMem, raidSup, onboardVid, crosfireSup, sliSup, sata6GBS, sataExpress, onboardEther) VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)'
args = (Manufacturer, partNum, formFactor, socket, chipSet, memSlots, memType, maxMem, raidSup, onboardVid, crosFire_Sup, sli_Sup, sata6GBS, sataExpress, onboard_Ether)
try:
conn = mysql.connector.connect(host = self.host_Address, database = self.database_Name, user = self.userName, password = self.db_Password)
if conn.is_connected():
print 'MySQL Database Connection Established'
cursor = conn.cursor()
cursor.execute(test_Query, args)
conn.commit()
print 'Data Inserted!!!'
except Error as e:
print ('ERROR: ',e)
finally:
cursor.close()
conn.close()