I am trying to create a function that will query a database using a string (or part of a string) inserted via raw_input.
The database PManDB has only one table ( accounts ) with 3 columns ( website, username, password ).
Example: if in the column website I have the string 'www.website.com' I want to obtain it inserting the values 'w' or 'web' or 'com' in the raw_input.
The full code of the project is here.
Below an extract without the function to print the database:
def SearchRecords(self):
search = raw_input("Insert value to be searched in the DB:")
db = MySQLdb.connect("localhost", "root", "pass", "PManDB")
cursor = db.cursor()
sql = """SELECT * FROM accounts WHERE website LIKE %s"""
data = (search)
try:
# Execute the SQL command
cursor.execute(sql, data)
print "The research has been made with success"
except:
# Rollback in case there is any error
print "There is an error"
db.rollback()
# disconnect from server
The idea is to add a cursor.fetchall() after the print statement "The research has been made with success" using the following sintax
result = cursor.fetchall()
print result
Or to add a for loop
result = cursor.fetchall()
for row in result:
print row[0]
But in both ways the code is not working.
Could you please help me understanding where am I wrong and why?
EDIT1: According to mata comment, I have added the error.
When I launch the application in PyCharm, introducing the modification proposed by mata, I receive the following error
Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)
The code updated is the following (still not working with the error above):
def SearchRecords(self):
search = raw_input("Insert value to be searched in the DB:")
db = MySQLdb.connect("localhost", "root", "studio", "PManDB")
cursor = db.cursor()
sql = """SELECT * FROM accounts WHERE website LIKE %s"""
data = ('%' + search + '%',)
cursor.execute(sql, data)
result = cursor.fetchall()
print result
Solved removing the db.close and the db.commit at the end of the function.
Related
I am inserting JSON data into a MySQL database
I am parsing the JSON and then inserting it into a MySQL db using the python connector
Through trial, I can see the error is associated with this piece of code
for steps in result['routes'][0]['legs'][0]['steps']:
query = ('SELECT leg_no FROM leg_data WHERE travel_mode = %s AND Orig_lat = %s AND Orig_lng = %s AND Dest_lat = %s AND Dest_lng = %s AND time_stamp = %s')
if steps['travel_mode'] == "pub_tran":
travel_mode = steps['travel_mode']
Orig_lat = steps['var_1']['dep']['lat']
Orig_lng = steps['var_1']['dep']['lng']
Dest_lat = steps['var_1']['arr']['lat']
Dest_lng = steps['var_1']['arr']['lng']
time_stamp = leg['_sent_time_stamp']
if steps['travel_mode'] =="a_pied":
query = ('SELECT leg_no FROM leg_data WHERE travel_mode = %s AND Orig_lat = %s AND Orig_lng = %s AND Dest_lat = %s AND Dest_lng = %s AND time_stamp = %s')
travel_mode = steps['travel_mode']
Orig_lat = steps['var_2']['lat']
Orig_lng = steps['var_2']['lng']
Dest_lat = steps['var_2']['lat']
Dest_lng = steps['var_2']['lng']
time_stamp = leg['_sent_time_stamp']
cursor.execute(query,(travel_mode, Orig_lat, Orig_lng, Dest_lat, Dest_lng, time_stamp))
leg_no = cursor.fetchone()[0]
print(leg_no)
I have inserted higher level details and am now searching the database to associate this lower level information with its parent. The only way to find this unique value is to search via the origin and destination coordinates with the time_stamp. I believe the logic is sound and by printing the leg_no immediately after this section, I can see values which appear at first inspection to be correct
However, when added to the rest of the code, it causes subsequent sections where more data is inserted using the cursor to fail with this error -
raise errors.InternalError("Unread result found.")
mysql.connector.errors.InternalError: Unread result found.
The issue seems similar to MySQL Unread Result with Python
Is the query too complex and needs splitting or is there another issue?
If the query is indeed too complex, can anyone advise how best to split this?
EDIT As per #Gord's help, Ive tried to dump any unread results
cursor.execute(query,(leg_travel_mode, leg_Orig_lat, leg_Orig_lng, leg_Dest_lat, leg_Dest_lng))
leg_no = cursor.fetchone()[0]
try:
cursor.fetchall()
except mysql.connector.errors.InterfaceError as ie:
if ie.msg == 'No result set to fetch from.':
pass
else:
raise
cursor.execute(query,(leg_travel_mode, leg_Orig_lat, leg_Orig_lng, leg_Dest_lat, leg_Dest_lng, time_stamp))
But, I still get
raise errors.InternalError("Unread result found.")
mysql.connector.errors.InternalError: Unread result found.
[Finished in 3.3s with exit code 1]
scratches head
EDIT 2 - when I print the ie.msg, I get -
No result set to fetch from
All that was required was for buffered to be set to true!
cursor = cnx.cursor(buffered=True)
The reason is that without a buffered cursor, the results are "lazily" loaded, meaning that "fetchone" actually only fetches one row from the full result set of the query. When you will use the same cursor again, it will complain that you still have n-1 results (where n is the result set amount) waiting to be fetched. However, when you use a buffered cursor the connector fetches ALL rows behind the scenes and you just take one from the connector so the mysql db won't complain.
I was able to recreate your issue. MySQL Connector/Python apparently doesn't like it if you retrieve multiple rows and don't fetch them all before closing the cursor or using it to retrieve some other stuff. For example
import mysql.connector
cnxn = mysql.connector.connect(
host='127.0.0.1',
user='root',
password='whatever',
database='mydb')
crsr = cnxn.cursor()
crsr.execute("DROP TABLE IF EXISTS pytest")
crsr.execute("""
CREATE TABLE pytest (
id INT(11) NOT NULL AUTO_INCREMENT,
firstname VARCHAR(20),
PRIMARY KEY (id)
)
""")
crsr.execute("INSERT INTO pytest (firstname) VALUES ('Gord')")
crsr.execute("INSERT INTO pytest (firstname) VALUES ('Anne')")
cnxn.commit()
crsr.execute("SELECT firstname FROM pytest")
fname = crsr.fetchone()[0]
print(fname)
crsr.execute("SELECT firstname FROM pytest") # InternalError: Unread result found.
If you only expect (or care about) one row then you can put a LIMIT on your query
crsr.execute("SELECT firstname FROM pytest LIMIT 0, 1")
fname = crsr.fetchone()[0]
print(fname)
crsr.execute("SELECT firstname FROM pytest") # OK now
or you can use fetchall() to get rid of any unread results after you have finished working with the rows you retrieved.
crsr.execute("SELECT firstname FROM pytest")
fname = crsr.fetchone()[0]
print(fname)
try:
crsr.fetchall() # fetch (and discard) remaining rows
except mysql.connector.errors.InterfaceError as ie:
if ie.msg == 'No result set to fetch from.':
# no problem, we were just at the end of the result set
pass
else:
raise
crsr.execute("SELECT firstname FROM pytest") # OK now
cursor.reset() is really what you want.
fetchall() is not good because you may end up moving unnecessary data from the database to your client.
The problem is about the buffer, maybe you disconnected from the previous MySQL connection and now it cannot perform the next statement. There are two ways to give the buffer to the cursor. First, only to the particular cursor using the following command:
import mysql.connector
cnx = mysql.connector.connect()
# Only this particular cursor will buffer results
cursor = cnx.cursor(buffered=True)
Alternatively, you could enable buffer for any cursor you use:
import mysql.connector
# All cursors created from cnx2 will be buffered by default
cnx2 = mysql.connector.connect(buffered=True)
cursor = cnx.cursor()
In case you disconnected from MySQL, the latter works for you.
Enjoy coding
If you want to get only one result from a request, and want after to reuse the same connexion for other requests, limit your sql select request to 1 using "limit 1" at the end of your request.
ex "Select field from table where x=1 limit 1;"
This method is faster using "buffered=True"
Set the consume_results argument on the connect() method to True.
cnx = mysql.connector.connect(
host="localhost",
user="user",
password="password",
database="database",
consume_results=True
)
Now instead of throwing an exception, it basically does fetchall().
Unfortunately this still makes it slow, if you have a lot of unread rows.
There is also a possibility that your connection to MySQL Workbench is disconnected. Establish the connection again. This solved the problem for me.
cursor.reset()
and then create tables and load entries
Would setting the cursor within the for loop, executing it, and then closing it again in the loop help?
Like:
for steps in result['routes'][0]['legs'][0]['steps']:
cursor = cnx.cursor()
....
leg_no = cursor.fetchone()[0]
cursor.close()
print(leg_no)
I have a localhost SQL Server running and am able to connect to it successfully. However, I am running into the problem of data not transfering over from temp csv files. Using Python import pyodbc for Server connection.
I've tried with Python Import pymssql but have had worse results so I've stuck with pyodbc. I've also tried closing the cursor each time or just at the end but not to any luck.
Here is a piece of code that I am using. Towards the bottom are two different csv styles. One is a temp in which is used to fill the SQL Server table. The other is for my personal use to make sure I am actually gathering information at the moment but, in the long term will be removed so only the temp csv is used.
#_retry(max_retry=1, timeout=1)
def blocked_outbound_utm_scada():
# OTHER CODE EXISTS HERE!!!
# GET Search Results and add to a temp CSV File then send to MS SQL Server
service_search_results_str = '/services/search/jobs/%s/results?output_mode=csv&count=0' % sid
search_results = (_service.request(_host + service_search_results_str, 'GET',
headers={'Authorization': 'Splunk %s' % session_key},
body={})[1]).decode('utf-8')
with tempfile.NamedTemporaryFile(mode='w+t', suffix='.csv', delete=False) as temp_csv:
temp_csv.writelines(search_results)
temp_csv.close()
try:
cursor.execute("BULK INSERT Blocked_Outbound_UTM_Scada FROM '%s' WITH ("
"FIELDTERMINATOR='\t', ROWTERMINATOR='\n', FirstRow = 2);" % temp_csv.name)
conn.commit()
except pyodbc.ProgrammingError:
cursor.execute("CREATE TABLE Blocked_Outbound_UTM_Scada ("
"Date_Time varchar(25),"
"Src_IP varchar(225),"
"Desktop_IP varchar(225));")
conn.commit()
finally:
cursor.execute("BULK INSERT Blocked_Outbound_UTM_Scada FROM '%s' WITH ("
"FIELDTERMINATOR='\t', ROWTERMINATOR='\n', FirstRow = 2);" % temp_csv.name)
conn.commit()
os.remove(temp_csv.name)
with open(_global_path + '/blocked_outbound_utm_scada.csv', 'a', newline='') as w:
w.write(search_results)
w.close()
I'm just trying to get the information into SQL Server but the code seems to be ignoring cursor.commit(). Any help is appreciated in figuring out what is wrong.
Thanks in Advance!
Try it without the conn.commit .
I do not understand why or how does it work but it seems, as well to me, that pyodbc ignores the commit clause.
Try change autocommit parameter in pymssql.connect()
conn = pymssql.connect(host=my_host, user=my_user, password=my_password, database=my_database, autocommit=True)
conn = pymssql.connect(host=my_host, user=my_user, password=my_password, database=my_database, autocommit=False)
This question already has answers here:
Does Python support MySQL prepared statements?
(7 answers)
How to use variables in SQL statement in Python?
(5 answers)
Closed 10 months ago.
I am new to python and i am facing problem while fetching data from mysql db while i am passing parameters in mysql query i think my mysql syntax is incorrect .
Here is the Error displayed on Screen Like this.
Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator at webmaster#localhost to inform them of the time this error occurred, and the actions you performed just before this error.
More information about this error may be available in the server error log.
Apache/2.4.6 (Ubuntu) Server at localhost Port 80
Here Is My Code For Select query in that I want to fetch data from get parameter of Url.
#!/usr/bin/python2.7
import cgi;
import cgitb; cgitb.enable();
import time, calendar;
print "Content-type: text/html\n\n";
print "<h1>Hello Python</h1>";
#!/usr/bin/python
import MySQLdb
# Create instance of FieldStorage
form = cgi.FieldStorage()
# Get data from fields
first_name = form.getvalue('first_name')
last_name = form.getvalue('last_name')
# Open database connection
db = MySQLdb.connect("localhost","root","123456789","testdrive" )
# prepare a cursor object using cursor() method
cursor = db.cursor()
# Prepare SQL query to INSERT a record into the database
sqlstmt = "SELECT * FROM EMPLOYEE WHERE FIRST_NAME = %(first_name)s AND LAST_NAME = %(last_name)s"
try:
# Execute the SQL command
cursor.execute(sqlstmt, {'first_name': first_name, 'last_name': last_name})
# Fetch all the rows in a list of lists.
results = cursor.fetchall()
for row in results:
fname = row[0]
lname = row[1]
age = row[2]
sex = row[3]
income = row[4]
# Now print fetched result
print "fname=%s,lname=%s,age=%d,sex=%s,income=%d" % \
(fname, lname, age, sex, income )
except:
print "Error: unable to fecth data"
# disconnect from server
db.close()
You have an error in this line
sql = "SELECT * FROM EMPLOYEE WHERE FIRST_NAME = '".first_name."' AND LAST_NAME = '".last_name."'"
This isn't PHP, meaning you can't concat strings and variables like this. I assume you wan't to make prepared statements. In that case, you should read following reply.
Relevant part of your code would look like this:
cursor.execute("SELECT * FROM EMPLOYEE WHERE FIRST_NAME = %s AND LAST_NAME = %s", [first_name, last_name])
First off, you should try abstracting all that into a single function you can call outside of CGI, but that's a whole other exercise now. Anyway, if you had done that you can get the stacktrace much easier to see what you did wrong, however, I can see you have a syntax error in the code you helpfully included
sql = "SELECT * FROM EMPLOYEE WHERE FIRST_NAME = '".first_name."' AND LAST_NAME = '".last_name."'"
Python string concatenation uses the + operator, not . like it is in PHP.
Second, this code is not secure. See http://xkcd.com/327/
To fix this, the cursor.execute method provides a second argument to fill out the tokens, this is what you should do
sqlstmt = "SELECT * FROM EMPLOYEE WHERE FIRST_NAME = %(first_name)s AND LAST_NAME = %(last_name)s"
try:
cursor.execute(sqlstmt, {'first_name': first_name, 'last_name': last_name})
...
I think we don't need fetchall() here, which maybe legacy from sqlite code. Just do something like:
for row in cursor:
x = row[0]
y = row[1]
...
Try this code
import mysql.connector
from mysql.connector import Error
try:
mySQLconnection = mysql.connector.connect(host='localhost',
database='python_database',
user='username',
password='passw0rd')
sql_select_Query = "select * from tablename"
cursor = mySQLconnection .cursor()
cursor.execute(sql_select_Query)
records = cursor.fetchall()
for row in records:
print("Sr No = ", row[0], )
print("Name = ", row[1])
print("Age = ", row[2])
print("Gender = ", row[3], "\n")
cursor.close()
except Error as e :
print ("Error connecting MySQL", e)
finally:
#closing database connection.
if(mySQLconnection .is_connected()):
connection.close()
print("MySQL connection is closed Now"
Ref
I can't quite seem to figure this one out. I've just started playing with Python today so I'm not exactly a pro.
I want to get the user to input a value and then that value to be inserted into my SQL query. What I've managed to do so far from googling is below but it doesn't seem to work.
import pyodbc
try:
con = pyodbc.connect(r'DRIVER={SQL Server};Server=server;DATABASE=MSTAR;UID=ADMIN;PWD=***;Trusted_Connection=yes')
cursor = con.cursor()
except Exception as e:
sys.exit('Cant connect')
Machine=input('Enter MachineID: ')
cursor.execute("""SELECT TOP 1 [STATE_ID] FROM [STAR].[dbo].[P_GPS] where MACHINE_ID like %s""", (Machine))
result = cursor.fetchall()
print result
I'm having an issue getting my python script to update my sqlite db.
The first part seems to work fine:
conn = sqlite3.connect('/Users/test/Desktop/my-accounts.db')
currentAccount = None
for row in conn.execute('SELECT email FROM accounts WHERE active=0'):
currentAccount = row[0]
print "Checking out: ",currentAccount
break
if currentAccount is None:
print "No available accounts"
Then this next part I want to take the variable currentAccount and update the row in the db where that value is.
else:
conn.execute('UPDATE accounts SET active=1 WHERE email=?', [currentAccount,])
conn.close()
I don't get any errors in the console but the db does not update. The email column is a VARCHAR and the active column is an INT.
Thanks.
SOLUTION was to add
conn.commit() after execute()
try to add conn.commit() after conn.execute("XXX"). Sometimes sqlite3 doesn't auto commit the execution.