I have a part in my python script that I need to insert some data into a table on a mysql database example below:
insert_data = "INSERT into test (test_date,test1,test2) values (%s,%s,%s)"
cur.execute(insert_data,(test_date,test1,test2))
db.commit()
db.close()
I have a couple of questions what is incorrect with this syntax and how is possible to change the VALUES to timestamp instead of %s for string? Note the column names in the database are the same as the data stored in the variables in my script.
THanks
try this:
import MySQLdb
import time
import datetime
ts = time.time()
timestamp = datetime.datetime.fromtimestamp(ts).strftime('%Y-%m-%d %H:%M:%S')
conn = MySQLdb.connect(host= "localhost",
user="root",
passwd="newpassword",
db="db1")
x = conn.cursor()
try:
x.execute("""INSERT into test (test_date,test1,test2) values(%s,%s,%s)""",(timestamp,test1,test2))
conn.commit()
except:
conn.rollback()
conn.close()
Timestamp creating can be done in one line, no need to use time.time(), just:
from datetime import datetime
timestamp = datetime.now().strftime('%Y-%m-%d %H:%M:%S')
Simply use the database NOW() function, e.g.
timestamp="NOW()"
insert_data = "INSERT into test (test_date,test1,test2) values (%s,%s,%s)"
cur.execute(insert_data,(test_date,test1,test2,timestamp))
db.commit()
db.close()
Related
In console.log I am getting the correct argument value, but when I try to add the argument value in the insert statement it is passing as 0. As a whole, the code works fine without errors, but for assg_id, instead of the actual value, it is inserting 0.
import pandas as pd
import sys
print ("parameters from nodejs", str(sys.argv[1]))
df = pd.read_csv("./userSetupData.csv")
df.head()
import mysql.connector as msql
from mysql.connector import Error
try:
conn = msql.connect(host='localhost', database='pythonTest', user='root', password='0000', auth_plugin='mysql_native_password')
if conn.is_connected():
cursor = conn.cursor()
cursor.execute("select database();")
record = cursor.fetchone()
assgn_id = str(sys.argv[1])
print('Checking the parameter value', assgn_id)
for i,row in df.iterrows():
#here %S means string values
sql = "INSERT INTO pythonTest.usr_stg VALUES (%s,%s,%s,%s,%s,assgn_id)"
cursor.execute(sql, tuple(row))
print("Record inserted")
# the connection is not auto committed by default, so we must commit to save our changes
conn.commit()
except Error as e:
print("Error while connecting to MySQL", e)
sql has normal string and it treats assgn_id as normal text in string, not Python's variable
You would have to use f-string and "{assgn_id}" to put value in this string
sql = f"INSERT INTO pythonTest.usr_stg VALUES (%s,%s,%s,%s,%s, {assgn_id})"
Or you should put assgn_id in tuple with parameters (and use %s in query)
sql = "INSERT INTO pythonTest.usr_stg VALUES (%s,%s,%s,%s,%s, %s)"
cursor.execute(sql, tuple(row.to_list() + [assgn_id]) )
I'm trying to store a mySQL query result in a pandas DataFrame using pymysql and am running into errors building the dataframe. Found a similar question here and here, but it looks like there are pymysql-specific errors being thrown:
import pandas as pd
import datetime
import pymysql
# dummy values
connection = pymysql.connect(user='username', password='password', databse='database_name', host='host')
start_date = datetime.datetime(2017,11,15)
end_date = datetime.datetime(2017,11,16)
try:
with connection.cursor() as cursor:
query = "SELECT * FROM orders WHERE date_time BETWEEN %s AND %s"
cursor.execute(query, (start_date, end_date))
df = pd.DataFrame(data=cursor.fetchall(), index = None, columns = cursor.keys())
finally:
connection.close()
returns: AttributeError: 'Cursor' object has no attribute 'keys'
If I drop the index and columns arguments:
try:
with connection.cursor() as cursor:
query = "SELECT * FROM orders WHERE date_time BETWEEN %s AND %s"
cursor.execute(query, (start_date, end_date))
df = pd.DataFrame(cursor.fetchall())
finally:
connection.close()
returns ValueError: DataFrame constructor not properly called!
Thanks in advance!
Use Pandas.read_sql() for this:
query = "SELECT * FROM orders WHERE date_time BETWEEN ? AND ?"
df = pd.read_sql(query, connection, params=(start_date, end_date))
Thank you for your suggestion to use pandas.read_sql(). It works with executing a stored procedure as well! I tested it in MSSQL 2017 environment.
Below is an example (I hope it helps others):
def database_query_to_df(connection, stored_proc, start_date, end_date):
# Define a query
query ="SET NOCOUNT ON; EXEC " + stored_proc + " ?, ? " + "; SET NOCOUNT OFF"
# Pass the parameters to the query, execute it, and store the results in a data frame
df = pd.read_sql(query, connection, params=(start_date, end_date))
return df
Try This:
import pandas as pd
import pymysql
mysql_connection = pymysql.connect(host='localhost', user='root', password='', db='test', charset='utf8')
sql = "SELECT * FROM `brands`"
df = pd.read_sql(sql, mysql_connection, index_col='brand_id')
print(df)
My goal is to take two variables, xdate and xtime and store them into an sqlite database in two separate columns using a python scripts. My code is
from datetime import datetime
import sqlite3 as mydb
import sys
con = mydb.connect('testTime.db')
def logTime():
i=datetime.now()
xdate = i.strftime('%Y-%m-%d')
xtime = i.strftime('%H-%M-%S')
return xdate, xtime
z=logTime()
this is where I get hung up I tried
try:
with con:
cur = con.cursor
cur.execute('INSERT INTO DT(Date, Time) Values (?,?)' (z[0],z[1]))
data = cur.fetchone()
print (data)
con.commit()
except:
with con:
cur=con.cursor()
cur.execute("CREATE TABLE DT(Date, Time)')
cur.commit()
I keep getting none when I try to fetch the data.
Any tips or recommended readings??
You are executing a insert query, it's result is not having any thing to fetch. You should run a select query and then fetch the data.
fetchone()
Fetches the next row of a query result set, returning a single sequence, or None when no more data is available.
An example -
>>> cur.execute('INSERT INTO DT(Date, Time) Values (?,?)', (z[0],z[1]))
<sqlite3.Cursor object at 0x0353DF60>
>>> print cur.fetchone()
None
>>> cur.execute('SELECT Date, Time from DT')
<sqlite3.Cursor object at 0x0353DF60>
>>> print cur.fetchone()
(u'2016-02-25', u'12-46-16')
The following logic works with the mysqldb module (see python mysqldb multiple cursors for one connection), but I am getting the following error with mysql.connector on cursor2.execute(sql)
"Unread result found."
I realize that I can use a join to combine these 2 simple sql statements and avoid the need for a second cursor, but my real world example is more complex and requires a second sql statement.
Assuming I need to execute 2 separate sql statements (1 for the loop and 1 inside the loop), how should this be done with the mysql.connector module?
import datetime
import mysql.connector
db = mysql.connector.connect(user='alan', password='please', host='machine1', database='mydb')
cursor1 = db.cursor()
cursor2 = db.cursor()
sql = """
SELECT userid,
username,
date
FROM user
WHERE date BETWEEN %s AND %s
"""
start_date = datetime.date(1999, 1, 1)
end_date = datetime.date(2014, 12, 31)
cursor1.execute(sql, (start_date, end_date))
for (userid, username, date) in cursor1:
sql = """
select count(*)
from request
where assigned = '%s'
""" % (userid)
cursor2.execute(sql)
requestcount = cursor2.fetchone()[0]
print userid, requestcount
cursor2.close()
cursor1.close()
db.close()
This mysqldb version works just fine:
import datetime
import MySQLdb
db = MySQLdb.connect(user='alan', passwd='please', host='machine1', db='mydb')
cursor1 = db.cursor()
cursor2 = db.cursor()
sql = """
SELECT userid,
username,
date
FROM user
WHERE date BETWEEN %s AND %s
"""
start_date = datetime.date(1999, 1, 1)
end_date = datetime.date(2014, 12, 31)
cursor1.execute(sql, (start_date, end_date))
for (userid, username, date) in cursor1:
sql = """
select count(*)
from request
where assigned = '%s'
""" % (userid)
cursor2.execute(sql)
requestcount = cursor2.fetchone()[0]
print userid, requestcount
cursor2.close()
cursor1.close()
db.close()
MySQL Connector/Python is, by default, non-buffering. This means the data is not fetched automatically and you need to 'consume' all rows. (It works with MySQLdb because that driver is buffering by default.)
Using Connector/Python you have to use the buffered-argument set to True for cursor you use as iterator. In the OP's question, this would be cursor1:
cursor1 = db.cursor(buffered=True)
cursor2 = db.cursor()
You can also use buffered=True as connection argument to make all cursor buffering instantiated by this connection buffering.
i want insert my data to my MySQL database but still can't, here is my code:
import ue9
d = ue9.UE9()
import datetime
from time import gmtime, strftime
import MySQLdb
timee = "%Y-%m-%d %H:%M:%S"
noww = strftime(timee, gmtime())
print noww
db = MySQLdb.connect("localhost","root","root","temperature")
cursor = db.cursor()
sql = ("INSERT INTO Mydate(datenow) VALUES(%s)",(noww))
try:
cursor.execute(sql)
db.commit()
except:
db.rollback()
db.close()
but still can't connect to database can anyone help me ?
You miss a comma in the list of values:
sql = ("INSERT INTO Mydate(datenow) VALUES(%s)",(noww,))