Inserting an array into mysql with Python - python

I am building this array with 40k entries.
array = [(value1, value2, value3),(value1, value2, value3),(value1, value2, value3) .... ]
Is it possible to insert this into mysql in python something like:
cursor.execute('''INSERT IGNORE into %s VALUES *array here*''' % (table_name, array))
I am having trouble passing the array variable into mysql correctly. Any help appreciated.

Yes you can do it with executemany:
cursor.executemany('INSERT IGNORE into %s VALUES(%s, %s, %s)'%table_name, sql_data)
Note: you shouldn't use % to pass values to the database, instead you need to pass them in the second parameter of execute/executemany. i used % for the table name because the first parameter is the prepared query string.

This is Use for me try it.
mydb = mysql.connector.connect(
host="localhost",
user="root",
passwd="",
database="test"
)
mycursor = mydb.cursor()
print("Insert Process... Please Wait...")
for r in range(1,tdrows+1):
a = []
for c in range(1,tdcols+1):
a.append(driver.find_element_by_xpath("//*[#id='DataTables_Table_0']/tbody/tr["+str(r)+"]/td["+str(c)+"]").text)
sql = "INSERT IGNORE into test_table(id,fname,lname) VALUES (%s, %s, %s)"
val = (a)
mycursor.execute(sql,val)
mydb.commit()
print(mycursor.rowcount, "Record Inserted.")

Related

How to insert a list in postgres through python

I have a list
A=[1,2,3,3,4,5,6,8,90,8,6,5]
I want to put this list into a postgres table
After making a cursor and connection
I tried this
for i in A:
cusror.execute("Insert into schema1.table1 Values (%s)" ,i)
connection.commit()
But getting an error
TypeError: Not all arguments converted during string formatting.
Can someone help me out please
Use this function I will provide, just make sure to change TABLE_NAME and the columns for the ones you will be inserting:
import psycopg2
def bulkInsert(records):
try:
connection = psycopg2.connect(user="sysadmin",
password="pynative##29",
host="127.0.0.1",
port="5432",
database="postgres_db")
cursor = connection.cursor()
# Here replace the table and the columns
sql_insert_query = """ INSERT INTO TABLE_NAME (id, model, price)
VALUES (%s,%s,%s) """
# executemany() to insert multiple rows
result = cursor.executemany(sql_insert_query, records)
connection.commit()
print(cursor.rowcount, "Record inserted successfully into mobile table")
except (Exception, psycopg2.Error) as error:
print("Failed inserting record into mobile table {}".format(error))
finally:
# closing database connection.
if connection:
cursor.close()
connection.close()
print("PostgreSQL connection is closed")
# Example of how to use the function
records_to_insert = [(4, 'LG', 800), (5, 'One Plus 6', 950)]
bulkInsert(records_to_insert)
The 2nd argument to cursor.execute() should be a tuple. Try:
for i in A:
cursor.execute("Insert into schema1.table1 Values (%s)", (i,))
connection.commit()
Noting down a point from documentation -
For positional variables binding, the second argument must always be a sequence, even if it contains a single variable (remember that Python requires a comma to create a single element tuple):
>>> cur.execute("INSERT INTO foo VALUES (%s)", "bar") # WRONG
>>> cur.execute("INSERT INTO foo VALUES (%s)", ("bar")) # WRONG
>>> cur.execute("INSERT INTO foo VALUES (%s)", ("bar",)) # correct
>>> cur.execute("INSERT INTO foo VALUES (%s)", ["bar"]) # correct
Correct answer for your issue would be
for i in A:
cusror.execute("Insert into schema1.table1 Values (%s)" ,(i,))
connection.commit()
Using psycopg2 Fast execution helpers:
import psycopg2
from psycopg2.extras import execute_batch,execute_values
con = psycopg2.connect("dbname=test user=postgres host=localhost port=5432")
cur = con.cursor
A=[1,2,3,3,4,5,6,8,90,8,6,5]
param_list = [[id] for id in A]
#Using execute_batch
sql = "Insert into public.table1 values (%s)"
execute_batch(cur, sql, param_list)
con.commit()
#Using execute_values
sql = "Insert into public.table1 values %s"
execute_values(cur, sql, param_list)
con.commit()

Executing multiple SQL queries with Python Flask

I have a python function which should execute 2 SQL queries. I have found that it is impossible to execute 2 queries in one command at once, so as a workaround I created a list of my queries and try to iterate over it with execute command. However nothing is added to MySQL table. Here is the code:
#app.route('/addComment', methods=['POST'])
def addComment():
try:
if session.get('user'):
_description = request.form['description']
_user = session.get('user')
_term_id = request.form['termID']
_time = datetime.now()
operation = ['"INSERT INTO comments (description, user, termID, time) VALUES (%s, %s, %s, %s)", (_description, _user, _term_id, _time)', '"INSERT INTO history (user, term, time) VALUES (%s, %s, %s)", (_user, _term_id, _time)']
conn = mysql.connect()
cursor = conn.cursor()
for item in operation:
cursor.execute()
conn.commit()
data = cursor.fetchall()
if len(data) == 0:
conn.commit()
return json.dumps({'status':'OK'})
else:
return json.dumps({'status':'ERROR'})
except Exception as e:
return json.dumps({'status':'Unauthorized access'})
finally:
cursor.close()
conn.close()
Could you please help me?
Errors in your code lies in the following areas:
A. On iteration sql statement is not passed to execute()
Should be:
for item in operation:
cursor.execute(item)
conn.commit()
B. Invalid parameterization
'"INSERT INTO comments (description, user, termID, time) VALUES (%s, %s, %s, %s)", (_description, _user, _term_id, _time)'
This string statement doesn't apply variables to SQL statement string. Depending on your value types you should decide whether to add ' (apostrophe) or not. More safely would be to pass parameters to .execute() function. Example below.
cursor.execute(
"INSERT INTO comments (description, user, termID, time) VALUES (:description, :user, :term_id, :time)",
description=_description,
user=_user,
term_id=_term_id,
time=_time
)

: Unknown column in 'field list'

Was trying to set time and the other value to record table but showing: Unknown column 'time' in 'field list', I define time into VARCHAR(255) AND make the value specific to "e" because it is char type, can anyone here help me out with this, much appreciated
import mysql.connector
import datetime
d = datetime.datetime.now()
e = d.strftime("%Y-%m-%d %H:%M:%S")
mydb = mysql.connector.connect(
host="localhost",
user="root",
password="asd619248636",
database="mydatabase"
)
mycursor = mydb.cursor()
mycursor.excute=("CREATE TABLE record (temperature FLOAT(20) , humidity FLOAT(20), time VARCHAR(255))")
#mycursor.execute("SHOW TABLES")
#for x in mycursor:
#print(x)
sql = "INSERT INTO record (temperature,humidity,time) VALUES (%s, %s, %s)"
val = (3.2,6.5,"e")
mycursor.execute(sql,val)
mydb.commit()
print(mycursor.rowcount, "record inserted.")
I think time is a keyword in mysql according to this
So in order to use it as a column name you will need to enclose it with backticks:
create_table_sql = "CREATE TABLE record (temperature FLOAT(20) , humidity FLOAT(20), `time` VARCHAR(255))"
sql = "INSERT INTO record (temperature,humidity,`time`) VALUES (%s, %s, %s)"

Python scraper how to add mysql insert

What is wrong in my code ? I want to add %s to my mysql db.
titlux = tree.xpath('//*[#id="offer-title"]/h1/text()')
pretx = tree.xpath('//*[#id="offer-price-stock"]/div[3]/span/#content')
print "%s," % titlux
print "%s," % pretx
print "\n"
conn = ..............
x = conn.cursor()
try:
x.execute ("""INSERT INTO produse (titlu, pret) VALUES (%s, %s)""")
conn.commit()
except:
conn.rollback()
conn.close()
You're missing the replacement variables and some quotes in your SQL insert. Change it to:
x.execute ("""INSERT INTO produse (titlu, pret) VALUES ("%s", "%s")""" % (titlux[0], pretx[0]))
#Alastair has the right answer but if you want to see the query you're using
print "INSERT INTO produse (titlu, pret) VALUES (%s, %s)" % (titlux, pretx)
x.execute ("""INSERT INTO produse (titlu, pret) VALUES (%s, %s)""" % (titlux, pretx))

Problem with inserting into MySQL database from Python

I am having trouble inserting a record into a MySQL database from python. This is what I am doing.
def testMain2():
conn = MySQLdb.connect(charset='utf8', host="localhost", user="root", passwd="root", db="epf")
cursor = conn.cursor()
tableName = "test_table"
columnsDef = "(export_date BIGINT, storefront_id INT, genre_id INT, album_id INT, album_rank INT)"
exStr = """CREATE TABLE %s %s""" % (tableName, columnsDef)
cursor.execute(exStr)
#Escape the record
values = ["1305104402172", "12", "34", "56", "78"]
values = [conn.literal(aField) for aField in values]
stringList = "(%s)" % (", ".join(values))
columns = "(export_date, storefront_id, genre_id, album_id, album_rank)"
insertStmt = """INSERT INTO %s %s VALUES %s""" % (tableName, columns, stringList)
cursor.execute(insertStmt)
cursor.close()
conn.close()
The table is created however nothing is in the table. I can run the INSERT statement successfully in Terminal with the same credentials.
Any suggestions on what I may be doing wrong?
You haven't committed the transaction.
conn.commit()
(The MySQLdb library sets autocommit to False when connecting to MySQL. This means that you need to manually call commit or your changes will never make it into the database.)

Categories