The problem is that from the query result in the database:
cur1 = con.cursor()
result1 = ("SELECT DDATE FROM TABLE(NULL)")
cur1.execute(result1)
result1 = cur1.fetchone()
Result from the query - 43949.0
need put the result in the next query, replacing the first two "NULL" values in it, which select:
cur = con.cursor()
POS = (SELECT ST1,ST2 FROM SOMETABLE(**NULL**, **NULL**, NULL, NULL)")
cur.execute(POS)
POS = cur.fetchall()
The result should be a successful request like this: POS = (SELECT ST1,ST2 FROM SOMETABLE(43949.0, 43949.0, NULL, NULL)")
If you want to execute a second statement with input from the first statement, you can use a parameterized statement, and pass parameters from the first to the second in your python code.
For example, something like this:
cur = con.cursor()
cur.execute('select output1 from step1(null)')
result1 = cur.fetchone()
cur.execute('select output1, output2 from step2(?, ?, null, null)', (result1[0], result1[0]))
result2 = cur.fetchall()
Alternatively, you can join the stored procedures together to do this in one query. For example:
select s2.*
from step1(null) s1
cross join step2(s1.output1, s1.output1, null, null) s2
Contrary to normal tables, using a cross join with a stored procedure does not produce a cross-product, but instead behaves as a lateral join.
Related
I need to insert multiple values into a table after checking if it doesn't exist using psycopg2.
The query am using:
WITH data(name,proj_id) as (
VALUES ('hello',123),('hey',123)
)
INSERT INTO keywords(name,proj_id)
SELECT d.name,d.proj_id FROM data d
WHERE NOT EXISTS (SELECT 1 FROM keywords u2 WHERE
u2.name=d.name AND u2.proj_id=d.proj_id)
But how to format or add the values section from tuple to ('hello',123),('hey',123) in query.
As suggested in the comment, assuming that your connection is already established as conn one of the ways would be:
from typing import Iterator, Dict, Any
def insert_execute_values_iterator(connection, keywords: Iterator[Dict[str, Any]], page_size: int = 1000) -> None:
with connection.cursor() as cursor:
psycopg2.extras.execute_values(
cursor,
""" WITH data(name,proj_id) as (VALUES %s)
INSERT INTO keywords(name,proj_id)
SELECT d.name,d.proj_id FROM data d
WHERE NOT EXISTS (SELECT 1 FROM keywords u2 WHERE
u2.name=d.name AND u2.proj_id=d.proj_id);""",
(( keyword['name'],
keyword['proj_id'] ) for keyword in keywords),
page_size=page_size)
insert_execute_values_iterator(conn,{'hello':123,'hey':123})
insert_query = """WITH data(name, proj_id) as (
VALUES (%s,%s)
)
INSERT INTO keywords(name, proj_id)
SELECT d.name,d.proj_id FROM data d
WHERE NOT EXISTS (
SELECT 1 FROM keywords u2
WHERE u2.name = d.name AND u2.proj_id = d.proj_id)"""
tuple_values = (('hello',123),('hey',123))
psycopg2.extras.execute_batch(cursor,insert_query,tuple_values)
Beginners question here. I wish to populate a table with many rows of data straight from a query I'm running in the same session. I wish to do it using with excutemany(). currently, I insert each row as a tuple, as shown in the script below.
Select Query to get the needed data:
This query returns data with 4 columns Parking_ID, Snapshot_Date, Snapshot_Time, Parking_Stat
park_set_stat_query = "SET #row_number = 0;"
park_set_stat_query2 = "SET #row_number2 = 0;"
# one time load to catch only the changes done in the input table
park_change_stat_query = """select in1.Parking_ID,
in1.Snapshot_Date as Snapshot_Date,
in1.Snapshot_Time as Snapshot_Time,
in1.Parking_Stat
from (SELECT
Parking_ID,
Snapshot_Date,
Snapshot_Time,
Parking_Stat,
(#row_number:=#row_number + 1) AS num1
from Fact_Parking_Stat_Input
WHERE Parking_Stat<>0) as in1
left join (SELECT
Parking_ID,
Snapshot_Date,
Snapshot_Time,
Parking_Stat,
(#row_number2:=#row_number2 + 1)+1 AS num2
from Fact_Parking_Stat_Input
WHERE Parking_Stat<>0) as in2
on in1.Parking_ID=in2.Parking_ID and in1.num1=in2.num2
WHERE (CASE WHEN in1.Parking_Stat<>in2.Parking_Stat THEN 1 ELSE 0 END=1) OR num1=1"""
Here is the insert part of the script:
as you can see below I insert each row to the destination table Fact_Parking_Stat_Input_Alter
mycursor = connection.cursor()
mycursor2 = connection.cursor()
mycursor.execute(park_set_stat_query)
mycursor.execute(park_set_stat_query2)
mycursor.execute(park_change_stat_query)
# # keep only changes in a staging table named Fact_Parking_Stat_Input_Alter
qSQLresults = mycursor.fetchall()
for row in qSQLresults:
Parking_ID = row[0]
Snapshot_Date = row[1]
Snapshot_Time = row[2]
Parking_Stat = row[3]
#SQL query to INSERT a record into the table Fact_Parking_Stat_Input_Alter.
mycursor2.execute('''INSERT into Fact_Parking_Stat_Input_Alter (Parking_ID, Snapshot_Date, Snapshot_Time, Parking_Stat)
values (%s, %s, %s, %s)''',
(Parking_ID, Snapshot_Date, Snapshot_Time, Parking_Stat))
# Commit your changes in the database
connection.commit()
mycursor.close()
mycursor2.close()
connection.close()
How can I improve the code so it will insert the data in on insert command?
Thanks
Amir
MYSQL has an INSERT INTO command that is probably far more efficient than query it in python, pulling it and re-iserting
https://www.mysqltutorial.org/mysql-insert-into-select/
I have two variables to insert in my table.
user_id - int
marks - float
and I am having this data for multiple users like this:
user_ids = (-,-,-,-,-,-,-) **TUPLE**
marks = (-,-,-,-,-,-,-,-) **TUPLE**
I want to insert this data into my table using executemany and I am executing this query in my flask snippet:
con = pymysql.connect(
host=host,
user=user,
password=password,
db=db,
charset=charset,
cursorclass=pymysql.cursors.DictCursor,
port=port,
)
cur = con.cursor()
percs = calcattnonull()
# percs contains list of dictionaries.
# {[<'user_id'>: <'marks'>], [<'user_id'>: <'marks'>]........}
id_ = []
perc_ = []
final = []
for perc in tqdm(percs):
id_.append(perc["user_id"])
perc_.append(perc["att_perc"])
id_ = tuple(id_)
perc_ = tuple(perc_)
final.append(id_)
final.append(perc_)
cur.executemany(
"UPDATE dream_offline_calculate SET (user_id,att_percentage) VALUES (?,?)",
final,
)
con.commit()
I am getting this error again and again:
TypeError: not all arguments converted during string formatting
Thanks in advance for helping me.
executemany takes an iterable of the same placeholders you would use when calling execute several times.
So if your original query would be
cur.execute(
"UPDATE dream_offline_calculate SET (user_id,att_percentage) VALUES (?,?)",
(user_id, att_perc),
)
the equivalent executemany would be
cur.executemany(
"UPDATE dream_offline_calculate SET (user_id,att_percentage) VALUES (?,?)",
[(user_id, att_perc)],
)
So that said, simply
cur.executemany(
"UPDATE dream_offline_calculate SET (user_id,att_percentage) VALUES (?,?)",
[(perc["user_id"], perc["att_perc"]) for perc in percs],
)
should do the trick.
I am new to psycopg2. I have to insert data into the table with no duplicates. So, first I created a temporary table where I dumped all the data. And then, I check and add the data to the actual table.
Here is the code till now:
for eachline in content:
pmid ,first_name, last_name,initial,article_title,journal,language = eachline.split("\t")
cur.execute ("INSERT INTO AUTHOR_PMID(pmid, Author_lastname, Author_firstname, Author_initial,Article_title)
SELECT DISTINCT (pmid, Author_lastname, Author_firstname, Author_initial,Article_title)
FROM AUTHOR_PMID WHERE NOT EXISTS (SELECT "X" FROM AUTHOR_pmid_temp
WHERE
AUTHOR_pmid_temp.pmid = AUTHOR_PMID.pmid
AND AUTHOR_pmid_temp.Author_lastname = AUTHOR_PMID.Author_lastname
AND AUTHOR_pmid_temp.Author_firstname = AUTHOR_PMID.Author_firstname
AND AUTHOR_pmid_temp.Author_initial = AUTHOR_PMID.Author_initial
AND AUTHOR_pmid_temp.Article_title = AUTHOR_PMID.Article_title);")
con.commit()
error: syntax error.
Where am i going wrong?
Try inserting query with triple quotes instead of single like below
for eachline in content:
pmid ,first_name, last_name,initial,article_title,journal,language = eachline.split("\t")
cur.execute ("""INSERT INTO AUTHOR_PMID(pmid, Author_lastname, Author_firstname, Author_initial,Article_title)
SELECT DISTINCT (pmid, Author_lastname, Author_firstname, Author_initial,Article_title)
FROM AUTHOR_PMID WHERE NOT EXISTS (SELECT "X" FROM AUTHOR_pmid_temp
WHERE
AUTHOR_pmid_temp.pmid = AUTHOR_PMID.pmid
AND AUTHOR_pmid_temp.Author_lastname = AUTHOR_PMID.Author_lastname
AND AUTHOR_pmid_temp.Author_firstname = AUTHOR_PMID.Author_firstname
AND AUTHOR_pmid_temp.Author_initial = AUTHOR_PMID.Author_initial
AND AUTHOR_pmid_temp.Article_title = AUTHOR_PMID.Article_title);""")
con.commit()
For more info, please check here !!!
I have a file temperature.txt with columns:
city
avghigh
avglow
coldmonth
coldavghigh
coldavglow
warmmonth
warmavghigh
warmavglow
I need to return the names of the cities which have the same average low temperature.
I also have this function:
def run_query(db, q, args=None):
conn = sqlite3.connect(db)
cur = conn.cursor()
if args is None:
cur.execute(q)
else:
cur.execute(q, args)
results = cur.fetchall()
cur.close()
conn.commit()
conn.close()
return results
all I got thus far (If it's correct is)
return run_query(noname.db, ('Select Cities, AvgLow from Table')
In SQL, this can be easily done using a self join on the table to get matching AvgLow for different cities like this:
Select
t.Cities,
t.AvgLow
from Table1 t
INNER JOIN Table1 t1 ON t.AvgLow = t1.AvgLow
and t.Cities <> t1.Cities
ORDER BY t.AvgLow;
SQLite Demo