I am using python and postgresql. I have a table with 6 column. One id and 5 entries. I want to copy the id and most repeated entry in 5 entries to a new table.
I have done this:
import psycopg2
connection=psycopg2.connect("dbname=homedb user=ria")
cursor=connection.cursor()
l_dict= {'licence_id':1}
cursor.execute("SELECT * FROM im_entry.usr_table")
rows=cursor.fetchall()
cursor.execute("INSERT INTO im_entry.pr_table (image_1d) SELECT image_1d FROM im_entry.usr_table")
for row in rows:
p = findmax(row) #to get most repeated entry from first table
.................
.................
Then how can I enter this p value to the new table?
Please help me
p is a tuple so you can create a new execute with the INSERT statement passing the tuple (or part):
cursor.execute("INSERT INTO new_table (x, ...) VALUES (%s, ...)", p)
where:
(x, ....) contains the column names
(%s, ...) %s is repeated for each column
Related
I have a dataframe named Data2 and I wish to put values of it inside a postgresql table. For reasons, I cannot use to_sql as some of the values in Data2 are numpy arrays.
This is Data2's schema:
cursor.execute(
"""
DROP TABLE IF EXISTS Data2;
CREATE TABLE Data2 (
time timestamp without time zone,
u bytea,
v bytea,
w bytea,
spd bytea,
dir bytea,
temp bytea
);
"""
)
My code segment:
for col in Data2_mcw.columns:
for row in Data2_mcw.index:
value = Data2_mcw[col].loc[row]
if type(value).__module__ == np.__name__:
value = pickle.dumps(value)
cursor.execute(
"""
INSERT INTO Data2_mcw(%s)
VALUES (%s)
"""
,
(col.replace('\"',''),value)
)
Error generated:
psycopg2.errors.SyntaxError: syntax error at or near "'time'"
LINE 2: INSERT INTO Data2_mcw('time')
How do I rectify this error?
Any help would be much appreciated!
There are two problems I see with this code.
The first problem is that you cannot use bind parameters for column names, only for values. The first of the two %s placeholders in your SQL string is invalid. You will have to use string concatenation to set column names, something like the following (assuming you are using Python 3.6+):
cursor.execute(
f"""
INSERT INTO Data2_mcw({col})
VALUES (%s)
""",
(value,))
The second problem is that a SQL INSERT statement inserts an entire row. It does not insert a single value into an already-existing row, as you seem to be expecting it to.
Suppose your dataframe Data2_mcw looks like this:
a b c
0 1 2 7
1 3 4 9
Clearly, this dataframe has six values in it. If you were to run your code on this dataframe, then it would insert six rows into your database table, one for each value, and the data in your table would look like the following:
a b c
1
3
2
4
7
9
I'm guessing you don't want this: you'd rather your database table contained the following two rows instead:
a b c
1 2 7
3 4 9
Instead of inserting one value at a time, you will have to insert one entire row at time. This means you have to swap your two loops around, build the SQL string up once beforehand, and collect together all the values for a row before passing it to the database. Something like the following should hopefully work (please note that I don't have a Postgres database to test this against):
column_names = ",".join(Data2_mcw.columns)
placeholders = ",".join(["%s"] * len(Data2_mcw.columns))
sql = f"INSERT INTO Data2_mcw({column_names}) VALUES ({placeholders})"
for row in Data2_mcw.index:
values = []
for col in Data2_mcw.columns:
value = Data2_mcw[col].loc[row]
if type(value).__module__ == np.__name__:
value = pickle.dumps(value)
values.append(value)
cursor.execute(sql, values)
I have a list contains many lists in python.
my_list = [['city', 'state'], ['tampa', 'florida'], ['miami','florida']]
The nested list at index 0 contains the column headers, and rest of the nested lists contain corresponding values. How would I insert this into sql server using pyodbc or slqalchemy? I have been using pandas pd.to_sql and want to make this a process in pure python. Any help would be greatly appreciated.
expected output table would look like:
city |state
-------------
tampa|florida
miami|florida
Since the column names are coming from your list you have to build a query string to insert the values. Column names and table names can't be parameterised with placeholders (?).
import pyodbc
conn = pyodbc.connect(my_connection_string)
cursor = conn.cursor()
my_list = [['city', 'state'], ['tampa', 'florida'], ['miami','florida']]
columns = ','.join(my_list[0]) #String of column names
values = ','.join(['?'] * len(my_list[0])) #Placeholders for values
query = "INSERT INTO mytable({0}) VALUES ({1})".format(columns, values)
#Loop through rest of list, inserting data
for l in my_list[1:]:
cursor.execute(query, l)
conn.commit() #save changes
Update:
If you have a large number of records to insert you can do that in one go using executemany. Change the code like this:
columns = ','.join(my_list[0]) #String of column names
values = ','.join(['?'] * len(my_list[0])) #Placeholders for values
#Bulk insert
query = "INSERT INTO mytable({0}) VALUES ({1})".format(columns, values)
cursor.executemany(query, my_list[1:])
conn.commit() #save change
Assuming conn is already open connection to your database:
cursor = conn.cursor()
for row in my_list:
cursor.execute('INSERT INTO my_table (city, state) VALUES (?, ?)', row)
cursor.commit()
Since the columns value are are the first elemnts in the array, just do:
q ="""CREATE TABLE IF NOT EXISTS stud_data (`{col1}` VARCHAR(250),`{col2}` VARCHAR(250); """
sql_cmd = q.format(col1 = my_list[0][0],col2 = my_list[0][1])
mcursor.execute(sql)#Create the table with columns
Now to add the values to the table, do:
for i in range(1,len(my_list)-1):
sql = "INSERT IGNORE into test_table(city,state) VALUES (%s, %s)"
mycursor.execute(sql,my_list[i][0],my_list[i][1])
mycursor.commit()
print(mycursor.rowcount, "Record Inserted.")#Get count of rows after insertion
I have to insert a list having integer values into a column of MySQL database table using Python. First I tried
cur.execute("insert into time_Interval (name, time_interval_list) values (%s, %s)",\
# (user_name, interval_list))
Where "interval_list" is the list variable having values that I want to insert into "time_interval_list" column of "time_Interval" table. Code is working but does not insert the values. Then I tried this solution changing the code as
var_string = ','.join('?' * len(interval_list))
cur.execute("insert into time_Interval (name, time_interval_list) values (%s, %s)",\
(user_name, var_string))
Again, code was working fine but inserts a list of question mark. I am unable to find the solution.
How to do this correctly. Any suggestion will be appreciated
Try changing
var_string = ','.join('?' * len(interval_list))
to
var_string = ','.join(interval_list)
The following code helped me bulk insert data into a database table using Python.
You can first create a tuple of tuples from the data you want to insert as follows
var_string = ','.join(cur.mogrify("(%s,%s)", x).decode('utf-8') for x in interval_list)
The interval_list would include both name and time_interval_list values whehn creating the tuple of tuples. Then run exceute command giving the tuple of tuples as input to the values.
cur.execute("INSERT INTO time_Interval (name, time_interval_list) VALUES " + var_string)
This command bulk inserts data into a database table very quickly.
I'm attempting to update my sqlite db with 2 python lists. I have a sqlite db with three fields. Name, number, date. I also have three python lists with similar names. I'm trying to figure out a way to update my sqlite db with data from these 2 lists. I can get the db created, and even get a single column filled, but I cant seem to update it correctly or at all. Is there a way to INSERT both lists at once? Rather than INSERT a single column and then UPDATE the db with the other?
Here is what I have so far:
name_list = []
number_list = []
date = now.date()
strDate = date.strftime("%B %Y")
tableName = strDate
sqlTable = 'CREATE TABLE IF NOT EXISTS ' + tableName + '(name text, number integer, date text)'
c.execute(sqlTable)
conn.commit()
for i in name_list:
c.execute('INSERT INTO January2018(names) VALUES (?)', [i])
conn.commit()
I can't seem to get past this point. I still need to add another list of data (number_list) and attach the date to each row.
Here's what I have on that:
for i in number_list:
c.execute('UPDATE myTable SET number = ? WHERE name', [i])
conn.commit()
Any help would be much appreciated. And if you need more information, please let me know.
You can use executemany with zip:
c.executemany('INSERT INTO January2018 (name, number) VALUES (?, ?)', zip(name_list, number_list))
conn.commit()
I want to create a simple database that has a list of names and ID numbers. I want there to be 2 columns and a total of 4 rows - one for each person. The error I keep getting is:
OperationalError: table contacts has 2 columns but 4 values were
supplied
Here is my code:
import sqlite3
conn = sqlite3.connect("test2.db")
cursor = conn.cursor()
cursor.execute('''CREATE TABLE IF NOT EXISTS contacts (name text, id_num text)''')
list = [('Taylor', '001'), ('Tory', '002'), ('Joe', '003'), ('Steve', '004')]
cursor.executemany("INSERT INTO contacts VALUES (?,?,?,?)", list)
conn.commit()
You only have two values per row in your list so you need to tell the INSERT INTO statement to expect only two values rather than four:
cursor.executemany("INSERT INTO contacts VALUES (?,?)", list)