We Keep Coding

Insert nested arrays into sql from python

I have a list contains many lists in python.
my_list = [['city', 'state'], ['tampa', 'florida'], ['miami','florida']]
The nested list at index 0 contains the column headers, and rest of the nested lists contain corresponding values. How would I insert this into sql server using pyodbc or slqalchemy? I have been using pandas pd.to_sql and want to make this a process in pure python. Any help would be greatly appreciated.
expected output table would look like:
city |state
-------------
tampa|florida
miami|florida

Since the column names are coming from your list you have to build a query string to insert the values. Column names and table names can't be parameterised with placeholders (?).
import pyodbc
conn = pyodbc.connect(my_connection_string)
cursor = conn.cursor()
my_list = [['city', 'state'], ['tampa', 'florida'], ['miami','florida']]
columns = ','.join(my_list[0]) #String of column names
values = ','.join(['?'] * len(my_list[0])) #Placeholders for values
query = "INSERT INTO mytable({0}) VALUES ({1})".format(columns, values)
#Loop through rest of list, inserting data
for l in my_list[1:]:
cursor.execute(query, l)
conn.commit() #save changes
Update:
If you have a large number of records to insert you can do that in one go using executemany. Change the code like this:
columns = ','.join(my_list[0]) #String of column names
values = ','.join(['?'] * len(my_list[0])) #Placeholders for values
#Bulk insert
query = "INSERT INTO mytable({0}) VALUES ({1})".format(columns, values)
cursor.executemany(query, my_list[1:])
conn.commit() #save change

Assuming conn is already open connection to your database:
cursor = conn.cursor()
for row in my_list:
cursor.execute('INSERT INTO my_table (city, state) VALUES (?, ?)', row)
cursor.commit()

Since the columns value are are the first elemnts in the array, just do:
q ="""CREATE TABLE IF NOT EXISTS stud_data (`{col1}` VARCHAR(250),`{col2}` VARCHAR(250); """
sql_cmd = q.format(col1 = my_list[0][0],col2 = my_list[0][1])
mcursor.execute(sql)#Create the table with columns
Now to add the values to the table, do:
for i in range(1,len(my_list)-1):
sql = "INSERT IGNORE into test_table(city,state) VALUES (%s, %s)"
mycursor.execute(sql,my_list[i][0],my_list[i][1])
mycursor.commit()
print(mycursor.rowcount, "Record Inserted.")#Get count of rows after insertion

How to make automatically items id numeration?

I'm trying to insert some data into SQL database, and the problem is that I'm really green on this. So the MAIN problem is that How can I sort all the items in table? I have 3 main things: ID, CARNUM, TIME. But in this 'Insertion' I have to type the id manually. How can I make that the system would create a numeric id numeration automatically?
Here's the insertion code:
postgres_insert_query = """ INSERT INTO Vartotojai (ID, CARNUM, TIME) VALUES (%s,%s,%s)"""
record_to_insert = (id, car_numb, Reg_Tikslus_Laikas)
cursor.execute(postgres_insert_query, record_to_insert)
connection.commit()
count = cursor.rowcount
print (count, "Record inserted successfully into mobile table")
pgadmin sort
pgadmin table

You could change the datatype of ID to serial, which is an auto incrementing integer. Meaning that you don't have to manually enter an ID when inserting into the database.
Read more about datatype serial: source

Set Sqlite query results as variables [duplicate]

This question already has answers here:
How can I get dict from sqlite query?
(16 answers)
Closed 4 years ago.
Issue:
Hi, right now I am making queries to sqlite and assigning the result to variables like this:
Table structure: rowid, name, something
cursor.execute("SELECT * FROM my_table WHERE my_condition = 'ExampleForSO'")
found_record = cursor.fetchone()
record_id = found_record[0]
record_name = found_record[1]
record_something = found_record[2]
print(record_name)
However, it's very possible that someday I have to add a new column to the table. Let's put the example of adding that column:
Table structure: rowid, age, name, something
In that scenario, if we run the same code, name and something will be assigned wrongly and the print will not get me the name but the age, so I have to edit the code manually to fit the current index. However, I am working now with tables of more than 100 fields for a complex UI and doing this is tiresome.
Desired output:
I am wondering if there is a better way to catch results by using dicts or something like this:
Note for lurkers: The next snipped is made up code that does not works, do not use it.
cursor.execute_to(my_dict,
'''SELECT rowid as my_dict["id"],
name as my_dict["name"],
something as my_dict["something"]
FROM my_table WHERE my_condition = "ExampleForSO"''')
print(my_dict['name'])
I am probably wrong with this approach, but that's close to what I want. That way if I don't access the results as an index, and if add a new column, no matter where it's, the output would be the same.
What is the correct way to achieve it? Is there any other alternatives?

You can use namedtuple and then specify connection.row_factory in sqlite. Example:
import sqlite3
from collections import namedtuple
# specify my row structure using namedtuple
MyRecord = namedtuple('MyRecord', 'record_id record_name record_something')
con = sqlite3.connect(":memory:")
con.isolation_level = None
con.row_factory = lambda cursor, row: MyRecord(*row)
cur = con.cursor()
cur.execute("CREATE TABLE my_table (record_id integer PRIMARY KEY, record_name text NOT NULL, record_something text NOT NULL)")
cur.execute("INSERT INTO my_table (record_name, record_something) VALUES (?, ?)", ('Andrej', 'This is something'))
cur.execute("INSERT INTO my_table (record_name, record_something) VALUES (?, ?)", ('Andrej', 'This is something too'))
cur.execute("INSERT INTO my_table (record_name, record_something) VALUES (?, ?)", ('Adrika', 'This is new!'))
for row in cur.execute("SELECT * FROM my_table WHERE record_name LIKE 'A%'"):
print(f'ID={row.record_id} NAME={row.record_name} SOMETHING={row.record_something}')
con.close()
Prints:
ID=1 NAME=Andrej SOMETHING=This is something
ID=2 NAME=Andrej SOMETHING=This is something too
ID=3 NAME=Adrika SOMETHING=This is new!

How to insert a list containing values seperated by comma into MySQL database using python

I have to insert a list having integer values into a column of MySQL database table using Python. First I tried
cur.execute("insert into time_Interval (name, time_interval_list) values (%s, %s)",\
# (user_name, interval_list))
Where "interval_list" is the list variable having values that I want to insert into "time_interval_list" column of "time_Interval" table. Code is working but does not insert the values. Then I tried this solution changing the code as
var_string = ','.join('?' * len(interval_list))
cur.execute("insert into time_Interval (name, time_interval_list) values (%s, %s)",\
(user_name, var_string))
Again, code was working fine but inserts a list of question mark. I am unable to find the solution.
How to do this correctly. Any suggestion will be appreciated

Try changing
var_string = ','.join('?' * len(interval_list))
to
var_string = ','.join(interval_list)

The following code helped me bulk insert data into a database table using Python.
You can first create a tuple of tuples from the data you want to insert as follows
var_string = ','.join(cur.mogrify("(%s,%s)", x).decode('utf-8') for x in interval_list)
The interval_list would include both name and time_interval_list values whehn creating the tuple of tuples. Then run exceute command giving the tuple of tuples as input to the values.
cur.execute("INSERT INTO time_Interval (name, time_interval_list) VALUES " + var_string)
This command bulk inserts data into a database table very quickly.

How to add multiple Columns into Sqlite3 from a for loop in Python

Admittedly I a still very new to both Python and Sqlite3, and I am attempting to add the contents of two lists into a database so that one list is in the first column and the second list shows up in the second column. To this point, I have been unsuccessful. I am defenitely making a fundamental error, and the error message that I get is this: "sqlite3.InterfaceError: Error binding parameter 0 - probably unsupported type."
my code is this:
import sqlite3
names = ['Tom', 'Dick', 'Harry']
ids = ['A452', 'B698', 'Kd9f']
conn = sqlite3.connect('testforinput.db')
c = conn.cursor()
c.execute("CREATE TABLE thetable(name TEXT, id TEXT)")
index = 0
for link in names:
idofperson = ids[index]
c.execute("INSERT INTO thetable(name, id)VALUES(?, ?)", ( [link], idofperson ))
index+=1
conn.commit()
conn.close()
The error occurs because of the for loop specifically the "idofperson" variable
The desired outcome is that I would like to have two columns created in sql one being name and the other being id.
Any help would be greatly appreciated.

I think you just change
index =0
for link in names:
idofperson = ids[index]
c.execute("INSERT INTO thetable(name, id)VALUES(?, ?)", ( [link], idofperson ))
to this (use enumrate and change [list] to list, because you pass a list into a column need TEXT type):
for index, link in enumrable(names):
idofperson = ids[index]
c.execute("INSERT INTO thetable(name, id)VALUES(?, ?)", ( link, idofperson ))

your variable index is not increasing.try using the enumerate on for loop. or just add index += 1 after execute

the error is occurring because of the unsupported data type you are trying to push in, you can't store list as it is, you need to change to another supported data types, i like this solution ....it worked for me https://stackoverflow.com/a/18622264/6180263

for your problem, try this:
import sqlite3
names = ['Tom', 'Dick', 'Harry']
ids = ['A452', 'B698', 'Kd9f']
data = zip(names, ids)
conn = sqlite3.connect('testforinput.db')
c = conn.cursor()
c.execute("CREATE TABLE thetable(name TEXT, id TEXT)")
for d in data:
sql = "INSERT INTO thetable (name, id) VALUES ('%s', '%s'); " % d
c.execute(sql)
conn.commit()
conn.close()
I suggest change data to a list of dict, like this [{'name':'Tom', 'id': 'A452'}, {'name':'dick', 'id':'B698'}..]
and you can generate insert sql by data, this make the insert more flexible.