We Keep Coding

Insert into with returning into :array

i want to add multiple rows to table in oracle and get back added values to python list
the returned column is the primary key and is generated on the oracle side
the code I wrote doesn't work, which is understandable, but I don't know how to write otherwise. help me please
out_id = cursor.var(cx_Oracle.NUMBER)
batch = []
for i in range(3):
batch.append({'val': i})
ins = 'INSERT INTO table (col1) VALUES (:val) RETURNING table.col2 into :out_id'
cursor.executemany(ins, batch)
print(list_out_id.getvalue())
I get an error
ORA-01036: illegal variable name/number

#e.burenina
A few observations here:
I am assuming col2 is an AUTO INCREMENT Number field
I am assuming you meant to use out_id instead of list_out_id in the last line
arraysize parameter should be included when you define the cursor variable out-id. i.e., out_id = cursor.var(cx_Oracle.NUMBER,arraysize=3).
cx_Oracle.NUMBER will return any number in Python float format(e.g., 0.0, 1.0 etc.). So if the returned data is an integer,
please use out_id = cursor.var(int,arraysize=3)
The batch list should include both the input 'val' and the output variable out_id
batch.append({'val':i,'out_id':out_id})
The following code should return the required values:
import cx_Oracle
# Establish the database connection - Add your DB details here
connection = cx_Oracle.connect(user="hr", password="hr", dsn="localhost/orclpdb")
# Obtain a cursor
cursor = connection.cursor()
# set the input and output variables
out_id = cursor.var(int, arraysize=3)
batch = []
for i in range(3):
batch.append({'val': i, 'out_id': out_id})
# set the SQL statement
ins = 'INSERT INTO table(col1) VALUES (:val) RETURNING table.col2 into :out_id'
# run the query and fetch the output into 'out_id' variable
cursor.executemany(ins, batch)
# print the out_id in a list
print(out_id.values)
Note: Please ensure that you do a connection.commit() at the end, in case you want to commit your code changes to the DB
Also make changes to the arraysize parameter based on the number of rows returned by your INSERT statement.

For printing output in the format
[{'val': 1, 'out_id': id_1}, {'val': 2, 'out_id': id_2}, {...}]
the following code will work
import cx_Oracle
# Establish the database connection - Add your DB details here
connection = cx_Oracle.connect(
user="hr", password="hr", dsn="localhost/orclpdb")
# Obtain a cursor cursor = connection.cursor()
# set the input and output variables
out_id = cursor.var(int, arraysize=3)
batch = []
for i in range(3):
batch.append({'val': i, 'out_id': out_id})
# set the SQL statement
ins = 'INSERT INTO table(col1) VALUES (:val) RETURNING table.col2 into :out_id'
# run the query and fetch the output into 'out_id' variable
cursor.executemany(ins, batch)
#print the output in the required format
for i in range(3):
batch[i]['out_id'] = batch[i]['out_id'].getvalue(i)
print(batch)
print(out_id.values)
# Do a commit to the table if required
# connection.commit()

I want to insert tables with variable lengths into an sqlite database but I am getting an error stating: Incorrect number of bindings supplied

Because I want to insert tables with varying sizes I need to be able to use a string variable in my insert statement for example like so:
numvals = '?,?'
vals = 'delays_values[0], delays_values[1]'
converted_valz = 'ID int ,Description text'
c.execute("CREATE TABLE DELAYS ("+converted_valz+")")
c.execute("INSERT INTO DELAYS VALUES ("+numvals+" )", (vals))
However this gives me the aforementioned error which says exactly: sqlite3.ProgrammingError: Incorrect number of bindings supplied. The current statement uses 19, and there are 349 supplied.
For some reason, if I replace vals with the hardcoded values it actually contains like so it actually works but then I am forced to hardcode values which defeats the purpose of all of this. Please help!:
numvals = '?,?'
vals = 'delays_values[0], delays_values[1]'
converted_valz = 'ID int ,Description text'
c.execute("CREATE TABLE DELAYS ("+converted_valz+")")
c.execute("INSERT INTO DELAYS VALUES ("+numvals+" )", (delays_values[0], delays_values[1]))

This might be an oversimplification that may cause problems later on, but an easy fix is to evaluate your vals string with eval(vals) in the end:
import sqlite3
conn = sqlite3.connect('example.db')
c = conn.cursor()
delays_values = (1,2)
numvals = '?,?'
vals = 'delays_values[0], delays_values[1]'
converted_valz = 'ID int ,Description text'
c.execute("CREATE TABLE DELAYS ("+converted_valz+")")
c.execute("INSERT INTO DELAYS VALUES ("+numvals+" )", eval(vals))
Don't know why vals is a string to begin with though to be honest...

Why are my SQL Query parameters not returning proper vales?

I'm trying to create an SQL queries for a large list of records (>42 million) to insert into a remote database. Right now I'm building queries in the format INSERT INTO tablename (columnnames) VALUES (values)
tablename, columnnames, and values are all of varying length so I'm generating a number of placeholders equal to the number of values required.
The result is I have a string called sqcommand that looks like INSERT INTO ColName (?,?,?) VALUES (?,?,?); and a list of parameters that looks like ([Name1, Name2, Name3, Val1, Val2, Val3]).
When try to execute the query as db.execute(sqlcommand, params) I get errors indicating I'm trying to insert into columns "#P1", "#P2", "#P3" et cetera. Why aren't the values from my list properly translating? Where is it getting "#P1" from? I know I don't have a column of that name and as far as I can tell I'm not referencing a column of that name yet the execute method is still trying to use it.
UPDATE: As per request, the full code is below, modified to avoid anything that might be private. The end result of this is to move data, row by row, from an sqlite3 db file to an AWS SQL server.
newDB = pyodbc.connect(newDataBase)
oldDB = sqlite3.connect(oldDatabase)
tables = oldDB.execute("SELECT * FROM sqlite_master WHERE type='table';").fetchall()
t0 = datetime.now()
for table in tables:
print('Parsing:', str(table[1]))
t1 = datetime.now()
colInfo = oldDB.execute('PRAGMA table_info('+table[1]+');').fetchall()
cols = list()
cph = ""
i = 0
for col in colInfo:
cph += "?,"
cols.append(str(col[1]))
rowCount = oldDB.execute("SELECT COUNT(*) FROM "+table[1]+" ;").fetchall()
count = 0
while count <= int(rowCount[0][0]):
params = list()
params.append(cols)
count += 1
row = oldDB.execute("SELECT * FROM "+table[1]+" LIMIT 1;").fetchone()
ph = ""
for val in row:
ph += "?,"
params = params.append(str(val))
ph = ph[:-1]
cph = cph[:-1]
print(str(table[1]))
sqlcommand = "INSERT INTO "+str(table[1])+" ("+cph+") VALUES ("+ph+");"
print(sqlcommand)
print(params)
newDB.execute(sqlcommand, params)
sqlcommand = "DELETE FROM ? WHERE ? = ?;"
oldDB.execute(sqlcommand, (str(table[1]), cols[0], vals[0],))
newDB.commit()

Unbeknownst to me, column names can't be passed as parameters. Panagiotis Kanavos answered this in a comment. I guess I'll have to figure out a different way to generate the queries. Thank you all very much, I appreciate it.

Using instance variables in SQLite3 update?

Ok so basically I'm trying to update an existing SQLite3 Database with instance variables (typ and lvl)
#Set variables
typ = 'Test'
lvl = 6
#Print Databse
print("\nHere's a listing of all the records in the table:\n")
for row in cursor.execute("SELECT rowid, * FROM fieldmap ORDER BY rowid"):
print(row)
#Update Info
sql = """
UPDATE fieldmap
SET buildtype = typ, buildlevel = lvl
WHERE rowid = 11
"""
cursor.execute(sql)
#Print Databse
print("\nHere's a listing of all the records in the table:\n")
for row in cursor.execute("SELECT rowid, * FROM fieldmap ORDER BY rowid"):
print(row)
As an Error I'm getting
sqlite3.OperationalError: no such column: typ
Now I basically know the problem is that my variable is inserted with the wrong syntax but I can not for the life of me find the correct one. It works with strings and ints just fine like this:
sql = """
UPDATE fieldmap
SET buildtype = 'house', buildlevel = 3
WHERE rowid = 11
"""
But as soon as I switch to the variables it throws the error.

Your query is not actually inserting the values of the variables typ and lvl into the query string. As written the query is trying to reference columns named typ and lvl, but these don't exist in the table.
Try writing is as a parameterised query:
sql = """
UPDATE fieldmap
SET buildtype = ?, buildlevel = ?
WHERE rowid = 11
"""
cursor.execute(sql, (typ, lvl))
The ? acts as a placeholder in the query string which is replaced by the values in the tuple passed to execute(). This is a secure way to construct the query and avoids SQL injection vulnerabilities.

Hey I think you should use ORM to manipulate with SQL database.
SQLAlchemy is your friend. I use that with SQLite, MySQL, PostgreSQL. It is fantastic.
That can make you get away from this syntax error since SQL does take commas and quotation marks as importance.
For hard coding, you may try this:
sql = """
UPDATE fieldmap
SET buildtype = '%s', buildlevel = 3
WHERE rowid = 11
""" % (house)
This can solve your problem temporarily but not for the long run. ORM is your friend.
Hope this could be helpful!

Populating sqlite3 database in Python loop

I have constructed a database but when I loop through my data to populate it, I get the following error:
OperationalError: no such column: tmp1
Code:
with con:
cur = con.cursor()
cur.execute("CREATE TABLE TESTTABLE(X REAL, Y REAL)")
for i in xrange(0,5):
tmp1 = array[i,0]
tmp2 = array[i,1]
with con:
cur.execute("""INSERT INTO TESTTABLE VALUES(tmp1,tmp2)""")
Basically I have a big array that I want to transfer into a database. This probably isn't the most efficient way of going about it. Suggestions?

If you want to insert values into a row, you need to pass those values along as SQL parameters to the .execute() call:
with con:
for i in xrange(0,5):
tmp1 = array[i, 0]
tmp2 = array[i, 1]
cur.execute("""INSERT INTO TESTTABLE VALUES(?, ?)""", (tmp1, tmp2))
The ? characters are parameters, and they are filled, in order, by values takes from the second argument to .execute(), a tuple. The above code will insert the numbers 0 through to 4 as pairs into the database.
Names in the SQL code have no correlation to names you define in Python, values can only be passed in explicitly.