I'm trying to fill in a SQLite database using the Python library sqlite3. In this example, the idea is simply to read data from a file and populate the database, however, although it seems the table is being created, the database is not growing and values are not written to it.
The code I have is below:
import sqlite3
def update_db(c, key, val):
sql = ''' UPDATE atable SET f1 = ? WHERE id = ?; '''
c.execute(sql, (val, key))
def create_table(c):
sql = ''' CREATE TABLE atable (id text PRIMARY KEY, f1 integer DEFAULT 0); '''
c.execute(sql)
with sqlite3.connect('test.db') as conn:
c = conn.cursor()
create_table(c)
with open('file1.txt') as f1:
for line in f1:
l = line.strip().split()
update_db(c, l[0], int(l[1]))
conn.commit()
This code runs without errors, but when trying to query this database, either with Python:
with sqlite3.connect('test.db') as conn:
c = conn.cursor()
c.execute('SELECT * FROM atable;')
for row in c:
print(row)
or in the SQLite command interface:
$ sqlite3 test.db
SQLite version 3.8.10.2 2015-05-20 18:17:19
Enter ".help" for usage hints.
sqlite> SELECT * FROM atable;
sqlite> .tables
atable
sqlite>
the output is always empty (but it looks the table was correctly create). What am I doing wrong here?
The 'file1.txt' for testing is this:
foo 2
bar 0
baz 1
The SQL syntax for adding rows to your table is:
INSERT INTO atable (id, f1) VALUES (?, ?);
UPDATE will do nothing if there are not already rows in the table.
If you want to insert or replace an existing row, sqlite also supports an INSERT OR REPLACE command.
Related
in my python code I insert a value into a table.
In the table, there is a sequence which automatically assigns an ID.
After the insert, I want to get this it back in to my python application:
import cx_Oracle, sys
with cx_Oracle.connect(user=ORA_USER,password=ORA_PWD,dsn=ORA_DSN) as conn:
with conn.cursor() as cur:
cur.execute("Insert into my_table columns(data) values ('Hello')")
conn.commit()
with cx_Oracle.connect(user=ORA_USER,password=ORA_PWD,dsn=ORA_DSN) as conn:
with conn.cursor() as cur:
r = cur.execute("select id from my_table where data = 'Hello'")
print(r)
if r is None:
print("Cannot retrieve ID")
sys.exit()
Unfortunately, the result set r is always "None" even though the value has been inserted properly (checked via sqldeveloper).
What am I doing wrong?
I even open a new connection to be sure to grab the value...
After calling execute() for a SELECT statement you need to call fetchone(), fetchmany() or fetchall() as shown in the cx_Oracle documentation SQL Queries.
Or you can use an iterator:
with connection.cursor() as cursor:
try:
sql = """select systimestamp from dual"""
for r in cursor.execute(sql):
print(r)
sql = """select 123 from dual"""
(c_id,) = cursor.execute(sql).fetchone()
print(c_id)
except oracledb.Error as e:
error, = e.args
print(sql)
print('*'.rjust(error.offset+1, ' '))
print(error.message)
However to get an automatically generated ID returned without the overhead of an additional SELECT, you can change the INSERT statement to use a RETURNING INTO clause. There is an example in the cx_Oracle documentation DML RETURNING Bind Variables that shows an UPDATE. You can use similar syntax with INSERT.
With the table:
CREATE TABLE mytable
(myid NUMBER(11) GENERATED BY DEFAULT ON NULL AS IDENTITY (START WITH 1),
mydata VARCHAR2(20));
You can insert and get the generated key like:
myidvar = cursor.var(int)
sql = "INSERT INTO mytable (mydata) VALUES ('abc') RETURNING myid INTO :bv"
cursor.execute(sql, bv=myidvar)
i, = myidvar.getvalue()
print(i)
If you just want a unique identifier you get the ROWID of an inserted row without needing a bind variable. Simple access cursor.lastrowid after executing an INSERT.
I want to insert a new row in my table by using the python-mariadb connector. For that I prefer to use the SET clause.
For some reason it does work if I only want to save ints (i.e y=2), but when I use a string, the following error occurs
Unknown column 'myString' in 'field list'
It seems it thinks the content of the string is a column name? Any idea how to fix that (I can do it with INSERT INTO ... VALUES ..., but I want to use the SET clause here). From my understanding, it should save both an int and a str without throwing an error
Thank you.
See the code example below
def myfunction():
x = 1
y ='myString'
db = connect_db()
cur = db.cursor()
sql = "INSERT INTO Table SET col1={}, col2={}"
cur.execute(sql.format(x, y))
db.commit()
db.close()
return
Here the MariaDB Connector, but this should be fine as it works for other db functions.
import mariadb
def connect_db():
db = mariadb.connect(
user="user",
password="123",
host="localhost",
port=3306,
database="DB"
)
db.autocommit = False
return db
you are not using right syntax for insert
sql = "INSERT INTO Table (col1,col2) values({}, {})"
but if you want to update an existing row:
sql = "UPDATE Table SET col1={}, col2={} WHERE id = {}"
and probably you need a where clause
The code in question produces the SQL statement:
INSERT INTO Table SET col1=1, col2=myString;
This is incorrect syntax, and strings must be in single-quotes:
INSERT INTO Table (col1, col2) VALUES (1, 'myString');
def myfunction():
x = 1
y ='myString'
db = connect_db()
cur = db.cursor()
sql = "INSERT INTO Table (col1, COL2) VALUES ({}, '{}')"
cur.execute(sql.format(x, y))
db.commit()
db.close()
return
But the above is fragile. Don't use string building methods to create SQL statements, it is much better to use parameter binding.
def myfunction():
x = 1
y ='myString'
db = connect_db()
cur = db.cursor()
sql = "INSERT INTO Table (col1, col2) VALUES (?, ?)"
cur.execute(sql, (x, y))
db.commit()
db.close()
return
The MariaDB connector documentation explains these things.
Retrieving Data
Once you have the initial code in place you can start working with the data. The first thing you should do is try to
retrieve information from the database. Here is code for a query
against the employees database:
cur.execute(
"SELECT first_name,last_name FROM employees WHERE first_name=?",
(some_name,))
MariaDB Connector/Python uses prepared statements, sanitizing and inserting the values from the tuple into the position
of the question marks (?). This is safer than inserting through
f-strings or format specifiers when working with user provided
information.
The query results are stored in a list in the cursor object. To view
the results, you can loop over the cursor.
Adding Data
Using the same execute() method with an INSERT statement, you can add rows to the table.
cursor.execute(
"INSERT INTO employees (first_name,last_name) VALUES (?, ?)",
(first_name, last_name))
I am trying to clean raw json data by parsing and inserting it into a table of an sqlite db.
I have 22 columns in my table and want to find a way of looping through them so I don't need to write 22 loops which insert the data or a single column.
I have simplified the approach I am trying with the following:
import sqlite3
conn = sqlite3.connect('cdata.sqlite')
cur = conn.cursor()
column = 'name'
value = 'test'
cur.execute('''INSERT INTO COMPANY (?)
VALUES (?)''',(column,),(value,))
conn.commit()
conn.close()
This doesn't work at the moment and return the error TypeError: function takes at most 2 arguments (3 given).
Does anyone know if it is possible to write an SQLite insert statement using 2 parameters like this or another way I might be able to iterate through the columns?
Sample as below:
import sqlite3
conn = sqlite3.connect("cdata.sqlite")
cur = conn.cursor()
column = ("name", "age")
table = f"CREATE TABLE IF NOT EXISTS COMPANY ({column[0]} text, {column[1]} text);"
cur.execute(table)
name = "hello"
age = "1"
sql_stmt = f"INSERT INTO COMPANY({column[0]},{column[1]}) VALUES ('{name}', '{age}')"
cur.execute(sql_stmt)
with conn:
cur.execute("SELECT * FROM COMPANY")
print(cur.fetchall())
conn.commit()
conn.close()
When trying to insert rows into a table with a unique index, it appears to simply silently not insert.
I've captured the behaviour in the following program: on the second call to test_insert I should get an integrity violation on the unique key. But nothing. Also, if I take the c.execute(query, [id_to_test]) line and duplicate itself below it, I do receive the proper integrity constraint as expected. What's happening here?
import sqlite3
def test_insert(id_to_test):
conn = sqlite3.connect('test.db')
c = conn.cursor()
query = '''INSERT INTO test(unique_id)
VALUES(?)'''
c.execute(query, [id_to_test])
def setup_table():
conn = sqlite3.connect('test.db')
c = conn.cursor()
c.execute('''DROP TABLE IF EXISTS test''')
c.execute('''CREATE TABLE test (unique_id text)''')
c.execute('''CREATE UNIQUE INDEX test_unique_id ON test (unique_id)''')
if __name__ == '__main__':
setup_table()
test_insert('test_id')
test_insert('test_id')
test_insert('test_id')
At the end of database operations, commit the changes to the database:
conn.commit()
I can't insert data into sqlite table.
Here is my code:
import sqlite3
connection = sqlite3.connect('mydata.db')
cursor = connection.cursor()
a=raw_input('name')
a=str(a)
b=raw_input('theme')
b=str(b)
c=raw_input('language')
c=str(c)
sql="INSERT INTO Website (Website, Theme, Language) VALUES (%, %, %)",(a,b,c)
cursor.execute(sql)
connection.commit()
connection.close()
For some reason it doesn't work.
the extended call syntax is f(*args):
cursor.execute(*sql)
sqlite uses '?' placeholder:
conn.execute('insert into sometable values (?,?,?)', (a,b,c))
raw_input() already returns a string. a = str(a) is unnecessary