When trying to insert rows into a table with a unique index, it appears to simply silently not insert.
I've captured the behaviour in the following program: on the second call to test_insert I should get an integrity violation on the unique key. But nothing. Also, if I take the c.execute(query, [id_to_test]) line and duplicate itself below it, I do receive the proper integrity constraint as expected. What's happening here?
import sqlite3
def test_insert(id_to_test):
conn = sqlite3.connect('test.db')
c = conn.cursor()
query = '''INSERT INTO test(unique_id)
VALUES(?)'''
c.execute(query, [id_to_test])
def setup_table():
conn = sqlite3.connect('test.db')
c = conn.cursor()
c.execute('''DROP TABLE IF EXISTS test''')
c.execute('''CREATE TABLE test (unique_id text)''')
c.execute('''CREATE UNIQUE INDEX test_unique_id ON test (unique_id)''')
if __name__ == '__main__':
setup_table()
test_insert('test_id')
test_insert('test_id')
test_insert('test_id')
At the end of database operations, commit the changes to the database:
conn.commit()
Related
in my python code I insert a value into a table.
In the table, there is a sequence which automatically assigns an ID.
After the insert, I want to get this it back in to my python application:
import cx_Oracle, sys
with cx_Oracle.connect(user=ORA_USER,password=ORA_PWD,dsn=ORA_DSN) as conn:
with conn.cursor() as cur:
cur.execute("Insert into my_table columns(data) values ('Hello')")
conn.commit()
with cx_Oracle.connect(user=ORA_USER,password=ORA_PWD,dsn=ORA_DSN) as conn:
with conn.cursor() as cur:
r = cur.execute("select id from my_table where data = 'Hello'")
print(r)
if r is None:
print("Cannot retrieve ID")
sys.exit()
Unfortunately, the result set r is always "None" even though the value has been inserted properly (checked via sqldeveloper).
What am I doing wrong?
I even open a new connection to be sure to grab the value...
After calling execute() for a SELECT statement you need to call fetchone(), fetchmany() or fetchall() as shown in the cx_Oracle documentation SQL Queries.
Or you can use an iterator:
with connection.cursor() as cursor:
try:
sql = """select systimestamp from dual"""
for r in cursor.execute(sql):
print(r)
sql = """select 123 from dual"""
(c_id,) = cursor.execute(sql).fetchone()
print(c_id)
except oracledb.Error as e:
error, = e.args
print(sql)
print('*'.rjust(error.offset+1, ' '))
print(error.message)
However to get an automatically generated ID returned without the overhead of an additional SELECT, you can change the INSERT statement to use a RETURNING INTO clause. There is an example in the cx_Oracle documentation DML RETURNING Bind Variables that shows an UPDATE. You can use similar syntax with INSERT.
With the table:
CREATE TABLE mytable
(myid NUMBER(11) GENERATED BY DEFAULT ON NULL AS IDENTITY (START WITH 1),
mydata VARCHAR2(20));
You can insert and get the generated key like:
myidvar = cursor.var(int)
sql = "INSERT INTO mytable (mydata) VALUES ('abc') RETURNING myid INTO :bv"
cursor.execute(sql, bv=myidvar)
i, = myidvar.getvalue()
print(i)
If you just want a unique identifier you get the ROWID of an inserted row without needing a bind variable. Simple access cursor.lastrowid after executing an INSERT.
I want to insert a new row in my table by using the python-mariadb connector. For that I prefer to use the SET clause.
For some reason it does work if I only want to save ints (i.e y=2), but when I use a string, the following error occurs
Unknown column 'myString' in 'field list'
It seems it thinks the content of the string is a column name? Any idea how to fix that (I can do it with INSERT INTO ... VALUES ..., but I want to use the SET clause here). From my understanding, it should save both an int and a str without throwing an error
Thank you.
See the code example below
def myfunction():
x = 1
y ='myString'
db = connect_db()
cur = db.cursor()
sql = "INSERT INTO Table SET col1={}, col2={}"
cur.execute(sql.format(x, y))
db.commit()
db.close()
return
Here the MariaDB Connector, but this should be fine as it works for other db functions.
import mariadb
def connect_db():
db = mariadb.connect(
user="user",
password="123",
host="localhost",
port=3306,
database="DB"
)
db.autocommit = False
return db
you are not using right syntax for insert
sql = "INSERT INTO Table (col1,col2) values({}, {})"
but if you want to update an existing row:
sql = "UPDATE Table SET col1={}, col2={} WHERE id = {}"
and probably you need a where clause
The code in question produces the SQL statement:
INSERT INTO Table SET col1=1, col2=myString;
This is incorrect syntax, and strings must be in single-quotes:
INSERT INTO Table (col1, col2) VALUES (1, 'myString');
def myfunction():
x = 1
y ='myString'
db = connect_db()
cur = db.cursor()
sql = "INSERT INTO Table (col1, COL2) VALUES ({}, '{}')"
cur.execute(sql.format(x, y))
db.commit()
db.close()
return
But the above is fragile. Don't use string building methods to create SQL statements, it is much better to use parameter binding.
def myfunction():
x = 1
y ='myString'
db = connect_db()
cur = db.cursor()
sql = "INSERT INTO Table (col1, col2) VALUES (?, ?)"
cur.execute(sql, (x, y))
db.commit()
db.close()
return
The MariaDB connector documentation explains these things.
Retrieving Data
Once you have the initial code in place you can start working with the data. The first thing you should do is try to
retrieve information from the database. Here is code for a query
against the employees database:
cur.execute(
"SELECT first_name,last_name FROM employees WHERE first_name=?",
(some_name,))
MariaDB Connector/Python uses prepared statements, sanitizing and inserting the values from the tuple into the position
of the question marks (?). This is safer than inserting through
f-strings or format specifiers when working with user provided
information.
The query results are stored in a list in the cursor object. To view
the results, you can loop over the cursor.
Adding Data
Using the same execute() method with an INSERT statement, you can add rows to the table.
cursor.execute(
"INSERT INTO employees (first_name,last_name) VALUES (?, ?)",
(first_name, last_name))
I'm trying to fill in a SQLite database using the Python library sqlite3. In this example, the idea is simply to read data from a file and populate the database, however, although it seems the table is being created, the database is not growing and values are not written to it.
The code I have is below:
import sqlite3
def update_db(c, key, val):
sql = ''' UPDATE atable SET f1 = ? WHERE id = ?; '''
c.execute(sql, (val, key))
def create_table(c):
sql = ''' CREATE TABLE atable (id text PRIMARY KEY, f1 integer DEFAULT 0); '''
c.execute(sql)
with sqlite3.connect('test.db') as conn:
c = conn.cursor()
create_table(c)
with open('file1.txt') as f1:
for line in f1:
l = line.strip().split()
update_db(c, l[0], int(l[1]))
conn.commit()
This code runs without errors, but when trying to query this database, either with Python:
with sqlite3.connect('test.db') as conn:
c = conn.cursor()
c.execute('SELECT * FROM atable;')
for row in c:
print(row)
or in the SQLite command interface:
$ sqlite3 test.db
SQLite version 3.8.10.2 2015-05-20 18:17:19
Enter ".help" for usage hints.
sqlite> SELECT * FROM atable;
sqlite> .tables
atable
sqlite>
the output is always empty (but it looks the table was correctly create). What am I doing wrong here?
The 'file1.txt' for testing is this:
foo 2
bar 0
baz 1
The SQL syntax for adding rows to your table is:
INSERT INTO atable (id, f1) VALUES (?, ?);
UPDATE will do nothing if there are not already rows in the table.
If you want to insert or replace an existing row, sqlite also supports an INSERT OR REPLACE command.
I am trying to query MS SQL Server for a table.column, then insert this output into a sqlite table.
This example has one numeric column in the SQL Server source table.
I think I've almost got it by scouring the other answers.
Please let me know what I am missing.
import sqlite3
import pyodbc
#def connect_msss():
ODBC_Prod = ODBC_Prod
SQLSN = SQLSN
SQLpass = SQLpass
conn_str = ('DSN='+ODBC_Prod+';UID='+SQLSN+';PWD='+SQLpass)
conn = pyodbc.connect(conn_str)
#def connect_sqlite():
sl3Conn = sqlite3.connect('server_test.db')
c = sl3Conn.cursor()
c.execute('CREATE TABLE IF NOT EXISTS mrn_test (PTMRN NUMERIC)')
#def query_msss():
cur = conn.cursor()
cur.execute("SELECT TOP 50 PTMRN FROM dbo.atl_1234_mrntest")
rows = cur.fetchall()
for row in rows:
c.execute("INSERT INTO mrn_test VALUES (?)", row)
conn.commit()
#connect_msss()
#connect_sqlite()
#query_msss()
Error 1:
c.execute('CREATE TABLE IF NOT EXISTS mrn_test
(PTMRN NUMERIC)')
Out[117]: <sqlite3.Cursor at 0x2d1a742fc70>
Error 2:
cur = conn.cursor() cur.execute("SELECT TOP 50 PTMRN FROM
dbo.atl_1234_mrntest")
Out[118]: <pyodbc.Cursor at 0x2d1a731b990>
You're not committing the executed changes on the sqlite connection, after the c.execute step you're committing the MySQL DB connection. I think you need to replace conn.commit() at the end with sl3Conn.commit().
I apologize in advance for asking such a basic question but I am new to SQlite3 and having trouble starting. I am trying to build a database with one table. I used the following code to build a table.
import sqlite3
conn = sqlite3.connect('example.db')
c = conn.cursor()
c.execute('''CREATE TABLE mytable
(start, end, score)''')
but whenever I try to update or access the table it seems that it doesnt exist or maybe it exists in a different database. I also tried creating a table called example.mytable but I got the error:
sqlite3.OperationalError: unknown database example
What am I missing?
Thanks
I think that a commit is needed after inserts (schema changes such as new tables should automatically commit). I would suggest adding the full path to your database as well to make sure you are accessing the same location next time round.
Here is an extension on your code:
import sqlite3
def create():
try:
c.execute("""CREATE TABLE mytable
(start, end, score)""")
except:
pass
def insert():
c.execute("""INSERT INTO mytable (start, end, score)
values(1, 99, 123)""")
def select(verbose=True):
sql = "SELECT * FROM mytable"
recs = c.execute(sql)
if verbose:
for row in recs:
print row
db_path = r'C:\Users\Prosserc\Documents\Geocoding\test.db'
conn = sqlite3.connect(db_path)
c = conn.cursor()
create()
insert()
conn.commit() #commit needed
select()
c.close()
Output:
(1, 99, 123)
After closing the program if I log onto the SQLite database the data is still there.
import sqlite3;
import pandas as pd;
con=None
def getConnection():
databaseFile="./test.db"
global con
if con == None:
con=sqlite3.connect(databaseFile)
return con
def createTable(con):
try:
c = con.cursor()
c.execute("""CREATE TABLE IF NOT EXISTS Movie
(start, end, score)""")
except Exception as e:
pass
def insert(con):
c = con.cursor()
c.execute("""INSERT INTO Movie (start, end, score)
values(1, 99, 123)""")
def queryExec():
con=getConnection()
createTable(con)
insert(con)
# r = con.execute("""SELECT * FROM Movie""")
result=pd.read_sql_query("select * from Movie;",con)
return result
r = queryExec()
print(r)