How to retrieve inserted id after inserting row in SQLite using Python? I have table like this:
id INT AUTOINCREMENT PRIMARY KEY,
username VARCHAR(50),
password VARCHAR(50)
I insert a new row with example data username="test" and password="test". How do I retrieve the generated id in a transaction safe way? This is for a website solution, where two people may be inserting data at the same time. I know I can get the last read row, but I don't think that is transaction safe. Can somebody give me some advice?
You could use cursor.lastrowid (see "Optional DB API Extensions"):
connection=sqlite3.connect(':memory:')
cursor=connection.cursor()
cursor.execute('''CREATE TABLE foo (id integer primary key autoincrement ,
username varchar(50),
password varchar(50))''')
cursor.execute('INSERT INTO foo (username,password) VALUES (?,?)',
('test','test'))
print(cursor.lastrowid)
# 1
If two people are inserting at the same time, as long as they are using different cursors, cursor.lastrowid will return the id for the last row that cursor inserted:
cursor.execute('INSERT INTO foo (username,password) VALUES (?,?)',
('blah','blah'))
cursor2=connection.cursor()
cursor2.execute('INSERT INTO foo (username,password) VALUES (?,?)',
('blah','blah'))
print(cursor2.lastrowid)
# 3
print(cursor.lastrowid)
# 2
cursor.execute('INSERT INTO foo (id,username,password) VALUES (?,?,?)',
(100,'blah','blah'))
print(cursor.lastrowid)
# 100
Note that lastrowid returns None when you insert more than one row at a time with executemany:
cursor.executemany('INSERT INTO foo (username,password) VALUES (?,?)',
(('baz','bar'),('bing','bop')))
print(cursor.lastrowid)
# None
All credits to #Martijn Pieters in the comments:
You can use the function last_insert_rowid():
The last_insert_rowid() function returns the ROWID of the last row insert from the database connection which invoked the function. The last_insert_rowid() SQL function is a wrapper around the sqlite3_last_insert_rowid() C/C++ interface function.
SQLite 3.35's RETURNING clause:
CREATE TABLE users (
id INTEGER PRIMARY KEY,
first_name TEXT,
last_name TEXT
);
INSERT INTO users (first_name, last_name)
VALUES ('Jane', 'Doe')
RETURNING id;
returns requested columns of the inserted row in INSERT, UPDATE and DELETE statements. Python usage:
cursor.execute('INSERT INTO users (first_name, last_name) VALUES (?,?)'
' RETURNING id',
('Jane', 'Doe'))
row = cursor.fetchone()
(inserted_id, ) = row if row else None
Related
I am working with MySQL but I have some unexpected behaviour.
I have past experience with SQLite but I guess I am missing something here.
Using the query SELECT * FROM tableName I would expect the content of the table to be the output.
Instead I get an int, being the count of rows in the table.
Here is the piece of code I am using.
import MySQLdb
conn=MySQLdb.connect(host="xxx",user="xxx",passwd="xxx")
cursor = conn.cursor()
cursor.execute("create database if not exists Test;")
cursor.execute("use Test;")
cursor.execute("create table if not exists City (id int not null primary key auto_increment, city varchar(50), unique(city));")
cursor.execute("insert into City (city) values ('Firenze');")
cursor.execute("insert into City (city) values ('Roma');")
conn.commit()
print(cursor.execute("select city from City;"))
I would expect to get:
Firenze
Roma
Instead I get:
2
If I run the same query from a SQL client I get the expected output. Any clever idea?
Thanks :)
You are missing the FetchAll() function in your code.
Fetch all is nothing but fetching the data of last executed statement.
import MySQLdb
conn=MySQLdb.connect(host="xxx",user="xxx",passwd="xxx")
cursor = conn.cursor()
cursor.execute("create database if not exists Test;")
cursor.execute("use Test;")
cursor.execute("create table if not exists City (id int not null primary key
auto_increment, city varchar(50), unique(city));")
cursor.execute("insert into City (city) values ('Firenze');")
cursor.execute("insert into City (city) values ('Roma');")
conn.commit()
print(cursor.execute("select city from City;"))
myresult = mycursor.fetchall()
for x in myresult:
print(x)
The thing is print(cursor.execute("select city from City;") returns you the number of rows or rows count.
For the complete records use something like this
myresult = cursor.fetchall()
for x in myresult:
print(x)
I have that query in a python program:
And i should create a multidimensional array (if it possible) or four arrays from this query for every column from the query.
Can you suggest an elegant way to solve it?
conn = #connection to the server
cursor=conn.cursor()
query = (" select id, name, phone, city from guest")
cursor.execute(query)
results = cursor.fetchall
for i in results:
print i
cursor.close()
conn.close()
Not elegant but it may assist to unravel the mysterious Python Connector Cursor Class and transfers the list of tuples (see Copperfield comment) with the data from the query, into a list (phoneList) of dictionaries (entries) with details of each entry in the database, that might be easier to work with in your python script:
# ref: https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursor.html
import mysql.connector
db = 'test'
table = 'phonebook'
phoneList = []
drop_table = ("DROP TABLE IF EXISTS {};").format(table)
# By default, the starting value for AUTO_INCREMENT is 1, and it will increment by 1 for each new record.
# To let the AUTO_INCREMENT sequence start with another value, use the following SQL statement:
# ALTER TABLE phonebook AUTO_INCREMENT=100;
create_table = ("CREATE TABLE {} ("
"id int NOT NULL AUTO_INCREMENT,"
"name varchar(30) NOT NULL,"
"phone varchar(30) NOT NULL,"
"city varchar(30) NOT NULL,"
"PRIMARY KEY (id))"
" ENGINE=InnoDB DEFAULT CHARSET=latin1;").format(table)
Names = {'Bill':{'phone':'55123123','city':'Melbourne'},
'Mary':{'phone':'77111123','city':'Sydney'},
'Sue':{'phone':'55888123','city':'Melbourne'},
'Harry':{'phone':'77777123','city':'Sydney'},
'Fred':{'phone':'88123444','city':'Yongala'},
'Peter':{'phone':'55999123','city':'Melbourne'}}
cnx = mysql.connector.connect(user='mysqluser', password='xxxx',host='127.0.0.1',database=db)
cursor = cnx.cursor(dictionary=True) # key to using **row format
cursor.execute(drop_table)
cursor.execute(create_table)
# populate db
for name,detail in dict.items(Names):
sql = ("INSERT INTO {} (name,phone,city) VALUES ('{}','{}','{}')".format(table,name,detail['phone'],detail['city']))
cursor.execute(sql)
sql = ("SELECT id,name,phone,city FROM {}".format(table))
cursor.execute(sql)
for row in cursor:
print("{id} {name} {phone} {city}".format(**row))
phoneList.append(row)
print phoneList[0]['name'],phoneList[0]['city']
print phoneList[3]['name'],phoneList[3]['phone']
for entries in phoneList: # list of dictionaries
print entries['name'],entries
for entries in phoneList:
for k,v in dict.items(entries):
print k,v
print "\n"
cnx.close()
I'm having a bit of trouble trying to fix a problem I'm having in retrieving the last insert id from a query in SQLite3 using Python.
Here's a sample of my code:
import sqlite3
# Setup our SQLite Database
conn = sqlite3.connect('value_serve.db')
conn.execute("PRAGMA foreign_keys = 1") # Enable Foreign Keys
cursor = conn.cursor()
# Create table for Categories
conn.executescript('DROP TABLE IF EXISTS Category;')
conn.execute('''CREATE TABLE Category (
id INTEGER PRIMARY KEY AUTOINCREMENT,
category CHAR(132),
description TEXT,
parent_id INT,
FOREIGN KEY (parent_id) REFERENCES Category (id)
);''')
conn.execute("INSERT INTO Category (category, parent_id) VALUES ('Food', NULL)")
food_category = cursor.lastrowid
conn.execute("INSERT INTO Category (category, parent_id) VALUES ('Beverage', NULL)")
beverage_category = cursor.lastrowid
...
conn.commit() # Commit to Database
No matter what I do, when I try to get the value of 'food_category' I get a return value of 'None'.
Any help would be appreciated, thanks in advance.
The lastrowid value is set per cursor, and only visible to that cursor.
You need to execute your query on the cursor that executed the query to get the last row id. You are asking an arbitrary cursor, one that never actually is used to execute the query for a last row id, but that cursor can't know that value.
If you actually execute the query on the cursor object, it works:
cursor.execute("INSERT INTO Category (category, parent_id) VALUES ('Food', NULL)")
food_category = cursor.lastrowid
The connection.execute() function creates a new (local) cursor for that query and the last row id is only visible on that local cursor. That cursor is returned when you use connection.execute(), so you could get the same value from that return value:
cursor_used = conn.execute("INSERT INTO Category (category, parent_id) VALUES ('Food', NULL)")
food_category = cursor_used.lastrowid
I am able to get my Python code to run print the desired results, but my problem is with the SQLite table. I was asked to apply this SQL command to the tables:
SELECT hex(User.name || Course.title || Member.role ) AS X
FROM User JOIN Member JOIN Course
ON User.id = Member.user_id AND Member.course_id = Course.id
ORDER BY X
I was able to execute the command in SQLite, but according to the instructions for this project, X is supposed to start with 416 in row one of the results column produced. However, the X I got for row 1 in the results was:
43616C6962736933313030
Here is what I wrote in Python so far:
import sqlite3
import json
#Working with Java and Sqlite
conn = sqlite3.connect('rosterdb.sqlite')
cur = conn.cursor()
cur.executescript('''
DROP TABLE IF EXISTS User;
DROP TABLE IF EXISTS Member;
DROP TABLE IF EXISTS Course;
CREATE TABLE User(
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT UNIQUE,
name TEXT UNIQUE
);
CREATE TABLE Member(
user_id INTEGER UNIQUE,
course_id INTEGER UNIQUE,
role INTEGER,
PRIMARY KEY (user_id, course_id)
);
CREATE TABLE Course(
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT UNIQUE,
title TEXT UNIQUE
);
''')
#primary key for junction table is composite of both user_id and course_id
fname = raw_input("Enter file name:")
if (len(fname) < 1): fname = 'roster_data.json'
#prompts for file name
str_data = open(fname).read()
json_data = json.loads(str_data)
#opens the file and reads it all
#loads the json data and now is a python list
for entry in json_data:
title = entry[1];
name = entry [0];
role = entry[2];
#["Charley, "sill0", 1] represents the name, course title, and role
print name, title, role
cur.execute('''INSERT or IGNORE INTO User (name)
VALUES (?)''', (name, ))
cur.execute('SELECT id FROM User WHERE name = ?',(name, ))
user_id = cur.fetchone()[0]
cur.execute('''INSERT or IGNORE INTO Course (title)
VALUES (?)''', (title, ))
cur.execute('SELECT id FROM Course WHERE title = ?', (title, ))
course_id = cur.fetchone()[0]
cur.execute('''INSERT or REPLACE INTO Member (user_id, course_id, role)
VALUES (?,?,?)''', (user_id, course_id, role))
#INSERT, SELECT AND FETCHONE STATEMENTS
conn.commit()
Here is the JSON data that I was working with. It is about course registration for students: roster_data.json Here is the link to it:
https://pr4e.dr-chuck.com/tsugi/mod/sql-intro/roster_data.php?PHPSESSID=9addd537cfe55c03585d2bfaa757f6b0
I am not sure if I implemented the "role" key correctly. Thank you for your inputs!
The problem is that you made Member.course_id unique. Thus you can have no more members than courses. Using REPLACE in INSERT or REPLACE into Member hides this error.
Just drop UNIQUE constraint on Member.course and you will get expected result.
I got this code to save some values in a Sql-DB via python. But it doesn't work. Could you please help me to find the problem?
def saveValue(art, value, timestamp):
# Missing: Check if table setting exists or create it
# First check if row exists
cursor.execute('SELECT count(*) FROM setting WHERE parameter=%s', art)
existing = cursor.fetchone()
if existing[0]==0:
cursor.execute('INSERT INTO setting (parameter, wert) VALUES (%s, NULL)', art)
db.commit()
# Update row
cursor.execute('UPDATE setting set value=%s WHERE parameter=%s', value, art)
db.commit()
# Check if measure-table exisits or create ist
cursor.execute('SHOW TABLES LIKE measure_%s', art)
result = cursor.fetchone()
if not result:
cursor.execute('CREATE TABLE measure_%s (id INT NOT NULL AUTO_INCREMENT, time int(11), value float(6), PRIMARY KEY (id))', art)
db.commit()
# Insert measure value
sql = 'INSERT INTO measure_%s (time, value) VALUES (%s, %s)'
args= art, timestamp, value
cursor.execute(sql,args)
db.commit()
Maybe it is even possible to optimize/minimize the code?