I am using sqlite, and I have a Python code as follows:
...
cur.execute("insert or ignore into books (title, authors, ...) \
values (:title, :authors, ..."), locals())
...
bookId = cur.lastrowid
If the ignore part of the select statement applies
then the value of cur.lastrowid is 0.
But this is not what I want. I'd like to get books.id value from the database
in any case.
Should I use select statement or is there smarter way to achieve it?
My temporary solution:
if bookId == 0:
cur.execute("select id from books where \
title = :title and authors = :authors", locals())
bookId = cur.fetchone()[0]
There is no better way.
To read some existing value from the database, there's only SELECT.
Try using on duplicate key update id=LAST_INSERT_ID(id) for the PK. Seems like lastrowID will be correct from then on out.
Related
I'm aiming to automate the insertion of book metadata in a mysql database.
I'm working on creating a dictionary based on a set of authors, so I can later replace, with the dictionary, the author by its author_id in every book object having this particular author in its author field. This way, my data will be prepared for its insertion in the database.
My example set being
authors_set = {'Maurice Blanchot', 'Paul Celan', 'Jean-Pierre Martinet', 'Yves Citton'}
I'm trying this:
AUTHOR_EXISTENCE = "SELECT id, full_name FROM authors WHERE full_name = %s"
def create_dict(iterable, query):
dictionary = {}
mysql_db = ms.connect(**mysql_conn_params)
cur = mysql_db.cursor(buffered=True)
for author in iterable:
if cur.execute(query, [author]):
for id, value in cur:
dictionary[value] = id
else:
dictionary[author] = author
return dictionary
authors_dict = create_dict(authors_set, AUTHOR_EXISTENCE)
Which never returns anything, although I know at least one author should be in the dictionary. I don't see what I'm doing different from the MySql documentation.
EDIT:
When I am printing cur after cur.execute(query, [author]), my terminal prints:
MySQLCursorBuffered: SELECT id, full_name FROM authors WHERE ..
I guess I should see the authors name at the end. My code doesn't look wrong to my eye though. Must be somehow.
END EDIT
But I have tried something which worked each time I'm searching for one author only:
mysql_db = ms.connect(**mysql_conn_params)
cur = mysql_db.cursor(buffered=True)
cur.execute(AUTHOR_EXISTENCE, ['Yves Citton'])
for id, value in cur:
print(id, value)
And it returns : 2 Yves Citton, like it should.
Somehow neither the nice MySql documentation, my searches nor my knowledge gave me a clue about the solution.
Could anyone help me ?
Thank you for your attention
So I found a solution, even if I can't quite tell the difference : I wrote the "execute" part with a tuple for value instead of a list, and I deleted the else clause because I'm creating the dictionary after all authors are inserted in the database.
def create_dictionary(query, iterable):
dictionary = {}
connexion = ms.connect(**mysql_conn_params)
cursor = connexion.cursor(buffered=True)
for value in iterable:
cursor.execute(query, (value, )) #change is here
for id, name in cursor:
dictionary[name] = id
connexion.close()
return dictionary
I have 3 MariaDB-SQL tables and I want to insert some data:
Restaurant
ID_Restaurant (Primary Key, Auto Increment)
RestaurantName
Location
Restaurant_has_Request
ID_Restaurant(Foreign Key)
ID_Request (Foreign Key)
Request
ID_Request (Primary Key, Auto Increment)
Date
Adults
One Restaurant has 0 or infinity Requests. One Request can have 1 or infinity Restaurants.
I am iterating through a request site for only one Request. That means I want to treat first one Request and then I save the related one or more Restaurants. After I have done this another Request will be treated and so on.
I have the following Python Code for inserting the data:
cursor.execute('insert into Restaurant(RestaurantName, Location) values(%s, %s)',(RestaurantName, Location))
# ID from last insert
ID_Restaurant_Cache = cursor.lastrowid
cursor.execute('insert into Request(Date, Adults) values(%s, %s)',(Date, Adults))
# ID from last insert
ID_Request_Cache = cursor.lastrowid
cursor.execute('insert into Restaurant_has_Request(ID_Restaurant, ID_Request) values(%s, %s)',(ID_Restaurant_Cache, ID_Request_Cache))
The problem is that I have still duplicates... How can I modify the Python code that I use the existing entry from a Restaurant, when it already exists (RestaurantName & Location is already in the database)?
I have also duplicates for the Request. I want to use the same Request ID for one iteration and then I want to use another Request ID.
Thank you :)
You need to check whether the restaurant already exists before inserting.
cursor.execute('select ID_Restaurant from Restaurant where RestaurantName = %s', (RestaurantName,))
row = cursor.fetchone()
if row:
ID_Restaurant_Cache = row[0]
else:
cursor.execute('insert into Restaurant(RestaurantName, Location) values(%s, %s)',(RestaurantName, Location))
ID_Restaurant_Cache = cursor.lastrowid
You should also add a unique index on the RestaurantName column, to prevent duplicates.
Please provide SHOW CREATE TABLE.
Shouldn't Restaurant_has_Request have
PRIMARY KEY(ID_restaurant, ID_request),
INDEX(ID_request, ID_restaurant)
I'd like to have returned to me (via cx_oracle in python) the value of the Identity that's created for a row that I'm inserting. I think I can figure out the python bit on my own, if someone could please state how to modify my SQL statement to get the ID of the newly-created row.
I have a table that's created with something like the following:
CREATE TABLE hypervisor
(
id NUMBER GENERATED BY DEFAULT AS IDENTITY (
START WITH 1 NOCACHE ORDER ) NOT NULL ,
name VARCHAR2 (50)
)
LOGGING ;
ALTER TABLE hypervisor ADD CONSTRAINT hypervisor_PK PRIMARY KEY ( id ) ;
And I have SQL that's similar to the following:
insert into hypervisor ( name ) values ('my hypervisor')
Is there an easy way to obtain the id of the newly inserted row? I'm happy to modify my SQL statement to have it returned, if that's possible.
Most of the google hits on this issue were for version 11 and below, which don't support automatically-generated identity columns so hopefully someone here can help out.
Taking what user2502422 said above and adding the python bit:
newest_id_wrapper = cursor.var(cx_Oracle.STRING)
sql_params = { "newest_id_sql_param" : newest_id_wrapper }
sql = "insert into hypervisor ( name ) values ('my hypervisor') " + \
"returning id into :python_var"
cursor.execute(sql, sql_params)
newest_id=newest_id_wrapper.getvalue()
This example taken from learncodeshare.net has helped me grasp the correct syntax.
cur = con.cursor()
new_id = cur.var(cx_Oracle.NUMBER)
statement = 'insert into cx_people(name, age, notes) values (:1, :2, :3) returning id into :4'
cur.execute(statement, ('Sandy', 31, 'I like horses', new_id))
sandy_id = new_id.getvalue()
pet_statement = 'insert into cx_pets (name, owner, type) values (:1, :2, :3)'
cur.execute(pet_statement, ('Big Red', sandy_id, 'horse'))
con.commit()
It's only slightly different from ragerdl's answer, but different enough to be added here I believe!
Notice the absence of sql_params = { "newest_id_sql_param" : newest_id_wrapper }
Use the returning clause of the insert statement.
insert into hypervisor (name ) values ('my hypervisor')
returning id into :python_var
You said you could handle the Python bit ? You should be able to "bind" the return parameter in your program.
I liked the answer by Marco Polo, but it is incomplete.
The answer from FelDev is good too but does not address named parameters.
Here is a more complete example from code I wrote with a simplified table (less fields). I have omitted code on how to set up a cursor since that is well documented elsewhere.
import cx_Oracle
INSERT_A_LOG = '''INSERT INTO A_LOG(A_KEY, REGION, DIR_NAME, FILENAME)
VALUES(A_KEY_Sequence.nextval, :REGION, :DIR_NAME, :FILENAME)
RETURNING A_KEY INTO :A_LOG_ID'''
CURSOR = None
class DataProcessor(Process):
# Other code for setting up connection to DB and storing it in CURSOR
def save_log_entry(self, row):
global CURSOR
# Oracle variable to hold value of last insert
log_var = CURSOR.var(cx_Oracle.NUMBER)
row['A_LOG_ID'] = log_var
row['REGION'] = 'R7' # Other entries set elsewhere
try:
# This will fail unless row.keys() =
# ['REGION', 'DIR_NAME', 'FILE_NAME', 'A_LOG_ID']
CURSOR.execute(INSERT_A_LOG, row)
except Exception as e:
row['REJCTN_CD'] = 'InsertFailed'
raise
# Get last inserted ID from Oracle for update
self.last_log_id = log_var.getvalue()
print('Insert id was {}'.format(self.last_log_id))
Agreeing with the older answers. However, depending on your version of cx_Oracle (7.0 and newer), var.getvalue() might return an array instead of a scalar.
This is to support multiple return values as stated in this comment.
Also note, that cx_Oracle is deprecated and has moved to oracledb now.
Example:
newId = cur.var(oracledb.NUMBER, outconverter=int)
sql = """insert into Locations(latitude, longitude) values (:latitude, :longitude) returning locationId into :newId"""
sqlParam = [latitude, longitude, newId]
cur.execute(sql, sqlParam)
newIdValue = newId.getvalue()
newIdValue would return [1] instead of 1
I am writing a program in which two variables are selected from QCombobBoxes which are populated with results from a MySQL query. I then take these variable and insert them into a MySQLdb statement that inserts the variables into a different MySQL table. The first variable works fine, however on the second I get this error,
TypeError: 'NoneType' object has no attribute '__getitem__'
The code is identical for both variables, with the exception of different names
name = str(self.item_name.currentText())
cur.execute("SELECT item_id FROM Items WHERE name = '%s';"), name
db.commit()
results = cur.fetchone()
item_name = results[0]
personnel_name = str(self.purchaser_name.currentText())
cur.execute("SELECT personnel_id FROM Personnel WHERE name = '%s';"), personnel_name
db.commit()
results = cur.fetchone()
purchaser_id = results[0]
After playing with it, it looks like cur.execute("SELECT item_id FROM Items WHERE name = '%s';"), name is inserting an extra pair of quotation marks around the value that replaces %s Does anyone know why it's doing this and how to stop it? I coded both variables exactly the same, and it seems that name is getting an extra pair of quotes from MySQL
This is code that populates QComboBox:
#Get list of items currently in the database
cur = db.cursor()
cur.execute("SELECT name FROM Items")
db.commit()
results = cur.fetchall()
for name in results:
self.item_name.addItem(name[0])
#Get list of purchaser names
cur.execute("SELECT name FROM Personnel")
db.commit()
results = cur.fetchall()
for name in results:
self.purchaser_name.addItem(name[0])
If I manually insert a variable, it works fine. ex: cur.execute("SELECT item_id FROM Items WHERE name = 'Wire';") Only when I use string formatting with %s does the error occurr.
c.execute("SELECT * FROM sometable WHERE some_condition=?","My Condition")
you should always use the ? placeholders for this kind of thing
[edit]
try 1. cur.execute("SELECT item_id FROM Items WHERE name = '%s';"%(name,))
or 2. cur.execute("SELECT item_id FROM Items WHERE name = %s;", (name,))
from my brief reading, I think that mysql driver will automatically quote %s arguments
my conclusion is that cur.execute("SELECT item_id FROM Items WHERE name = %s;", (name,)) is the most correct way to do this(to avoid injection etc).
I am using SQLAlchemy without the ORM, i.e. using hand-crafted SQL statements to directly interact with the backend database. I am using PG as my backend database (psycopg2 as DB driver) in this instance - I don't know if that affects the answer.
I have statements like this,for brevity, assume that conn is a valid connection to the database:
conn.execute("INSERT INTO user (name, country_id) VALUES ('Homer', 123)")
Assume also that the user table consists of the columns (id [SERIAL PRIMARY KEY], name, country_id)
How may I obtain the id of the new user, ideally, without hitting the database again?
You might be able to use the RETURNING clause of the INSERT statement like this:
result = conn.execute("INSERT INTO user (name, country_id) VALUES ('Homer', 123)
RETURNING *")
If you only want the resulting id:
result = conn.execute("INSERT INTO user (name, country_id) VALUES ('Homer', 123)
RETURNING id")
[new_id] = result.fetchone()
User lastrowid
result = conn.execute("INSERT INTO user (name, country_id) VALUES ('Homer', 123)")
result.lastrowid
Current SQLAlchemy documentation suggests
result.inserted_primary_key should work!
Python + SQLAlchemy
after commit, you get the primary_key column id (autoincremeted) updated in your object.
db.session.add(new_usr)
db.session.commit() #will insert the new_usr data into database AND retrieve id
idd = new_usr.usrID # usrID is the autoincremented primary_key column.
return jsonify(idd),201 #usrID = 12, correct id from table User in Database.
this question has been asked many times on stackoverflow and no answer I have seen is comprehensive. Googling 'sqlalchemy insert get id of new row' brings up a lot of them.
There are three levels to SQLAlchemy.
Top: the ORM.
Middle: Database abstraction (DBA) with Table classes etc.
Bottom: SQL using the text function.
To an OO programmer the ORM level looks natural, but to a database programmer it looks ugly and the ORM gets in the way. The DBA layer is an OK compromise. The SQL layer looks natural to database programmers and would look alien to an OO-only programmer.
Each level has it own syntax, similar but different enough to be frustrating. On top of this there is almost too much documentation online, very hard to find the answer.
I will describe how to get the inserted id AT THE SQL LAYER for the RDBMS I use.
Table: User(user_id integer primary autoincrement key, user_name string)
conn: Is a Connection obtained within SQLAlchemy to the DBMS you are using.
SQLite
======
insstmt = text(
'''INSERT INTO user (user_name)
VALUES (:usernm) ''' )
# Execute within a transaction (optional)
txn = conn.begin()
result = conn.execute(insstmt, usernm='Jane Doe')
# The id!
recid = result.lastrowid
txn.commit()
MS SQL Server
=============
insstmt = text(
'''INSERT INTO user (user_name)
OUTPUT inserted.record_id
VALUES (:usernm) ''' )
txn = conn.begin()
result = conn.execute(insstmt, usernm='Jane Doe')
# The id!
recid = result.fetchone()[0]
txn.commit()
MariaDB/MySQL
=============
insstmt = text(
'''INSERT INTO user (user_name)
VALUES (:usernm) ''' )
txn = conn.begin()
result = conn.execute(insstmt, usernm='Jane Doe')
# The id!
recid = conn.execute(text('SELECT LAST_INSERT_ID()')).fetchone()[0]
txn.commit()
Postgres
========
insstmt = text(
'''INSERT INTO user (user_name)
VALUES (:usernm)
RETURNING user_id ''' )
txn = conn.begin()
result = conn.execute(insstmt, usernm='Jane Doe')
# The id!
recid = result.fetchone()[0]
txn.commit()
result.inserted_primary_key
Worked for me. The only thing to note is that this returns a list that contains that last_insert_id.
Make sure you use fetchrow/fetch to receive the returning object
insert_stmt = user.insert().values(name="homer", country_id="123").returning(user.c.id)
row_id = await conn.fetchrow(insert_stmt)
For Postgress inserts from python code is simple to use "RETURNING" keyword with the "col_id" (name of the column which you want to get the last inserted row id) in insert statement at end
syntax -
from sqlalchemy import create_engine
conn_string = "postgresql://USERNAME:PSWD#HOSTNAME/DATABASE_NAME"
db = create_engine(conn_string)
conn = db.connect()
INSERT INTO emp_table (col_id, Name ,Age)
VALUES(3,'xyz',30) RETURNING col_id;
or
(if col_id column is auto increment)
insert_sql = (INSERT INTO emp_table (Name ,Age)
VALUES('xyz',30) RETURNING col_id;)
result = conn.execute(insert_sql)
[last_row_id] = result.fetchone()
print(last_row_id)
#output = 3
ex -