We Keep Coding

MySql & Python problem : author found one by one but not inside function

I'm aiming to automate the insertion of book metadata in a mysql database.
I'm working on creating a dictionary based on a set of authors, so I can later replace, with the dictionary, the author by its author_id in every book object having this particular author in its author field. This way, my data will be prepared for its insertion in the database.
My example set being
authors_set = {'Maurice Blanchot', 'Paul Celan', 'Jean-Pierre Martinet', 'Yves Citton'}
I'm trying this:
AUTHOR_EXISTENCE = "SELECT id, full_name FROM authors WHERE full_name = %s"
def create_dict(iterable, query):
dictionary = {}
mysql_db = ms.connect(**mysql_conn_params)
cur = mysql_db.cursor(buffered=True)
for author in iterable:
if cur.execute(query, [author]):
for id, value in cur:
dictionary[value] = id
else:
dictionary[author] = author
return dictionary
authors_dict = create_dict(authors_set, AUTHOR_EXISTENCE)
Which never returns anything, although I know at least one author should be in the dictionary. I don't see what I'm doing different from the MySql documentation.
EDIT:
When I am printing cur after cur.execute(query, [author]), my terminal prints:
MySQLCursorBuffered: SELECT id, full_name FROM authors WHERE ..
I guess I should see the authors name at the end. My code doesn't look wrong to my eye though. Must be somehow.
END EDIT
But I have tried something which worked each time I'm searching for one author only:
mysql_db = ms.connect(**mysql_conn_params)
cur = mysql_db.cursor(buffered=True)
cur.execute(AUTHOR_EXISTENCE, ['Yves Citton'])
for id, value in cur:
print(id, value)
And it returns : 2 Yves Citton, like it should.
Somehow neither the nice MySql documentation, my searches nor my knowledge gave me a clue about the solution.
Could anyone help me ?
Thank you for your attention

So I found a solution, even if I can't quite tell the difference : I wrote the "execute" part with a tuple for value instead of a list, and I deleted the else clause because I'm creating the dictionary after all authors are inserted in the database.
def create_dictionary(query, iterable):
dictionary = {}
connexion = ms.connect(**mysql_conn_params)
cursor = connexion.cursor(buffered=True)
for value in iterable:
cursor.execute(query, (value, )) #change is here
for id, name in cursor:
dictionary[name] = id
connexion.close()
return dictionary

Python Sqlite UPDATE multiple values

Is there a way to do these two updates in a single instruction?
cur.execute("UPDATE table_name1 SET email = 'foo#bar.com' WHERE id = 4")
cur.execute("UPDATE table_name1 SET phone = '0400-123-456' WHERE id = 4")
I've tried all sort of variations but can't get it to work.
Edit: I want to pass email, phone and I'd as parameters.

You're solution opens you up to SQL injections. If you read the first section of the documentation, it specifically says not to do it the way you are proposing:
Never do this -- insecure!
symbol = 'RHAT'
c.execute("SELECT * FROM stocks WHERE symbol = '%s'" % symbol)
Do this instead
t = ('RHAT',)
c.execute('SELECT * FROM stocks WHERE symbol=?', t)
So you should change your code to something along the following lines:
conn = sqlite3.connect('connex.db')
cur = conn.cursor()
mobileval = '0400-123-456'
emailval = 'foo#bar.com'
constrain = 4
q = "UPDATE licontacts310317 SET (?, ?)
WHERE (?)=(?)"
cur.execute(q, (liemailval, limobileval, id, constrain) )
conn.commit()
conn.close()
I haven't tested it, but hopefully you get the idea =)

The following works: Its just standard SQL at this point.
cur.execute("""UPDATE table_name1
SET email = 'foo#bar.com', phone = '0400-123-456'
WHERE id = 4""")

I was facing a similar issue with my own code and was able to get my code working using the following:
cur.execute("UPDATE licontacts310317 SET liemail=?, limobile=? WHERE id=? ", (liemailval, limobileval, constrain))
Someone else already commented this, but it's better to use the ? placeholder and not the string formatting operations because those leave your db vulnerable to SQL injection attacks (basically, hackers).

OK. I made a solution that works with parameters.
First thanks to David for his original answer. It had a small syntax error (corrected in the comments for that answer) but it was enough to help me work out how to get it working without parametising.
(Note:I think David posted his reply before I edited the question to add the need to working with parameters.)
Then this answer helped me parametise the solution.
Here is my solution to the question. I'm poting it in case someone else has the same problem because I did quite a bit of searching before posting the original question and couldn't find the answer.
conn = sqlite3.connect('connex.db')
cur = conn.cursor()
mobileval = '0400-123-456'
emailval = 'foo#bar.com'
constrain = 4
cur.execute("UPDATE licontacts310317 SET liemail=%s, limobile=%s
WHERE %s=?" % (liemailval, limobileval, id), (constrain,))
conn.commit()
conn.close()

Use Dictionaries!
They seem to work well:
cur.execute(
"""UPDATE table_name1
SET email =:email,
phone =:phone
WHERE id = 4
""",
{"email": "foo#bar.com", "phone": '0400-123-456'}
)
So you can just post a dictionary in like so, provided they contain the keys:
cur.execute(
"""UPDATE table_name1
SET email =:email,
phone =:phone
WHERE id = 4
""",
the_dictionary
)
Where the_dictionary = {"email": "foo#bar.com", "phone": "0400-123-456"}. You can put in as many as you'd like. This seems more readable as well I feel.

Why does Psycopg2 return list of tuples in with Stored Procedure?

I have been using Psycopg2 to read stored procedures from Postgres successfully and getting a nice tuple returned, which has been easy to deal with. For example...
def authenticate(user, password):
conn = psycopg2.connect("dbname=MyDB host=localhost port=5433 user=postgres password=mypwd")
cur = conn.cursor()
retrieved_pwd = None
retrieved_userid = None
retrieved_user = None
retrieved_teamname = None
cur.execute("""
select "email", "password", "userid", "teamname"
from "RegisteredUsers"
where "email" = '%s'
""" % user)
for row in cur:
print row
The row that prints would give me ('user#gmail.com ', '84894531656894hashedpassword5161651165 ', 36, 'test ')
However, when I run the following code to read a row of fixtures with a Stored Procedure, I get (what looks to me like) an unholy mess.
def get_from_sql(userid):
conn = psycopg2.connect("dbname=MyDB host=localhost port=5433 user=postgres password=pwd")
fixture_cursor = conn.cursor()
callproc_params = [userid]
fixture_cursor.execute("select sppresentedfixtures(%s)", callproc_params)
for row in fixture_cursor:
print row
The resulting output:
('(5,"2015-08-28 21:00:00","2015-08-20 08:00:00","2015-08-25 17:00:00","Team ",,"Team ",,"Final ")',)
I have researched the cursor class and cannot understand why it outputs like this for a stored procedure. When executing within Postgres, the output is in a perfect Tuple. Using Psycopg2 adds onto the tuple and I don't understand why?
How do I change this so I get a tidy tuple? What am I not understanding about the request that I am making that gives me this result?
I have tried the callproc function and get an equally unhelpful output. Any thoughts on this would be great.

This is because you're SELECTing the result of the function directly. Your function returns a set of things, and each "thing" happens to be a tuple, so you're getting a list of stringified tuples back. What you want is this:
SELECT * FROM sppresentedfixtures(...)
But this doesn't work, because you'll get the error:
ERROR: a column definition list is required for functions returning "record"
The solution is to return a table instead:
CREATE OR REPLACE FUNCTION sppresentedfixtures(useridentity integer) RETURNS TABLE(
Fixture_No int,
Fixture_Date timestamp,
...
) AS
$BODY$
select
"Fixtures"."Fixture_No",
"Fixtures"."Fixture_Date",
...
from "Fixtures" ...
$BODY$ LANGUAGE sql

PyODBC fetch result from a simple SELECT clause

I have a SQL script like follows:
DECLARE #AGE INT = ?
, #NAME VARCHAR(20) = ?
INSERT INTO [dbo].[new_table] (AGE, NAME)
SELECT #AGE, #NAME
SELECT ID = CAST(SCOPE_IDENTITY() AS INT)
In the table, there is an IDENTITY column ID which is defined as INT. Thus the values of the ID column is increasing as new rows are inserted. And my goal is to take out the ID that just inserted.
The above code works fine in SQL.
Then I tried to run it in python, using following code:
conn = pyodbc.connect("driver={SQL Server}; server= MyServer; database= MyDatabase"; trusted_connection=true")
cursor = conn .cursor()
SQL_command = """
DECLARE #AGE INT = ?
, #NAME VARCHAR(20) = ?
INSERT INTO [dbo].[new_table] (AGE, NAME)
SELECT #AGE, #NAME
SELECT ID = CAST(SCOPE_IDENTITY() AS INT)
"""
cursor.execute(SQL_command, 23, 'TOM')
result = cursor.fetchall()
However, I've got following error message:
Traceback (most recent call last):
File "C:\Users\wwang\Documents\Aptana Studio 3 Workspace\ComparablesBuilder\test.py", line 119, in
result = cursor.fetchall()
pyodbc.ProgrammingError: No results. Previous SQL was not a query.
So, may I know why the same code cannot work in python? Is my usage of pyodbc is incorrect?
Many thanks.

It's possible that the multiple statements in your Sql Batch are being interpreted as separate result sets to the Python driver - the first row-count returning statement is the INSERT statement, which could be the culprit.
Try adding SET NOCOUNT ON; before your statements, to suppress row counts from non-queries:
SET NOCOUNT ON;
DECLARE #AGE INT = ?
, #NAME VARCHAR(20) = ?
...
SELECT ID = CAST(SCOPE_IDENTITY() AS INT);
Edit
IIRC, some drivers are also dependent on Sql Server's ROW COUNT to parse result sets correctly. So if the above fails, you might also try:
SET NOCOUNT ON;
DECLARE ...;
INSERT ...;
SET NOCOUNT OFF; -- i.e. Turn back on again so the 1 row count can be returned.
SELECT CAST(SCOPE_IDENTITY() AS INT) AS ID;

Getting the id of the last record inserted for Postgresql SERIAL KEY with Python

I am using SQLAlchemy without the ORM, i.e. using hand-crafted SQL statements to directly interact with the backend database. I am using PG as my backend database (psycopg2 as DB driver) in this instance - I don't know if that affects the answer.
I have statements like this,for brevity, assume that conn is a valid connection to the database:
conn.execute("INSERT INTO user (name, country_id) VALUES ('Homer', 123)")
Assume also that the user table consists of the columns (id [SERIAL PRIMARY KEY], name, country_id)
How may I obtain the id of the new user, ideally, without hitting the database again?

You might be able to use the RETURNING clause of the INSERT statement like this:
result = conn.execute("INSERT INTO user (name, country_id) VALUES ('Homer', 123)
RETURNING *")
If you only want the resulting id:
result = conn.execute("INSERT INTO user (name, country_id) VALUES ('Homer', 123)
RETURNING id")
[new_id] = result.fetchone()

User lastrowid
result = conn.execute("INSERT INTO user (name, country_id) VALUES ('Homer', 123)")
result.lastrowid

Current SQLAlchemy documentation suggests
result.inserted_primary_key should work!

Python + SQLAlchemy
after commit, you get the primary_key column id (autoincremeted) updated in your object.
db.session.add(new_usr)
db.session.commit() #will insert the new_usr data into database AND retrieve id
idd = new_usr.usrID # usrID is the autoincremented primary_key column.
return jsonify(idd),201 #usrID = 12, correct id from table User in Database.

this question has been asked many times on stackoverflow and no answer I have seen is comprehensive. Googling 'sqlalchemy insert get id of new row' brings up a lot of them.
There are three levels to SQLAlchemy.
Top: the ORM.
Middle: Database abstraction (DBA) with Table classes etc.
Bottom: SQL using the text function.
To an OO programmer the ORM level looks natural, but to a database programmer it looks ugly and the ORM gets in the way. The DBA layer is an OK compromise. The SQL layer looks natural to database programmers and would look alien to an OO-only programmer.
Each level has it own syntax, similar but different enough to be frustrating. On top of this there is almost too much documentation online, very hard to find the answer.
I will describe how to get the inserted id AT THE SQL LAYER for the RDBMS I use.
Table: User(user_id integer primary autoincrement key, user_name string)
conn: Is a Connection obtained within SQLAlchemy to the DBMS you are using.
SQLite
======
insstmt = text(
'''INSERT INTO user (user_name)
VALUES (:usernm) ''' )
# Execute within a transaction (optional)
txn = conn.begin()
result = conn.execute(insstmt, usernm='Jane Doe')
# The id!
recid = result.lastrowid
txn.commit()
MS SQL Server
=============
insstmt = text(
'''INSERT INTO user (user_name)
OUTPUT inserted.record_id
VALUES (:usernm) ''' )
txn = conn.begin()
result = conn.execute(insstmt, usernm='Jane Doe')
# The id!
recid = result.fetchone()[0]
txn.commit()
MariaDB/MySQL
=============
insstmt = text(
'''INSERT INTO user (user_name)
VALUES (:usernm) ''' )
txn = conn.begin()
result = conn.execute(insstmt, usernm='Jane Doe')
# The id!
recid = conn.execute(text('SELECT LAST_INSERT_ID()')).fetchone()[0]
txn.commit()
Postgres
========
insstmt = text(
'''INSERT INTO user (user_name)
VALUES (:usernm)
RETURNING user_id ''' )
txn = conn.begin()
result = conn.execute(insstmt, usernm='Jane Doe')
# The id!
recid = result.fetchone()[0]
txn.commit()

result.inserted_primary_key
Worked for me. The only thing to note is that this returns a list that contains that last_insert_id.

Make sure you use fetchrow/fetch to receive the returning object
insert_stmt = user.insert().values(name="homer", country_id="123").returning(user.c.id)
row_id = await conn.fetchrow(insert_stmt)

For Postgress inserts from python code is simple to use "RETURNING" keyword with the "col_id" (name of the column which you want to get the last inserted row id) in insert statement at end
syntax -
from sqlalchemy import create_engine
conn_string = "postgresql://USERNAME:PSWD#HOSTNAME/DATABASE_NAME"
db = create_engine(conn_string)
conn = db.connect()
INSERT INTO emp_table (col_id, Name ,Age)
VALUES(3,'xyz',30) RETURNING col_id;
or
(if col_id column is auto increment)
insert_sql = (INSERT INTO emp_table (Name ,Age)
VALUES('xyz',30) RETURNING col_id;)
result = conn.execute(insert_sql)
[last_row_id] = result.fetchone()
print(last_row_id)
#output = 3
ex -