Can someone please explain to me why the defined test().commit() does not work as varcon.commit()? Everything else seem to work fine. (using vagrant virtualbox of ubuntu-trusty-32)
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import psycopg2
varcon = psycopg2.connect('dbname=tournament')
def test():
try:
psycopg2.connect("dbname=tournament")
except:
print("Connection to Tournament Database Failed")
else:
return psycopg2.connect('dbname=tournament')
def writer():
#db = psycopg2.connect('dbname=tournament')
c =varcon.cursor()
c.execute('select * from players')
data = c.fetchall()
c.execute("insert into players (name) values ('Joe Smith')")
varcon.commit()
varcon.close
print(data)
def writer2():
#db = psycopg2.connect('dbname=tournament')
c =test().cursor()
c.execute('select * from players')
data = c.fetchall()
c.execute("insert into players (name) values ('Joe Smith')")
test().commit()
test().close
print(data)
writer2() #this seem not commited, but database registers the insert by observing the serial promotion
#writer() # this works as expected
maybe this is because the return statement in a function block (def) is not equal to an assignment like =
The psycopg2 connection function returns a connection object and this is assigned to conn or varcon
conn = psycopg2.connect("dbname=test user=postgres password=secret")
http://initd.org/psycopg/docs/module.html
the test() function also returns the psycopg2 connection object but in writer2 it is not assigned to a variable (memory place) meaning that there is no reference
this also explains why the database connection is established (initialized in the test function) but the commit does not work (broken reference)
(http://www.icu-project.org/docs/papers/cpp_report/the_anatomy_of_the_assignment_operator.html)
maybe try
ami=test()
ami.commit()
to establish a reference
Every time you call psycopg2.connect() you open new connection to database.
So effectively your code executes SQL in one connection, commits another, and then closes third connection. Even in your test() function you are opening two different connections.
I use the following pattern to access PostgreSQL:
conn = psycopg2.connect(DSN)
with conn:
with conn.cursor() as curs:
...
curs.execute(SQL1)
with conn:
with conn.cursor() as curs:
...
curs.execute(SQL2)
conn.close()
with statement ensures transaction is opened and properly committed around your SQL. It also automatically rolls transaction back in case your code inside with raises an exception.
Reference: http://initd.org/psycopg/docs/usage.html#with-statement
Related
I want to insert given values from my docker app-service to the MariaDB-service.
The connection has been established because I can execute SELECT * FROM via the MariaDB.connection.cursor.
First of all I create the connection:
def get_conn() -> mariadb.connection:
try:
conn = mariadb.connect(
user="XXX",
database="XXX",
password="XXX",
host="db",
port=33030,
)
except mariadb.Error as e:
print(f'Error connecting to MariaDB Platform: {e}')
sys.exit(1)
return conn
Then I create a mariadb.connection.cursor-Object:
def get_cur() -> mariadb.connection.cursor:
conn = get_conn()
cur = conn.cursor()
return cur
Finally I want to insert new values in the table testing:
def write_data():
cursor = get_cur()
conn = get_conn()
cursor.execute('INSERT INTO testing (title) VALUE ("2nd automatic entry");')
print("Executed Query")
conn.commit()
cursor.close()
conn.close()
print("Closed Connection")
return True
To test, if the entries are inserted, I started with 1 manual entry, then executed the write_data()-function and to finish of I inserted a 2nd manual entry via the console.
After the procedure the table looks like:
Note that the ìd is on AUTO_INCREMENT. So the function write_data() was not skipped entirely, because the 2nd manual entry got the id 3 and not 2.
You're committing a transaction in a different connection than the one your cursor belongs to.
get_conn() creates a new database connection and returns it.
get_cur() calls get_conn, that gets it a new connection, retrieves a cursor object that belongs to it, and returns it.
In your main code, you call get_conn - that gives you connection A.
Then you obtain a cursor by calling get_cur - that creates a connection B and returns a cursor belonging to it.
You run execute on the cursor object (Connection B) but commit the connection you got in the first call (Connection A).
PS: This was a really fun problem to debug, thanks :)
It's really easy, in a new table with new code, to unintentionally do an INSERT without a COMMIT. That is especially true using the Python connector, which doesn't use autocommit. A dropped connection with an open transaction rolls back the transaction. And, a rolled-back INSERT does not release the autoincremented ID value for reuse.
This kind of thing happens, and it's no cause for alarm.
A wise database programmer won't rely on a set of autoincrementing IDs with no gaps in it.
I was trying to use the python connector code given in the MySQL documentation and test it on a small database already created, but it aborts. The code is just supposed to connect to the db and add a new email adress.
import mysql.connector
from mysql.connector import errorcode
cnx = mysql.connector.connect(user='root', password='pwd', host='localhost', database='db')
cursor = cnx.cursor()
add_email = ("INSERT INTO employee(MID, Email) VALUES (%s,%s)")
email_details = (NULL, "a#a.de")
cursor.execute(add_email, email_details)
cnx.commit()
input("data entered successfully")
cnx.close()
By setting breakpoints I found out that the problem probably lies in the cursor.execute() statement. (I used Null as the first %s since MID is Auto Incrementing btw)
To solve this problem NULL (for the autoincrementing "MID") needs to be replaced with None.
I'm using Visual Studio 2017 with a Python Console environment. I have a MySQL database set up which I can connect to successfully. I can also Insert data into the DB. Now I'm trying to display/fetch data from it.
I connect fine, and it seems I'm fetching data from my database, but nothing is actually printing to the console. I want to be able to fetch and display data, but nothing is displaying at all.
How do I actually display the data I select?
#importing module Like Namespace in .Net
import pypyodbc
#creating connection Object which will contain SQL Server Connection
connection = pypyodbc.connect('Driver={SQL Server};Server=DESKTOP-NJR6F8V\SQLEXPRESS;Data Source=DESKTOP-NJR6F8V\SQLEXPRESS;Integrated Security=True;Connect Timeout=30;Encrypt=False;TrustServerCertificate=True;ApplicationIntent=ReadWrite;MultiSubnetFailover=False')
cursor = connection.cursor()
SQLCommand = ("SELECT ID FROM MyAI_DB.dbo.WordDefinitions WHERE ID > 117000")
#Processing Query
cursor.execute(SQLCommand)
#Commiting any pending transaction to the database.
connection.commit()
#closing connection
#connection.close()
I figured it out. I failed to include the right Print statement. Which was:
print(cursor.fetchone())
I also had the connection.commit statement in the wrong place (it was inserted even executing the Print statement). The final code that worked was this:
#importing module Like Namespace in .Net
import pypyodbc
#creating connection Object which will contain SQL Server Connection
connection = pypyodbc.connect('Driver={SQL Server};Server=DESKTOP-NJR6F8V\SQLEXPRESS;Data Source=DESKTOP-NJR6F8V\SQLEXPRESS;Integrated Security=True;Connect Timeout=30;Encrypt=False;TrustServerCertificate=True;ApplicationIntent=ReadWrite;MultiSubnetFailover=False')
cursor = connection.cursor()
SQLCommand = ("SELECT * FROM MyAI_DB.dbo.WordDefinitions")
#Processing Query
cursor.execute(SQLCommand)
#Commiting any pending transaction to the database.
print(cursor.fetchone())
connection.commit()
#closing connection
#connection.close()
I use this code to retreive an id. It works:
db = MySQLdb.connect("localhost","root","","proyectoacademias" )
cursor = db.cursor()
sql = "SELECT id FROM test WHERE url=\'"
sql = sql + self.start_urls[0]
sql = sql + "\'"
cursor.execute(sql)
data = cursor.fetchone()
for row in data:
self.id_paper_web=str(row)
db.close()
It gives me the id of the current row I have to update...
But then I try to update or to insert, it doesn't work....
def guardarDatos(self):
db = MySQLdb.connect("localhost","root","","proyectoacademias" )
cursor = db.cursor()
sql = "UPDATE test SET abstract=\'"+str(self.abstracto)+"\', fecha_consulta=\'"+str(self.fecha_consulta)+"\', anio_publicacion=\'"+str(self.anio_publicacion)+"\', probabilidad="+str(self.probabilidad)+" WHERE id = "+str(self.id_paper_web)
print "\n\n\n"+sql+"\n\n\n"
cursor.execute(sql)
for i in range (len(self.nombres)):
sql = "INSERT INTO test_autores VALUES (\'"+self.nombres.keys()[i]+"\', "+str(self.id_paper_web)+", \'"+self.instituciones[self.nombres[self.nombres.keys()[i]]]+"\', "+str((i+1))+")"
print "\n\n\n"+sql+"\n\n\n"
cursor.execute(sql)
db.close()
I print every sql query I sent and they seem to be fine... no exceptions thrown, just no updates or inserts in the database...
you must commit ... or set the db to auto commit
db.commit()
lots of py sqlite3 tutorials out there
By default, the sqlite3 module opens transactions implicitly before a
Data Modification Language (DML) statement (i.e.
INSERT/UPDATE/DELETE/REPLACE), and commits transactions implicitly
before a non-DML, non-query statement (i. e. anything other than
SELECT or the aforementioned).
So if you are within a transaction and issue a command like CREATE
TABLE ..., VACUUM, PRAGMA, the sqlite3 module will commit implicitly
before executing that command. There are two reasons for doing that.
The first is that some of these commands don’t work within
transactions. The other reason is that sqlite3 needs to keep track of
the transaction state (if a transaction is active or not).
You can control which kind of BEGIN statements sqlite3 implicitly
executes (or none at all) via the isolation_level parameter to the
connect() call, or via the isolation_level property of connections.
If you want autocommit mode, then set isolation_level to None.
Otherwise leave it at its default, which will result in a plain
“BEGIN” statement, or set it to one of SQLite’s supported isolation
levels: “DEFERRED”, “IMMEDIATE” or “EXCLUSIVE”.
http://docs.python.org/library/sqlite3.html Section 11.13.6
I use psycopg2 for accessing my postgres database in python. My function should create a new database, the code looks like this:
def createDB(host, username, dbname):
adminuser = settings.DB_ADMIN_USER
adminpass = settings.DB_ADMIN_PASS
try:
conn=psycopg2.connect(user=adminuser, password=adminpass, host=host)
cur = conn.cursor()
cur.execute("CREATE DATABASE %s OWNER %s" % (nospecial(dbname), nospecial(username)))
conn.commit()
except Exception, e:
raise e
finally:
cur.close()
conn.close()
def nospecial(s):
pattern = re.compile('[^a-zA-Z0-9_]+')
return pattern.sub('', s)
When I call createDB my postgres server throws an error:
CREATE DATABASE cannot run inside a transaction block
with the errorcode 25001 which stands for "ACTIVE SQL TRANSACTION".
I'm pretty sure that there is no other connection running at the same time and every connection I used before calling createDB is shut down.
It looks like your cursor() is actually a transaction:
http://initd.org/psycopg/docs/cursor.html#cursor
Cursors created from the same
connection are not isolated, i.e., any
changes done to the database by a
cursor are immediately visible by the
other cursors. Cursors created from
different connections can or can not
be isolated, depending on the
connections’ isolation level. See also
rollback() and commit() methods.
Skip the cursor and just execute your query. Drop commit() as well, you can't commit when you don't have a transaction open.