This is probably a stupid question but I cannot find the information in the documentation for pymysql. What is the pymysql default cursorclass? When I do not specify a cursor class on connection to the database my queries return a list for each row in the response.
When I specify pymysql.cursors.DictCursor I get a dictionary response. I would like to be able to change between them for different connections within a script.
I've written a little function with a context manager to yield the cursor but it requires me to specify the name of the cursorclass each time. I know I can get around this, but knowing the name of the default cursorclass would be nice.
from contextlib import contextmanager
import pymysql
#contextmanager
def openDb(host=DB_HOST, database=DB_DATABASE,
user=DB_USER, cursor=DB_CURSOR):
"""
Simple context manager for opening a db connection
"""
with pymysql.connect(host=host, database=database, user=user,
cursorclass=cursor) as cur:
yield cur
I could probably write this as:
#contextmanager
def openDb(host=DB_HOST, database=DB_DATABASE,
user=DB_USER, cursor=None):
"""
Simple context manager for opening a db connection
"""
if cursor:
with pymysql.connect(host=host, database=database, user=user,
cursorclass=cursor) as cur:
yield cur
else:
with pymysql.connect(host=host, database=database, user=user) as cur:
yield cur
and let it default to whatever the default cursorclass is, but I would prefer to be explicit.
Of course as soon as I post this I find the answer in via:
>>> import pymysql
>>> help(pymysql.cursors)
Help on module pymysql.cursors in pymysql:
NAME
pymysql.cursors - # -*- coding: utf-8 -*-
CLASSES
builtins.object
Cursor
SSCursor
DictCursorMixin
DictCursor(DictCursorMixin, Cursor)
SSDictCursor(DictCursorMixin, SSCursor)
pymysql.cursors.Cursor is the answer. Documentation...
Related
I currently have many scripts that connect to the same MSSQL database. I make the connection in each of the scripts, but for ease of use I want to put the connection in a module and call that module from my script. The code in my module connect_to_db.pyc looks like this:
import pyodbc
def sql_connect():
server="some_server.net"
port="1433"
user = "my_username#my_domain"
server="my_server"
database="my_database"
conn = pyodbc.connect('DRIVER={SQL Server};SERVER=my_server,1433',
user=user,
password=password,
database=database)
c=conn.cursor()
Then, in my script I try to call this module and run a query:
from connect_to_db import sql_connect
sql_connect()
c.execute("SELECT * FROM table")
I get the error that the name c is not defined. I tried to define it as a global too, but it don't help. It must have something to do with my lack of understanding modules, but I can't figure out what.
You can return cursor in your sql_connect function
import pyodbc
def sql_connect():
server="some_server.net"
port="1433"
user = "my_username#my_domain"
server="my_server"
database="my_database"
conn = pyodbc.connect('DRIVER={SQL Server};SERVER=my_server,1433',
user=user,
password=password,
database=database)
return conn.cursor()
And then you can use it as
from connect_to_db import sql_connect
c = sql_connect()
c.execute("SELECT * FROM table")
You are indeed missing a bit there:
in your function sql_connect, you assign to a local variable named c.
That variable is not existant outside your function.
If you want a connection variable to exist on module level, maybe try the following attempt:
In your "connect_to_db.py":
import pyodbc
def sql_connect():
server="some_server.net"
port="1433"
user = "my_username#my_domain"
server="my_server"
database="my_database"
conn = pyodbc.connect('DRIVER={SQL Server};SERVER=my_server,1433',
user=user,
password=password,
database=database)
return conn.cursor()
cursor = sql_connect()
This creates a varibale "cursor" on the level of the module.
In another module, simply perform
from connect_to_db import cursor
to import the module's "cursor" member.
This should do the trick.
Hint: Please be advised that this approach may not be very elegant, in terms of software-engineering.
Edit:
Maybe, you may want to dive deeper into object-oriented programming?
class MSSQLConnector(object):
def __init__(self, server, port, database, user, password):
self.server = server
self.port = port
self.conn = pyodbc.connect('DRIVER={SQL Server};SERVER='{0},
{1}.format((self.server, self.port)), user, password, database)
def open_cursor(self):
return self.conn.cursor()
Which would be used in this fashion:
connector = MSSQLConnector("my_server", "1433", "my_database", "username", "secret-password")
cursor = connector.open_cursor()
Can someone please explain to me why the defined test().commit() does not work as varcon.commit()? Everything else seem to work fine. (using vagrant virtualbox of ubuntu-trusty-32)
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import psycopg2
varcon = psycopg2.connect('dbname=tournament')
def test():
try:
psycopg2.connect("dbname=tournament")
except:
print("Connection to Tournament Database Failed")
else:
return psycopg2.connect('dbname=tournament')
def writer():
#db = psycopg2.connect('dbname=tournament')
c =varcon.cursor()
c.execute('select * from players')
data = c.fetchall()
c.execute("insert into players (name) values ('Joe Smith')")
varcon.commit()
varcon.close
print(data)
def writer2():
#db = psycopg2.connect('dbname=tournament')
c =test().cursor()
c.execute('select * from players')
data = c.fetchall()
c.execute("insert into players (name) values ('Joe Smith')")
test().commit()
test().close
print(data)
writer2() #this seem not commited, but database registers the insert by observing the serial promotion
#writer() # this works as expected
maybe this is because the return statement in a function block (def) is not equal to an assignment like =
The psycopg2 connection function returns a connection object and this is assigned to conn or varcon
conn = psycopg2.connect("dbname=test user=postgres password=secret")
http://initd.org/psycopg/docs/module.html
the test() function also returns the psycopg2 connection object but in writer2 it is not assigned to a variable (memory place) meaning that there is no reference
this also explains why the database connection is established (initialized in the test function) but the commit does not work (broken reference)
(http://www.icu-project.org/docs/papers/cpp_report/the_anatomy_of_the_assignment_operator.html)
maybe try
ami=test()
ami.commit()
to establish a reference
Every time you call psycopg2.connect() you open new connection to database.
So effectively your code executes SQL in one connection, commits another, and then closes third connection. Even in your test() function you are opening two different connections.
I use the following pattern to access PostgreSQL:
conn = psycopg2.connect(DSN)
with conn:
with conn.cursor() as curs:
...
curs.execute(SQL1)
with conn:
with conn.cursor() as curs:
...
curs.execute(SQL2)
conn.close()
with statement ensures transaction is opened and properly committed around your SQL. It also automatically rolls transaction back in case your code inside with raises an exception.
Reference: http://initd.org/psycopg/docs/usage.html#with-statement
I've the code like this below. Is it necessary to close mysql connection because whenever my home page is requested, a new sql connection will be created?
I randomly get Connection Limit error. But I'm not sure if the DB connection is the problem.
#app.route("Home", methods=["GET"])
def get_home_page():
db = mysql.connect(host, user, password, db_name, charset='utf8', use_unicode=True)
...
It is good practice to close the connection. You can put your codes inside a try..finally block.
#app.route("Home", methods=["GET"])
def get_home_page():
db = mysql.connect(host, user, password, db_name, charset='utf8', use_unicode=True)
try:
... do something ...
finally:
db.close()
from my experience, close session after use it take significant difference amount of time to response in api whom i've experienced in flask
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
try:
db.session.query( Any Model ...
except:
finally:
db.close_all_sessions
I have read the Flask documentation and Python documentation to try to understand what these codes do. I know that it is initialise the Database but would like to know in very detail and with normal language, easy language for beginner.
Can anyone please explain me about this?
import sqlite3
from contextlib import closing
DATABASE = 'flaskr.db'
def connect_db():
return sqlite3.connect(app.config[’DATABASE’])
def init_db():
with closing(connect_db()) as db:
with app.open_resource(’schema.sql’, mode=’r’) as f:
db.cursor().executescript(f.read())
db.commit()
import sqlite3
Imports Data base connectors
from contextlib import closing
No idea why its used.
Documentetion here
DATABASE = 'flaskr.db'
Defines database,in this case it is a db file.
def connect_db():
return sqlite3.connect(app.config[’DATABASE’])
Connect DB method which returns a sqlite3 connection, using sqlite3 import and invoking connect() on it.
Hint: try >>> dir(sqlite3) after importing in your python console
def init_db():
with closing(connect_db()) as db:
with app.open_resource(’schema.sql’, mode=’r’) as f:
db.cursor().executescript(f.read())
db.commit()
Initialize database method, it takes a schema.sql and executes it on DB by f.read method.
After executing you need to commit changes to db, hence db.commit()
You can find it explained clearly here
I am used to (spoiled by?) python's SQLite interface to deal with SQL databases. One nice feature in python's SQLite's API the "context manager," i.e., python's with statement. I usually execute queries in the following way:
import as sqlite
with sqlite.connect(db_filename) as conn:
query = "INSERT OR IGNORE INTO shapes VALUES (?,?);"
results = conn.execute(query, ("ID1","triangle"))
With the code above, if my query modifies the database and I forget to run conn.commit(),the context manager runs it for me automatically upon exiting the with statement. It also handles exceptions nicely: if an exception occurs before I commit anything, then the database is rolled back.
I am now using the MySQLdb interface, which doesn't seem to support a similar context manager out of the box. How do I create my own? There is a related question here, but it doesn't offer a complete solution.
Previously, MySQLdb connections were context managers.
As of this commit on 2018-12-04, however, MySQLdb connections are no longer context managers,
and users must explicitly call conn.commit() or conn.rollback(), or write their own context manager, such as the one below.
You could use something like this:
import config
import MySQLdb
import MySQLdb.cursors as mc
import _mysql_exceptions
import contextlib
DictCursor = mc.DictCursor
SSCursor = mc.SSCursor
SSDictCursor = mc.SSDictCursor
Cursor = mc.Cursor
#contextlib.contextmanager
def connection(cursorclass=Cursor,
host=config.HOST, user=config.USER,
passwd=config.PASS, dbname=config.MYDB,
driver=MySQLdb):
connection = driver.connect(
host=host, user=user, passwd=passwd, db=dbname,
cursorclass=cursorclass)
try:
yield connection
except Exception:
connection.rollback()
raise
else:
connection.commit()
finally:
connection.close()
#contextlib.contextmanager
def cursor(cursorclass=Cursor, host=config.HOST, user=config.USER,
passwd=config.PASS, dbname=config.MYDB):
with connection(cursorclass, host, user, passwd, dbname) as conn:
cursor = conn.cursor()
try:
yield cursor
finally:
cursor.close()
with cursor(SSDictCursor) as cur:
print(cur)
connection = cur.connection
print(connection)
sql = 'select * from table'
cur.execute(sql)
for row in cur:
print(row)
To use it you would place config.py in your PYTHONPATH and define the HOST, USER, PASS, MYDB variables there.
Think things have changed since this question was originally asked. Somewhat confusingly (from my point of view at least), for recent versions of MySQLdb, if you use a connection in a context you get a cursor (as per the oursql example), not something that closes automatically (as you would if you opened a file for instance).
Here's what I do:
from contextlib import closing
with closing(getConnection()) as conn: #ensure that the connection is closed
with conn as cursor: #cursor will now auto-commit
cursor.execute('SELECT * FROM tablename')