I am trying to create a database with pg8000 driver of postgressql , but unable to create. Creating a db manually and then connecting to it works fine with me, but i need to create db with my code. I am getting error "sqlalchemy.exc.ProgrammingError: (pg8000.ProgrammingError)". I have tried below code for creating a db.
from sqlalchemy import create_engine
dburl = "postgresql+pg8000://user:pswd#myip:5432/postgres/"
engine = create_engine(dburl)
conn = engine.connect()
conn.execute("COMMIT")
conn.execute("CREATE DATABASE qux")
I also tried with below -
from sqlalchemy import create_engine
from sqlalchemy.engine import url
settings ={"drivername" : "postgresql+pg8000", "host" : "myip","port" : 5432,"username" : "user","password" : "pswd","database" : "MyTestDB"}
db=create_engine(url.URL(**settings))
db.execute("commit")
This is the exact Error i am getting """sqlalchemy.exc.ProgrammingError: (pg8000.ProgrammingError) ('ERROR', '25001', 'CREATE DATABASE cannot run inside a transaction block') [SQL: 'create database workDB']""""
Please suggest as to how i can create this db...
Here's a solution:
from sqlalchemy import create_engine
dburl = "postgresql+pg8000://user:pswd#myip:5432/postgres/"
engine = create_engine(dburl)
conn = engine.connect()
con.rollback() # Make sure we're not in a transaction
con.autocommit = True # Turn on autocommit
conn.execute("CREATE DATABASE qux")
con.autocommit = False # Turn autocommit back off again
The docs talk about this problem of executing commands that can't be run in a transaction. The thing is that pg8000 automatically executes a begin transaction before any execute() if there isn't already a transaction in progress. This is fine until you come to execute a command that can't be executed inside a transaction. In that case you have to enter autocommit mode, which implicitly starts a transaction before the statement and commits it afterwards, but automatically avoids doing this if (like CREATE DATABASE) the statement can't be executed within a transaction.
Related
I have a postgres db running on docker. I am able to access this db via my sql client Dbeaver and when I run select statements I see the expected results.
I would like to be able to query this db via a Python script and after some searching found psycopg2 package.
When I run the code below it 'looks' like it's successful, the conn and cursor objects appear as a variables.
import pandas as pd
import psycopg2
# connect to db
conn = psycopg2.connect(
host="localhost",
database="postgres",
user="postgres",
password="example")
# create a cursor
cur = conn.cursor()
However, when trying to query the db using cur.connect(), , variable ex_data is None. This exact same query via my sql client returns a table of data.
ex_data = cur.execute('select * from myschema.blah limit 10;')
How can I query my db via Python using psycopg2? Desired result wold be a data frame with the result set from the query string above.
I have a mysql database that I am currently trying to connect python to. I have successfully created a table with python but when it comes to inserting data in the table my code does not work.
import pymysql
conn = pymysql.connect(
host = Admin.Host,
user = Admin.User,
password= Admin.Password,
db = "thesystem"
)
a = conn.cursor()
sql ="INSERT INTO `a`(`try`) VALUES ('testing')"
a.execute(sql)
I have tried this exact command in the Mysql command Line client and it worked.
INSERT INTO `a`(`try`) VALUES ('testing');
I have also managed to create a table with this same code. I can't figure out why the code runs successfully but does not work.
The resolution is that pymysql requires that you call conn.commit() after making a change. By default, pymysql is not autocommit.
Code example at https://github.com/PyMySQL/PyMySQL says:
. . .
# connection is not autocommit by default. So you must commit to save
# your changes.
connection.commit()
. . .
It does not seem like SqlAlchemy supports calling stored procedures. Did anybody find a workaround for this that works with SQL Server?
Sample procedure:
CREATE PROCEDURE list_lock_set #name varchar (5), #requester varchar(30)
AS
BEGIN
SET NOCOUNT ON;
INSERT INTO list_lock (name, requester, acquired) values (#name, #requester, GETDATE())
RETURN 0
END
GO
This works:
import pyodbc
dbh = pyodbc.connect(driver=''{SQL Server}'', server=srv, database=db, uid=uid, pwd=pwd)
dbc = dbh.cursor()
dbc.execute("list_lock_set ?, ?", ['bbc', 'pyodbc'])
dbc.commit()
This does not produce an error but also but does not work:
from sqlalchemy import create_engine
engine = create_engine('mssql+pyodbc://usr:passw#srv/db?driver=SQL Server', echo=True)
engine.execute("list_lock_set ?, ?", ['bbc', 'sqlalchemy'])
Thank you.
EDIT: It appears the best solution is to fish out pyodbc cursor from the engine:
cursor = engine.raw_connection().cursor()
cursor.execute("list_lock_set ?, ?", ['bbc', 'using cursor'])
cursor.commit()
I can also obtain pyodbc Connection:
engine.raw_connection().connection
and set autocommit=True, but that might interfere with engine logic. Many thanks to #Batman.
To have it working in sqlalchemy I managed to do it this way:
from sqlalchemy import create_engine
engine = create_engine('mssql+pyodbc://usr:passw#srv/db?driver=SQL Server', echo=True)
with engine.begin() as conn:
conn.execute("exec dbo.your_proc")
I remember this giving me grief before too. From memory either session.execute() or connection.execute() worked for me. There's also a callproc() method, which is probably the right way to go. http://docs.sqlalchemy.org/en/latest/core/connections.html
Also, I've had issues in the past with MSSQL which seemed to be due to something asynchronous happening where the method was returning before the procedure was finished, which resulted in unpredictable results on the database. I found that putting a time.sleep(1) (or whatever the appropriate number is) right after the call fixed this.
Yes, Even I was facing the same issue. SQLAlchemy was executing the procedure but no actions were happening.
So I tweaked it by adding connection.execute("exec <Procedure_name>")
Following on from #MATEITA LUCIAN:
Add input parameters to the stored procedure
from sqlalchemy import create_engine
engine = create_engine('mssql+pyodbc://usr:passw#srv/db?driver=SQLServer', echo=True)
with engine.begin() as conn:
conn.execute('EXEC dbo.your_proc #textinput_param = "sometext", #intinput_param = 4')
For those that aren't sure about how or what to refer to with the '....//usr:passw#srv/db?driver=SQL server', echo=True... part of creating the engine I would suggest reading this and choosing Option 1 - Provide a DSN.
Retrieve output from the stored procedure
from sqlalchemy import create_engine
engine = create_engine('mssql+pyodbc://usr:passw#srv/db?driver=SQLServer', echo=True)
with engine.begin() as conn:
result = conn.execute('EXEC dbo.your_proc #textinput_param = "sometext", #intinput_param = 4')
for row in result:
print('text_output: %s \t int_output: %d' % (row['text_field'], row['int_field']))
Something peculiar I've noticed is that any changes committed to the DB outside of the session (such as ones made in MySQL's Workbench) are not recognised in the sqlAlchemy session. I have to close and open a new session for sqlAlchemy to recognise it.
For example, a row I deleted manually is still fetched from sqlAlchemy.
This is how I initialise the session:
engine = create_engine('mysql://{}:{}#{}/{}'.format(username, password, host, schema), pool_recycle=3600)
Session = sessionmaker(bind=engine)
session = Session()
metadata = MetaData()
How can I get sqlAlchemy to recognise them?
My sqlAlchemy version is 0.9.4 and my MySQL version is 5.5.34. We use only sqlAlchemy's Core (no ORM).
To be able to read committed data from others transactions you'll need to set transaction isolation level to READ COMMITTED. For sqlalchemy and mysql:
To set isolation level using create_engine():
engine = create_engine(
"mysql://scott:tiger#localhost/test",
isolation_level="READ COMMITTED")
To set using per-connection execution options:
connection = engine.connect()
connection = connection.execution_options(
isolation_level="READ COMMITTED")
source
I'm currently stuck with the following error when I try to perform an add using a session with SQLAlchemy.
sqlalchemy.exc.ProgrammingError: (ProgrammingError) (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '%s)' at line 1") b'INSERT INTO ctr_status (`Name`) VALUES (%s)' ('Test',)
Here is the code:
from sqlalchemy import MetaData, Table
from sqlalchemy import create_engine, orm
# Login omitted...
url = 'mysql+mysqldb://{user}:{password}#{host}:{port}/{database}'.format(user=user, password=password, host=host, port=port, database=database)
e = create_engine(url, echo=True)
metadata = MetaData()
metadata.bind = e
metadata.create_all()
ctr_status = Table('ctr_status', metadata, autoload=True)
class CTRStatus(object):
pass
orm.mapper(CTRStatus, ctr_status)
new_status = CTRStatus()
new_status.Name = 'Test'
session_maker = orm.sessionmaker(bind=e, autoflush=True, autocommit=False, expire_on_commit=True)
session = orm.scoped_session(session_maker)
session.add(new_status)
# Crash here at flush
session.flush()
I'm confused as to what I'm doing wrong. I'm currently using MySQL 5.5, Python 3.3.2 and SQLAlchemy-0.8.2.win32-py3.3.
Here are some of the links I have been following:
SQL Alchemy - Tutorial
SQL Alchemy - Sessions
I managed to get it working by switching the adapter to OurSQL and changing the url to use mysql+oursql.
I would like to add that it was surprisingly non-trivial to get OurSQL working on my setup for Python 3.3.
I had to delete oursqlx/oursql.c from the initial downloaded files such that it be auto-generated by Cython.
I also had to manually modify the setup.py because it was pointing to the wrong registry path for the mysql_root variable in setup_windowsish(). Essentially, the current implementation doesn't seem to support MySQL Server 5.5 nor does it support if the registry path is in HKEY_CURRENT_USER instead of HKEY_LOCAL_MACHINE.