I am using Python 3.x within an app hosted on Heroku with Basic PostgreSQL instance and Im using the psycopg2 library (as "lite")
Recently started hanging when I call Execute on the cursor object.
I am not sure how to troubleshoot this and would appreciate any thoughts.
Here is how I instantiate Connection:
def getConnection():
urlparse.uses_netloc.append("postgres")
url = urlparse.urlparse(os.environ["HEROKU_STUFF_HERE"])
con = lite.connect(
database=url.path[1:],
user=url.username,
password=url.password,
host=url.hostname,
port=url.port
)
return con
Here is how I generally execute non-return type statements:
def ExecuteSQL(sql):
con = DB.getConnection()
with con:
cur = con.cursor()
print("In")
cur.execute(sql)
print("out")
The app NEVER sees the light after printing the word "In" to the console.
I've left try except blocks out intentionally so as to blow the thing up.....no dice.
pretty much doesnt matter what the SQL Statement is, I get the same result
Running the same sql in a sql client tool executes instantly.
Im also not sure how to detect if this statement even makes it into Postgresql.....
Thanks for any help you can offer
psycopg2 opened the transaction which is not closed until a commit() or rollback(). Therefore your changes were not persistent (docs).
def ExecuteSQL(sql):
con = DB.getConnection()
with con:
cur = con.cursor()
cur.execute(sql)
con.commit()
or you could just do the following:
def ExecuteSQL(sql):
con = DB.getConnection()
with con:
con.autocommit = True
cur = con.cursor()
cur.execute(sql)
Related
Kindly look at the code below. I'm using opentelemetry for tracing. Psycopg2Instrumentor for PostgreSQL tracing. Here only the "show server_version" SQL statement is getting traced. But the SQL statement in execute method is not traced. I think it's because of using NamedTupleCursor cursor_factory. If I remove NamedTupleCursor, it's tracing the main SQL statements. Could you please help me to trace the main SQL statement without removing NamedTupleCursor?
def self.get_connection():
#conn = create_connection()
with conn.cursor() as curs:
curs.execute("show server_version") ---> this sql statement is getting tracked
return conn
def execute()
with self.get_connection() as conn:
with conn.cursor(cursor_factory=NamedTupleCursor) as curs:
curs.execute("Sql statements"). ---> this sql statement is **not** getting tracked```
Below is the code snippet for working with Psycopg2Instrumentor for PostgreSQL tracing. The instrumentation code to be updated on passing cursor_factory in cursor parameter, rather than setting it in connection. For now, the below works for me and tracing got captured.
import psycopg2
from opentelemetry.instrumentation.psycopg2 import Psycopg2Instrumentor
Psycopg2Instrumentor().instrument()
#add your cursor factory in connection method
cnx = psycopg2.connect(
host=host, database=DBname, user=user, password=password, cursor_factory=RealDictCursor)
#remove the cursor factory from cursor method
cursor = cnx.cursor()
cursor.execute("SELECT statement")
cursor.close()
cnx.close()
Thanks to the thread (Psycopg2Instrumentor doesn't work for cursors with non-default cursor_factory) and #RaguramGopi
i've been trying to get some data from my db by using below code, but the code is not working. is there any mistake that i made in the code, if so how can i fix it.
NOTE: i took the below code from just a script not a django or flesk web app.
def db():
conn = psycopg2.connect(
"dbname=mydb user=postgres password=****** host=*.*.*.*")
cur = conn.cursor()
cur.execute("""SELECT * FROM MddPublisher""")
query_results = cur.fetchall()
print(query_results)
db()
ERROR: psycopg2.errors.UndefinedTable: relation "mddpublisher" does not exist LINE 1: SELECT * FROM MddPublisher
additionally,i want to show below code to prove that connection is ok. the problem is that i can't receive data from my db whenever i try to execute select command through python.
def print_tables():
conn = psycopg2.connect(
"dbname=mydb user=postgres password=***** host=*.*.*.*.*")
cur = conn.cursor()
cur.execute("""SELECT table_name FROM information_schema.tables
WHERE table_schema = 'public'""")
for table in cur.fetchall():
print(table)
print_tables()
OUTPUT:
('MddPublisher',)
This is probably an issue with case sensitivity. Postgresql names are usually normalized to lower case. However, when used inside double quotes, they keep their case. So, to access a table named MddPublisher you must write it like "MddPublisher".
All the gory details are in Section 4.1.1, Identifiers and Key Words in the Postgresql 14 docs.
I am running some code on jupyter notebook, there are some SQL queries to an aws redshift database.
The problem is after executing these queries even when I am not running anything on the notebook looks like the tables stay read-locked.
When I close the terminal running the notebook the lock releases.
Sample of running code
def met():
con=psycopg2.connect(host=
,user=
,password=
,port=
,dbname =)
table_data = pd.read_sql_query(query, con)
con.close()
Your cursor might be staying open, you should wrap it in a try-except-finally in order to ensure it closes when you are done with your query:
cursor = None
try:
cursor = db.cursor()
cursor.execute("""SELECT foo FROM bar""")
module.rase_unexpected_error()
cursor.commit()
except BaseException:
if cursor is not None:
cursor.rollback()
finally:
if cursor is not None:
cursor.close()
check out this question for more info: use try/except with psycopg2 or "with closing"?
I'm using Visual Studio 2017 with a Python Console environment. I have a MySQL database set up which I can connect to successfully. I can also Insert data into the DB. Now I'm trying to display/fetch data from it.
I connect fine, and it seems I'm fetching data from my database, but nothing is actually printing to the console. I want to be able to fetch and display data, but nothing is displaying at all.
How do I actually display the data I select?
#importing module Like Namespace in .Net
import pypyodbc
#creating connection Object which will contain SQL Server Connection
connection = pypyodbc.connect('Driver={SQL Server};Server=DESKTOP-NJR6F8V\SQLEXPRESS;Data Source=DESKTOP-NJR6F8V\SQLEXPRESS;Integrated Security=True;Connect Timeout=30;Encrypt=False;TrustServerCertificate=True;ApplicationIntent=ReadWrite;MultiSubnetFailover=False')
cursor = connection.cursor()
SQLCommand = ("SELECT ID FROM MyAI_DB.dbo.WordDefinitions WHERE ID > 117000")
#Processing Query
cursor.execute(SQLCommand)
#Commiting any pending transaction to the database.
connection.commit()
#closing connection
#connection.close()
I figured it out. I failed to include the right Print statement. Which was:
print(cursor.fetchone())
I also had the connection.commit statement in the wrong place (it was inserted even executing the Print statement). The final code that worked was this:
#importing module Like Namespace in .Net
import pypyodbc
#creating connection Object which will contain SQL Server Connection
connection = pypyodbc.connect('Driver={SQL Server};Server=DESKTOP-NJR6F8V\SQLEXPRESS;Data Source=DESKTOP-NJR6F8V\SQLEXPRESS;Integrated Security=True;Connect Timeout=30;Encrypt=False;TrustServerCertificate=True;ApplicationIntent=ReadWrite;MultiSubnetFailover=False')
cursor = connection.cursor()
SQLCommand = ("SELECT * FROM MyAI_DB.dbo.WordDefinitions")
#Processing Query
cursor.execute(SQLCommand)
#Commiting any pending transaction to the database.
print(cursor.fetchone())
connection.commit()
#closing connection
#connection.close()
I'm new to mySQL and Python.
I have code to insert data from Python into mySQL,
conn = MySQLdb.connect(host="localhost", user="root", passwd="kokoblack", db="mydb")
for i in range(0,len(allnames)):
try:
query = "INSERT INTO resumes (applicant, jobtitle, lastworkdate, lastupdate, url) values ("
query = query + "'"+allnames[i]+"'," +"'"+alltitles[i]+"',"+ "'"+alldates[i]+"'," + "'"+allupdates[i]+"'," + "'"+alllinks[i]+"')"
x = conn.cursor()
x.execute(query)
row = x.fetchall()
except:
print "error"
It seems to be working fine, because "error" never appears. Instead, many rows of "1L" appear in my Python shell. However, when I go to MySQL, the "resumes" table in "mydb" remains completely empty.
I have no idea what could be wrong, could it be that I am not connected to MySQL's server properly when I'm viewing the table in MySQL? Help please.
(I only use import MySQLdb, is that enough?)
use commit to commit the changes that you have done
MySQLdb has autocommit off by default, which may be confusing at first
You could do commit like this
conn.commit()
or
conn.autocommit(True) Right after the connection is created with the DB