We Keep Coding

Error message "no pq wrapper available." popped up when import psycopg?

I'm only running my code which the first line is
import psycopg
and then the ImportError "no pq wrapper available" just popped up.
There's not really a solution i could find anywhere, so that's why i'm asking here.

There is a new way.
I was having the exact same issue, and I really couldn't be bothered to hunt it down, as It is working on WSL with IntelliJ on Windows or on WSL with Ubuntu but not in Jupyter lab on Windows (I'll reply back if I find the issue). So, I thought this is a good opportunity to use import pg8000. pg8000 is a pure python implementation for connecting to Postgres, and so far seems similar in operation (I tested it with a cursor and local Postgres). Install pg8000 with pip3 install pg8000, then...
Basic Ingredients
#!/usr/bin/env python3
import pg8000 as pg
def db_conn():
conn = pg.Connection(host='localhost',
database='dev_database',
user='postgres',
password='yourPassword!')
return conn
# Now you have a connection, query your data with a cursor
conn = db_conn()
cur = conn.cursor()
cur.execute('SELECT * FROM tab1')
data = cur.fetchall()
# Now 'data' contains your data, or use the new way with conn.run
if __name__ == '__main__':
print('Grabbing data from tab1...')
for row in conn.run('SELECT * FROM tab1'):
print(row)
conn.close()
The conn.run method returns a tuple with a list for every row, but no issue to be read into a pandas DataFrame, and as it's an iterable (no I don't recommend this, but it's a lazy Saturday 😄)
df = pd.DataFrame([row for row in conn.run('SELECT * FROM journal;')])
I'll get my coat 😅

pandas to_sql() gives a SADeprecationWarning

The to_sql() function in pandas is now producing a SADeprecationWarning.
df.to_sql(name=tablename, con=c, if_exists='append', index=False )
[..]/lib/python3.8/site-packages/pandas/io/sql.py:1430: SADeprecationWarning:The Connection.run_callable() method is deprecated and will be removed in a future release. Use a context manager instead. (deprecated since: 1.4)
I was getting this even with df.read_sql() command, when running sql select statements. Changing it to the original df.read_sql_query() that it wraps around, got rid of it. I'm suspecting there would be some linkage there.
So, question is, how to do I write a dataframe table to SQL without it getting deprecated in a future release? What does "use a context manager" mean, how can I implement that?
Versions:
pandas: 1.1.5 | SQLAlchemy: 1.4.0 | pyodbc: 4.0.30 | Python: 3.8.0
Working with a mssql database.
OS: Linux Mint Xfce, 18.04. Using a python virtual environment.
If it matters, connection created like so:
conn_str = r'mssql+pyodbc:///?odbc_connect={}'.format(dbString).strip()
sqlEngine = sqlalchemy.create_engine(conn_str,echo=False, pool_recycle=3600)
c = sqlEngine.connect()
And after the db operation,
c.close()
Doing so keeps the main connection sqlEngine "alive" between api calls and lets me use a pooled connection rather than having to connect anew.

Update: according to the pandas team, this will be fixed in Pandas 1.2.4 which as of the time of writing has not been released yet.
Adding this as an answer since Google led here but the accepted answer is not applicable.
Our surrounding code that uses Pandas does use a context manager:
with get_engine(dbname).connect() as conn:
df = pd.read_sql(stmt, conn, **kwargs)
return df
In my case, this error is being thrown from within pandas itself, not in the surrounding code that uses pandas:
/Users/tommycarpenter/Development/python-indexapi/.tox/py37/lib/python3.7/site-packages/pandas/io/sql.py:1430: SADeprecationWarning: The Engine.run_callable() method is deprecated and will be removed in a future release. Use the Engine.connect() context manager instead. (deprecated since: 1.4)
The snippet from pandas itself is:
def has_table(self, name, schema=None):
return self.connectable.run_callable(
self.connectable.dialect.has_table, name, schema or self.meta.schema
)
I raised an issue: https://github.com/pandas-dev/pandas/issues/40825

You could try...
connection_string = r'mssql+pyodbc:///?odbc_connect={}'.format(dbString).strip()
engine = sqlalchemy.create_engine(connection_string, echo=False, pool_recycle=3600)
with engine.connect() as connection:
df.to_sql(name=tablename, con=connection, if_exists='append', index=False)
This approach uses a ContextManager. The ContextManager of the engine returns a connection and automatically invokes connection.close() on it, see. Read more about ContextManager here. Another useful thing to know is, that a connection is a ContextManager as well and handles transactions for you. This means it begins and ends a transaction and in case of an error it automatically invokes a rollback.

pyodbc MERGE INTO error: HY000: The driver did not supply an error

I'm trying to execute many (~1000) MERGE INTO statements into oracledb 11.2.0.4.0(64bit) using python 3.9.2(64bit) and pyodbc 4.0.30(64bit). However, all the statements return an exception:
HY000: The driver did not supply an error
I've tried everything I can think of to solve this problem, but no luck. I tried changing code, encodings/decodings and ODBC driver from oracle home 12.1(64bit) to oracle home 19.1(64bit). I also tried using pyodbc 4.0.22 in which case the error just changed into:
<class 'pyodbc.ProgrammingError'> returned a result with an error set
Which is not any more helpful error than the first one. The issue I assume cannot be the MERGE INTO statement itself, because when I try running them directly in the database shell, it completes without issue.
Below is my code. I guess I should also mention the commands and parameters are read from stdin before being executed, and oracledb is using utf8 characterset.
cmds = sys.stdin.readlines()
comms = json.loads(cmds[0])
conn = pyodbc.connect(connstring)
conn.setencoding(encoding="utf-8")
cursor = conn.cursor()
cursor.execute("""ALTER SESSION SET NLS_DATE_FORMAT='YYYY-MM-DD"T"HH24:MI:SS.....'""")
for comm in comms:
params = [(None) if str(x)=='None' or str(x)=='NULL' else (x) for x in comm["params"]]
try:
cursor.execute(comm["sql"],params)
except Exception as e:
print(e)
conn.commit()
conn.close()
Edit: Another things worth mentioning for sure - this issue began after python2.7 to 3.9.2 update. The code itself didn't require any changes at all in this particular location, though.

I've had my share of HY000 errors in the past. It almost always came down to a syntax error in the SQL query. Double check all your double and single quotes, and makes sure the query works when run independently in an SQL session to your database.

python and sqlite3, check if I can use fts5 extension?

I recently found out that FTS5 extension has been released.
What is the best way to check if my application can use it on users system?
Just python3 version check, or sqlite3.sqlite_version check according to release page?? Or something else?

/ this was previously edit of the OP post, but I moved it down here to keep the question clear
so this feels like it could work, found it in the question here
import sqlite3
con = sqlite3.connect(':memory:')
cur = con.cursor()
cur.execute('pragma compile_options;')
available_pragmas = cur.fetchall()
con.close()
print(available_pragmas)
if ('ENABLE_FTS5',) in available_pragmas:
print('YES')
else:
print('NO')
But what is strange is that I run it in a few virtual machines, and none had fts4 enabled, yet I used it happily like nothing... maybe its fts3/fts4 being rather the same, or maybe its just all wrong.
/edit
from the documentation
Note that enabling FTS3 also makes FTS4 available. There is not a separate SQLITE_ENABLE_FTS4 compile-time option. A build of SQLite either supports both FTS3 and FTS4 or it supports neither.

The documentation you link to mentions that FTS5 is disabled by default. Did you enable it when compiling SQLite?
One quick way to know is to use the peewee ORM:
from playhouse.sqlite_ext import FTS5Model
FTS5Model.fts5_installed()
The above will return True if you can use FTS5. You can install peewee with pip install peewee.
You could also use the apsw wrapper, which includes FTS5 by default since version 3.11.0-r1. See the build instructions and use the --enable-all-extensions flag. The apsw wrapper uses the amalgamation.
EDIT:
Here is the code from the peewee source demonstrating how this is done:
def fts5_installed(cls):
if sqlite3.sqlite_version_info[:3] < FTS5_MIN_VERSION:
return False
# Test in-memory DB to determine if the FTS5 extension is installed.
tmp_db = sqlite3.connect(':memory:')
try:
tmp_db.execute('CREATE VIRTUAL TABLE fts5test USING fts5 (data);')
except:
try:
sqlite3.enable_load_extension(True)
sqlite3.load_extension('fts5')
except:
return False
else:
cls._meta.database.load_extension('fts5')
finally:
tmp_db.close()
return True

Connecting Maya 2011 with Mysqldb

I am using Maya 2011(64bit) and MySQL 5.5 (64 bit) in Windows 7 (64 bit) machine. I tried to connect maya with Mysqldb through python. So i copied the connector files into maya\python\lib\site packages.
I was able to import MYsqldb module without any error. But when i tried call the cursor object (for querying), I found that Maya is not recognizing the cursor object.
Here is my sample code:
import MySQLdb as mb
import maya.cmds as cmds
def mysql_connect(hostname, username, password, dbname):
db = mb.connect(host=hostname,user=username,passwd=password,db=dbname)
db = mysql_connect("localhost", “root”, “test”, “mydbt")
dbcursor = db.cursor()
dbcursor.execute("select * from maya")
But the code throws the following error :
Error: AttributeError: ‘NoneType’ object has no attribute ‘cursor’ #
I tried verifying the env-path variables, replacing the connector files but the problem persists.
Since being a beginner, i am un-able to identify the exact issue.
I request for your valuable suggestions

You are not returning anything from mysql_connect function. So it returns None. When you do:
db = mysql_connect("localhost", “root”, “test”, “mydbt")
db becomes None. Try changing:
db = mb.connect(host=hostname,user=username,passwd=password,db=dbname)
with
return mb.connect(host=hostname,user=username,passwd=password,db=dbname)
That being said, I'm not sure defining a function to make a single thing is useful. Better to have something like this:
import MySQLdb as mb
import maya.cmds as cmds
db = mb.connect(host="localhost",user=“root”,passwd=“test”,db=“mydbt")
dbcursor = db.cursor()
dbcursor.execute("select * from maya")

Here, you have two things assigned to db. It appears that mysql_connect("localhost", “root”, “test”, “mydbt") is returning None, so when you call db.cursor() later, you get that error.
Make sure you're overwriting the db variable correctly (in this case, it looks like you aren't).