We Keep Coding

Python -> MySQL "select * from brand WHERE text='L\'Orial'" (quote inside search text)

From Python 3.9 i'm trying to do a MySql query like this
select * from brand WHERE text='L\'Orial'.
It works fine from phpMyAdmin but fails from python for all text including quote "'"
brand = "L'Orial"
where = f"text='{brand}'"
brand_pk_id = self.getPrimaryKeyIfExistInTable('brand', where)
def getPrimaryKeyIfExistInTable(self, table, where, key='id'):
try:
sql = f"SELECT {key} FROM {table} WHERE {where}"
self.cursor.execute(sql)
result = self.cursor.fetchone()
return result[key if self.bUseDictCursor else 0] if result else None
except pymysql.MySQLError as e:
logging.error(e)
return None
I can see that python escapes all quotes, which probably causes the problem, but can not figure out how to handle it properly !!
If I turn it around and use query LIKE with underscore( _ ) as wildcard:
brand = "L_Orial"
sql = f"SELECT {key} FROM {table} WHERE text LIKE '{brand}'"
It works fine, but this is not what I want !!

If I am understanding your question correctly, your problem is as follows:
Your query must exactly read:
SELECT * from brand WHERE text='L\'Orial'
But you are currently getting something like this, when you use python to execute the query:
SELECT * from brand WHERE text='L'Orial'
If this is indeed the issue, you should be able to resolve this by simply escaping the backslash that you need to have in the query. The complete python string for your query would be:
# Python String:
"SELECT * from brand WHERE text='L\\'Orial'"
# Resulting Query
SELECT * from brand WHERE text='L\'Orial'
If you wanted to automatically fix this issue for all brands that might include a ', you can simply replace the ' with \\' before making the query. Example:
brand = "L'Orial"
brand = brand.replace("'", "\\'")
# New Python string:
# "L\\'Orial"
# Output in SQL
# "L\'Orial"

Had to fire up my local instance just to make a point.
First, some prep work...
import pymysql
table = 'ps_carrier'
key = 'id_carrier'
mysql = {
"charset": "utf8",
"database": "mystore",
"host": "localhost",
"password": "secret",
"user": "justin"
}
As somebody suggested in the comments, the following
sql = "SELECT %s FROM %s WHERE %s"
where = "name='UPS'"
with pymysql.connect(**mysql) as conn:
with conn.cursor() as cur:
cur.execute(sql, (key, table, where))
Raises an error as expected since all the (string) params are quoted, even the table name!
Traceback (most recent call last):
File "<stdin>", line 3, in <module>
...
File "C:\Python38\site-packages\pymysql\err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
pymysql.err.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ''ps_carrier' WHERE 'name=\\'UPS\\''' at line 1")
If you can trust the inputs for the table name, the key, and the column name(s) then perhaps a simple query builder can help.
params = {'name': 'UPS'} # add more key--value pairs here
# use backticks in case we need to escape reserved words (OP uses MySQL)
where = " AND ".join(f"`{k}` = %s" for k in params.keys()) # .keys() just to be explicit
args = tuple([v for v in params.values()])
# backticks again
sql = f"SELECT `{key}` FROM `{table}` WHERE {where}"
print(sql)
print(args)
with pymysql.connect(**mysql) as conn:
with conn.cursor() as cur:
cur.execute(sql, args)
print(cur.fetchall())
If you need something more elaborate, there are a few modules such as Mysql Simple Query Builder and PyPika - Python Query Builder that you may want to look at (I've not used any of these.)

Are spatial queries possible using Python and cx_Oracle?

I am trying to execute a spatial query on an Oracle spatial table via python using the cx_Oracle package.
I can make generic queries successfully, but when I try a spatial query it results in errors.
This is what I have tried:
import cx_Oracle
...
lon = -120.494352
lat = 36.585289
# open a connection to oracle
con = cx_Oracle.connect('myuser/mypass#spatialdb')
# create a cursor
cur = con.cursor()
# Create and populate Oracle objects
typeObj = con.gettype("MDSYS.SDO_GEOMETRY")
elementInfoTypeObj = con.gettype("MDSYS.SDO_ELEM_INFO_ARRAY")
ordinateTypeObj = con.gettype("MDSYS.SDO_ORDINATE_ARRAY")
obj = typeObj.newobject()
obj.SDO_GTYPE = 2001
obj.SDO_SRID = 8307
obj.SDO_ELEM_INFO = elementInfoTypeObj.newobject()
obj.SDO_ELEM_INFO.extend([1, 1, 1])
obj.SDO_ORDINATES = ordinateTypeObj.newobject()
obj.SDO_ORDINATES.extend([lon, lat])
print("Created object", obj)
# set up a distance-calculating sql statement
sql = "select id into :id from spatialtbl s where sdo_nn(s.geometry, :obj, 'sdo_num_res=1', 1) = 'TRUE'"
try:
# execute the distance sql
cur.execute(sql, id=id, obj=obj)
print(f'The id is {id.getvalue()}')
except cx_Oracle.Error as error:
print(error)
which results in the error:
ORA-01036: illegal variable name/number
Can anyone tell me what I may be doing wrong code-wise or if spatial queries are even possible using Python and cx_Oracle? The cx_Oracle documentation doesn't specifically address this as far as I can tell/find.

There is a brief mention in the documentation:
Binding Spatial Datatypes
Here are two examples from the cx_Oracle source code repository:
InsertGeometry.py
SpatialToGeoPandas.py
Here's a presentation from the recent Oracle Conference:
Analyzing Location-based Patterns with Python and Oracle Database
The download links are there but may not be obvious pdf and zip.
In your example, you probably need to do at least id = cursor.var(int), see Bind Direction so cx_Oracle knows what to do with the value you are getting from the DB.

I think the "select into" was the problem (reserved for pl/sql?).
By doing the following I was able to obtain the answer:
# set up a distance-calculating sql statement
sql = """select id from spatialtbl s where sdo_nn(s.geometry, :ob, 'sdo_num_res=1', 1) = 'TRUE'"""
try:
# execute the distance sql
cur.execute(sql, ob=obj)
id = cur.fetchone()
print(f'The id is {id}')
except cx_Oracle.Error as error:
print(error)

Python variables in MySQL execute command

I've looked for an answer everywhere and didn't manage to find any suitable one.
This is my code:
conn = pymysql.Connect(host="host", user="user", passwd="password", db="database")
dbhandler = conn.cursor()
table_name = today_date.split(" ")[0]
execute_it = """CREATE TABLE %s (
USERNAME CHAR(20) NOT NULL,
X CHAR(10),
Y INT,
Z INT,
A INT)"""
try:
dbhandler.execute(execute_it, table_name)
except:
print("\n----------------------------\nFailed to create table.")
Now I've tried to do it like this.
I tried with % separating in execute.
I tried with ? instead of %s.
I tried it with many more options and yet none of them worked for me and I failed to create the table
This is the exception I get:
(1064, "You have an error in your SQL syntax; check the manual that
corresponds to your MariaDB server version for the right syntax to use
near ''11/14/18' (\n USERNAME CHAR(20) NOT NULL, \n
X CHAR(10' at line 1")
Using 5.5.52-MariaDB.
Thank you!
EDIT:
Managed to get through it.
Thanks Pavel Francírek for the help.

Problem is not in placeholder, but in date format. Character "/" is not allowed in table name. Try something like:
table_name = today_date.split(" ")[0].replace("/","")
I assume that all numbers in your date format are 2-digit.

using python 2.7 to query sqlite3 database and getting "sqlite3 operational error no such table"

My simple test code is listed below. I created the table already and can query it using the SQLite Manager add-in on Firefox so I know the table and data exist. When I run the query in python (and using the python shell) I get the no such table error
def TroyTest(self, acctno):
conn = sqlite3.connect('TroyData.db')
curs = conn.cursor()
v1 = curs.execute('''
SELECT acctvalue
FROM balancedata
WHERE acctno = ? ''', acctno)
print v1
conn.close()

When you pass SQLite a non-existing path, it'll happily open a new database for you, instead of telling you that the file did not exist before. When you do that, it'll be empty and you'll instead get a "No such table" error.
You are using a relative path to the database, meaning it'll try to open the database in the current directory, and that is probably not where you think it is..
The remedy is to use an absolute path instead:
conn = sqlite3.connect('/full/path/to/TroyData.db')
You need to loop over the cursor to see results:
curs.execute('''
SELECT acctvalue
FROM balancedata
WHERE acctno = ? ''', acctno)
for row in curs:
print row[0]
or call fetchone():
print curs.fetchone() # prints whole row tuple

The problem is the SQL statment. you must specify the db name and after the table name...
'''SELECT * FROM db_name.table_name WHERE acctno = ? '''

Insert data from file into database

I have a .sql file with multiple insert statements ( 1000 + ) and I want to run the statements in this file into my Oracle database.
For now, im using a python with odbc to connect to my database with the following:
import pyodbc
from ConfigParser import SafeConfigParser
def db_call(self, cfgFile, sql):
parser = SafeConfigParser()
parser.read(cfgFile)
dsn = parser.get('odbc', 'dsn')
uid = parser.get('odbc', 'user')
pwd = parser.get('odbc', 'pass')
try:
con = pyodbc.connect('DSN=' + dsn + ';PWD=' + pwd + ';UID=' + pwd)
cur = con.cursor()
cur.execute(sql)
con.commit()
except pyodbc.DatabaseError, e:
print 'Error %s' % e
sys.exit(1)
finally:
if con and cur:
cur.close()
con.close()
with open('theFile.sql','r') as f:
cfgFile = 'c:\\dbinfo\\connectionInfo.cfg'
#here goes the code to insert the contents into the database using db_call_many
statements = f.read()
db_call(cfgFile,statements)
But when i run it i receive the following error:
pyodbc.Error: ('HY000', '[HY000] [Oracle][ODBC][Ora]ORA-00911: invalid character\n (911) (SQLExecDirectW)')
But all the content of the file are only:
INSERT INTO table (movie,genre) VALUES ('moviename','horror');
Edit
Adding print '<{}>'.format(statements) before the db_db_call(cfgFile,statements) i get the results(100+):
<INSERT INTO table (movie,genre) VALUES ('moviename','horror');INSERT INTO table (movie,genre) VALUES ('moviename_b','horror');INSERT INTO table (movie,genre) VALUES ('moviename_c','horror');>
Thanks for your time on reading this.

Now it's somewhat clarified - you have a lot of separate SQL statements such as INSERT INTO table (movie,genre) VALUES ('moviename','horror');
Then, you're effectively after cur.executescript() than the current state (I have no idea if pyodbc supports that part of the DB API, but any reason, you can't just execute an execute to the database itself?

When you read a file using read() function, the end line (\n) at the end of file is read too. I think you should use db_call(cfgFile,statements[:-1]) to eliminate the end line.