I'm new in Python and Mysql.
I'm just trying to learn and understand how SQL and Python works together.
I'm trying to build up a database connection via SqlPool. I found the related code here:
class MySQLPool(object):
.
.
.
def execute(self, sql, args=None, commit=False):
""" Execute a sql """
# get connection form connection pool.
conn = self.pool.get_connection()
cursor = conn.cursor()
if args:
cursor.execute(sql, args) #so if there is args in the sql select then it should be run like this: (select * from table , args)
else:
cursor.execute(sql) #if there is no args then it will run the simple select statement
if commit is True:
conn.commit()
self.close(conn, cursor)
return None
else:
res = cursor.fetchall() #else it will fetch all the records
self.close(conn, cursor)
return res
Then in the main.py I'm trying to use this def:
l = 'saxonb'
def testing(request, bid, *args, **kwargs):
mysql_pool = MySQLPool()
query = """select AZON, LOGIN, JELSZO from tmrecords.luzer WHERE LOGIN='?'"""
result = mysql_pool.execute(query)
print('RESULT : \n', result)
query = """select AZON, LOGIN, JELSZO from tmrecords.luzer WHERE LOGIN='?'"""
testing(MySQLPool, query, l)
It should be found 1 record but it is coming back with an empty result.
I have tried several placeholders like ? or %s but it does not work properly and I can't see what the problem is.
It might be a del calling problem? I mean I try to call the def incorrectly?
Or placeholder problem?
Can anyone help me in this? I would really appreciate any helps. Thanks. Hexa
Related
I'm really new with Python and currently I'm trying to understand how it works and I'm building small codes for practising.
I have a table contains the details of the users: called luzer.
I want to update the password field (called: JELSZO in the table) is a varbinary (1000).
The MysqlDB connection works via a pool.
So, I use this def to execute Sql queries and commit updates etc...
def execute(self, sql, args=None, commit=False):
""" Execute a sql """
# get connection form connection pool.
conn = self.pool.get_connection()
cursor = conn.cursor()
if args:
cursor.execute(sql, args)
else:
cursor.execute(sql)
if commit is True:
conn.commit()
self.close(conn, cursor)
return None
else:
res = cursor.fetchall()
self.close(conn, cursor)
return res
And this is how I try to update the password field (JELSZO)
sql_update_query = "Update luzer SET JELSZO = %s where AZON = %s" #the AZON is the userid in the table.
pas2 = testing(MySQLPool, sql_update_query, (jelszoid1, loginid, ), True) #if the commit = True then it should run the conn.commit() above.
It runs without any error but when I try to check if it commited the update succesfully then I see that nothing happend.
The password is a binary string (generated using Fernet Key).
I would really appreciate if you have any idea what could go wrong here?
I solved this bloody issue. I used a different named def for it.
def update(request, bid, *args, **kwargs):
mysql_pool = MySQLPool()
query = sql_update_query
result = mysql_pool.execute(query, (jelszoid1, loginid,), True)
print('RESULT : \n', result)
return result
while the original (def testing now I renamed to 'selection') function stayed as it was:
def selection(request, bid, *args, **kwargs):
mysql_pool = MySQLPool()
query = query1
result = mysql_pool.execute(query, (loginid,), False)
print('RESULT : \n', result)
return result
As you can see the only difference is between the parameters of the function call.
I was stupid because I disregarded the basic sql rule that the select queries do not need commit.
From Python 3.9 i'm trying to do a MySql query like this
select * from brand WHERE text='L\'Orial'.
It works fine from phpMyAdmin but fails from python for all text including quote "'"
brand = "L'Orial"
where = f"text='{brand}'"
brand_pk_id = self.getPrimaryKeyIfExistInTable('brand', where)
def getPrimaryKeyIfExistInTable(self, table, where, key='id'):
try:
sql = f"SELECT {key} FROM {table} WHERE {where}"
self.cursor.execute(sql)
result = self.cursor.fetchone()
return result[key if self.bUseDictCursor else 0] if result else None
except pymysql.MySQLError as e:
logging.error(e)
return None
I can see that python escapes all quotes, which probably causes the problem, but can not figure out how to handle it properly !!
If I turn it around and use query LIKE with underscore( _ ) as wildcard:
brand = "L_Orial"
sql = f"SELECT {key} FROM {table} WHERE text LIKE '{brand}'"
It works fine, but this is not what I want !!
If I am understanding your question correctly, your problem is as follows:
Your query must exactly read:
SELECT * from brand WHERE text='L\'Orial'
But you are currently getting something like this, when you use python to execute the query:
SELECT * from brand WHERE text='L'Orial'
If this is indeed the issue, you should be able to resolve this by simply escaping the backslash that you need to have in the query. The complete python string for your query would be:
# Python String:
"SELECT * from brand WHERE text='L\\'Orial'"
# Resulting Query
SELECT * from brand WHERE text='L\'Orial'
If you wanted to automatically fix this issue for all brands that might include a ', you can simply replace the ' with \\' before making the query. Example:
brand = "L'Orial"
brand = brand.replace("'", "\\'")
# New Python string:
# "L\\'Orial"
# Output in SQL
# "L\'Orial"
Had to fire up my local instance just to make a point.
First, some prep work...
import pymysql
table = 'ps_carrier'
key = 'id_carrier'
mysql = {
"charset": "utf8",
"database": "mystore",
"host": "localhost",
"password": "secret",
"user": "justin"
}
As somebody suggested in the comments, the following
sql = "SELECT %s FROM %s WHERE %s"
where = "name='UPS'"
with pymysql.connect(**mysql) as conn:
with conn.cursor() as cur:
cur.execute(sql, (key, table, where))
Raises an error as expected since all the (string) params are quoted, even the table name!
Traceback (most recent call last):
File "<stdin>", line 3, in <module>
...
File "C:\Python38\site-packages\pymysql\err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
pymysql.err.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ''ps_carrier' WHERE 'name=\\'UPS\\''' at line 1")
If you can trust the inputs for the table name, the key, and the column name(s) then perhaps a simple query builder can help.
params = {'name': 'UPS'} # add more key--value pairs here
# use backticks in case we need to escape reserved words (OP uses MySQL)
where = " AND ".join(f"`{k}` = %s" for k in params.keys()) # .keys() just to be explicit
args = tuple([v for v in params.values()])
# backticks again
sql = f"SELECT `{key}` FROM `{table}` WHERE {where}"
print(sql)
print(args)
with pymysql.connect(**mysql) as conn:
with conn.cursor() as cur:
cur.execute(sql, args)
print(cur.fetchall())
If you need something more elaborate, there are a few modules such as Mysql Simple Query Builder and PyPika - Python Query Builder that you may want to look at (I've not used any of these.)
I am writing a generic function to query a database where I want to use *args to get values. Here's the function
def get_latest_value(col_name, *args):
query = "select * from latest_value where {column} in (%s)"
query = query.format(**dict(column=col_name))
results = pd.read_sql_query(query, engine, params = [args])
return(results)
However, when I run this, I get this error:
ProgrammingError: (MySQLdb._exceptions.ProgrammingError) not all arguments converted during bytes formatting
[SQL: select * from latest_value where amfi_code in (%s)]
[parameters: (100370, 100371)]
(Background on this error at: http://sqlalche.me/e/f405)
I even tried converting the *args to a list before passing it, but that didn't help either.
Not able to figure out how to pass the values in *args to the IN clause. Any help is appreciated.
Also, is using string concatenation for column name this ways safe from sql injection attacks?
Update:
I changed the code as follows:
def get_latest_value(col_name, *args):
query = "select * from latest_value where {column} in (%s)"
query = query.format(**dict(column=col_name))
conditions = ", ".join(map(str, args))
results = pd.read_sql_query(query, engine, params = [conditions])
return(results)
I don't get an error now, but I only get the results for the first value in args.
I could not figure out exactly what, but I figured that sqlalchemy has a problem dealing with this scenario. It can be solved by using psycopg2 instead.
I wrote the code as follows
conn = psycopg2.connect(user='', host='', password='', database='')
query = "select * from latest_value where {column} = any(%s)"
query = query.format(**dict(column=col_name))
results = pd.read_sql_query(query, conn, params=(args,))
Note the change in the query. Instead of using in (%s), I have used =any(%s) as seen in the psycopg2 documentation.
Also note that for the above to work, args needs to be a Python list, not a tuple.
I am writing a python script to search if an element pre-exists in my sql database.
def create_connection(db_file):
try:
conn = sqlite3.connect(db_file)
print(sqlite3.version)
except Error as e:
print(e)
main(conn) #function with inserts creates table and inserts values and calls find_values() function
def find_values(conn):
sql = ''' SELECT link_ID from links_table where link_ID="l_1234" '''
conn.execute(sql)
conn.commit()
Here I am comparing to something which I have already entered in my code like "l_1234". How will I compare it to something which is dynamic like a user entered variable?
Also how will I find if the element was found?
Have a look to this post :
How do I use SQL parameters with python?
or this one :
How to use variables in SQL statement in Python?
From one of the last post:
# Do this instead
t = ('RHAT',)
c.execute('SELECT * FROM stocks WHERE symbol=?', t)
print c.fetchone()
I want to execute the sql_query given in below function and doing this way :
def get_globalid(self):
cursor = self.connect()
sql_query = "REPLACE INTO globalid (stub) VALUES ('a'); SELECT LAST_INSERT_ID() as id"
cursor = self.query(sql_query, cursor)
for row in cursor:
actionid = row[0]
return actionid
connect() function I am using to make a connection and it is defined separately .The query function i am using above is to execute any query passed to it . It is defined as :
def query(self,query,cursor):
cursor.execute(query)
self.close()
return cursor
It is not working properly . Is there anything I am missing ? Is there any mysqli function for python like that in php (multi_query) ?
mysql-python can't execute semicolon separated multiple query statement.
If you are looking for last_insert_id you can try with this:
conmy = MySQLdb.connect(host, user, passwd, db)
cursor = conmy.cursor()
sql_query = "REPLACE INTO globalid (stub) VALUES ('a')"
cursor.execute(sql)
last_insert_id = conmy.insert_id()
conmy.close()