I have a python function that takes in a list of people (first name, last name, and date of birth stored in a dict), loops through this list and queries the database person by person, and returns a list containing the results found.
So for example if the user would want to query 3 people using the list [{'name': 'Alice'}, {'name': 'Bob'}, {'name': 'Charlie'}], the function would query the database 3 times, each time getting a list of potential matches that gets put into another list. The could return a list of 3 lists each containing matching results for Alice, Bob and Charlie. The function looks like the following.
query_string = "SELECT * FROM some_db"
def improved_batch_client_check(user_requests, db_connection):
responses = []
for one_request in user_requests:
first_name = one_request.get('FirstName')
last_name = one_request.get('LastName')
dob = one_request.get('BirthDate')
query_string_end = " WHERE db.NAME LIKE '%{}%' AND db.NAME LIKE '%{}%'".format(first_name, last_name)
if dob is None:
pass
else:
query_string_end += " AND db.DOB = '{}'".format(str(dob))
df_candidates = pandas.read_sql(query_string + query_string_end, db_connection)
responses.append(df_candidates.to_dict('records'))
return jsonify(responses)
What I want to do is to improve efficiency by having to call the function pandas.read_sql() only once by somehow submitting a list of different queries that returns dataframes in a similar manner to what is used below. What is the easiest way to do it in Python?
In order to make it one query I would create a query of or's, that is -
select * from table where (request1) or (request2) ...
Each request is of the form if dob is not given or -
db.NAME LIKE '%{}%' AND db.NAME LIKE '%{}%'".format(first_name, last_name)
and if dob is given -
db.NAME LIKE '%{}%' AND db.NAME LIKE '%{}%' AND db.DOB = '{}'".format(first_name, last_name, str(dob))
Putting it all together -
def improved_batch_client_check(user_requests, db_connection):
requests = []
for one_request in user_requests:
first_name = one_request.get('FirstName')
last_name = one_request.get('LastName')
dob = one_request.get('BirthDate')
curr_request = "db.NAME LIKE '%{}%' AND db.NAME LIKE '%{}%'".format(first_name, last_name)
if dob is not None:
curr_request += " AND db.DOB = '{}'".format(str(dob))
requests.append(curr_request)
query_string = "SELECT * FROM some_db WHERE " + 'or '.join([f'({x})' for x in requests])
df = pandas.read_sql(query_string, db_connection)
return jsonify (df.to_dict('records'))
Related
I will try and explain this the best I can. I have a list of lets say ids that I can store in a python list, I need to be able to loop through the list and pass the single id to a update query for SQL.
Here is what I have, I have two separate databases which are hosted on two separate servers with a table in each, one has a column containing a voicemail value and the other needs to be updated with that value based on the ID.
I have a working piece of code right now but you need to provide the ID manually, I would like to store all the IDS in a list and then just loop through it.
my code so far:
def add_vm():
ID = input("ID: ").upper()
checkTwo = 1
while checkTwo == 1:
if len(ID) == 6:
checkTwo = 0
elif len(ID) != 6:
print("\n[bold red]ID entered is incorrect")
ID = input("\nID: ")
promote_tag = "My tag here"
getVoicemail = """select VOICEMAIL from Database1 where ID = ?"""
cursorTwo.execute(getVoicemail, ID)
for row in cursorTwo:
row_to_list5 = [elem for elem in row]
voicemail = str(row_to_list5[0])
voicemailFormat = '+1' + voicemail
update_Query1 = """update table1 set column1 = ?, promote_type = 1, voicemail = ? where ID = ?"""
cursor.execute(update_Query1, promote_tag, voicemailFormat, ID)
conn.commit()
print("Record updated")
while True:
add_vm()
What I would like to do is figure out if its possible to take all my ids that I have and just create a python list then somehow loop through that list providing the id to both the getVoicemail and also updateQuery lines
I have an PyQt5 dialog with a table, 3 buttons (delete row, add row and filter) and 5 comboboxes, in those comboboxes I have all the items written in an external SQL databes (for example I have customers, drivers ecc..), the user can choose an item from each combobox, afetr choosing it when the users press the filter button(the object name is 'bottonefiltra') automatically the table will be filtered based on which customer, driver ecc the user selected. Here's my code and here's where I'm stuck :
def __init__(self):
super(modificaviaggi, self).__init__()
uic.loadUi('modifica_reg.ui', self)
bottonefiltra = self.bottone_filtra
bottonefiltra.clicked.connect(self.filtertable)
def filtertable(self):
self.tabella_registrazioni.clear()
self.tabella_registrazioni.blockSignals(True)
startdate = self.input_dal.date().toPyDate()
enddate = self.input_al.date().toPyDate()
farticle = self.filtro_articolo.currentText()
fwarehouse = self.filtro_magazzino.currentText()
fcustomer = self.filtro_cliente.currentText()
fplate = self.filtro_targa.currentText()
fdriver = self.filtro_autista.currentText()
How can I create a query that changes based on the item selected without creating 30 if statements? I'll explain it better, to do what I want I'd most likely create an if to check which combobox has an item selected like this :
if farticle != " " :
if fwarehouse != " ":
if fcustomer != " " :
if fplate != " " :
if fdriver != " " :
then I'll execute a query where like :
c.execute('SELECT * FROM Viaggi WHERE DataDDT < ? AND DataDDT > ? AND Ragione_sociale = ? AND Targa = ? AND Autista = ? AND Magazzino = ? AND Articolo = ?',
(fcustomer, fplate, fdriver, fwarehouse, farticle))
but everytime I have to write like 10-15 different queries based on the combination of which items are selected(maybe 1 combo has an item selected and third no ecc...). How can I do it without writing 20 queries and without making 20 If statements to check which combo has an item selected ?
A possibility would be to pair each combo with a field name, then cycle through them to build both the query and the value list.
Somewhere in the __init__ create a list with those pairs:
self.comboFields = (
(self.first_combo, 'FirstField'),
(self.second_combo, 'SecondField'),
...
)
Then iterate through it and build the query:
query = 'SELECT * FROM Viaggi WHERE DataDDT < ? AND DataDDT > ?'
values = [startdate, enddate]
for combo, field in self.comboFields:
value = combo.currentText()
if value != ' ':
query += ' AND {} = ?'.format(field)
values.append(value)
c.execute(query, values)
I have a code, below, where a user can put a list of allergies. I want my code to come up with a list of recipes, that exclude the ingredients from the user allergy list. I made a test code but cannot figure out how to exclude all three ingredients at the same time.
userallergy = conn.execute ('SELECT Allergies from User where userid = 4')
userallergy = userallergy.fetchall()
userallergy = userallergy[0][0].replace(" ","")
listallergy=list(userallergy.split(","))
listallergy = ["'%" + i.capitalize() + "%'" for i in listallergy]
print([listallergy])
query='SELECT RecipeName FROM Recipess where Ingredients Not LIKE {}'.format(listallergy[1])
print(query)
aller = conn.execute(query)
saferecipe = aller.fetchall()
print(saferecipe)
Use REGEXP for a clean approach(Though you can't use it directly, check out the answer of Problem with regexp python and sqlite ):
def regexp(expr, item):
reg = re.compile(expr)
return reg.search(item) is not None
conn.create_function("REGEXP", 2, regexp)
allergies = "|".join([i.capitalize() for i in listallergy])
print(allergies)
query='SELECT RecipeName FROM Recipess where Ingredients Not REGXP \'{}\''.format(allergies)
print(query)
A simple approach is to join all the statements by AND operator(but as #alexis mentioned in comments it comes with it's own issues):
allergies = " AND ".join(["Ingredients Not LIKE '%" + i.capitalize() + "%'" for i in listallergy])
print(allergies)
query='SELECT RecipeName FROM Recipess where {}'.format(allergies)
print(query)
You may use MySQL REGEX 'OR' to exclude all three. Your query should then look like this:
query = f"SELECT RecipeName FROM `Recipess` WHERE Ingredients NOT REGEXP '{('|').join(listallergy)}'"
I will read the input into the following web form:
via Flask thus:
def query():
if request.method == "POST":
job_title_input = request.form['title']
description_operator = request.form['DescriptionOperator']
description_input = request.form['description']
salary_operator = request.form['SalaryOperator']
salary_input = request.form['salary']
location_input = request.form['location']
The values received will be used to query this table in SQLite:
CREATE TABLE scrapedjobs
(id integer primary key autoincrement not null,
huburl text,
business_name text,
location text,
jobtitle text,
description text,
joburl text,
salary int,
date_first_scraped date)
It's my first time attempting to handle of the permutations in a web form's submission, and after cascading down through just the first two fields, I can see already that it's is getting convoluted:
base_query = """SELECT * FROM scrapedjobs"""
if job_title_input != '':
base_query = base_query + """ WHERE jobtitle IN ('""" + job_title_input + """')"""
job_title_input_flag = """ AND"""
else:
job_title_input_flag = """"""
if description_input != '':
if job_title_input_flag != '':
base_query = base_query + """ AND description """ + description_operator + " " + "('" + description_input + "')"
description_input_flag = """ AND"""
else:
base_query = base_query + """ WHERE description """ + description_operator + " " + "('" + description_input + "')"
description_input_flag = """ AND"""
else:
description_input_flag = """"""
The further down I go, the more nesting of if/else's will be needed, and it's not sustainable if new fields are added to the form. A major flaw also is that I'm setting things up whereby each field depends on whether the previous fields had a value, and so on. The reality of course is that in a straight query like this without grouping or joins or the like, order is irrelevant, once I can take care of when I need AND (in case a field did have a value) and when I need instead WHERE. Worse still, I will have to parameterize all of this when I pass it to cursor.execute() in sqlite3.
I'm not the first person to every read in values from a multiple form and build appropriate SELECTs - what patterns or more efficient techniques can I take advantage of?
I've been there and I know how it will end. Manually constructing the queries like in your case is highly error-prone, not readable and leads to supporting nightmares. Instead, look into abstracting things away with sqlalchemy and flask-sqlalchemy specifically. With SQLAlchemy you can compose queries in pieces and it will build the final query correctly.
As a real world example, study the Flask Mega-Tutorial (sqlalchemy is used in chapter 4 and later).
Using Python's mysql.connector, how can I select all records that match a tuple?
For example:
ids = (b'6TRsHMzWSrq0zeembo0Vbg',
b'7Qoq53lKTk-RZOi830t3MA',
b'7ZGO9S3DTcGHjwrfpV0E0A')
And then the query would do something like:
query = SELECT first_name
FROM users
WHERE id = (ids)
Returning records for all users whose ID appears in the tuple
Try doing this
query = "SELECT first_name FROM users WHERE id IN " + str(list(ids))
cursor.execute(query)
On second thoughts, the following should also work
query = "SELECT first_name FROM users WHERE id IN " + str(ids)
EDIT
OP mentions in comments that the ids are binary data returned from a previous query. In which case, taking hint from this answer and using BINARY operator, the following query should work
query = "SELECT first_name FROM users WHERE BINARY id IN " + str(ids) ;