I'm trying to search a database, by iterating through a list of search values. I'm almost there as this works for integers but not strings. The code below won't work, but if I replace the list values with numbers it does:
mycursor = mydb.cursor()
lst = []
select_query = "SELECT * FROM fruit WHERE content LIKE "
ids = ["apple", "pear"]
for x in ids:
var = select_query + str(x)
mycursor.execute(var)
lst.extend(mycursor.fetchall())
print(lst)
It's because you have to enclose strings in quotation marks in SQL. So for example
SELECT * FROM fruit WHERE content LIKE 'pears'
will work, and it will only work with the single quotations around "pears". Even better, type conversion can (and should) be done automatically with psycopg2:
select_query = "SELECT * FROM fruit WHERE content LIKE %s"
...
mycursor.execute(select_query, (x,))
https://www.psycopg.org/docs/usage.html#strings-adaptation
Related
Hello and thank you for taking the time to read this. I want to extract large amounts of text from one database to another. The problem is that when I read the text from the database it contains a lot of "\N" "\n". I would really appreciate it if someone could point me in the right direction or tell what is going on here exactly and why the command fetchall is behaving like this...
Thank you in advance!
Example of the text in the TABLE:ourdata (I want to read from):
Example of text in the TABLE product_d I am writing to:
This is the code I am using:
import sqlite3
database_path = "E:/Thesis stuff/Python/database/"
conn = sqlite3.connect(database_path + 'test2.db')
c = conn.cursor()
current_number = 0
# creates the table to which the descriptions go
c.execute("CREATE TABLE IF NOT EXISTS product_d (product_description TEXT)")
conn.commit()
c.execute("SELECT description FROM ourdata")
text_to_format = c.fetchall()
print(text_to_format[0])
text_list = []
# make a list of the descriptions
for text in text_to_format:
text = str(text)
text_list.append(text)
# put all the elements of the list into the new table
for item in text_list:
c.execute("INSERT INTO product_d (product_description) VALUES (?)", (text_list[current_number],))
print("at number " + str(current_number))
current_number += 1
conn.commit()
c.close()
conn.close()
The giveaway is the parentheses around the string in the second example. fetchall() returns a list of tuples, and in your "make a list of the descriptions" block, you're explicitly converting those tuples to strings. Instead, what you want to do is simply grab the first (and only, in this case) element of the tuple. It should be as simple as changing this line:
text = str(text)
to this:
text = text[0]
I have a list in Python i.e:
['E/123', 'E/145']
I want to add this to a SQL statement that I made:
WHERE GrantInformation.GrantRefNumber LIKE 'E/123'
OR GrantInformation.GrantRefNumber LIKE 'E/145'
The code I have is:
items = []
i = 0
while 1:
i += 1
item = input('Enter item %d: '%i)
if item == '':
break
items.append(item)
print(items)
refList = " OR GrantInformation.GrantRefNumber LIKE ".join(items)
The problem is, when I insert that String into my SQL it is a String so it is looking for WHERE GrantInformation.GrantRefNumber Like 'ES/P004355/1 OR GrantInformation.GrantRefNumber LIKE ES/P001452/1'
Which obviously returns nothing as 'ES/P004355/1 OR GrantInformation.GrantRefNumber LIKE ES/P001452/1' does not appear in the field.
How do i do it so the ' GrantInformation.GrantRefNumber LIKE ' is not a String?
Thank you
The preferred way to do this, is to use a ORM like SQLAlchemy, which
does the query construction for you and you dont have to make the string concentrations yourself.
join(), adds the string between all the items in the array, that is passed as argument. You would need to add the condition into the array as well:
>>> items = ['A', 'B']
>>> " OR ".join(["GrantInformation.GrantRefNumber LIKE '%s'" % num for num in items])
"GrantInformation.GrantRefNumber LIKE 'A' OR GrantInformation.GrantRefNumber LIKE 'B'"
I use Python2.7 on Windows 7 and a mysql server, connection by pymssql.
My Problem: I have a very big Database and I like to select the ID's of objects matching one of several words(string) from a list, I give to my program.
In this query there must be a LIKE %...% expression for these words of my list, too.
So far I connected my Python-Script to my Database and defined a cursor.
Then I made a small list with the words, I am searching for and I created some placeholders for my query later:
wortliste = ['Bruch', 'Verwerfung']
placeholders = ','.join(['%s'] * len(wortliste))
Here is my Query:
query = """ SELECT BO_INDEX FROM GeoTest.dbo.Tabelle_Bohrung
WHERE BO_BEMERKUNG IN ({})""".format(placeholders)
When I am searching for a single word, here for example for the word 'Bruch', my query would look like this:
query = """ SELECT BO_INDEX FROM GeoTest.dbo.Tabelle_Bohrung
WHERE BO_BEMERKUNG LIKE '%Bruch%'"""
This query for a single word matches the right Id's (=BO_INDEX).
The query with the placeholders doesn't crash, but it didn't match anything :(
But I like to loop my database for a couple of words and append the matching ID's for every word (string) in my list(=wortliste) and append it to a new list.
I really dont't know how to solve this problem!
I am grateful for every new way to solve this challenge!
Thanks!
EDIT 2:
If you want to loop over your list and append to the output (using your example):
words = ['ab', 'cd', 'ef']
abfrage_list = []
for w in words:
# Generate a query
query = """ SELECT BO_INDEX FROM GeoTest.dbo.Tabelle_Bohrung
WHERE BO_BEMERKUNG LIKE '%%%s%%' """ % w
# Execute that query and get results
cur.execute(query)
result_all = cur.fetchall()
# Add those results to your final list
for i in result_all:
abfrage_list.append(i)
EDIT:
For your example with multiple likes:
query = """ SELECT BO_INDEX FROM GeoTest.dbo.Tabelle_Bohrung
WHERE BO_BEMERKUNG LIKE '%ab%'
OR O_BEMERKUNG LIKE '%cd%'
OR O_BEMERKUNG LIKE '%ef%' """
query = """ SELECT BO_INDEX FROM GeoTest.dbo.Tabelle_Bohrung
WHERE {params}""".format(
params=" OR ".join("BO_BEMERKUNG LIKE '%%%s%%' \n" % w for w in wortliste)
)
print(query)
Prints:
SELECT BO_INDEX FROM GeoTest.dbo.Tabelle_Bohrung
WHERE BO_BEMERKUNG LIKE '%Bruch%'
OR BO_BEMERKUNG LIKE '%Verwerfung%'
Your placeholders doesn't contain any of the items from your word list, use:
placeholders = ','.join("'%s'" % w for w in wortliste)
For example:
wortliste = ['Bruch', 'Verwerfung']
print(','.join(['%s'] * len(wortliste)))
print(','.join("'%s'" % w for w in wortliste))
Prints:
%s,%s
'Bruch','Verwerfung'
for the Example of the following list = ['ab','cd','ef']
query = """ SELECT BO_INDEX FROM GeoTest.dbo.Tabelle_Bohrung
WHERE BO_BEMERKUNG LIKE '%ab%'
OR O_BEMERKUNG LIKE '%cd%'
OR O_BEMERKUNG LIKE '%ef%' """
cur.execute(query)
result_all = cur.fetchall()
abfrage_list = []
for i in result_all:
abfrage_list.append(i)
But I need this procedure for possibly hundreds of strings in this list.
i need to loop over this list and i need the LIKE expression in the query, otherwise it won't catch anything.
In order to simplify some of my code I have decided to move queries and HTML code to txt files. However, an issue has come up: most of my queries and HTML that I normally keep inside the code have variable in the middle. For example, I have this in my code:
count = 0
for x in reviewers:
query = """select *
from mytable
where reviewer = """ + reviewers[count]
cur.execute(query)
count = count + 1
#do more stuff
The question is, how do I save queries or HTML code in txt files and then add variables in the middle of the strings?
Thanks!!
Ok so here is the solution I came up with I hope it helps
So you can save the Queries in text files in the form
SELECT * from %s where id = %d
And once you get the query you can place your variable in it. I am assuming that I already got the query from file.
query = "SELECT * from %s where id = %d"
completeQuery=query% ('myTable', 21)
print completeQuery
The output will be
SELECT * from myTable where id = 21
Reference
I'm still not sure what you want, Here's a way to read a file and add a variable name in the text
query = ""
f = open("query_file",'r')
query = f.read() # read the query file into a string
f.close()
for x in reviewers:
query = query+reviewers[count] # add variable name in the string assuming reviewers[count] gives a string
cur.execute(query)
count = count + 1
#do more stuff
EDIT
An important point strings in Python are immutable
if you want to modify string then you'd have to create a new string
for e.g
query = "Select from Table"
you want to make it Select Col from Table
here is what you do:-
add_me = "Col"
new_string = query[:-10] + add_me + query[6:]
now new_string string will have Select Col from Table
I'm a newbie to Python. I have the code below which basically extracts some 5 rows from a MySql table.
if keyword.lower() == 'last5':
cursor.execute('SELECT income_Ref, income_Amount FROM model_income WHERE company_MobileNumber = %s ORDER BY income_Datetime DESC LIMIT 5', [sender])
result = cursor.fetchall()
result = [list(row) for row in result]
self.respond(result)
return ()
The self.respond() kinda works like print() but it sends a text message instead.I am having trouble formatting the output. What I get from the code above is a list of lists which looks likes this:
[[u'2014-11-06fd753b-inc', u'50.0'], [u'2014-11-067d724b-inc', u'50.0'], [u'2014-11-067557d6-inc', u'50.0']]
I really wish I new how to make it look like this:
Ref:2014-11-06fd753b-inc Amount:50.0
Ref:2014-11-067d724b-inc Amount:50.0
Ref:2014-11-067557d6-inc Amount:50.0
That includes prefixing 'Ref' and 'Amount' before each respective field. I swear I have searched high and low on how to do that for a week now but I'm failing to crack it.
You can use:
self.respond("\n".join("Ref:{} Amount:{}".format(ref, amt) for ref, amt in result))
Not really beginner material, but...
Joining a string (created by formatting "Ref:{} Amount:{}") with "\n" using a generator
if keyword.lower() == 'last5':
cursor.execute('SELECT income_Ref, income_Amount FROM model_income WHERE company_MobileNumber = %s ORDER BY income_Datetime DESC LIMIT 5', [sender])
result = cursor.fetchall()
result = [list(row) for row in result]
self.respond("\n".join("Ref:{} Amount:{}".format(ref, amt) for ref, amt in result))
return ()
I left the list comprehension result = [list(row) for row in result] because I don't know what fetchall() actually returns.
Output:
Ref:2014-11-06fd753b-inc Amount:50.0
Ref:2014-11-067d724b-inc Amount:50.0
Ref:2014-11-067557d6-inc Amount:50.0
["Ref:" + i[0][2:-1] + " Amount: " + i[1][2:-1] + "\n" for i in result]