We Keep Coding

Python: sqlite3.OperationalError: near "<": syntax error (updating sqlite3 field with html source code)

I would like to change all the occurence of sometext (matching all case combination like sometext, SOMETEXT, SOmeTExt) in the note field of the table itemNotes of a sqlite.
Because sqlite doesn't support case insensitive update (I already ask 2 question here and here ), I am using python and regex.
But I get this error : sqlite3.OperationalError: near "<": syntax error matching line of cursor.execute(f'REPLACE
I dont' have a <in my code so I think it's coming from the note field which contains html source code.
Here is my code:
keyword ="sometext"
replacement_word="abc"
# load sqlite3
db = sqlite3.connect(path_to_sqlite)
cursor = db.cursor()
# search for all therm
cursor.execute(f'SELECT * FROM itemNotes WHERE note like "%{keyword}%"')
print("\nfetch one:")
# itemID_list = cursor.fetchone()
# pour chacun des result, change
for row in cursor.fetchall():
# print(each)
row_regex = re.compile(re.escape(keyword), re.IGNORECASE)
row_regex_replaced = row_regex.sub(replacement_word, row[2])
rowindex = row[0]
cursor.execute(
f'REPLACE INTO itemNotes (note) VALUES ({row_regex_replaced}) where itemID = {rowindex}')
After looking up "sql injection", this is what I came up with :
sql = "REPLACE INTO itemNotes (note) VALUES (?) where itemID = (?)"
data = (row_regex_replaced, rowindex,)
cursor.execute(sql, data)
But now I am getting this error : sqlite3.OperationalError: near "where": syntax error

From the sqlite doc:
The REPLACE command is an alias for the "INSERT OR REPLACE" variant of
the INSERT command.
An INSERT doesn't have a WHERE clause. The query needs to be written as a "regular" INSERT, and the system will "decide" whether to replace. Something like INSERT into itemNotes (itemID,note) VALUES (?,?). (NB the order of the data list in the above example would need to change).

Using python psychopg2 to get a count on all variables in the database that contain a specific string

I have a list, words = [word1, word2, word3, ...]
I want to use sql to return the number of times each word appears in Column A of an sql file. I can't figure out how to pass a variable into my sql query. Any help would be appreciated! My code so far looks like:
import psycopg2 as sql
for word in words
conn = sql.connect(**params)
c = conn.cursor()
#Create query and parameters to get usernames and ids
Query = """ SELECT COUNT(Column A) FROM file
WHERE Column A SIMILAR TO '% **VARIABLE WORD** %'
LIMIT 1000; """
try:
c.execute(Query)
except:
conn.commit()
print("Error in Query")
Result = c.fetchall()
Also, will this count return the total number of times the word appears or just the number of lines of column A in which it appears? (Will the count of the in "the team won the game" return one or two?)

The replaceable parameter flag used by psycopg2 is "%s", and to use a plain "%" in a query with replaceable parameters you need to double it (i.e., "%%"). So your code should look like:
Query = """SELECT COUNT(Column_A) FROM file
WHERE Column_A SIMILAR TO '%%%s%%'
LIMIT 1000;"""
try:
c.execute(Query, word)
This should return the number of lines in which the word appears, not the total number of occurrences of the word in all lines.
Your example has a space in the column name used; I've substituted an underscore, but if the column name really contains a space, the name should be double-quoted in this query.

Give a list of strings to a sql-query [Python]

I use Python2.7 on Windows 7 and a mysql server, connection by pymssql.
My Problem: I have a very big Database and I like to select the ID's of objects matching one of several words(string) from a list, I give to my program.
In this query there must be a LIKE %...% expression for these words of my list, too.
So far I connected my Python-Script to my Database and defined a cursor.
Then I made a small list with the words, I am searching for and I created some placeholders for my query later:
wortliste = ['Bruch', 'Verwerfung']
placeholders = ','.join(['%s'] * len(wortliste))
Here is my Query:
query = """ SELECT BO_INDEX FROM GeoTest.dbo.Tabelle_Bohrung
WHERE BO_BEMERKUNG IN ({})""".format(placeholders)
When I am searching for a single word, here for example for the word 'Bruch', my query would look like this:
query = """ SELECT BO_INDEX FROM GeoTest.dbo.Tabelle_Bohrung
WHERE BO_BEMERKUNG LIKE '%Bruch%'"""
This query for a single word matches the right Id's (=BO_INDEX).
The query with the placeholders doesn't crash, but it didn't match anything :(
But I like to loop my database for a couple of words and append the matching ID's for every word (string) in my list(=wortliste) and append it to a new list.
I really dont't know how to solve this problem!
I am grateful for every new way to solve this challenge!
Thanks!

EDIT 2:
If you want to loop over your list and append to the output (using your example):
words = ['ab', 'cd', 'ef']
abfrage_list = []
for w in words:
# Generate a query
query = """ SELECT BO_INDEX FROM GeoTest.dbo.Tabelle_Bohrung
WHERE BO_BEMERKUNG LIKE '%%%s%%' """ % w
# Execute that query and get results
cur.execute(query)
result_all = cur.fetchall()
# Add those results to your final list
for i in result_all:
abfrage_list.append(i)
EDIT:
For your example with multiple likes:
query = """ SELECT BO_INDEX FROM GeoTest.dbo.Tabelle_Bohrung
WHERE BO_BEMERKUNG LIKE '%ab%'
OR O_BEMERKUNG LIKE '%cd%'
OR O_BEMERKUNG LIKE '%ef%' """
query = """ SELECT BO_INDEX FROM GeoTest.dbo.Tabelle_Bohrung
WHERE {params}""".format(
params=" OR ".join("BO_BEMERKUNG LIKE '%%%s%%' \n" % w for w in wortliste)
)
print(query)
Prints:
SELECT BO_INDEX FROM GeoTest.dbo.Tabelle_Bohrung
WHERE BO_BEMERKUNG LIKE '%Bruch%'
OR BO_BEMERKUNG LIKE '%Verwerfung%'
Your placeholders doesn't contain any of the items from your word list, use:
placeholders = ','.join("'%s'" % w for w in wortliste)
For example:
wortliste = ['Bruch', 'Verwerfung']
print(','.join(['%s'] * len(wortliste)))
print(','.join("'%s'" % w for w in wortliste))
Prints:
%s,%s
'Bruch','Verwerfung'

for the Example of the following list = ['ab','cd','ef']
query = """ SELECT BO_INDEX FROM GeoTest.dbo.Tabelle_Bohrung
WHERE BO_BEMERKUNG LIKE '%ab%'
OR O_BEMERKUNG LIKE '%cd%'
OR O_BEMERKUNG LIKE '%ef%' """
cur.execute(query)
result_all = cur.fetchall()
abfrage_list = []
for i in result_all:
abfrage_list.append(i)
But I need this procedure for possibly hundreds of strings in this list.
i need to loop over this list and i need the LIKE expression in the query, otherwise it won't catch anything.

Google search from Python program

I'm trying to take an input file, read each line, search google with that line and print all the search results from the query ONLY IF the result is from a specific website. A simple example to illustrate my point, if I search dog I only want results printed from wikipedia, whether that be one result or ten results from wikipedia. My problem is I've been getting really weird results. Below is my Python code which contains a specific URL I want results from.
My program
inputFile = open("small.txt", 'r') # Makes File object
outputFile = open("results1.txt", "w")
dictionary = {} # Our "hash table"
compare = "www.someurl.com/" # urls will compare against this string
from googlesearch import GoogleSearch
for line in inputFile.read().splitlines():
lineToRead = line
dictionary[lineToRead] = [] #initialzed to empty list
gs = GoogleSearch(lineToRead)
for url in gs.top_urls():
print url # check to make sure this is printing URLs
compare2 = url
if compare in compare2: #compare the two URLs, if they match
dictionary[lineToRead].append(url) #write out query string to dictionary key & append EACH url that matches
inputFile.close()
for i in dictionary:
print i # this print is a test that shows what the query was in google (dictionary key)
outputFile.write(i+"\n")
for j in dictionary[i]:
print j # this print is a test that shows the results from the query which should look like correct URL: "www.medicaldepartmentstore.com/..."(dictionary value(s))
outputFile.write(j+"\n") #write results for the query string to the output file.
My output file is incorrect, the way it's supposed to be formatted is
query string
http://www.
http://www.
http://www.
query string
http://www.
query string
http://www.medical...
http://www.medical...

Can you limit the scope of the results to the specific site (e.g. wikipedia) at the time of the query? For example, using:
gs = GoogleSearch("site:wikipedia.com %s" % query) #as shown in https://pypi.python.org/pypi/googlesearch/0.7.0
This would instruct Google to return only the results from that domain, so you won't need to filter them after seeing the results.

I think #Cahit has the right idea. The only reason you would be getting lines of just the query string is because the domain you were looking for wasn't in the top_urls(). You can verify this by checking if the array contained in the dictionary for a given key is empty
for i in dictionary:
outputFile.write("%s: " % str(i))
if len(dictionary[i]) == 0:
outputFile.write("No results in top_urls\n")
else:
outputFile.write("%s\n" % ", ".join(dictionary[i]))

Read txt file and add a variable to the text - Python

In order to simplify some of my code I have decided to move queries and HTML code to txt files. However, an issue has come up: most of my queries and HTML that I normally keep inside the code have variable in the middle. For example, I have this in my code:
count = 0
for x in reviewers:
query = """select *
from mytable
where reviewer = """ + reviewers[count]
cur.execute(query)
count = count + 1
#do more stuff
The question is, how do I save queries or HTML code in txt files and then add variables in the middle of the strings?
Thanks!!

Ok so here is the solution I came up with I hope it helps
So you can save the Queries in text files in the form
SELECT * from %s where id = %d
And once you get the query you can place your variable in it. I am assuming that I already got the query from file.
query = "SELECT * from %s where id = %d"
completeQuery=query% ('myTable', 21)
print completeQuery
The output will be
SELECT * from myTable where id = 21
Reference

I'm still not sure what you want, Here's a way to read a file and add a variable name in the text
query = ""
f = open("query_file",'r')
query = f.read() # read the query file into a string
f.close()
for x in reviewers:
query = query+reviewers[count] # add variable name in the string assuming reviewers[count] gives a string
cur.execute(query)
count = count + 1
#do more stuff
EDIT
An important point strings in Python are immutable
if you want to modify string then you'd have to create a new string
for e.g
query = "Select from Table"
you want to make it Select Col from Table
here is what you do:-
add_me = "Col"
new_string = query[:-10] + add_me + query[6:]
now new_string string will have Select Col from Table