I am trying to make query in mongodb where I don't know how many variables I have from the beginning.
You get an array with variables(strings) and then mongo should return via the $or operator so there are no duplicates.
def findRecipes(IngredientsHome):
#Query and return all the alternatives
arrayCompleteSearch = [];
if len(IngredientsHome) > 1:
#theHardStuuff
#Create the query based on ingredientsarray
argsCount = len(IngredientsHome)
argsFunction = "";
for i in range(0, len(IngredientsHome)):
if i < (len(IngredientsHome)-1):
argsFunction += ("{"+'"_Ingredients":' + '"' + IngredientsHome[i] + '"' "}, ")
else:
argsFunction += "{" + '"_Ingredients":' + '"' + IngredientsHome[i] + '"' + "}"
if I then do like this
print("coll.find({" + '$or' + ':' + '[' + argsFunction + ']' "})")
the commandline will print(not execute) the command i want to do in mongodb but I don't know how to make it work in mongo and return the results to an array. I am using pymongo and there i don't if it's possible to use a string as a full query. In sql I could put the whole "query" as a string "SELECT * FROM ..." while pymongo need : outside the string, more like coll.find({"String":"String"}). Is there a similar method or another way to make this query when I don't know how many expressions I have from the beginning in my mongodb query?
Instead of constructing your query as a string, use PyMongo methods to query your database, for example:
coll.find({'_Ingredients': IngredientsHome[i]})
or you can pass the Python list and use $in in your query:
coll.find({'_Ingredients': {'$in': IngredientsHome}})
Read more about how to use PyMongo here.
Related
I have a few parameters typed in a string that need to be extracted & substituted using the string format() function. The code snippet is as follows:
# Filter list (output from another script)
filters ='recipient_id=100, reporting_date=2020-10-12'
# Fetch SQL file from location
sql_file = 'configs/sql_files/{sql_name}.sql'.format(sql_name=sql_name)
file_path = os.path.realpath(__file__)
final_file_path = "/".join(file_path.split("/")[:-2]) + "/" + sql_file
with open(final_file_path) as sql_file:
# Pass in filters to the SQL snippet. The SQL snippet has two parameters
# recipient_id & reporting_date
sql = sql_file.read().format(filters)
try:
sf = get_sql_client()
except Exception as e:
print("Error connecting to the DB!")
sys.exit()
df = sf.fetch_df(sql)
The code snippet fails as "filters" is being passed as a string.
sql_file.read().format('recipient_id=100, reporting_date=2020-10-12')
Instead, it should be passed as below:
sql_file.read().format(recipient_id=100, reporting_date='2020-10-12')
Is there a way to extract the "filters" string & format it as above?
SQL file sample:
SELECT columns..
FROM A
join B on <condition>
WHERE true
AND REPORTING_LEVEL_1_ID = '{recipient_id}'
AND date_trunc('day', delivered_date_pt) >= DATEADD(day, -7, last_day('{reporting_date}'::date, 'week') + 1)
AND date_trunc('day', delivered_date_pt) <= last_day('{reporting_date}'::date, 'week')
try using double quotes like so...
filters ="recipient_id=100, reporting_date='2020-10-12'"
Assuming the sql file with contents like that and for filtering information formatting as shown (as a series of name and value pairs, separated by comma and space characters), you could create dictionary from the latter and then pass it to the format() method as shown (using a ** prefix to unpack the items in it).
filter_data = 'recipient_id=100, reporting_date=2020-10-12'
final_file_path = 'sql_file'
with open(final_file_path) as sql_file:
pairs = (pair.split('=') for pair in filter_data.replace(',', '').split())
mapping = {k: v for (k, v) in pairs}
sql = sql_file.read().format(**mapping)
print(sql)
Output:
SELECT columns..
FROM A
join B on <condition>
WHERE true
AND REPORTING_LEVEL_1_ID = '100'
AND date_trunc('day', delivered_date_pt) >= DATEADD(day, -7, last_day('2020-10-12'::date, 'week') + 1)
AND date_trunc('day', delivered_date_pt) <= last_day('2020-10-12'::date, 'week')
I need to format an SQL query, and it looks like this:
s += " t{}.{} = '{}' and".format(t_c, filter_c, filter_value)
but when the filter_value is something like m's it will result in
psycopg2.errors.SyntaxError: syntax error
if I use the double quote, it will say there's no such column
Any way I can resolve this problem, please?
Caused by injection vulnerability. Use parameters for filter_value and let the database API handle it.
If the table/schema names are coming from user input, whitelist those too. Parameters aren't possible for table names).
try this:
def format_value( string ):
if '\'' in string:
k = string.split('\'')
string_list = list(map(lambda x: '\'' + x+ '\'', k ))
return ''.join(string_list )
else:
return string
then
filter_value = format_value(filter_value)
before you pass filter_value to your query
if year in Year:
#print 'executing'
for rows in range(1,sheet.nrows):
records = []
FIP = str(sheet.cell(rows, 1).value)
for cols in range(9,sheet.ncols):
records.append(str(sheet.cell(rows,cols).value))
cur.execute("UPDATE " + str(table_name) + " SET " + (str(variables[0]) + "= \'{0}\', ".format(records[0])
+ str(variables[1]) + " = \'{0}\', ".format(records[1])
+ str(variables[2]) + " = \'{0}\', ".format(records[2])
+ str(variables[3]) + " = \'{0}\', ".format(records[3])
+ str(variables[4]) + " = \'{0}\',".format(records[4])
+ str(variables[5]) + " = \'{0}\', ".format(records[5])
+ str(variables[6]) + " = \'{0}\' ".format(records[6])+
"WHERE DATA_Year='2010'AND FIPS='{0}'".format(FIP)))
The above code is updating 7 columns whose names are stored in the list 'variables'.
I want to make it dynamic so that if number of elements(columns) in the list 'variables' is increased, it should update all the columns and not just 7.
I tried doing that using this code:
if year in Year:
#print 'executing'
for rows in range(1,sheet.nrows):
records = []
FIP = str(sheet.cell(rows, 1).value)
for cols in range(9,sheet.ncols):
records.append(str(sheet.cell(rows,cols).value))
for x in range(0,len(variables)):
#print x
cur.execute("UPDATE " + str(table_name) + " SET " + (str(variables[x])) + "= \'{0}\', ".format(records[x])
+ "WHERE DATA_Year='2010' AND FIPS='{0}'".format(FIP))
But I am getting the error:
pypyodbc.ProgrammingError: (u'42000', u"[42000] [Microsoft][ODBC SQL Server Driver][SQL Server]Incorrect syntax near the keyword 'WHERE'.")
It would be great if someone can help me figure out what's wrong with my code and whether there is a alternate way of doing what I am trying to do.
You will find it easier to use parameter-substitution. See params, and note that execute takes a sequence argument. Then that line starts to look something like,
cur.execute(sql, records)
If memory permits (it probably does), you may find executemany performs better: you call it once with an array of records.
With that in mind, the dynamic part of your question comes into focus. Construct the parameterized query string as you iterate over cols. When that's done, you should have a matched set (in effect) of parameter placeholders and elements in records. Then tack on the WHERE clause, append FIP to records, and execute.
HTH.
So I am defining a function for use in a ArcGIS tool that will verify attributes, catch errors, and obtain user input to rectify those error. I want the tool to select and zoom to the segment that is being currently assessed so that they can make an informed decision. This is what I have been using, and it works well. But the CONVWGID is the variable that will be changing, and I'm not sure how to input that variable into an SQL statement without causing errors.
This is how I had tested the logic:
def selectzoom():
arcpy.SelectLayerByAttribute_management(Convwks, "NEW_SELECTION", " [CONVWGID] = 10000001")
mxd = arcpy.mapping.MapDocument('CURRENT')
df = arcpy.mapping.ListDataFrames(mxd, "Layers") [0]
df.zoomToSelectedFeatures()
arcpy.RefreshActiveView()
Then I needed to work the variable into the function in order to accept different CONVWGID values, which gives me a Runtime/TypeError that I should have known would happen.
Runtime error -
Traceback (most recent call last): - File "string", line 1, in module - TypeError: cannot concatenate 'str' and 'int' objects
def selectzoom(convwkgid):
delimfield = '" [CONVWGID] = ' + convwkgid + ' "'
arcpy.SelectLayerByAttribute_management(Convwks, "NEW_SELECTION", delimfield)
mxd = arcpy.mapping.MapDocument('CURRENT')
df = arcpy.mapping.ListDataFrames(mxd, "Layers") [0]
df.zoomToSelectedFeatures()
arcpy.RefreshActiveView()
And when I alter the delimfield line to change the integer into a string, it selects all of the attributes in the entire feature class. Not just the one that had been passed via the function call.
delimfield = '"[CONVWGID] = ' + str(convwkgid) + '"'
I'm not amazing with SQL and maybe I'm missing something basic with this statement, but I can't figure out why it won't work when I'm basically giving it the same information:
"[CONVWGID] = 10000001"
'"[CONVWGID] = ' + str(convwkgid) + '"'
It turned out to be the extra inclusion of Double quotes inside of my single quotes that raised this problem.
Thanks to #Emil Brundage for the help!
Let's say convwkgid = 10000001
'"[CONVWGID] = ' + str(convwkgid) + '"' doesn't equal "[CONVWGID] = 10000001"
'"[CONVWGID] = ' + str(convwkgid) + '"' would actually be '"CONVWGID] = 10000001"'
Try instead:
'[CONVWGID] = ' + str(convwkgid)
I want to write mulitiple values in a text file using python.
I wrote the following line in my code:
text_file.write("sA" + str(chart_count) + ".Name = " + str(State_name.groups())[2:-3] + "\n")
Note: State_name.groups() is a regex captured word. So it is captured as a tuple and to remove the ( ) brackets from the tuple I have used string slicing.
Now the output comes as:
sA0.Name = GLASS_OPEN
No problem here
But I want the output to be like this:
sA0.Name = 'GLASS_HATCH_OPENED_PROTECTION_FCT'
I want the variable value to be enclosed inside the single quotes.
Does this work for you?
text_file.write("sA" + str(chart_count) + ".Name = '" + str(State_name.groups())[2:-3] + "'\n")
# ^single quote here and here^