We Keep Coding

how to force the code to parse specific POST call response

I'm making three post calls one by one and getting tree response. My code is always taking last response and passing it on. The question is how to force the code to parse second or first response not last one.
This is my method with for loop and range3.
def create_card_on_the_board_call(self, id_list, card_count, card_name):
global create_card
for i in range(card_count):
create_card = self.rest_api_helper.post_call(
self.base_url + '/cards?' + 'key=' + test_config[
'key'] + '&' + 'token=' + test_config[
'token'] + '&' + f'idList={id_list}' + '&' + f'name={card_name}', headers=None)
return create_card.json()
response looks like that received response {"id":"5f9dcffaa96f144d311deaa5","checkItemStates":[]}
I need id from first or second response not third. Also I need all three calls.
and this is how I'm calling the post
card_count = 3
card_name = 'someCard'
create_card_on_the_board = api_board.create_card_on_the_board_call(id_list=create_list_on_the_board["id"],
card_count=card_count, card_name=card_name)
now my method is looking like this:
def create_card_on_the_board_call(self, id_list, card_count, card_name):
global create_card
responses = []
for i in range(card_count):
create_card = self.rest_api_helper.post_call(
self.base_url + '/cards?' + 'key=' + test_config[
'key'] + '&' + 'token=' + test_config[
'token'] + '&' + f'idList={id_list}' + '&' + f'name={card_name}', headers=None)
responses.append(create_card)
return responses[2].json()
and I could select which card I want to pass but still dont know how to use as a proper argument in next call? e.g.
new_card_name = 'NewCardName2'
update_card = api_board.update_card_call(card_id=create_card_on_the_board['id'], new_card_name=new_card_name)
def update_card_call(self, card_id, new_card_name):
update_card = self.rest_api_helper.put_call(
self.base_url + '/cards/' + f'{card_id}' + '?' + 'key=' + test_config[
'key'] + '&' + 'token=' + test_config[
'token'] + '&' + f'name={new_card_name}', headers=None)
return update_card.json()
so here I want to use different create_card_on_the_board but I dont know how? Still taking the same card as in return responses[2].json(). I tried card_id=create_card_on_the_board[1]['id']
but its not working as this is not a list

You could put all of the responses in a list so you could get them all and then decide what to do with them. Here's how you'd do that:
def create_card_on_the_board_call(self, id_list, card_count, card_name):
responses = []
for i in range(card_count):
create_card = self.rest_api_helper.post_call(
self.base_url + '/cards?' + 'key=' + test_config[
'key'] + '&' + 'token=' + test_config[
'token'] + '&' + f'idList={id_list}' + '&' + f'name={card_name}', headers=None)
responses.append(create_card)
// Logic here determines which response to return and sets `x = 0, 1 or 2`
return responses[x].json()
So now you have a list named responses after your loop completes, and you can do whatever you want with it. I put a comment to show kinda what you seem to be talking about in your question. You could just return the whole list with return responses and let the caller worry about which one to do what with.

Selenium: Check if element exist i loop, if true, click, if not continue

I'm trying to solve a issue which I cant wrap my head around whats the best solution for Selenium.
I want to check if a element exist and is displayed. If it is, click it and close it and continue with the rest of the code code. I want evaluate this for each loop to go through my list.
Example/pseudo code as of now:
...
for i in (urlList):
browser.get(i)
if #element exist and is displayed (//div[contains(#class,'wizard-tooltip-btn skip'))
# Find it and click/close it (browser.find_element_by_xpath(".//div[contains(#class,'wizard-tooltip-btn skip')]").click())
else:
browser.find_element_by_xpath(".//div[#data-name='data-window']").click()
title_element = browser.find_element_by_class_name('pane-legend-title__description')
ticker = browser.find_element_by_class_name('chart-data-window-header')
tickerClean = ticker.text.split(',')[0]
latestClose = browser.find_element_by_xpath('/html/body/div[1]/div[3]/div/div[1]/div[1]/div[3]/div/div[2]/div[1]/div[1]/div[2]/div[4]/div[2]')
latestBase = browser.find_element_by_xpath('/html/body/div[1]/div[3]/div/div[1]/div[1]/div[3]/div/div[2]/div[1]/div[3]/div[2]/div[9]/div[2]/span')
data.append(title_element.text + ',' + tickerClean + ',' + latestBase.text + ',' + latestClose.text)
print('Done' + title_element.text + ',' + tickerClean + ',' + latestBase.text + ',' + latestClose.text)
...

I would wrap the .click() in a try, catch statement. This way you can try and click the element every loop, but catch the exception that is thrown when it's not present/visible.
NOTE: If you are using Implicit Waits, each element check will take
however many seconds you have set the Implicit Wait to. I would not
recommend using Implicit, and use Explicit Waits instead in your
Selenium scripts. HERE you will find the docs on both.
Here is an example using your example:
from selenium.common.exceptions import NoSuchElementException, ElementNotVisibleException
for i in (urlList):
browser.get(i)
try:
browser.find_element_by_xpath(".//div[contains(#class,'wizard-tooltip-btn skip')]").click()
except (ElementNotVisibleException, NoSuchElementException):
pass
browser.find_element_by_xpath(".//div[#data-name='data-window']").click()
title_element = browser.find_element_by_class_name('pane-legend-title__description')
ticker = browser.find_element_by_class_name('chart-data-window-header')
tickerClean = ticker.text.split(',')[0]
latestClose = browser.find_element_by_xpath('/html/body/div[1]/div[3]/div/div[1]/div[1]/div[3]/div/div[2]/div[1]/div[1]/div[2]/div[4]/div[2]')
latestBase = browser.find_element_by_xpath('/html/body/div[1]/div[3]/div/div[1]/div[1]/div[3]/div/div[2]/div[1]/div[3]/div[2]/div[9]/div[2]/span')
data.append(title_element.text + ',' + tickerClean + ',' + latestBase.text + ',' + latestClose.text)
print('Done' + title_element.text + ',' + tickerClean + ',' + latestBase.text + ',' + latestClose.text)
If you are against the try, catch method, and would rather not deal with catching exceptions, you can use find_elements to Truthy check the existence of the element in a list, and click it if it passes, like this:
for i in (urlList):
browser.get(i)
if browser.find_elements_by_xpath(".//div[contains(#class,'wizard-tooltip-btn skip')]"):
browser.find_element_by_xpath(".//div[contains(#class,'wizard-tooltip-btn skip')]").click()
browser.find_element_by_xpath(".//div[#data-name='data-window']").click()
title_element = browser.find_element_by_class_name('pane-legend-title__description')
ticker = browser.find_element_by_class_name('chart-data-window-header')
tickerClean = ticker.text.split(',')[0]
latestClose = browser.find_element_by_xpath('/html/body/div[1]/div[3]/div/div[1]/div[1]/div[3]/div/div[2]/div[1]/div[1]/div[2]/div[4]/div[2]')
latestBase = browser.find_element_by_xpath('/html/body/div[1]/div[3]/div/div[1]/div[1]/div[3]/div/div[2]/div[1]/div[3]/div[2]/div[9]/div[2]/span')
data.append(title_element.text + ',' + tickerClean + ',' + latestBase.text + ',' + latestClose.text)
print('Done' + title_element.text + ',' + tickerClean + ',' + latestBase.text + ',' + latestClose.text)
I could not tell by the wording of the question, but if you did not want to run the rest of the statements in the loop after you found and clicked the element, add a continue after it passes to whichever example you chose, as such:
try:
browser.find_element_by_xpath(".//div[contains(#class,'wizard-tooltip-btn skip')]").click()
continue
except (ElementNotVisibleException, NoSuchElementException):
pass
or:
if browser.find_elements_by_xpath(".//div[contains(#class,'wizard-tooltip-btn skip')]"):
browser.find_element_by_xpath(".//div[contains(#class,'wizard-tooltip-btn skip')]").click()
continue
The continue will go to the next item in the loop without running the statements after the click.

Change API call in for loop python

I am trying to call several api datasets within a for loop in order to change the call and then append those datasets together into a larger dataframe.
I have written this code which works to call the first dataset but then returns this error for the next call.
`url = base + "max=" + maxrec + "&" "type=" + item + "&" + "freq=" + freq + "&" + "px=" +px + "&" + "ps=" + str(ps) + "&" + "r="+ r + "&" + "p=" + p + "&" + "rg=" +rg + "&" + "cc=" + cc + "&" + "fmt=" + fmt
TypeError: must be str, not Response`
Here is my current code
import requests
import pandas as pd
base = "http://comtrade.un.org/api/get?"
maxrec = "50000"
item = "C"
freq = "A"
px="H0"
ps="all"
r="all"
p="0"
rg="2"
cc="AG2"
fmt="json"
comtrade = pd.DataFrame(columns=[])
for year in range(1991,2018):
ps="{}".format(year)
url = base + "max=" + maxrec + "&" "type=" + item + "&" + "freq=" + freq + "&" + "px=" +px + "&" + "ps=" + str(ps) + "&" + "r="+ r + "&" + "p=" + p + "&" + "rg=" +rg + "&" + "cc=" + cc + "&" + "fmt=" + fmt
r = requests.get(url)
x = r.json()
new = pd.DataFrame(x["dataset"])
comtrade = comtrade.append(new)

Let requests assemble the URL for you.
common_params = {
"max": maxrec,
"type": item,
"freq": freq,
# etc
}
for year in range(1991,2018):
response = requests.get(base, params=dict(common_params, ps=str(year))
response_data = response.json()
new = pd.DataFrame(response_data["dataset"])
comtrade = comtrade.append(new)

Disclaimer: the other answer is correct and you should use it.
However, your actual problem stems from the fact that you are overriding r here:
r = requests.get(url)
x = r.json()
During the next iteration r will still be that value and not the one you initialized it with in the first place. You could simply rename it to result to avoid that problem. Better let the requests library do the work though.

Automatic labeling of LDA generated topics

I'm trying to categorize customer feedback and I ran an LDA in python and got the following output for 10 topics:
(0, u'0.559*"delivery" + 0.124*"area" + 0.018*"mile" + 0.016*"option" + 0.012*"partner" + 0.011*"traffic" + 0.011*"hub" + 0.011*"thanks" + 0.010*"city" + 0.009*"way"')
(1, u'0.397*"package" + 0.073*"address" + 0.055*"time" + 0.047*"customer" + 0.045*"apartment" + 0.037*"delivery" + 0.031*"number" + 0.026*"item" + 0.021*"support" + 0.018*"door"')
(2, u'0.190*"time" + 0.127*"order" + 0.113*"minute" + 0.075*"pickup" + 0.074*"restaurant" + 0.031*"food" + 0.027*"support" + 0.027*"delivery" + 0.026*"pick" + 0.018*"min"')
(3, u'0.072*"code" + 0.067*"gps" + 0.053*"map" + 0.050*"street" + 0.047*"building" + 0.043*"address" + 0.042*"navigation" + 0.039*"access" + 0.035*"point" + 0.028*"gate"')
(4, u'0.434*"hour" + 0.068*"time" + 0.034*"min" + 0.032*"amount" + 0.024*"pay" + 0.019*"gas" + 0.018*"road" + 0.017*"today" + 0.016*"traffic" + 0.014*"load"')
(5, u'0.245*"route" + 0.154*"warehouse" + 0.043*"minute" + 0.039*"need" + 0.039*"today" + 0.026*"box" + 0.025*"facility" + 0.025*"bag" + 0.022*"end" + 0.020*"manager"')
(6, u'0.371*"location" + 0.110*"pick" + 0.097*"system" + 0.040*"im" + 0.038*"employee" + 0.022*"evening" + 0.018*"issue" + 0.015*"request" + 0.014*"while" + 0.013*"delivers"')
(7, u'0.182*"schedule" + 0.181*"please" + 0.059*"morning" + 0.050*"application" + 0.040*"payment" + 0.026*"change" + 0.025*"advance" + 0.025*"slot" + 0.020*"date" + 0.020*"tomorrow"')
(8, u'0.138*"stop" + 0.110*"work" + 0.062*"name" + 0.055*"account" + 0.046*"home" + 0.043*"guy" + 0.030*"address" + 0.026*"city" + 0.025*"everything" + 0.025*"feature"')
Is there a way to automatically label them? I do have a csv file which has feedbacks manually labeled, but I do not want to supply these labels myself. I want the model to create labels. Is it possible?

The comments here link to another SO answer that links to a paper. Let's say you wanted to do the minimum to try to make this work. Here is an MVP-style solution that has worked for me: search Google for the terms, then look for keywords in the response.
Here is some working, though hacky, code:
pip install cssselect
then
from urllib.parse import urlencode, urlparse, parse_qs
from lxml.html import fromstring
from requests import get
from collections import Counter
def get_srp_text(search_term):
raw = get(f"https://www.google.com/search?q={topic_terms}").text
page = fromstring(raw)
blob = ""
for result in page.cssselect("a"):
for res in result.findall("div"):
blob += ' '
blob += res.text if res.text else " "
blob += ' '
return blob
def blob_cleaner(blob):
clean_blob = blob.replace(r'[\/,\(,\),\:,_,-,\-]', ' ')
return ''.join(e for e in blob if e.isalnum() or e.isspace())
def get_name_from_srp_blob(clean_blob):
blob_tokens = list(filter(bool, map(lambda x: x if len(x) > 2 else '', clean_blob.split(' '))))
c = Counter(blob_tokens)
most_common = c.most_common(10)
name = f"{most_common[0][0]}-{most_common[1][0]}"
return name
pipeline = lambda x: get_name_from_srp_blob(blob_cleaner(get_srp_text(x)))
Then you can just get the topic words from your model, e.g.
topic_terms = "delivery area mile option partner traffic hub thanks city way"
name = pipeline(topic_terms)
print(name)
>>> City-Transportation
and
topic_terms = "package address time customer apartment delivery number item support door"
name = pipeline(topic_terms)
print(name)
>>> Parcel-Package
You could improve this up a lot. For example, you could use POS tags to only find the most common nouns, then use those for the name. Or find the most common adjective and noun, and make the name "Adjective Noun". Even better, you could get the text from the linked sites, then run YAKE to extract keywords.
Regardless, this demonstrates a simple way to automatically name clusters, without directly using machine learning (though, Google is most certainly using it to generate the search results, so you are benefitting from it).

Concatenating strings containing many quotations results in slashes in output

I am trying to build a string that needs to contain specific double and single quotation characters for executing a SQL expression.
I need my output to be formatted like this:
" "Full_Stree" = 'ALLENDALE RD' "
where the value of ALLENDALE RD will be a variable defined through a For Loop. In the following code sample, the variable tOS is what I am trying to pass into the query variable.
tOS = "ALLENDALE RD"
query = '" "Full_Stree" = ' + "'" + tOS + "' " + '"'
and when I print the value of query variable I get this output:
'" "Full_Stree" = \'ALLENDALE RD\' "'
The slashes are causing my query to fail. I also tried using a modulus operator to pass the value of the tOS variable, but get the same results:
where = '" "Full_Stree" = \'%s\' "' % (tOS)
print where
'" "Full_Stree" = \'ALLENDALE RD\' "'
How can I get my string concatenated into the correct format, leaving the slashes out of the expression?

What you are seeing is the repr of your string.
>>> s = '" "Full_Stree" = \'ALLENDALE RD\' "'
>>> s # without print console displays the repr
'" "Full_Stree" = \'ALLENDALE RD\' "'
>>> print s # with print the string itself is displayed
" "Full_Stree" = 'ALLENDALE RD' "
Your real problem is the extra quotes at the beginning and end of your where-clause.
This
query = '" "Full_Stree" = ' + "'" + tOS + "' " + '"'
should be
query = '"Full_Stree" = ' + "'" + tOS + "'"
It is more clearly written as
query = """"Full_Stree" = '%s'""" % tOS
The ArcGis docs recommend something more like this
dataset = '/path/to/featureclass/shapefile/or/table'
field = arcpy.AddFieldDelimiters(dataset, 'Full_Stree')
whereclause = "%s = '%s'" % (field, tOS)
arcpy.AddFieldDelimiters makes sure that the field name includes the proper quoting style for the dataset you are using (some use double-quotes and some use square brackets).

Somehow the way I already tried worked out:
where = '" "Full_Stree" = \'%s\' "' % (tOS)
print where
'" "Full_Stree" = \'ALLENDALE RD\' "'

Can't you just use triple quotes?
a=""" "Full_Street" = 'ALLENDALE RD' """
print a
"Full_Street" = 'ALLENDALE RD'