We Keep Coding

pulling records with python

Having some trouble with this. I know that I have to use the offset value to pull more than 100 records, here is what I currently have:
AIRTABLE_BASE_ID = 'airtablebaseid'
AIRTABLE_API_KEY = 'airtableapikey'
AIRTABLE_TABLE_NAME = 'airtablename'
endpoint = f'https://api.airtable.com/v0/{AIRTABLE_BASE_ID}/{AIRTABLE_TABLE_NAME}?filterByFormula=AND(NOT(%7BSent+to+Payroll%7D+%3D+%22Sent%22)%2CNOT(%7BSent+to+Payroll%7D+%3D+%22Skipped%22))'
def get_airtable():
headers = {
"Authorization": f"Bearer {AIRTABLE_API_KEY}"
}
response = requests.get(endpoint, headers=headers)
return response
recordList = []
recordIDs = []
recordTimeStamp = []
response = get_airtable()
data = response.json()
for record in data['records']:
recordList.append(record['fields'])
recordIDs.append(record['id'])
recordTimeStamp.append(record['createdTime'])
print(record)

Airtable can't give you more than 100 records per request, it works, as you said it, with pagination and offset.
The first request you made without offset parameter will return you a payload with a offset field set to n, you have to pass that offset to your next request to get the n next records, and so on...
response = get_airtable()
data = response.json()
OFFSET = data['offset'] #not sur if "offset" is at root of the response
endpoint = f'https://api.airtable.com/v0/{AIRTABLE_BASE_ID}/{AIRTABLE_TABLE_NAME}?offset={OFFSET}&filterByFormula=AND(NOT(%7BSent+to+Payroll%7D+%3D+%22Sent%22)%2CNOT(%7BSent+to+Payroll%7D+%3D+%22Skipped%22))'

Get genres of music from listening history

I have the following csv file 1 which contains the features of the songs that have been listened to by a spotify user. The features I extracted with the following code block [2]. I would now like to add the information relating to the genre of the song, what can I do? Or is there a way to go back to the genre of the song starting from the bpm?
[2]
def get_id(track_name: str,artist:str, token: str) -> str:
headers = {
'Accept': 'application/json',
'Content-Type': 'application/json',
'Authorization': f'Bearer ' + token,
}
track_artist = track_name+ " " + artist
params = [
('q',track_artist ),#q is the search query parameter
('type', 'track'),
]
try:
response = requests.get('https://api.spotify.com/v1/search',
headers = headers, params = params, timeout = 10)
json = response.json()
first_result = json['tracks']['items'][0]
track_id = first_result['id']
return track_id
except:
return None
# Get track_id for streaming history
spotify_data["track_id"] = spotify_data.apply(lambda x: get_id(x["track_name"],
x["artist_name"],
token),axis=1)
# get track's feature
my_feature = pd.DataFrame(columns=["song_id","energy", "liveness","tempo","speechiness",
"acousticness","instrumentalness","danceability",
"duration_ms","loudness","valence",
"mode","key","genre"])
In case you need it I can show you all the code I use, I didn't do it just to avoid pasting useless code.

Facing a problem in solving captcha using a service

I grab the site-key using a some-what hardcoded method, but since the len is always the same it's ok.
After that I use the 2captcha API Documentation so that I can POST the key and GET the token for the captcha back. I face two major problems:
1) I always get the site-key wrong error, but the sitekey is correct according to their example (their sitekey is 40chars long and my sitekeys are 40 too)
2) I tried creating a POST function externally and trying it out seeing if it is a bug, but using the Py2Captcha documentation I always get the following error:
This is captcha key grabbing.
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Captcha Key~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
url=driver.find_element_by_css_selector("iframe[role='presentation']").get_attribute('src')
print(url)
keygoogle = url[53:93]
print('Site Key = ', keygoogle)
This is the captcha key solving block:
answer = ""
answer_id = 0
api_key = '--------------------------------'
data_post = {'key': api_key, 'method': 'userrecaptcha', 'googlekey': keygoogle, "pageurl": mainurl}
response = requests.post(url = 'https://2captcha.com/in.php', data = data_post )
print(response)
print("Waiting for server response.")
for x in range(15):
time.sleep(1)
if x == 5:
print('Downloading info..')
elif x == 10:
print('Processing info..')
elif x == 14:
print('Solving captcha..')
data_request = {
'key': api_key,
'action': answer,
'id': answer_id,
}
requests.get(url ='https://2captcha.com/res.php', data=data_request)
print(answer)
def captcha():
google_key = keygoogle
url = mainurl
client = TwoCaptchaClient(client_key=api_key)
task = GoogleReCaptchaV2Task(googlekey=google_key, pageurl=mainurl)
job = client.create_task(task)
token = job.get_solution_response()
return token
print(captcha())
What I haven't included is the part where the token gets posted into the answer field, I am not sure how to do that yet but I will find a way for sure!
EDIT:
This is the value I get from printing this:
response = requests.post(url = 'https://2captcha.com/in.php', data = data_post )
And this is the value I get from print('Site Key = ', keygoogle)
Site Key = Lc3HAsUAAAAACsN7CgY9MMVxo2M09n_e4heJEiZ&

This is my way of grabbing the correct key:
url=driver.find_element_by_css_selector("iframe[role='presentation']").get_attribute('src')
keygoogle = url[52:92]
And this is my POST function:
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Captcha Solve~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
answer = ""
answer_id = 0
data_post = {'key': api_key, 'method': 'userrecaptcha', 'googlekey': keygoogle, "pageurl": mainurl}
response = requests.post(url = 'https://2captcha.com/in.php', data = data_post )
response = response.text[3:]
print("Waiting for server response.")
for x in range(30):
time.sleep(1)
if x == 8:
print('Downloading info..')
elif x == 15:
print('Processing info..')
data_request = {'key': api_key,'id': int(response),'action': 'get'}
response = requests.get(url='https://2captcha.com/res.php', params=data_request)
token = response.text.split('|')[0]
while response.text == 'CAPCHA_NOT_READY':
print('Waiting for Capcha..')
time.sleep(5)
response = requests.get(url='https://2captcha.com/res.php', params=data_request)
token = response
print(token)

The & on the end of that sitekey definitely isn't supposed to be there. Use a regex instead of what you're doing there with the indexes.

Zendesk API search result index

I write a python script to do GET and PUT method in zendesk API and successfully get the data I wanted and do some updates to the tickets.
below method resulting this ticket number "6442" and put method is intended to remove the tags
from urllib.parse import urlencode
import json
import requests
# Set the credentials
credentials = 'some email', 'some password'
session = requests.Session()
session.auth = credentials
# Set the GET parameters
params_noreply_window = {
'query': 'type:ticket tags:test status<closed',
}
params_oustide_businesshour = {
'query': 'type:ticket tags:send_whatsapp_obh status:new',
}
url_search1 = 'https://propertypro.zendesk.com/api/v2/search.json?' + \
urlencode(params_noreply_window)
url_search2 = 'https://propertypro.zendesk.com/api/v2/search.json?' + \
urlencode(params_oustide_businesshour)
response_noreply_window = session.get(url_search1)
response_oustide_businesshour = session.get(url_search2)
# -----------------------------------------------------------------------------
if response_noreply_window.status_code != 200 | response_oustide_businesshour.status_code != 200:
print('Status 1:', response_noreply_window.status_code + 'Status 2:', response_oustide_businesshour.status_code,
'Problem with the request. Exiting.')
exit()
# Print the subject of each ticket in the results
data_noreply_window = response_noreply_window.json()
data_oustide_businesshour = response_oustide_businesshour.json()
# Ticket to update
# Create a list containing the values of the id field
# for each dictionary that is an element of the list data
id_merged1 = [result['id'] for result in data_noreply_window['results']]
print(type(id_merged1))
print(id_merged1)
id_merged2 = [result['id'] for result in data_oustide_businesshour['results']]
print(type(id_merged2))
print(id_merged2)
# Join value of list by using comma separated
id_merged1_joined = ','.join(map(str, id_merged1))
print(id_merged1_joined)
id_merged2_joined = ','.join(map(str, id_merged2))
print(id_merged2_joined)
# Package the data in a dictionary matching the expected JSON
data_comment1 = {"ticket":
{
"remove_tags": ["test"]
}
}
data_comment2 = {"ticket":
{
"remove_tags": ["send_whatsapp_obh"]
}
}
# Encode the data to create a JSON payload
payload1 = json.dumps(data_comment1)
payload2 = json.dumps(data_comment2)
print("**Start**")
# Set the request parameters
url_put_comments1 = 'https://propertypro.zendesk.com/api/v2/tickets/update_many.json?' +\
'ids=' + id_merged1_joined
url_put_comments2 = 'https://propertypro.zendesk.com/api/v2/tickets/update_many.json?' +\
'ids=' + id_merged2_joined
user = 'some email'
pwd = 'some password'
headers = {'content-type': 'application/json'}
# Do the HTTP put request
response_request_noreply = requests.put(url_put_comments1, data=payload1,
auth=(user, pwd), headers=headers)
response_request_obh = requests.put(url_put_comments2, data=payload2,
auth=(user, pwd), headers=headers)
# Check for HTTP codes other than 200
if response_request_noreply.status_code != 200 | response_request_obh.status_code != 200:
print('Status 1:', response_request_noreply.status_code +
'Status 1:', response_request_obh.status_code,
'Problem with the request. Exiting.')
exit()
# Report success
print('Successfully added comment to tickets')
However, after running my python code and do another GET method, the same ticket number still appears and I need to wait in random time to get the result I intend which is return 'null' since I have updated the ticket by using PUT method.
Can anyone explain me how does the Zendesk API works? and my apology for my incorrect sentences in explaining my concern.

Adding all the integers returned from a for loop?

I've just started with python and I'm writing a script that returns the number of clicks for a bunch of links. I'm using a for loop to iterate through all the links and return the click count for each:
for url in short_urls:
query_params = {
'access_token': ACCESS_TOKEN,
'link': url.values()}
endpoint = ENDPOINT
response = requests.get(endpoint, params = query_params)
data = json.loads(response.content)
print "link_clicks:" + " " + str(data['data']['link_clicks'])
The results returned into terminal look like this:
link_clicks: 938
link_clicks: 63
link_clicks: 3921
link_clicks: 47
link_clicks: 21
What would be the best way to add all of those numbers and return a total? Ideally, I would like to be able to print the total as "total link_clicks: (total)" in terminal.
EDIT:
Thank you all for the answers.
I guess I was overcomplicating the solution in my head, for some reason I thought it would require creating a list to which each new integer from the for loop would have to be added and then individually adding each element of that new list via indexing to grab the total.

You aren't really "returning" anything. You're just printing text to the console.
If you want to return a value, you need to use the return statement:
values = []
for url in short_urls:
...
values.append(number)
return values
Or you can use yield to make a generator:
for url in short_urls:
...
yield number
Both of which can be summed up using the sum function:
total = sum(your_function())

You use sum() to add up elements in a list.
nums = []
for url in short_urls:
query_params = {
'access_token': ACCESS_TOKEN,
'link': url.values()}
endpoint = "https://api-ssl.bitly.com/v3/link/clicks"
response = requests.get(endpoint, params = query_params)
data = json.loads(response.content)
print "link_clicks:" + " " + str(data['data']['link_clicks'])
nums.append(data['data']['link_clicks'])
print 'total link_clicks:', sum(nums)
Or simply add them in a variable..
total_links_clicks = 0
for url in short_urls:
query_params = {
'access_token': ACCESS_TOKEN,
'link': url.values()}
endpoint = "https://api-ssl.bitly.com/v3/link/clicks"
response = requests.get(endpoint, params = query_params)
data = json.loads(response.content)
print "link_clicks:" + " " + str(data['data']['link_clicks'])
total_links_clicks += data['data']['link_clicks']
print 'total link_clicks:', total_links_clicks