I am trying to call all the results from this API and into one big list. Right now the default value is set to 50. By having the paginated section in the code I am still getting 50 results. How will I be able to get more results?
nextRecordKey allows to paginate through the api
import os
import requests
import json
import requests
from http import HTTPStatus
client_id = ""
client_secret = ""
os.environ["DX_GATEWAY"] = "http://api.com"
os.environ["DX_CLIENT_ID"] = client_id
os.environ["DX_CLIENT_SECRET"] = client_secret
dx_request = requests.Request()
path = "/path/to/api"
params = {
"Type": "abc",
"Id": "def",
"limit": 999,
"Category": "abc"
}
params_str = "&".join([f"{k}={v}" for k, v in params.items()])
url = "?".join([path, params_str])
vulns = requests.get(
url=url,
version=1,
)
if vulns.status_code != int(HTTPStatus.OK):
raise RuntimeError("API call did not return expected response: " + str(vulns))
## loop through paginated API ##.
response_data = vulns.json()
while vulns["nextRecordKey"]:
vulns=requests.get(vulns["nextRecordKey"]).json()
print(json.dumps(response_data))```
Related
The documentation of the API is here, and I try to implement this line in python
//retrieve entries created on a specific day (use the date_created field)
//this example returns entries created on September 10, 2019
https://localhost/wp-json/gf/v2/entries?search={"field_filters": [{"key":"date_created","value":"09/10/2019","operator":"is"}]}
But when I try to do with python in the following code, I got an error:
import json
import oauthlib
from requests_oauthlib import OAuth1Session
consumer_key = ""
client_secret = ""
session = OAuth1Session(consumer_key,
client_secret=client_secret,signature_type=oauthlib.oauth1.SIGNATURE_TYPE_QUERY)
url = 'https://localhost/wp-json/gf/v2/entries?search={"field_filters": [{"key":"date_created","value":"09/01/2023","operator":"is"}]}'
r = session.get(url)
print(r.content)
The error message is :
ValueError: Error trying to decode a non urlencoded string. Found invalid characters: {']', '['} in the string: 'search=%7B%22field_filters%22:%20[%7B%22key%22:%22date_created%22,%22value%22:%2209/01/2023%22,%22operator%22:%22is%22%7D]%7D'. Please ensure the request/response body is x-www-form-urlencoded.
One solution is to parameterize the url:
import requests
import json
url = 'https://localhost/wp-json/gf/v2/entries'
params = {
"search": {"field_filters": [{"key":"date_created","value":"09/01/2023","operator":"is"}]}
}
headers = {'Content-type': 'application/json'}
response = session.get(url, params=params, headers=headers)
print(response.json())
But in the retrieved entries, the data is not filtered with the specified date.
In the official documentation, they gave a date in this format "09/01/2023", but in my dataset, the format is: "2023-01-10 19:16:59"
Do I have to transform the format ? I tried a different format for the date
date_created = "09/01/2023"
date_created = datetime.strptime(date_created, "%d/%m/%Y").strftime("%Y-%m-%d %H:%M:%S")
What alternative solutions can I test ?
What if you use urllib.parse.urlencode function, so your code would looks like:
import json
import oauthlib
from requests_oauthlib import OAuth1Session
import urllib.parse
consumer_key = ""
client_secret = ""
session = OAuth1Session(consumer_key,
client_secret=client_secret,signature_type=oauthlib.oauth1.SIGNATURE_TYPE_QUERY)
params = {
"search": {"field_filters": [{"key":"date_created","value":"09/01/2023","operator":"is"}]}
}
encoded_params = urllib.parse.urlencode(params)
url = f'https://localhost/wp-json/gf/v2/entries?{encoded_params}'
r = session.get(url)
print(r.content)
hope that helps
I had the same problem and found a solution with this code:
params = {
'search': json.dumps({
'field_filters': [
{ 'key': 'date_created', 'value': '2023-01-01', 'operator': 'is' }
],
'mode': 'all'
})
}
encoded_params = urllib.parse.urlencode(params, quote_via=urllib.parse.quote)
url = 'http://localhost/depot_git/wp-json/gf/v2/forms/1/entries?' + encoded_params + '&paging[page_size]=999999999' # nombre de réponses par page forcé manuellement
I'm not really sure what permitted it to work as I'm an absolute beginner with Python, but I found that you need double quotes in the URL ( " ) instead of simple quotes ( ' ), so the solution by William Castrillon wasn't enough.
As for the date format, Gravity Forms seems to understand DD/MM/YYYY. It doesn't need a time either.
I am trying to call an API using an access token.
Currently I am getting the token like so:
import requests
auth_url = "https://oauth.thegivenurl.com/connect/token"
client_id = "SomeClient_ID"
client_secret = "SomeClient_Secret"
scope = "SomeScope"
grant_type = "client_credentials"
data = {
"grant_type": grant_type,
"client_id": client_id,
"client_secret": client_secret,
"scope": scope
}
auth_response = requests.post(auth_url, data=data)
print(auth_response.content)
This produces something like the following:
{"access_token":"eyJhbGdiOihSUzh1NhIsImtpZCI6IjlVUHVwYnBkTXN2RDZ0Ry1ZcDVRUlEiLCJ0eXAiOiJhdCtqd3QifQ.eyJuYmYiOjE2MzM4OTcxNDIsImV4cCI6hTYzMhkwMDc0MiwiaXNzIjoiaHR0cHM6Ly9vYXV0aC56YW1iaW9uLmNvbSIsImF1ZCI6ImFwaTEiLCJjbGllbnRfaWQiOiIwN0I3RkZEOC1GMDJCLTRERDAtODY2OS0zRURBNzUyRTMyNkQiLCJzY29wZSI6WyJhcGkxIl19.GU6lynvQYAAmycEPKbLgHE-Ck189x-a-rVz6QojkBIVpSLu_sSAX2I19-GlTjVWeLKoMVxqEfVq_qIaaQYa5KFmMLHRxP6J-RUgGK8f_APKjX2VNoMyGyAbZ0qXAJCvUTh4CPaRbZ6pexEishzr4-w3JN-hJLiv3-QH2y_JZ_V_KoAyu8ANupIog-Hdg8coI3wyh86OeOSAWJA1AdkK5kcuwC890n60YVOWqmUiAwPRQrTGh2mnflho2O3EZGkHiRPsiJgjowheD9_Wi6AZO0kplHiJHvbuq1PV6lwDddoSdAIKkDscB0AF53sYlgJlugVbtU0gdbXjdyBZvUjWBgw","expires_in":3600,"token_type":"Bearer","scope":"api1"}
Now I would like to call the API and pass the token in a header, but I am unsure how to do this and I have had a couple of goes at online resources
One of my attempts were:
url = "https://anotherurl.com/api/SecuredApi/StaffDetails"
head = {'Authorization': 'token {}'.format(auth_response)}
response = requests.get(url, headers=head)
print(response)
but this gives me a 403 error
Please assist me with pointing out my error
EDIT:
Thanks to #RaniSharim I have made some changes. I now have
import requests
import json
auth_url = "https://oauth.thegivenurl.com/connect/token"
client_id = "SomeClient_ID"
client_secret = "SomeClient_Secret"
scope = "SomeScope"
grant_type = "client_credentials"
data = {
"grant_type": grant_type,
"client_id": client_id,
"client_secret": client_secret,
"scope": scope
}
dateparams = {
"StartDateTime": "2021-01-01 00:00:00",
"EndDateTime" : "2021-10-11 23:59:00"
}
auth_response = requests.post(auth_url, data=data)
# print(auth_response.content)
authjson = auth_response.content
authdata = json.loads(authjson)
token = (authdata['access_token'])
# print(token)
head = {"Authorization": "Bearer " + token}
response = requests.get(url, headers=head, params=dateparams)
print(response.content)
This looks better but I am now getting a 400 error:
"message":"The date range you have specified is in an invalid format. The required format is yyyy-MM-dd HH:mm:ss","status":400}
As best I can see my date ranges are already in the requested format, as set here:
dateparams = {
"StartDateTime": "2021-01-01 00:00:00",
"EndDateTime" : "2021-10-11 23:59:00"
}
Normally, 400 means front-end error, but when you do GET request
dateparams = {
"StartDateTime": "2021-01-01 00:00:00",
}
r = requests.get(url, params=dateparams)
print(r.url)
GET url will turn into sth like this:
https://oauth.thegivenurl.com/connect/token?StartDateTime=2021-01-01+00%3A00%3A00
see the str
2021-01-01+00%3A00%3A00
So if back-end can't handle this right, you'll get this error too
but you can use GET in another way:
requests.get(url, json=dateparams)
This will send your json params perfectly
I am using this python script to take a response from Progresso API:
http://docs.progresso.apiary.io/#reference/behaviour/behaviour-events-collection/get-behaviour-events
from urllib2 import Request, urlopen
import smtplib import gspread
from oauth2client.service_account import ServiceAccountCredentialseaders = {
'Authorization': 'Bearer [CURRENT_TOKEN]'
}
request = Request('https://private-anon-ae5edf57e7-progresso.apiary-
mock.com/BMEvents/?Behaviour=new', headers=headers)
response_body = urlopen(request).read()
scope = ['https://spreadsheets.google.com/feeds']
credentials = ServiceAccountCredentials.from_json_keyfile_name('ProgressoAPI-
2f6ecaa6635c.json', scope)
gc = gspread.authorize(credentials)
wks = gc.open("Progresso Test").sheet1
wks.clear()
cell_list = wks.range('A1:H20')
for cell in cell_list:
cell.value = response_body
wks.update_cells(cell_list)
I know the cell.value = response body is wrong and I don't know how I can get it right - I am stuck.
it appears in every cell like this:
"{
""BehaviourEntryId"": 13798177,
""LearnerId"": 245277,
""LearnerCode"": ""2009-0080"",
""RegGroup"": ""U6-RWE"",
""Behaviour"": ""Negative"",
""IncidentDate"": ""2017-02-07"",
""Subject"": ""BE"",
""Location"": ""CLS"",
""Published"": ""Yes"",
""Creator"": ""DhDr"",
""Editor"": null,
""Assignee"": ""DiRo"",
""Status"": ""Completed"",
""Details"": [
{
""Category"": ""CL"",
""Type"": ""CLatt"",
""Severity"": ""S2"",
""point"": 0
},
{
""Category"": ""CL"",
""Type"": ""CLBEH"",
""Severity"": ""S2"",
""point"": 2
}
],
""Comments"": [
{
""BehaviourEntryCommentId"": 5648278,
""Confidential"": true,
""Comment"": ""Asked to go to the toilet and went to the one furthest away just to waste time.""
},
{
""BehaviourEntryCommentId"": 5648279,
""Confidential"": false,
""Comment"": ""Spat gum out on floor""
},
{
""BehaviourEntryCommentId"": 5648280,
""Confidential"": false,
""Comment"": ""Was rude to memeber of Staff""
}
],
""Actions"": [
""HTO"",
""ISO""
]
}"
How do I separate the text to how I want in the cell range and bulk update it?
If you mean something like two columns with one row being "BehaviourEntryId" and the other row being 13798177, you can try something like this:
import json
response = json.loads(response_body) #decode the json response string, returns a dict
response_pairs = list(response.items)
for i in range(1, len(response_body)+1):
current_pair = response_pairs[i-1]
current_key = current_pair[0]
current_value = current_pair[1]
wks.update_acell('A{}'.format(i), current_key)
wks.update_acell('B{}'.format(i), current_value)
Am working with an API that asks to specify a file (Excel in my case) to upload to the API. Documentation specifies: JSON Property = file, Data Type = FileUpload. My question: What is FileUpload. I tried simply specifying the name of the file (e.g. c:\test\Data.xls) but obviously that does not work.
I am working with an API for FastField Mobile Forms (www.fastfield.com). Documentation shown in screen shot, as is code and result. Somehow, I am not posting the file data to the API correctly.
So this is the Python code I am attempting to run
import requests
import json
from requests.auth import HTTPBasicAuth
import base64
# Get session token, this must be specified in header of subsequent request and returns a JSON object ready for insertion into header
rqstResponse = requests.post('https://manage.fastfieldforms.com/api/authenticate', auth=HTTPBasicAuth('***', '***'))
jsonObj = json.loads(rqstResponse.content)
sessionToken = jsonObj['data']['sessionToken']
headers = {'X-Gatekeeper-SessionToken': sessionToken}
# run this code to get listIds - which are hard coded further down
rqstResponse = requests.get("https://manage.fastfieldforms.com/api/globallists", headers=headers)
print (rqstResponse.content)
del rqstResponse
# Read file and convert to binary string
filePath = r"J:\Properties\PropGIS\proj\20150820140457_TelecoMapping\data\Survey_Feb17\FastField_Test01.xlsx"
with open(filePath, 'r') as f:
filecontents = f.read()
fileDataEncoded = base64.b64encode(filecontents)
# create JSON
payloadDictObj = {}
payloadDictObj['file'] = fileDataEncoded
payloadDictObj['id'] = "03c804cb-b983-4e4c-956b-96ac23da16b2"
#payloadDictObj['listname'] = "Test02"
serializedJsonStr = json.dumps(payloadDictObj)
print serializedJsonStr
# Update Global List
rqstResponse = requests.post("https://manage.fastfieldforms.com/api//globallist", data=serializedJsonStr, headers=headers)
print (rqstResponse.content)
del rqstResponse
# --------------------
# Response
{
"code": 200,
"data": {
"searchResults": [
{
"id": 7793,
"accountId": 43600,
"name": "Test01",
"active": true,
"createdAt": "2017-05-24T06:37:28.49Z",
"updatedAt": "2017-05-24T06:37:28.49Z",
"version": 1,
"listId": "03c804cb-b983-4e4c-956b-96ac23da16b2",
"path": "{ bucket:'fastfield-globallists', key:'43600/ca4b89df75db4ef8b513d15d59f654d8.csv'}"
}
]
}
}
{"id": "03c804cb-b983-4e4c-956b-96ac23da16b2", "file": "UEsDB...qaJXQ=="}
{
"code": 403,
"error": "listname name is required",
"data": {}
}
OK. For what it's worth. This is how I finally got this to work. I now realise that this question was more about my understanding, or lack thereof, of the Python request module.
import requests
import json
from requests.auth import HTTPBasicAuth
import base64
# Get session token, this must be specified in header of subsequent request and returns a JSON object ready for insertion into header
rqstResponse = requests.post('https://manage.fastfieldforms.com/api/authenticate', auth=HTTPBasicAuth('XXX', 'XXX'))
jsonObj = json.loads(rqstResponse.content)
sessionToken = jsonObj['data']['sessionToken']
headers = {'X-Gatekeeper-SessionToken': sessionToken}
def getGloballistsDetails():
# run this code to get listIds - which are hard coded further down
rqstResponse = requests.get("https://manage.fastfieldforms.com/api/globallists", headers=headers)
print (rqstResponse.content)
del rqstResponse
def createGloballist(listname, filePath):
# usage example createGloballist("test01", r"c:\temp\test.xlsx")
files = {'file': open(filePath, 'rb')}
data = {'listname': listname}
rqstResponse = requests.post("https://manage.fastfieldforms.com/api//globallist", files=files, data=data, headers=headers)
print (rqstResponse.content)
del rqstResponse
def updateGloballist(id, filePath):
# usage example createGloballist("f03c7db1-cfea-4486-8350-53381ac048b4", r"c:\temp\test.xlsx")
files = files = {'file': open(filePath, 'rb')}
data = {'id': id}
rqstResponse = requests.post("https://manage.fastfieldforms.com/api//globallist", files=files, data=data, headers=headers)
print (rqstResponse.content)
del rqstResponse
filePath = r"J:\Properties\PropGIS\proj\20150820140457_TelecoMapping\data\Survey_Feb17\FastField_Test01.xlsx"
getGloballistsDetails()
#createGloballist("Test02", filePath)
updateGloballist('f03c7db1-cfea-4486-8350-53381ac048b4', filePath)
I am trying to send a message via Firebase to a certain client. This is my current (test) code:
import json
import requests
import urllib
def send_message():
server = "https://fcm.googleapis.com/fcm/send"
api_key = "xxx"
user_token = "xxx"
headers = {'Content-Type': 'application/json', 'Authorization': 'key=' + api_key}
data = {"type": "dataUpdate"}
payload = {"data": data, "to": user_token}
payload = json.dumps(payload)
res = requests.post(server, headers=headers, json=payload)
return res
which produces the following error, returned by Firebase:
JSON_PARSING_ERROR: Unexpected token END OF FILE at position 0.
The following JSON sent to Firebase seems correct to me:
{
"data": {
"type":"dataUpdate"
},
"to":"xxx"
}
which is in the format described by the Firebase documentation. Any idea why Firebase doesn't accept the given data?
When you use json=payload as a parameter to requests.post() you don't need to specify 'Content-Type': 'application/json' in the header. In addition, you are passing a string when the parameter should be payload as a dict (ie. no need for json.dumps())
Try this:
def send_message():
server = "https://fcm.googleapis.com/fcm/send"
api_key = "xxx"
user_token = "xxx"
headers = {'Authorization': 'key=' + api_key}
data = {"type": "dataUpdate"}
payload = {"data": data, "to": user_token}
res = requests.post(server, headers=headers, json=payload)
return res