We Keep Coding

Json To Angular

I am using MSGraph Api with Python as a backend and angular as a frontend to pull data but given output is showing in json format with all metadata i want to show specific data from it. How can i do that.
Python Code:
#app.route('/getChannel',methods=['GET','POST'])
def Channels():
token = _get_token_from_cache(app_config.SCOPE)
if not token:
return redirect(url_for("login"))
channel_data = requests.get(
app_config.ChannelURL,
headers={'Authorization': 'Bearer ' + token['access_token']},
).json()['value']
return render_template('team.html',result=channel_data)
json output
[
{
"createdDateTime": "2021-05-08T04:47:39.67Z",
"description": null,
"displayName": "General",
"email": "",
"id": "19:c9f316c845794a21a1ff2ba50be2bb1e#thread.tacv2",
"isFavoriteByDefault": null,
"membershipType": "standard",
"webUrl":
},
{
"createdDateTime": "2021-05-08T05:00:11.348Z",
"description": null,
"displayName": "Developement",
"email": "",
"id": "19:da1aff92f51f416c9390881e2fa70716#thread.tacv2",
"isFavoriteByDefault": true,
"membershipType": "standard",
"webUrl":
},
{
"createdDateTime": "2021-05-09T15:16:46.27Z",
"description": "testing",
"displayName": "channel1",
"email": "",
"id": "19:e81758e4858e4f598d31214282d6c100#thread.tacv2",
"isFavoriteByDefault": false,
"membershipType": "standard",
"webUrl":
}
]

Getting a request error 422 upon get request

I have seen some posts about this but most of them used requests.post and I want a get request.
I am making a program that simply goes to a url and gets a list of orders.
This is the example response:
{
"page": 1,
"pageCount": 22,
"orderCount": 214,
"orders": [
{
"id": "c1497823-370c-4c7a-82cd-dacddb36fc30",
"productId": "1a641ba5-38df-4acb-86f7-f5c031e538a0",
"email": "demoemail#autobuy.io",
"ipAddress": "127.0.0.1",
"total": 18,
"currency": "USD",
"gateway": "Stripe",
"isComplete": true,
"createdAtUtc": "2019-10-15T01:44:10.5446599+00:00"
},
{
"id": "228f4ca4-5001-4c19-8350-f960f13d35a7",
"productId": "a0041cc0-2bc6-40a2-9084-5880bae5ecec",
"email": "demoemail#autobuy.io",
"ipAddress": "127.0.0.1",
"total": 50,
"currency": "USD",
"gateway": "Stripe",
"isComplete": true,
"createdAtUtc": "2019-10-15T01:43:17.8322919+00:00"
},
{
"id": "71aed9b2-4bd2-4a49-9e6a-82119e6e05bf",
"productId": "2e0ac75b-bfea-42f1-ad60-ead17825162a",
"email": "demoemail#autobuy.io",
"ipAddress": "127.0.0.1",
"total": 6,
"currency": "USD",
"gateway": "Stripe",
"isComplete": true,
"createdAtUtc": "2019-10-14T23:54:44.6217478+00:00"
}
]
}
This is what I get:
<Response [422]>
My code:
import requests
url = "https://autobuy.io/api/Orders?page=1"
headers = {
"Cache-Control": "no-cache",
"Pragma": "no-cache",
"APIKey": "<APIKEY>"
}
req = requests.get(url, headers=headers)
print(req)
Documentation for the api I'mtrying to use https://api.autobuy.io/?version=latest
edit: on their site it says
HEADERS
APIKey
2cdbdc48-b297-41ad-a234-329db0d2dbea
AutoBuy Api Key found in your shop settings
but then when I remove my Cache-Control and Pragma headers I get an error that has to do with my headers being cached (cause the site is behind a CDN?)

Ended up being a bug in the site, my dashboard was showing an invalid key.

How can I format this JSON String into a JSON object?

I have the following JSON document being sent from an SNS HTTPs subscription to an API Gateway endpoint (backed by a Python2.7 Lambda function). In this case, I have a Cloudwatch Alarm that's configured to send to SNS, which then sends to this Gateway endpoint. SNS packages that alarm (which is JSON) into the "body" field of the message it sends to Gateway.
I need to extract the JSON document out of the "body" field, but by this point it's been properly mangled with escape characters and new lines and json.loads() is not liking it at all.
How can I read the value of "body" back into a JSON document using Python2.7 within a Lambda function? I've tried cleaning it by removing '\n' and '\', but I'm just striking out!
Here is the JSON as received by Lambda:
{
"body": "{\n \"Type\" : \"Notification\",\n \"MessageId\" : \"944c9xxx3-c98d636ff2c7\",\n \"TopicArn\" : \"arn:aws:sns:us-west-2:xxx6xx:sxxxr-sns-topic\",\n \"Subject\" : \"ALARM: \\\"hhh\\\" in US West (Oregon)\",\n \"Message\" : \"{\\\"AlarmName\\\":\\\"hhh\\\",\\\"AlarmDescription\\\":null,\\\"AWSAccountId\\\":\\\"8xxx\\\",\\\"NewStateValue\\\":\\\"ALARM\\\",\\\"NewStateReason\\\":\\\"Threshold Crossed: 1 out of the last 1 datapoints [0.333370380661336 (13/06/18 18:06:00)] was greater than or equal to the threshold (0.1) (minimum 1 datapoint for OK -> ALARM transition).\\\",\\\"StateChangeTime\\\":\\\"2018-06-13T18:16:56.457+0000\\\",\\\"Region\\\":\\\"US West (Oregon)\\\",\\\"OldStateValue\\\":\\\"INSUFFICIENT_DATA\\\",\\\"Trigger\\\":{\\\"MetricName\\\":\\\"CPUUtilization\\\",\\\"Namespace\\\":\\\"AWS/EC2\\\",\\\"StatisticType\\\":\\\"Statistic\\\",\\\"Statistic\\\":\\\"AVERAGE\\\",\\\"Unit\\\":null,\\\"Dimensions\\\":[{\\\"name\\\":\\\"InstanceId\\\",\\\"value\\\":\\\"i-07bxxx26\\\"}],\\\"Period\\\":300,\\\"EvaluationPeriods\\\":1,\\\"ComparisonOperator\\\":\\\"GreaterThanOrEqualToThreshold\\\",\\\"Threshold\\\":0.1,\\\"TreatMissingData\\\":\\\"\\\",\\\"EvaluateLowSampleCountPercentile\\\":\\\"\\\"}}\",\n \"Timestamp\" : \"2018-06-13T18:16:56.486Z\",\n \"SignatureVersion\" : \"1\",\n \"Signature\" : \"fFunXkjjxxxvF7Kmxxx\",\n \"SigningCertURL\" : \"https://sns.us-west-2.amazonaws.com/SimpleNotificationService-xxx.pem\",\n \"UnsubscribeURL\" : \"https://sns.us-west-2.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=axxxd\"\n}",
"resource": "/message",
"requestContext": {
"requestTime": "13/Jun/2018:18:16:56 +0000",
"protocol": "HTTP/1.1",
"resourceId": "m4sxxxq",
"apiId": "2v2cthhh",
"resourcePath": "/message",
"httpMethod": "POST",
"requestId": "f41e8-8cbd-57ad9e625d12",
"extendedRequestId": "xxx",
"path": "/stage/message",
"stage": "stage",
"requestTimeEpoch": 1528913816627,
"identity": {
"userArn": null,
"cognitoAuthenticationType": null,
"accessKey": null,
"caller": null,
"userAgent": "Amazon Simple Notification Service Agent",
"user": null,
"cognitoIdentityPoolId": null,
"cognitoIdentityId": null,
"cognitoAuthenticationProvider": null,
"sourceIp": "xxx",
"accountId": null
},
"accountId": "xxx"
},
"queryStringParameters": {
"id": "CBxxx69"
},
"httpMethod": "POST",
"pathParameters": null,
"headers": {
"Content-Type": "text/plain; charset=UTF-8",
"Via": "1.1 xxx.cloudfront.net (CloudFront)",
"Accept-Encoding": "gzip,deflate",
"CloudFront-Is-SmartTV-Viewer": "false",
"x-amz-sns-subscription-arn": "arn:aws:sns:us-west-2:xxx:sxxx-nxxx-sns-topic:xxx",
"CloudFront-Forwarded-Proto": "https",
"X-Forwarded-For": "54.240.xxx, 54.182.xxx",
"CloudFront-Viewer-Country": "US",
"User-Agent": "Amazon Simple Notification Service Agent",
"X-Amzn-Trace-Id": "Root=1-5b21xxx53acea6642317ed4",
"x-amz-sns-topic-arn": "arn:aws:sns:us-west-2:xxxx:sxxxier-sns-topic",
"Host": "2vxxx.execute-api.us-west-2.amazonaws.com",
"X-Forwarded-Proto": "https",
"X-Amz-Cf-Id": "xxx",
"CloudFront-Is-Tablet-Viewer": "false",
"X-Forwarded-Port": "443",
"x-amz-sns-message-type": "Notification",
"CloudFront-Is-Mobile-Viewer": "false",
"x-amz-sns-message-id": "xxx",
"CloudFront-Is-Desktop-Viewer": "true"
},
"stageVariables": null,
"path": "/message",
"isBase64Encoded": false
}

if i use your pasted sample as a raw string, it works well:
>>> j = r'''...your sample pasted here...'''
>>> data = json.loads(j)
>>> bodydata = json.loads(data['body'])
>>> bodydata['Type']
u'Notification'
seems, that what you pasted above is the repr form, printed out with Python

Need help formatting/parsing this list

I'm not a developer so sorry if this is a dumb question or if my terminology is incorrect. I'm writing a script to make calls to our CMDB's API but i'm not sure how to handle the data that is being sent back from it. It appears to be a list type but I can't reference anything by key names. Is there a way to convert it to something that i can easily manipulate and pull data out of?
Here is my code:
import requests
import json
r=requests.post('API.URL', data={'grant_type': 'password', 'client_id':'#######', 'username': 'user', 'password': 'password'})
json_data = json.loads(r.content)
token = json_data['access_token']
data ={
"filters": [
{
"fieldId": "937905400191ae67dd03ab4b79968fcbaa264b1a75",
"operator": "eq",
"value": "hostname"
}
],
"fields":[
'9426b6ddf3cb971488517145e39efc5aa7f16fec46',
'9343f8800b3917f26533954918a6388ae8c863507f',
'9379053db492ece14816704ef5a9e3e567e217511b',
'9343f93fc4c8422bcf24e74a9a86035bb7d0248b00',
'941ba290776d6f51ce35664246927b958330a753b2'
],
"association": "Configuration Item",
"busObId": "93dada9f640056ce1dc67b4d4bb801f69104894dc8",
"includeAllFields": 'false',
"pageNumber": 0,
"pageSize": 300,
"scope": "Global",
"scopeOwner": "(None)",
"searchName": "APItest"
}
payload = json.dumps(data)
headers = {'Content-Type': 'application/json', 'Accept': 'application/json', 'Authorization':'bearer '+token}
search=requests.post('http://API.URL', headers=headers, data=payload)
search_json = json.loads(search.content)
bo = search_json['businessObjects']
print(bo)
Here's the response:
[
{
"busObRecId": "9423ad7d617390fdc956ee4302a69d0ccf1a37a4c1",
"hasError": false,
"links": [
{
"url": "http://URL",
"name": "Delete Record"
}
],
"fields": [
{
"displayName": "Business Sponsor",
"name": "Business Sponsor",
"value": "",
"html": null,
"dirty": false,
"fieldId": "9426b6ddf3cb971488517145e39efc5aa7f16fec46"
},
{
"displayName": "Owned By",
"name": "Owned By",
"value": "John Doe",
"html": null,
"dirty": false,
"fieldId": "9343f8800b3917f26533954918a6388ae8c863507f"
},
{
"displayName": "Asset Status",
"name": "Asset Status",
"value": "Active",
"html": null,
"dirty": false,
"fieldId": "9379053db492ece14816704ef5a9e3e567e217511b"
},
{
"displayName": "Description",
"name": "Description",
"value": "Automation Server",
"html": null,
"dirty": false,
"fieldId": "9343f93fc4c8422bcf24e74a9a86035bb7d0248b00"
},
{
"displayName": "Data Center Location",
"name": "Data Center Location",
"value": "",
"html": null,
"dirty": false,
"fieldId": "941ba290776d6f51ce35664246927b958330a753b2"
}
],
"errorMessage": null,
"busObPublicId": "9423ad7d617390fdc956ee4302a69d0ccf1a37a4c1",
"busObId": "93dada9f640056ce1dc67b4d4bb801f69104894dc8",
"errorCode": null
}
]
type() shows the object bo as a list and len() says it only has one element so I'm not sure how to pull data out of it without hacking away at it stripping out characters.

The reason why you cannot reference anything by key names is the fact that your output is a list. A list which is only containing one single dictionary element. If you print out
bo[0]
you get the whole data, without the [ and ] symbols. As for the dictionary now we can access different elements by keys, e.g.:
print(bo["busObId"])
will return the following value:
93dada9f640056ce1dc67b4d4bb801f69104894dc
Let's say you would want to print out the fieldId of the first element of "fields". You can do it the following way:
print(bo[0]["fields"][0]["fieldId"])
Hope this helped.

Iterating through JSON in Python using an OFFSET

I am trying to use the HubSpot CRM API to get "All Deals".
The API endpoint is: https://api.hubapi.com/deals/v1/deal/all?hapikey=demo
The JSON returned looks like this...
{
"deals": [
{
"portalId": 62515,
"dealId": 18039629,
"isDeleted": false,
"associations": {
"associatedVids": [],
"associatedCompanyIds": [],
"associatedDealIds": []
},
"properties": {
"dealname": {
"value": "Company",
"timestamp": 1457040864519,
"source": "API",
"sourceId": null
},
"amount": {
"value": "10",
"timestamp": 1457040864519,
"source": "API",
"sourceId": null
},
"closedate": {
"value": "",
"timestamp": 1457040864519,
"source": "API",
"sourceId": null
},
"hubspot_owner_id": {
"value": "11626092",
"timestamp": 1457046177648,
"source": "SALESFORCE",
"sourceId": null
},
"hs_lastmodifieddate": {
"value": "1457046177662",
"timestamp": 1457046177662,
"source": "CALCULATED",
"sourceId": null
},
"hubspot_owner_assigneddate": {
"value": "1457046177648",
"timestamp": 1457046177648,
"source": "SALESFORCE",
"sourceId": null
},
"num_associated_contacts": {
"value": "0",
"timestamp": 0,
"source": "CALCULATED",
"sourceId": null
},
"hs_createdate": {
"value": "1457040864535",
"timestamp": 1457040864535,
"source": null,
"sourceId": null
},
"createdate": {
"value": "1457040864535",
"timestamp": 1457040864535,
"source": null,
"sourceId": null
},
"hs_salesforceopportunityid": {
"value": "00628000007nRyuAAE",
"timestamp": 1457046177648,
"source": "SALESFORCE",
"sourceId": null
}
},
"imports": []
},
{
"portalId": 62515,
"dealId": 18040854,
"isDeleted": false,
"associations": {
"associatedVids": [],
"associatedCompanyIds": [],
"associatedDealIds": []
},
"properties": {
"dealname": {
"value": "5678",
"timestamp": 1457042290572,
"source": "API",
"sourceId": null
},
"amount": {
"value": "750000.0",
"timestamp": 1457042290572,
"source": "API",
"sourceId": null
},
"closedate": {
"value": "",
"timestamp": 1457042290572,
"source": "API",
"sourceId": null
},
"hs_lastmodifieddate": {
"value": "1457042290592",
"timestamp": 1457042290592,
"source": "CALCULATED",
"sourceId": null
},
"num_associated_contacts": {
"value": "0",
"timestamp": 0,
"source": "CALCULATED",
"sourceId": null
},
"hs_createdate": {
"value": "1457042290592",
"timestamp": 1457042290592,
"source": null,
"sourceId": null
},
"createdate": {
"value": "1457042290592",
"timestamp": 1457042290592,
"source": null,
"sourceId": null
}
},
"imports": []
}
],
"hasMore": true,
"offset": 1467187
}
And I understand that if hasMore==true, then you are supposed to grab the offset and include it in another API call something like this: https://api.hubapi.com/deals/v1/deal/all?hapikey=demo&offset=1467187
And then keep doing that until hasMore==false.
I am using the following code to extract the first chunk of JSON from the API:
import requests
url = "https://api.hubapi.com/deals/v1/deal/all"
querystring = {"hapikey":"demo"}
headers = {
'cache-control': "no-cache"
}
response = requests.request("GET", url, headers=headers, params=querystring)
print(response.text)
So... my question is that now I am getting my JSON, how do I:
1) Read one chunk of JSON
2) If hasMore==true then go do #1 again
3) ElseIf hasMore==false then combine ALL the JSON from ALL iterations of #1 above into one big JSON
4) Return the value from #3
Any help please?

Working solution
import json
import requests
url = "https://api.hubapi.com/deals/v1/deal/all"
querystring = {"hapikey":"demo"}
headers = {
'cache-control': "no-cache"
}
all_deals = []
response = requests.request("GET", url, headers=headers, params=querystring).json()
for deal in response['deals']:
all_deals.append(deal)
hasMore = response['hasMore']
offset = response['offset']
while hasMore:
querystring = {
"hapikey":"demo",
"offset":offset
}
response = requests.request("GET", url, headers=headers, params=querystring).json()
for deal in response['deals']:
all_deals.append(deal)
hasMore = response['hasMore']
offset = response['offset']
print(json.dumps(all_deals))