I am working with the request module within python to grab certain fields within the JSON response.
import json
fn = 'download.json'
data = json
response = requests.get('http://api.appannie.com/v1/accounts/1000/apps/mysuperapp/sales?break_down=application+iap&start_date=2013-10-01&end_date=2013-10-02', \
auth=('username', 'password'))
data = response.json()
print(data)
This works in python, as the response is the following:
{'prev_page': None, 'currency': 'USD', 'next_page': None, 'sales_list': [{'revenue': {'ad': '0.00', 'iap': {'refunds': '0.00', 'sales': '0.00', 'promotions': '0.00'}, 'app': {'refunds': '0.00', 'updates': '0.00', 'downloads': '0.00', 'promotions': '0.00'}},
'units': {'iap': {'refunds': 0, 'sales': 0, 'promotions': 0}, 'app': {'refunds': 0, 'updates': 0, 'downloads': 2000, 'promotions': 0}}, 'country': 'all', 'date': 'all'}], 'iap_sales': [], 'page_num': 1, 'code': 200, 'page_index': 0}
The question is how do I parse this to get my downloads number within the 'app' block - namely the "2000" value?
After the response.json() data is already a dictionary otherwise response.json() would raise an exception. Therefore you can access it just like any other dictionary.
You can use the loads() method of json -
import json
response = requests.get('http://api.appannie.com/v1/accounts/1000/apps/mysuperapp/sales?break_down=application+iap&start_date=2013-10-01&end_date=2013-10-02',
auth=('username', 'password'))
data = json.loads(response.json()) # data is a dictionary now
sales_list = data.get('sales_list')
for sales in sales_list:
print sales['revenue']['app']
You can use json.loads:
import json
import requests
response = requests.get(...)
json_data = json.loads(response.text)
This converts a given string into a dictionary which allows you to access your JSON data easily within your code.
Related
I want to pick "ocr_text" in this json
How can I pick ocr_text with json.loads
{'message': 'Success', 'result': [{'message': 'Success', 'input': '1.jpg', 'prediction': [{'id': 'a6447ad9-80f7-4bce-bb5e-588bef3874e6', 'label': 'number_plate', 'xmin': 93, 'ymin': 405, 'xmax': 248, 'ymax': 445, 'score': 0.99992895, **'ocr_text': 'MH 02 CB 4545'**, 'type': 'field', 'status': 'correctly_predicted', 'page_no': 0, 'label_id': '45aaf761-4b60-42e9-b9a7-21d7ea8b927a'}], auto=compress&expires=1670532718&or=90&s=373803a82f093ab6b3b68d530f85f294', 'original_with_long_expiry': 'https://nnts.imgix.net/uploadedfiles/59aedc47-df0d-4e93-a52d-dd7076da1287/PredictionImages/658c79d6-c4c7-4ce3-8dfc-41d8884d5719.jpeg?expires=1686070318&or=0&s=849652a08454ccca0ac5cfb779c0cba3'}, 'uploadedfiles/59aedc47-df0d-4e93-a52d-dd7076da1287/RawPredictions/1-2022-12-08T16-51-56.347.jpg': {'original': 'https://nanonets.s3.us-west-2.amazonaws.com/uploadedfiles/59aedc47-df0d-4e93-a52d-dd7076da1287/RawPredictions/1-2022-12-08T16-51-56.347.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA5F4WPNNTLX3QHN4W%2F20221208%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20221208T165158Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&response-cache-control=no-cache&X-Amz-Signature=6ccfc59eb43ffe89dda229ca2a91f09f883596014c7ab0bba6028432f506438d', 'original_compressed': '', 'thumbnail': '', 'acw_rotate_90': '', 'acw_rotate_180': '', 'acw_rotate_270': '', 'original_with_long_expiry': ''}}}
There's an error with the JSON you provided, however I will try my best to answer.
if the json data is within a string in the code, you can use loads like so:
import json
json_string = ... # this is the string with json data
json_dict = json.loads(json_string)
print(json_dict["ocr_text"])
first, I use json.loads to load json data from the json_string into a dictionary (json_dict)
then, I treat it as a regular dictionary with the square brackets.
The python json module documentation has more info if you'd like.
I am sending requests to a crypto network for data on accounts. You get sent back information, but I haven't yet encountered lists being sent in JSON until now. I want to parse certain information, but am having trouble because the JSON is a list and is not as easy to parse compared to normal JSON data.
import requests
import json
url = ' https://s1.ripple.com:51234/'
payload = {
"method": "account_objects",
"params": [
{
"account": "r9cZA1mLK5R5Am25ArfXFmqgNwjZgnfk59",
"ledger_index": "validated",
"type": "state",
"deletion_blockers_only": False,
"limit": 10
}
]
}
response = requests.post(url, data=json.dumps(payload))
print(response.text)
data = response.text
parsed = json.loads(data)
price = parsed['result']
price = price['account_objects']
for Balance in price:
print(Balance)
You will receive all the tokens the account holds and the value. I can not figure out how to parse this correctly and receive the correct one. This particular test account has a lot of tokens so I will only show the first tokens info.
RESULT
{'Balance': {'currency': 'ASP', 'issuer': 'rrrrrrrrrrrrrrrrrrrrBZbvji', 'value': '0'}, 'Flags': 65536, 'HighLimit': {'currency': 'ASP', 'issuer': 'r9cZA1mLK5R5Am25ArfXFmqgNwjZgnfk59', 'value': '0'}, 'HighNode': '0', 'LedgerEntryType': 'RippleState', 'LowLimit': {'currency': 'ASP', 'issuer': 'r3vi7mWxru9rJCxETCyA1CHvzL96eZWx5z', 'value': '10'}, 'LowNode': '0', 'PreviousTxnID': 'BF7555B0F018E3C5E2A3FF9437A1A5092F32903BE246202F988181B9CED0D862', 'PreviousTxnLgrSeq': 1438879, 'index': '2243B0B630EA6F7330B654EFA53E27A7609D9484E535AB11B7F946DF3D247CE9'}
I want to get the first bit of info, here. {'Balance': {'currency': 'ASP', 'issuer': 'rrrrrrrrrrrrrrrrrrrrBZbvji', 'value': '0'},
Specifically 'value' and the number
I have tried to take parse 'Balance' but since it is a list it is not as straight forward.
You're mixing up lists and dictionaries. In order to access a dictionary by key, you need to invoke the key, as such:
for Balance in price:
print(Balance['Balance'])
Yields the following results:
{'currency': 'CHF', 'issuer': 'rrrrrrrrrrrrrrrrrrrrBZbvji', 'value': '-0.3488146605801446'}
{'currency': 'BTC', 'issuer': 'rrrrrrrrrrrrrrrrrrrrBZbvji', 'value': '0'}
{'currency': 'USD', 'issuer': 'rrrrrrrrrrrrrrrrrrrrBZbvji', 'value': '-11.68225001668339'}
If you only wanted to extract the value, you simply dive one level deeper:
for Balance in price:
print(Balance['Balance']['value')
Which yields:
-0.3488146605801446
0
-11.68225001668339
I assume that under price['account_objects'] you have a list of dictionaries? And then in each dictionary you have in one of the keys: 'Balance': {'currency': 'ASP', 'issuer': 'rrrrrrrrrrrrrrrrrrrrBZbvji', 'value': '0'. If so, why don't you iterate over the list and then access each dictionary, like:
account_objects = price['account_objects']
for account_object in price:
print(account_object['Balance'])
How can I output the value of 'Number#en' from my response? I am struggling to understand the nested structure. Thanks
Response from my api
{
'count': 1, 'total': 1,
'data': [
{'id': '6a3d7026-43f3-67zt-9211-99dfc6fee82e',
'name': 'test',
'properties': {'Description#en': 'test', 'Number#en': '20934120'}}],
what I have trying to print the value
response = requests.get(url, headers=headers, data=payload)
data_text = json.loads(response.text)
print(data_text[data]['properties.Number#en'])
data_text['data'] is a list of dictionary so to access the Number#en, you should use
data_text['data'][0]['properties']['Number#en']
im using an API to get some responses from surveygizmo. It works, but it is changing the question to [question(1)], [question(2)]...
import surveygizmo as sg
client = sg.SurveyGizmo(
api_version='v4',
# example
api_token = "api_token",
api_token_secret = "api_token_secret."
)
survey_id = "survey_id"
responses = client.api.surveyresponse.list(survey_id)
pages = responses['total_pages']
data = []
responses
I got the following answer:
{'result_ok': True,
'total_count': 5,
'page': 1,
'total_pages': 1,
'results_per_page': 50,
'data': [{'id': '1',
'contact_id': '',
'status': 'Complete',
'is_test_data': '0',
'datesubmitted': '2020-01-22 16:07:30',
'SessionID': '1579727226_5e28b97a9ff992.53369554',
'Language': 'Portuguese (Brazil)',
'datestarted': '2020-01-22 16:07:30',
'iLinkID': '9342723',
'sResponseComment': '',
'responseID': '1',
'[question(2)]': 'Sim',
'[question(3)]': 'Assunto',
'[question(4)]': '8',
...
I need to show the question as it was made. How it is possible to do that?
I found the answer. api_version='v4' has some limitations, the question text came with api_version='v5'.
I'm doing some scraping and looking at pages like this one (https://www.endomondo.com/rest/v1/users/20261627/workouts/526622897), but I have not been able to fully retrieve the JSON content.I have tried using both of the following sets of code, but each returns an incomplete JSON object:
url = 'https://www.endomondo.com/rest/v1/users/%s/workouts/%s'%(string_use_user, string_use_workout)
print(url)
response = urlopen(url)
try:
reader = codecs.getreader("utf-8")
print(reader(response))
jsonresponse = json.load(reader(response))
print(jsonresponse)
and similarly using the response library instead of urllib also fails to retrieve the full JSON
url = 'https://www.endomondo.com/rest/v1/users/%s/workouts/%s'%(string_use_user, string_use_workout)
print("using this url %s"%url)
r = requests.get(url)
try:
print(r.json())
jsonresponse = r.json()# json.loads(response.read())
In both cases I get about 1/4 of the JSON. For example, in this case:
https://www.endomondo.com/rest/v1/users/20261627/workouts/526622897
I received:
{'feed_id': 281475471235835, 'id': 526622897, 'duration': 4082.0, 'local_start_time': '2015-05-21T09:30:45.000+02:00', 'calories': 1073.0, 'tagged_users': [], 'altitude_max': 69.9523, 'sport': 0, 'distance': 11.115419387817383, 'altitud\
e_min': 14.9908, 'include_in_stats': True, 'hydration': 0.545339, 'start_time': '2015-05-21T07:30:45.000Z', 'ascent': 137.162, 'is_live': False, 'pb_count': 2, 'playlist': [], 'is_peptalk_allowed': False, 'weather': {'wind_speed': 11, '\
temperature': 12, 'wind_direction': 13, 'type': 3, 'humidity': 81}, 'speed_max': 24.8596, 'author': {'name': 'gfdgfd', 'id': 20261627, 'last_name': 'gdsgsk', 'gender': 0, 'expand': 'abs', 'picture': {'url': 'https://www.endom\
ondo.com/resources/gfx/picture/18511427/thumbnail.jpg'}, 'first_name': 'gdsgds', 'viewer_friendship': 1, 'is_premium': False}, 'sharing': [{'share_time': '2015-05-21T08:45:19.000Z', 'type': 0, 'share_id': 1635690786663532}], 'show_map':\
0, 'pictures': [], 'hashtags': [], 'descent': 150.621, 'speed_avg': 9.80291763746756, 'expand': 'full', 'show_workout': 0, 'points': {'expand': 'ref', 'id': 2199549878449}}
I am not receiving the long arrays within the data. I am also not even recovering all of the non-array data.
I ran the original page through a JSON validator, and it's fine. Similarly, I ran the JSON I do receive through a validator, and it's also fine - it doesn't show any signs of missing things unless I compare with the original.
I would appreciate any advice about how to troubleshoot this. Thanks.
Looks like this API is doing some User-Agent sniffing and only sending the complete content for what it considers to be actual web browsers.
Once you set a User-Agent header with the UA string of a common browser, you get the full response:
>>> UA = 'Mozilla/5.0 (X11; Linux x86_64; rv:36.0) Gecko/20100101 Firefox/36.0'
>>> url = 'https://www.endomondo.com/rest/v1/users/20261627/workouts/526622897'
>>> r = requests.get(url, headers={'User-Agent': UA})
>>>
>>> print len(r.content)
96412
See the requests docs for more details on setting custom headers.