To be clear, I am practicing my Python skills using CoinMarketCaps API.
The below code works great:
import json
# 1 JSON string in list, works
info_1_response = ['{"status": {"timestamp": "2023-01-25T22:59:58.760Z", "error_code": 0, "error_message": null, "elapsed": 16, "credit_count": 1, "notice": null}, "data": {"BTC": {"id": 1, "name": "Bitcoin", "symbol": "BTC"}}}']
for response in info_1_response:
info_1_dict = json.loads(response)
#print(info_1_dict) #works
data = info_1_dict['data']['BTC']
print(f"only id = {data['id']}")
OUTPUT: only id = 1
However, if I have 2 responses in a list, how would I got about getting the ID for each symbol (BTC/ETH)? Code:
info_2_response = ['{"status": {"timestamp": "2023-01-25T22:59:58.760Z", "error_code": 0, "error_message": null, "elapsed": 16, "credit_count": 1, "notice": null}, "data": {"BTC": {"id": 1, "name": "Bitcoin", "symbol": "BTC"}}}', '{"status": {"timestamp": "2023-01-25T22:59:59.087Z", "error_code": 0, "error_message": null, "elapsed": 16, "credit_count": 1, "notice": null}, "data": {"ETH": {"id": 1027, "name": "Ethereum", "symbol": "ETH"}}}']
for response in info_2_response:
info_2_dict = json.loads(response)
#print(info_2_dict) #works
print(info_2_dict['data']) #works
OUTPUT:
{'BTC': {'id': 1, 'name': 'Bitcoin', 'symbol': 'BTC'}}
{'ETH': {'id': 1027, 'name': 'Ethereum', 'symbol': 'ETH'}}
But what if I only wanted the ID? It seems as if I would need a dynamic parameter as so:
data = info_2_dict['data']['DYNAMIC PARAMETER-(BTC/ETH)']
print(f"only id = {data['id']}")
Desired Output: 1, 1027
is this possible?
Just iterate another level:
>>> for response in info_2_response:
... response = json.loads(response)
... for coin, data in response['data'].items():
... print(coin, data['id'])
...
BTC 1
ETH 1027
We can remove some boilerplate from the body of the loop by using map(json.loads, ...), and since you know there is only one item in the dict, you can get fancy and use iterable unpacking:
>>> for response in map(json.loads, info_2_response):
... [(coin, data), *_] = response['data'].items()
... print(coin, data['id'])
...
BTC 1
ETH 1027
And if you expect there to only be one, you might want an error thrown, so you can do:
>>> for response in map(json.loads, info_2_response):
... [(coin, data)] = response['data'].items()
... print(coin, data['id'])
...
BTC 1
ETH 1027
So note, in the top version, [(coin, data), *_] = response['data'].items() will not fail if there is more than one item in the dict, the rest of the items get assigned to a list called _, which we are ignoring. But that's just a conventional name for a "throwaway" variable.
However, the other version would fail:
>>> response
{'status': {'timestamp': '2023-01-25T22:59:59.087Z', 'error_code': 0, 'error_message': None, 'elapsed': 16, 'credit_count': 1, 'notice': None}, 'data': {'ETH': {'id': 1027, 'name': 'Ethereum', 'symbol': 'ETH'}}}
>>> response['data']['FOO'] = "FOO STUFF"
>>> [(coin, data)] = response['data'].items()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: too many values to unpack (expected 1)
>>>
As long as the data dictionary only has a single key, you can do something like this:
import json
info_2_response = [
"""{"status": {"timestamp": "2023-01-25T22:59:58.760Z", "error_code": 0, "error_message": null, "elapsed": 16, "credit_count": 1, "notice": null},
"data": {"BTC": {"id": 1, "name": "Bitcoin", "symbol": "BTC"}}}""",
"""{"status": {"timestamp": "2023-01-25T22:59:59.087Z", "error_code": 0, "error_message": null, "elapsed": 16, "credit_count": 1, "notice": null},
"data": {"ETH": {"id": 1027, "name": "Ethereum", "symbol": "ETH"}}}""",
]
for response in info_2_response:
info_2_dict = json.loads(response)
print(list(info_2_dict["data"].values())[0]["id"])
This will print:
1
1027
The code works by using the .values() method of a dictionary to get a list of values. Since there's only a single value, we just take the first item from the list and then look up the id attribute.
We can expand the compound statement to make the operations a little more clear:
for response in info_2_response:
info_2_dict = json.loads(response)
all_values = info_2_dict["data"].values()
first_value = list(all_values)[0]
id = first_value["id"]
print(id)
Related
I have the below python dictionary stored as dictPython
{
"paging": {"count": 10, "start": 0, "links": []},
"elements": [
{
"organizationalTarget~": {
"vanityName": "vv",
"localizedName": "ViV",
"name": {
"localized": {"en_US": "ViV"},
"preferredLocale": {"country": "US", "language": "en"},
},
"primaryOrganizationType": "NONE",
"locations": [],
"id": 109,
},
"role": "ADMINISTRATOR",
},
],
}
I need to get the values of vanityName, localizedName and also the values from name->localized and name->preferredLocale.
I tried dictPython.keys() and it returned dict_keys(['paging', 'elements']).
Also I tried dictPython.values() and it returned me what is inside of the parenthesis({}).
I need to get [vv, ViV, ViV, US, en]
I am writing this in a form of answer, so I can get to explain it better without the comments characters limit
a dict in python is an efficient key/value structure or data type
for example dict_ = {'key1': 'val1', 'key2': 'val2'} to fetch key1 we can do it in 2 different ways
dict_.get(key1) this returns the value of the key in this case val1, this method has its advantage, that if the key1 is wrong or not found it returns None so no exceptions are raised. You can do dict_.get(key1, 'returning this string if the key is not found')
dict_['key1'] doing the same .get(...) but will raise a KeyError if the key is not found
So to answer your question after this introduction,
a dict can be thought of as nested dictionaries and/or objects inside of one another
to get your values you can do the following
# Fetch base dictionary to make code more readable
base_dict = dict_["elements"][0]["organizationalTarget~"]
# fetch name_dict following the same approach as above code
name_dict = base_dict["name"]
localized_dict = name_dict["localized"]
preferred_locale_dict = name_dict ["preferredLocale"]
so now we fetch all of the wanted data in their corresponding locations from your given dictionary, now to print the results, we can do the following
results_arr = []
for key1, key2 in zip(localized_dict, preferredLocale_dict):
results_arr.append(localized_dict.get(key1))
results_arr.append(preferred_locale_dict.get(key2))
print(results_arr)
What about:
dic = {
"paging": {"count": 10, "start": 0, "links": []},
"elements": [
{
"organizationalTarget~": {
"vanityName": "vv",
"localizedName": "ViV",
"name": {
"localized": {"en_US": "ViV"},
"preferredLocale": {"country": "US", "language": "en"},
},
"primaryOrganizationType": "NONE",
"locations": [],
"id": 109,
},
"role": "ADMINISTRATOR",
},
],
}
base = dic["elements"][0]["organizationalTarget~"]
c = base["name"]["localized"]
d = base["name"]["preferredLocale"]
output = [base["vanityName"], base["localizedName"]]
output.extend([c[key] for key in c])
output.extend([d[key] for key in d])
print(output)
outputs:
['vv', 'ViV', 'ViV', 'US', 'en']
So something like this?
[[x['organizationalTarget~']['vanityName'],
x['organizationalTarget~']['localizedName'],
x['organizationalTarget~']['name']['localized']['en_US'],
x['organizationalTarget~']['name']['preferredLocale']['country'],
x['organizationalTarget~']['name']['preferredLocale']['language'],
] for x in s['elements']]
I am trying to create lists from json datas by pulling one by one and append them to the lists. However, some variables does not given in all json files. For example: for the json file below, data does not have ['statistics']['aerialLost'] , so it return Key Error. My Expected solution is when json file does not have key, append 'None' value to the list and continue.
Code
s_aerialLost = []
s_aerialWon = []
s_duelLost = []
s_duelWon = []
players = ['Martin Linnes', 'Christian Luyindama', 'Marcão', 'Ömer Bayram', 'Oghenekaro Etebo', 'Muhammed Kerem Aktürkoğlu', 'Gedson Fernandes', 'Emre Kılınç', 'Ryan Babel', 'Mostafa Mohamed', 'Florent Hadergjonaj', 'Tomáš Břečka', 'Duško Tošić', 'Oussama Haddadi', 'Kristijan Bistrović', 'Aytaç Kara', 'Haris Hajradinović', 'Armin Hodžić', 'Gilbert Koomson', 'Isaac Kiese Thelin']
players_id = [109569, 867191, 840951, 68335, 839110, 903324, 862055, 202032, 1876, 873551, 354860, 152971, 14557, 867180, 796658, 128196, 254979, 138127, 341107, 178743]
for player, player_id in zip(players, players_id):
url = base_url + str(player_id)
data = requests.request("GET", url).json()
## just added 4 data for simplify
accurateLongBalls = str(data['statistics']['accurateLongBalls'])
aerialLost = str(data['statistics']['aerialLost'])
aerialWon = str(data['statistics']['aerialWon'])
duelLost = str(data['statistics']['duelLost'])
s_aerialLost.append()
s_aerialWon.append()
s_duelLost.append()
s_duelWon.append()
Json File
{
"player": {
"name": "Martin Linnes",
"slug": "martin-linnes",
"shortName": "M. Linnes",
"position": "D",
"userCount": 339,
"id": 109569,
"marketValueCurrency": "€",
"dateOfBirthTimestamp": 685324800
},
"team": {
"name": "Galatasaray",
"slug": "galatasaray",
"shortName": "Galatasaray",
"gender": "M",
"userCount": 100254,
"nameCode": "GAL",
"national": false,
"type": 0,
"id": 3061,
"teamColors": {
"primary": "#ff9900",
"secondary": "#ff0000",
"text": "#ff0000"
}
},
"statistics": {
"totalPass": 32,
"accuratePass": 22,
"totalLongBalls": 7,
"accurateLongBalls": 3,
"totalCross": 2,
"aerialWon": 1,
"duelLost": 2,
"duelWon": 7,
"totalContest": 3,
"wonContest": 2,
"totalClearance": 4,
"totalTackle": 3,
"wasFouled": 1,
"fouls": 1,
"minutesPlayed": 82,
"touches": 63,
"rating": 7.3,
"possessionLostCtrl": 18,
"keyPass": 1
},
"position": "D"
}
Error
KeyError: 'aerialLost'
Use .get(). You can specify a default value to return if the key is not found, and it defaults to None.
So you can use
aerialLost = str(data.get('statistics', {}).get('aerialLost'))
The first call defaults to an empty dictionary so that there's something to make the second .get() call on. The second call just returns the default None.
I have this sample json data, and need to grab only the MAC addresses so I can convert the mac to a list of manufacturers later.
[
{
"aps": {
"00:20:90:B3:16:25": {
"ssid": "",
"encryption": "Open",
"hidden": 1,
"channel": 11,
"signal": -23,
"wps": 0,
"last_seen": 1594356454,
"clients": []
},
"06:AA:A0:84:7F:D8": {
"ssid": "",
"encryption": "Open",
"hidden": 1,
"channel": 6,
"signal": -75,
"wps": 0,
"last_seen": 1594356452,
"clients": []
},
"1E:51:A4:D4:B7:29": {
"ssid": "",
"encryption": "WPA Mixed PSK (CCMP TKIP)",
"hidden": 1,
"channel": 11,
"signal": -63,
"wps": 0,
"last_seen": 1594356448,
"clients": []
}
}
}
]
This is my python program so far, but im not sure how to isolate the MAC address
import json
f = open('recon_data.json',)
data = json.load(f)
print(data["aps"])
f.close()
I get an error every time I run the program weather im asking for aps or ssid information
Traceback (most recent call last):
File "recon.py", line 12, in
print(data["ssid"])
TypeError: list indices must be integers or slices, not str
This is because the data you're loading is a list. Try data[0]["aps"]
As for getting all the mac addresses they are the keys in that dict so you can just use list on that inner dict to get all the keys:
import json
with open('recon_data.json') as f
data = json.load(f)
print(list(data[0]['aps']))
This will print a list of all the MAC addresses
['00:20:90:B3:16:25', '06:AA:A0:84:7F:D8', '1E:51:A4:D4:B7:29']
The text file contains dictionary of dictionary. In that text file for exmaple "2018" acts as they further "8" is the month which is value for "2018" but key for next dictionary. I want to fetch the "total_queries_count","total_dislike","unique_users" values.
{"2018":
{"8":{ "total_queries_count": 4,
"queries_without_teachers": 3,
"non_teacher_queries": 1,
"total_dislike": 0,
"unique_users": [", "landmark", "232843"],
"user_dislike": 0
},
"9":{ "total_queries_count": 1021,
"queries_without_teachers": 0,
"non_teacher_queries": 1021,
"total_dislike": 0,
"unique_users": [", "1465146", "14657", "dfgf", "1123", "456", "1461546", "Ra", "siva", "234", "ramesh", "3456", "23", "43567", "sfdf", "sdsd", "ra", "sddff", "1234", "rames", "RAM", "444", "123", "333", "RAM", "789", "itassistant", "rame", "12345"],
"user_dislike": 0},
"10": {"total_queries_count": 352,
"queries_without_teachers": 1,
"non_teacher_queries": 351,
"total_dislike": 0,
"unique_users": [", "1465146", "777", "43567", "1234", "456", "123456", "12345", "232843"],
"user_dislike": 0
},
"11": {"total_queries_count": 180,
"queries_without_teachers": 0,
"non_teacher_queries": 180,
"total_dislike": 12,
"unique_users": [", "75757575", "9000115", "9000157", "9000494", "9000164", "123453"],
"user_dislike": 12},
"12": {"total_queries_count": 266,
"queries_without_teachers": 0,
"non_teacher_queries": 266,
"total_dislike": 16,
"unique_users": [", "131422", "121550", "9000508", "9000560", "9000115", "9000371", "9000372", "93979", "146625", "114586", "165937", "9000494", "9000463", "38404", "129458", "62948", "125143", "9000179", "9000145", "9000001", "9000164", "81849", "102663", "9000123", "105407", "33517", "21344", "9000213", "202074", "9000103", "18187", "9000342", "9000125", "9000100", "9000187", "18341", "9000181", "168802", "9000529", "12345", "110127", "9000134", "100190", "9000352", "9000156", "9000055", "tcs_hariharas", "9000078", "204101", "9000050", "9000139"],
"user_dislike": 16}
}
}
Check https://docs.python.org/3/tutorial/datastructures.html#dictionaries
You can access needed keys like this:
# assuming your initial nested dict is called 'data'
data["2018"]["8"]["total_queries_count"]
If you want to aggregate data for all years and months in one place, you can do this:
overall_queries = 0
overall_dislikes = 0
users = set() # this is a set not a list in order to preserve uniqueness of users
for year in data: # year is a key in data dict
for month in data[year]: # month is a key in data[year] dict
users.update(data[year][month]["unique_users"])
overall_queries += data[year][month]["total_queries_count"]
overall_dislikes += data[year][month]["total_dislike"]
If you want to keep your result separated by years you can do this:
result = {}
for year in data:
overall_queries = 0
overall_dislikes = 0
users = set()
for month in data[year]:
overall_queries += data[year][month]["total_queries_count"]
overall_dislikes += data[year][month]["total_dislike"]
users.update(data[year][month]["unique_users"])
result[year] = {
"overall_queries": overall_queries,
"overall_dislikes": overall_dislikes,
"users": users,
}
Result:
{'2018': {'overall_dislikes': 28,
'overall_queries': 1823,
'users': {'100190',
'102663',
'105407',
'110127',
...}}}
I have a json file
{
"rows": [
{
"votes": {
"funny": 0,
"useful": 1,
"cool": 0
},
"user_id": "zvNimI98mrmhgNOOrzOiGg",
"review_id": "I7Kte2FwXWPCwdm7ispu1A",
"text": "Pretty good dinner with a nice selection of food"
},
{
"votes": {
"funny": 2,
"useful": 5,
"cool": 0
},
"user_id": "Au3Qs-AAZEWu2_4gIMwRgw",
"review_id": "SSlO5u2nIJ8PoAKAgN5m3Q",
"text": "Yeah, thats right a five freakin star rating."
}
]
}
I just want to read the "text" one by one i.e. I want to access the first "text", do some operation on it, and then move onto the next "text".
It's a simple matter to open a file, read the contents as JSON, then iterate over the data you get:
import json
with open("my_data.json") as my_data_file:
my_data = json.load(my_data_file)
for row in my_data["rows"]:
do_something(row["text"])
You can simply access the data like in a dict, since your current json data is already one:
>>> text = """{
"rows": [
{
"votes": {
"funny": 0,
"useful": 1,
"cool": 0
},
"user_id": "zvNimI98mrmhgNOOrzOiGg",
"review_id": "I7Kte2FwXWPCwdm7ispu1A",
"text": "Pretty good dinner with a nice selection of food"
},
{
"votes": {
"funny": 2,
"useful": 5,
"cool": 0
},
"user_id": "Au3Qs-AAZEWu2_4gIMwRgw",
"review_id": "SSlO5u2nIJ8PoAKAgN5m3Q",
"text": "Yeah, thats right a five freakin star rating."
}
]
}"""
Assuming the above is your json text, (which can be obtained using a simple
with open("json_file.txt", "r") as f: text = f.read(), you can now get convert the json into a dictionary format using
>>> import json
>>> json_data = json.loads(text)
To access the data, you can now operae normally as you would on a dict.
So, in a list comprehension, this becomes:
>>> print [d["text"] for d in json_data["rows"]]
['Pretty good dinner with a nice selection of food',
'Yeah, thats right a five freakin star rating.']
And in a loop, this becomes
>>> for d in json_data["rows"]:
... print d["text"]
Pretty good dinner with a nice selection of food
Yeah, thats right a five freakin star rating.
Note that the json is not read line by line, it is converted in entirety and only then the required fields are accessed.