I am trying to learn how to manipulate data in python.
I have the following data in a txt file
{"summonerId":000000,"games":[{"gameId":111111,"invalid":false,"gameMode":"CLASSIC","gameType":"MATCHED_GAME","subType":"NORMAL","mapId":11,"teamId":200,"championId":89,"spell1":3,"spell2":4,"level":30,"ipEarned":237,"createDate":1443314494341,"fellowPlayers":[{"summonerId":46350758,"teamId":100,"championId":157}],"stats":{"level":15,"goldEarned":10173,"numDeaths":5,"minionsKilled":48,"championsKilled":1,"goldSpent":9205,"totalDamageDealt":48752,"totalDamageTaken":23464,"team":200,"win":true,"largestMultiKill":1,"physicalDamageDealtPlayer":9064,"magicDamageDealtPlayer":35714,"physicalDamageTaken":18944,"magicDamageTaken":4005,"timePlayed":1831,"totalHeal":4129,"totalUnitsHealed":5,"assists":24,"item0":3401,"item1":2049,"item2":3117,"item3":3068,"item4":3075,"item5":1028,"item6":3340,"magicDamageDealtToChampions":9062,"physicalDamageDealtToChampions":3348,"totalDamageDealtToChampions":12411,"trueDamageDealtPlayer":3974,"trueDamageTaken":514,"wardKilled":1,"wardPlaced":16,"totalTimeCrowdControlDealt":104,"playerRole":2,"playerPosition":4}]}
My end goal is to be able to display a specific piece of information from the "stats" dictionary.
When I run the following code
import json
matches = open('testdata.txt', 'r')
output = matches.read()
data=json.loads(output)
display = data["games"]
print("Info: " + str(display))
The output is everything that corresponds to the "games" key as I would expect.
When I try
import json
matches = open('testdata.txt', 'r')
output = matches.read()
data=json.loads(output)
display = data["games"]["stats"]
print("Info: " + str(display))
I receive: TypeError: list indices must be integers, not str
I'm not really sure how to proceed given that the key is clearly a string and not an integer...
Your data["games"] value is a list; each element in that list is a dictionary, and it is those dictionaries in the list that (may) have the 'stats' key. A list can contain 0 or more elements; in this specific case there is just 1 but there could be more or none.
Loop over the list of dictionaries, or pick a specific dictionary from the list with indexing. Since there is only one in your specific example, you could just index that 1 element with the 0 index:
display = data["games"][0]["stats"]
Related
I have a list of personal data(id_code,birth_year,born_in) and i want to sort the any arrays in list but i have a problem in this work.
my list data :
data = [
'id_code:3211238576;birth_year:1350;born_in:Boushehr',
'id_code:9801233575;born_in:Argentina;birth_year:1360',
'born_in:Portugal;id_code:0219206431;birth_year:1358',
'id_code:0021678913;born_in:Shiraz;birth_year:1120',
'id_code:1101102135;born_in:Gilan;birth_year:1152',
]
The code I wrote and has an bug:
for i in data:
s = ''.join(sorted(i))
print(s)
my code output:
01112233355678:::;;B___abbcddeeehhhiiinnooorrrrstuy
00112333556789:::;;A___aabbcddeeeghiiiinnnnoorrrrtty
00111223345689:::;;P___aabbcddeeghiiilnnooorrrrttuy
00011112236789:::;;S___aabbcddeehhiiiinnoorrrrtyz
00111111122355:::;;G___aabbcddeehiiiilnnnoorrrty
But! The code to i want to have in output(True answer):
id_code:3211238576,born_in:Boushehr,birth_year:1350
id_code:9801233575,born_in:Argentina,birth_year:1360
id_code:0219206431,born_in:Portugal,birth_year:1358
id_code:0021678913,born_in:Shiraz,birth_year:1120
id_code:1101102135,born_in:Gilan,birth_year:1152
Please help me to solve this problem
Assuming you want your fields to be in specific order, try this one: (I put comments in code for clarification):
data = [
'id_code:3211238576;birth_year:1350;born_in:Boushehr',
'id_code:9801233575;born_in:Argentina;birth_year:1360',
'born_in:Portugal;id_code:0219206431;birth_year:1358',
'id_code:0021678913;born_in:Shiraz;birth_year:1120',
'id_code:1101102135;born_in:Gilan;birth_year:1152',
]
def sorter(x: str):
# getting the field name
field = x.split(':')[0]
# returning it's index from "sorted_by" list
return sorted_by.index(field)
# The index of these fields will be used for sorting in "sorter" function.
sorted_by = ['id_code', 'born_in', 'birth_year']
result = []
for item in data:
# splitting the fields
splited = item.split(';')
splited.sort(key=sorter)
# building the line back and append it
result.append(';'.join(splited))
for i in result:
print(i)
output :
id_code:3211238576;born_in:Boushehr;birth_year:1350
id_code:9801233575;born_in:Argentina;birth_year:1360
id_code:0219206431;born_in:Portugal;birth_year:1358
id_code:0021678913;born_in:Shiraz;birth_year:1120
id_code:1101102135;born_in:Gilan;birth_year:1152
Now you can easily change the fields order in sorted_by list and see the result.
Try
out = [';'.join(reversed(sorted(x.split(';')))) for x in data]
print(out)
This takes every element of the data list and splits it in three strings, each of which contains one of the three attributes. Then, it arranges the three strings in reversed alphabetical order and joins them back into one string, separated by ';'
I have a list of dictionaries in a json file.
I have iterated through the list and each dictionary to obtain two specific key:value pairs from each dictionary for each element.
i.e. List[dictionary{i(key_x:value_x, key_y:value_y)}]
My question is now:
How do I place these two new key: value pairs in a new list/dictionary/array/tuple, representing the two key: value pairs extracted for each listed element in the original?
To be clear:
ORIGINAL_LIST (i.e. with each element being a nested dictionary) =
[{"a":{"blah":"blah",
"key_1":value_a1,
"key_2":value_a2,
"key_3":value_a3,
"key_4":value_a4,
"key_5":value_a5,},
"b":"something_a"},
{"a":{"blah":"blah",
"key_1":value_b1,
"key_2":value_b2,
"key_3":value_b3,
"key_4":value_b4,
"key_5":value_b5,},
"b":"something_b"}]
So my code so far is:
import json
from collections import *
from pprint import pprint
json_file = "/some/path/to/json/file"
with open(json_file) as json_data:
data = json.load(json_data)
json_data.close()
for i in data:
event = dict(i)
event_key_b = event.get('b')
event_key_2 = event.get('key_2')
print(event_key_b)#print value of "b" for each nested dict for 'i'
print(event_key_2)#print value of "key_2" for each nested dict for 'i'
To be clear:
FINAL_LIST(i.e. with each element being a nested dictionary) =
[{"b":"something_a", "key_2":value_2},
{"b":"something_b", "key_2":value_2}]
So I have an answer to getting the keys into individual dictionaries, as follows in the code below. The only problem is that the value for 'key_2' in the original json dictionaries is either an int value or it is "" for values which are 0. My script just returns 'None' for all instances of value_2 for key_2. How can I get it to read the appropriate values for 'value_2'? I want to only return dictionaries for cases where 'value_2' > 0 (i.e. where value_2 != "")
Below is the current code:
import json
from pprint import pprint
json_file = "/some/path/to/json/file"
with open(json_file) as json_data:
data = json.load(json_data)
json_data.close()
for i in data:
event_key_b = event.get('b')
for x in i:
event_key_2 = event.get('key_2')
x = {'b' : something_b, 'key_2' : value_2}
print(x)
Also, if there are any more elegant solutions anyone can think of I would really be interested in learning them ... Some of the json files I'm looking at can range from 200 dictionary entries in the original list to 2,000,000. I'm planning to feed my parsed results into a message queue for processing by a different service and any efficiencies in the code will help for scalability in processing. Also if anyone has any recommendations to give on Redis vs. RabbitMQ, I'd really appreciate it
I have two values number and sys_id. I have made seperate list for both of the values. How can save it in any other data structure like dictionary, or something else because i those list of number and sys_id are related. I am doing it in Python
Below Is the code what i have done
ticket_num.append(resp['result'][idx]['number'])
sys_id.append(resp['result'][idx]['sys_id']) ```
This is making two list one for ticket_num and sys_id. As Ticket number and sys_id are related for example ticket_num = ['INC00012','INC00013','INC00014' ] ,
sys_id = ['644323432sfasesdf213', '644323432dfgdfkdskrwwr', 'f283044423fdldsf09']
As this list are related like ticket_num[0] is directly link with sys_id[0]
So can i make a dictionary that contains ticket_num, sys_id directly without creating lists(for e.g. : {ticket_num : '...' , sys_id = '....' , ..... }
Use zip with dict
Ex:
ticket_num = ['INC00012','INC00013','INC00014' ]
sys_id = ['644323432sfasesdf213', '644323432dfgdfkdskrwwr', 'f283044423fdldsf09']
print(dict(zip(ticket_num, sys_id)))
Output:
{'INC00012': '644323432sfasesdf213',
'INC00013': '644323432dfgdfkdskrwwr',
'INC00014': 'f283044423fdldsf09'}
Welcome to Stackoverflow.
Do you actually need the lists of ticket numbers and IDs? If not that you could instead consider building the structure you need instead of the lists.
You don't say whether you want to be able to look up IDs from ticket numbers or vice versa. This solution allows you to do either:
idx_from_ticket = {}
ticket_from_idx = {}
# In the loop that produces the values, instead of the current appends ...
temp = resp['result'][idx]
idx = temp['sys_id]
number = temp['number']
idx_from_ticket[number] = idx
ticket_from_idx[idx] = number
The two dictionaries can then be used to correlate the IDs and ticket numbers. If you want to actually do something else then I hope this code gives you enough clues.
If you do already have the lists and want to retain them then the zip function is your friend.
idx_from_ticket = dict(zip(ticket_num, sys_id))
ticket_from_idx = dict(zip(sys_id, ticket_num))
zip, when called with two argument, yields a sequence of two-element tuples, which the
dict function assumes are key/value pairs.
I am asking an ElasticSearch database to provide me with a list of indices and their creation dates using Python 2.7 and the Requests package. The idea is to quickly calculate which indices have exceeded the retention policy and need to be put to sleep.
The request works perfectly and the results are exactly what I want. However, when I run the code below, when I try to convert the json result to a dict, the type of theDict is correct but it reports a size of 1, when there should be at least a couple dozen entries. What am I doing wrong? I have a feeling it's something really dumb but I just can't snag it! :)
import json
import requests
esEndPoint = "https://localhost:9200"
retrieveString = "/_cat/indices?h=index,creation.date.string&format=json&s=creation.date"
# Gets the current indices and their creation dates
def retrieveIndicesAndDates():
try:
theResult = requests.get(esEndPoint+retrieveString)
print (theResult.content)
except Exception as e:
print("Unable to retrieve list of indices with creation dates.")
print("Error: "+e)
exit(3)
return theResult.content
def main():
theDict = dict(json.loads(retrieveIndicesAndDates()))
print(type(theDict)) # Reports correct type
print(len(theDict)) # Always outputs "1" ??
for index, creationdate in theDict.items():
print("Index: ",index,", Creation date: ",theDict[index])
return
The json the call returns:
[{"index":".kibana","creation.date.string":"2017-09-14T15:01:38.611Z"},{"index":"logstash-2018.07.23","creation.date.string":"2018-07-23T00:00:01.024Z"},{"index":"cwl-2018.07.23","creation.date.string":"2018-07-23T00:00:03.877Z"},{"index":"k8s-testing-internet-2018.07.23","creation.date.string":"2018-07-23T14:19:10.024Z"},{"index":"logstash-2018.07.24","creation.date.string":"2018-07-24T00:00:01.023Z"},{"index":"k8s-testing-internet-2018.07.24","creation.date.string":"2018-07-24T00:00:01.275Z"},{"index":"cwl-2018.07.24","creation.date.string":"2018-07-24T00:00:02.157Z"},{"index":"k8s-testing-internet-2018.07.25","creation.date.string":"2018-07-25T00:00:01.022Z"},{"index":"logstash-2018.07.25","creation.date.string":"2018-07-25T00:00:01.186Z"},{"index":"cwl-2018.07.25","creation.date.string":"2018-07-25T00:00:04.012Z"},{"index":"logstash-2018.07.26","creation.date.string":"2018-07-26T00:00:01.026Z"},{"index":"k8s-testing-internet-2018.07.26","creation.date.string":"2018-07-26T00:00:01.185Z"},{"index":"cwl-2018.07.26","creation.date.string":"2018-07-26T00:00:02.587Z"},{"index":"k8s-testing-internet-2018.07.27","creation.date.string":"2018-07-27T00:00:01.027Z"},{"index":"logstash-2018.07.27","creation.date.string":"2018-07-27T00:00:01.144Z"},{"index":"cwl-2018.07.27","creation.date.string":"2018-07-27T00:00:04.485Z"},{"index":"ctl-2018.07.27","creation.date.string":"2018-07-27T09:02:09.854Z"},{"index":"cfl-2018.07.27","creation.date.string":"2018-07-27T11:12:44.681Z"},{"index":"elb-2018.07.27","creation.date.string":"2018-07-27T11:13:51.340Z"},{"index":"cfl-2018.07.24","creation.date.string":"2018-07-27T11:45:23.697Z"},{"index":"cfl-2018.07.23","creation.date.string":"2018-07-27T11:45:24.646Z"},{"index":"cfl-2018.07.25","creation.date.string":"2018-07-27T11:45:25.700Z"},{"index":"cfl-2018.07.26","creation.date.string":"2018-07-27T11:45:26.341Z"},{"index":"elb-2018.07.24","creation.date.string":"2018-07-27T11:45:27.440Z"},{"index":"elb-2018.07.25","creation.date.string":"2018-07-27T11:45:29.572Z"},{"index":"elb-2018.07.26","creation.date.string":"2018-07-27T11:45:36.170Z"},{"index":"logstash-2018.07.28","creation.date.string":"2018-07-28T00:00:01.023Z"},{"index":"k8s-testing-internet-2018.07.28","creation.date.string":"2018-07-28T00:00:01.316Z"},{"index":"cwl-2018.07.28","creation.date.string":"2018-07-28T00:00:03.945Z"},{"index":"elb-2018.07.28","creation.date.string":"2018-07-28T00:00:53.992Z"},{"index":"ctl-2018.07.28","creation.date.string":"2018-07-28T00:07:19.543Z"},{"index":"k8s-testing-internet-2018.07.29","creation.date.string":"2018-07-29T00:00:01.026Z"},{"index":"logstash-2018.07.29","creation.date.string":"2018-07-29T00:00:01.378Z"},{"index":"cwl-2018.07.29","creation.date.string":"2018-07-29T00:00:04.100Z"},{"index":"elb-2018.07.29","creation.date.string":"2018-07-29T00:00:59.241Z"},{"index":"ctl-2018.07.29","creation.date.string":"2018-07-29T00:06:44.199Z"},{"index":"logstash-2018.07.30","creation.date.string":"2018-07-30T00:00:01.024Z"},{"index":"k8s-testing-internet-2018.07.30","creation.date.string":"2018-07-30T00:00:01.179Z"},{"index":"cwl-2018.07.30","creation.date.string":"2018-07-30T00:00:04.417Z"},{"index":"elb-2018.07.30","creation.date.string":"2018-07-30T00:01:01.442Z"},{"index":"ctl-2018.07.30","creation.date.string":"2018-07-30T00:08:28.936Z"},{"index":"cfl-2018.07.30","creation.date.string":"2018-07-30T06:52:16.739Z"}]
Your error is trying to convert a list of dicts to a dict:
theDict = dict(json.loads(retrieveIndicesAndDates()))
# ^^^^^ ^
That would only work for a dict of lists. It would be redundant, though.
Just use the reply directly. Each entry is a dict with the appropriate keys:
data = json.loads(retrieveIndicesAndDates())
for entry in data:
print("Index: ", entry["index"], ", Creation date: ", entry["creation.date.string"])
So what happens when you do convert that list to a dict? Why is there just one entry?
The dict understands three initialisation methods: keywords, mappings and iterables. A list fits the last one.
Initialisation from an iterable goes through it and expects key-value iterables as elements. If one were to do it manually, it would look like this:
def sequence2dict(sequence):
map = {}
for element in sequence:
key, value = element
map[key] = value
return map
Notice how each element is unpacked via iteration? In the reply each element is a dict with two entries. Iteration on that yields the two keys but ignores the values.
key, value = {"index":".kibana","creation.date.string":"2017-09-14T15:01:38.611Z"}
print(key, '=>', value) # prints "index => creation.date.string"
To the dict constructor, every element in the reply has the same key-value pair: "index" and "creation.date.string". Since keys in a dict are unique, all elements collapse to the same entry: {"index": "creation.date.string"}.
I've run into the following issue, with my code below. Basically I have a list of objects with an id and a corresponding weight, and I have another list of id's. I want to use only the weights of the objects matching the id's in the second list.
d_weights = [{'d_id':'foo', 'weight': -0.7427}, ...]
d_ids = ['foo', ...]
for dtc_id in d_ids:
d_weight = next((d['weight'] for d in d_weights if d['d_id'] == dtc_id), "")
print str(d_weight)
if str(d_weight) != "":
print "not empty string! "+str(d_weight)
The output for this is:
-0.7427
0.0789
-0.0039
-0.2436
-0.0417
not empty string! -0.0417
Why is only the last one not empty when I can print them fine and they are obviously not equal to an empty string? How do I check that the next() actually returned something before using it?
You haven't correct algorithm.
So d_weight = next((d['weight'] for d in d_weights if d['d_id'] == dtc_id), "") iterate only once.
On every cycle for weight_dict in d_weights: you've got only first dict of d_weights.
Without more data, i can't reproduce your output.
In my case it works fine:
-0.7427
not empty string! -0.7427
-0.327
not empty string! -0.327
Correct code you can find in DhiaTN's answer.
just iterate of the list of the keys and get the values from each dict :
for weight_dict in d_weights
for key in d_ids:
print weight_dict.get(key, "")