I am new to python.I want read a json file and add random values to it.The json contains subset too.I am unable to solve this.
sample.json
{"name": "Kash","age": 12,"loc": {"loc1":"Uk","loc2":"Usa"}}
import json
import random
f=open("sample.json")
data=json.load(f)
def iterate(dictionary):
for key, value in dictionary.items():
dictionary[key]=random.randrange(1,10)
print(dictionary)
if isinstance(value, dict):
iterate(value)
return dictionary
iterate(data)
Output I Got
{'name': 8, 'age': 12, 'loc': {'loc1': 'tc', 'loc2': 'cbe'}}
{'name': 8, 'age': 6, 'loc': {'loc1': 'tc', 'loc2': 'cbe'}}
{'name': 8, 'age': 6, 'loc': 9}
{'loc1': 5, 'loc2': 'cbe'}
{'loc1': 5, 'loc2': 1}
===========================================
Output Expected
{"name": 15,"age": 85,"loc": {"loc1":52,"loc2":36}}
dictionary[key] = random.randrange(1,10)
This is performing:
dictionary['loc'] = some_number
So you lose the nested dict that was already there.
You only want to modify keys that do not have a dict as a value.
def iterate(dictionary):
for key, value in dictionary.items():
if isinstance(value, dict):
iterate(value)
else:
dictionary[key] = random.randrange(1, 10)
return dictionary
If your JSON can contain lists - you will need to handle that case too.
Related
I have a list of Dicts as follows
[{"Sender":"bob","Receiver":"alice","Amount":50},{"Sender":"bob","Receiver":"alice","Amount":60},{"Sender":"bob","Receiver":"alice","Amount":70},{"Sender":"joe","Receiver":"bob","Amount":50},{"Sender":"joe","Receiver":"bob","Amount":150},{"Sender":"alice","Receiver":"bob","Amount":100},{"Sender":"bob","Receiver":"kyle","Amount":260}]
What i need is to sum up the totals per each unique sender/receiver pair, as well as how many total "transactions" there were per pair, as shown below in my desired output
[{"Sender":"bob","Receiver":"alice","Total":180,"Count":3},{"Sender":"joe","Receiver":"bob","Total":"200","Count":2},{"Sender":"alice","Receiver":"bob","Total":"100","Count":1}, {"Sender":"bob","Receiver":"kyle","Total":260,"Count":1}]
What i'm currently doing to get the "total" is
total = sum(a['Amount'] for a in transactions).
But this simply sums up all of the amounts across all pairs, i need the total for each unique pair of sender/receiver i would't know where to begin getting the "count" numbers, either.
Make a new dictionary where the key is the sender/receiver pair.
Iterate over the list of senders/receivers. If that sender/receiver pair does not exist in the new dict, create it. Otherwise increment the count for that pair by one.
newdict = {}
for row in transactions:
sender = row['Sender']
receiver = row['Receiver']
amount = row['Amount']
key = f'{sender},{receiver}'
if key in newdict:
newdict[key]['Total'] += amount
newdict[key]['Count'] += 1
else:
newdict[key] = {'Count': 1, 'Total': amount}
This solution produces a single dict instead of a list of dicts.
lookup for existing value and keep updating it. e.g.
transactions = [
{"Sender":"bob","Receiver":"alice","Amount":50},
{"Sender":"bob","Receiver":"alice","Amount":60},
{"Sender":"bob","Receiver":"alice","Amount":70},
{"Sender":"joe","Receiver":"bob","Amount":50},
{"Sender":"joe","Receiver":"bob","Amount":150},
{"Sender":"alice","Receiver":"bob","Amount":100},
{"Sender":"bob","Receiver":"kyle","Amount":260}
]
output = []
def find_transaction(sender: str, receiver: str):
for o in output:
if o["Sender"] == sender and o["Receiver"] == receiver:
return o
return None
for tx in transactions:
existing = find_transaction(tx["Sender"], tx["Receiver"])
if existing:
existing["Total"] += tx["Amount"]
existing["Count"] += 1
else:
output.append({
"Sender": tx["Sender"],
"Receiver": tx["Receiver"],
"Total": tx["Amount"],
"Count": 1
})
print(output)
Use a dict to group the amounts, using (sender, receiver) as key;
Rebuild your list of dicts from the dict of amounts.
ll = [{"Sender":"bob","Receiver":"alice","Amount":50},{"Sender":"bob","Receiver":"alice","Amount":60},{"Sender":"bob","Receiver":"alice","Amount":70},{"Sender":"joe","Receiver":"bob","Amount":50},{"Sender":"joe","Receiver":"bob","Amount":150},{"Sender":"alice","Receiver":"bob","Amount":100},{"Sender":"bob","Receiver":"kyle","Amount":260}]
# GROUP BY (SENDER, RECEIVER) AND REDUCE BY KEY
d = {}
for transaction in ll:
k = (transaction['Sender'], transaction['Receiver'])
d[k] = d.get(k, 0) + transaction['Amount']
# print(d)
# d = {('bob', 'alice'): 180, ('joe', 'bob'): 200, ('alice', 'bob'): 100, ('bob', 'kyle'): 260}
# REBUILD LIST OF DICT
new_ll = [{'Sender': s, 'Receiver': r, 'Amount': a} for (s,r),a in d.items()]
print(new_ll)
# [{'Sender': 'bob', 'Receiver': 'alice', 'Amount': 180}, {'Sender': 'joe', 'Receiver': 'bob', 'Amount': 200}, {'Sender': 'alice', 'Receiver': 'bob', 'Amount': 100}, {'Sender': 'bob', 'Receiver': 'kyle', 'Amount': 260}]
These kinds of group-by-key and reduce-by-key operations are extremely common. Using a dict to group by key is the best method. There is also a library function in module more_itertools:
from more_itertools import map_reduce
from operator import itemgetter
ll = ll = [{"Sender":"bob","Receiver":"alice","Amount":50},{"Sender":"bob","Receiver":"alice","Amount":60},{"Sender":"bob","Receiver":"alice","Amount":70},{"Sender":"joe","Receiver":"bob","Amount":50},{"Sender":"joe","Receiver":"bob","Amount":150},{"Sender":"alice","Receiver":"bob","Amount":100},{"Sender":"bob","Receiver":"kyle","Amount":260}]
d = map_reduce(ll, keyfunc=itemgetter('Sender', 'Receiver'), valuefunc=itemgetter('Amount'), reducefunc=sum)
print(d)
# defaultdict(None, {('bob', 'alice'): 180, ('joe', 'bob'): 200, ('alice', 'bob'): 100, ('bob', 'kyle'): 260})
new_ll = [{'Sender': s, 'Receiver': r, 'Amount': a} for (s,r),a in d.items()]
I have a list:
List_ = ["Peter", "Peter", "Susan"]
I want to make a dictonary like this:
Dict_ = {"Name": "Peter", "Count": 2, "Name": "Susan", "Count": 1}
Dict_ = {}
Dict_new = {}
for text in List_:
if text not in Dict_:
Dict_[text] = 1
else:
Dict_[text] += 1
for key, values in Dict_.items():
Dict_new["Name"] = key
Dict_new["Count"] = values
print(Dict_new)
It is printing only last ones:
{"Name": "Susan", "Count": 1}
Here is the implementation that you can use according to what you would like :
from collections import Counter
# Your data
my_list = ["Peter", "Peter", "Susan"]
# Count the occurrences
counted = Counter(my_list)
# Your format
counted_list = []
for key, value in counted.items():
counted_list.append({"Name": key, "Count": value})
print(counted_list)
And output will be :
[{'Name': 'Peter', 'Count': 2}, {'Name': 'Susan', 'Count': 1}]
As noted in comments, a dictionary can only have each key once.
You may want a list of dictionaries, built with help from collections.Counter and a list comprehension.
>>> from collections import Counter
>>> List_ = ["Peter", "Peter", "Susan"]
>>> [{'name': k, 'count': v} for k, v in Counter(List_).items()]
[{'name': 'Peter', 'count': 2}, {'name': 'Susan', 'count': 1}]
In addition to using collections.Counter you could use a defaultdict.
>>> from collections import defaultdict
>>> d = defaultdict(int)
>>> for n in List_:
... d[n] += 1
...
>>> d
defaultdict(<class 'int'>, {'Peter': 2, 'Susan': 1})
>>> [{'name': k, 'count': v} for k, v in d.items()]
[{'name': 'Peter', 'count': 2}, {'name': 'Susan', 'count': 1}]
You can use the following code to achieve what you are trying to do.
List_ = ["Peter", "Peter", "Susan"]
dict_ = {}
for name in List_:
if name in dict_:
dict_[name] += 1
else:
dict_[name] = 1
print(dict_)
Generates the following output where key is the name and value is the count.
{'Peter': 2, 'Susan': 1}
I uploaded a a csv file using DictReader so I essentially have a list of dictionaries. For example I have a called reader with the following:
[{'name': 'Jack', 'hits:' :7, 'misses:': 12, 'year': 10},
{'name': 'Lisa', 'hits': 5, 'misses': 3,' year': 8},
{'name': 'Jack', 'hits': 5, 'misses ':7, 'year': 9}]
I am using a loop to create lists like the following:
name = []
hits = []
for row in reader:
name.append(row["name"])
hits.append(row["hits"])
However I don't want duplicates in my list so where there is a duplicate name I am only interested in the names with the highest year. So basically I want to end up with the following
name = [Jack, Lisa]
hits = [7,5]
What is the best way to go about this
TRY:
reader = sorted(reader, key = lambda i: i['year'], reverse=True)
name = []
hits = []
for row in reader:
if row['name'] in name:
continue
name.append(row["name"])
hits.append(row["hits"])
Idea is to sort the list of dict based on year and then iterate over the list.
import pandas as pd
data = [{'name': 'Jack', 'hits' :7, 'misses': 12, 'year': 10},
{'name': 'Lisa', 'hits': 5, 'misses': 3,'year': 8},
{'name': 'Jack', 'hits': 5, 'misses':7, 'year': 9}]
df = pd.DataFrame(data).sort_values(by=['name','year'],ascending=False).groupby('name').first()
dict(zip(df.index,df['hits']))
In pure Python (no libraries):
people = {} # maps "name" -> "info"
for record in csv_reader:
# do we have someone with that name already?
old_record = people.get(record['name'], {})
# what's their year (defaulting to -1)
old_year = old_record.get('year', -1)
# if this record is more up to date
if record['year'] > old_year:
# replace the old record
people[record['name']] = record
# -- then, you can pull out your name and year lists
name = list(people.keys())
year = list(r['year'] for r in people.values())
If you want to learn Pandas
import pandas as pd
df = pd.read_csv('yourdata.csv')
df.groupby(['name']).max()
Solution without pandas:
lst = [
{"name": "Jack", "hits": 7, "misses:": 12, "year": 10},
{"name": "Lisa", "hits": 5, "misses": 3, " year": 8},
{"name": "Jack", "hits": 5, "misses ": 7, "year": 9},
]
out = {}
for d in lst:
out.setdefault(d["name"], []).append(d)
name = [*out]
hits = [max(i["hits"] for i in v) for v in out.values()]
print(name)
print(hits)
Prints:
['Jack', 'Lisa']
[7, 5]
I have a list of dictionaries in python and I would like to override old value with duplicate value. Please let me know how can I do.
{'message': [{'name': 'raghav', 'id': 10}, {'name': 'raghav', 'id': 11}]}
Output should be:
{'message': [ {'name': 'raghav', 'id': 11}]}
I don't know what you mean by "override old value with duplicate value". If you mean just picking the second dict from the list, you could:
print({k: [v[1]] for (k, v) in data.items()})
If the idea is to update the "name" with a newer value of "id" as you move along the list, then maybe:
def merge_records(data):
records = data['message']
users = {}
for record in records:
name = record['name']
id_ = record['id']
users[name] = id_
new_records = []
for name, id_ in users.items():
new_records.append({'name': name, 'id': id_})
return {'message': new_records}
But, if you have any control over how the data is represented, you might reconsider. You probably want a different data structure.
Here you go:
d = {'message': [{'name': 'raghav', 'id': 10}, {'name': 'raghav', 'id': 11}]}
#loop over outer dictionary
for key, value in d.items():
d[key] = [dict([t for k in value for t in k.items()])]
print(d)
Edit:
As per your requirement:
d = {'message': [ {'name': 'raghav', 'id': 11}, {'name': 'krish', 'id': 20}, {'name': 'anu', 'id': 30}]}
for key, value in d.items():
print [dict((k1,v1)) for k1,v1 in dict([tuple(i.items()) for i in value for val in i.items()]).items()]
I have JSON in this format. How can I print each value and go inside the objects. Also, the json can vary and also the name, so I need a generic solution.
output = {'name':'StackOverflow',
'competitors':[{ 'competitor':'bing',
'link':'bing.com'},
{ 'competitor':'google',
'link':'google.com'}],
'acquisition': {'acquired_day': 16,
'acquired_month': 12,
'acquired_year': 2013,
'acquiring_company': {'name': 'Viggle',
'permalink': 'viggle'}}}
You can use isinstance to check if something is a dict or a list. Something like this may work, but I haven't checked it.
output = {'name': 'StackOverflow',
'competitors': [{'competitor': 'bing',
'link': 'bing.com'},
{'competitor': 'google',
'link': 'google.com'}],
'acquisition': {'acquired_day': 16,
'acquired_month': 12,
'acquired_year': 2013,
'acquiring_company': {'name': 'Viggle',
'permalink': 'viggle'}}}
def traverse(obj):
if isinstance(obj, dict):
for key, value in obj.iteritems():
print('dict_key', key)
traverse(value)
elif isinstance(obj, list):
for value in obj:
traverse(value)
else:
print('value', obj)
traverse(output)