I have a json file with players structured as so
[
{
"Player_Name": "Rory McIlroy",
"Tournament": [
{
"Name": "Arnold Palmer Invitational presented by Mastercard",
"Points": "68.10",
"Salary": "12200.00"
},
{
"Name": "World Golf Championships-Mexico Championship",
"Points": "103.30",
"Salary": "12200.00"
},
{
"Name": "The Genesis Invitational",
"Points": "88.60",
"Salary": "12200.00"
},
{
"Name": "Farmers Insurance Open",
"Points": "107.30",
"Salary": "12200.00"
},
{
"Name": "World Golf Championships-HSBC Champions",
"Points": "138.70",
"Salary": "12400.00"
},
{
"Name": "The ZOZO Championship",
"Points": "103.40",
"Salary": "12300.00"
}
]
}]
When I run this code
import json
import numpy as np
import pandas as pd
from itertools import groupby
# using json open the player objects file and set it equal to data
with open('Active_PGA_Player_Objects.json') as json_file:
data = json.load(json_file)
with open('Players_DK.json') as json_file:
Players_DK = json.load(json_file)
results = []
for k,g in groupby(sorted(data, key=lambda x:x['Player_Name']), lambda x:x['Player_Name']):
results.append({'Player_Name':k, 'Tournament':[i['Tournament'][0] for i in g]})
for obj in results:
for x in Players_DK:
if obj['Player_Name'] == x['Name']:
obj['Average'] = x['AvgPointsPerGame']
i = 0
points_results = []
while i < len(results):
j = 0
while j < len(results[i]['Tournament']):
difference = (int(float(results[i]['Tournament'][j]['Points'])) - (results[i]['Average']))
points_results.append(round(difference,2))
j += 1
i += 1
with open('PGA_Player_Objects_w_Average.json', 'w') as my_file:
json.dump(results, my_file)
my list comes back like this
[{
"Player_Name": "Rory McIlroy",
"Tournament": [
{
"Name": "Arnold Palmer Invitational presented by Mastercard",
"Points": "68.10",
"Salary": "12200.00"
}
],
"Average": 96.19
}]
Can someone explain to me why when I update the specific dictionary it deletes all but the first value from the nested Tournament list? My goal here is to add each players average to their corresponding dictionary so that I can take each average and subtract it from each score. When I try to do this though I'm only able to perform it on the one value left in the list.
Just for what it's worth, I'd go back and really think about what each line is really doing. You're also making things harder on yourself by calling variables obj or x. Calculating the average can be done like:
for player in data: # data is poorly named, try players or players_data
player['Average'] = sum(float(tourny['Points']) for tourny in player['Tournament']) / len(player['Tournament'])
for tourny in player['Tournament']:
tourny['Difference'] = float(tourny['Points']) - float(player['Average'])
leaving you with:
{'Player_Name': 'Rory McIlroy',
'Tournament': [{
'Name': 'Arnold Palmer Invitational presented by Mastercard',
'Points': '68.10',
'Salary': '12200.00',
'Difference': -33.46666666666667},
{
'Name': 'World Golf Championships-Mexico Championship',
'Points': '103.30',
'Salary': '12200.00',
'Difference': 1.7333333333333343}, # .....etc
'Average': 101.566666666666666
}
When you use names in your code that describe what they're representing, a huge number of optimizations become immediately obvious. Give it a go!
I hope everyone is doing well.
I need a little help where I need to get all the strings from a variable and need to store into a single list in python.
For example -
I have json file from where I am getting ids and all the ids are getting stored into a variable called id as below when I run print(id)
17298626-991c-e490-bae6-47079c6e2202
17298496-19bd-2f89-7b5f-881921abc632
17298698-3e17-7a9b-b337-aacfd9483b1b
172986ac-d91d-c4ea-2e50-d53700480dd0
172986d0-18aa-6f51-9c62-6cb087ad31e5
172986f4-80f0-5c21-3aee-12f22a5f4322
17298712-a4ac-7b36-08e9-8512fa8322dd
17298747-8cc6-d9d0-8d05-50adf228c029
1729875c-050f-9a99-4850-bb0e6ad35fb0
1729875f-0d50-dc94-5515-b4891c40d81c
17298761-c26b-3ce5-e77e-db412c38a5b4
172987c8-2b5d-0d94-c365-e8407b0a8860
1729881a-e583-2b54-3a52-d092020d9c1d
1729881c-64a2-67cf-d561-6e5e38ed14cb
172987ec-7a20-7eb6-3ebe-a9fb621bb566
17298813-7ac4-258b-d6f9-aaf43f9147b1
17298813-f1ef-d28a-0817-5f3b86c3cf23
17298828-b62b-9ee6-248b-521b0663226e
17298825-7449-2fcb-378e-13671cb4688a
I want these all values to be stored into a single list.
Can some please help me out with this.
Below is the code I am using:
import json
with open('requests.json') as f:
data = json.load(f)
print(type(data))
for i in data:
if 'traceId' in i:
id = i['traceId']
newid = id.split()
#print(type(newid))
print(newid)
And below is my json file looks like:
[
{
"id": "376287298-hjd8-jfjb-khkf-6479280283e9",
"submittedTime": 1591692502558,
"traceId": "17298626-991c-e490-bae6-47079c6e2202",
"userName": "ABC",
"onlyChanged": true,
"description": "Not Required",
"startTime": 1591694487929,
"result": "NONE",
"state": "EXECUTING",
"paused": false,
"application": {
"id": "16b22a09-a840-f4d9-f42a-64fd73fece57",
"name": "XYZ"
},
"applicationProcess": {
"id": "dihihdosfj9279278yrie8ue",
"name": "Deploy",
"version": 12
},
"environment": {
"id": "fkjdshkjdshglkjdshgldshldsh03r937837",
"name": "DEV"
},
"snapshot": {
"id": "djnglkfdglki98478yhgjh48yr844h",
"name": "DEV_snapshot"
},
},
{
"id": "17298495-f060-3e9d-7097-1f86d5160789",
"submittedTime": 1591692844597,
"traceId": "17298496-19bd-2f89-7b5f-881921abc632",
"userName": "UYT,
"onlyChanged": true,
"startTime": 1591692845543,
"result": "NONE",
"state": "EXECUTING",
"paused": false,
"application": {
"id": "osfodsho883793hgjbv98r3098w",
"name": "QA"
},
"applicationProcess": {
"id": "owjfoew028r2uoieroiehojehfoef",
"name": "EDC",
"version": 5
},
"environment": {
"id": "16cf69c5-4194-e557-707d-0663afdbceba",
"name": "DTESTU"
},
}
]
From where I am trying to get the traceId.
you could use simple split method like the follwing:
ids = '''17298626-991c-e490-bae6-47079c6e2202 17298496-19bd-2f89-7b5f-881921abc632 17298698-3e17-7a9b-b337-aacfd9483b1b 172986ac-d91d-c4ea-2e50-d53700480dd0 172986d0-18aa-6f51-9c62-6cb087ad31e5 172986f4-80f0-5c21-3aee-12f22a5f4322 17298712-a4ac-7b36-08e9-8512fa8322dd 17298747-8cc6-d9d0-8d05-50adf228c029 1729875c-050f-9a99-4850-bb0e6ad35fb0 1729875f-0d50-dc94-5515-b4891c40d81c 17298761-c26b-3ce5-e77e-db412c38a5b4 172987c8-2b5d-0d94-c365-e8407b0a8860 1729881a-e583-2b54-3a52-d092020d9c1d 1729881c-64a2-67cf-d561-6e5e38ed14cb 172987ec-7a20-7eb6-3ebe-a9fb621bb566 17298813-7ac4-258b-d6f9-aaf43f9147b1 17298813-f1ef-d28a-0817-5f3b86c3cf23 17298828-b62b-9ee6-248b-521b0663226e 17298825-7449-2fcb-378e-13671cb4688a'''
l = ids.split(" ")
print(l)
This will give the following result, I assumed that the separator needed is simple space you can adjust properly:
['17298626-991c-e490-bae6-47079c6e2202', '17298496-19bd-2f89-7b5f-881921abc632', '17298698-3e17-7a9b-b337-aacfd9483b1b', '172986ac-d91d-c4ea-2e50-d53700480dd0', '172986d0-18aa-6f51-9c62-6cb087ad31e5', '172986f4-80f0-5c21-3aee-12f22a5f4322', '17298712-a4ac-7b36-08e9-8512fa8322dd', '17298747-8cc6-d9d0-8d05-50adf228c029', '1729875c-050f-9a99-4850-bb0e6ad35fb0', '1729875f-0d50-dc94-5515-b4891c40d81c', '17298761-c26b-3ce5-e77e-db412c38a5b4', '172987c8-2b5d-0d94-c365-e8407b0a8860', '1729881a-e583-2b54-3a52-d092020d9c1d', '1729881c-64a2-67cf-d561-6e5e38ed14cb', '172987ec-7a20-7eb6-3ebe-a9fb621bb566', '17298813-7ac4-258b-d6f9-aaf43f9147b1', '17298813-f1ef-d28a-0817-5f3b86c3cf23', '17298828-b62b-9ee6-248b-521b0663226e', '17298825-7449-2fcb-378e-13671cb4688a']
Edit
You get list of lists because each iteration you read only 1 id, so what you need to do is to initiate an empty list and append each id to it in the following way:
l = []
for i in data
if 'traceId' in i:
id = i['traceId']
l.append(id)
you can append the ids variable to the list such as,
#list declaration
l1=[]
#this must be in your loop
l1.append(ids)
I'm assuming you get the id as a str type value. Using id.split() will return a list of all ids in one single Python list, as each id is separated by space here in your example.
id = """17298626-991c-e490-bae6-47079c6e2202 17298496-19bd-2f89-7b5f-881921abc632
17298698-3e17-7a9b-b337-aacfd9483b1b 172986ac-d91d-c4ea-2e50-d53700480dd0
172986d0-18aa-6f51-9c62-6cb087ad31e5 172986f4-80f0-5c21-3aee-12f22a5f4322
17298712-a4ac-7b36-08e9-8512fa8322dd 17298747-8cc6-d9d0-8d05-50adf228c029
1729875c-050f-9a99-4850-bb0e6ad35fb0 1729875f-0d50-dc94-5515-b4891c40d81c
17298761-c26b-3ce5-e77e-db412c38a5b4 172987c8-2b5d-0d94-c365-e8407b0a8860
1729881a-e583-2b54-3a52-d092020d9c1d 1729881c-64a2-67cf-d561-6e5e38ed14cb
172987ec-7a20-7eb6-3ebe-a9fb621bb566 17298813-7ac4-258b-d6f9-aaf43f9147b1
17298813-f1ef-d28a-0817-5f3b86c3cf23 17298828-b62b-9ee6-248b-521b0663226e
17298825-7449-2fcb-378e-13671cb4688a"""
id_list = id.split()
print(id_list)
Output:
['17298626-991c-e490-bae6-47079c6e2202', '17298496-19bd-2f89-7b5f-881921abc632',
'17298698-3e17-7a9b-b337-aacfd9483b1b', '172986ac-d91d-c4ea-2e50-d53700480dd0',
'172986d0-18aa-6f51-9c62-6cb087ad31e5', '172986f4-80f0-5c21-3aee-12f22a5f4322',
'17298712-a4ac-7b36-08e9-8512fa8322dd', '17298747-8cc6-d9d0-8d05-50adf228c029',
'1729875c-050f-9a99-4850-bb0e6ad35fb0', '1729875f-0d50-dc94-5515-b4891c40d81c',
'17298761-c26b-3ce5-e77e-db412c38a5b4', '172987c8-2b5d-0d94-c365-e8407b0a8860',
'1729881a-e583-2b54-3a52-d092020d9c1d', '1729881c-64a2-67cf-d561-6e5e38ed14cb',
'172987ec-7a20-7eb6-3ebe-a9fb621bb566', '17298813-7ac4-258b-d6f9-aaf43f9147b1',
'17298813-f1ef-d28a-0817-5f3b86c3cf23', '17298828-b62b-9ee6-248b-521b0663226e',
'17298825-7449-2fcb-378e-13671cb4688a']
split() splits by default with space as a separator. You can use the sep argument to use any other separator if needed.
i need to filter out data from a list that i get:
[{"end": 1547230999000, "attributes": {}, "metric": "system.cpu.idle", "interval": 20, "start": 1547227400000, "length": 180, "query_index": 0, "aggr": null, "scope": "host:osboxes", "pointlist": [[1547227400000.0, 99.6485366821289], [1547227420000.0, 99.60060119628906], [1547227440000.0, 99.40513610839844], "expression": "system.cpu.idle{host:osboxes}", "unit": [{"family": "percentage", "scale_factor": 1.0, "name": "percent", "short_name": "%", "plural": "percent", "id": 17}, null], "display_name": "system.cpu.idle"}], "to_date": 1547231000000, "resp_version": 1, "query": "system.cpu.idle{*}by{host}", "message": "", "group_by": ["host"]}
i only want the data that comes after the pointlist key and before the expression key, thought about regex for this problem, i'm not sure about it though.
2) from the tuples i'd get after the 1st filter say for example:
[1547227420000.0, 99.60060119628906] i need just the 2nd value in each one in some kind of structure.
Tried using regex but i cant seem to find the correct rule to grab just the stuff i want.
Here's the whole json ( dictionary) that i get as an input:
[{"end": 1547230999000, "attributes": {}, "metric": "system.cpu.idle", "interval": 20, "start": 1547227400000, "length": 180, "query_index": 0, "aggr": null, "scope": "host:osboxes", "pointlist": [[1547227400000.0, 99.6485366821289], [1547227420000.0, 99.60060119628906], [1547227440000.0, 99.40513610839844], [1547227460000.0, 99.5660171508789], [1547227480000.0, 99.68238067626953], [1547227500000.0, 99.58213806152344], [1547227520000.0, 99.56404876708984], [1547227540000.0, 99.59886169433594], [1547227560000.0, 99.33905792236328], [1547227580000.0, 99.49874877929688], [1547227600000.0, 99.69874572753906], [1547227620000.0, 99.58246231079102], [1547227640000.0, 99.01371002197266], [1547227660000.0, 99.53114318847656], [1547227680000.0, 99.48202896118164], [1547227700000.0, 99.49647521972656], [1547227720000.0, 99.68254089355469], [1547227740000.0, 99.43094635009766], [1547227760000.0, 99.38209533691406], [1547227780000.0, 99.6488265991211], [1547227800000.0, 99.42307662963867], [1547227820000.0, 99.28117370605469], [1547227840000.0, 99.51512908935547], [1547227860000.0, 96.35371780395508], [1547227880000.0, 99.0471420288086], [1547227900000.0, 99.59866333007812], [1547227920000.0, 99.41494750976562], [1547227940000.0, 99.4984130859375], [1547227960000.0, 99.5489501953125], [1547227980000.0, 99.48962783813477], [1547228000000.0, 99.58173370361328], [1547228020000.0, 99.63229370117188], [1547228040000.0, 99.38098907470703], [1547228060000.0, 99.46452331542969], [1547228080000.0, 99.5501480102539], [1547228100000.0, 99.46395111083984], [1547228120000.0, 99.6651611328125], [1547228140000.0, 99.66544342041016], [1547228160000.0, 99.5067024230957], [1547228180000.0, 99.53192901611328], [1547228200000.0, 99.58263397216797], [1547228220000.0, 99.4233169555664], [1547228240000.0, 99.51488494873047], [1547228260000.0, 99.69884490966797], [1547228280000.0, 99.17123413085938], [1547228300000.0, 99.48178100585938], [1547228320000.0, 99.61544799804688], [1547228340000.0, 99.38138961791992], [1547228360000.0, 99.49983215332031], [1547228380000.0, 99.58074951171875], [1547228400000.0, 99.44026565551758], [1547228420000.0, 99.56558227539062], [1547228440000.0, 99.61634826660156], [1547228460000.0, 99.2971076965332], [1547228480000.0, 99.514404296875], [1547228500000.0, 99.56529235839844], [1547228520000.0, 99.48181915283203], [1547228540000.0, 99.49799346923828], [1547228560000.0, 99.56507110595703], [1547228580000.0, 99.47320556640625], [1547228600000.0, 99.49816131591797], [1547228620000.0, 99.59886169433594], [1547228640000.0, 99.0047836303711], [1547228660000.0, 99.48117065429688], [1547228680000.0, 99.66544342041016], [1547228700000.0, 99.49843215942383], [1547228720000.0, 99.48194885253906], [1547228740000.0, 99.63235473632812], [1547228760000.0, 99.36409378051758], [1547228780000.0, 93.1688461303711], [1547228800000.0, 99.34782409667969], [1547228820000.0, 99.46506118774414], [1547228840000.0, 99.33065795898438], [1547228860000.0, 99.59893035888672], [1547228880000.0, 99.47415924072266], [1547228900000.0, 99.46299743652344], [1547228920000.0, 99.5824966430664], [1547228940000.0, 99.39748764038086], [1547228960000.0, 99.46452331542969], [1547228980000.0, 99.71566772460938], [1547229000000.0, 99.4896354675293], [1547229020000.0, 99.481689453125], [1547229040000.0, 99.48186492919922], [1547229060000.0, 99.43965148925781], [1547229080000.0, 99.41500854492188], [1547229100000.0, 99.56536102294922], [1547229120000.0, 99.45612716674805], [1547229140000.0, 99.28033447265625], [1547229160000.0, 98.72547149658203], [1547229180000.0, 99.36448669433594], [1547229200000.0, 99.39749145507812], [1547229220000.0, 99.55000305175781], [1547229240000.0, 99.32996368408203], [1547229260000.0, 99.43115234375], [1547229280000.0, 99.41422271728516], [1547229300000.0, 99.41427993774414], [1547229320000.0, 99.4978256225586], [1547229340000.0, 99.63327026367188], [1547229360000.0, 99.45573425292969], [1547229380000.0, 99.04618835449219], [1547229400000.0, 99.56463623046875], [1547229420000.0, 99.42306137084961], [1547229440000.0, 99.36380004882812], [1547229460000.0, 99.6164779663086], [1547229480000.0, 99.48064422607422], [1547229500000.0, 99.44741821289062], [1547229520000.0, 99.5820083618164], [1547229540000.0, 99.23918914794922], [1547229560000.0, 99.38034057617188], [1547229580000.0, 99.58187103271484], [1547229600000.0, 99.47303771972656], [1547229620000.0, 99.44770050048828], [1547229640000.0, 99.56521606445312], [1547229660000.0, 99.36420822143555], [1547229680000.0, 93.31424713134766], [1547229700000.0, 99.19745635986328], [1547229720000.0, 99.3642807006836], [1547229740000.0, 99.3148422241211], [1547229760000.0, 99.41403198242188], [1547229780000.0, 98.98696899414062], [1547229800000.0, 99.36422729492188], [1547229820000.0, 99.59711456298828], [1547229840000.0, 99.41479110717773], [1547229860000.0, 99.4476089477539], [1547229880000.0, 99.59845733642578], [1547229900000.0, 99.42321014404297], [1547229920000.0, 99.46488189697266], [1547229940000.0, 99.59845733642578], [1547229960000.0, 99.51408767700195], [1547229980000.0, 99.53137969970703], [1547230000000.0, 99.59893035888672], "expression": "system.cpu.idle{host:osboxes}", "unit": [{"family": "percentage", "scale_factor": 1.0, "name": "percent", "short_name": "%", "plural": "percent", "id": 17}, null], "display_name": "system.cpu.idle"}], "to_date": 1547231000000, "resp_version": 1, "query": "system.cpu.idle{*}by{host}", "message": "", "group_by": ["host"]}
So the end result should achieve something of that nature:
structure = [99.6485366821289,99.60060119628906,99.40513610839844...and so on]`
I'm assuming you're dealing with regular Python objects, so regexes aren't the way to go.
If you are reading some json in, you can do this beforehand:
import json
with open("source_file.json", "r") as fh:
data = json.load(fh)
Collecting the data you need is then:
new_list = []
for obj in data:
new_list.extend(second for _, second in obj['pointlist'])
I'm guessing here that in case you have multiple "pointlist" instances you'd want to gather all of them. If you know there will be only one, this would work just as well:
new_list = [second for _, second in data[0]['pointlist']]
This is known as a list comprehension and is a quick way to process lists.
The _, second is called destructuring. Here _ is a dummy name, you could write first, second just as well, and the technique works whenever you have a list or tuple with a known number of items.
I have the next JSON that I get from a URL:
[{
"id": 1,
"version": 23,
"external_id": "2312",
"url": "https://example.com/432",
"type": "typeA",
"date": "2",
"notes": "notes",
"title": "title",
"abstract": "dsadasdas",
"details": "something",
"accuracy": 0,
"reliability": 0,
"severity": 12,
"thing": "32132",
"other": [
"aaaaaaaaaaaaaaaaaa",
"bbbbbbbbbbbbbb",
"cccccccccccccccc",
"dddddddddddddd",
"eeeeeeeeee"
],
"nana": 8
},
{
"id": 2,
"version": 23,
"external_id": "2312",
"url": "https://example.com/432",
"type": "typeA",
"date": "2",
"notes": "notes",
"title": "title",
"abstract": "dsadasdas",
"details": "something",
"accuracy": 0,
"reliability": 0,
"severity": 12,
"thing": "32132",
"other": [
"aaaaaaaaaaaaaaaaaa",
"bbbbbbbbbbbbbb",
"cccccccccccccccc",
"dddddddddddddd",
"eeeeeeeeee"
],
"nana": 8
}]
My code:
import json
import urllib2
data = json.load(urllib2.urlopen('http://someurl/path/to/json'))
print data
I want to know how to access to the part "abstract" of the object that has "id" equal to 2 for example. The part "id" is unique so I can use id to index my searchs.
Thanks!
Here's one way to do it. You can create a generator via a generator expression, call next to iterate that generator once, and get back the desired object.
item = next((item for item in data if item['id'] == 2), None)
if item:
print item['abstract']
See also Python: get a dict from a list based on something inside the dict
EDIT : If you'd like access to all elements of the list that have a given key value (for example, id == 2) you can do one of two things. You can either create a list via comprehension (as shown in the other answer), or you can alter my solution:
my_gen = (item for item in data if item['id'] == 2)
for item in my_gen:
print item
In the loop, item will iterate over those items in your list which satisfy the given condition (here, id == 2).
You can use list comprehention to filter:
import json
j = """[{"id":1,"version":23,"external_id":"2312","url":"https://example.com/432","type":"typeA","date":"2","notes":"notes","title":"title","abstract":"dsadasdas","details":"something","accuracy":0,"reliability":0,"severity":12,"thing":"32132","other":["aaaaaaaaaaaaaaaaaa","bbbbbbbbbbbbbb","cccccccccccccccc","dddddddddddddd","eeeeeeeeee"],"nana":8},{"id":2,"version":23,"external_id":"2312","url":"https://example.com/432","type":"typeA","date":"2","notes":"notes","title":"title","abstract":"dsadasdas","details":"something","accuracy":0,"reliability":0,"severity":12,"thing":"32132","other":["aaaaaaaaaaaaaaaaaa","bbbbbbbbbbbbbb","cccccccccccccccc","dddddddddddddd","eeeeeeeeee"],"nana":8}]"""
dicto = json.loads(j)
results = [x for x in dicto if "id" in x and x["id"]==2]
And then you can print the 'abstract' values like so:
for result in results:
if "abstract" in result:
print result["abstract"]
import urllib2
import json
data = json.load(urllib2.urlopen('http://someurl/path/to/json'))
your_id = raw_input('enter the id')
for each in data:
if each['id'] == your_id:
print each['abstract']
In the above code data is list and each is a dict you can easily access the dict object.