I have query set which returns a json string after converted list.
[{"pid": 1, "loc": "KL", "sid": 1, "sd": "south-1"},
{"pid": 1, "loc": "KL", "sid": 2, "sd": "north-5"},
{"pid": 1, "loc": "KL", "sid": 3, "sd": "west-3"}
]
I have tried many serializer options but no idea how to make the above as:
[{"pid": 1,
"s": [{"sid": 1, "sd": "south-1",
"sid": 2, "sd": "north-5",
"sid": 3, "sd": "west-3"
}]
}]
Firstly, there's an error in your expected output. You probably meant:
[{"pid": 1,
"s": [{"sid": 1, "sd": "south-1"},
{"sid": 2, "sd": "north-5"},
{"sid": 3, "sd": "west-3"}
],
"loc": "KL"
}]
ie s should be a list of dictionaries and not one dict (and clashing keys). I've added "loc": "KL" since that looks like it's missing.
Assuming each query returns only the same pid and loc, you can create s as a list with each sid and sd in the original query:
>>> q = ... # as above
>>> r = {"pid": q[0]["pid"], "loc": q[0]["loc"]} # since pid and loc are always the same
>>> r["s"] = [{"sid": x["sid"], "sd": x["sd"]} for x in q]
>>> print r
[{'pid': 1,
's': [{'sid': 1, 'sd': 'south-1'},
{'sid': 2, 'sd': 'north-5'},
{'sid': 3, 'sd': 'west-3'}
],
'loc': 'KL'
}]
>>> print json.dumps(r) # gives the output as a json string
Related
I have below JSON file from which I want to extract only
("workers": {"usersRunning": 1, "usersWaiting": 0, "total": 8, "jobsWaiting": 0, "inUse": 4})
part, and then put it into csv file or text file (tab deliminated). I am new to python so any help will be apprciated..
{
"workers": {
"usersRunning": 1,
"usersWaiting": 0,
"total": 8,
"jobsWaiting": 0,
"inUse": 4
},
"users": {
"activeUsers": 1,
"activity": [{
"maxWorkers": 4,
"inProgress": 4,
"displayName": "abc",
"waiting": 0
}]
}
}
I recommend using pandas. It has methods to read the json and you can use dataframe filtering to find the data you need.
Examples here: https://www.listendata.com/2019/07/how-to-filter-pandas-dataframe.html
I would recommend you use pandas and convert to excel
The following is sample that will help you to get your answer
json_ = {"workers": {"usersRunning": 1, "usersWaiting": 0, "total": 8, "jobsWaiting": 0, "inUse": 4}, "users": {"activeUsers": 1, "activity": [{"maxWorkers": 4, "inProgress": 4, "displayName": "abc", "waiting": 0}]}}
import pandas as pd
df = pd.DataFrame(data= json_['workers'], index=[0])
df.to_excel('json_.xlsx')
I am trying to create lists from json datas by pulling one by one and append them to the lists. However, some variables does not given in all json files. For example: for the json file below, data does not have ['statistics']['aerialLost'] , so it return Key Error. My Expected solution is when json file does not have key, append 'None' value to the list and continue.
Code
s_aerialLost = []
s_aerialWon = []
s_duelLost = []
s_duelWon = []
players = ['Martin Linnes', 'Christian Luyindama', 'Marcão', 'Ömer Bayram', 'Oghenekaro Etebo', 'Muhammed Kerem Aktürkoğlu', 'Gedson Fernandes', 'Emre Kılınç', 'Ryan Babel', 'Mostafa Mohamed', 'Florent Hadergjonaj', 'Tomáš Břečka', 'Duško Tošić', 'Oussama Haddadi', 'Kristijan Bistrović', 'Aytaç Kara', 'Haris Hajradinović', 'Armin Hodžić', 'Gilbert Koomson', 'Isaac Kiese Thelin']
players_id = [109569, 867191, 840951, 68335, 839110, 903324, 862055, 202032, 1876, 873551, 354860, 152971, 14557, 867180, 796658, 128196, 254979, 138127, 341107, 178743]
for player, player_id in zip(players, players_id):
url = base_url + str(player_id)
data = requests.request("GET", url).json()
## just added 4 data for simplify
accurateLongBalls = str(data['statistics']['accurateLongBalls'])
aerialLost = str(data['statistics']['aerialLost'])
aerialWon = str(data['statistics']['aerialWon'])
duelLost = str(data['statistics']['duelLost'])
s_aerialLost.append()
s_aerialWon.append()
s_duelLost.append()
s_duelWon.append()
Json File
{
"player": {
"name": "Martin Linnes",
"slug": "martin-linnes",
"shortName": "M. Linnes",
"position": "D",
"userCount": 339,
"id": 109569,
"marketValueCurrency": "€",
"dateOfBirthTimestamp": 685324800
},
"team": {
"name": "Galatasaray",
"slug": "galatasaray",
"shortName": "Galatasaray",
"gender": "M",
"userCount": 100254,
"nameCode": "GAL",
"national": false,
"type": 0,
"id": 3061,
"teamColors": {
"primary": "#ff9900",
"secondary": "#ff0000",
"text": "#ff0000"
}
},
"statistics": {
"totalPass": 32,
"accuratePass": 22,
"totalLongBalls": 7,
"accurateLongBalls": 3,
"totalCross": 2,
"aerialWon": 1,
"duelLost": 2,
"duelWon": 7,
"totalContest": 3,
"wonContest": 2,
"totalClearance": 4,
"totalTackle": 3,
"wasFouled": 1,
"fouls": 1,
"minutesPlayed": 82,
"touches": 63,
"rating": 7.3,
"possessionLostCtrl": 18,
"keyPass": 1
},
"position": "D"
}
Error
KeyError: 'aerialLost'
Use .get(). You can specify a default value to return if the key is not found, and it defaults to None.
So you can use
aerialLost = str(data.get('statistics', {}).get('aerialLost'))
The first call defaults to an empty dictionary so that there's something to make the second .get() call on. The second call just returns the default None.
I want to get all parent keys for all items in a nested python dictionary with unlimited levels. Take an analogy, if you think of a nested dictionary as a directory containing sub-directories, the behaviour I want is similar to what glob.glob(dir, recursive=True) does.
For example, suppose we have the following dictionary:
sample_dict = {
"key_1": {
"sub_key_1": 1,
"sub_key_2": 2,
},
"key_2": {
"sub_key_1": 3,
"sub_key_2": {
"sub_sub_key_1": 4,
},
},
}
I want to get the full "path" of every value in the dictionary:
["key_1", "sub_key_1", 1]
["key_1", "sub_key_2", 2]
["key_2", "sub_key_1", 3]
["key_2", "sub_key_2", "sub_sub_key_1", 4]
Just wondering if there is a clean way to do that?
Using generators can often simplify the code for these type of tasks and make them much more readable while avoiding passing explicit state arguments to the function. You get a generator instead of a list, but this is a good thing because you can evaluate lazily if you want to. For example:
def getpaths(d):
if not isinstance(d, dict):
yield [d]
else:
yield from ([k] + w for k, v in d.items() for w in getpaths(v))
result = list(getpaths(sample_dict))
Result will be:
[['key_1', 'sub_key_1', 1],
['key_1', 'sub_key_2', 2],
['key_2', 'sub_key_1', 3],
['key_2', 'sub_key_2', 'sub_sub_key_1', 4]]
You can solve it recursively
sample_dict = {
"key_1": {
"sub_key_1": 1,
"sub_key_2": 2,
},
"key_2": {
"sub_key_1": 3,
"sub_key_2": {
"sub_sub_key_1": 4,
},
}
}
def full_paths(sample_dict, paths=[], parent_keys=[]):
for key in sample_dict.keys():
if type(sample_dict[key]) is dict:
full_paths(sample_dict[key], paths=paths, parent_keys=(parent_keys + [key]))
else:
paths.append(parent_keys + [key] + [sample_dict[key]])
return paths
print(full_paths(sample_dict))
You can use this solution.
sample_dict = {
"key_1": {
"sub_key_1": 1,
"sub_key_2": 2,
},
"key_2": {
"sub_key_1": 3,
"sub_key_2": {
"sub_sub_key_1": 4,
},
},
}
def key_find(sample_dict, li=[]):
for key, val in sample_dict.items():
if isinstance(val, dict):
key_find(val, li=li + [key])
else:
print(li + [key] + [val])
key_find(sample_dict)
I have the following Python code:
myArray = [{ "id": 1, "desc": "foo", "specs": { "width": 1, "height": 1}}, { "id": 2, "desc": "bar", "specs": { "width": 2, "height": 2, "color": "black"}}, { "id": 3, "desc": "foobar"}]
print len(myArray)
myArray_filtered = filter(lambda item : hasattr(item, "specs") and hasattr(item.specs, "color"), myArray)
print len(myArray_filtered)
I expect to get length 1 on second print, but it is 0. Can you tell me what's wrong with my code?
Given your nested structure, you could use dict.get with some default values:
>>> myArray_filtered = list(filter(lambda d: d.get("specs", {}).get("color") is not None, myArray))
>>> len(myArray_filtered)
1
>>> myArray_filtered
[{'id': 2, 'desc': 'bar', 'specs': {'width': 2, 'height': 2, 'color': 'black'}}]
myArray_filtered = [v for v in myArray if v.get('specs', {}).get('color')]
print(len(myArray_filtered))
Slightly simpler just using list comprehensions.
And you can add to the condition:
myArray_filtered = [v for v in myArray if v.get('specs', {}).get('color') and v.get('specs', {}).get('width') == 2]
print(len(myArray_filtered))
I have a JSON decoder set up with a specific decoder function:
data.tankDecoder = JSONDecoder(object_hook=tankFromJSON)
tankFromJSON:
def tankFromJSON(obj):
print("object", obj)
humans = [HumanFish(human["name"], human["position"][0],
human["position"][1], human["size"])
for human in obj["humans"]]
bots = [RobotFish(bot["position"][0], bot["position"][1], bot["size"])
for bot in obj["bots"]]
tank = Tank(obj["canvasSize"], humans)
tank.grass = obj["grass"]
tank.bots = bots
print(tank)
return tank
The JSON I get looks something like this:
{
"canvasSize": 600,
"newBotOffset": 50,
"grass": [
[583, 588],
...,
[409, 575],
[496, 574]
],
"bots": [],
"humans": [{
"name": ["127.0.0.1", 50014],
"acceleration": [0, 0],
"maxSpeed": 3,
"speed": [0, 0],
"accelerationRate": 1,
"position": [300.0, 300.0],
"foodHistory": [],
"efficiency": 0.4,
"size": 20,
"color": "green"
}],
"maxBots": 20
}
For some reason the value of obj in the tankFromJSON function becomes the first dictionary in the humans list instead of the whole JSON itself.
Can anyone explain?
During decoding the object hook is called once for every JSON object that is encountered. What you are seeing is that the hook is called on inner objects first, and then on the outer objects next as the deserialization unwinds.
In other words, the whole JSON object is processed, but any objects it contains are processed first in an inside-out fashion, as this example demonstrates:
s = '''
{
"outer": {
"middle": {
"inner": [1, 2, 3]
}
}
}
'''
def hook(obj):
print(obj)
return obj
decoder = JSONDecoder(object_hook=hook)
decoder.decode(s)
The above prints:
{'inner': [1, 2, 3]}
{'middle': {'inner': [1, 2, 3]}}
{'outer': {'middle': {'inner': [1, 2, 3]}}}