We Keep Coding

Update json data with context in Python using jsonpath-ng

following Update json nodes in Python using jsonpath, would like to know how one might update the JSON data given a certain context.
So, say we pick the exact same JSON example:
{
"SchemeId": 10,
"nominations": [
{
"nominationId": 1
}
]
}
But this time, would like to double the value of the original value, hence some lambda function is needed which takes into account the current node value.

No need for lambdas; for example, to double SchemeId, something like this should work:
data = json.loads("""the json string above""")
jsonpath_expr = parse('$.SchemeId')
jsonpath_expr.find(data)
val = jsonpath_expr.find(data)[0].value
jsonpath_expr.update(data, val*2)
print(json.dumps(data, indent=2))
Output:
{
"SchemeId": 20,
"nominations": [
{
"nominationId": 1
}
]
}

Here is example with lambda expression:
import json
from jsonpath_ng import parse
settings = '''{
"choices": {
"atm": {
"cs": "Strom",
"en": "Tree"
},
"bar": {
"cs": "Dům",
"en": "House"
},
"sea": {
"cs": "Moře",
"en": "Sea"
}
}
}'''
json_data = json.loads(settings)
pattern = parse('$.choices.*')
def magic(f: dict, to_lang='cs'):
return f[to_lang]
pattern.update(json_data,
lambda data_field, data, field: data.update({field: magic(data[field])}))
json_data
returns
{
'choices': {
'atm': 'Strom',
'bar': 'Dům',
'sea': 'Moře'
}
}

Why code not working correctly for printing having { in python

I have a dictionary below:
event = {
"body-json": {},
"params": {
"path": {
"matchphrase": "term"
},
"querystring": {
"dataproduct.keyword": "health"
},
"header": {
"Accept": "application/json"
}
},
"resource-path": "/{matchphrase}"
}
I would like to access the above event dictionary keys & values and frame a new dictionary as follows:
{"query": {"term" : {"dataproduct.keyword": "health"}}}
Here is the code what I tried:
a = event['params']['path']['matchphrase'] #term
b = list(event['params']['querystring'].keys())[0] #dataproduct.keyword
c = list(event['params']['querystring'].values())[0] #health
body=f"{query: {{a} : {{b}: {c}}}}"
print(body)
Am I missing something ?

This should work :
body = {"query":{str(a):{str(b):str(c)}}}
print(body)

The escaping is wrong.
Try this instead:
body = f'{{"query": {{{a!r}: {{{b!r}: {c!r}}}}}}}'
I've also added !r which will return the real representation (repr) of the object (so you don't need to artificially add quotes).

you can create a dictionary and then get a string version of it using json.dumps.
import json
event = {
"body-json": {},
"params": {
"path": {"matchphrase": "term"},
"querystring": {"dataproduct.keyword": "health"},
"header": {"Accept": "application/json"},
},
"resource-path": {"matchphrase}"},
}
a = event["params"]["path"]["matchphrase"] # term
b = list(event["params"]["querystring"].keys())[0] # dataproduct.keyword
c = list(event["params"]["querystring"].values())[0] # health
result = {"query": {a: {b: c}}}
print(json.dumps(result))
Output:
{"query": {"term": {"dataproduct.keyword": "health"}}}

Dictionary length is equal to 3 but when trying to access an index receiving KeyError

I am attempting to parse a json response that looks like this:
{
"links": {
"next": "http://www.neowsapp.com/rest/v1/feed?start_date=2015-09-08&end_date=2015-09-09&detailed=false&api_key=xxx",
"prev": "http://www.neowsapp.com/rest/v1/feed?start_date=2015-09-06&end_date=2015-09-07&detailed=false&api_key=xxx",
"self": "http://www.neowsapp.com/rest/v1/feed?start_date=2015-09-07&end_date=2015-09-08&detailed=false&api_key=xxx"
},
"element_count": 22,
"near_earth_objects": {
"2015-09-08": [
{
"links": {
"self": "http://www.neowsapp.com/rest/v1/neo/3726710?api_key=xxx"
},
"id": "3726710",
"neo_reference_id": "3726710",
"name": "(2015 RC)",
"nasa_jpl_url": "http://ssd.jpl.nasa.gov/sbdb.cgi?sstr=3726710",
"absolute_magnitude_h": 24.3,
"estimated_diameter": {
"kilometers": {
"estimated_diameter_min": 0.0366906138,
"estimated_diameter_max": 0.0820427065
},
"meters": {
"estimated_diameter_min": 36.6906137531,
"estimated_diameter_max": 82.0427064882
},
"miles": {
"estimated_diameter_min": 0.0227984834,
"estimated_diameter_max": 0.0509789586
},
"feet": {
"estimated_diameter_min": 120.3760332259,
"estimated_diameter_max": 269.1689931548
}
},
"is_potentially_hazardous_asteroid": false,
"close_approach_data": [
{
"close_approach_date": "2015-09-08",
"close_approach_date_full": "2015-Sep-08 09:45",
"epoch_date_close_approach": 1441705500000,
"relative_velocity": {
"kilometers_per_second": "19.4850295284",
"kilometers_per_hour": "70146.106302123",
"miles_per_hour": "43586.0625520053"
},
"miss_distance": {
"astronomical": "0.0269230459",
"lunar": "10.4730648551",
"kilometers": "4027630.320552233",
"miles": "2502653.4316094954"
},
"orbiting_body": "Earth"
}
],
"is_sentry_object": false
},
}
I am trying to figure out how to parse through to get "miss_distance" dictionary values ? I am unable to wrap my head around it.
Here is what I have been able to do so far:
After I get a Response object from request.get()
response = request.get(url
I convert the response object to json object
data = response.json() #this returns dictionary object
I try to parse the first level of the dictionary:
for i in data:
if i == "near_earth_objects":
dataset1 = data["near_earth_objects"]["2015-09-08"]
#this returns the next object which is of type list
Please someone can explain me :
1. How to decipher this response in the first place.
2. How can I move forward in parsing the response object and get to miss_distance dictionary ?
Please any pointers/help is appreciated.
Thank you

Your data will will have multiple dictionaries for the each date, near earth object, and close approach:
near_earth_objects = data['near_earth_objects']
for date in near_earth_objects:
objects = near_earth_objects[date]
for object in objects:
close_approach_data = object['close_approach_data']
for close_approach in close_approach_data:
print(close_approach['miss_distance'])

The code below gives you a table of date, miss_distances for every object for every date
import json
raw_json = '''
{
"near_earth_objects": {
"2015-09-08": [
{
"close_approach_data": [
{
"miss_distance": {
"astronomical": "0.0269230459",
"lunar": "10.4730648551",
"kilometers": "4027630.320552233",
"miles": "2502653.4316094954"
},
"orbiting_body": "Earth"
}
]
}
]
}
}
'''
if __name__ == "__main__":
parsed = json.loads(raw_json)
# assuming this json includes more than one near_earch_object spread across dates
near_objects = []
for date, near_objs in parsed['near_earth_objects'].items():
for obj in near_objs:
for appr in obj['close_approach_data']:
o = {
'date': date,
'miss_distances': appr['miss_distance']
}
near_objects.append(o)
print(near_objects)
output:
[
{'date': '2015-09-08',
'miss_distances': {
'astronomical': '0.0269230459',
'lunar': '10.4730648551',
'kilometers': '4027630.320552233',
'miles': '2502653.4316094954'
}
}
]

Grab element from json dump

I'm using the following python code to connect to a jsonrpc server and nick some song information. However, I can't work out how to get the current title in to a variable to print elsewhere. Here is the code:
TracksInfo = []
for song in playingSongs:
data = { "id":1,
"method":"slim.request",
"params":[ "",
["songinfo",0,100, "track_id:%s" % song, "tags:GPASIediqtymkovrfijnCYXRTIuwxN"]
]
}
params = json.dumps(data, sort_keys=True, indent=4)
conn.request("POST", "/jsonrpc.js", params)
httpResponse = conn.getresponse()
data = httpResponse.read()
responce = json.loads(data)
print json.dumps(responce, sort_keys=True, indent=4)
TrackInfo = responce['result']["songinfo_loop"][0]
TracksInfo.append(TrackInfo)
This brings me back the data in json format and the print json.dump brings back:
pi#raspberrypi ~/pithon $ sudo python tom3.py
{
"id": 1,
"method": "slim.request",
"params": [
"",
[
"songinfo",
"0",
100,
"track_id:-140501481178464",
"tags:GPASIediqtymkovrfijnCYXRTIuwxN"
]
],
"result": {
"songinfo_loop": [
{
"id": "-140501481178464"
},
{
"title": "Witchcraft"
},
{
"artist": "Pendulum"
},
{
"duration": "253"
},
{
"tracknum": "1"
},
{
"type": "Ogg Vorbis (Spotify)"
},
{
"bitrate": "320k VBR"
},
{
"coverart": "0"
},
{
"url": "spotify:track:2A7ZZ1tjaluKYMlT3ItSfN"
},
{
"remote": 1
}
]
}
}
What i'm trying to get is result.songinfoloop.title (but I tried that!)

The songinfo_loop structure is.. peculiar. It is a list of dictionaries each with just one key.
Loop through it until you have one with a title:
TrackInfo = next(d['title'] for d in responce['result']["songinfo_loop"] if 'title' in d)
TracksInfo.append(TrackInfo)
A better option would be to 'collapse' all those dictionaries into one:
songinfo = reduce(lambda d, p: d.update(p) or d,
responce['result']["songinfo_loop"], {})
TracksInfo.append(songinfo['title'])

songinfo_loop is a list not a dict. That means you need to call it by position, or loop through it and find the dict with a key value of "title"
positional:
responce["result"]["songinfo_loop"][1]["title"]
loop:
for info in responce["result"]["songinfo_loop"]:
if "title" in info.keys():
print info["title"]
break
else:
print "no song title found"
Really, it seems like you would want to have the songinfo_loop be a dict, not a list. But if you need to leave it as a list, this is how you would pull the title.

The result is really a standard python dict, so you can use
responce["result"]["songinfoloop"]["title"]
which should work

How to format unicode strings to utf-8 in Python?

I'm reading in a JSON string which is littered with u'string' style strings. Example:
[
{
"!\/award\/award_honor\/honored_for": {
"award": {
"id": "\/en\/spiel_des_jahres"
},
"year": {
"value": "1996"
}
},
"guid": "#9202a8c04000641f80000000003a0ee6",
"type": "\/games\/game",
"id": "\/en\/el_grande",
"name": "El Grande"
},
{
"!\/award\/award_honor\/honored_for": {
"award": {
"id": "\/en\/spiel_des_jahres"
},
"year": {
"value": "1995"
}
},
"guid": "#9202a8c04000641f80000000000495ec",
"type": "\/games\/game",
"id": "\/en\/settlers_of_catan",
"name": "Settlers of Catan"
}
]
If I assign name = result.name. Then when I log of pass that value to a Django template, it displays as u'Dominion'
How do I format it to display as Dominion?
++ UPDATE ++
I think the problem has to do with printing values from a list or dictionary. For example:
result = freebase.mqlread(query)
games = {}
count = 0
r = result[0]
name = r.name
games["name"] = name,
self.response.out.write(games["name"])
self.response.out.write(name)
This displays as:
(u'Dominion',) // saved response to dictionary, and then printed
Dominion // when calling the value directly from the response
I need to iterate through an array of JSON items and the values are being shown with the unicode. Why?

The comma at the end of games["name"] = name, makes it a 1-tuple. Remove it.

>>> # example
>>> s = u"Jägermütze"
>>> s.encode("utf-8")
'J\xc3\xa4germ\xc3\xbctze'
>>> print s.encode("utf-8") # on a utf-8 terminal
Jägermütze
Don't know much about Django, but not accepting snicode strings seems unpythonic to me.

You can use str(your string) to do this.