We Keep Coding

How to get value from JSON - Python

If anyone can help me to get a value from a json response. I intend to get only the first incidence of id where the type is CLIENTREF.
I am reading a list to get some values to do a GET and check if the response is other than null "".
This is the JSON structure:
This is my code so far
def checkParent(list):
for item in list:
cdwParenturl = f"http://cdwu/cdw/counterparties/{item}/?yyyy-mm-dd={dateinplay}"
r = requests.get(cdwParenturl).json()
jsonpath_expression = parse("$..identifier[*]")
# $..identifier[?(#.type == 'CLIENTREF')].id
parentId = []
for match in jsonpath_expression.find(r):
# print(f"match id: {match.value}")
thisdict = match.value
if thisdict["type"] == "CLIENTREF":
# print(thisdict["id"])
parentId.append(thisdict["id"])
elif thisdict["id"] == "":
print(colored(f"The parent {item} does not have ultimateParent", "red"))
print(colored(f"All counterparties have parent", "green"))
checkParent(r_mheu_trade)
print(f "match id: {match.value}") I get this, What I need is the first id related to the type: CLIENTREF
The error that appears on my console is this:
Traceback (most recent call last):
File "h:\DESKTOP\test_check\checkCounterpartie.py", line 114, in <module>
checkParent(r_mheu_trade)
File "h:\DESKTOP\test_check\checkCounterpartie.py", line 106, in checkParent
if thisdict["type"] == "CLIENTREF":
KeyError: 'type'

With the help of pyzer I found a solution to my problem. This is how the code looks like:
# Request to get all UltimateParent
def checkParent(murex):
for item in murex:
cdwParenturl = f"http://cdwu/cdw/counterparties/{item}/?yyyy-mm-dd={dateinplay}"
r = requests.get(cdwParenturl).json()
clientref = r[0]["riskUltimateParent"]["identifier"][1]
if clientref == "":
print(colored(f"The parent {item} does not have ultimateParent", "red"))
else:
print(colored(f"All counterparties have parent", "green"))
checkParent(r_mheu_trade)

Python Parsing Trace File

I have a trace file downloaded from chrome://bookmarks and it looks like this
{"args":{"name":"Chrome_ChildIOThread"},"cat":"__metadata","name":"thread_name","ph":"M","pid":24050,"tid":13059,"ts":0},
{"args":{"name":"Compositor"},"cat":"__metadata","name":"thread_name","ph":"M","pid":24050,"tid":42243,"ts":0},
{"args":{"name":"CompositorTileWorker1"},"cat":"__metadata","name":"thread_name","ph":"M","pid":24050,"tid":23555,"ts":0},
{"args":{"name":"CompositorTileWorker2"},"cat":"__metadata","name":"thread_name","ph":"M","pid":24050,"tid":40963,"ts":0},
{"args":{"name":"ThreadPoolForegroundWorker"},"cat":"__metadata","name":"thread_name","ph":"M","pid":24050,"tid":12035,"ts":0},
{"args":{"name":"Chrome_ChildIOThread"},"cat":"__metadata","name":"thread_name","ph":"M","pid":17240,"tid":13315,"ts":0},
{"args":{"name":"Chrome_ChildIOThread"},"cat":"__metadata","name":"thread_name","ph":"M","pid":17247,"tid":13059,"ts":0},
{"args":{"name":"Chrome_ChildIOThread"},"cat":"__metadata","name":"thread_name","ph":"M","pid":17244,"tid":18435,"ts":0},
I would like to read this file and load into a dictionary to get the traces that name is equal to Chrome_ChildIOThread and so on. How can I do that?

Let's assume that the data you've shown as input is in a file called bookmarks.txt then this should suffice:
D = {'data': []}
with open('bookmarks.txt') as bm:
for line in bm:
try:
i = line.rindex('}') + 1
j = eval(line[:i])
if j['args']['name'] == 'Chrome_ChildIOThread':
D['data'].append(j)
except (ValueError, KeyError):
pass
print(D)

Python shelve not saving nested keys

I am using the shelve module to save some Python objects (strings in the example).
When I am trying to save an object as a nested key, then it is not being saved.
class Car:
def __init__(self, ID):
self.id = ID
def save_to_shelf(self):
with shelve.open('shelf') as shelf:
if not shelf.get('users'):
shelf['users'] = {}
print(shelf['users'])
try:
shelf['users'][self.id] = "hello"
except Exception as e:
print(e)
print('saved')
print(shelf['users'])
car = Car(123)
car.save_to_shelf()
The code should print:
{}
saved
{123: hello}
But instead it prints:
{}
saved
{}
Meaning that it is not getting saved
If I print the value of the key, it gives KeyError
shelf['users'][self.id] = "hello"
print(shelf['users'][self.id])
Error
Traceback (most recent call last):
File "P:/Python practice/Shelll/main.py", line 3, in <module>
car.save_to_shelf()
File "P:/Python practice/Shelll\db.py", line 20, in save_to_shelf
print(shelf['users'][self.id])
KeyError: '123'
I am able to save it if I do the following while saving
Instead of
with shelve.open('shelf') as shelf:
shelf['users'][self.id] = "hello"
This works
with shelve.open('shelf') as shelf:
USERS = shelf['users']
USERS[self.id] = self.id
shelf['users'] = USERS
# or this works
# with shelve.open('shelf') as shelf:
# shelf['users'] = {self.id:"hello"}
I want to understand the reason behind this. As far as I know, shelve objects work as a dictionary. So the way I was saving earlier should work but does not.

'exceptions.TypeError' when executing python script

I'm getting the following error when executing the following script:
Error Type: <type 'exceptions.TypeError'>
Error Contents: 'NoneType' object is not iterable
Traceback (most recent call last):
File "addon.py", line 75, in <module>
plugin.run()
File "xbmcswift2/plugin.py", line 332, in run
items = self._dispatch(self.request.path)
File "/plugin.py", line 306, in _dispatch
listitems = view_func(**items)
File "/addon.py", line 42, in all_episodes
items = thisiscriminal.compile_playable_podcast(playable_podcast)
File "/lib/thisiscriminal.py", line 121, in compile_playable_podcast
for podcast in playable_podcast:
TypeError: 'NoneType' object is not iterable
The code in question is as follows, any advice would be greatly appreciated as I have no idea what I'm doing wrong:
def get_playable_podcast(soup):
"""
#param: parsed html page
"""
r = urllib.urlopen('https://thisiscriminal.com/wp-json/criminal/v1/episodes?posts=10000&page=1')
data = json.loads(r.read().decode('utf-8'))
for post in data['posts']:
print post['title']
print post['episodeNumber']
print post['audioSource']
print post['image']['medium']
subjects = []
item = {
'title': post['title'],
'audioSource': post['audioSource'],
'episodeNumber': post['episodeNumber'],
'medium': post['image']['medium']
}
subjects.append(item)
print subjects
def compile_playable_podcast(playable_podcast):
"""
#para: list containing dict of key/values pairs for playable podcasts
"""
items = []
for podcast in playable_podcast:
items.append({
post['title']: podcast['title']['episodeNumber'],
post['audioSource']: podcast['audioSource'],
post['image']['medium']: podcast['medium'],
'is_playable': True,})
return items

I assume your script does something alike to the following,
podcast = get_playable_podcast(soup)
compiled = compile_playable_podcast(podcast)
The problem is that get_playable_podcast has no return statement. In such a case, Python defaults to returning None - which you then pass into compile_playable_podcast. Since None is not iterable, compile_playable_podcast rightfully raises a TypeError.
Now, the solution is of course to return the podcast list you're building in get_playable_podcast, like so,
def get_playable_podcast(soup):
"""
#param: parsed html page
"""
r = urllib.urlopen('https://thisiscriminal.com/wp-json/criminal/v1/episodes?posts=10000&page=1')
data = json.loads(r.read().decode('utf-8'))
subjects = []
for post in data['posts']:
print post['title']
print post['episodeNumber']
print post['audioSource']
print post['image']['medium']
item = {
'title': post['title'],
'audioSource': post['audioSource'],
'episodeNumber': post['episodeNumber'],
'medium': post['image']['medium']
}
subjects.append(item)
print subjects
return subjects
Beside this, it may be worthwhile to carefully check your script for unused parameters and/or duplicate code.

How to tackle a NoneTypeError?

I have a function which returns a list. But it has nothing to return. I want to handle the TypeError.
I have tried this:
def isLookingAround(lst):
res = []
body = []
for data in lst:
res += isLookingAt(normalize2(data))
body += isBodyDirection(normalize2(data))
if most_body == "front" or most_body == "backward":
if ('lookL' in res and 'lookR' in res):
return 'lookingAround'
elif most_body == "left" or most_body == "right":
if ('lookF' in res and 'lookB' in res):
return 'lookingAround'
Error:
Traceback (most recent call last):
File "action_detector.py", line 201, in <module>
write_labels(input_source, labels)
File "action_detector.py", line 179, in write_labels
for itr, word in enumerate(lbls):
TypeError: 'NoneType' object is not iterable
I am still getting the error with labels.append(detectors.isLookingAround(back_Data)) . I would appreciate your help.

To check data is empty. you can use below code
if data:
l.append(data)

In order to handle the NoneType exception, you can surround the statement within try-except blocks.
For example, you can do something like,
try:
labels.append(detectors.isLookingAround(back_Data))
except TypeError:
# Do something. Like,
return
It will "catch" the error, and then you can handle it properly. However, the error seems to come from isLookingAround method.
You can re-check that, and if you're unable to resolve it, post a new question, I guess.