We Keep Coding

How to extract info from the dictionaries within a list?

I'm new to Python, trying to gather data from a json file that consists of a list that contains info inside dictionaries as follows. How do I extract the "count" data from this? (Without using list comprehension)
{
"stats":[
{
"name":"Ron",
"count":98
},
{
"name":"Sandy",
"count":89
},
{
"name":"Sam",
"count":77
}
]
}

Index the list using the stats key then iterate through it
data = {
"stats":[
{
"name":"Ron",
"count":98
},
{
"name":"Sandy",
"count":89
},
{
"name":"Sam",
"count":77
}
]
}
for stat in data['stats']:
count = stat['count']

Consider the dictionary data stored in a variable source.
source = {
"stats":[
{
"name":"Ron",
"count":98
},
{
"name":"Sandy",
"count":89
},
{
"name":"Sam",
"count":77
}
]
}
Now to access the count field inside of "stats" we use indexing.
For example, to view the count of "Ron" you would write:
print(source['stats'][0]['count'])
This will result in 98
Similarly, for "Sam" it will be
print(source['stats'][2]['count'])
And the result will be 77
In short, we first index the key of dictionary, then the array position and then provide the filed from array of which you want the data.
I hope it helped.

Simply append all those values to do calculations:
count_values = []
for dic in data['stats']:
count_values.append(dic['count'])
# Do anything with count_values
print(count_values)

According to the Zen of Python, "Simple is better than complex."
Thus, list comprehension is actually the best way to extract the information you need and still have it available for further processing (in the form of a list of count values).
d = <your dict-list>
count_data_list = [ x['count'] for x in d['stats'] ]
If not, and your intention is to process the "count" data as it is extracted, I'd suggest a for-loop:
d = <your dict-list>
for x in d['stats']:
count_data = x['count']
<process "count_data">

using a map function will do that in a single line
>>> result = list(map(lambda x: x['count'], data['stats']))
[98, 89, 77]

How to populate optional fields in a dictionary comprehension?

I'm working on some code that processes a json database with very detailed information into a simpler format. It copies some of the fields and reserializes others into a new json file.
I'm currently using a dictionary comprehension like this MVCE:
converted_data = {
raw_item['name']: {
'state': raw_item['field_a'],
'variations': [variant for variant in raw_item['field_b']['variations']]
} for raw_item in json.loads(my_file.read())
}
An example file (not the actual data being used) is this:
[
{
"name": "Object A",
"field_a": "foo",
"field_b": {
"bar": "baz",
"variants": [
"foo",
"bar",
"baz"
]
}
},
{
"name": "Object B",
"field_a": "foo",
"field_b": {
"bar": "baz",
}
}
]
The challenge is that not all items contain variations. I see two potential solutions:
Use an if statement to conditionally apply the variations field into the dictionary.
Include an empty variations field for all items and fill it if the raw item contains variations.
I'll probably settle on the 2nd solution. However, is there a way to conditionally include a particular field within a dictionary comprehension?
Edit: In other words, is approach 1 possible inside a dictionary comprehension?
An example of the desired output (using a dictionary comprehension) would be as follows:
{
"Object A": {
"state": "foo",
"variants": ["foo", "bar", "baz"]
},
"Object B": {
"state": "foo"
}
}
I've found some other questions that change the entries conditionally or filter the entries, but these don't unconditionally create an item where a particular field (in the item) is conditionally absent.

I'm not sure you realise you can use the if inside an assignment, which seems like a very clean way to solve it to me:
converted_data = {
raw_item['name']: {
'state': raw_item['field_a'],
'variants': [] if 'variants' not in raw_item['field_b'] else
[str(variant) for variant in raw_item['field_b']['variants']]
} for raw_item in example
}
(Note: using str() instead of undefined function that was given in initial example)
After clarification of the question, here's an alternate solution that adds a different dictionary (missing the empty 'variations' key if there is none:
converted_data = {
raw_item['name']: {
'state': raw_item['field_a'],
'variants': [str(variant) for variant in raw_item['field_b']['variants']]
} if 'variants' in raw_item['field_b'] else {
'state': raw_item['field_a'],
} for raw_item in example
}
If the question actually is: can a key/value pair in a dictionary literal be optional (which would solve your problem) then the answer is simply "no". But the above achieves the same for this simple example.
If the real life situation is more complicated, simply construct the dictionary as in the first solution given here and then use del(dictionary['key']) to remove any added keys that have a None or [] value after construction.
For example, after the first example, converted_data could be cleaned up with:
for item in converted_data.values:
if not item['variants']:
del(item['variants'])

You could pass the process out to a function?
def check_age(age):
return age >= 18
my_dic = {
"age": 25,
"allowed_to_drink": check_age(25)
}
You end up with the value as the result of the function call
{'age': 25, 'allowed_to_drink': True}
How you would implement this I don't know, but some food for thought.

For each loop with JSON object python

Alright, so I'm struggling a little bit with trying to parse my JSON object.
My aim is to grab the certain JSON key and return it's value.
JSON File
{
"files": {
"resources": [
{
"name": "filename",
"hash": "0x001"
},
{
"name": "filename2",
"hash": "0x002"
}
]
}
}
I've developed a function which allows me to parse the JSON code above
Function
def parsePatcher():
url = '{0}/{1}'.format(downloadServer, patcherName)
patch = urllib2.urlopen(url)
data = json.loads(patch.read())
patch.close()
return data
Okay so now I would like to do a foreach statement which prints out each name and hash inside the "resources": [] object.
Foreach statement
for name, hash in patcher["files"]["resources"]:
print name
print hash
But it only prints out "name" and "hash" not "filename" and "0x001"
Am I doing something incorrect here?

By using name, hash as the for loop target, you are unpacking the dictionary:
>>> d = {"name": "filename", "hash": "0x001"}
>>> name, hash = d
>>> name
'name'
>>> hash
'hash'
This happens because iteration over a dictionary only produces the keys:
>>> list(d)
['name', 'hash']
and unpacking uses iteration to produce the values to be assigned to the target names.
That that worked at all is subject to random events even, on Python 3.3 and newer with hash randomisation enabled by default, the order of those two keys could equally be reversed.
Just use one name to assign the dictionary to, and use subscription on that dictionary:
for resource in patcher["files"]["resources"]:
print resource['name']
print resource['hash']

So what you intend to do is :
for dic in x["files"]["resources"]:
print dic['name'],dic['hash']
You need to iterate on those dictionaries in that array resources.

The problem seems to be you have a list of dictionaries, first get each element of the list, and then ask the element (which is the dictionary) for the values for keys name and hash
EDIT: this is tested and works
mydict = {"files": { "resources": [{ "name": "filename", "hash": "0x001"},{ "name": "filename2", "hash": "0x002"}]} }
for element in mydict["files"]["resources"]:
for d in element:
print d, element[d]

If in case you have multiple files and multiple resources inside it. This generalized solution works.
for keys in patcher:
for indices in patcher[keys].keys():
print(patcher[keys][indices])
Checked output from myside
for keys in patcher:
... for indices in patcher[keys].keys():
... print(patcher[keys][indices])
...
[{'hash': '0x001', 'name': 'filename'}, {'hash': '0x002', 'name': 'filename2'}]

Python - Searching JSON

I have JSON output as follows:
{
"service": [{
"name": ["Production"],
"id": ["256212"]
}, {
"name": ["Non-Production"],
"id": ["256213"]
}]
}
I wish to find all ID's where the pair contains "Non-Production" as a name.
I was thinking along the lines of running a loop to check, something like this:
data = json.load(urllib2.urlopen(URL))
for key, value in data.iteritems():
if "Non-Production" in key[value]: print key[value]
However, I can't seem to get the name and ID from the "service" tree, it returns:
if "Non-Production" in key[value]: print key[value]
TypeError: string indices must be integers
Assumptions:
The JSON is in a fixed format, this can't be changed
I do not have root access, and unable to install any additional packages
Essentially the goal is to obtain a list of ID's of non production "services" in the most optimal way.

Here you go:
data = {
"service": [
{"name": ["Production"],
"id": ["256212"]
},
{"name": ["Non-Production"],
"id": ["256213"]}
]
}
for item in data["service"]:
if "Non-Production" in item["name"]:
print(item["id"])

Whatever I see JSON I think about functionnal programming ! Anyone else ?!
I think it is a better idea if you use function like concat or flat, filter and reduce, etc.
Egg one liner:
[s.get('id', [0])[0] for s in filter(lambda srv : "Non-Production" not in srv.get('name', []), data.get('service', {}))]
EDIT:
I updated the code, even if data = {}, the result will be [] an empty id list.

Getting a specific element from a JSON string

I have a JSON string that i got into python that looks something like this:
{"count": 100,
"facets": null,
"previous_page": null,
"results": [{"updated_at": "2013-09-17T13:45:13Z", "test_id": 194037042, "customer_id":
203793326, "id": 1954182}]
There are more elements but this is a small cut of what i need. Basically, results has a list of 100 dictionaries that contain the elements above "updated_at, test_id, customer_id, id" What i need to do is the following:
I need to get a list of all of the values of JUST the id. I am not sure how to go about this though, i have tried doing things like:
for i in my_dict['results']:
print i['id']
but i get an error message that says:
print i['id']
TypeError: string indices must be integers
What am i doing wrong?

You are not doing anything obviously wrong. Your data includes 'null' elements which are not proper python. This works fine.
my_dict = {"count": 100,
"facets": None,
"previous_page": None,
"results": [{"updated_at": "2013-09-17T13:45:13Z", "test_id": 194037042, "customer_id": 203793326, "id": 1954182}]
}
for i in my_dict['results']:
print i['id']
The error you have implies that one of your list items is a string and when you are trying to get the ['id'] element the error rightly tells you that list indices (a string is a list of characters) must be integers.

It looks like you haven't loaded the json into Python with json.loads(json_string)
And even after you do that, your "results" dict is actually going to be a list of dicts.
Try this:
import json
json_str = '{"count": 100, "facets": null, "previous_page": null, "results": [{"updated_at": "2013-09-17T13:45:13Z", "test_id": 194037042, "customer_id": 203793326, "id": 1954182}, {"updated_at": "2013-09-18T13:45:13Z", "test_id": 194037043, "customer_id": 203793327, "id": 1954183}]}'
data = json.loads(json_str)
result_ids = [result['id'] for result in data['results'] if 'id' in result]
Output:
[1954182, 1954183]
This code would then output a list containing 1954182 and 1954183.
Use a list comprehension here for greater speed and less lines of code. It also ensures that the result dict has an 'id' attribute before trying to access it.