We Keep Coding

How to get each values of a column in a data frame that contains a list of dictionaries?

As you can see I have a column that contains an array of dictionaries. I need to see if a key in any item has a certain value and return the row if it does.
0 [{'id': 473172988, 'node_id': 'MDU6TGFiZWw0NzM...
1 [{'id': 473172988, 'node_id': 'MDU6TGFiZWw0NzM...
2 [{'id': 473172988, 'node_id': 'MDU6TGFiZWw0NzM...
3 [{'id': 473173351, 'node_id': 'MDU6TGFiZWw0NzM...
Is there a straightforward approach for this?
The datatype of the column is an object.

You would need to give the exact format of your dictionary, but on the general principle you should loop over the elements:
key = 'xxx'
value = 'yyy'
out = [any(d.get(key) == value for d in l) for l in df['your_column']]
# slicing rows
df[out]

Python 3.9.5: One dictionary assignment is overwriting multiple keys [BUG?]

I am reading a .csv called courses. Each row corresponds to a course which has an id, a name, and a teacher. They are to be stored in a Dict. An example:
list_courses = {
1: {'id': 1, 'name': 'Biology', 'teacher': 'Mr. D'},
...
}
While iterating the rows using enumerate(file_csv.readlines()) I am performing the following:
list_courses={}
for idx, row in enumerate(file_csv.readlines()):
# Skip blank rows.
if row.isspace(): continue
# If we're using the row, turn it into a list.
row = row.strip().split(",")
# If it's the header row, take note of the header. Use these values for the dictionaries' keys.
# As of 3.7 a Dict remembers the order in which the keys were inserted.
# Since the order is constant, simply load each other row into the corresponding key.
if not idx:
sheet_item = dict.fromkeys(row)
continue
# Loop through the keys in sheet_item. Assign the value found in the row, converting to int where necessary.
for idx, key in enumerate(list(sheet_item)):
sheet_item[key] = int(row[idx].strip()) if key == 'id' or key == 'mark' else row[idx].strip()
# Course list
print("ADDING COURSE WITH ID {} TO THE DICTIONARY:".format(sheet_item['id']))
list_courses[sheet_item['id']] = sheet_item
print("\tADDED: {}".format(sheet_item))
print("\tDICT : {}".format(list_courses))
Thus, the list_courses dictionary is printed after each sheet_item is added to it.
Now comes the issue - when reading in two courses, I expect that list_courses should read:
list_courses = {
1: {'id': 1, 'name': 'Biology', 'teacher': 'Mr. D'},
2: {'id': 2, 'name': 'History', 'teacher': 'Mrs. P'}
}
However, the output of my print statements (substantiated by errors later in my program) is:
ADDING COURSE WITH ID 1 TO THE DICTIONARY:
ADDED: {'id': 1, 'name': 'Biology', 'teacher': 'Mr. D'}
DICT : {1: {'id': 1, 'name': 'Biology', 'teacher': 'Mr. D'}}
ADDING COURSE WITH ID 2 TO THE DICTIONARY:
ADDED: {'id': 2, 'name': 'History', 'teacher': 'Mrs. P'}
DICT : {1: {'id': 2, 'name': 'History', 'teacher': 'Mrs. P'}, 2: {'id': 2, 'name': 'History', 'teacher': 'Mrs. P'}}
Thus, the id with which the sheet_item is being added to courses_list is correct (1 or 2), however the assignment which occurs for the second course appears to be overwriting the value for key 1. I'm not even sure how this is possible. Please let me know your thoughts.

You're using the same dictionary for both the header and all the rows. You never create any new dictionaries after the header. Key assignments are overwriting previous ones, because there are no new dictionaries to write to.
Store the keys in a list, and make a new sheet_item before the for loop:
list_courses={}
keys = None # Let Python know this is defined
for idx, row in enumerate(file_csv.readlines()):
# Skip blank rows.
if row.isspace(): continue
# If we're using the row, turn it into a list.
row = row.strip().split(",")
# If it's the header row, take note of the header. Use these values for the dictionaries' keys.
# As of 3.7 a Dict remembers the order in which the keys were inserted.
# Since the order is constant, simply load each other row into the corresponding key.
if not idx:
keys = row
continue
sheet_item = {}
# Loop through the keys in sheet_item. Assign the value found in the row, converting to int where necessary.
for idx, key in enumerate(keys):
sheet_item[key] = int(row[idx].strip()) if key == 'id' or key == 'mark' else row[idx].strip()
# Course list
print("ADDING COURSE WITH ID {} TO THE DICTIONARY:".format(sheet_item['id']))
list_courses[sheet_item['id']] = sheet_item
print("\tADDED: {}".format(sheet_item))
print("\tDICT : {}".format(list_courses))

How to iterate through this nested dictionary within a list using for loop

I have a list of nested dictionaries that I want to get specific values and put into a dictionary like this:
vid = [{'a':{'display':'axe', 'desc':'red'}, 'b':{'confidence':'good'}},
{'a':{'display':'book', 'desc':'blue'}, 'b':{'confidence':'poor'}},
{'a':{'display':'apple', 'desc':'green'}, 'b':{'confidence':'good'}}
]
I saw previous questions similar to this, but I still can't get the values such as 'axe' and 'red'. I would like the new dict to have a 'Description', 'Confidence' and other columns with the values from the nested dict.
I have tried this for loop:
new_dict = {}
for x in range(len(vid)):
for y in vid[x]['a']:
desc = y['desc']
new_dict['Description'] = desc
I got many errors but mostly this error:
TypeError: string indices must be integers
Can someone please help solve how to get the values from the nested dictionary?

You don't need to iterate through the keys in the dictionary (the inner for-loop), just access the value you want.
vid = [{'a':{'display':'axe', 'desc':'red'}, 'b':{'confidence':'good'} },
{'a':{'display':'book', 'desc':'blue'}, 'b':{'confidence':'poor'}},
{'a':{'display':'apple', 'desc':'green'}, 'b':{'confidence':'good'}}
]
new_dict = {}
list_of_dicts = []
for x in range(len(vid)):
desc = vid[x]['a']['desc']
list_of_dicts.append({'desc': desc})

I have found a temporary solution for this. I decided to use the pandas dataframe instead.
df = pd.DataFrame(columns = ['Desc'])
for x in range(len(vid)):
desc = vid[x]['a']['desc']
df.loc[len(df)] = [desc]

so you want to write this to csv later so pandas will help you a lot for this problem using pandas you can get the desc by
import pandas as pd
new_dict = {}
df = pd.DataFrame(vid)
for index, row in df.iterrows() :
new_dict['description'] = row['a']['desc']
a b
0 {'display': 'axe', 'desc': 'red'} {'confidence': 'good'}
1 {'display': 'book', 'desc': 'blue'} {'confidence': 'poor'}
2 {'display': 'apple', 'desc': 'green'} {'confidence': 'good'}
this is how dataframe looks like a b are column of the dataframe and your nested dicts are rows of dataframe

Try using this list comprehension:
d = [{'Description': i['a']['desc'], 'Confidence': i['b']['confidence']} for i in vid]
print(d)

how to get data in dict from list and store back in dict form? [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
Here is the dictionary form
abc = {
'if1': {'name': 'data', 'date': '80980'},
'if2': {'name': 'data_1', 'date': '9886878'},
'if3': {'name': 'data', 'date': '0987667'},
'if4': {'name': 'data__5', 'date': '0987667'},
'if5': {'date': '0987667'}
}
and I am trying to apply a filter using the NAME when I give input filter in the form of a list
list_item = ['data','data_1']
it should give me the output dates as follows
{
data:['80980', '0987667'],
data_1:['9886878']
}
please help me to resolve this issue.

For the resulting dictionary we create a defaultdict with an empty list as the default value. Then we loop over all values of 'abc' and check if we have an entry with the key 'name' and the corresponding value is in list_items. If this is the case we can use the name as a key for the resulting dictionary and append the value of the element with the key 'date'.
abc = {
'if1': {'name': 'data', 'date': '80980'},
'if2': {'name': 'data_1', 'date': '9886878'},
'if3': {'name': 'data', 'date': '0987667'},
'if4': {'name': 'data__5', 'date': '0987667'},
'if5': {'date': '0987667'}
}
list_item = ['data','data_1']
import collections
result = collections.defaultdict(list)
for item in abc.values():
if item.get('name', None) in list_item:
result[item['name']].append(item['date'])
print(result)
An other approach is looping over the values in 'list_item'.
result = {}
for key in list_item:
result[key] = [item['date'] for item in abc.values() if item.get('name', None) == key]
print(result)
Using a dictionary comprehension you can transform the last solution into a one-liner (but I prefer a more readable style):
result = {key:[item['date'] for item in abc.values() if item.get('name', None) == key] for key in list_item}

This dict is so complex you can't just filter it, you have to convert it to something more useful.
abc= { 'if1':{'name':'data','date':'80980'}, 'if2':{'name':'data_1','date':'9886878'}, 'if3':{'name':'data','date':'0987667'}, 'if4':{'name':'data__5','date':'0987667'}}
[The dictionary was made wrongly, I assumed 2nd if4 is a typo and I deleted it when I copied the text.]
First, let's flatten it by removing the inside dictionary, making date our dict's value:
formatted1 = {(key, subdict['name']): subdict['date'] for key, subdict in abc.items() if 'name' in subdict}
I kept the original key as part of the new key because otherwise we'd overwrite our data entry.
Our new dict looks like that:
{('if1', 'data'): '80980', ('if2', 'data_1'): '9886878', ('if3', 'data'): '0987667', ('if4', 'data__5'): '0987667'}
Now it's easier to work with. Let's do a simple loop to format it further:
formatted2 = {}
for (_, key), value in formatted1.items(): # we'll be skipping first part of the key, hence I didn't give it a meaningful name
elem = formatted2.get(key, [])
elem.append(value)
formatted2[key] = elem
Our even newer dict looks now like that:
{'data': ['80980', '0987667'], 'data_1': ['9886878'], 'data__5': ['0987667']}
And now it's finally in a form we can easily filter!
list_item = ['data','data_1']
result = {k: formatted2[k] for k in formatted2 if k in list_item}
Result:
{'data': ['80980', '0987667'], 'data_1': ['9886878']}

Reading and manipulating JSON elements on Python

I'm a python newbie and I need to read and manipulate elements from a json file, but I keep getting errors and I have no idea how to move forward. This is my code:
import json
with open('file.txt', 'r') as json_data:
d = json.load(json_data)
An example of the dataset:
[
{
'id': 1,
'name': 'a',
'city': 'Paris'
},
{
'id': 2,
'name': 'b',
'city': 'Mons'
},
{
'id': 3,
'name': 'c.',
'city': 'Leuven'
}
]
When I try to get only id or name, I get this error:
city = d['city']
TypeError Traceback (most recent call last)
in ()
----> 1 city = d['city']
TypeError: list indices must be integers or slices, not str
Then I tried this:
city = d[:]['city']
TypeError Traceback (most recent call last)
in ()
----> 1 city = d[:]['city']
TypeError: list indices must be integers or slices, not str
Any ideas? Thanks!

You more likely don't want to know the array index of the element you are looking for.
With some pythonic flavor, you can create tuples with a list comprehension like this:
arr = [(dict['id'], dict['city']) for dict in d]
The output would be
[(1, 'Paris'),
(2, 'Mons'),
(3, 'Leuven')]
Then, you have the possibility to get only specific items in your tuples if needed.
Ex:
arr = [(dict['id'], dict['city']) for dict in d if 's' in dict['city']]
which would return id and name for every entry that contain 's' in the city property.

As this is a dictionary within a list you must provide a list index before calling value by the key. This should work:
dict = d[0]
city = dict['city']
Or you can simply use:
city = d[0]['city']

You can write a loop to go through each object
final=[]
for obj in d:
final.append(obj['city'])
Or you can try using this
final = [obj['city'] for obj in d]
Or if you only need the first value then
print(d[0]['city'])
Output
'Paris'
Since your data is a list of dictionaries, you'll have to use the index value to get the data in the dictionary