I have a list of json objects (dictionaries) ds_list
ds_list = [ { "status": "NEW" }, { "status": "UP_TO_DATE" }]
I need to modify an attribute of each object.
So here is my solution:
if we_are_processing:
result = list(map(lambda ds: ds.update({'status': 'PROCESSING'}) or ds, ds_list))
result = [ { "status": "PROCESSING" }, { "status": "PROCESSING" }]
It works, but I don't like it very much, in particular update() and or ds.
What is more pythonic (readable) way of implementing it?
The Pythonic way is to use for loop:
ds_list = [ { "status": "NEW" }, { "status": "UP_TO_DATE" }]
for item in ds_list:
item['status'] = 'PENDING'
# [{'status': 'PENDING'}, {'status': 'PENDING'}]
Related
I'm working on taking a JSON feed and filtering out only the items I want from my list. I'm appending the items I'd like to keep to each list identifier. However, when I convert to JSON the output is incorrect. You can see the ACTUAL OUTPUT example below. The target output below is what I'm actually expecting. I've tried orienting the list with index and records, but no luck.
#TARGET OUTPUT
{
"id":"1",
"Name":"xxx",
"Image":"https://xxx.xxx.png",
},
{
"id":"2",
"Name":"xx2",
"Image":"https://xx2.xxx.png",
}
#ACTUAL OUTPUT
{
"id": ["1","2",]
},
{
"image":["https://xxx.xxx.png","https://xx2.xxx.png"]
},
{
"name":["xxx", "xx2"]
},
#CODE
# JSON feed
{
"document": {
"id": "1",
"image": "https://xxx.xxx.png",
"name": "xxx",
},
},
{
"document": {
"id": "2",
"image": "https://xx2.xxx.png",
"name": "xx2",
},
},
# create list array
list = {'id':[], 'Name': [], 'Image': []}
links = {'id': [], 'Image': []}
# loop through and append items
def getData(hits):
for item in filter(None, hits):
item = item['document']
list['id'].append(item['id'])
links['id'].append(item['id'])
links['Image'].append(item['image'])
list['Image'].append(item['image'])
list['Name'].append(item['name'])
# get first page
pageNum = 1
data = getDataPerPage(pageNum)
try:
itemsNo = data['found']
getData(data['hits'])
while itemsNo > 24:
itemsNo -= 24
pageNum += 1
data = getDataPerPage(pageNum)
getData(data['hits'])
except:
print("broken")
# save list to json
with open('./output/data_chart.json', 'w') as f:
f.write(json.dumps(list))
When you receive multiple JSON objects, those are in the form of a list (so between []). You could:
covert JSON string to python dictionary using json.loads()
filter using the dict
dump dictionary into a JSON string using json.dumps()
input = """[
{"document":
{"id": "1","image": "https://xxx.xxx.png","name": "xxx"}},
{"document":
{"id": "2","image": "https://xx2.xxx.png","name": "xx2"}}
]"""
input_dic = json.loads(input)
tmp = []
for item in input_dic:
tmp.append(json.dumps(item["document"]))
output = json.dumps(tmp)
print(output)
Hope I got your question.
It's not 100% clear what you have or what you want, but with a few assumptions (input is list of dict, desired output is list of dict):
json_obj = [
{
"document": {
"id": "1",
"image": "https://xxx.xxx.png",
"name": "xxx",
},
},
{
"document": {
"id": "2",
"image": "https://xx2.xxx.png",
"name": "xx2",
},
},
]
desired_output = [x["document"] for x in json_obj]
print(desired_output)
I have a nested dict(as below). Goal is to to extract the values of "req_key1" and "rkey1", and append them to a list.
raw_json = {
"first_key": {
"f_sub_key1": "some_value",
"f_sub_key2": "some_value"
},
"second_key": {
"another_key": [{
"s_sub_key1": [{
"date": "2022-01-01",
"day": {
"key1": "value_1",
"keyn": "value_n"
}
}],
"s_sub_key2": [{
"req_key1": "req_value1",
"req_key2": {
"rkey1": "rvalue_1",
"rkeyn": "rvalue_n"
}
}]
}]
}
}
I am able to append the values to a list and below is my approach.
emp_ls = []
filtered_key = raw_json["second_key"]["another_key"]
for i in filtered_key:
for k in i.get("s_sub_key2"):
emp_ls.append({"first_val": k.get("req_key1"), "second_val": k["req_key2"].get("rkey1") })
print(emp_ls)
Is it a good approach i.e. it can be used in production or there can be another approach to do this task?
I want to pop an item from a list of lists.
So I have a scan for all dynamo Items and want to "pop" one field from each list.
For example:
response = table.query(
KeyConditionExpression=Key('userId').eq(userId)
)
agentList = response['Items']
The List:
"result": [
{
"credentials": {
"key": "xxx",
"secret": "xxxx"
},
"active": true,
"totalImported": "12345",
}]
From this example, I have a bunch of Results and for every result list, I will remove the Item "credentials" like
agentList.pop('credentials')
However, this isn't working
You're doing the pop() on a list. So, you can specify a position in the list, but, as a list has no key values, you can't use a string. Hence the error.
I'm not sure exactly what you're trying to do, but, assuming you want to remove the 'credentials' from every item in the list you could do something like:
agentList = [
{
"credentials": {
"key": "xxx",
"secret": "xxxx"
},
"active": True,
"totalImported": "12345",
},
{
"credentials": {
"key": "yyy",
"secret": "yyy"
},
"active": True,
"totalImported": "2222",
}
]
for result in agentList:
result.pop('credentials')
print(agentList)
Which would result in:
[{'active': True, 'totalImported': '12345'}, {'active': True, 'totalImported': '2222'}]
I finally found this way:
print(items)
agentList =[]
for result in items['Items']:
agentList.append(result)
result.pop('credentials')
Appending the List each by each and afterwards "popping" the "credentials" field.
I managed to scrape some content and organize it as a nested dictionary like this.
country_data = {
"US": {
"People": [
{
"Title": "Pres.",
"Name": "Joe"
},
{
"Title": "Vice",
"Name": "Harris"
}
]
}
}
Then I have this list
tw_usernames = ['#user1', '#user2']
that I'd like to use to add each item list to each entry of the People nested dictionary. I have done some research about list and dict comprehension but I cannot find something to make it work, so I tried with this basic code but of course it's not exactly what I want, as it returns all the items list.
for firstdict in country_data.values():
for dictpeople in firstdict.values():
for name in dictpeople:
name['Twitter'] = tw_usernames
print(name)
So how would you do it to get dictionary like this?
country_data = {
"US": {
"People": [
{
"Title": "Pres.",
"Name": "Joe",
"Twitter": "#user1"
},
{
"Title": "Vice",
"Name": "Harris",
"Twitter": "#user2"
}
]
}
}
Thanks in advance for any tip that could teach me.
Try this. I have just added index in your logic. It will pick the username according to the index
idx = 0
tw_usernames = ['#user1', '#user2']
for firstdict in country_data.values():
for dictpeople in firstdict.values():
for name in dictpeople:
name['Twitter'] = tw_usernames[idx]
idx+=1
print(country_data)
output
{
"US":{
"People":[
{
"Title":"Pres.",
"Name":"Joe",
"Twitter":"#user1"
},
{
"Title":"Vice",
"Name":"Harris",
"Twitter":"#user2"
}
]
}
}
You can zip the list of sub-dicts with the list of usernames and iterate over the resulting sequence of pairs to add a Twitter key to each sub-dict:
for person, username in zip(country_data['US']['People'], tw_usernames):
person['Twitter'] = username
With your sample input, country_data would become:
{'US': {'People': [{'Title': 'Pres.', 'Name': 'Joe', 'Twitter': '#user1'}, {'Title': 'Vice', 'Name': 'Harris', 'Twitter': '#user2'}]}}
Source code
def add_twitter_usernames_to_users(country_data: dict, tw_usernames: [str]):
for tw_username in tw_usernames:
country_data["US"]["People"][tw_usernames.index(tw_username)]["Twitter"] = tw_username
return country_data
Test
def test_add_twitter_usernames_to_users():
country_data = {
"US": {
"People": [
{
"Title": "Pres.",
"Name": "Joe"
},
{
"Title": "Vice",
"Name": "Harris"
}
]
}
}
tw_usernames = ['#user1', '#user2']
updated_country_data: dict = so.add_twitter_usernames_to_users(country_data, tw_usernames)
assert updated_country_data["US"]["People"][0]["Twitter"] == "#user1"
assert updated_country_data["US"]["People"][1]["Twitter"] == "#user2"
What are the options for extracting value from JSON depending on other parameters (using python)? For example, JSON:
"list": [
{
"name": "value",
"id": "123456789"
},
{
"name": "needed-value",
"id": "987654321"
}
]
When using json_name["list"][0]["id"] it obviously returns 123456789. Is there a way to indicate "name" value "needed-value" so i could get 987654321 in return?
For example:
import json as j
s = '''
{
"list": [
{
"name": "value",
"id": "123456789"
},
{
"name": "needed-value",
"id": "987654321"
}
]
}
'''
js = j.loads(s)
print [x["id"] for x in js["list"] if x["name"] == "needed-value"]
The best way to handle this is to refactor the json as a single dictionary. Since "name" and "id" are redundant you can make the dictionary with the value from "name" as the key and the value from "id" as the value.
import json
j = '''{
"list":[
{
"name": "value",
"id": "123456789"
},{
"name": "needed-value",
"id": "987654321"
}
]
}'''
jlist = json.loads(j)['list']
d = {jd['name']: jd['id'] for jd in jlist}
print(d) ##{'value': '123456789', 'needed-value': '987654321'}
Now you can iterate the items like you normally would from a dictionary.
for k, v in d.items():
print(k, v)
# value 123456789
# needed-value 987654321
And since the names are now hashed, you can check membership more efficiently than continually querying the list.
assert 'needed-value' in d
jsn = {
"list": [
{
"name": "value",
"id": "123456789"
},
{
"name": "needed-value",
"id": "987654321"
}
]
}
def get_id(list, name):
for el in list:
if el['name'] == name:
yield el['id']
print(list(get_id(jsn['list'], 'needed-value')))
Python innately treats JSON as a list of dictionaries. With this in mind, you can call the index of the list you need to be returned since you know it's location in the list (and child dictionary).
In your case, I would use list[1]["id"]
If, however, you don't know where the position of your needed value is within the list, the you can run an old fashioned for loop this way:
for user in list:
if user["name"] == "needed_value":
return user["id"]
This is assuming you only have one unique needed_value in your list.