Related
I'm trying to create a dictionary of specific key values from a list of dictionaries. I believe my code is not flattening out the dictionaries when i put in chunkdata.extend(pythondict[0][1][2], chunkdata will return with the whole 1st 2nd and 3rd dictionaries where i want something like the "name" key pair for all the dictionaries that return in the response.
chunkdata = []
for chunk in chunklist:
url3 = "some URL"
headers = {'accept': 'application/json',
response = requests.request("GET", url3, headers=headers)
time.sleep(5)
print(response.text)
pythondict = json.loads(response.text)
print(pythondict)
chunkdata.extend(pythondict['name']['age']['date']
pythondict output
[{'data': {'name': 'jon', 'age': '30', 'date': '2020-01-05', 'time': '1', 'color': 'blue'}, {'data': {'name': 'phil', 'age': '33', 'date': '2020-01-05', 'time': '1', 'color': 'blue'}, {'data': {'name': 'ted', 'age': '25', 'date': '2020-01-05', 'time':'1', 'color': 'blue'}]
Traceback (most recent call last):
File line 84, in <module>
chunkdata.extend(pythondict['name']['age']['date']
TypeError: list indices must be integers or slices, not str
Use requests.json() for parsing. It is more reliable and accurate.
Note: Response header MUST contain Content-Type: application/json in the header in order for .json() method to work
I figured out that the json format you get is not right here. I was not able to make out the necessity of the 'data:' prior to each element.
It would be better to modify it in the following form:
python_dict=[{'name': 'jon', 'age': '30', 'date': '2020-01-05', 'time': '1', 'color': 'blue'}, {'name': 'phil', 'age': '33', 'date': '2020-01-05', 'time': '1', 'color': 'blue'}, {'name': 'ted', 'age': '25', 'date': '2020-01-05', 'time':'1', 'color': 'blue'}]
Modify the relevant part of the code as follows:
chunkdata=[]
for x in range(len(python_dict)):
temp_list=[python_dict[x]['name'],python_dict[x]['age'],python_dict[x]['date'],python_dict[x]['time'],python_dict[x]['color']]
chunkdata.append(temp_list)
print(chunkdata)
chunkdata will be a list of lists that you can keep appending into. The output for chunkdata is as follows:
[['jon', '30', '2020-01-05', '1', 'blue'], ['phil', '33',
'2020-01-05', '1', 'blue'], ['ted', '25', '2020-01-05', '1', 'blue']]
I am declaring a method named add_to_cart(db, itemid, quantity). Whenever the method is called, it will look into the database for the session data. Session data contains a list of dictionaries. The purpose of this method is to create a new entry (dictionary) to the list or update the value of existing entry.
Dictionary has the following keys: id, quantity
So far I have developed the following code. First of all after fetching the data from the database, I am matching the itemid with the dictionary key: 'id'. If the itemid does not match with any of the values of the dictionaries, then it will append a new dictionary to that list.
def add_to_cart(db, itemid, quantity):
# ......
row = cursor.fetchone()
if row is not None:
cart = json.loads(row['data'])
for dic in cart:
if str(dic.get("id")) == str(itemid):
dic['quantity'] = int(dic['quantity']) + quantity
data = json.dumps(cart)
# update the 'data' to the database
break
else:
if counter == len(cart):
item = {
'id': itemid,
'quantity': quantity
}
cart.append(item)
data = json.dumps(cart)
# update the 'data' to the database
break
Let the initial cart is like:
[{'id': '40', 'quantity': '2'}, {'id': '41', 'quantity': '5'}]
When I add 1 more of the item 40 to the cart, this should be like:
[{'id': '40', 'quantity': '3'}, {'id': '41', 'quantity': '5'}]
but I am getting :
[{'id': '40', 'quantity': '2'}, {'id': '41', 'quantity': '5'}, {'id': '40', 'quantity': '1'}]
You are adding a new dictionary to the list when you do cart.append(item),
hence the list
[{'id': '40', 'quantity': '2'}, {'id': '41', 'quantity': '5'}]
ends up being
[{'id': '40', 'quantity': '2'}, {'id': '41', 'quantity': '5'}, {'id': '40', 'quantity': '1'}]
But you want to find the matching id in that list of dictionaries, and add to the quantity for that dictionary.
So the code will look like as below:
li = [{'id': '40', 'quantity': '2'}, {'id': '41', 'quantity': '5'}]
def add_elem(li, id, to_add):
#Iterate over the dictionaries
for item in li:
#If the id is found
if str(id) in item.values():
#Increment the quantity
item['quantity'] = str(int(item['quantity']) + to_add)
#Return the updated list
return li
print(add_elem(li, 40, 1))
The output will be
[{'id': '40', 'quantity': '3'}, {'id': '41', 'quantity': '5'}]
The problem seems to be that you are going simply adding a new dict to the list (cart), through append. You need to go through the list, find the dict with the itemid you need, then add to the quantity.
Try this -
for dict in cart:
if dict[itemid] == itemid:
dict['quantity'] += str(quantity)
break
item = {
'id': itemid,
'quantity': quantity
}
cart.append(item)
I'm having difficulty with iterating through the nested list table below. I understand how to iterate through the table once, but to go a level deeper and iterate through each nested list, I am stuck on the correct syntax to use. In iterating through the sublists, I am trying to cast each 'age' and 'years experience' to an integer, perform the operation 'age' - 'years experience', and append the value (as a string) to each sublist.
table = [
['first_name', 'last_name', 'age', 'years experience', 'salary'],
['James', 'Butt', '29', '8', '887174.4'],
['Josephine', 'Darakjy', '59', '39', '1051267.9'],
['Art', 'Venere', '22', '2', '47104.2'],
['Lenna', 'Paprocki', '33', '7', '343240.2'],
['Donette', 'Foller', '26', '2', '273541.4'],
['Simona', 'Morasca', '35', '15', '960967.0'],
['Mitsue', 'Tollner', '51', '31', '162776.7'],
['Leota', 'Dilliard', '64', '39', '464595.5'],
['Sage', 'Wieser', '27', '9', '819519.7'],
['Kris', 'Marrier', '59', '33', '327505.55000000005'],
['Minna', 'Amigon', '45', '23', '571227.05'],
['Abel', 'Maclead', '46', '23', '247927.25'],
['Kiley', 'Caldarera', '33', '7', '179182.8'],
['Graciela', 'Ruta', '48', '21', '136978.95'],
['Cammy', 'Albares', '29', '9', '1016378.95'],
['Mattie', 'Poquette', '39', '15', '86458.75'],
['Meaghan', 'Garufi', '21', '3', '260256.5'],
['Gladys', 'Rim', '52', '26', '827390.5'],
['Yuki', 'Whobrey', '32', '10', '652737.0'],
['Fletcher', 'Flosi', '59', '37', '954975.15']]
##Exercise 3 (rows as lists): Iterate over each row and append the following values:
#If it is the first row then extend it with the following ['Started Working', 'Salary / Experience']
#Start work age (age - years experience)
#Salary / Experience ratio = (salary / divided by experience)
for i, v in enumerate(table):
extension = ['Started Working', 'Salary/Experience']
if i == 0:
v.extend(extension)
print(i,v) #test to print out the index and nested list values
#for index, value in enumerate(v):
# age =
#exp =
#start_work = age - exp
#print(index, value) test to print out the index and each value in the nested list
Pass the argument start to enumerate, enumerate(table, 1) in your case,
table = [['first_name', 'last_name', 'age', 'years experience', 'salary'],
['James', 'Butt', '29', '8', '887174.4'],
['Josephine', 'Darakjy', '59', '39', '1051267.9'],
['Art', 'Venere', '22', '2', '47104.2']]
table[0].extend(['Started Working', 'Salary/Experience'])
for idx, row in enumerate(table[1:], 1):
start_work_age = int(row[2]) - int(row[3])
ratio = float(row[4]) / int(row[3])
table[idx].extend([str(start_work_age), str(ratio)])
print(table)
# Output
[['first_name', 'last_name', 'age', 'years experience', 'salary', 'Started Working', 'Salary/Experience'],
['James', 'Butt', '29', '8', '887174.4', '21', '110896.8'],
['Josephine', 'Darakjy', '59', '39', '1051267.9', '20', '26955.5871795'],
['Art', 'Venere', '22', '2', '47104.2', '20', '23552.1']]
If you can convert the space to an underscore in years experience you can use collections.namedtuple to make your life simpler:
from collections import namedtuple
table = [
['first_name', 'last_name', 'age', 'years_experience', 'salary'],
['James', 'Butt', '29', '8', '887174.4'],
['Josephine', 'Darakjy', '59', '39', '1051267.9'],
['Art', 'Venere', '22', '2', '47104.2'],
# ...
]
workerv1 = namedtuple('workerv1', ','.join(table[0]))
for i,v in enumerate(table):
worker = workerv1(*v)
if i == 0:
swage = 'Started Working'
sex_ratio = 'S/Ex ratio'
else:
swage = int(worker.age) - int(worker.years_experience)
sex_ratio = float(worker.salary) / float(worker.years_experience)
print("{w.first_name},{w.last_name},{w.age},{w.years_experience},{w.salary},{0},{1}".format(
swage, sex_ratio, w=worker))
I have this list of dicts that I'm maintaining as a master list:
orig_list = [
{ 'cpu': '4', 'mem': '4', 'name': 'server1', 'drives': '4', 'nics': '1' }
{ 'cpu': '1', 'mem': '2', 'name': 'server2', 'drives': '2', 'nics': '2' }
{ 'cpu': '2', 'mem': '8', 'name': 'server3', 'drives': '1', 'nics': '1' }
]
However, I need to perform actions on things inside this list of dicts, like:
def modifyVM(local_list)
local_temp_list = []
for item in local_list :
'''
Tons of VM processy things happen here.
'''
item['cpu'] = 4
item['notes'] = 'updated cpu'
local_temp_list.append(item)
return local_temp_list
temp_list []
for item in orig_list :
if item['cpu'] < 4
temp_list.append(item)
result_list = modifyVM(temp_list)
At this point, result_list contains:
result_list = [
{ 'cpu': '4', 'mem': '2', 'name': 'server2', 'drives': '2', 'nics': '2' }
{ 'cpu': '4', 'mem': '8', 'name': 'server3', 'drives': '1', 'nics': '1' }
]
So my questions are:
1) What is the most efficient way to update orig_list with the results of result_list? I'm hoping to end up with:
orig_list = [
{ 'cpu': '4', 'mem': '4', 'name': 'server1', 'drives': '4', 'nics': '1' }
{ 'cpu': '4', 'mem': '2', 'name': 'server2', 'drives': '2', 'nics': '2' 'notes': 'updated cpu' }
{ 'cpu': '4', 'mem': '8', 'name': 'server3', 'drives': '1', 'nics': '1' 'notes': 'updated cpu' }
]
2) Is there a way to update orig_list without ever creating secondary lists?
Thank you in advance.
Collections store references to the objects.
So the code you posted is already modifying the items in "orig_list" as well, cause all the lists are storing references to the same original dictionaries.
As for the second part of your question, you don't need to create a new list. You can modify the objects directly, and next time you iterate the list you'll see the updated values.
Like for example:
orig_list = [
{ 'cpu': 4, 'mem': '4', 'name': 'server1', 'drives': '4', 'nics': '1' },
{ 'cpu': 1, 'mem': '2', 'name': 'server2', 'drives': '2', 'nics': '2' },
{ 'cpu': 2, 'mem': '8', 'name': 'server3', 'drives': '1', 'nics': '1' }
]
print orig_list
for item in orig_list :
if item['cpu'] < 4:
item['cpu'] = 4
print orig_list
Output of first print:
[{'mem': '4', 'nics': '1', 'drives': '4', 'cpu': 4, 'name': 'server1'},
{'mem': '2', 'nics': '2', 'drives': '2', 'cpu': 1, 'name': 'server2'},
{'mem': '8', 'nics': '1', 'drives': '1', 'cpu': 2, 'name': 'server3'}]
And second print:
[{'mem': '4', 'nics': '1', 'drives': '4', 'cpu': 4, 'name': 'server1'},
{'mem': '2', 'nics': '2', 'drives': '2', 'cpu': 4, 'name': 'server2'},
{'mem': '8', 'nics': '1', 'drives': '1', 'cpu': 4, 'name': 'server3'}]
No, you don't need to create a separate list, just use list comprehension.
Just iterate through the list and check if value of cpu key is less than 4. If value is less than 4, then update value the cpu key to 4 and add an extra key notes having value as 'updated_cpu'. Value of orig_list after iteration finishes is the desired result.
>>> orig_list = [{'cpu': 4, 'drives': '4', 'mem': '4', 'name': 'server1', 'nics': '1'},
{'cpu': 1, 'drives': '2', 'mem': '2', 'name': 'server2', 'nics': '2'},
{'cpu': 2, 'drives': '1', 'mem': '8', 'name': 'server3', 'nics': '1'}]
>>> for item in orig_list:
if item['cpu']<4:
item['cpu']=4
item['notes'] = 'updated cpu'
>>> orig_list
[{'cpu': 4, 'drives': '4', 'mem': '4', 'name': 'server1', 'nics': '1'},
{'cpu': 4, 'drives': '2', 'mem': '2', 'name': 'server2', 'nics': '2', 'notes': 'updated cpu'},
{'cpu': 4, 'drives': '1', 'mem': '8', 'name': 'server3', 'nics': '1', 'notes': 'updated cpu'}]
Thank you for all the input! I flagged eugenioy's post as the answer because he posted first. Both the answer from him and from Rahul Gupta are very efficient ways to update a list of dictionaries.
However, I kept trying other ways because these answers, as efficient as they are, still do one other thing I've always been told is taboo: modifying the list you're iterating over.
Keep in mind, that I'm still learning Python. So if some of my "revelations" here are mundain, they are new and "wow" to me. To that effect, I'm adding the answer that I actually ended up implementing.
Here's the finished code:
def modifyVM(local_list, l_orig_list)
for item in local_list[:] :
l_orig_list.remove(item)
'''
Tons of VM processy things happen here.
'''
item['cpu'] = 4
item['notes'] = 'updated cpu'
l_orig_list.append(item)
temp_list []
for item in orig_list[:] :
if item['cpu'] < 4
temp_list.append(item)
modifyVM(temp_list, orig_list)
I change this line:
def modifyVM(local_list)
To this line:
def modifyVM(local_list, l_orig_list)
In this way, I'm passing in both the list I want to use as well as the list I want to update.
Next I changed:
for item in local_list :
To this line:
for item in local_list[:] :
This causes "item" to iterate through a slice (copy) of "local_list" that contains everything.
I also added:
l_orig_list.remove(item)
And:
l_orig_list.append(item)
This solved several problems for me.
1) This avoids the potential of modifying any list that's being iterated through.
2) This allows "orig_list" to be updated as processes are happening, which cuts down on the "secondary lists" that are created and maintained.
3) The "orig_list" that's passed into the function and "l_orig_list" are linked variables until a hard assignment (i.e. l_orig_list = 'anything') is made. (Again, thank you to everyone that answered! This was some great "secret sauce" learning for me, and y'all pointed it out.) So, avoiding the "=", I'm able to update "l_orig_list" and have "orig_list" updated as well.
4) This also allows the movement of items from one list to another if needed (i.e. list items that generate errors can be removed from "orig_list" and placed in any other list, like "bad_list" for example.
In closing, I'd like to give recognition to Steven Rumbalski. When I read your comment, I was like, "Of course!!!" However, I spent 2 days on it before realizing that dictionaries cannot be sorted. I had to narrow down the technical problem I was facing to ask a question here. Sorting was an unstated requirement for other parts of the script. So GREAT suggestion, and I'll probably use that for another script.
given in a string the following information:
[:T102684-1 coord="107,20,885,18":]27.[:/T102684-1:] [:T102684-2
coord="140,16,885,18":]A.[:/T102684-2:] [:T102684-3
coord="162,57,885,18":]Francke[:/T102684-3:][:T102684-4
coord="228,5,885,18":]:[:/T102684-4:] [:T102684-5
coord="240,27,885,18":]Die[:/T102684-5:] [:T102684-6
coord="274,42,885,18":]alpine[:/T102684-6:] [:T102684-7
coord="325,64,885,18":]Literatur[:/T102684-7:] [:T102684-8
coord="398,25,885,18":]des[:/T102684-8:] [:T102684-9
coord="427,46,885,18":]Jahres[:/T102684-9:] [:T102684-10
coord="480,33,885,18":]1888[:/T102684-10:] [:T102684-11
coord="527,29,885,18":]475[:/T102684-11:]
How can I extract the Tab-ID (here: T102684), the Token-ID (the number after the "-"), the coordinates (107,20,885,18) and the token itself ("27.") ?
I used simple find-methods, but it doesn't work...
for tok in ele.text.split():
print tok.find("[:T")
print tok.rfind(":]")
print tok[(tok.find("[:T")+2):tok.rfind("-")]
Thanks for any help!
You can use regex for this:
>>> import re
>>> s = '[:T102684-1 coord="107,20,885,18":]27.[:/T102684-1:] [:T102684-2 coord="140,16,885,18":]A.[:/T102684-2:] [:T102684-3 coord="162,57,885,18":]Francke[:/T102684-3:][:T102684-4 coord="228,5,885,18":]:[:/T102684-4:] [:T102684-5 coord="240,27,885,18":]Die[:/T102684-5:] [:T102684-6 coord="274,42,885,18":]alpine[:/T102684-6:] [:T102684-7 coord="325,64,885,18":]Literatur[:/T102684-7:] [:T102684-8 coord="398,25,885,18":]des[:/T102684-8:] [:T102684-9 coord="427,46,885,18":]Jahres[:/T102684-9:] [:T102684-10 coord="480,33,885,18":]1888[:/T102684-10:] [:T102684-11 coord="527,29,885,18":]475[:/T102684-11:]'
>>> r = re.compile(r'''\[:/?T(?P<token_id>\d+)-(?P<id>\d+)\s+coord="
(?P<coord>(\d+,\d+,\d+,\d+))":\](?P<token>\w+)''', flags=re.VERBOSE)
>>> for m in r.finditer(s):
print m.groupdict()
{'token_id': '102684', 'token': '27', 'id': '1', 'coord': '107,20,885,18'}
{'token_id': '102684', 'token': 'A', 'id': '2', 'coord': '140,16,885,18'}
{'token_id': '102684', 'token': 'Francke', 'id': '3', 'coord': '162,57,885,18'}
{'token_id': '102684', 'token': 'Die', 'id': '5', 'coord': '240,27,885,18'}
{'token_id': '102684', 'token': 'alpine', 'id': '6', 'coord': '274,42,885,18'}
{'token_id': '102684', 'token': 'Literatur', 'id': '7', 'coord': '325,64,885,18'}
{'token_id': '102684', 'token': 'des', 'id': '8', 'coord': '398,25,885,18'}
{'token_id': '102684', 'token': 'Jahres', 'id': '9', 'coord': '427,46,885,18'}
{'token_id': '102684', 'token': '1888', 'id': '10', 'coord': '480,33,885,18'}
{'token_id': '102684', 'token': '475', 'id': '11', 'coord': '527,29,885,18'}