Iterate over keys and all values in MultiDict - python

I have a dictionary
params = ImmutableMultiDict([('dataStore', 'tardis'), ('symbol', '1'), ('symbol', '2')])
I want to be able to iterate through the dictionary and get a list of all the values and their keys. However, when I try to do it, it only gets the first symbol key value pair and ignores the other one.
for k in params:
print(params.get(k))

If I understand you correctly you want to iterate over all keys, including duplicates, right? Then you could use the items(multi=False) method with multi set to True.
Documentation:
items(multi=False)
Return an iterator of (key, value) pairs.
Parameters: multi – If set to True the iterator returned will have a pair for each value of each key. Otherwise it will only contain
pairs for the first value of each key.
If I misunderstood you and you want a list of all entries to a single key have a look at jonrsharpe's answer.

If you read the docs for MultiDict, from which ImmutableMultiDict is derived, you can see:
It behaves like a normal dict thus all dict functions will only return the first value when multiple values for one key are found.
However, the API includes an additional method, .getlist, for this purpose. There's an example of its use in the docs, too:
>>> d = MultiDict([('a', 'b'), ('a', 'c')])
# ...
>>> d.getlist('a')
['b', 'c']

Related

Why do we need a dict.update() method in python instead of just assigning the values to the corresponding keys?

I have been working with dictionaries that I have to modify within different parts of my code. I am trying to make sure if I do not miss anything about there is no need for dict_update() in any scenario.
So the reasons to use update() method is either to add a new key-value pair to current dictionary, or update the value of your existing ones.
But wait!?
Aren't they already possible by just doing:
>>>test_dict = {'1':11,'2':1445}
>>>test_dict['1'] = 645
>>>test_dict
{'1': 645, '2': 1445}
>>>test_dict[5]=123
>>>test_dict
{'1': 645, '2': 1445, 5: 123}
In what case it would be crucial to use it ? I am curious.
Many thanks
1. You can update many keys on the same statement.
my_dict.update(other_dict)
In this case you don't have to know how many keys are in the other_dict. You'll just be sure that all of them will be updated on my_dict.
2. You can use any iterable of key/value pairs with dict.update
As per the documentation you can use another dictionary, kwargs, list of tuples, or even generators that yield tuples of len 2.
3. You can use the update method as an argument for functions that expect a function argument.
Example:
def update_value(key, value, update_function):
update_function([(key, value)])
update_value("k", 3, update_on_the_db) # suppose you have a update_on_the_db function
update_value("k", 3, my_dict.update) # this will update on the dict
d.update(n) is basically an efficient implementation of the loop
for key, value in n.items():
d[key] = value
But syntactically, it also lets you specify explicit key-value pairs without building a dict, either using keyword argumetns
d.update(a=1, b=2)
or an iterable of pairs:
d.update([('a', 1), ('b', 2)])

Iterating over dictionaries within dictionaries, dictionary object turning into string?

test = {'a':{'aa':'value','ab':'value'},'b':{'aa':'value','ab':'value'}}
#test 1
for x in test:
print(x['aa'])
#test 2
for x in test:
print(test[x]['aa'])
Why does test 1 give me a TypeError: string indices must be integers but test 2 pass?
Does the for loop turn the dictionary into a string?
If you iterate over a dictionary, you iterate over the keys. So that means in the first loop, x = 'a', and in the second x = 'b' (or vice versa, since dictionaries are unordered). It thus simply "ignores" the values. It makes no sense to index a string with a string (well there is no straightforward interpretation for 'a'['aa'], or at least not really one I can come up with that would be "logical" for a signifcant number of programmers).
Although this may look quite strange, it is quite consistent with the fact that a membership check for example also works on the keys (if we write 'a' in some_dict, it does not look to the values either).
If you want to use the values, you need to iterate over .values(), so:
for x in test.values():
print(x['aa'])
If you however use your second thest, then this works, since then x is a key (for example 'a'), and hence test[x] will fetch you the corresponding value. If you then process test[x] further, you thus process the values of the dictionary.
You can iterate concurrently over keys and values with .items():
for k, x in test.items():
# ...
pass
Here in the first iteration k will be 'a' and x will be {'aa':'value','ab':'value'}, in the second iteration k will be 'b' and x will be {'aa':'value','ab':'value'} (again the iterations can be swapped, since dictionaries are unordered).
If you thus are interested in the outer key, and the value that is associated with the 'aa' key of the corresponding subdictionary, you can use:
for k, x in test.items():
v = x['aa']
print(k, v)
When you iterate over a dictionary with a for, you're not iterating over the items, but over the keys ('a', 'b'). These are just strings that mean nothing. That's why you have to do it as on test 2. You could also iterate over the items with test.items().

How does this function to remove duplicate characters from a string in python work?

I was looking up how to create a function that removes duplicate characters from a string in python and found this on stack overflow:
from collections import OrderedDict
def remove_duplicates (foo) :
print " ".join(OrderedDict.fromkeys(foo))
It works, but how? I've searched what OrderedDict and fromkeys mean but I can't find anything that explains how it works in this context.
I will give it a shot:
OrderedDict are dictionaries that store keys in order they are added. Normal dictionaries don't. If you look at doc of fromkeys, you find:
OD.fromkeys(S[, v]) -> New ordered dictionary with keys from S.
So the fromkeys class method, creates an OrderedDict using items in the input iterable S (in my example characters from a string) as keys. In a dictionary, keys are unique, so duplicate items in S are ignored.
For example:
s = "abbcdece" # example string with duplicate characters
print(OrderedDict.fromkeys(s))
This results in an OrderedDict:
OrderedDict([('a', None), ('b', None), ('c', None), ('d', None), ('e', None)])
Then " ".join(some_iterable) takes an iterable and joins its elements using a space in this case. It uses only keys, as iterating through a dictionary is done by its keys. For example:
for k in OrderedDict.fromkeys(s): # k is a key of the OrderedDict
print(k)
Results in:
a
b
c
d
e
Subsequently, call to join:
print(" ".join(OrderedDict.fromkeys(s)))
will print out:
a b c d e
Using set
Sometimes, people use a set for this:
print( " ".join(set(s)))
# c a b d e
But unlike sets in C++, sets in python do not guarantee order. So using a set will give you unique values easily, but they might be in a different order then they are in the original list or string (as in the above example).
Hope this helps a bit.
By list comprehension
print ' '.join([character for index, character in enumerate(foo) if character not in foo[:index]])

Right way to initialize an OrderedDict using its constructor such that it retains order of initial data?

What's the correct way to initialize an ordered dictionary (OD) so that it retains the order of initial data?
from collections import OrderedDict
# Obviously wrong because regular dict loses order
d = OrderedDict({'b':2, 'a':1})
# An OD is represented by a list of tuples, so would this work?
d = OrderedDict([('b',2), ('a', 1)])
# What about using a list comprehension, will 'd' preserve the order of 'l'
l = ['b', 'a', 'c', 'aa']
d = OrderedDict([(i,i) for i in l])
Question:
Will an OrderedDict preserve the order of a list of tuples, or tuple of tuples or tuple of lists or list of lists etc. passed at the time of initialization (2nd & 3rd example above)?
How does one go about verifying if OrderedDict actually maintains an order? Since a dict has an unpredictable order, what if my test vectors luckily have the same initial order as the unpredictable order of a dict? For example, if instead of d = OrderedDict({'b':2, 'a':1}) I write d = OrderedDict({'a':1, 'b':2}), I can wrongly conclude that the order is preserved. In this case, I found out that a dict is ordered alphabetically, but that may not be always true. What's a reliable way to use a counterexample to verify whether a data structure preserves order or not, short of trying test vectors repeatedly until one breaks?
P.S. I'll just leave this here for reference: "The OrderedDict constructor and update() method both accept keyword arguments, but their order is lost because Python’s function call semantics pass-in keyword arguments using a regular unordered dictionary"
P.P.S : Hopefully, in future, OrderedDict will preserve the order of kwargs also (example 1): http://bugs.python.org/issue16991
The OrderedDict will preserve any order that it has access to. The only way to pass ordered data to it to initialize is to pass a list (or, more generally, an iterable) of key-value pairs, as in your last two examples. As the documentation you linked to says, the OrderedDict does not have access to any order when you pass in keyword arguments or a dict argument, since any order there is removed before the OrderedDict constructor sees it.
Note that using a list comprehension in your last example doesn't change anything. There's no difference between OrderedDict([(i,i) for i in l]) and OrderedDict([('b', 'b'), ('a', 'a'), ('c', 'c'), ('aa', 'aa')]). The list comprehension is evaluated and creates the list and it is passed in; OrderedDict knows nothing about how it was created.
# An OD is represented by a list of tuples, so would this work?
d = OrderedDict([('b', 2), ('a', 1)])
Yes, that will work. By definition, a list is always ordered the way it is represented. This goes for list-comprehension too, the list generated is in the same way the data was provided (i.e. source from a list it will be deterministic, sourced from a set or dict not so much).
How does one go about verifying if OrderedDict actually maintains an order. Since a dict has an unpredictable order, what if my test vectors luckily has the same initial order as the unpredictable order of a dict?. For example, if instead of d = OrderedDict({'b':2, 'a':1}) I write d = OrderedDict({'a':1, 'b':2}), I can wrongly conclude that the order is preserved. In this case, I found out that a dict is order alphabetically, but that may not be always true. i.e. what's a reliable way to use a counter example to verify if a data structure preserves order or not short of trying test vectors repeatedly until one breaks.
You keep your source list of 2-tuple around for reference, and use that as your test data for your test cases when you do unit tests. Iterate through them and ensure the order is maintained.
It is also possible (and a little more efficient) to use a generator expression:
d = OrderedDict((i, i) for i in l)
Obviously, the benefit is negligible in this trivial case for l, but if l corresponds to an iterator or was yielding results from a generator, e.g. used to parse and iterate through a large file, then the difference could be very substantial (e.g. avoiding to load the entire contents onto memory). For example:
def mygen(filepath):
with open(filepath, 'r') as f:
for line in f:
yield [int(field) for field line.split()]
d = OrderedDict((i, sum(numbers)) for i, numbers in enumerate(mygen(filepath)))

sorting list of tuples by arbitrary key

order = ['w','x','a','z']
[(object,'a'),(object,'x'),(object,'z'),(object,'a'),(object,'w')]
How do I sort the above list of tuples by the second element according the the key list provided by 'order'?
UPDATE on 11/18/13:
I found a much better approach to a variation of this question where the keys are certain to be unique, detailed in this question: Python: using a dict to speed sorting of a list of tuples.
My above question doesn't quite apply because the give list of tuples has two tuples with the key value of 'a'.
You can use sorted, and give as the key a function that returns the index of the second value of each tuple in the order list.
>>> sorted(mylist,key=lambda x: order.index(x[1]))
[('object', 'w'), ('object', 'x'), ('object', 'a'), ('object', 'a'), ('object', 'z')]
Beware, this fails whenever a value from the tuples is not present within the order list.
Edit:
In order to be a little more secure, you could use :
sorted(mylist,key=lambda x: x[1] in order and order.index(x[1]) or len(order)+1)
This will put all entries with a key that is missing from order list at the end of the resulting list.

Categories