I have face this weird behavior I can not find explications about.
MWE:
l = [1]
l += {'a': 2}
l
[1, 'a']
l + {'B': 3}
Traceback (most recent call last):
File "<input>", line 1, in <module>
TypeError: can only concatenate list (not "dict") to list
Basically, when I += python does not raise an error and append the key to the list while when I only compute the + I get the expected TypeError.
Note: this is Python 3.6.10
l += ... is actually calling object.__iadd__(self, other) and modifies the object in-place when l is mutable
The reason (as #DeepSpace explains in his comment) is that when you do l += {'a': 2} the operation updates l in place only and only if l is mutable. On the other hand, the operation l + {'a': 2} is not done in place resulting into list + dictionary -> TypeError.
(see here)
l = [1]
l = l.__iadd__({'a': 2})
l
#[1, 'a']
is not the same as + that calls object.__add__(self, other)
l + {'B': 3}
TypeError: can only concatenate list (not "dict") to list
So as the authors say this is not a bug. When you Do a += b it is like b come to a's house and changing it the way that a like it to be. what the Authors say is when you do a + b it cannot be decided which one's style will get prioritized. and no one knows where will the result of a + b will go until you execute it. So you can't decide whose style it would be. if it is a style it would be [1, 'a']'s, and if it is b style it would be an error. and therefore it cannot be decided who will get the priority. So I don't personally agree with that statement. because when you take the call stack a is in a higher place than b. when there is a expression like a + b you first call a.__add__(self, other) if a.__add__ is NotImplemented (in this case it is implemented). then you call a.__radd__(self, other). which means call other.__add__ in this case b.__add__. I am telling this based on the place of the call stack and the python community may have more important reasons to do this.
Related
There are dozens of questions on turning a Python dict with some number of elements into a list of tuples. I am looking for a shortcut to turn a one element dict into a tuple.
Ideally, it would:
Be idiomatic and simple (ie, not a subclass of dict, not a function, etc).
Throw an error if there are more that one element in the dict.
Support multiple assignment by tuple unpacking.
Not destroy the dict.
I am looking for a shortcut to do this (that is not destructive to the dict):
k,v=unpack_this({'unknown_key':'some value'})
* does not work here.
I have come up with these that work:
k,v=next(iter(di.items())) # have to call 'iter' since 'dict_items' is not
Or:
k,v=(next(((k,v) for k,v in di.items())))
Or even:
k,v=next(zip(di.keys(), di.values()))
Finally, the best I can come up with:
k,v=list(di.items())[0] # probably the best...
Which can be wrapped into a function if I want a length check:
def f(di):
if (len(di)==1): return list(di.items())[0]
raise ValueError(f'Too many items to unpack. Expected 2, got {len(di)*2}')
These methods seem super clumsy and none throw an error if there is more than one element.
Is there an idiomatic shortcut that I am missing?
Why not :
next(iter(d.items())) if len(d)==1 else (None,None)
>>> d = {'a': 1}
>>> d.popitem()
('a', 1)
This will return the pair associated with the last key added, with no error error if the dict has multiple keys, but the same length check you've made in your function f can be used.
def f(di):
if len(di) == 1:
return d.popitem()
raise ValueError(f'Dict has multiple keys')
#chepner has the right approach with .popitem():
>>> d = {'a': 1}
>>> d.popitem()
('a', 1) # d is now {}
This will return the pair associated with the last key added, with no error error if the dict has multiple keys. It will also destroy d.
You can keep d intact by using this idiom to create a new dict and pop off the last item from the new dict created with {**d}:
>>> {**d}.popitem()
('a', 1)
>>> d
{'a': 1}
As far as a 1 line unpackable tuple that throws an error if the dict has more than one item, you can use a Python conditional expression with an alternate that throws the error desired:
# success: k=='a' and v==1
>>> d={'a':1}
>>> k,v={**d}.popitem() if len(d)==1 else 'wrong number of values'
# len(d)!=1 - pass a string to k,v which is too long
>>> d={'a':1,'b':2}
>>> k,v={**d}.popitem() if len(d)==1 else 'wrong number of values'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: too many values to unpack (expected 2)
If you object to different types being passed, you could do:
>>> k,v={**d}.popitem() if len(d)==1 else {}.fromkeys('too many keys').items()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: too many values to unpack (expected 2)
I have face this weird behavior I can not find explications about.
MWE:
l = [1]
l += {'a': 2}
l
[1, 'a']
l + {'B': 3}
Traceback (most recent call last):
File "<input>", line 1, in <module>
TypeError: can only concatenate list (not "dict") to list
Basically, when I += python does not raise an error and append the key to the list while when I only compute the + I get the expected TypeError.
Note: this is Python 3.6.10
l += ... is actually calling object.__iadd__(self, other) and modifies the object in-place when l is mutable
The reason (as #DeepSpace explains in his comment) is that when you do l += {'a': 2} the operation updates l in place only and only if l is mutable. On the other hand, the operation l + {'a': 2} is not done in place resulting into list + dictionary -> TypeError.
(see here)
l = [1]
l = l.__iadd__({'a': 2})
l
#[1, 'a']
is not the same as + that calls object.__add__(self, other)
l + {'B': 3}
TypeError: can only concatenate list (not "dict") to list
So as the authors say this is not a bug. When you Do a += b it is like b come to a's house and changing it the way that a like it to be. what the Authors say is when you do a + b it cannot be decided which one's style will get prioritized. and no one knows where will the result of a + b will go until you execute it. So you can't decide whose style it would be. if it is a style it would be [1, 'a']'s, and if it is b style it would be an error. and therefore it cannot be decided who will get the priority. So I don't personally agree with that statement. because when you take the call stack a is in a higher place than b. when there is a expression like a + b you first call a.__add__(self, other) if a.__add__ is NotImplemented (in this case it is implemented). then you call a.__radd__(self, other). which means call other.__add__ in this case b.__add__. I am telling this based on the place of the call stack and the python community may have more important reasons to do this.
I'm trying to improve the computing time of my code so I want to replace for loops with map functions.
For each key in the dictionary I check if it is bigger than a specific value and inserting it to a new dictionary under the same key:
My original code is:
dict1={'a':-1,'b':0,'c':1,'d':2,'e':3}
dict_filt = {}
for key in dict1.keys():
if dict1[key]>1:
dict_filt[key] = dict1[key]*10
print (dict_filt)
output is: {'d': 20, 'e': 30}
and this works
but when I try with map:
dict1={'a':-1,'b':0,'c':1,'d':2,'e':3}
dict_filt = {}
def for_filter (key):
if dict1[key]>1:
dict_filt[key] = dict1[key]*10
map (for_filter ,dict1.keys())
print (dict_filt)
I get an empty dictionary
I tried to make it work with lambda:
map (lambda x: for_filter(x) ,dict1.keys())
or define the dictionarys as global but it still doesnt work.
I'll be glad to get some help
I don't need the original dictionary so if it's simpler to work on one dictionary it's still ok
Use a dictionary-comprehension instead of map:
{k: v * 10 for k, v in dict1.items() if v > 1}
Code:
dict1 = {'a':-1,'b':0,'c':1,'d':2,'e':3}
print({k: v * 10 for k, v in dict1.items() if v > 1})
# {'d': 20, 'e': 30}
map is lazy: if you do not consume the values, the function for_filter is not applied. Since you are using a side effect to populate dict_filt, nothing will happen unless you force the evaluation of the map:
Replace:
map(for_filter, dict1.keys())
By:
list(map(for_filter, dict1)) # you don't need keys here
And you will get the expected result.
But note that this is a misuse of map. You should use a dict comprehension (see #Austin's answer).
EDIT: More on map and lazyness.
TLDR;
Look at the doc:
map(function, iterable, ...)
Return an iterator that applies function to every item of iterable, yielding the results.
Explanation
Consider the following function:
>>> def f(x):
... print("x =", x)
... return x
...
This function returns its parameter and performs a side effect (print the value). Let's try to apply this function to a simple range with the map function:
>>> m = map(f, range(5))
Nothing is printed! Let's look at the value of m:
>>> m
<map object at 0x7f91d35cccc0>
We were expecting [0, 1, 2, 3, 4] but we got a strange <map object at 0x7f91d35cccc0>. That's lazyness: map does not really apply the function but creates an iterator. This iterator returns, on each next call, a value:
>>> next(m)
x = 0
0
That value is the result of the application of the function f to the next element of the mapped iterable (range). Here, 0 is the returned value and x = 0 the result of the print side effect. What is important here is that this value does not exist before you pull it out of the iterator. Hence the side effect is not performed before you pull the vlaue out of the iterator.
If we continue to call next, we'll exhaust the iterator:
...
>>> next(m)
x = 4
4
>>> next(m)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration
Another way to get all the values of the iterator is to create a list. That's not a cast, but rather the constrution of a new object and the consumption of the old one:
>>> m = map(f, range(5))
>>> list(m)
x = 0
x = 1
x = 2
x = 3
x = 4
[0, 1, 2, 3, 4]
We see that the side effect print is performed for every element of the range, and then the list [0, 1, 2, 3, 4] is returned.
In your case, the function doesn't print anything, but makes an assignement to an external variable dict_filt. The function is not applied unless you consume the map iterator.
I repeat: do not use map (or any list/dict comprehension) to perform a side effect (map comes from the functional world where side effect do not exist).
This question already has answers here:
How do I pass a variable by reference?
(39 answers)
Closed 8 months ago.
I am not sure I understand the concept of Python's call by object style of passing function arguments (explained here http://effbot.org/zone/call-by-object.htm). There don't seem to be enough examples to clarify this concept well (or my google-fu is probably weak! :D)
I wrote this little contrived Python program to try to understand this concept
def foo( itnumber, ittuple, itlist, itdict ):
itnumber +=1
print id(itnumber) , itnumber
print id(ittuple) , ittuple
itlist.append(3.4)
print id(itlist) , itlist
itdict['mary'] = 2.3
print id(itdict), itdict
# Initialize a number, a tuple, a list and a dictionary
tnumber = 1
print id( tnumber ), tnumber
ttuple = (1, 2, 3)
print id( ttuple ) , ttuple
tlist = [1, 2, 3]
print id( tlist ) , tlist
tdict = tel = {'jack': 4098, 'sape': 4139}
print '-------'
# Invoke a function and test it
foo(tnumber, ttuple, tlist , tdict)
print '-------'
#Test behaviour after the function call is over
print id(tnumber) , tnumber
print id(ttuple) , ttuple
print id(tlist) , tlist
print id(tdict), tdict
The output of the program is
146739376 1
3075201660 (1, 2, 3)
3075103916 [1, 2, 3]
3075193004 {'sape': 4139, 'jack': 4098}
---------
146739364 2
3075201660 (1, 2, 3)
3075103916 [1, 2, 3, 3.4]
3075193004 {'sape': 4139, 'jack': 4098, 'mary': 2.3}
---------
146739376 1
3075201660 (1, 2, 3)
3075103916 [1, 2, 3, 3.4]
3075193004 {'sape': 4139, 'jack': 4098, 'mary': 2.3}
As you can see , except for the integer that was passed, the object id's (which as I understand refers to memeory location) remain unchanged.
So in the case of the integer, it was (effectively) passed by value and the other data structure were (effectively) passed by reference. I tried changing the list , the number and the dictionary to just test if the data-structures were changed in place. The number was not bu the list and the
dictionary were.
I use the word effectively above, since the 'call-by-object' style of argument passing seems to behave both ways depending on the data-structure passed in the above code
For more complicated data structures, (say numpy arrays etc), is there any quick rule of thumb to
recognize which arguments will be passed by reference and which ones passed by value?
The key difference is that in C-style language, a variable is a box in memory in which you put stuff. In Python, a variable is a name.
Python is neither call-by-reference nor call-by-value. It's something much more sensible! (In fact, I learned Python before I learned the more common languages, so call-by-value and call-by-reference seem very strange to me.)
In Python, there are things and there are names. Lists, integers, strings, and custom objects are all things. x, y, and z are names. Writing
x = []
means "construct a new thing [] and give it the name x". Writing
x = []
foo = lambda x: x.append(None)
foo(x)
means "construct a new thing [] with name x, construct a new function (which is another thing) with name foo, and call foo on the thing with name x". Now foo just appends None to whatever it received, so this reduces to "append None to the the empty list". Writing
x = 0
def foo(x):
x += 1
foo(x)
means "construct a new thing 0 with name x, construct a new function foo, and call foo on x". Inside foo, the assignment just says "rename x to 1 plus what it used to be", but that doesn't change the thing 0.
Others have already posted good answers. One more thing that I think will help:
x = expr
evaluates expr and binds x to the result. On the other hand:
x.operate()
does something to x and hence can change it (resulting in the same underlying object having a different value).
The funny cases come in with things like:
x += expr
which translate into either x = x + expr (rebinding) or x.__iadd__(expr) (modifying), sometimes in very peculiar ways:
>>> x = 1
>>> x += 2
>>> x
3
(so x was rebound, since integers are immutable)
>>> x = ([1], 2)
>>> x
([1], 2)
>>> x[0] += [3]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment
>>> x
([1, 3], 2)
Here x[0], which is itself mutable, was mutated in-place; but then Python also attempted to mutate x itself (as with x.__iadd__), which errored-out because tuples are immutable. But by then x[0] was already mutated!
Numbers, strings, and tuples in Python are immutable; using augmented assignment will rebind the name.
Your other types are merely mutated, and remain the same object.
What's going on here?
>>> a = {1: "a", 2: "b"}
>>> del a[1]
>>> a
{2: 'b'}
>>> a = {1: "a", 2: "b"}
>>> del a[:]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unhashable type
>>> a.clear()
>>> a
{}
Why must I call dict.clear?
a[:] is a special case of a slicing operation, which is defined only for sequences. It is a short form of a[0:len(a)]. A dict is not a sequence.
a[:] is a quick way to make a shallow copy of a list (and tuple). See towards the bottom of the docs for clarification on different types of copying.
Thus, it would reason to say that del a[:] on a dictionary doesn't really make much sense.
If you want to delete the entire dictionary, you can simply do del a
>>> a = {1: "a", 2: "b"}
>>> del a
>>> a
Traceback (most recent call last):
File "<pyshell#8>", line 1, in <module>
a
NameError: name 'a' is not defined
>>>
The reason is rather simple: It isn't defined. There's no reason why it couldn't work, if someone put the effort into catching the empty slice and delegating to the appropriate method.
On the other hand, this would violate a symmetry principle with assigning and getting the same slice, so probably would not gain acceptance.
If this functionality is crucial to your system, you could subclass the dict builtin to add this function.
import sys
class newdict(dict):
def __delslice__(self,i,j):
if i==0 and j==sys.maxint:
self.clear()
else:
dict.__delslice__(self,i,j)
When you're doing del a[1] you're not deleting the first element in dictionary, you're deleting the element with key 1. Dictionary does not have any ordering and thus slicing algorigthm (in this case at least).
That's why with the dict a = {1 : 'one', 5 : 'five'} you can do del a[1] and cannot do del a[2] or del a[0]