When I execute the following:
a = {'a': 1, 'b':3, 'c': 5}
b = [2,4,6]
c = list(b)
c.extend(a.values())
print c
It prints out [2, 4, 6, 1, 5, 3] as I expected, but when I try to do the list copy and extension in one line:
a = {'a': 1, 'b':3, 'c': 5}
b = [2,4,6]
d = list(b).extend(a.values())
print d
It prints None. Why are these different?
Because list.extend() does not return the modified list but operate on the list itself.
I guess you may want to reuse d. If you want to create a new d to hold the result after extending and do it in one line, try:
a = {'a': 1, 'b':3, 'c': 5}
b = [2,4,6]
d = list(b) + list(a.values())
print d
Two points to note:
dictionary.value() returns a view object but not a plain list
values / keys in a default Python dictionary is randomly ordered
What you assign to d variable is the result of the list(b).extend function. This method does not return anything - it extends existing mutable object. list(b) is modified but since you did not save it anywhere the statement won't have any effect.
You don't even need to make the variable d. extend() will extend the list in place so just use this:
a = {'a': 1, 'b':3, 'c': 5}
b = [2,4,6]
b.extend(a.values())
print b
This will give you the desired output: [2, 4, 6, 5, 3, 1]
EDIT:
According to the OP he wanted b to remain unchanged. All you need to do is make a copy of b and then extend that. You can do that like this:
a = {'a': 1, 'b':3, 'c': 5}
b = [2,4,6]
c = b[:]
c.extend(a.values())
print c
Related
I have a simple scenario where Python (3.7, tested also 3.5) does not seem to behave as I expect.
putting it simply:
a = [{"c":1, "d":2}]
a
[{'c': 1, 'd': 2}]
b = a + a
b
[{'c': 1, 'd': 2}, {'c': 1, 'd': 2}]
b[0]
{'c': 1, 'd': 2}
b[0]['c'] = 3
b
[{'c': 3, 'd': 2}, {'c': 3, 'd': 2}]
Changing the value of an entry in the first dictionary in b, also updates the corresponding entry in the 2nd dictionary.
I have tried b = a.copy() + a.copy() but got the same result.
Does anyone know a way around it?
You should use deepcopy
copy returns only a shallow copy so since your dictionary is inside the list copy will create new list but the dictionary inside the list will still reference the same dictionary.
shallow copy would work if you had this case:
a = {"c":1, "d":2}
b = [a.copy(), a.copy()]
But in your case you need to use deepcopy
from copy import deepcopy
b = deepcopy(a) + deepcopy(a)
Long-hand this is how it would look like:
class TestClass(object):
def f(num):
"""In general, a complicated function."""
return num
self.a = f(1)
self.b = f(2)
self.c = f(3)
self.d = f(4)
self.e = f(5)
I'm thinking dictionary methods could help, but how?
As you said you better to use a dictionary.And as a more pythonic way you can use a dictionary comprehension.You can use enumerate to create a sequence of keys for your dictionary based on your items index. :
>>> my_dict = {'a{}'.format(i):f(j) for i,j in enumerate([3,4,5,1,2])}
{'a1': 4, 'a0': 3, 'a3': 1, 'a2': 5, 'a4': 2}
And for accessing to each value you can use a simple indexing :
>>> my_dict['a3']
1
Also if you want to use custom names for your keys you can use zip function to zip the variable names with values the use if within a dict comprehension:
>>> var_names=['a','b','c','d','e']
>>> values=[1,2,3,4,5]
>>>
>>> my_dict = {i:f(j) for i,j in zip(var_names,values)}
>>> my_dict
{'a': 1, 'c': 3, 'b': 2, 'e': 5, 'd': 4}
You're going in the wrong direction - if you want to assign several references based on the same function, you should be storing them in a data structure like a list instead of in discrete, manually-entered variables. You can unpack them like that later if you want, but you should start with a data structure. It then becomes easier to map() each value in an iterable to this function, and then turn it into a list.
def f(num):
"""In general, a complicated function."""
return num
my_numbers = list(map(f, range(1, 6)))
Your numbers were a tidy range this time so I just used a range() object, but you can use any iterable you like, such as [1, 2, 3] or (4, 2, 3).
I have a list
In [4]: a = [1, 2, 3, 3, 2, 4]
from which I would like to remove duplicates via a comprehension using a sentinel list (see below why):
In [8]: [x if x not in seen else seen.append(x) for x in a]
Out[8]: [1, 2, 3, 3, 2, 4]
It seems that seen is not taken into account (neither updated, not checked). Why is it so?
As for the reason why using a convoluted method: The list I have is of the form
[{'a': 3, 'b': 4}, {'a': 10, 'b': 4}, {'a': 5, 'b': 5}]
and I want to remove duplicates based on the value of a specific key (b in the case above, to leave [{'a': 3, 'b': 4}, {'a': 5, 'b': 5}] (I do not care which dict is removed). The idea would be to build a sentinel list with the values of b and keep only the dicts without b equal to any element in that sentinel list.
Since x is not in seen, you are never adding it to seen either; the else branch is not executed when x not in seen is true.
However, you are using a conditional expression; it always produces a value; either x or the result of seen.append() (which is None), so you are not filtering, you are mapping here.
If you wanted to filter, move the test to an if section after the for loop:
seen = set()
[x for x in a if not (x in seen or seen.add(x))]
Since you were using seen.append() I presume you were using a list; I switched you to a set() instead, as membership tests are way faster using a set.
So x is excluded only if a) x in seen is true (so we have already seen it), or seen.append(x) returned a true value (None is not true). Yes, this works, if only a little convoluted.
Demo:
>>> a = [1, 2, 3, 3, 2, 4]
>>> seen = set()
>>> [x for x in a if not (x in seen or seen.add(x))]
[1, 2, 3, 4]
>>> seen
set([1, 2, 3, 4])
Applying this to your specific problem:
>>> a = [{'a': 3, 'b': 4}, {'a': 10, 'b': 4}, {'a': 5, 'b': 5}]
>>> seen = set()
>>> [entry for entry in a if not (entry['b'] in seen or seen.add(entry['b']))]
[{'a': 3, 'b': 4}, {'a': 5, 'b': 5}]
You never execute the else part of the if, because you do not update when you match the first time. You could do this:
[seen.append(x) or x for x in lst if x not in seen]
This way the or returns the last value (and executes the update using append (which always returns None, to let the or continue looking for truth-y value).
Maybe you can use the fact that dict keys are a set for this. If you want to prioritize the last items use reversed (last item is prioritized here):
>>> lst = [{'a': 3, 'b': 4}, {'a': 10, 'b': 4}, {'a': 5, 'b': 5}]
>>> filtered = {item['b']: item for item in reversed(lst)}
>>> filtered.values()
[{'a': 3, 'b': 4}, {'a': 5, 'b': 5}]
This uses 'b' as the key to map a value to, so only a single elemnt can be mapped to a value of 'b', which effectively creates a set over 'b'.
note: this will return the values in random order. To fix it nicely, for big datasets, I'd create another mapping, of each object to it's index in the original list (O(n)), and use that mapping as a sorting function of the final result (O(n*log(n))). That's beyond the scope of this answer.
I'm always queasy making use of operator precedence as execution flow control. I feel that the below is marginally more explicit and palatable, although it does carry the additional cost of tuple creation.
b_values = set()
[(item, b_values.add(item['b']))[0] for item in original_list
if item['b'] not in b_values]
But really when you're maintaining/updating some sort of state, I think the best format is the simple for-loop:
output_list = []
b_values = set()
for item in original_list:
if item['b'] not in b_values:
output_list.append(item)
b_values.add(item['b'])
I am playing about with shallow copies in python. I came across a gotcha that I was not expecting.
My assumption was that a shallow copy was a new instance of a class with references to the objects in the class. This behavior is shown below.
>>> a = { 'a': 1, 'b':2, 'c': [[1,2],2,3,4,5,6] }
>>> c = copy.copy(a['c'])
>>> d = a['c']
>>> a['c'] is c
False
>>> a['c'][0] is c[0]
True
>>> a['c'] is d
True
>>> a['c'][0] is d[0]
True
What surprised me was the following. As the elements of the shallow copied list are references to the elements in the list from a I assumed when I changed the mutable 1st element it would also change in a.
>>> c[0] = [3,3]
>>> c
[[3, 3], 2, 3, 4, 5, 6]
>>> a
{'a': 1, 'c': [[1, 2], 2, 3, 4, 5, 6], 'b': 2}
>>> a['c'][0] is c[0]
False
I see since the change the first element is no longer a reference of a.
My Question:
If I changed all elements in the list would it be similar to a deepcopy?
As the elements of the shallow copied list are references to the elements in the list from 'a' I assumed when I changed the mutable 1st element it would also change in 'a'.
The 1st element of c is a list, which is mutable. So, if you actually did mutate it, the result would be visible in a. For example:
>>> a = { 'a': 1, 'b':2, 'c': [[1],2,3,4,5,6] }
>>> c = copy.copy(a['c'])
>>> c[0].append(0)
>>> a
{'a': 1, 'b':2, 'c': [[1, 0], 2, 3, 4, 5, 6]}
But you didn't mutate it; you just replaced it with a different value.
The fact that both the original value ([1, 2]) and the new one ([3, 3]) are mutable is irrelevant; you aren't mutating anything (except for c, of course… but c is, as you already know, a shallow copy of a['c'], not the same object).
So:
If I changed all elements in the list would it be similar to a deepcopy?
No, on two counts. Changing shared elements means you're changing all references. Replacing all elements of the list would be "similar to a deepcopy"… but not the same, unless you replaced them with deepcopy-like copies of the originals. If you replace them with shallow copies, you only push the exact same issue one level down. For example:
>>> a = [[[0]]]
>>> b = copy.copy(a[0])
>>> b[0] = copy.copy(b[0])
>>> a[0] is b
False
>>> a[0][0] is b[0]
False
>>> a[0][0][0] is b[0][0]
True
(In your example, you're replacing them with entirely different and unrelated values, which isn't really like a copy at all… but I think I know what you meant.)
While reading up the documentation for dict.copy(), it says that it makes a shallow copy of the dictionary. Same goes for the book I am following (Beazley's Python Reference), which says:
The m.copy() method makes a shallow
copy of the items contained in a
mapping object and places them in a
new mapping object.
Consider this:
>>> original = dict(a=1, b=2)
>>> new = original.copy()
>>> new.update({'c': 3})
>>> original
{'a': 1, 'b': 2}
>>> new
{'a': 1, 'c': 3, 'b': 2}
So I assumed this would update the value of original (and add 'c': 3) also since I was doing a shallow copy. Like if you do it for a list:
>>> original = [1, 2, 3]
>>> new = original
>>> new.append(4)
>>> new, original
([1, 2, 3, 4], [1, 2, 3, 4])
This works as expected.
Since both are shallow copies, why is that the dict.copy() doesn't work as I expect it to? Or my understanding of shallow vs deep copying is flawed?
By "shallow copying" it means the content of the dictionary is not copied by value, but just creating a new reference.
>>> a = {1: [1,2,3]}
>>> b = a.copy()
>>> a, b
({1: [1, 2, 3]}, {1: [1, 2, 3]})
>>> a[1].append(4)
>>> a, b
({1: [1, 2, 3, 4]}, {1: [1, 2, 3, 4]})
In contrast, a deep copy will copy all contents by value.
>>> import copy
>>> c = copy.deepcopy(a)
>>> a, c
({1: [1, 2, 3, 4]}, {1: [1, 2, 3, 4]})
>>> a[1].append(5)
>>> a, c
({1: [1, 2, 3, 4, 5]}, {1: [1, 2, 3, 4]})
So:
b = a: Reference assignment, Make a and b points to the same object.
b = a.copy(): Shallow copying, a and b will become two isolated objects, but their contents still share the same reference
b = copy.deepcopy(a): Deep copying, a and b's structure and content become completely isolated.
Take this example:
original = dict(a=1, b=2, c=dict(d=4, e=5))
new = original.copy()
Now let's change a value in the 'shallow' (first) level:
new['a'] = 10
# new = {'a': 10, 'b': 2, 'c': {'d': 4, 'e': 5}}
# original = {'a': 1, 'b': 2, 'c': {'d': 4, 'e': 5}}
# no change in original, since ['a'] is an immutable integer
Now let's change a value one level deeper:
new['c']['d'] = 40
# new = {'a': 10, 'b': 2, 'c': {'d': 40, 'e': 5}}
# original = {'a': 1, 'b': 2, 'c': {'d': 40, 'e': 5}}
# new['c'] points to the same original['d'] mutable dictionary, so it will be changed
It's not a matter of deep copy or shallow copy, none of what you're doing is deep copy.
Here:
>>> new = original
you're creating a new reference to the the list/dict referenced by original.
while here:
>>> new = original.copy()
>>> # or
>>> new = list(original) # dict(original)
you're creating a new list/dict which is filled with a copy of the references of objects contained in the original container.
Adding to kennytm's answer. When you do a shallow copy parent.copy() a new dictionary is created with same keys,but the values are not copied they are referenced.If you add a new value to parent_copy it won't effect parent because parent_copy is a new dictionary not reference.
parent = {1: [1,2,3]}
parent_copy = parent.copy()
parent_reference = parent
print id(parent),id(parent_copy),id(parent_reference)
#140690938288400 140690938290536 140690938288400
print id(parent[1]),id(parent_copy[1]),id(parent_reference[1])
#140690938137128 140690938137128 140690938137128
parent_copy[1].append(4)
parent_copy[2] = ['new']
print parent, parent_copy, parent_reference
#{1: [1, 2, 3, 4]} {1: [1, 2, 3, 4], 2: ['new']} {1: [1, 2, 3, 4]}
The hash(id) value of parent[1], parent_copy[1] are identical which implies [1,2,3] of parent[1] and parent_copy[1] stored at id 140690938288400.
But hash of parent and parent_copy are different which implies
They are different dictionaries and parent_copy is a new dictionary having values reference to values of parent
"new" and "original" are different dicts, that's why you can update just one of them.. The items are shallow-copied, not the dict itself.
In your second part, you should use new = original.copy()
.copy and = are different things.
Contents are shallow copied.
So if the original dict contains a list or another dictionary, modifying one them in the original or its shallow copy will modify them (the list or the dict) in the other.