This question already has answers here:
How to copy a dictionary and only edit the copy
(23 answers)
Closed 8 years ago.
I'm expecting my frustration to be overridden with some enlightenment - here's a minimal version of the script to demonstrate the problem:
First I create a dictionary:
dic = {
'foo':{},
'bar':{}
}
Then we instantiate a template dictionary that can be iteratively appended
to keys of dic:
appendic= {
'is':'', # '' is a terminal value to be replaced later
}
So here we append appendic to each of the keys in dic:
dic['foo'] = appendic
dic['bar'] = appendic
Now we replace the terminal values, '', with something meaningful:
dic['foo']['is'] = 'foo'
dic['bar']['is'] = 'bar'
At this point, my intuition tells me that if we call:
print(dic['foo']['is']) we get 'foo'
But instead Python returns 'bar' ... to my un-trained mind that is counter-intuitive.
Questions:
How can I tell Python to keep the keys of dic independent?
Why is this the default behaviour? What use cases does this have?
When you assign a appendic to two different keys, Python doesn't make a copy. It assigns a reference instead.
As a result, both dic['please_make_me_Foo'] and dic['dont_make_him_Bar'] refer to the same object. These are not separate dictionaries, they are both the same object, the one appendic also references to.
If you expected these to be separate dictionaries, create a copy of appendic instead. The dict.copy() method creates a shallow copy of a dictionary:
dic['please_make_me_Foo']= appendic.copy()
dic['dont_make_him_Bar'] = appendic.copy()
Shallow means that a new dictionary is created and all references to keys and values contained are copied over.
If appendic itself contains values that are also dictionaries, these would not be copied. The new copy and appendic would both refer to the same values. In most cases, that's not a problem because most primitive values (strings, integers, etc.) are immutable, and you never notice references are shared as you replace such values with new ones.
You make a dict:
appendic= {
'Python_made_me':''
}
Add it to your other dict twice
dic['please_make_me_Foo']= appendic
dic['dont_make_him_Bar'] = appendic
And set the single dict's Python_made_me value twice
dic['please_make_me_Foo']['Python_made_me'] = 'Foo'
dic['dont_make_him_Bar']['Python_made_me'] = 'Bar'
But because they're the same dict, the second line overwrites the first
If you need to copy it, you need to use the copy method:
dic['please_make_me_Foo']= appendic.copy()
dic['dont_make_him_Bar'] = appendic.copy()
ok, I'm just going to write this as a complement to the other answers. When you manipulate a dictionary, you manipulate the reference to an instance, which is the root cause of your mistake. Using hex(id(foo)) you get the memory address of foo, so let's show the address of d instance in the following example to make that tangible:
>>> hex(id(d))
'0x10bd95e60'
>>> hex(id(e[1]))
'0x10bd95e60'
>>> hex(id(f[1]))
'0x10bd95e60'
so if you add or remove values from e[1], you're actually changing the same instance as the one pointed by d, and as a dictionary is mutable, i.e. you can change values within.
Now you're wondering why that does not happen when you're handling integers? Because, in fact it does, it's just that integers are not mutable:
>>> i = 1
>>> hex(id(i))
'0x10ba51e90'
>>> j = i
>>> hex(id(j))
'0x10ba51e90'
>>> i = 2
>>> hex(id(i))
'0x10ba51eb0'
i.e. i is pointing to another place in the memory.
It's possible to create a mutable integer though, by using a class:
>>> class Integer:
... def __init__(self, i):
... self.i = i
...
>>> i = Integer(2)
>>> hex(id(i))
'0x10bd9b410'
>>> j = i
>>> hex(id(j))
'0x10bd9b410'
>>> j.i = 2
>>> i.i
2
>>> hex(id(i))
'0x10bd9b410'
In order to create a new instance of the same dictionary, you need to use the copy() member of a dict:
>>> hex(id(d))
'0x10bd95e60'
>>> w = d.copy()
>>> x = d.copy()
>>> y = d.copy()
>>> hex(id(w))
'0x10bd96128'
>>> hex(id(x))
'0x10bd95f80'
>>> hex(id(y))
'0x10bd96098'
dic['please_make_me_Foo']= appendic
dic['dont_make_him_Bar'] = appendic
appendic is an object - you are assigning a reference to the same object to both keys in dic. So when you change one, you change both.
Try this instead:
dic['please_make_me_Foo']= appendic.copy()
dic['dont_make_him_Bar'] = appendic.copy()
Related
class Object:
def __init__(self,dict):
self.dict = dict
a = Object({1:"hello",2:"lol"})
b = Object(a.dict)
b.dict.pop(1) #remove the element with key 1
print(a.dict, b.dict)
>>{2: 'lol'} {2: 'lol'}
for some reason the "a" object's dictionary gets modified too.
I've tried the same thing with a different attribute, like an int variable, and the problem didn't happen. I really don't know what to do :(
you are not copying the dictionary you are just pointing both a and b to the same dict, thats why its changing both.
you can use dict.copy() to create a shallow copy.
b = Object(a.dict.copy())
this will copy the dict but not any nested dictionaries, for that you need a deep copy.
import copy
b = copy.deepcopy(a.dict)
This question already has answers here:
How to copy a dictionary and only edit the copy
(23 answers)
Closed 7 days ago.
I have a nested dictionary, let's call it dictionary d. The key of this dictionary is an integer, and the value of each key is another dictionary. I'm trying a simple code on python 2.7 to update the value of one outer key, but it seems that it's updating the values of ALL of the outer key.
Hope these codes will make it easier to understand. Here's my input.
>>> template = {'mean':0,'median':0}
>>> d[0] = template
>>> d[1] = template
>>> d[0]['mean'] = 1
>>> d
and then here's the output:
{0: {'mean':1, 'median':0}, 1:{'mean':1,'median':0}}
you see, I only assigned '1' to d[0]['mean'], but somehow the d[1]['mean'] is also updated. If i increase the number of keys in the d, it will just change ALL of the ['mean'] values on all d keys.
Am I doing anything wrong here? Is this a bug?
>>> d[0] = template
>>> d[1] = template
These two statements made both d[0] and d[1] refer to the same object, template. Now you can access the dictionary with three names, template, d[0] and d[1]. So doing:
d[0]['mean'] = 1
modifies a dictionary object, which can be referred with the other names mentioned above.
To get this working as you expected, you can create a copy of the template object, like this
>>> d[0] = template.copy()
>>> d[1] = template.copy()
Now, d[0] and d[1] refer to two different dictionary objects.
In the following code I make a deep copy of _data dictionary so that I don't change the original.
My question is how can I get the new version v7 from the variable my_str
i.e, my_str should point me to v7 and not v6 anymore. I want to use the same variable my_str and not construct a new one.
_data = {"version":"v6"}
my_str = "{version}".format(**_data)
import copy
new_data = copy.deepcopy(_data)
new_data["version"] = "v7"
print(my_str) # I expected "v7" and not "v6" here
There is absolutely no reason you should have 'expected "v7" and not "v6"'.
my_dict isn't a dictionary; it's a string:
>>> _data = {"version":"v6"}
>>> my_dict = "{version}".format(**_data)
>>> my_dict
'v6'
>>> type(my_dict)
<type 'str'>
There is no connection between my_dict and _data, and strings are immutable (can't be changed in-place). Even if there was some magic connection between the two, you have deliberately made new_data a copy of _data before updating it, which would have removed that connection.
There is no way to "update" my_dict, you have to create a new string from the new, altered dictionary:
my_dict = "{version}".format(**new_data)
or why not just access the value?
new_data['version']
How can I generate a variable name from a string (say a concatenation of a letter and a number) ?
In Matlab, this task can be easily done using genvarname
Here's a really bad way (undefined behavior), but I think it shows the path to a more reasonable technique.
Your current namespace is really a dictionary under the covers:
>>> local_namespace = locals()
>>> name = "myVarName"
>>> local_namespace[name] = 'VarData'
>>> myVarName
'VarData'
But that's not very DRY - you have to write the name of the variable twice! It would be nice to use a variable that stored the name of our dynamically created variable so we didn't have to type it twice:
>>> name
'myVarName'
obviously doesn't work for this. But we can use our dictionary again:
>>> local_namespace[name]
'VarData'
So now we can store and recall the value associated with our variable. But wait - there's no need to use the special locals() dictionary for this - an ordinary dictionary will do!
>>> d = {}
>>> d[name] = 'VarData'
>>> d[name]
'VarData'
And now we have all these added benefits, like being able to keep track of the names of several of these variables in a list:
>>> l = []
>>> l.append('myVarName')
>>> l.append('anotherVarName')
Dictionaries even do this for us:
>>> d['anotherVarName'] = 123
>>> d.keys()
['myVarName', 'anotherVarName']
Unless you're doing terrifically wacky things, it's hard to imagine how constructing variable names could be more useful than using a dictionary.
You can use exec("").
But you really(!!!) don't want to.
>>> name="myVarName"
>>> exec(name+"='VarData'")
>>> myVarName
'VarData'
My code will be more clear I think-
someList = list()
foo = {'a':'b'}
someList.append(foo)
print someList
>>> [{'a':'b'}]
defaultbazz = {'a':2, 'b':'t', 'c':'gg'}
for k, v in defaultbazz.iteritems():
foo[k] = v
print someList
>>> [{'a': 2, 'c': 'gg', 'b': 't'}]
Shouldn't the last print be [{'a':'b'}]? I didn't updated the someList, I want it as is..
It's seems to me uninterpreted behavior..
But if that's how python works, how can I find workaround? Even setting a new dict updates the original one dict.. I mean:
someList = list()
foo = {'a':'b'}
someList.append(foo)
print someList
>>> [{'a':'b'}]
bar = foo
defaultbazz = {'a':2, 'b':'t', 'c':'gg'}
for k, v in defaultbazz.iteritems():
bar[k] = v
print someList
>>> [{'a': 2, 'c': 'gg', 'b': 't'}]
I'll be thankful if someone can maybe explain me why it's happen..
It looks like you are expecting your dict to be copied when you add it to a list or assign it to a new variable, but that is not how Python operates. If you assign a dict -- actually, if you assign any object -- you are not creating a new object, but instead you are simply giving your object a new name. (An object can have multiple names.)
So, when you edit your object under the new name, the single instance of that object changes, and that change is visible when you access the object through any name.
If you want to copy your object, then you can do this:
bar = dict(foo)
or
bar = foo.copy()
To simplify:
a = {2: 3}
b = [a]
b contains a "reference" to a (and is a dict which is mutable) - so if a is modified then accessing a via the list b, will display the modified a.
You have to explicitly create a copy of a, which can be done in this case as:
b = [dict(a)]
But you should look at the copy module for copy.copy() and copy.deepcopy()
Dictionaries are mutable objects, hence the result of your script.
I guess you want a new object, i.e. a copy of the original one:
import copy
someList.append(copy.copy(foo))
Variables in Python are just names of objects. If you change the object from any name "attached" to it, you will see the changes from every other name. Python never creates copies automatically for you, in particular:
someList.append(foo)
doesn't create a copy of foo and put it on someList, it appends the object that the name foo refers to onto the list.
You can create a second name for this object
bar = foo
but this does not create a copy either. In particular
foo['x'] = 42
and
bar['x'] = 42
will then operate on exactly the same object. You can verify this by printing the memory address of the object:
print id(foo), id(bar)
and see that they are the same.
If you need a copy in Python, you'll need to create one explicitly. Depending on what you need, the copy module -- either copy.copy() or copy.deepcopy() -- will do what you want:
import copy
bar = copy.copy(foo)
print id(foo), id(bar)
should now print different memory locations.
Dicts are mutable, which means that they can change. It's because foo is inside of someList and you're changing foo in the for-loop. Take a look at this simple example:
a_dict = {'a':'b'}
a_list = [a_dict]
print a_list # [{'a':'b'}]
#change the dict
a_dict['a'] = 'c'
print a_list # [{'a':'c'}]