We Keep Coding

Python: can view objects show keys of zombie dictionaries?

Here is my example code:
#!/usr/bin/env python3
myDict = {'a':1, 'b':2, 'c':3}
myKeys = myDict.keys()
# myDict = {} # toggle this line
myDict['y'] = 4
for myKey in myDict:
print('D ' + myKey)
for myKey in myKeys:
print('K ' + myKey)
If you run the program as shown here (with the line in the middle commented out), you get this output, which is exactly what I expected. The lines with the prefix D (loop over dictionary) have the same values as the lines with the prefix K (loop over keys of dictionary):
D a
D b
D c
D y
K a
K b
K c
K y
Now remove the hash and activate the line that was commented out. When you run the modified program, you get this:
D y
K a
K b
K c
But I expected one of these behaviors:
either:
After myDict = {} was executed, myKeys has become empty too (since it is a view object that always views the keys of its parent dictionary). Adding the item with the key 'y' should result in this output:
D y
K y
or:
After myDict = {} was executed, a new version of myDict was created and the previous version of myDict was destroyed, so myKeys has no longer any parent dictionary and therefor is pointing to null. So in this case looping over myKeys should throw an error.
But to me is looks as if the old version of myDict has become some kind of zombie, and myKeys is displaying the keys of this zombie.
So, here are my questions:
Is it true, that in my program myKeys shows keys of a zombie dictionary if the line in the middle is activated?
Is it reliable behavior, that myKeys will always show the keys of the previous version of the dictionary? (That would be very helpful for a program that I'm writing just now.)
Is there a way to revive the zombie dictionary? (I have all keys. Can I get the values too?)

The fundamental issue in your understanding here is that:
myDict = {}
Does nothing to the original dict, assignment never mutates, so the object that myDict was referring to is unmodified. Python objects will stay alive as long as something refers to them. CPython uses reference counting, and as an implementation detail, will reclaim objects immediately when their reference count reaches zero.
However, the dict_keys view object you created internally references the dictionary it is acting as a view over, so the original dictionary still has at least one reference to it, since the view object is alive. Note, though, the dict_keys API does not expose this reference, so if it is the only reference, you cannot really access your dict anymore (in any sane way).
Is it reliable behavior, that myKeys will always show the keys of the previous version of the dictionary?
You are misunderstanding the behavior. It is showing the keys of the same dictionary it has always been showing. There is no previous dictionary. The view is over the object, it doesn't care or know which variables happen to be referring to that object at any given time.
It's the same reason that if you do:
x = 'foo'
container = []
container.append(x)
x = 'bar'
print(container[0])
will still print "foo". Objects have no idea what names are referencing them at any given time, and should not care. And when one reference changes, the other references don't magically get updated.

clearing a dictionary but keeping the keys

Is it possible to clear all the entries within a dictionary but keep all the keys?
For example if I had:
my_dic={
"colour":[],
"number":[]
}
I put some stuff in them:
my_dic["colour"]='Red'
my_dic["number"]='2'
I can clear these by:
my_dic["colour"] = []
my_dic["number"] = []
But this is long winded if I want to clear a large dictionary quickly, is there a quicker way perhaps using for? I want to keep the keys ["colour"], ["number"], without having to recreate them, just clear all the entries within them.

You can simply clear all lists in a loop:
for value in my_dic.values():
del value[:]
Note the value[:] slice deletion; we are removing all indices in the list, not the value reference itself.
Note that if you are using Python 2 you probably want to use my_dic.itervalues() instead of my_dic.values() to avoid creating a new list object for the loop.
Demo:
>>> my_dic = {'colour': ['foo', 'bar'], 'number': [42, 81]}
>>> for value in my_dic.values():
... del value[:]
...
>>> my_dic
{'colour': [], 'number': []}
You could also replace all values with new empty lists:
my_dic.update((key, []) for key in my_dic)
or replace the whole dictionary entirely:
my_dic = {key: [] for key in my_dic}
Take into account these two approaches will not update other references to either the lists (first approach) or the whole dictionary (second approach).

You no need to delete keys from dictionary:
for key in my_dict:
my_dict[key] = []

One liner:
my_dict = dict.fromkeys(my_dict, None)
You can also replace the None type with other values that are immutable. A mutable type such as a list will cause all of the values in your new dictionary to be the same list.
For mutable types you would have to populate the dictionary with distinct instances of that type as others have shown.

Python dictionary

I am having some trouble understanding this, I have tried to reduce the problem to this set of code
for k in y.keys():
if k in dateDict.keys():
if yearDict[k] in dict1:
dict1[yearDict[k]].extend(y[k])
else:
dict1[yearDict[k]] = y[k]
if yearDict[k] in dict2:
dict2[yearDict[k]].extend(y[k])
else:
dict2[yearDict[k]] = y[k]
else:
continue
I have two dictionaries y and dateDict to begin with. For a matching key for y in dateDict, I am populating two other dictionaries dict1 and dict2, hashed with keys from some other dictionary yearDict. Unfortunately the result are duplicated in dict1 and dict2, I have values repeating themselves. Any idea what could be happening?
Also I notice that this code works as expected,
for k in y.keys():
if k in dateDict.keys():
if yearDict[k] in dict1:
dict1[yearDict[k]].extend(y[k])
else:
dict1[yearDict[k]] = y[k]
else:
continue

If y[k] is a list (which it looks like), the same list will be assigned to everywhere where is it used. Dictionaries do not make copies of the elements when they are assigned, they just keep references to their objects. In your example, both keys in dict1 and dict2 will point to the same object.
Later, when it is modified, the same elements will be appended with the new values, once for each map. To prevent this, you can create a new list when initially assigning:
dictl[yearDict[k]] = list(y[k])
However, it is always good to know the Python standard library. This code could be made much more readable, and without the error, by using collections.defaultdict:
from collections import defaultdict
# This goes wherever the dictionaries
# where initially defined.
dict1 = defaultdict(list)
dict2 = defaultdict(list)
# You can get the value here, no need to search it later.
for k, value in y.items():
if k in dateDict.keys():
# No need to call this everywhere.
new_key = yearDict[k]
# Note the defaultdict magic.
dict1[new_key].extend(value)
dict2[new_key].extend(value)
# No need for the 'continue' at the end either.
When asked for a key that does not exist yet, the defaultdict will create a new one on the fly -- so you don't have to care about initialization, or creating copies of you values.

Why don't Python dictionaries treat keys independently in this script? [duplicate]

This question already has answers here:
How to copy a dictionary and only edit the copy
(23 answers)
Closed 8 years ago.
I'm expecting my frustration to be overridden with some enlightenment - here's a minimal version of the script to demonstrate the problem:
First I create a dictionary:
dic = {
'foo':{},
'bar':{}
}
Then we instantiate a template dictionary that can be iteratively appended
to keys of dic:
appendic= {
'is':'', # '' is a terminal value to be replaced later
}
So here we append appendic to each of the keys in dic:
dic['foo'] = appendic
dic['bar'] = appendic
Now we replace the terminal values, '', with something meaningful:
dic['foo']['is'] = 'foo'
dic['bar']['is'] = 'bar'
At this point, my intuition tells me that if we call:
print(dic['foo']['is']) we get 'foo'
But instead Python returns 'bar' ... to my un-trained mind that is counter-intuitive.
Questions:
How can I tell Python to keep the keys of dic independent?
Why is this the default behaviour? What use cases does this have?

When you assign a appendic to two different keys, Python doesn't make a copy. It assigns a reference instead.
As a result, both dic['please_make_me_Foo'] and dic['dont_make_him_Bar'] refer to the same object. These are not separate dictionaries, they are both the same object, the one appendic also references to.
If you expected these to be separate dictionaries, create a copy of appendic instead. The dict.copy() method creates a shallow copy of a dictionary:
dic['please_make_me_Foo']= appendic.copy()
dic['dont_make_him_Bar'] = appendic.copy()
Shallow means that a new dictionary is created and all references to keys and values contained are copied over.
If appendic itself contains values that are also dictionaries, these would not be copied. The new copy and appendic would both refer to the same values. In most cases, that's not a problem because most primitive values (strings, integers, etc.) are immutable, and you never notice references are shared as you replace such values with new ones.

You make a dict:
appendic= {
'Python_made_me':''
}
Add it to your other dict twice
dic['please_make_me_Foo']= appendic
dic['dont_make_him_Bar'] = appendic
And set the single dict's Python_made_me value twice
dic['please_make_me_Foo']['Python_made_me'] = 'Foo'
dic['dont_make_him_Bar']['Python_made_me'] = 'Bar'
But because they're the same dict, the second line overwrites the first
If you need to copy it, you need to use the copy method:
dic['please_make_me_Foo']= appendic.copy()
dic['dont_make_him_Bar'] = appendic.copy()

ok, I'm just going to write this as a complement to the other answers. When you manipulate a dictionary, you manipulate the reference to an instance, which is the root cause of your mistake. Using hex(id(foo)) you get the memory address of foo, so let's show the address of d instance in the following example to make that tangible:
>>> hex(id(d))
'0x10bd95e60'
>>> hex(id(e[1]))
'0x10bd95e60'
>>> hex(id(f[1]))
'0x10bd95e60'
so if you add or remove values from e[1], you're actually changing the same instance as the one pointed by d, and as a dictionary is mutable, i.e. you can change values within.
Now you're wondering why that does not happen when you're handling integers? Because, in fact it does, it's just that integers are not mutable:
>>> i = 1
>>> hex(id(i))
'0x10ba51e90'
>>> j = i
>>> hex(id(j))
'0x10ba51e90'
>>> i = 2
>>> hex(id(i))
'0x10ba51eb0'
i.e. i is pointing to another place in the memory.
It's possible to create a mutable integer though, by using a class:
>>> class Integer:
... def __init__(self, i):
... self.i = i
...
>>> i = Integer(2)
>>> hex(id(i))
'0x10bd9b410'
>>> j = i
>>> hex(id(j))
'0x10bd9b410'
>>> j.i = 2
>>> i.i
2
>>> hex(id(i))
'0x10bd9b410'
In order to create a new instance of the same dictionary, you need to use the copy() member of a dict:
>>> hex(id(d))
'0x10bd95e60'
>>> w = d.copy()
>>> x = d.copy()
>>> y = d.copy()
>>> hex(id(w))
'0x10bd96128'
>>> hex(id(x))
'0x10bd95f80'
>>> hex(id(y))
'0x10bd96098'

dic['please_make_me_Foo']= appendic
dic['dont_make_him_Bar'] = appendic
appendic is an object - you are assigning a reference to the same object to both keys in dic. So when you change one, you change both.
Try this instead:
dic['please_make_me_Foo']= appendic.copy()
dic['dont_make_him_Bar'] = appendic.copy()

Increment items in nested python dict

I have a nested Python Dict and I am trying to take values from a list and then iterate them into a Dict's values as such:
for row in rows:
Dict[A][AA][AAA] += 1
However, when I print my dict, it appears to be adding all of the increments to all of the Dict entries. By which I mean that instead of this:
{KeyA:{KeyAA:{KeyAAA:5}}}
{KeyB:{KeyBB:{KeyBBB:10}}}
I am getting this:
{KeyA:{KeyAA:{KeyAAA:15}}}
{KeyB:{KeyBB:{KeyBBB:15}}}
I'm a bit stumped.
EDIT:
This is how the Dicts were created:
I first skim through a long table that contains a type classification. While I'm doing that, I create a new entry into the main Dict. At the same time, I'm collecting all of the unique classifications into a subDict so that I can add this to the main Dict later on:
Dict = {}
subDict = {}
for row in skimRows:
Dict[row[0]] = {"Type":row[1],"Assoc":{}} # Save each ID and origin Type to Dict
if item not in subDict: # Check to see if unique item already exists in subDict
subDict[item] = 0
Here is evidently where I was going wrong. I was then taking the subDict and plunking this into the main Dict, not realising the inserted subDict was retaining its relationship to the original subDict object:
for key in Dict: # After initial iteration and Type collection, add new subDict to each Dict key
Dict[key]["Assoc"] = subDict
SOLUTION:
Per the correct answer below, I fixed it by adding .copy()
for key in Dict: # After initial iteration and Type collection, add new subDict to each Dict key
Dict[key]["Assoc"] = subDict.copy()

Your innermost dictionaries are shared, not unique objects:
>>> somedict = {}
>>> somedict['foo'] = {'bar': 0}
>>> somedict['spam'] = somedict['foo']
>>> somedict['foo']['bar'] += 1
>>> somedict['spam']
{'bar': 1}
>>> somedict['foo'] is somedict['spam']
True
The two keys foo and spam both are referring to the same object here, one dictionary object holding a key bar.
You should not reuse your dictionaries like this. Either create a new empty dictiorary:
somedict['spam'] = {'bar': 0}
or create a (shallow) copy:
somedict['spam'] = somedict['foo'].copy()