We Keep Coding

Default dict with a non-trivial default

I want to create a "default dict" that performs a non-trivial operation on the missing key (like a DB lookup, for example). I've seen some old answers on here, like Using the key in collections.defaultdict, that recommend subclassing collections.defaultdict.
While this makes sense, is there a reason to use defaultdict at this point. Why not simply subclass dict and override its __missing__ method instead? Does defaultdict provide something else that I'd gain by subclassing it?

What does defaultdict add?
According to the documentation, the only difference between a defaultdict and a built-in dict is:
It overrides one method and adds one writable instance variable.
The one method is the __missing__ method which is called when a key that is not present is accessed.
And the one writable instance variable is the default_factory - a callable with no arguments used by __missing__ to determine the default value to be used with missing keys.
Roughly equivalent to:
def __missing__(self, key):
if self.default_factory is None:
raise KeyError(key)
self[key] = self.default_factory()
return self[key]
When to inherit at all?
It is important to make it clear that the only reason you would even need to create a subclass is if your default value for missing keys is dependent of the actual key. If your default factory doesn't need to key - no matter how complicated the logic is, you can just use defaultdict instead of inheriting from it. If the logic is too much for a lambda, you can still of course create a function and use it:
def calc():
# very long code
# calculating a static new key
# (maybe a DB request to fetch the latest record...)
return new_value
d = defaultdict(calc)
If you actually need the key itself for the calculation of the default value, then you need to inherit:
When to inherit from defaultdict?
The main advantage is if you want to be able to have a dynamic factory (i.e. change the default_factory during runtime) this saves you the bother of implementing that yourself (no need to override __init__...).
But, note that this means you will have to take in account the existence of this default_factory when you override __missing__, as can be seen in this answer.
When to inherit from dict
When you don't care about dynamically changing the factory and can be satisfied with a static one throughout the existence of the dict.
In this case you simply override the __missing__ method and implement the factory with whatever complicated logic you have dependent of the key.

MutableSequence to pass as a list in isinstance() check

I built a custom list-like class based on collections.MutableSequence:
class MyList(collections.MutableSequence):
etc... behave mostly like a list...
value = MyList([1,2,3])
Before processing list data, a third-party library runs a check:
def check_correct_type(value):
assert isinstance(value, list)
I do not wish to convert my custom list-like object to a built-in list before passing it to the third-party library.
Is there an elegant way to make an instance of MyList appear as though it was an instance of list in the isinstance(MyList([1,2,3]), list) check?

No, there is no way instances of your class can pass that test without inheriting from list. You have to subclass list to pass that test.
You can try inheriting from both MutableSequence and list; any method or attribute not implemented by your class or by MutableSequence will then be looked up on list, so you may get extra methods that you don't want this way and those may behave unexpectedly:
class MyList(collections.MutableSequence, list):
You could also monkeypatch the check_correct_type() function, provided it really is a stand-alone function like that:
def my_check_correct_type(value):
assert isinstance(value, collections.MutableSequence)
third_party_library.check_correct_type = my_check_correct_type

Can I just partially override __setattr__?

I'm imitating the behavior of the ConfigParser module to write a highly specialized parser that exploits some well-defined structure in the configuration files for a particular application I work with. The files follow the standard INI structure:
[SectionA]
key1=value1
key2=value2
[SectionB]
key3=value3
key4=value4
For my application, the sections are largely irrelevant; there is no overlap between keys from different sections and all the users only remember the key names, never which section they're supposed to go in. As such, I'd like to override __getattr__ and __setattr__ in the MyParser class I'm creating to allow shortcuts like this:
config = MyParser('myfile.cfg')
config.key2 = 'foo'
The __setattr__ method would first try to find a section called key2 and set that to 'foo' if it exists. Assuming there's no such section, it would look inside each section for a key called key2. If the key exists, then it gets set to the new value. If it doesn't exist, the parser would finally raise an AttributeError.
I've built a test implementation of this, but the problem is that I also want a couple straight-up attributes exempt from this behavior. I want config.filename to be a simple string containing the name of the original file and config.content to be the dictionary that holds the dictionaries for each section.
Is there a clean way to set up the filename and content attributes in the constructor such that they will avoid being overlooked by my custom getters and setters? Will python look for attributes in the object's __dict__ before calling the custom __setattr__?

pass filename, content to super class to handle it
class MyParser(object):
def __setattr__(self, k, v):
if k in ['filename', 'content']:
super(MyParser, self).__setattr__(k, v)
else:
# mydict.update(mynewattr) # dict handles other attrs

I think it might be cleaner to present a dictionary-like interface for the contents of the file and leave attribute access for internal purposes. However, that's just my opinion.
To answer your question, __setattr__() is called prior to checking in __dict__, so you can implement it as something like this:
class MyParser(object):
specials = ("filename", "content")
def __setattr__(self, attr, value):
if attr in MyParser.specials:
self.__dict__[attr] = value
else:
# Implement your special behaviour here

what is the dict class used for

Can someone explain what the dict class is used for? This snippet is from Dive Into Python
class FileInfo(dict):
"store file metadata"
def __init__(self, filename=None):
self["name"] = filename
I understand the assignment of key=value pairs with self['name'] = filename but what does inheriting the dict class have to do with this? Please help me understand.

If you're not familiar with inheritance concept of object-oriented programming have a look at least at this wiki article (though, that's only for introduction and may be not for the best one).
In python we use this syntax to define class A as subclass of class B:
class A(B):
pass # empty class
In your example, as FileInfo class is inherited from standard dict type you can use instances of that class as dictionaries (as they have all methods that regular dict object has). Besides other things that allows you assign values by key like that (dict provides method for handing this operation):
self['name'] = filename
Is that the explanation you want or you don't understand something else?

It's for creating your own customized Dictionary type.
You can override __init__, __getitem__ and __setitem__ methods for your own special purposes to extend dictionary's usage.
Read the next section in the Dive into Python text: we use such inheritance to be able to work with file information just the way we do using a normal dictionary.
# From the example on the next section
>>> f = fileinfo.FileInfo("/music/_singles/kairo.mp3")
>>> f["name"]
'/music/_singles/kairo.mp3'
The fileinfo class is designed in a way that it receives a file name in its constructor, then lets the user get file information just the way you get the values from an ordinary dictionary.
Another usage of such a class is to create dictionaries which control their data. For example you want a dictionary who does a special thing when things are assigned to, or read from its 'sensor' key. You could define your special __setitem__ function which is sensitive with the key name:
def __setitem__(self, key, item):
self.data[key] = item
if key == "sensor":
print("Sensor activated!")
Or for example you want to return a special value each time user reads the 'temperature' key. For this you subclass a __getitem__ function:
def __getitem__(self, key):
if key == "temperature":
return CurrentWeatherTemperature()
else:
return self.data[key]

When an Class in Python inherits from another Class, it means that any of the methods defined on the inherited Class are, by nature, defined on the newly created Class.
So when FileInfo inherits dict it means all of the functionality of the dict class is now available to FileInfo, in addition to anything that FileInfo may declare, or more importantly, override by re-defining the method or parameter.
Since the dict Object in Python allows for key/value name pairs, this enables FileInfo to have access to that same mechanism.

Override reversed(...) in Python 2.5

I need a custom __reverse__ function for my class that I am deploying on App Engine, so it needs to work with Python 2.5. Is there a __future__ import or a workaround I could use?
Subclassing list won't work, as I need my class to be a subclass of dict.
EDIT:
Using OrderedDict will not solve the problems, because the dict keys are not the same the same as the list items.
This is the object I'm trying to create:
My object needs to provide the same attributes as a list, i.e. support iter(obj) and reverse(obj).
The elements must be instances of a special third party class.
Each elements is associated with a key.
Internally, need to access these objects using their keys. That's why I'd put them in a mapping.
I've revised my implementation to be a list subclass instead of a dict subclass, so here's what I have now:
class Foo(list):
pat = {}
def __init__(self):
for app in APPS: # these are strings
obj = SpecialClass(app)
self.append(obj)
self.pat[app] = obj
def __getitem__(self, item):
# Use object as a list
if isinstance(item, int):
return super(Foo, self).__getitem__(item)
# Use object as a dict
if item not in self.pat:
# Never raise a KeyError
self.pat[item] = SpecialClass(None)
return self.pat[item]
def __setitem__(self, item, value):
if isinstance(item, int):
return self.pat.__setitem__(item, value)
return super(Foo).__setitem__(item, value)
EDIT 2:
Now that my class is a subclass of list, my problem is resolved.

__reversed__ isn't supported in 2.5, so your only option if you really need to customize the reversed order of your collection, is to modify the places that you call reversed to use something else.
But I'm curious: if you are subclassing dict, then the order of items is arbitrary anyway, so what does reversed mean in this case?

Creating a custom __reversed__ is only possible since 2.6, so you can't simply implement that and have reversed working in 2.5. In 2.5 and below, you can however make your custom class still working with reversed by implementing the sequence protocol (i.e. implement both __len__ and __getitem__).
A different possibility would be to replace the built-in function reversed with a custom function that treats your custom class differently. This could work like this:
originalReversed = reversed
def myReversed ( seq ):
if isinstance( seq, MyCustomClass ):
# do something special
else:
return originalReversed( seq )
reversed = myReversed
However, I wouldn't recommend that as it changes the normal behaviour of built-in functions (obviously) and might confuse other users.. So you should rather implement the sequnce protocol to make reversed working.