We Keep Coding

Why do Python's empty classes and functions work as arbitrary data containers, but not other objects?

I've seen two different Python objects used to group arbitrary data together: empty classes and functions.
def struct():
pass
record = struct
record.number = 3
record.name = "Zoe"
class Struct:
pass
record = Struct()
record.number = 3
record.name = "Zoe"
Even if the class isn't empty, it seems to work so long as it's defined at runtime.
But when I got cocky and tried to do this with built-in functions or classes, it didn't work.
record = set()
record.number = 3
AttributeError: 'set' object has no attribute 'number'
record = pow
pow.number = 3
AttributeError: 'builtin_function_or_method' object has no attribute 'number'
Is there a fundamental difference between built-in and "custom" classes and functions that accounts for this behavior?

The difference is that both function objects and your Struct object have a __dict__ attribute, but set instances and built-in functions do not:
>>> def struct():
... pass
...
>>> record = struct
>>> record.number = 2
>>> struct.__dict__
{'number': 2}
>>> class Struct:
... pass
...
>>> record = Struct()
>>> record.number = 3
>>> record.__dict__
{'number': 3}
>>> record=set()
>>> record.__dict__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'set' object has no attribute '__dict__'
>>> pow.__dict__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'builtin_function_or_method' object has no attribute '__dict__'
In classes you can emulate the behavour using slots (although only on new-style classes):
>>> class StructWithSlots(object):
... __slots__ = []
...
>>> record = StructWithSlots()
>>> record.number = 3
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'StructWithSlots' object has no attribute 'number'
>>> record.__dict__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'StructWithSlots' object has no attribute '__dict__'

Built-in types are written in C and cannot be modified like that. But after the type/class unification introduced in py2.2 you can now inherit from a built-in types and override or add your own attributes to that subclass.
You can use the forbiddenfood package to add attributes to built-in types:
This project aims to give you the way to find heaven in tests, but it
might lead you to hell if you use it on production code.
>>> from forbiddenfruit import curse
>>> def words_of_wisdom(self):
... return self * "blah "
>>> curse(int, "words_of_wisdom", words_of_wisdom)
>>> assert (2).words_of_wisdom() == "blah blah "
And of course if you're cocky enough then you can create your own types in C and add such features to it.

Some built-ins can be more restrictive. Also, classes implemented with slots won't accept arbitrary attributes either.

If you want some simular protection in your own class, you can use the __setattr__() method.
class TestClass(object):
# Accept the attributes in this list
__valid_attributes = ["myattr1", "myattr2"]
def __setattr__(self, name, value):
if not name in TestClass.__valid_attributes:
raise AttributeError(
"{0} has no attribute '{1}'".format(self.__class__.__name__, name))
self.__dict__[name] = value
Now you can do something like this:
t = TestClass()
t.noattr = "test" # AttributeError: TestClass has no attribute 'noattr'
But "valid attributes" can still be set:
t = TestClass()
t.myattr1 = "test"
print(t.myattr1) # test

Why is getattr() so much slower than self.__dict__.get()?

The example below is from a REST database driver on Python 2.7.
In the __setattr__ method below, if I use the commented out getattr() line, it reduces the object instantiation performance from 600 rps to 230.
Why is getattr() so much slower than self.__dict__.get() in this case?
class Element(object):
def __init__(self, client):
self._client = client
self._data = {}
self._initialized = True
def __setattr__(self, key, value):
#_initialized = getattr(self, "_initialized", False)
_initialized = self.__dict__.get("_initialized", False)
if key in self.__dict__ or _initialized is False:
# set the attribute normally
object.__setattr__(self, key, value)
else:
# set the attribute as a data property
self._data[key] = value

In short: because getattr(foo,bar) does the same thing as foo.bar, which is not the same thing as just accessing the __dict__ property (for a start, getattr has to select the right __dict__, but there's a whole lot more going on).
An example for illustration:
>>> class A:
... a = 1
...
>>> class B(A):
... b = 2
...
>>> dir(B)
['__doc__', '__module__', 'a', 'b']
>>> B.a
1
>>> B.__dict__
{'__module__': '__main__', 'b': 2, '__doc__': None}
>>> B.__dict__['a']
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
KeyError: 'a'
>>> B.__dict__.get('a')
>>>
Details contained in, or linked to here: http://docs.python.org/reference/datamodel.html (search for "getattr").

Impossible to set an attribute to a string?

Usually, you can set an arbitrary attribute to a custom object, for instance
----------
>>> a=A()
>>> a.foo=42
>>> a.__dict__
{'foo': 42}
>>>
----------
On the other hand, you can't do the same binding with a string object :
----------
>>> a=str("bar")
>>> a.foo=42
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'str' object has no attribute 'foo'
>>> a.__dict__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'str' object has no attribute '__dict__'
>>>
----------
Why ?

Because the str type is a type wich does not has an attribute dict. From the docs, "Classes" section:
A class has a namespace implemented by a dictionary object.
Class attribute references are translated to lookups in this
dictionary, e.g., C.x is translated to C.__dict__["x"]
You can also enforce something similar on custom objects:
>>> class X(object):
... __slots__=('a', )
...
>>> a = X()
>>> a.a = 2
>>> a.foo = 2
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'X' object has no attribute 'foo'
In general, you should not be setting nor modifying fields of objects that you are not supposed to. The documentation of the specific data type should reference you what fields are available for public modification.
For example, an ReadOnlyPoint object, where the x and y coordinates are set only on object construction:
>>> class ReadOnlyPoint(object):
... __slots__ = ('_x', '_y')
... def __init__(self, x, y):
... self._x = x
... self._y = y
... def getx(self):
... return self._x
... def gety(self):
... return self._y
... x = property(getx)
... y = property(gety)
...
>>> p = ReadOnlyPoint(2, 3)
>>> print p.x, p.y
2 3
>>> p.x = 9
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: can't set attribute
>>> p._x = 9
>>> print p.x, p.y
9 3
While the x and y properties are read-only, accessing the object internals allows you to alter the object's state.
The inhability to add a new field to an str object is an implementation detail, specific to the Python version that you are using.

http://docs.python.org/reference/datamodel.html
If the class has a setattr() or delattr() method, this is
called instead of updating the instance dictionary directly.
http://docs.python.org/reference/datamodel.html#object.setattr

How do I override __getattr__ without breaking the default behavior?

How do I override the __getattr__ method of a class without breaking the default behavior?

Overriding __getattr__ should be fine -- __getattr__ is only called as a last resort i.e. if there are no attributes in the instance that match the name. For instance, if you access foo.bar, then __getattr__ will only be called if foo has no attribute called bar. If the attribute is one you don't want to handle, raise AttributeError:
class Foo(object):
def __getattr__(self, name):
if some_predicate(name):
# ...
else:
# Default behaviour
raise AttributeError
However, unlike __getattr__, __getattribute__ will be called first (only works for new style classes i.e. those that inherit from object). In this case, you can preserve default behaviour like so:
class Foo(object):
def __getattribute__(self, name):
if some_predicate(name):
# ...
else:
# Default behaviour
return object.__getattribute__(self, name)
See the Python docs for more.

class A(object):
def __init__(self):
self.a = 42
def __getattr__(self, attr):
if attr in ["b", "c"]:
return 42
raise AttributeError("%r object has no attribute %r" %
(self.__class__.__name__, attr))
>>> a = A()
>>> a.a
42
>>> a.b
42
>>> a.missing
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 8, in __getattr__
AttributeError: 'A' object has no attribute 'missing'
>>> hasattr(a, "b")
True
>>> hasattr(a, "missing")
False

To extend Michael answer, if you want to maintain the default behavior using __getattr__, you can do it like so:
class Foo(object):
def __getattr__(self, name):
if name == 'something':
return 42
# Default behaviour
return self.__getattribute__(name)
Now the exception message is more descriptive:
>>> foo.something
42
>>> foo.error
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 5, in __getattr__
AttributeError: 'Foo' object has no attribute 'error'

How to assign a new class attribute via __dict__?

I want to assign a class attribute via a string object - but how?
Example:
class test(object):
pass
a = test()
test.value = 5
a.value
# -> 5
test.__dict__['value']
# -> 5
# BUT:
attr_name = 'next_value'
test.__dict__[attr_name] = 10
# -> 'dictproxy' object does not support item assignment

There is a builtin function for this:
setattr(test, attr_name, 10)
Reference: http://docs.python.org/library/functions.html#setattr
Example:
>>> class a(object): pass
>>> a.__dict__['wut'] = 4
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'dictproxy' object does not support item assignment
>>> setattr(a, 'wut', 7)
>>> a.wut
7