We Keep Coding

Copying 'special' methods in Python 3.x [duplicate]

This question already has answers here:
Overriding special methods on an instance
(5 answers)
Closed 5 years ago.
Suppose I want to run the following code using Python 3.6.3:
class Foo:
def bar(self):
return 1
def __len__(self):
return 2
class FooWrapper:
def __init__(self, foo):
self.bar = foo.bar
self.__len__ = foo.__len__
f = Foo()
print(f.bar())
print(f.__len__())
print(len(f))
w = FooWrapper(Foo())
print(w.bar())
print(w.__len__())
print(len(w))
Here's the output:
1
2
2
1
2
TypeError: object of type 'FooWrapper' has no len()
So __len__() works, but len() does not? What gives and how go I properly copy __len__ method from Foo to FooWrapper?
By the way, the following behavior is universal for all 'special' methods, not only __len__: for example, __iter__ or __getitem__ do not work either (unless called directly)

The reason this happens is because the special methods have to be on an object's class, not on the instance.
len will look for __len__ in the FooWrapper class. BTW, although this looks like it "works", you are actually adding foo.__len__, i.e. , the methoud already bound to the foo instance of Foo to your FooWrapper object. That might be the intent, but you have to be aware of this.
The easiest way for this to work is to make FooWrapper have itself a __len__ method that will call the wrapped instance's __len__:
class FooWrapper:
def __init__(self, foo):
self.foo = foo
self.bar = foo.bar
self.__len__ = foo.__len__
def __len__(self):
return len(self.foo)
Does that mean that for any and all special methods hace to explicitly exist in the wrapper class? Yes, it does - and it is one of the pains for creating proxies that behave just the same as the wrapped object.
That is because the special methods' checking for existence and calling is done directly in C, and not using Python's lenghty lookup mechanisms, as tat would be too inefficient.
It is possible to create a wrapper-class factory thing that would inspect the object and create a brand new wrapper class,with all meaningful special methods proxied, though - but I think that would be too advanced for what you have in mind right now.
You'd better just use explicit special methods, or explicit access to the wrapped object in the remainder of the code. (Like, when you will need to use __iter__ from a wrapped object, instead of doing just for x in wrapper, do for x in wrapper.wrapped )

Assigning (instead of defining) a __getitem__ magic method breaks indexing [duplicate]

This question already has answers here:
Why won't dynamically adding a `__call__` method to an instance work?
(2 answers)
Closed 4 years ago.
I have a wrapper class similar to this (strongly simplified) example:
class wrap(object):
def __init__(self):
self._data = range(10)
def __getitem__(self, key):
return self._data.__getitem__(key)
I can use it like this:
w = wrap()
print w[2] # yields "2"
I thought I could optimize and get rid of one function call by changing to this:
class wrap(object):
def __init__(self):
self._data = range(10)
self.__getitem__ = self._data.__getitem__
However, I receive a
TypeError: 'wrap' object does not support indexing
for the print w[2] line with the latter version.
The direct call to the method, i.e., print w.__getitem__(2), works in both cases...
Why does the assignment version not allow indexing?
EDIT: Regarding the "closed for duplication"
I agree that the linked question has the same answer. It is, however, not at all clear that they are the same question. In particular, someone who does not know the answer here, also does not know that there is an overarching "type of problem" at work. Thus, it won't be clear that they find the answer in a seemingly unrelated question about __call__.

Special methods (essentially anything with two underscores on each end) have to be defined on the class. The internal lookup procedure for special methods completely skips the instance dict. Among other things, this is so if you do
class Foo(object):
def __repr__(self):
return 'Foo()'
the __repr__ method you defined is only used for instances of Foo, and not for repr(Foo).

Why doesn't Python allow referencing a class inside its definition?

Python (3 and 2) doesn't allow you to reference a class inside its body (except in methods):
class A:
static_attribute = A()
This raises a NameError in the second line because 'A' is not defined, while this
class A:
def method(self):
return A('argument')
works fine.
In other languages, for example Java, the former is no problem and it is advantageous in many situations, like implementing singletons.
Why isn't this possible in Python? What are the reasons for this decision?
EDIT:
I edited my other question so it asks only for ways to "circumvent" this restriction, while this questions asks for its motivation / technical details.

Python is a dynamically typed language, and executes statements as you import the module. There is no compiled definition of a class object, the object is created by executing the class statement.
Python essentially executes the class body like a function, taking the resulting local namespace to form the body. Thus the following code:
class Foo(object):
bar = baz
translates roughly to:
def _Foo_body():
bar = baz
return locals()
Foo = type('Foo', (object,), _Foo_body())
As a result, the name for the class is not assigned to until the class statement has completed executing. You can't use the name inside the class statement until that statement has completed, in the same way that you can't use a function until the def statement has completed defining it.
This does mean you can dynamically create classes on the fly:
def class_with_base(base_class):
class Foo(base_class):
pass
return Foo
You can store those classes in a list:
classes = [class_with_base(base) for base in list_of_bases]
Now you have a list of classes with no global names referring to them anywhere. Without a global name, I can't rely on such a name existing in a method either; return Foo won't work as there is no Foo global for that to refer to.
Next, Python supports a concept called a metaclass, which produces classes just like a class produces instances. The type() function above is the default metaclass, but you are free to supply your own for a class. A metaclass is free to produce whatever it likes really, even things that are bit classes! As such Python cannot, up front, know what kind of object a class statement will produce and can't make assumptions about what it'll end up binding the name used to. See What is a metaclass in Python?
All this is not something you can do in a statically typed language like Java.

A class statement is executed just like any other statement. Your first example is (roughly) equivalent to
a = A()
A = type('A', (), {'static_attribute': a})
The first line obviously raises a NameError, because A isn't yet bound to anything.
In your second example, A isn't referenced until method is actually called, by which time A does refer to the class.

Essentially, a class does not exist until its entire definition is compiled in its entirety. This is similar to end blocks that are explicitly written in other languages, and Python utilizes implicit end blocks which are determined by indentation.

The other answers are great at explaining why you can't reference the class by name within the class, but you can use class methods to access the class.
The #classmethod decorator annotes a method that will be passed the class type, instead of the usual class instance (self). This is similar to Java's static method (there's also a #staticmethod decorator, which is a little different).
For a singleton, you can access a class instance to store an object instance (Attributes defined at the class level are the fields defined as static in a Java class):
class A(object):
instance = None
#classmethod
def get_singleton(cls):
if cls.instance is None:
print "Creating new instance"
cls.instance = cls()
return cls.instance
>>> a1 = A.get_singleton()
Creating new instance
>>> a2 = A.get_singleton()
>>> print a1 is a2
True
You can also use class methods to make java-style "static" methods:
class Name(object):
def __init__(self, name):
self.name = name
#classmethod
def make_as_victoria(cls):
return cls("Victoria")
#classmethod
def make_as_stephen(cls):
return cls("Stephen")
>>> victoria = Name.make_as_victoria()
>>> stephen = Name.make_as_stephen()
>>> print victoria.name
Victoria
>>> print stephen.name
Stephen

The answer is "just because".
It has nothing to do with the type system of Python, or it being dynamic. It has to do with the order in which a newly introduced type is initialized.
Some months ago I developed an object system for the language TXR, in which this works:
1> (defstruct foo nil (:static bar (new foo)))
#
2> (new foo)
#S(foo)
3> *2.bar
#S(foo)
Here, bar is a static slot ("class variable") in foo. It is initialized by an expression which constructs a foo.
Why that works can be understood from the function-based API for the instantiation of a new type, where the static class initialization is performed by a function which is passed in. The defstruct macro compiles a call to make-struct-type in which the (new foo) expression ends up in the body of the anonymous function that is passed for the static-initfun argument. This function is called after the type is registered under the foo symbol already.
We could easily patch the C implementation of make_struct_type so that this breaks. The last few lines of that function are:
sethash(struct_type_hash, name, stype);
if (super) {
mpush(stype, mkloc(su->dvtypes, super));
memcpy(st->stslot, su->stslot, sizeof (val) * su->nstslots);
}
call_stinitfun_chain(st, stype);
return stype;
}
The call_stinifun_chain does the initialization which ends up evaluating (new foo) and storing it in the bar static slot, and the sethash call is what registers the type under its name.
If we simply reverse the order in which these functions are called, the language and type system will still be the same, and almost everything will work as before. Yet, the (:static bar (new foo)) slot specifier will fail.
I put the calls in that order because I wanted the language-controlled aspects of the type to be as complete as possible before exposing it to the user-definable initializations.
I can't think of any reason for foo not to be known at the time when that struct type is being initialized, let alone a good reason. It is legitimate for static construction to create an instance. For example, we could use it to create a "singleton".
This looks like a bug in Python.

Overriding __getattr__ method for one instance of a class? [duplicate]

This question already has answers here:
Overriding special methods on an instance
(5 answers)
Closed 5 years ago.
I'm working on a project right now that deals with functions in an abstract mathematical sense. Without boring the reader with the details, I'll say that I had the following structure in an earlier version:
class Foo(Bar):
def __init__(self, a, b):
self.a = a
self.b = b
self.sub_unit = Foo(a, not b)
Of course, that's not quite the change I'm making to the arguments to Foo, but suffice to say, it is necessary that this property, if accessed repeatedly, result in an indefinitely long chain of Foo objects. Obviously, this results in an infinite recursion when one instantiates Foo. I solved this in the earlier version by removing the last line of init and adding the following to the Foo class:
def __getattr__(self, attr: str):
if attr == 'sub_unit':
return Foo(self.a, not self.b)
else:
return super().__getattr__(attr)
This worked quite well, as I could calculate the next object in the chain as needed.
In going over the code, though, I realize that for other reasons, I need an instance of Bar, not a sub-class of it. To see if I could override the getattr for a single instance, I tried the following:
>>> foo = Bar(a=1, b=2) # sub_unit doesn't get set here.
>>> foo.__getattr__ = lambda attr: 'foo'
>>> foo.a
1
>>> foo.__getattr__('a')
'foo'
What is happening here that I don't understand? Why isn't foo.a calling foo.__getattr__('a')?
Is there a good way to overwrite __getattr__ for a single instance, or is my best bet to re-factor all the code I have that reads sub_unit and friends to call those as functions, to handle this special case?

When you lookup the attribute a with foo.a, python looks it up in the instance's property dictionary. When it is not found, the __getattr__ method will then be called.
On the contrary, if a exists in the instance, __getattr__ will not be called.

Monkey patch __del__ to new function

For specific debugging purposes I'd like to wrap the del function of an arbitrary object to perform extra tasks like write the last value of the object to a file.
Ideally I want to write
monkey(x)
and it should mean that the final value of x is printed when x is deleted
Now I figured that del is a class method. So the following is a start:
class Test:
def __str__(self):
return "Test"
def p(self):
print(str(self))
def monkey(x):
x.__class__.__del__=p
a=Test()
monkey(a)
del a
However if I want to monkey specific objects only I suppose I need to dynamically rewrite their class to a new one?! Moreover I need to do this anyway, since I cannot access del of built-in types?
Anyone knows how to implement that?

While special 'double underscore' methods like __del__, __str__, __repr__, etc. can be monkey-patched on the instance level, they'll just be ignored, unless they are called directly (e.g., if you take Omnifarious's answer: del a won't print a thing, but a.__del__() would).
If you still want to monkey patch a single instance a of class A at runtime, the solution is to dynamically create a class A1 which is derived from A, and then change a's class to the newly-created A1. Yes, this is possible, and a will behave as if nothing has changed - except that now it includes your monkey patched method.
Here's a solution based on a generic function I wrote for another question:
Python method resolution mystery
def override(p, methods):
oldType = type(p)
newType = type(oldType.__name__ + "_Override", (oldType,), methods)
p.__class__ = newType
class Test(object):
def __str__(self):
return "Test"
def p(self):
print(str(self))
def monkey(x):
override(x, {"__del__": p})
a=Test()
b=Test()
monkey(a)
print "Deleting a:"
del a
print "Deleting b:"
del b

del a deletes the name 'a' from the namespace, but not the object referenced by that name. See this:
>>> x = 7
>>> y = x
>>> del x
>>> print y
7
Also, some_object.__del__ is not guaranteed to be called at all.
Also, I already answered your question here (in german).

You can also inherit from some base class and override the __del__ method (then only thing you would need would be to override class when constructing an object).
Or you can use super built-in method.

Edit: This won't actually work, and I'm leaving it here largely as a warning to others.
You can monkey patch an individual object. self will not get passed to functions that you monkey patch in this way, but that's easily remedied with functools.partial.
Example:
def monkey_class(x):
x.__class__.__del__ = p
def monkey_object(x):
x.__del__ = functools.partial(p, x)