I came across this as a bit of a surprise while trying to work out another question.
This seemed extremely odd to me, I thought it was worth asking the question. Why doesn't __getattr__ appear to work with with?
if I make this object:
class FileHolder(object):
def __init__(self,*args,**kwargs):
self.f= file(*args,**kwargs)
def __getattr__(self,item):
return getattr(self.f,item)
and using it with with,
>>> a= FileHolder("a","w")
>>> a.write
<built-in method write of file object at 0x018D75F8>
>>> with a as f:
... print f
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: __exit__
>>> a.__exit__
<built-in method __exit__ of file object at 0x018D75F8>
Why does this happen?
EDIT
>>> object.__exit__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: type object 'object' has no attribute '__exit__'
It definitely isn't inheriting __exit__
The with statement opcode SETUP_WITH looks up __exit__ as a "special method lookup", which ignores __getattr__ and __getattribute__ on new-style classes (but not on old-style classes). See this mailing list thread for more information, where they discuss adding the special method lookup semantics to with (which they eventually do). See also special method lookup for new-style classes for a detailed discussion on why these special methods are looked up in this way.
In particular, special method lookup also bypasses __getattr__ on the type object. So, even though the documentation says the method is looked up as type(mgr).__exit__, this code doesn't work:
class M(type):
def __getattr__(*args): return lambda: 0
class X(object):
__metaclass__ = M
x = X()
type(x).__exit__ # works, returns a lambda
with x: pass # fails, AttributeError
I can't say for sure, but after reading over the PEP describing the with statement:
http://www.python.org/dev/peps/pep-0343/
This jumped out at me:
A new statement is proposed with the syntax:
with EXPR as VAR:
BLOCK
....
The translation of the above statement is:
mgr = (EXPR)
exit = type(mgr).__exit__ # Not calling it yet
value = type(mgr).__enter__(mgr)
....
Right there. The with statement does not call __getattr__(__exit__) but calls type(a).__exit__ which does not exist giving the error.
So you just need to define those:
class FileHolder(object):
def __init__(self,*args,**kwargs):
self.f= file(*args,**kwargs)
def __enter__(self,*args,**kwargs):
return self.f.__enter__(*args,**kwargs)
def __exit__(self,*args,**kwargs):
self.f.__exit__(*args,**kwargs)
def __getattr__(self,item):
return getattr(self.f,item)
The previous answers has explained the fact that __getattr__ does not work with __enter__ and __exit__. I'm here to give my thinking of why it SHOULD NOT work.
The only reason we define __enter__ and __exit__ methods on an object is that we need to use it in with statement. The two methods help us get and release a resource implicitly, so we usually define them like this:
class Resource(object):
...
def __enter__(self):
return self
def __exit__(self, *exc):
self.close()
then you can write some code like this:
with Resource() as resource: # __enter__ is called and returns a value as `resource`
do_something_with_resource()
# `resource.__exit__` is called
As you have noticed, the resource we get and release is exactly an instance of the class we defined.
What if we hold a resource as an attribute and proxy its __enter__ and __exit__ with __getattr__? We write some code like this:
class ResourceProxy(object):
def __init__(self):
self._resource = Resource()
def __getattr__(self, key):
return getattr(self._resource, key)
Assuming __getattr__ works fine with __enter__ and __exit__, here is what will happen in with statement:
with ResourceProxy() as resource: # proxied __enter__ is called
# now `resource` is NOT a ResourceProxy instance, because what we called is `_resource.__enter__`
do_something_with_resource()
# `_resource.__exit__` is called and closed itself properly.
# Here is nothing to do with ResourceProxy, because it has never enter `with` context
The behavior above is strange and probably not as the user expected, for the following two reasons:
the resource entered into with context is not the object we sent in.
when exiting with context, __exit__ method of the proxied object is called, instead of the outer object we sent in. You may think it might help if we add an __exit__ definition on the outer class, but the answer is not, because the outer class has never enter with context.
To conclude, if we make __getattr__ works with __enter__ and __exit__, it will result in bad behaviors. It's not a good design.
Related
While executing the following code:
class Test():
def __init__(self):
self.hi_there()
self.a = 5
def hi_there(self):
print(self.a)
new_object = Test()
new_object.hi_there()
I have received an error:
Traceback (most recent call last):
File "/root/a.py", line 241, in <module>
new_object = Test()
File "/root/a.py", line 233, in __init__
self.hello()
File "/root/a.py", line 238, in hello
print(self.a)
AttributeError: 'Test' object has no attribute 'a'
Why do we need to specify the self inside the function while the object is not initialized yet? The possibility to call hi_there() function means that the object is already set, but how come if other variables attributed to this instances haven't been initialized yet?
What is the self inside the __init__ function if it's not a "full" object yet?
Clearly this part of code works:
class Test():
def __init__(self):
#self.hi_there()
self.a = 5
self.hi_there()
def hi_there(self):
print(self.a)
new_object = Test()
new_object.hi_there()
I come from C++ world, there you have to declare the variables before you assign them.
I fully understand your the use of self. Although I don't understand what is the use of self inside__init__() if the self object is not fully initialized.
There is no magic. By the time __init__ is called, the object is created and its methods defined, but you have the chance to set all the instance attributes and do all other initialization. If you look at execution in __init__:
def __init__(self):
self.hi_there()
self.a = 5
def hi_there(self):
print(self.a)
the first thing that happens in __init__ is that hi_there is called. The method already exists, so the function call works, and we drop into hi_there(), which does print(self.a). But this is the problem: self.a isn't set yet, since this only happens in the second line of __init__, but we called hi_there from the first line of __init__. Execution hasn't reached the line where you set self.a = 5, so there's no way that the method call self.hi_there() issued before this assignment can use self.a. This is why you get the AttributeError.
Actually, the object has already been created when __init__ is called. That's why you need self as a parameter. And because of the way Python works internally, you don't have access to the objects without self (Bear in mind that it doesn't need to be called self, you can call it anything you want as long as it is a valid name. The instance is always the first parameter of a method, whatever it's name is.).
The truth is that __init__ doesn't create the object, it just initializes it. There is a class method called __new__, which is in charge of creating the instance and returning it. That's where the object is created.
Now, when does the object get it's a attribute. That's in __init__, but you do have access to it's methods inside of __init__. I'm not completely knowledable about how the creation of the objects works, but methods are already set once you get to that point. That doesn't happen with values, so they are not available until you define them yourself in __init__.
Basically Python creates the object, gives it it's methods, and then gives you the instance so you can initialize it's attributes.
EDIT
Another thing I forgot to mention. Just like you define __init__, you can define __new__ yourself. It's not very common, but you do it when you need to modify the actual object's creation. I've only seen it when defining metaclasses (What are metaclasses in Python?). Another method you can define in that case is __call__, giving you even more control.
Not sure what you meant here, but I guess the first code sample should call an hello() function instead of the hi_there() function.
Someone corrects me if I'm wrong, but in Python, defining a class, or a function is dynamic. By this I mean, defining a class or a function happens at runtime: these are regular statements that are executed just like others.
This language feature allows powerful thing such as decorating the behavior of a function to enrich it with extra functionality (see decorators).
Therefore, when you create an instance of the Test class, you try to call the hello() function before you have set explicitly the value of a. Therefore, the Test class is not YET aware of its a attribute. It has to be read sequentially.
This may be a silly question but i am curious to know the answer.
As per official documentation, __init__ doesn't need return statement. Any particular reason why is it that way.
>>> class Complex:
... def __init__(self, realpart, imagpart):
... self.r = realpart
... self.i = imagpart
...
>>> x = Complex(3.0, -4.5)
>>> x.r, x.i
(3.0, -4.5)
__init__() is not a normal function. It is a special method Python uses to customize an instance of a class. It is part of Python's data model:
Called after the instance has been created (by __new__()), but before it is returned to the caller[...].
As you can see from above, when you create a new instance of a class, Python first calls __new_() - which is also a special method - to create a new instance of the class. Then __init__() is called to customize the new instance.
It wouldn't make sense to return anything from __init__(), since the class instance is already created. In fact, Python goes as far as raising an error to prevent this:
>>> class A:
... def __init__(self):
... return 'foo'
...
>>> A()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: __init__() should return None, not 'str'
>>>
If you want to know what exactly is going on behind the scenes, #eryksun provides a nice explanation:
To completely explain this story, you have to step back to the metaclass __call__ method. In particular the default type.__call__ in CPython calls __new__ and __init__ via their C slot functions, and it's slot_tp_init (defined in Objects/typeobject.c) that enforces the return value to be None. If you use a custom metaclass that overrides type.__call__, it can manually call the __new__ and __init__ methods of the class with no restriction on what __init__ can return -- as silly as that would be.
__init__ is called when you create a new instance of a class.
It's main use is initializing the instance variables, and it can be called only with an instance - so you can't call it before you create an instance anyways (what triggers it automatically).
For these reasons, __init__s have no reason to be able to return any value - it's simply not their use case.
I am unsuccessfully trying to get the magic with-statement methods __enter__ and __exit__ running on class-level:
class Spam():
#classmethod
def __enter__(cls):
return cls
#classmethod
def __exit__(cls, typ, value, tb):
cls.cleanup_stuff()
with Spam:
pass
However, this will result in an AttributeError:
Traceback (most recent call last):
File "./test.py", line 15, in <module>
with Spam:
AttributeError: __exit__
Is it possible to use the __enter__ and __exit__ methods on class-level anyway?
__enter__ and __exit__ are special methods, and as such only work correctly when defined on a object's type, not in it's instance dictionary.
Now Spam is a instance of type, and type(Spam).__enter__ and type(Spam).__exit__ do not exist. Therefore you get an attribute error.
To make this work, the methods would need to be declared on the metaclass of the class you want to use. Example:
class Spam(type):
def __enter__(cls):
print('enter')
return cls
def __exit__(cls, typ, value, tb):
print('exit')
class Eggs(metaclass=Spam):
pass
with Eggs:
pass
Now Eggs is an instance of Spam (type(Eggs) == Spam, and therefore type(Eggs).__enter__ and type(Eggs).__exit__ do exist).
However defining a metaclass just to use an instance of it as a context manager seems a little over the top. The more straight forward solution starting from your example would be to just use
with Spam():
pass
Or if you want to reuse the same instance later:
spam = Spam()
with spam:
pass
It seems that CPython doesn't call a bound method like instance.__exit__, it seeks over instance type, doing something like type(instance).__dict__['__exit__'] than calls it. And since type(Spam) is a special type object (not a Spam itself), it doesn't contain __exit__ method.
I tried to workaround that using metaclasses, but wasn't successful. __getattr__ doesn't work either.
See here: https://github.com/python/cpython/blob/2545fdbd4b4a6a77b132fccf816578f59b609be5/Objects/typeobject.c#L1362
Py_TYPE is similiar to type(self)
_PyType_LookupId walks over type(self).__dict__ (no __getattr__ call here)
Python 2 implementation is different, but main idea about getting type(self) applies to it too
I'm trying to define a class decorator which (among other things) wraps the constructor with some custom code.
I'm using a class decorator, and not inheritance, because I want the decorator to be applicable to multiple classes in a class hierarchy, and I want the wrapped code to execute for every decorated class in the hierarchy.
I can do something like:
def decorate(klass):
def Dec(klass):
def __init__(self,*a,**k):
# wrapping code here
super().__init__(*a,**k)
# wrapping code here
return Dec
And it works fine for simple test cases. But, i'm afraid replacing the class by another might cause subtle breakage (for instance if a decorated class decides to do arcane stuff referencing itself). In addition, it breaks the nice default repr string of the class (it shows up as "decorate..Dec" instead of whatever klass was originally).
I tried changing the class itself:
def decorate(klass):
old_init = klass.__init__
def new_init(self,*a,**k):
# wrapper code here
old_init(self,*a,**k)
# wrapper code here
klass.__init__ = new_init
return klass
This way, it maintains the proper class name and all, and it works fine as long as my constructor doesn't take any arguments. However, it breaks when applying it to, for instance, a type like str:
#decorate
class S(str):
pass
>>> s = S('foo')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "decorate.py", line 13, in new_init
old_init(self,*a,**k)
TypeError: object.__init__() takes no parameters
Identical code works perfectly if i'm not inheriting from str but from some dummy class.
If I override __new__ instead, it works for str but fails for a custom class:
def decorate(klass):
old_new = klass.__new__
def new_new(cls,*a,**k):
# wrapper code
i = old_init(cls,*a,**k)
# wrapper code
return i
klass.__new__ = new_new
return klass
#decorate
class Foo:
pass
#decorate
class Bar(Foo):
def __init__(self, x):
self.x = x
Then
>>> bar = Bar(1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "decorate.py", line 12, in new_new
i = old_init(cls,*a,**k)
File "decorate.py", line 12, in new_new
i = old_init(cls,*a,**k)
TypeError: object() takes no parameters
How is it possible that version changing __init__ fails ? How is it possible that by replacing a function with another one of the most generic signature type (*a,**k) and proxy the call to the original, I get a failure ?
Why does object.init seems to violate the convention of accepting at least one positional argument ?
How can I make it so it works in both scenarios ? I can't override both __init__ and __new__ to extend the same behaviour; which criterion should be used to dynamically know which one is the right one to hook ?
Should this really be implemented completely differently, for instance with a metaclass ?
This is inspired by a question I just saw, "Change what is returned by calling class instance", but was quickly answered with __repr__ (and accepted, so the questioner did not actually intend to call the instance).
Now calling an instance of a class can be done like this:
instance_of_object = object()
instance_of_object()
but we'll get an error, something like TypeError: 'object' object is not callable.
This behavior is defined in the CPython source here.
So to ensure we have this question on Stackoverflow:
How do you actually call an instance of a class in Python?
You call an instance of a class as in the following:
o = object() # create our instance
o() # call the instance
But this will typically give us an error.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'object' object is not callable
How can we call the instance as intended, and perhaps get something useful out of it?
We have to implement Python special method, __call__!
class Knight(object):
def __call__(self, foo, bar, baz=None):
print(foo)
print(bar)
print(bar)
print(bar)
print(baz)
Instantiate the class:
a_knight = Knight()
Now we can call the class instance:
a_knight('ni!', 'ichi', 'pitang-zoom-boing!')
which prints:
ni!
ichi
ichi
ichi
pitang-zoom-boing!
And we have now actually, and successfully, called an instance of the class!
The short answer is that the object class has no __call__ method (you can check that with "dir(object)"). When you create an instance of a class the __init__ method is called and when you call the instance, the __call__ method is called.
Up Votes for Everyone!
Thanks for posting the question and thanks for answering.
I thought I would just share my implementation in case that helps others ...
I have a class (called RTS) and it contains an SQL Query that I access using a 'get'. The class works fine as an independent endpoint. Now I want to call that class from within the program.
Using the answer above I added the following:
class RTS(Resource):
def __call__(self):
print("In RTS")
def get(self, user_id):
try: ...
In order to call the class from elsewhere in the program I added:
getGR = RTS.get(self, user_unique_id)
Voila - I got the same info I could check on Postman returned within the program.