I'm writing a simple XML serializer and caught a strange problem. If an object being serialized inherited from object explicitly everything works fine - I can serialize and deserialize it.
But if I don't specify object explicitly, it fails to deserialize despite the fact object is in mro:
i.e.
class ScanResults(object):
works fine, but
class ScanResults:
doesn't work
MRO :[<class 'aplib.scanresults.ScanResults'>, <class 'aplib.pyxmlser.pyxmlser'>, <type 'object'>]
Traceback (most recent call last):
...
classinst = classobj.__new__(classobj, None, None)
AttributeError: class AccessPoint has no attribute '__new__'
Any ideas is much appreciated.
Old-style classes (those not derived from object) don't have __new__().
One way to test if an instance is an old-style clase (which fail) is to use
type(my_instance) is types.InstanceType
which will evaluate to True for old-style classes.
some input about why use __new()__ and how to check in a more clean way for the creation of a new instance:
https://mail.python.org/pipermail/tutor/2008-April/061426.html
tips for overwritting the __new()__ method to be found here.
Related
I'm struggling with understanding which methods are called when an old-style object is created.
Doing a little research I found that this differentiation only applies for python 2.7 not for python 3, however, I'm testing the methods in code:
class OldClass():
pass
class NewClass(object):
pass
old = OldClass()
new = NewClass()
print(type(old))
print(type(new))
print(old.__class__) #It retrieves __main__
print(old.__new__) #Never called
print(old.__init__) #Never called
And this the output:
<type 'instance'>
<class '__main__.NewClass'>
__main__.OldClass
Traceback (most recent call last):
File "main.py", line 29, in <module>
print(old.__init__)
AttributeError: OldClass instance has no attribute '__init__'
I'm continuing in research for this topic, here are some links with some information related:
New Style class vs Old style: https://www.youtube.com/watch?v=KwpnXqnVx2o
stack overflow new style vs old style class: What is the difference between old style and new style classes in Python?
If I find something I will update the post
__init__ only exists and is only called if you define it. Since you didn't define __init__, it's not called. When it is defined, it's the only useful point of interception to creating the instance, and most of what you're trying to do works fine.
New-style classes can also use __new__ to hook instance construction (as opposed to initialization), and also allow for metaclasses (which can hook stuff in even crazier ways), but they're not available for old-style classes (defining __new__ doesn't change anything, and using metaclasses implicitly opts in to new-style classes).
I try to understand more thoroughly descriptor and explicit attribute name look-up order.
I read descriptor howto and it states as follows:
The details of invocation depend on whether obj is an object or a class:
...
For classes, the machinery is in type.__getattribute__() which transforms B.x into B.__dict__['x'].__get__(None, B)
I test this on __class__ since it is a data descriptor of object
In [47]: object.__class__
Out[47]: type
So, it returns type as expected since type class creates all classes including object class. Base on the 'descriptor howto', object.__class__ is turned into object.__dict__['__class__'].__get__(None, object).
However, When I run it, the output is the descriptor itself, not type
In [48]: object.__dict__['__class__'].__get__(None, object)
Out[48]: <attribute '__class__' of 'object' objects>
I guess it returns the descriptor itself because inside of this __get__ having some kind of code like:
if instance is None:
return self
So, I understand the reason of returning the descriptor itself when calling from class. The thing confuses me is the different ouputs
When it says 'B.x into B.__dict__['x'].__get__(None, B)', I expect outputs are the same. Why are they different?
The descriptor how-to is a simplification. It glosses over things like metaclasses, and the fact that classes are objects. Classes are objects, and they go through both "object-style" and "class-style" attribute lookup and descriptor handling. (The implementation can be found in type_getattro, if you want to independently verify this.)
A lookup for object.__class__ doesn't just go through object.__mro__; it also looks through type(object).__mro__. Descriptors found in type(object).__mro__ use "object-style" descriptor handling, treating the class as an instance of its metaclass, while descriptors found in object.__mro__ use "class-style" descriptor handling.
When you look up object.__class__, Python searches through type(object).__mro__. Since object is in type(object).__mro__, this search finds object.__dict__['__class__']. Since object.__dict__['__class__'] is a data descriptor (it has a __set__), this takes priority over the search through object.__mro__. Thus, treating object as an instance of object rather than as a class, Python performs
descr.__get__(object, type(object))
instead of
descr.__get__(None, object)
and the __get__ call returns type(object), which is type.
Your manual descr.__get__(None, object) call treats object as a class instead of as an instance of object. Invoked this way, the descriptor returns itself.
To demonstrate that __class__ isn't being special-cased here, we can create our own class that's an instance of itself, just like object is:
class DummyMeta(type):
pass
class SelfMeta(type, metaclass=DummyMeta):
#property
def x(self):
return 3
SelfMeta.__class__ = SelfMeta
print(SelfMeta.x)
print(SelfMeta.__dict__['x'].__get__(None, SelfMeta))
print(SelfMeta.__dict__['x'].__get__(SelfMeta, type(SelfMeta)))
Output:
3
<property object at 0x2aff9f04c5e8>
3
Just like with object.__class__, "object-style" descriptor handling happens here too. (Also, in case you're wondering, properties are data descriptors even if you don't write a setter.)
from my research, I understand that using getattr() allows one to get a method on a certain object such as:
getattr(obj, method)
is the equivalent to
obj.method()
However, I tried to use it in my code and the statement returns a memory location:
<bound method _Class.method of <Class instance at 0x7f412df0c320>>
I don't understand what I am doing wrong. Any help is appreciated.
Methods are just attributes, and getattr() retrieves attributes. As such
getattr(obj, 'method')
is the same as
obj.method
so without the () call primary, so no call to the method object made. Just add a call expression, to call the result of the getattr() method:
getattr(obj, 'method')()
__getattr__ is a hook method to allow a class to customise how missing attributes are handled. You should generally not need to call it directly.
I was trying to get some intuition about metaclass in Python. I have tried on both Python2.7 and and Python3.5. In Python3.5 I found every class we define is of type <class 'type'> whether we explicitly inherit or not. But if not inherited from type we can't use that class as a metaclass for another class.
>>> class foo:
pass
>>> class Metafoo(type):
pass
>>> foo
<class '__main__.foo'>
>>> Metafoo
<class '__main__.Metafoo'>
>>> type(foo)
<class 'type'>
>>> type(Metafoo)
<class 'type'>
>>>
>>> class foocls1(metaclass=foo):
pass
doing the above I get the following error:
Traceback (most recent call last):
File "<pyshell#52>", line 1, in <module>
class foocls1(metaclass=foo):
TypeError: object() takes no parameters
But that is not in the case while using Metafoo as metaclass for the new class
>>> class foocls3(metaclass=Metafoo):
pass
>>> foocls3
<class '__main__.foocls3'>
>>> type(foocls3)
<class '__main__.Metafoo'>
Can anyone explain why this is the case that we need to explicitly inherit if we want to use a class as metaclass in other class.
"type" is the base class for all class objects in Python 3, and in Python 2 post version 2.2. (It is just that on Python 2, you are supposed to inherit from object. Classes that do not explicitly inherit from "object" in Python 2 were called "old style classes", and kept for backward compatibility purposes, but were of little use.)
So, what happens is that inheritance: what are the superclasses of a class, and "metaclass" that is "since in Python a class is an object itself, what is the class of that object" are two different things. The classes you inherit from define the order of attribute (and method) lookup on the instances of your class, and therefore you have a common behavior.
The metaclass is rather "the class your class is built with", and although it can serve other purposes, is most used to modify the construction step of your classes itself. Search around and you will see metaclasses mostly implementing the __new__ and __init__ methods. (Although it is ok to have other methods there, but yu have to know what you are doing)
It happens that to build a class some actions are needed that are not required to build normal objects. These actions are performed on the native-code layer (C in CPython) and are not even reproducible in pure Python code, like populating the class's special method slots- the pointers to the functions that implement the __add__, __eq__ and such methods. The only class that does that in CPython is "type". So any code you want to use to build a class, i.e. as a metaclass, will have to at some point call type.__new__ method. (Just as any thing you want to create a new object in Python will at some point call the code in object.__new__.).
Your error happened not because Python goes checking whether you made a direct or indirect call to type.__new__ ahead of time. The error: "TypeError: object() takes no parameters" is simply due to the fact that __new__ method of the metaclass is passed 3 parameters (name, bases, namespace), while the same method in object is passed no parameters. (Both get te extra "cls", equivalent to self as well, but this is not counted).
You can use any callable as the metaclass, even an ordinary function. It is just that it will have to take the 3 explicit parameters. Whatever this callable returns is used as the class from that point on, but if at some point you don't call type.__new__ (even indirectly), you don't have a valid class to return.
For example, one could create a simple method just in order to be able to use class bodies as dictionary declarations:
def dictclass(name, bases, namespace):
return namespace
class mydict(metaclass=dictclass):
a = 1
b = 2
c = 3
mydict["a"]
So, one interesting fact with all that is that type is its own metaclass. (that is hardcoded in the Python implementation). But type itself also inherits from object:
In [21]: type.__class__
Out[21]: type
In [22]: type.__mro__
Out[22]: (type, object)
And just to end this: it is possible to create classes without calling type.__new__ and objects without calling object.__new__, but not ordinarily from pure Python code, as data structures at the C-API level have to be filled for both actions. One could either do a native-code implementation of functions to do it, or hack it with ctypes.
It's said that :
When it would yield a class method object, it is transformed into a
bound user-defined method object whose im_class and im_self attributes
are both C.
in the Reference
And I did an EX.
>>> class C(object) :
... #classmethod
... def cm(cls) : print cls
...
>>> C.cm
<bound method type.cm of <class '__main__.C'>>
>>> C.cm.im_self
<class '__main__.C'>
>>> C.cm.im_class
<type 'type'>
It's not hard for me to understand the phenomenon. But unfortunately, in the reference, it's told that im_self should be the same as im_class. How to explain the inconsistency?
I read that the same way you do. It appears that what Python is actually doing is not exactly what the documentation says.
It sets im_self to the class, and im_class to the class's type, i.e., its metaclass. The default metaclass for classes in Python is type. This is analogous to what happens with methods that are bound to instances: im_self is the instance and im_class is the type of the instance. In the case of #classmethod, in other words, the class is treated as the instance (which it is; it's an instance of type).
Possibly the behavior was changed without the documentation being updated, or the documentation was just wrong to begin with. I write documentation for a living and I can confirm that it is almost impossible to keep it 100% correct for something the size of Python -- especially for obscure details like this one.
The Python developers have a procedure for reporting bugs in the documentation. Give it a try!