number = 5
print(number.__class__)
print(number.__str__())
print(number.__add__(1))
#Output
<class 'int'>
5
6
I guess I'm trying to see if I understand this correctly.
So there's different ways to access the dunder methods. In the first case, it looks as though I'm accessing the method as though it were an attribute of the class int. In this case, is __class__ using an #property decorator to access it as though it were an attribute? If that's the case, then it makes sense.
Second and third make sense. __str___ is a dunder method defined in the class int that takes no arguments. You call it like a method. __add__ takes a argument, which is the number you're adding to it. So you have to call it like a method with an argument. So none of these use #property decorators.
I guess what makes it more confusing is that technically the __str__ can be made into an attribute with the #property decorator. I tested it out myself on my own class and it definitely works either way, so it seems a bit arbitrary which ones are accessed like attributes and which ones are accessed like methods. Assuming my theory is correct.
So there's different ways to access the dunder methods. In the first case, it looks as though I'm accessing the method as though it were an attribute of the class int.
That's because __class__ is an attribute, not a method.
Another distinction to make is that the way you access methods is always the same. Dunder methods are not magical in this. You use a . between the instance and the method name. Whether you are referring just to the method or calling it to get its result is another matter.
While this is a reasonable exercise to understand how Python works, remember that you should not usually call dunder methods directly. Python offers other syntax that will call these methods under the hood. For example, use str(number) instead of number.__str__() and a + b rather than a.__add__(b).
The main reason for dunder methods is that they allow you to override default behavior in your own classes. They aren't intended to be called directly.
Related
I want to use a method from another class.
Neither inheritance nor delegation is a good choice (to my understanding) because the existing class is too complicated to override and too expensive to instanciate.
Note that modifying the existing class is not allowed (legacy project, you know).
I came up with a way:
class Old:
def a(self):
print('Old.a')
class Mine:
b = Old.a
and it shows
>>> Mine().b()
Old.a
>>> Mine().b
<bound method Old.a of <__main__.Mine object at 0x...>>
It seems fine.
And I tried with some more complicated cases including property modification (like self.foo = 'bar'), everything seems okay.
My question:
What is actually happening when I define methods like that?
Will that safely do the trick for my need mentioned above?
Explanation
What's happening is that you are defining a callable class property of class Mine called b. However, this works:
m = Mine()
m.b()
But this won't:
Mine.b()
Why doesn't the second way work?
When you call a function of a class, python expects the first argument to be the actual object upon which the function was called. When you do this, the self argument is automatically passed into the function behind the scenes. Since we called Mine.b() without an instantiated instance of any object, no self was passed into b().
Will this "do the trick"?
As for whether this will do the trick, that depends.
As long as Mine can behave the same way as Old, python won't complain. This is because the python interpreter does not care about the "type" of self. As long as it walks like a duck and quacks like a duck, it's a duck (see duck typing). However, can you guarantee this? What if someone goes and changes the implementation of Old.a. Most of the time, as a client of another system we have no say when the private implementation of functions change.
A simpler solution might be to pull out the functionality you are missing into a separate module. Yes, there is some code duplication but at least you can be confident the code won't change from under you.
Ultimately, if you can guarantee the behavior of Old and Mine will be similar enough for the purposes of Old.a, python really shouldn't care.
This question already has answers here:
What is the purpose of the `self` parameter? Why is it needed?
(26 answers)
Closed 6 months ago.
Are you supposed to use self when referencing a member function in Python (within the same module)?
More generally, I was wondering when it is required to use self, not just for methods but for variables as well.
Adding an answer because Oskarbi's isn't explicit.
You use self when:
Defining an instance method. It is passed automatically as the first parameter when you call a method on an instance, and it is the instance on which the method was called.
Referencing a class or instance attribute from inside an instance method. Use it when you want to call a method or access a name (variable) on the instance the method was called on, from inside that method.
You don't use self when
You call an instance method normally. Using Oskarbi's example, if you do instance = MyClass(), you call MyClass.my_method as instance.my_method(some_var) not as instance.my_method(self, some_var).
You reference a class attribute from outside an instance method but inside the class definition.
You're inside a staticmethod.
These don'ts are just examples of when not to use self. The dos are when you should use it.
Use self to refer to instance variables and methods from other instance methods. Also put self as the first parameter in the definition of instance methods.
An example:
class MyClass(object):
my_var = None
def my_method(self, my_var):
self.my_var = my_var
self.my_other_method()
def my_other_method(self):
# do something...
There is nothing 'special' about the name self. It is the name preferred by convention by Pythonistas, to indicate what that parameter is expected to contain.
The Python runtime will pass a 'self' value when you call an instance method on an instance, whether you deliberately provide for it or not. This will usually result in an easily diagnosed/understood error (since the function will get called with the wrong number of parameters), but the use of *args can lead to rather more strange type errors.
The parameter is passed implicitly when you call an instance method on an instance. It contains the instance upon which you call the method. So you don't mention self in the function call because (a) as noted above, that wouldn't make any sense (there isn't a self in scope, in general, and self is not a keyword or special name or anything); (b) you've already indicated the instance to use (by writing my_instance.).
You can, of course, explicitly call an instance method by accessing it from the class. In this case, you'll need to pass the instance explicitly as the first parameter. You generally speaking don't want to do this. And you especially don't want to write code that considers the possibility that the first parameter is something else that's been explicitly passed in this way. This is akin to checking if (this == null) in C++: you don't do it, because if it could possibly mean anything, then the calling code is wrong, morally if not legally. (At least in Python you won't have problems with undefined behaviour, but it's still morally wrong.)
Within the instance method, since self is a parameter which has been assigned the instance as a value, you can write self.whatever to access attributes of the instance. Unlike in some other 'implicit this' style languages, the attribute names are not implicitly "in scope".
There are no other use cases for self, since again it's not a special name, and that is the one specific purpose that the naming convention addresses. If you needed to access a 'variable' (really an attribute) from another module, you would use the module name. If you wanted to access one from the current module, no prefix is needed, or really possible for that matter. (Well, you could explicitly look it up in the dict returned by globals(), but please don't do that.)
For instance variable and for methods it is mandatory to use self anytime.
I am trying to make a class that wraps a value that will be used across multiple other objects. For computational reasons, the aim is for this wrapped value to only be calculated once and the reference to the value passed around to its users. I don't believe this is possible in vanilla python due to its object container model. Instead, my approach is a wrapper class that is passed around, defined as follows:
class DynamicProperty():
def __init__(self, value = None):
# Value of the property
self.value: Any = value
def __repr__(self):
# Use value's repr instead
return repr(self.value)
def __getattr__(self, attr):
# Doesn't exist in wrapper, get it from the value
# instead
return getattr(self.value, attr)
The following works as expected:
wrappedString = DynamicProperty("foo")
wrappedString.upper() # 'FOO'
wrappedFloat = DynamicProperty(1.5)
wrappedFloat.__add__(2) # 3.5
However, implicitly calling __add__ through normal syntax fails:
wrappedFloat + 2 # TypeError: unsupported operand type(s) for
# +: 'DynamicProperty' and 'float'
Is there a way to intercept these implicit method calls without explicitly defining magic methods for DynamicProperty to call the method on its value attribute?
Talking about "passing by reference" will only confuse you. Keep that terminology to languages where you can have a choice on that, and where it makes a difference. In Python you always pass objects around - and this passing is the equivalent of "passing by reference" - for all objects - from None to int to a live asyncio network connection pool instance.
With that out of the way: the algorithm the language follows to retrieve attributes from an object is complicated, have details - implementing __getattr__ is just the tip of the iceberg. Reading the document called "Data Model" in its entirety will give you a better grasp of all the mechanisms involved in retrieving attributes.
That said, here is how it works for "magic" or "dunder" methods - (special functions with two underscores before and two after the name): when you use an operator that requires the existence of the method that implements it (like __add__ for +), the language checks the class of your object for the __add__ method - not the instance. And __getattr__ on the class can dynamically create attributes for instances of that class only.
But that is not the only problem: you could create a metaclass (inheriting from type) and put a __getattr__ method on this metaclass. For all querying you would do from Python, it would look like your object had the __add__ (or any other dunder method) in its class. However, for dunder methods, Python do not go through the normal attribute lookup mechanism - it "looks" directly at the class, if the dunder method is "physically" there. There are slots in the memory structure that holds the classes for each of the possible dunder methods - and they either refer to the corresponding method, or are "null" (this is "viewable" when coding in C on the Python side, the default dir will show these methods when they exist, or omit them if not). If they are not there, Python will just "say" the object does not implement that operation and period.
The way to work around that with a proxy object like you want is to create a proxy class that either features the dunder methods from the class you want to wrap, or features all possible methods, and upon being called, check if the underlying object actually implements the called method.
That is why "serious" code will rarely, if ever, offer true "transparent" proxy objects. There are exceptions, but from "Weakrefs", to "super()", to concurrent.futures, just to mention a few in the core language and stdlib, no one attempts a "fully working transparent proxy" - instead, the api is more like you call a ".value()" or ".result()" method on the wrapper to get to the original object itself.
However, it can be done, as I described above. I even have a small (long unmaintained) package on pypi that does that, wrapping a proxy for a future.
The code is at https://bitbucket.org/jsbueno/lelo/src/master/lelo/_lelo.py
The + operator in your case does not work, because DynamicProperty does not inherit from float. See:
>>> class Foo(float):
pass
>>> Foo(1.5) + 2
3.5
So, you'll need to do some kind of dynamic inheritance:
def get_dynamic_property(instance):
base = type(instance)
class DynamicProperty(base):
pass
return DynamicProperty(instance)
wrapped_string = get_dynamic_property("foo")
print(wrapped_string.upper())
wrapped_float = get_dynamic_property(1.5)
print(wrapped_float + 2)
Output:
FOO
3.5
The classify_class_attrs function from the inspect module can be used to determine what kind of object each of a class's attributes is, including whether a function is an instance method, a class method, or a static method. Here is an example:
from inspect import classify_class_attrs
class Example(object):
#classmethod
def my_class_method(cls):
pass
#staticmethod
def my_static_method():
pass
def my_instance_method(self):
pass
print classify_class_attrs(Example)
This will output a list of Attribute objects for each attribute on Example, with metadata about the attribute. The relevant ones in these case are:
Attribute(name='my_class_method', kind='class method', defining_class=<class '__main__.Example'>, object=<classmethod object at 0x100535398>)
Attribute(name='my_instance_method', kind='method', defining_class=<class '__main__.Example'>, object=<unbound method Example.my_instance_method>)
Attribute(name='my_static_method', kind='static method', defining_class=<class '__main__.Example'>, object=<staticmethod object at 0x100535558>)
However, it seems that many objects in Python's standard library can't be introspected this way. I'm guessing this has something to do with the fact that many of them are implemented in C. For example, datetime.datetime.now is described with this Attribute object by inspect.classify_class_attrs:
Attribute(name='now', kind='method', defining_class=<type 'datetime.datetime'>, object=<method 'now' of 'datetime.datetime' objects>)
If we compare this to the metadata returned about the attributes on Example, you'd probably draw the conclusion that datetime.datetime.now is an instance method. But it actually behaves as a class method!
from datetime import datetime
print datetime.now() # called from the class: 2014-09-12 16:13:33.890742
print datetime.now().now() # called from a datetime instance: 2014-09-12 16:13:33.891161
Is there a reliable way to determine whether a method on a stdlib class is a static, class, or instance method?
I think you can get much of what you want, distinguishing five kinds, without relying on anything that isn't documented by inspect:
Python instance methods
Python class methods
Python static methods
Builtin instance methods
Builtin class methods or static methods
But you can't distinguish those last two from each other with using CPython-specific implementation details.
(As far as I know, only 3.x has any builtin static methods in the stdlib… but of course even in 2.x, someone could always define one in an extension module.)
The details of what's available in inspect and even what it means are a little different in each version of Python, partly because things have changed between 2.x and 3.x, partly because inspect is basically a bunch of heuristics that have gradually improved over time.
But at least for CPython 2.6 and 2.7 and 3.3-3.5, the simplest way to distinguish builtin instance methods from the other two types is isbuiltin on the method looked up from the class. For a static method or class method, this will be True; for an instance method, False. For example:
>>> inspect.isbuiltin(str.maketrans)
True
>>> inspect.isbuiltin(datetime.datetime.now)
True
>>> inspect.isbuiltin(datetime.datetime.ctime)
False
Why does this work? Well, isbuiltin will:
Return true if the object is a built-in function or a bound built-in method.
When looked up on an instance, either a regular method or a classmethod-like method is bound. But when looked up on the class, a regular method is unbound, while a classmethod-like method is bound (to the class). And of course a staticmethod-like method ends up as a plain-old function when looked up either way. So, it's a bit indirect, but it will always be correct.*
What about class methods vs. static methods?
In CPython 3.x, builtin static and class method descriptors both return the exact same type when looked up on their class, and none of the documented attributes can be used to distinguish them either. And even if this weren't true, I think the way the reference is written, it's guaranteed that no functions in inspect would be able to distinguish them.
What if we turn to the descriptors themselves? Yes, there are ways we can distinguish them… but I don't think it's something guaranteed by the language:
>>> callable(str.__dict__['maketrans'])
False
>>> callable(datetime.datetime.__dict__['now'])
True
Why does this work? Well, static methods just use a staticmethod descriptor, exactly like in Python (but wrapping a builtin function instead of a function). But class and instance methods use a special descriptor type, instead of using classmethod wrapping a (builtin) function and the (builtin) function itself, as Python class and instance methods do. These special descriptor types, classmethod_descriptor and method_descriptor, are unbound (class and instance) methods, as well as being the descriptors that bind them. There are historical/implementation reasons for this to be true, but I don't think there's anything in the language reference that requires it to be true, or even implies it.
And if you're willing to rely on implementation artifacts, isinstance(m, staticmethod) seems a lot simpler…
All that being said, are there any implementations besides CPython that have both builtin staticmethods and classmethods? If not, remember that practicality beats purity…
* What it's really testing for is whether the thing is callable without an extra argument, but that's basically the same thing as the documented "function or bound method"; either way, it's what you want.
Let's say I have classes Base(object) and Derived(Base). These both implement a function foo, with Derived.foo overriding the version in Base.
However, in one of the methods, say Base.learn_to_foo, I want to call Base.foo instead of the derived version regardless of whether it was overridden. So, I call Base.foo(self) in that method:
class Base(object):
# ...
def learn_to_foo(self, x):
y = Base.foo(self, x)
# check if we foo'd correctly, do interesting stuff
This approach seems to work and from a domain standpoint, it makes perfect sense, but somehow it smells a bit fishy. Is this the way to go, or should I refactor?
The answer is NOT to use the super() function. The way you are doing is exactly right as you don't want to invoke the virtual method that is overridden in the super class. Since you seem to want the base class' exact implementation all the time, the only way is to get the base class' unbound method object back, bound it to self, which could be an instance of Base or Derived. Invoke the unbound method with self supplied explicitly as the first parameter gives you back a bound method. From this point forward, Base.foo will be acting on the instance self's data. This is perfectly acceptable and is the way Python deals with non-virtual method invocation. This is one of the nice things that Python allows you to do that Java does not.
It is recommended:
def learn_to_foo(self, x):
super(Derived, self).foo(x)
More information at http://docs.python.org/library/functions.html#super
An alternative is to use the 'super' built-in:
super(Derived, self).foo(x) # Python 2
super().foo(x) # Python 3