This question already has answers here:
Overriding special methods on an instance
(5 answers)
Closed 5 years ago.
I'm working on a project right now that deals with functions in an abstract mathematical sense. Without boring the reader with the details, I'll say that I had the following structure in an earlier version:
class Foo(Bar):
def __init__(self, a, b):
self.a = a
self.b = b
self.sub_unit = Foo(a, not b)
Of course, that's not quite the change I'm making to the arguments to Foo, but suffice to say, it is necessary that this property, if accessed repeatedly, result in an indefinitely long chain of Foo objects. Obviously, this results in an infinite recursion when one instantiates Foo. I solved this in the earlier version by removing the last line of init and adding the following to the Foo class:
def __getattr__(self, attr: str):
if attr == 'sub_unit':
return Foo(self.a, not self.b)
else:
return super().__getattr__(attr)
This worked quite well, as I could calculate the next object in the chain as needed.
In going over the code, though, I realize that for other reasons, I need an instance of Bar, not a sub-class of it. To see if I could override the getattr for a single instance, I tried the following:
>>> foo = Bar(a=1, b=2) # sub_unit doesn't get set here.
>>> foo.__getattr__ = lambda attr: 'foo'
>>> foo.a
1
>>> foo.__getattr__('a')
'foo'
What is happening here that I don't understand? Why isn't foo.a calling foo.__getattr__('a')?
Is there a good way to overwrite __getattr__ for a single instance, or is my best bet to re-factor all the code I have that reads sub_unit and friends to call those as functions, to handle this special case?
When you lookup the attribute a with foo.a, python looks it up in the instance's property dictionary. When it is not found, the __getattr__ method will then be called.
On the contrary, if a exists in the instance, __getattr__ will not be called.
Related
I was writing a class and at some point I decided that it would be nice to have a possibility to create an instance of the class using other instance of this same class. So basically the class would look like this:
class Test:
def __init__(self, a, b):
if type(b) == Test:
self = a / b
else:
self.a = a
self.b = b
def __rtruediv__(self, other):
return Test(other * self.b, self.a)
def __str__(self):
return f"{self.a = }, {self.b = }"
if __name__ == '__main__':
hf = Test(1, 2)
print(hf) # self.a = 1, self.b = 2
print(Test(1, hf)) # AttributeError: 'Test' object has no attribute 'a'
print(1 / Test(1, hf)) # AttributeError: 'Test' object has no attribute 'b'
However, when i tried to do it, I got the AttributeError (AttributeError: 'Test' object has no attribute 'a'). Interestingly enough, the code print(1/Test(1, hf)) gives the same attribute error about attribute b while going into the rtruediv func, so the object Test(1, hf) has methods that I defined. Why does that happen? Is something wrong with "self = ..."?
The reason this doesn't work is that self is just an ordinary variable like any other.
if type(b) == Test:
self = a / b
else:
self.a = a
self.b = b
In the if branch, we reassign a local variable, but it doesn't change the instance being constructed. It just makes a new instance, that will be discarded in a moment. The actual instance being constructed (the original value of self) is still there and remains in an uninitialized state (i.e. doesn't have a and b like it should).
I think your goal here is well-intentioned, though it will likely confuse Python programmers as it's not incredibly idiomatic. My recommendation is to simply set self.a and self.b in the constructor in all cases, as it's a more intuitive code design for the average Python coder.
However, what you want can be done. In Python, when you call Test(a, b), some complicated internal things happen.
First, we invoke __call__ on the metaclass of Test. In your example, the metaclass of Test isn't specified, so it defaults to [type], the built-in metaclass.
The default behavior of type.__call__ is to invoke the class method __new__ on your class (__new__ is implicitly a class method, even if you don't ask it to be, so you don't need the #classmethod decorator). Your class doesn't currently define __new__, so we inherit the default one from object.
object.__new__ actually creates the new object. This is written in low-level C and can't be replicated in Python. It truly is primitive.
Finally, object.__new__ calls Test.__init__ on the newly-constructed object and then returns the new object.
Aside from (3), which is truly a C-level primitive, we can easily inject into any part of this. We could write a metaclass and redefine __call__, but that's overkill for what we're doing. We're going to hook into (2) and define __new__. So rather than your __init__, consider
class Test:
def __new__(cls, a, b):
if type(b) == Test:
return a / b
else:
obj = object.__new__(cls)
obj.a = a
obj.b = b
return obj
In __init__, we're given the object to construct, and our only option is to construct that object (or throw an exception if we can't). But __new__ is lower-level. In __new__, it's our choice whether and how to create a new object. In the case where we're given two numbers, we delegate to the default __new__ on object (creating a new object, using the primitive C-level code discussed above), and then we initialize its instance variables. But in the case where b is a Test, we do something different and short-circuit the process entirely, calling a different method instead.
Again, I don't necessarily think this is super idiomatic Python code, but it is possible to do what you want.
I recently spent way too long debugging a piece of code, only to realize that the issue was I did not include a () after a command. What is the logic behind which commands require a () and which do not?
For example:
import pandas as pd
col1=['a','b','c','d','e']
col2=[1,2,3,4,5]
df=pd.DataFrame(list(zip(col1,col2)),columns=['col1','col2'])
df.columns
Returns Index(['col1', 'col2'], dtype='object') as expected. If we use .columns() we get an error.
Other commands it is the opposite:
df.isna()
Returns:
col1 col2
0 False False
1 False False
2 False False
3 False False
4 False False
but df.isna returns:
<bound method DataFrame.isna of col1 col2
0 a 1
1 b 2
2 c 3
3 d 4
4 e 5>
Which, while not throwing an error, is clearly not what we're looking for.
What's the logic behind which commands use a () and which do not?
I use pandas as an example here, but I think this is relevant to python more generally.
Because functions need parenthesis for their arguments, while variables do not, that's why it's list.append(<item>) but it's list.items.
If you call a function without the parenthesis like list.append what returns is a description of the function, not a description of what the function does, but a description of what it is.
As for classes, a call to a class with parenthesis initiates an object of that class, while a call to a class without the parenthesis point to the class itself, which means that if you were to execute print(SomeClass) you'd get <class '__main__.SomeClass'> which is a description of what it is, the same kind of response you'd get if you were to call a function without parenthesis.
What's the logic behind which commands use a () and which do not?
An object needs to have a __call__ method associated with it for it to called as a function using ():
class Test:
def __call__(self, arg):
print("Called with", arg)
t = Test() # The Test class object uses __call__ to create instances
t(5) # Then this line prints "Called with 5"
So, the difference is that columns doesn't have a __call__ method defined, while Index and DataFrame do.
TL;DR you just kinda have to know
Nominally, the parens are needed to call a function instead of just returning an object.
foo.bar # get the bar object
foo.bar() # call the bar object
Callable objects have a __call__ method. When python sees the (), it knows to call __call__. This is done at the C level.
In addition, python has the concept of a property. Its a callable data object that looks like a regular data object.
class Foo:
def __init__(self):
self._foo = "foo"
#property
def foo(self):
return "I am " + self._foo
#foo.setter
def foo(self, val):
assert isinstance(val, str)
self._foo = val + " you bet"
f = Foo()
f.foo = "Hello" # calls setter
print(f.foo) # calls getter
Similarly, when python sees array notation foo[1] it will call an object's __getitem__ or __setitem__ methods and the object is free to overload that call in any way it sees fit.
Finally, the object itself can intercept attribute access with __getattr__, __getattribute__ and __setattr__ methods, leaving everything up in the air. In fact, python doesn't really know what getting and setting attributes means. It is calling these methods. Most objects just use the default versions inherited from object. If the class is implemented in C, there is no end to what could be going on in the background.
Python is a dynamic language and many packages add abstractions to make it easier (?) to use their services. The downside is that you may spend more time with help text and documentation than one may like.
Object method vs Object attribute.
Objects has methods and attributes.
Methods require a parenthesis to call them -- even if the method does not require arguments.
Where as attributes are like variables are pointed to objects as the program progresses. You just call these attributes by their name (without parenthesis). Of course you may have to qualify both the methods and attributes with the object names as required.
This question already has answers here:
Overriding special methods on an instance
(5 answers)
Closed 5 years ago.
Suppose I want to run the following code using Python 3.6.3:
class Foo:
def bar(self):
return 1
def __len__(self):
return 2
class FooWrapper:
def __init__(self, foo):
self.bar = foo.bar
self.__len__ = foo.__len__
f = Foo()
print(f.bar())
print(f.__len__())
print(len(f))
w = FooWrapper(Foo())
print(w.bar())
print(w.__len__())
print(len(w))
Here's the output:
1
2
2
1
2
TypeError: object of type 'FooWrapper' has no len()
So __len__() works, but len() does not? What gives and how go I properly copy __len__ method from Foo to FooWrapper?
By the way, the following behavior is universal for all 'special' methods, not only __len__: for example, __iter__ or __getitem__ do not work either (unless called directly)
The reason this happens is because the special methods have to be on an object's class, not on the instance.
len will look for __len__ in the FooWrapper class. BTW, although this looks like it "works", you are actually adding foo.__len__, i.e. , the methoud already bound to the foo instance of Foo to your FooWrapper object. That might be the intent, but you have to be aware of this.
The easiest way for this to work is to make FooWrapper have itself a __len__ method that will call the wrapped instance's __len__:
class FooWrapper:
def __init__(self, foo):
self.foo = foo
self.bar = foo.bar
self.__len__ = foo.__len__
def __len__(self):
return len(self.foo)
Does that mean that for any and all special methods hace to explicitly exist in the wrapper class? Yes, it does - and it is one of the pains for creating proxies that behave just the same as the wrapped object.
That is because the special methods' checking for existence and calling is done directly in C, and not using Python's lenghty lookup mechanisms, as tat would be too inefficient.
It is possible to create a wrapper-class factory thing that would inspect the object and create a brand new wrapper class,with all meaningful special methods proxied, though - but I think that would be too advanced for what you have in mind right now.
You'd better just use explicit special methods, or explicit access to the wrapped object in the remainder of the code. (Like, when you will need to use __iter__ from a wrapped object, instead of doing just for x in wrapper, do for x in wrapper.wrapped )
This question already has answers here:
Why can't I replace the __str__ method of a Python object with another function?
(2 answers)
Closed 9 years ago.
I'm converting old Python code and replaced some classes with new style classes. The problem is that this broke the behavior of replacing __str__ and I have no idea why.
class OldStyle():
def __str__(self):
return 'original'
old = OldStyle()
print old
old.__str__ = lambda: 'modified'
print old
class NewStyle(object):
def __str__(self):
return 'original'
new = NewStyle()
print new
new.__str__ = lambda: 'modified'
print new
I expected
original
modified
original
modified
but I got
original
modified
original
original
That is, __str__ was not correctly replaced in the new style. Printing new.__str__ returns the new lambda correctly, but str(new) still doesn't call it. I thought it was some method lookup caching problem, but this happens even if the object was never printed before.
Why does that happen? I have never heard about this behavior difference and it only happens with __str__, other methods are replaced fine.
This is documented in the python Data model under special method names. Specifically:
For instance, if a class defines a method named __getitem__, and x is an instance of this class, then x[i] is roughly equivalent to x.__getitem__(i) for old-style classes and type(x).__getitem__(x, i) for new-style classes.
I believe that this allows new-style classes to be a little more efficient in terms of these operations because with the old style classes, python was forced to look up the attribute and then call the attribute, with new-style classes, python can just reference it as part of a C struct somewhere effectively pushing the lookups and calls into native C code. For example:
class Foo:
def __add__(self,other):
return 4 + other
class Bar(object):
def __add__(self,other):
return 4 + other
import timeit
print timeit.timeit('f + 5','from __main__ import Foo; f = Foo()')
print timeit.timeit('b + 5','from __main__ import Bar; b = Bar()')
For me (python2.7.3, OS-X 10.5.8), the new-style class is almost 4 times faster!
2.27801704407
0.602614879608
This works:
NewStyle.__str__ = lambda self: 'modified'
It seems like what we're seeing is that the __str__ method is tied to the class type now, rather than the instance. I'm not sure why that is. You can call new.__str__() explicitly and get the replaced method, but str(new) always calls NewStyle.__str__(new).
class Foo(object):
def tick(self):
print("something")
class Bar(object):
def __init__(self):
self.foo = Foo()
def tick(self):
#Here's what I do....
self.foo.tick()
#here's what my goal would be
self.foo()
b = Bar()
b.tick()
That's essentially my goal. From what I've gathered I could change the tick function to __call__ and that would allow me to do what I wanted. A couple of the other answers said that this would make a new instance of the object, does that mean that it would use self.foo's memory? or would it make a whole new object, newly instanced? or make a copy of self.foo?
Also a couple of drawbacks to this which may or may not manifest themselves come to mind. For a particular part of my program, I check to see if the object has a __call__ to determine if the argument I'm passing is a function or a variable, and I don't really think I would want to allow that to be called (even though, I suppose the class technically would be a function at that point.) Is there any way to distinguish between a function and a callable class?
Is there anything else that would make doing this undesirable (and is it a pythonic way to work?)? My next thought had been that given that other variable prefixed with __ cant be used outside their class, but that doesnt seem to be the case here.
Changing tick(self) to __call__(self) is the correct solution.
This has nothing to do with memory allocation. All __call__ signifies is the function which Python calls when you use (for an object foo) the syntax foo().
For your later question: to check whether something is an object, use isinstance or issubclass (possibly with the object class). And in my mind this answer is better than the accepted one for the "How do I determine if something is a function?" question you've probably seen.
To make a class instance callable, all you need to do is implement a __call__ method. Then, to call the __call__ method, you just do: instance(arg1,arg2,...) -- in other words, you call the instance the same way you would call a function.
Note that you can alias __call__ with some other method defined on the class if you want:
>>> class Foo(object):
... def tick(self):
... print ("something")
... __call__ = tick
...
>>> a = Foo()
>>> a()
something
>>> a.tick()
something