class _GhostLink(object):
toGhost = lambda filename: False
class _Mod_AllowGhosting_All(_GhostLink):
def _loop(self):
# ...
if self.__class__.toGhost(fileName) != oldGhost:...
produces:
Traceback (most recent call last):
File "bash\basher\mod_links.py", line 592, in Execute
changed = self._loop()
File "bash\basher\mod_links.py", line 587, in _loop
if self.__class__.toGhost(fileName) != oldGhost:
TypeError: unbound method <lambda>() must be called with _Mod_AllowGhosting_All instance as first argument (got Path instance instead)
while passing an instance as in if self.toGhost(fileName) != ... results in:
Traceback (most recent call last):
File "bash\basher\mod_links.py", line 592, in Execute
changed = self._loop()
File "bash\basher\mod_links.py", line 587, in _loop
if self.toGhost(fileName) != oldGhost:
TypeError: <lambda>() takes exactly 1 argument (2 given)
How come toGhost behaves as a classmethod instance method ?
EDIT: I know the difference of class,static etc methods - this is a syntactic question
Looks like you want a static method:
class _GhostLink(object):
toGhost = staticmethod(lambda filename: False)
or:
class _GhostLink(object):
#staticmethod
def toGhost(filename):
return False
The reason this happens is fundamentally that lambda and def do the same thing, except that def also assigns a variable, That is, both constructs produce a function.
The binding of a function (whether from lambda or def) into an instance method happens because functions are also descriptors; remember, in every single case:
foo = lambda (...): (...)
is identical to:
def foo(...):
return (...)
so when you say:
class _GhostLink(object):
toGhost = lambda filename: False
It's the same as if you had said:
class _GhostLink(object):
def toGhost(filename):
return False
So the moral of the story is that you should probably never use lambda as the right side of an assignment; it's not "better" or even different from using def. All it does is confuse.
Related
I have a python function that I am documenting with a standard docstring:
"""
Function description
Parameters
----------
p0 : numpy.ndarray, shape (...,4)
p0 description 1
p0 description 2
p1 : int, optional
p1 description
Returns
-------
Return value : my.Returnclass
"""
When I inherit this function from another class and override it, I would like to still make use of the same docstring but modify the return value accordingly. I created a decorator function for this purpose that looks as following and is being used like this:
from typing import Callable, get_type_hints
def modified_docstring(f: Callable) -> Callable:
"""
Decorator: Combine another function's docstring with a given docstring.
Parameters
----------
f : function
Function of which the docstring is taken.
"""
def replace_return_value(original_func, decorated_func):
original_docstring = original_func.__doc__
return_class_original = get_type_hints(original_func).get("return",None)
return_class_decorated = get_type_hints(decorated_func).get("return",None)
if return_class_decorated and return_class_original!=return_class_decorated:
class_name = return_class_decorated.__name__
class_module = return_class_decorated.__module__.split(".")[0]
return _re.sub(r'(Returns([ ]*\n[ ]*)-------([ ]*\n[ ]*)[A-Za-z0-9 ]* : )([A-Za-z0-9\.]*)\n',
fr'\1{class_module}.{class_name}\2',original_docstring)
else:
return original_docstring
def _decorator(func):
func.__doc__ = replace_return_value(f,func)
return func
return _decorator
class Superclass:
def my_super_function() -> 'Superclass':
"""
Function description
Returns
-------
Return value : test.Superclass
"""
return Superclass()
class DerivedClass(Superclass):
#modified_docstring(Superclass.my_super_function)
def my_derived_function() -> 'DerivedClass':
return DerivedClass()
However, because the typehints are not properly initiated in DerivedClass when the decorator function is being called upon module initialization, this returns the following error:
File "test.py", line 48, in <module>
class DerivedClass(Superclass):
File "test.py", line 50, in DerivedClass
def my_derived_function() -> 'DerivedClass':
File "test.py", line 24, in _decorator
func.__doc__ = replace_return_value(f.__doc__,f,func)
File "test.py", line 14, in replace_return_value
return_class_decorated = get_type_hints(decorated_func).get("return",None)
File "/usr/lib/python3.8/typing.py", line 1264, in get_type_hints
value = _eval_type(value, globalns, localns)
File "/usr/lib/python3.8/typing.py", line 270, in _eval_type
return t._evaluate(globalns, localns)
File "/usr/lib/python3.8/typing.py", line 518, in _evaluate
eval(self.__forward_code__, globalns, localns),
File "<string>", line 1, in <module>
NameError: name 'DerivedClass' is not defined
Is there a clean way to force the typehints to be initialized when the decorator function is being called, or to delay the initialization of the decorator functions until the typehints are properly initialized?
The problem is that 'DerivedClass' doesn't exist in the module globals yet when 'get_type_hints' is called.
Edit: Easiest solution:
wrap the call to 'get_type_hints' in a try except and when you get a NameError and func.__annotations__.get("return", None) returns a string just assume the function is a member function of the return values class and use the module of the function as the module.
Warning: Both easy-solutions will have a tough time with generics and may result in incorrect type hints when e.g. "List[MyDerivedCls]" is used it will most likely result in a type documentation of my_module.List[MyDerivedCls]
You can either avoid calling 'get_type_hints' or apply the actual docstring change in a separate call.
Easy solution - don't evaluate type hint and do what 'get_type_hints' does by hand:
Access the return value directly without evaluating the forward reference by using func.__annotations__.get("return", None). That way you'll get DerivedClass as a string. If you need the module where the class is defined you can look int the module of the function and search there and if the value doesn't exist assume that it's the class the function is defined in.
func_return = func.__annotations__.get("return", None)
if isinstance(decorated_func_return, str):
func_return_cls_name = func_return
func_return = sys.modules.get(func.__module__, {}).get(func_return, None)
if func_return is None:
# we haven't found the class in the module of the function - assume it's returning an instance of the class this function is a member of
func_return_module = func.__module__
if func_return is not None:
func_return_cls_name = func_return.__name__
func_return_module = func_return.__module__
You can find the fields defined on a function in the 'callable types' section on this page in the docs
More tedious solution: Delay the docstring modifications
Make it such that your modified_docstring decorator only marks the function as 'this function should have its docstring modified' and then make a call to a function that applies these modifications.
You cannot use a decorator there as 'DerivedClass' won't exist in the module yet when it's called.
That would look something like this.
def modified_docstring(original_func):
def _wrapper(decorated_func):
decorated_func.__my_docstring_marker__ = original_func
return decorated_func
return _wrapper
def apply_docstring_modifications(cls): # looks like a decorator but doesn't work if used as such
for func_name, func in inspect.getmembers(cls, inspect.isfunction):
if not hasattr(func, "__my_docstring_marker"):
continue
func = function_that_does_docstring_modification(func, func.__my_docstring_marker__)
setattr(cls, func_name, func)
class MyDerivedCls(...):
...
apply_docstring_modifications(MyDerivedCls)
Hope this helped.
Having this class:
class A(frozenset):
def __init__(self, *args):
frozenset.__init__(self, *args)
Executing A(range(2)) results in the following error:
Traceback (most recent call last):
File "<pyshell#65>", line 1, in <module>
A(range(2))
File "<pyshell#60>", line 3, in __init__
frozenset.__init__(self, *args)
TypeError: object.__init__() takes no parameters
Meanwhile, frozenset(range(2)) works, and if I inherit A from set instead, A(range(2)) also works.
If I pass to A's constructor 0 or more than 1 parameter, it works as it should (with 0 paramemeters creates an empty set, with 2 or more parameters raises TypeError: A expected at most 1 arguments, got 2).
Actually you need to override __new__ method (not __init__, __init__ method will accept an instance generated and returned by __new__ method) when subclassing frozenset to create a new frozenset from a passed iterable (as argument):
class A(frozenset):
def __new__(cls, *args):
self = super().__new__(cls, *args)
return self
print(A(range(2)))
print(A(range(2)).__class__.__bases__)
Sample output:
A({0, 1})
(<class 'frozenset'>,)
I encounter a surprising behaviour of the side_effect parameter in patch.object where the function replacing the original does not receive self
class Animal():
def __init__(self):
self.noise = 'Woof'
def make_noise(self):
return self.noise
def loud(self):
return self.noise.upper() + '!!'
from unittest.mock import patch
dog = Animal()
dog.make_noise()
with patch.object(Animal, 'make_noise', side_effect=loud):
dog.make_noise()
This give:
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "/lustre/home/production/Applications/python/python-3.4.4/lib/python3.4/unittest/mock.py", line 902, in __call__
return _mock_self._mock_call(*args, **kwargs)
File "/lustre/home/production/Applications/python/python-3.4.4/lib/python3.4/unittest/mock.py", line 968, in _mock_call
ret_val = effect(*args, **kwargs)
TypeError: loud() missing 1 required positional argument: 'self'
If I change the loud function to
def loud(*args, **kwargs):
print(args)
print(kwargs)
I get the following output:
()
{}
Is there a way to replace a function from an object and still receive self?
self is only supplied for bound methods (because functions are descriptors). A Mock object is not such a method, and the side_effect function is not bound, so self is indeed not going to be passed in.
If you must have access the instance in a side_effect object, you'll have to patch the function on the class with an actual function:
with patch.object(Animal, 'make_noise', new=loud):
Now make_noise is replaced by the loud function on the Animal class, so it'll be bound:
>>> with patch.object(Animal, 'make_noise', new=loud):
... dog.make_noise()
...
'WOOF!!'
The alternative is to set autospec=True, at which point mock will use a real function to mock out make_noise():
>>> with patch.object(Animal, 'make_noise', autospec=True, side_effect=loud):
... dog.make_noise()
...
'WOOF!!'
Also see the Mocking Unbound Methods section in the mock getting started section.
class Car:
# constructor
def __init__(self, make, model, year, mpg):
# instance variables
self.carMake = make
self.carModel=model
self.carYear = year
self.efficiency=mpg
self.gas = 0
# special method
def __str__(self):
return "%s %s %s"%(self.carYear, self.carMake, self.carModel)
def refuel(self,gallon):
if gallon < 0:
print("Sorry, amount cannot be negative")
else:
self.gas=self.gas+gallon
print (self.gas)
print("Added %.2f gallon of gas to the tank"%(self.gas))
def gas(self):
print(self.gas)
> Traceback (most recent call last): File "<pyshell#12>", line 1, in
> <module>
> c1.gas() TypeError: 'int' object is not callable
Your method gas and your instance attribute gas created in __init__ have the same name. The method is stored on the class, but is "shadowed" by the attribute stored on the instance, since Python first looks for names on the instance, then on the class and its parents.
So self.gas is an integer and you can't call it.
You have self.gas initialized to an int in the __init__() method, but then you define a method named gas() as well. Once __init__() runs, self.gas is an int. I'm guessing somewhere you are calling gas() on an instance of this class.
Rename your gas() method to something like print_gas(), or, wherever you're calling this, instead of doing c1.gas(), just do print c1.gas.
Consider this class Test in a file called test.py:
class Test:
def __init__(self):
self.x=3
def x(self):
print self.x
Now I import class Test in my console and see what methods it has:
>>> from test import Test
>>> [method for method in dir(Test) if callable(getattr(Test, method))]
['__init__', 'x']
Notice that it has the method x. Now let's create an instance of Test
>>> k=Test()
Let's see what methods we have
>>> [method for method in dir(k) if callable(getattr(k, method))]
['__init__']
>>>
As you can see the method x is no longer available. why?
When you created k as an instance of Test, it executes the __init__ method and sees self.x=3 which redefines x to be just a variable in self and your method x() is gone. So when you do k.x() it thinks that you are doing it on self.x that you set in __init__ which is not callable. However just k.x will work as I show below:
>>> k.x()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'int' object is not callable
>>> k.x
3
>>>
The conclusion is don't name your variables and methods the same.
So I've looked at similar questions, and I've found some solutions to this, but I can't quite figure out how to do this.
What I'm trying to do is add a method to a class from a string. I can do this with the setattr() method, but that won't let me use self as an attribute in the extra method. Here's an example: (and I apologize for the variable names, I always use yolo when I'm mocking up an idea)
class what:
def __init__(self):
s = 'def yolo(self):\n\tself.extra = "Hello"\n\tprint self.extra'
exec(s)
setattr(self,"yolo",yolo)
what().yolo()
returns this:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: yolo() takes exactly 1 argument (0 given)
and if s = 'def yolo():\n\tself.extra = "Hello"\n\tprint self.extra'
then I get this result:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 2, in yolo
NameError: global name 'self' is not defined
This essentially means that I cannot dynamically create methods for classes, which I know is bad practice and unpythonic, because the methods would be unable to access the variables that the rest of the class has access to.
I appreciate any help.
You have to bind your function to the class instance to turn it into a method. It can be done by wrapping it in types.MethodType:
import types
class what:
def __init__(self):
s = 'def yolo(self):\n\tself.extra = "Hello"\n\tprint self.extra'
exec(s)
self.yolo = types.MethodType(yolo, self)
what().yolo()
On a side note, why do you even need exec in this case? You can just as well write
import types
class what:
def __init__(self):
def yolo(self):
self.extra = "Hello"
print self.extra
self.yolo = types.MethodType(yolo, self)
what().yolo()
Edit: for the sake of completeness, one might prefer a solution through the descriptor protocol:
class what:
def __init__(self):
def yolo(self):
self.extra = "Hello"
print self.extra
self.yolo = yolo.__get__(self)
what().yolo()
Another way, seems more elegant to me:
class what:
pass
ld = {}
exec("""
def yolo(self):
self.extra = "Hello"
print(self.extra)
""", None, ld)
# print('locals got: {}'.format(ld))
for name, value in ld.items():
setattr(what, name, value)
what().yolo()