This question already has answers here:
Why does setattr fail on a bound method
(2 answers)
Closed 9 years ago.
I'm wondering if it is possible to use setattr to set an attribute to a method within a class like so because when I try I get an error which is going to be shown after the code:
class Test:
def getString(self, var):
setattr(self.getString, "string", var)
return self.getString
test = Test()
test.getString("myString").string
Which errors AttributeError: 'method' object has no attribute 'string' so I tried it without putting .string and just tried test.getString("myString") Same error, but then I tried it without the using the class just like this
def getString(var):
setattr(getString, "string", var)
return getString
getString("myString").string
It returned "myString" like I wanted it to, so how would I do this within a class and why does it work outside of one but inside of one?
type( test.getString ) is builtins.method and from the documentations ( methods ),
since method attributes are actually stored on the underlying function
object (meth.__func__), setting method attributes on bound methods is
disallowed. Attempting to set an attribute on a method results in an
AttributeError being raised.
There are (at least) two possible solutions depending on which behaviour you are looking for. One is to set the attribute on the class method:
class Test:
def getString(self, var):
setattr(Test.getString, "string", var)
return self.getString
test = Test()
test.getString("myString").string # > "myString"
test2 = Test()
test2.getString.string # > this is also "myString"
and the other is to use function objects:
class Test:
class getStringClass:
def __call__ ( self, var ):
setattr( self, "string", var )
return self
def __init__( self ):
self.getString = Test.getStringClass( )
test = Test( )
test.getString( "myString" ).string # > "myString"
test2 = Test()
test2.getString.string # > this is error, because it does not
# have the attribute 'string' yet
Functions are like most other objects in that you can freely add attributes to them. Methods, on the other hand... conceptually they're just functions, but they behave slightly differently (implicity pass self) and therefore are implemented with a bit of extra glue around functions.
Every time self.getString is evaluated, a new (bound) method object is created, which is a thin wrapper around the underlying function (which you can access as Test.getString). These method objects don't allow adding attributes, and even if they did, your code would not work because it juggles multiple distinct method objects (though they all wrap the same function).
You can't make this work with bound methods. Since you presumably want the string to be attached to the Test object (indirectly, by being attached to its method), you can make it an attribute of Test. You could even create your own object that behaves like a method but allows attributes (you'd have to explicitly add it in __init__), but honestly there's probably a better way that keeps data and methods separated. If, on the other hand, you want to attach this attribute to the underlying function (which would mean it's shared by all Test instances), you can just set the attribute on Test.getString.
So, I found a way but it's not really how I wanted to do it personally. If anyone does find another way to do the following code feel free to comment on how to do it.
class Test:
def getString(self, string):
setattr(self,"newString",self)
self.newString.string = string
return self.newString
Like I said, I don't feel like I accomplished anything by doing it that way, but it works for what I need and if you do find another way comment below.
Related
I am trying to unit test a block of code, and I'm running into issues with mocking the object's type to grab the right function from a dictionary.
For example:
my_func_dict = {
Foo: foo_func,
Bar: bar_func
FooBar: foobar_func
}
def generic_type_func(my_obj):
my_func = my_func_dict[type(my_obj)]
my_func()
With this code, I can swap between functions with a key lookup, and it's pretty efficient.
When I try to mock my_obj like this, I get a KeyError:
mock_obj = Mock(spec=Foo)
generic_type_func(mock_obj)
# OUTPUT:
# KeyError: <class 'unittest.mock.Mock'>
Because it's a mock type. Although, when I check isinstance(), it returns true:
is_instance_Foo = isinstance(mock_obj, Foo)
print(is_instance_foo)
# Output:
# True
Is there any way to retain the type() check, and using the dictionary lookup via a key, while still maintaining the ability to mock the input and return_type? Or perhaps a different pattern where I can retain the performance of a dictionary, but use isinstance() instead so I can mock the parameter? Looping over a list to check the type against every possible value is not preferred.
I managed to unit test this by moving the function to the parameter itself, and implicitly calling the function from the parent. I wanted to avoid this, because now the function manipulates the parent implicitly instead of explicitly from the parent itself. It looks like this now:
def generic_type_func(self, my_obj):
my_obj.my_func(self)
The function then modifies self as needed, but implicitly instead of an explicit function on the parent class.
This:
def my_func(self, parent):
self.foo_prop = parent
Rather than:
def my_foo_func(self, foo):
foo.foo_prop = self
This works fine with a mock, and I can mock that function easily. I've just hidden some of the functionality, and edit properties on the parent implicitly instead of explicitly from within the class I'm working in. Maybe this is preferable anyways, and it looks cleaner with less code on the parent class. Every instance must have my_func this way, which is enforced via an abstract base class.
In the following code,
# An example class with some variable and a method
class ExampleClass(object):
def __init__(self):
self.var = 10
def dummyPrint(self):
print ('Hello World!')
# Creating instance and printing the init variable
inst_a = ExampleClass()
# This prints --> __init__ variable = 10
print ('__init__ variable = %d' %(inst_a.var))
# This prints --> Hello World!
inst_a.dummyPrint()
# Creating a new attribute and printing it.
# This prints --> New variable = 20
inst_a.new_var = 20
print ('New variable = %d' %(inst_a.new_var))
# Trying to create new method, which will give error
inst_a.newDummyPrint()
I am able to create a new attribute (new_var) outside the class, using instance. And it works. Ideally, I was expecting it will not work.
Similarly I tried creating new method (newDummyPrint()); which will print AttributeError: 'ExampleClass' object has no attribute 'newDummyPrint' as I expected.
My question is,
Why did creating a new attribute worked?
Why creating a new method didn't work?
As already mentionned in comments, you are creating the new attribute here:
inst_a.new_var = 20
before reading it on the next line. You're NOT assigning newDummyPrint anywhere, so obviously the attribute resolution mechanism cannot find it and ends up raising an AtributeError. You'd get the very same result if you tried to access any other non-existing attribute, ie inst_a.whatever.
Note that since in Python everything is an object (including classes, functions etc), there are no real distinction between accessing a "data" attribute or a method - they are all attributes (whether class or instance ones), and the attribute resolution rules are the same. In the case of methods (or any other callable attribute), the call operation happens after the attribute has been resolved.
To dynamically create a new "method", you mainly have two solutions: creating as a class attribute (which will make it available to all other instances of the class), or as an instance attribute (which will - obviously - make it available only on this exact instance.
The first solution is as simple as it can be: define your function and bind it to the class:
# nb: inheriting from `object` for py2 compat
class Foo(object):
def __init__(self, var):
self.var = var
def bar(self, x):
return self.var * x
# testing before:
f = Foo(42)
try:
print(f.bar(2))
except AttribteError as e:
print(e)
# now binds the function to the class:
Foo.bar = bar
# and test it:
print(f.bar(2))
# and it's also available on other instances:
f2 = Foo(6)
print(f2.bar(7))
Creating per-instance method is a (very tiny) bit more involved - you have to manually get the method from the function and bind this method to the instance:
def baaz(self):
return "{}.var = {}".format(self, self.var)
# test before:
try:
print(f.baaz())
except AttributeError as e:
print(e)
# now binds the method to the instance
f.baaz = baaz.__get__(f, Foo)
# now `f` has a `baaz` method
print(f.baaz())
# but other Foo instances dont
try:
print(f2.baaz())
except AttributeError as e:
print(e)
You'll noticed I talked about functions in the first case and methods in the second case. A python "method" is actually just a thin callable wrapper around a function, an instance and a class, and is provided by the function type through the descriptor protocol - which is automagically invoked when the attribute is resolved on the class itself (=> is a class attribute implementin the descriptor protocol) but not when resolved on the instance. This why, in the second case, we have to manually invoke the descriptor protocol.
Also note that there are limitations on what's possible here: first, __magic__ methods (all methods named with two leading and two trailing underscores) are only looked up on the class itself so you cannot define them on a per-instance basis. Then, slots-based types and some builtin or C-coded types do not support dynamic attributes whatsoever. Those restrictions are mainly there for performance optimization reasons.
You can create new attributes on the fly when you are using an empty class definition emulating Pascal "record" or C "struct". Otherwise, what you are trying to do is not a good manner, or a good pattern for object-oriented programming. There are lots of books you can read about it. Generally speaking, you have to clearly tell in the class definition what an object of that class is, how it behaves: modifying its behavior on the fly (e.g. adding new methods) could lead to unknown results, which make your life impossible when reading that code a month later and even worse when you are debugging.
There is even an anti-pattern problem called Ambiguous Viewpoint:
Lack of clarification of the modeling viewpoint leads to problematic
ambiguities in object models.
Anyway, if you are playing with Python and you swear you'll never use this code in production, you can write new attributes which store lambda functions, e.g.
c = ExampleClass()
c.newMethod = lambda s1, s2: str(s1) + ' and ' + str(s2)
print(c.newMethod('string1', 'string2'))
# output is: string1 and string2
but this is very ugly, I would never do it.
In Python when we define class all its members including variables and methods also becomes attributes of that class. In following example MyClass1.a and MyClass1.mydef1 are attributes of class MyClass1.
class MyClass1:
a = 10
def mydef1(self):
return 0
ins1 = MyClass1() # create instance
print(MyClass1.a) # access class attribute which is class variable
print(MyClass1.mydef1) # No idea what to do with it so just printing
print(ins1.mydef1) # No idea what to do with it so just printing
Output
10
<function MyClass1.mydef1 at 0x0000000002122EA0>
<bound method MyClass1.mydef1 of <__main__.MyClass1 object at 0x000000000212D0F0>>
Here attribute a is a variable and it can be used like any other variable.
But mydef1 is a method, if it is not invoked and just used like MyClass1.mydef1 or ins1.mydef1, it returns object for that method(correct me if I am wrong).
So my question is, what can we do with the Class/instance methods without invoking it? Are there any use cases for it or is it just good to know thing?
An attribute of a class that happens to be a function becomes a method for instances or that class:
inst.foo(params, ...)
is internally translated into:
cls.foo(inst, params, ...)
That means that what is actually invoked is the attribute from the class of the instance, and the instance itself is prepended to the argument list. It is just Python syntax to invoke methods on objects.
In your example the correct uses would be:
print(MyClass1.mydef1(ins1)) # prints 0
print(ins1.mydef1()) # also prints 0
Well instance methods can be called with the appropriate parameters of course:
print(ins1.mydef1()) # no parameters, so empty parenthesis, this call should print "0" in your example instead of the method description
If you use it without the parenthesis, you are playing with reference to the function, I don't think you can have any use of it, except checking the list of methods available in a class or something like that.
In JavaScript, we can do the following to any object or function
const myFn = () => {};
Object.defineProperties(myFn, {
property: {
get: () => console.log('property accessed')
}
});
This will allow for a #property like syntax by defining a getter function for the property property.
myFn.property
// property accessed
Is there anything similar for functions in Python?
I know we can't use property since it's not a new-style class, and assigning a lambda with setattr will not work since it'll be a function.
Basically what I want to achieve is that whenever my_fn.property is to return a new instance of another class on each call.
What I currently have with setattr is this
setattr(my_fn, 'property', OtherClass())
My hopes are to design an API that looks like this my_fn.property.some_other_function().
I would prefer using a function as my_fn and not an instance of a class, even though I realize that it might be easier to implement.
Below is the gist of what I'm trying to achieve
def my_fn():
pass
my_fn = property('property', lambda: OtherClass())
my_fn.property
// will be a new instance of OtherClass on each call
It's not possible to do exactly what you want. The descriptor protocol that powers the property built-in is only invoked when:
The descriptor is defined on a class
The descriptor's name is accessed on an instance of said class
Problem is, the class behind functions defined in Python (aptly named function, exposed directly as types.FunctionType or indirectly by calling type() on any function defined at the Python layer) is a single shared, immutable class, so you can't add descriptors to it (and even if you could, they'd become attributes of every Python level function, not just one particular function).
The closest you can get to what you're attempting would be to define a callable class (defining __call__) that defines the descriptor you're interested in as well. Make a single instance of that class (you can throw away the class itself at this point) and it will behave as you expect. Make __call__ a staticmethod, and you'll avoid changing the signature to boot.
For example, the behavior you want could be achieved with:
class my_fn:
# Note: Using the name "property" for a property has issues if you define
# other properties later in the class; this is just for illustration
#property
def property(self):
return OtherClass()
#staticmethod
def __call__(...whatever args apply; no need for self...):
... function behavior goes here ...
my_fn = my_fn() # Replace class with instance of class that behaves like a function
Now you can call the "function" (really a functor, to use C++ parlance):
my_fn(...)
or access the property, getting a brand new OtherClass each time:
>>> type(my_fn.property) is type(my_fn.property)
True
>>> my_fn.property is my_fn.property
False
No, this isn't what you asked for (you seem set on having a plain function do this for you), but you're asking for a very JavaScript specific thing which doesn't exist in Python.
What you want is not currently possible, because the property would have to be set on the function type to be invoked correctly. And you are not allowed to monkeypatch the function type:
>>> type(my_fn).property = 'anything else'
TypeError: can't set attributes of built-in/extension type 'function'
The solution: use a callable class instead.
Note: What you want may become possible in Python 3.8 if PEP 575 is accepted.
I've got a question about defining functions and the self-parameter in python.
There is following code.
class Dictionaries(object):
__CSVDescription = ["ID", "States", "FilterTime", "Reaction", "DTC", "ActiveDischarge"]
def __makeDict(Lst):
return dict(zip(Lst, range(len(Lst))))
def getDict(self):
return self.__makeDict(self.__CSVDescription)
CSVDescription = __makeDict(__CSVDescription)
x = Dictionaries()
print x.CSVDescription
print x.getDict()
x.CSVDescription works fine. But print x.getDict() returns an error.
TypeError: __makeDict() takes exactly 1 argument (2 given)
I can add the self-parameter to the __makeDict() method, but then print x.CSVDescription wouldn't work.
How do I use the self-parameter correctly?
In python, the self parameter is implicitly passed to instance methods, unless the method is decorated with #staticmethod.
In this case, __makeDict doesn't need a reference to the object itself, so it can be made a static method so you can omit the self:
#staticmethod
def __makeDict(Lst): # ...
def getDict(self):
return self.__makeDict(self.__CSVDescription)
A solution using #staticmethod won't work here because calling the method from the class body itself doesn't invoke the descriptor protocol (this would also be a problem for normal methods if they were descriptors - but that isn't the case until after the class definition has been compiled). There are four major options here - but most of them could be seen as some level of code obfuscation, and would really need a comment to answer the question "why not just use a staticmethod?".
The first is, as #Marcus suggests, to always call the method from the class, not from an instance. That is, every time you would do self.__makeDict, do self.__class__.__makeDict instead. This will look strange, because it is a strange thing to do - in Python, you almost never need to call a method as Class.method, and the only time you do (in code written before super became available), using self.__class__ would be wrong.
In similar vein, but the other way around, you could make it a staticmethod and invoke the descriptor protocol manually in the class body - do: __makeDict.__get__(None, Dictionaries)(__lst).
Or, you could detect yourself what context its being called from by getting fancy with optional arguments:
def __makeDict(self, Lst=None):
if Lst is None:
Lst = self
...
But, by far the best way is to realise you're working in Python and not Java - put it outside the class.
def _makeDict(Lst):
...
class Dictionaries(object):
def getDict(self):
return _makeDict(self.__CSVDescription)
CSVDescription = _makeDict(__CSVDescription)