Say I am declaring a class C and a few of the declarations are very similar. I'd like to use a function f to reduce code repetition for these declarations. It's possible to just declare and use f as usual:
>>> class C(object):
... def f(num):
... return '<' + str(num) + '>'
... v = f(9)
... w = f(42)
...
>>> C.v
'<9>'
>>> C.w
'<42>'
>>> C.f(4)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unbound method f() must be called with C instance as first argument (got int instance instead)
Oops! I've inadvertently exposed f to the outside world, but it doesn't take a self argument (and can't for obvious reasons). One possibility would be to del the function after I use it:
>>> class C(object):
... def f(num):
... return '<' + str(num) + '>'
... v = f(9)
... del f
...
>>> C.v
'<9>'
>>> C.f
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: type object 'C' has no attribute 'f'
But what if I want to use f again later, after the declaration? It won't do to delete the function. I could make it "private" (i.e., prefix its name with __) and give it the #staticmethod treatment, but invoking staticmethod objects through abnormal channels gets very funky:
>>> class C(object):
... #staticmethod
... def __f(num):
... return '<' + str(num) + '>'
... v = __f.__get__(1)(9) # argument to __get__ is ignored...
...
>>> C.v
'<9>'
I have to use the above craziness because staticmethod objects, which are descriptors, are not themselves callable. I need to recover the function wrapped by the staticmethod object before I can call it.
There has got to be a better way to do this. How can I cleanly declare a function in a class, use it during its declaration, and also use it later from within the class? Should I even be doing this?
Quite simply, the solution is that f does not need to be a member of the class. I am assuming that your thought-process has gone through a Javaish language filter causing the mental block. It goes a little something like this:
def f(n):
return '<' + str(num) + '>'
class C(object):
v = f(9)
w = f(42)
Then when you want to use f again, just use it
>>> f(4)
'<4>'
I think the moral of the tale is "In Python, you don't have to force everything into a class".
Extending Ali A's answer,
if you really want to avoid f in the module namespace (and using a non-exported name like _f, or setting __all__ isn't sufficient), then
you could achieve this by creating the class within a closure.
def create_C():
def f(num):
return '<' + str(num) + '>'
class C(object):
v = f(9)
def method_using_f(self, x): return f(x*2)
return C
C=create_C()
del create_C
This way C has access to f within its definition and methods, but nothing else does (barring fairly involved introspection
of its methods (C.method_using_f.im_func.func_closure))
This is probably overkill for most purposes though - documenting that f is internal by using the "_" prefix nameing convention should
generally be sufficient.
[Edit] One other option is to hold a reference to the pre-wrapped function object in the methods you wish to use it in. For example, by setting it as a default argument:
class C(object):
def f(num):
return '<' + str(num) + '>'
v = f(9)
def method_using_f(self, x, f=f): return f(x*2)
del f
(Though I think the closure approach is probably better)
I believe you are trying to do this:
class C():
... class F():
... def __call__(self,num):
... return "<"+str(num)+">"
... f=F()
... v=f(9)
>>> C.v
'<9>'
>>> C.f(25)
'<25>'
>>>
Maybe there is better or more pythonic solution...
"declare a function in a class, use it during its declaration, and also use it later from within the class"
Sorry. Can't be done.
"Can't be done" doesn't seem to get along with Python
This is one possibility:
class _C:
# Do most of the function definitions in here
#classmethod
def f(cls):
return 'boo'
class C(_C):
# Do the subsequent decoration in here
v = _C.f()
One option: write a better staticmethod:
class staticfunc(object):
def __init__(self, func):
self.func = func
def __call__(self, *args, **kw):
return self.func(*args, **kw)
def __repr__(self):
return 'staticfunc(%r)' % self.func
Let's begin from the beginning.
"declare a function in a class, use it during its declaration, and also use it later from within the class"
Sorry. Can't be done. "In a class" contradicts "used during declaration".
In a class means created as part of the declaration.
Used during declaration means it exists outside the class. Often as a meta class. However, there are other ways.
It's not clear what C.w and C.v are supposed to be. Are they just strings? If so, an external function f is the best solution. The "not clutter the namespace" is a bit specious. After all, you want to use it again.
It's in the same module as C. That's why Python has modules. It binds the function and class together.
import myCmod
myCmod.C.w
myCmod.C.v
myCmod.f(42)
If w and v aren't simple strings, there's a really good solution that gives a lot of flexibility.
Generally, for class-level ("static") variables like this, we can use other classes. It's not possible to completely achieve the desired API, but this is close.
>>> class F(object):
def __init__( self, num ):
self.value= num
self.format= "<%d>" % ( num, )
>>> class C(object):
w= F(42)
v= F(9)
>>> C.w
<__main__.F object at 0x00C58C30>
>>> C.w.format
'<42>'
>>> C.v.format
'<9>'
The advantage of this is that F is a proper, first-class thing that can be extended. Not a "hidden" thing that we're trying to avoid exposing. It's a fact of life, so we might as well follow the Open/Closed principle and make it open to extension.
Related
Consider:
class A(object):
def f(self): print("A")
class B(A):
def f(self): print("B")
b = B()
I can call A.f on b by doing:
A.f(b)
Is there an easy way to "wrap" b such that wrap(b).f() calls A.f for any f?
Here is my solution which copies the methods from the most upper base class:
import types, copy
def get_all_method_names(clazz):
return [func for func in dir(clazz) if callable(getattr(clazz, func))]
def wrap(obj):
obj = copy.copy(obj)
obj_clazz = obj.__class__
base_clazz = obj_clazz.__bases__[-1] # the one which directly inherits from object
base_methods = get_all_method_names(base_clazz) # list of all method names in base_clazz
for base_method_name in base_methods:
base_method = getattr(base_clazz, base_method_name) # get the method object
if isinstance(base_method, types.FunctionType): # skip dunder methods like __class__, __init__
setattr(obj, base_method_name, base_method) # copy it into our object
return obj
# class declaration from question here
wrapped_b = wrap(b)
wrapped_b.f(wrapped_b) # prints A, unfortunately we have to pass the self parameter explicitly
b.f() # prints B, proof that the original object is untouched
This feels dirty to me, but it also seems to work. I'm not sure I'd rely on this for anything important.
import copy
def upcast(obj, clazz):
if not isinstance(obj, clazz): # make sure we're actually "upcasting"
raise TypeError()
wrapped = copy.copy(obj)
wrapped.__class__ = clazz
return wrapped
This results in
>>> a = A()
>>> a.f()
A
>>> b = B()
>>> b.f()
B
>>> upcast(b, A).f()
A
What I've really done here is essentially monkey-patch a clone of b and lied to it and told it it's actually an A, so when it comes time to resolve which version of f to call, it'll call the one from A.
Object Slicing is not supported in python the way it is done in C++ (The link you are pointing to takes a cpp example).
In Python Object Slicing is a rather different thing which means to slice up any object which supports sequence protocol (implements getitem() and len() methods).
Example :
A = [1,2,3,4,5,6,7,8]
print(A[1:3])
But in C++ Object Slicing is just cutting off the properties added by a base class instance when assigned to a parent class variable.
If I define the __iter__ method as follows, it won't work:
class A:
def __init__(self):
self.__iter__ = lambda: iter('text')
for i in A().__iter__():
print(i)
iter(A())
Result:
t
e
x
t
Traceback (most recent call last):
File "...\mytest.py", line 10, in <module>
iter(A())
TypeError: 'A' object is not iterable
As you can see, calling A().__iter__() works, but A() is not iterable.
However if I define __iter__ for the class, then it will work:
class A:
def __init__(self):
self.__class__.__iter__ = staticmethod(lambda: iter('text'))
# or:
# self.__class__.__iter__ = lambda s: iter('text')
for i in A():
print(i)
iter(A())
# will print:
# t
# e
# x
# t
Does anyone know why python has been designed like this? i.e. why __iter__ as instance variable does not work? Don't you find it unintuitive?
It is done by design. You can find the thorough description here: https://docs.python.org/3/reference/datamodel.html#special-method-lookup
Short answer: the special method must be set on the class object itself in order to be consistently invoked by the interpreter.
Long answer: the idea behind this is to speed up well-known constructions. In your example:
class A:
def __init__(self):
self.__iter__ = lambda: iter('text')
How often are you going to write a code like this in real life? So, what Python does - it skips a dictionary lookup of the instance, i.e. iter(A()) simply does not "see" that self.__iter__, which is actually self.__dict__['__iter__'] in this case.
It also skips all the __getattribute__ instance and metaclass lookup gaining a significant speedup.
I would like to do the following:
class A(object): pass
a = A()
a.__int__ = lambda self: 3
i = int(a)
Unfortunately, this throws:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: int() argument must be a string or a number, not 'A'
This only seems to work if I assign the "special" method to the class A instead of an instance of it. Is there any recourse?
One way I thought of was:
def __int__(self):
# No infinite loop
if type(self).__int__.im_func != self.__int__.im_func:
return self.__int__()
raise NotImplementedError()
But that looks rather ugly.
Thanks.
Python always looks up special methods on the class, not the instance (except in the old, aka "legacy", kind of classes -- they're deprecated and have gone away in Python 3, because of the quirky semantics that mostly comes from looking up special methods on the instance, so you really don't want to use them, believe me!-).
To make a special class whose instances can have special methods independent from each other, you need to give each instance its own class -- then you can assign special methods on the instance's (individual) class without affecting other instances, and live happily ever after. If you want to make it look like you're assigning to an attribute the instance, while actually assigning to an attribute of the individualized per-instance class, you can get that with a special __setattr__ implementation, of course.
Here's the simple case, with explicit "assign to class" syntax:
>>> class Individualist(object):
... def __init__(self):
... self.__class__ = type('GottaBeMe', (self.__class__, object), {})
...
>>> a = Individualist()
>>> b = Individualist()
>>> a.__class__.__int__ = lambda self: 23
>>> b.__class__.__int__ = lambda self: 42
>>> int(a)
23
>>> int(b)
42
>>>
and here's the fancy version, where you "make it look like" you're assigning the special method as an instance attribute (while behind the scene it still goes to the class of course):
>>> class Sophisticated(Individualist):
... def __setattr__(self, n, v):
... if n[:2]=='__' and n[-2:]=='__' and n!='__class__':
... setattr(self.__class__, n, v)
... else:
... object.__setattr__(self, n, v)
...
>>> c = Sophisticated()
>>> d = Sophisticated()
>>> c.__int__ = lambda self: 54
>>> d.__int__ = lambda self: 88
>>> int(c)
54
>>> int(d)
88
The only recourse that works for new-style classes is to have a method on the class that calls the attribute on the instance (if it exists):
class A(object):
def __int__(self):
if '__int__' in self.__dict__:
return self.__int__()
raise ValueError
a = A()
a.__int__ = lambda: 3
int(a)
Note that a.__int__ will not be a method (only functions that are attributes of the class will become methods) so self is not passed implicitly.
I have nothing to add about the specifics of overriding __int__. But I noticed one thing about your sample that bears discussing.
When you manually assign new methods to an object, "self" is not automatically passed in. I've modified your sample code to make my point clearer:
class A(object): pass
a = A()
a.foo = lambda self: 3
a.foo()
If you run this code, it throws an exception because you passed in 0 arguments to "foo" and 1 is required. If you remove the "self" it works fine.
Python only automatically prepends "self" to the arguments if it had to look up the method in the class of the object and the function it found is a "normal" function. (Examples of "abnormal" functions: class methods, callable objects, bound method objects.) If you stick callables in to the object itself they won't automatically get "self".
If you want self there, use a closure.
I would something like this in Python:
result = SomeClass(some_argument)
Here is the catch though. I don't want the result to be an instance but an immutable object (int, for example). Basically the hole role of a class is returning a value calculated from the argument. I am using a class and not a function for DRY purposes.
Since the above code won't work because it will always return an instance of SomeClass what would be the best alternative?
My only idea is to have a static method, but I don't like it:
result = SomeClass.static_method(some_argument)
You can override __new__. This is rarely a good idea and/or necessary though ...
>>> class Foo(object):
... def __new__(cls):
... return 1
...
>>> Foo()
1
>>> type(Foo())
<type 'int'>
If you don't return an instance of cls, __init__ will never be called.
Basically class methods are the way to go if you have a factory method.
About the result - it really depends on what kind of immutability you seek, but basically namedtuple does a great job for encapsulating things and is also immutable (like normal tuples):
from collections import namedtuple
class FactoryClass(object):
_result_type = namedtuple('ProductClass', ['prod', 'sum'])
#classmethod
def make_object(cls, arg1, arg2):
return cls._result_type(prod=arg1 * arg2, sum=arg1 + arg2)
>>> FactoryClass.make_object(2,3)
ProductClass(prod=6, sum=5)
>>> x = _
>>> x.prod = 3
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: can't set attribute
this works in the desired way:
class d:
def __init__(self,arg):
self.a = arg
def p(self):
print "a= ",self.a
x = d(1)
y = d(2)
x.p()
y.p()
yielding
a= 1
a= 2
i've tried eliminating the "self"s and using a global statement in __init__
class d:
def __init__(self,arg):
global a
a = arg
def p(self):
print "a= ",a
x = d(1)
y = d(2)
x.p()
y.p()
yielding, undesirably:
a= 2
a= 2
is there a way to write it without having to use "self"?
"self" is the way how Python works. So the answer is: No! If you want to cut hair: You don't have to use "self". Any other name will do also. ;-)
Python methods are just functions that are bound to the class or instance of a class. The only difference is that a method (aka bound function) expects the instance object as the first argument. Additionally when you invoke a method from an instance, it automatically passes the instance as the first argument. So by defining self in a method, you're telling it the namespace to work with.
This way when you specify self.a the method knows you're modifying the instance variable a that is part of the instance namespace.
Python scoping works from the inside out, so each function (or method) has its own namespace. If you create a variable a locally from within the method p (these names suck BTW), it is distinct from that of self.a. Example using your code:
class d:
def __init__(self,arg):
self.a = arg
def p(self):
a = self.a - 99
print "my a= ", a
print "instance a= ",self.a
x = d(1)
y = d(2)
x.p()
y.p()
Which yields:
my a= -98
instance a= 1
my a= -97
instance a= 2
Lastly, you don't have to call the first variable self. You could call it whatever you want, although you really shouldn't. It's convention to define and reference self from within methods, so if you care at all about other people reading your code without wanting to kill you, stick to the convention!
Further reading:
Python Classes tutorial
When you remove the self's, you end up having only one variable called a that will be shared not only amongst all your d objects but also in your entire execution environment.
You can't just eliminate the self's for this reason.