Is it possible to convert a python function into a class? [closed] - python

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 6 years ago.
Improve this question
I am new to Python coming from C++ background and this is the first time I am seeing a language which contains nothing but objects. I have just learned that class and functions are also just objects. So, is there a way to convert the following function to a class?
In [1]: def somefnc(a, b):
...: return a+b
...:
I have first tried assigning the __call__ variable to None to take away the "callable nature" from the function. But as you can see, the __call__ was successfully replaced by None but this didn't cause the function to stop adding numbers when called, though, somefnc.__call__(1,3) was working before assigning somefnc.__call__ to None
In [2]: somefnc.__dict__
Out[2]: {}
In [3]: somefnc.__call__
Out[3]: <method-wrapper '__call__' of function object at 0x7f282e8ff7b8>
In [4]: somefnc.__call__ = None
In [5]: x = somefnc(1, 2)
In [6]: print(x)
3
In [7]: somefnc.__call__
In [8]: print(somefnc.__call__(1, 2))
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
ipython-input-8-407663da97ca
in <module>()
----> 1 print(somefnc.__call__(1, 2))
TypeError: 'NoneType' object is not callable
In [9]: print (somefnc(1,2))
3
In [10]:
I am not doing this for developing purpose here, so claiming this to be a bad practice will not make any sense. I am just trying to understand Python very well. Of course, for development purpose, I could rather create a class than to convert a function to one!
After robbing the function off its ability to add two numbers, I am thinking of assigning a valid function to the attribute somefnc.__init__ and some members by modifying somefun.__dict__, to convert it to a class.

In Python functions are instances of the function class. So I'll give you a general answer about any class and instance.
In [10]: class Test:
...: def __getitem__(self, i):
...: return i
...:
In [11]: t = Test()
In [12]: t[0]
Out[12]: 0
In [13]: t.__getitem__(0)
Out[13]: 0
In [14]: t.__getitem__ = None
In [15]: t[0]
Out[15]: 0
In [16]: t.__getitem__(0)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-16-c72f91d2bfbc> in <module>()
----> 1 t.__getitem__(0)
TypeError: 'NoneType' object is not callable
In Python all special methods (the ones with double underscores in the prefix and the postfix) are accessed from the class when triggered via operators, not from the instance.
In [17]: class Test2:
...: def test(self, i):
...: return i
...:
In [18]: t = Test2()
In [19]: t.test(1)
Out[19]: 1
In [20]: t.test = None
In [20]: t.test(1)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-22-261b43cb55fe> in <module>()
----> 1 t.test(1)
TypeError: 'NoneType' object is not callable
All methods are accessed via the instance first, when accessed by name. The difference is due to different search mechanics. When you access a method/attribute by name, you invoke __getattribute__ which will first search in the instance's namespace by default. When you trigger a method via operators, __getattribute__ is not invoked. You can see it in the disassembly.
In [22] import dis
In [23]: def test():
...: return Test()[0]
...:
In [24]: dis.dis(test)
2 0 LOAD_GLOBAL 0 (Test)
3 CALL_FUNCTION 0 (0 positional, 0 keyword pair)
6 LOAD_CONST 1 (0)
9 BINARY_SUBSCR
10 RETURN_VALUE
In [25]: def test2():
...: return Test().__getitem__(0)
...:
In [26]: dis.dis(test2)
2 0 LOAD_GLOBAL 0 (Test)
3 CALL_FUNCTION 0 (0 positional, 0 keyword pair)
6 LOAD_ATTR 1 (__getitem__)
9 LOAD_CONST 1 (0)
12 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
15 RETURN_VALUE
As you can see, there is no LOAD_ATTR in the first case. The [] operator is assembled as a special virtual-machine instruction BINARY_SUBSCR.

Related

Is there a built in 'do nothing' function in python? [duplicate]

I'd like to point to a function that does nothing:
def identity(*args)
return args
my use case is something like this
try:
gettext.find(...)
...
_ = gettext.gettext
else:
_ = identity
Of course, I could use the identity defined above, but a built-in would certainly run faster (and avoid bugs introduced by my own).
Apparently, map and filter use None for the identity, but this is specific to their implementations.
>>> _=None
>>> _("hello")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'NoneType' object is not callable
Doing some more research, there is none, a feature was asked in issue 1673203 And from Raymond Hettinger said there won't be:
Better to let people write their own trivial pass-throughs
and think about the signature and time costs.
So a better way to do it is actually (a lambda avoids naming the function):
_ = lambda *args: args
advantage: takes any number of parameters
disadvantage: the result is a boxed version of the parameters
OR
_ = lambda x: x
advantage: doesn't change the type of the parameter
disadvantage: takes exactly 1 positional parameter
An identity function, as defined in https://en.wikipedia.org/wiki/Identity_function, takes a single argument and returns it unchanged:
def identity(x):
return x
What you are asking for when you say you want the signature def identity(*args) is not strictly an identity function, as you want it to take multiple arguments. That's fine, but then you hit a problem as Python functions don't return multiple results, so you have to find a way of cramming all of those arguments into one return value.
The usual way of returning "multiple values" in Python is to return a tuple of the values - technically that's one return value but it can be used in most contexts as if it were multiple values. But doing that here means you get
>>> def mv_identity(*args):
... return args
...
>>> mv_identity(1,2,3)
(1, 2, 3)
>>> # So far, so good. But what happens now with single arguments?
>>> mv_identity(1)
(1,)
And fixing that problem quickly gives other issues, as the various answers here have shown.
So, in summary, there's no identity function defined in Python because:
The formal definition (a single argument function) isn't that useful, and is trivial to write.
Extending the definition to multiple arguments is not well-defined in general, and you're far better off defining your own version that works the way you need it to for your particular situation.
For your precise case,
def dummy_gettext(message):
return message
is almost certainly what you want - a function that has the same calling convention and return as gettext.gettext, which returns its argument unchanged, and is clearly named to describe what it does and where it's intended to be used. I'd be pretty shocked if performance were a crucial consideration here.
yours will work fine. When the number of parameters is fix you can use an anonymous function like this:
lambda x: x
There is no a built-in identity function in Python. An imitation of the Haskell's id function would be:
identity = lambda x, *args: (x,) + args if args else x
Example usage:
identity(1)
1
identity(1,2)
(1, 2)
Since identity does nothing except returning the given arguments, I do not think that it is slower than a native implementation would be.
No, there isn't.
Note that your identity:
is equivalent to lambda *args: args
Will box its args - i.e.
In [6]: id = lambda *args: args
In [7]: id(3)
Out[7]: (3,)
So, you may want to use lambda arg: arg if you want a true identity function.
NB: This example will shadow the built-in id function (which you will probably never use).
If the speed does not matter, this should handle all cases:
def identity(*args, **kwargs):
if not args:
if not kwargs:
return None
elif len(kwargs) == 1:
return next(iter(kwargs.values()))
else:
return (*kwargs.values(),)
elif not kwargs:
if len(args) == 1:
return args[0]
else:
return args
else:
return (*args, *kwargs.values())
Examples of usage:
print(identity())
None
$identity(1)
1
$ identity(1, 2)
(1, 2)
$ identity(1, b=2)
(1, 2)
$ identity(a=1, b=2)
(1, 2)
$ identity(1, 2, c=3)
(1, 2, 3)
Stub of a single-argument function
gettext.gettext (the OP's example use case) accepts a single argument, message. If one needs a stub for it, there's no reason to return [message] instead of message (def identity(*args): return args). Thus both
_ = lambda message: message
def _(message):
return message
fit perfectly.
...but a built-in would certainly run faster (and avoid bugs introduced by my own).
Bugs in such a trivial case are barely relevant. For an argument of predefined type, say str, we can use str() itself as an identity function (because of string interning it even retains object identity, see id note below) and compare its performance with the lambda solution:
$ python3 -m timeit -s "f = lambda m: m" "f('foo')"
10000000 loops, best of 3: 0.0852 usec per loop
$ python3 -m timeit "str('foo')"
10000000 loops, best of 3: 0.107 usec per loop
A micro-optimisation is possible. For example, the following Cython code:
test.pyx
cpdef str f(str message):
return message
Then:
$ pip install runcython3
$ makecython3 test.pyx
$ python3 -m timeit -s "from test import f" "f('foo')"
10000000 loops, best of 3: 0.0317 usec per loop
Build-in object identity function
Don't confuse an identity function with the id built-in function which returns the 'identity' of an object (meaning a unique identifier for that particular object rather than that object's value, as compared with == operator), its memory address in CPython.
Lots of good answers and discussion are in this topic. I just want to note that, in OP's case where there is a single argument in the identity function, compile-wise it doesn't matter if you use a lambda or define a function (in which case you should probably define the function to stay PEP8 compliant). The bytecodes are functionally identical:
import dis
function_method = compile("def identity(x):\n return x\ny=identity(Type('x', (), dict()))", "foo", "exec")
dis.dis(function_method)
1 0 LOAD_CONST 0 (<code object identity at 0x7f52cc30b030, file "foo", line 1>)
2 LOAD_CONST 1 ('identity')
4 MAKE_FUNCTION 0
6 STORE_NAME 0 (identity)
3 8 LOAD_NAME 0 (identity)
10 LOAD_NAME 1 (Type)
12 LOAD_CONST 2 ('x')
14 LOAD_CONST 3 (())
16 LOAD_NAME 2 (dict)
18 CALL_FUNCTION 0
20 CALL_FUNCTION 3
22 CALL_FUNCTION 1
24 STORE_NAME 3 (y)
26 LOAD_CONST 4 (None)
28 RETURN_VALUE
Disassembly of <code object identity at 0x7f52cc30b030, file "foo", line 1>:
2 0 LOAD_FAST 0 (x)
2 RETURN_VALUE
And lambda
import dis
lambda_method = compile("identity = lambda x: x\ny=identity(Type('x', (), dict()))", "foo", "exec")
dis.dis(lambda_method)
1 0 LOAD_CONST 0 (<code object <lambda> at 0x7f52c9fbbd20, file "foo", line 1>)
2 LOAD_CONST 1 ('<lambda>')
4 MAKE_FUNCTION 0
6 STORE_NAME 0 (identity)
2 8 LOAD_NAME 0 (identity)
10 LOAD_NAME 1 (Type)
12 LOAD_CONST 2 ('x')
14 LOAD_CONST 3 (())
16 LOAD_NAME 2 (dict)
18 CALL_FUNCTION 0
20 CALL_FUNCTION 3
22 CALL_FUNCTION 1
24 STORE_NAME 3 (y)
26 LOAD_CONST 4 (None)
28 RETURN_VALUE
Disassembly of <code object <lambda> at 0x7f52c9fbbd20, file "foo", line 1>:
1 0 LOAD_FAST 0 (x)
2 RETURN_VALUE
Adding to all answers:
Notice there is an implicit convention in Python stdlib, where a HOF defaulting it's key parameter function to the identity function, interprets None as such.
E.g. sorted, heapq.merge, max, min, etc.
So, it is not bad idea to consider your HOF expecting key to following the same pattern.
That is, instead of:
def my_hof(x, key=lambda _: _):
...
(whis is totally right)
You could write:
def my_hof(x, key=None):
if key is None: key = lambda _: _
...
If you want.
The thread is pretty old. But still wanted to post this.
It is possible to build an identity method for both arguments and objects. In the example below, ObjOut is an identity for ObjIn. All other examples above haven't dealt with dict **kwargs.
class test(object):
def __init__(self,*args,**kwargs):
self.args = args
self.kwargs = kwargs
def identity (self):
return self
objIn=test('arg-1','arg-2','arg-3','arg-n',key1=1,key2=2,key3=3,keyn='n')
objOut=objIn.identity()
print('args=',objOut.args,'kwargs=',objOut.kwargs)
#If you want just the arguments to be printed...
print(test('arg-1','arg-2','arg-3','arg-n',key1=1,key2=2,key3=3,keyn='n').identity().args)
print(test('arg-1','arg-2','arg-3','arg-n',key1=1,key2=2,key3=3,keyn='n').identity().kwargs)
$ py test.py
args= ('arg-1', 'arg-2', 'arg-3', 'arg-n') kwargs= {'key1': 1, 'keyn': 'n', 'key2': 2, 'key3': 3}
('arg-1', 'arg-2', 'arg-3', 'arg-n')
{'key1': 1, 'keyn': 'n', 'key2': 2, 'key3': 3}

Is there an efficient replacement to "lambda x: x"? [duplicate]

I'd like to point to a function that does nothing:
def identity(*args)
return args
my use case is something like this
try:
gettext.find(...)
...
_ = gettext.gettext
else:
_ = identity
Of course, I could use the identity defined above, but a built-in would certainly run faster (and avoid bugs introduced by my own).
Apparently, map and filter use None for the identity, but this is specific to their implementations.
>>> _=None
>>> _("hello")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'NoneType' object is not callable
Doing some more research, there is none, a feature was asked in issue 1673203 And from Raymond Hettinger said there won't be:
Better to let people write their own trivial pass-throughs
and think about the signature and time costs.
So a better way to do it is actually (a lambda avoids naming the function):
_ = lambda *args: args
advantage: takes any number of parameters
disadvantage: the result is a boxed version of the parameters
OR
_ = lambda x: x
advantage: doesn't change the type of the parameter
disadvantage: takes exactly 1 positional parameter
An identity function, as defined in https://en.wikipedia.org/wiki/Identity_function, takes a single argument and returns it unchanged:
def identity(x):
return x
What you are asking for when you say you want the signature def identity(*args) is not strictly an identity function, as you want it to take multiple arguments. That's fine, but then you hit a problem as Python functions don't return multiple results, so you have to find a way of cramming all of those arguments into one return value.
The usual way of returning "multiple values" in Python is to return a tuple of the values - technically that's one return value but it can be used in most contexts as if it were multiple values. But doing that here means you get
>>> def mv_identity(*args):
... return args
...
>>> mv_identity(1,2,3)
(1, 2, 3)
>>> # So far, so good. But what happens now with single arguments?
>>> mv_identity(1)
(1,)
And fixing that problem quickly gives other issues, as the various answers here have shown.
So, in summary, there's no identity function defined in Python because:
The formal definition (a single argument function) isn't that useful, and is trivial to write.
Extending the definition to multiple arguments is not well-defined in general, and you're far better off defining your own version that works the way you need it to for your particular situation.
For your precise case,
def dummy_gettext(message):
return message
is almost certainly what you want - a function that has the same calling convention and return as gettext.gettext, which returns its argument unchanged, and is clearly named to describe what it does and where it's intended to be used. I'd be pretty shocked if performance were a crucial consideration here.
yours will work fine. When the number of parameters is fix you can use an anonymous function like this:
lambda x: x
There is no a built-in identity function in Python. An imitation of the Haskell's id function would be:
identity = lambda x, *args: (x,) + args if args else x
Example usage:
identity(1)
1
identity(1,2)
(1, 2)
Since identity does nothing except returning the given arguments, I do not think that it is slower than a native implementation would be.
No, there isn't.
Note that your identity:
is equivalent to lambda *args: args
Will box its args - i.e.
In [6]: id = lambda *args: args
In [7]: id(3)
Out[7]: (3,)
So, you may want to use lambda arg: arg if you want a true identity function.
NB: This example will shadow the built-in id function (which you will probably never use).
If the speed does not matter, this should handle all cases:
def identity(*args, **kwargs):
if not args:
if not kwargs:
return None
elif len(kwargs) == 1:
return next(iter(kwargs.values()))
else:
return (*kwargs.values(),)
elif not kwargs:
if len(args) == 1:
return args[0]
else:
return args
else:
return (*args, *kwargs.values())
Examples of usage:
print(identity())
None
$identity(1)
1
$ identity(1, 2)
(1, 2)
$ identity(1, b=2)
(1, 2)
$ identity(a=1, b=2)
(1, 2)
$ identity(1, 2, c=3)
(1, 2, 3)
Stub of a single-argument function
gettext.gettext (the OP's example use case) accepts a single argument, message. If one needs a stub for it, there's no reason to return [message] instead of message (def identity(*args): return args). Thus both
_ = lambda message: message
def _(message):
return message
fit perfectly.
...but a built-in would certainly run faster (and avoid bugs introduced by my own).
Bugs in such a trivial case are barely relevant. For an argument of predefined type, say str, we can use str() itself as an identity function (because of string interning it even retains object identity, see id note below) and compare its performance with the lambda solution:
$ python3 -m timeit -s "f = lambda m: m" "f('foo')"
10000000 loops, best of 3: 0.0852 usec per loop
$ python3 -m timeit "str('foo')"
10000000 loops, best of 3: 0.107 usec per loop
A micro-optimisation is possible. For example, the following Cython code:
test.pyx
cpdef str f(str message):
return message
Then:
$ pip install runcython3
$ makecython3 test.pyx
$ python3 -m timeit -s "from test import f" "f('foo')"
10000000 loops, best of 3: 0.0317 usec per loop
Build-in object identity function
Don't confuse an identity function with the id built-in function which returns the 'identity' of an object (meaning a unique identifier for that particular object rather than that object's value, as compared with == operator), its memory address in CPython.
Lots of good answers and discussion are in this topic. I just want to note that, in OP's case where there is a single argument in the identity function, compile-wise it doesn't matter if you use a lambda or define a function (in which case you should probably define the function to stay PEP8 compliant). The bytecodes are functionally identical:
import dis
function_method = compile("def identity(x):\n return x\ny=identity(Type('x', (), dict()))", "foo", "exec")
dis.dis(function_method)
1 0 LOAD_CONST 0 (<code object identity at 0x7f52cc30b030, file "foo", line 1>)
2 LOAD_CONST 1 ('identity')
4 MAKE_FUNCTION 0
6 STORE_NAME 0 (identity)
3 8 LOAD_NAME 0 (identity)
10 LOAD_NAME 1 (Type)
12 LOAD_CONST 2 ('x')
14 LOAD_CONST 3 (())
16 LOAD_NAME 2 (dict)
18 CALL_FUNCTION 0
20 CALL_FUNCTION 3
22 CALL_FUNCTION 1
24 STORE_NAME 3 (y)
26 LOAD_CONST 4 (None)
28 RETURN_VALUE
Disassembly of <code object identity at 0x7f52cc30b030, file "foo", line 1>:
2 0 LOAD_FAST 0 (x)
2 RETURN_VALUE
And lambda
import dis
lambda_method = compile("identity = lambda x: x\ny=identity(Type('x', (), dict()))", "foo", "exec")
dis.dis(lambda_method)
1 0 LOAD_CONST 0 (<code object <lambda> at 0x7f52c9fbbd20, file "foo", line 1>)
2 LOAD_CONST 1 ('<lambda>')
4 MAKE_FUNCTION 0
6 STORE_NAME 0 (identity)
2 8 LOAD_NAME 0 (identity)
10 LOAD_NAME 1 (Type)
12 LOAD_CONST 2 ('x')
14 LOAD_CONST 3 (())
16 LOAD_NAME 2 (dict)
18 CALL_FUNCTION 0
20 CALL_FUNCTION 3
22 CALL_FUNCTION 1
24 STORE_NAME 3 (y)
26 LOAD_CONST 4 (None)
28 RETURN_VALUE
Disassembly of <code object <lambda> at 0x7f52c9fbbd20, file "foo", line 1>:
1 0 LOAD_FAST 0 (x)
2 RETURN_VALUE
Adding to all answers:
Notice there is an implicit convention in Python stdlib, where a HOF defaulting it's key parameter function to the identity function, interprets None as such.
E.g. sorted, heapq.merge, max, min, etc.
So, it is not bad idea to consider your HOF expecting key to following the same pattern.
That is, instead of:
def my_hof(x, key=lambda _: _):
...
(whis is totally right)
You could write:
def my_hof(x, key=None):
if key is None: key = lambda _: _
...
If you want.
The thread is pretty old. But still wanted to post this.
It is possible to build an identity method for both arguments and objects. In the example below, ObjOut is an identity for ObjIn. All other examples above haven't dealt with dict **kwargs.
class test(object):
def __init__(self,*args,**kwargs):
self.args = args
self.kwargs = kwargs
def identity (self):
return self
objIn=test('arg-1','arg-2','arg-3','arg-n',key1=1,key2=2,key3=3,keyn='n')
objOut=objIn.identity()
print('args=',objOut.args,'kwargs=',objOut.kwargs)
#If you want just the arguments to be printed...
print(test('arg-1','arg-2','arg-3','arg-n',key1=1,key2=2,key3=3,keyn='n').identity().args)
print(test('arg-1','arg-2','arg-3','arg-n',key1=1,key2=2,key3=3,keyn='n').identity().kwargs)
$ py test.py
args= ('arg-1', 'arg-2', 'arg-3', 'arg-n') kwargs= {'key1': 1, 'keyn': 'n', 'key2': 2, 'key3': 3}
('arg-1', 'arg-2', 'arg-3', 'arg-n')
{'key1': 1, 'keyn': 'n', 'key2': 2, 'key3': 3}

Monkey patching operator overloads behaves differently in Python2 vs Python3 [duplicate]

This question already has answers here:
Overriding special methods on an instance
(5 answers)
Closed 6 years ago.
Consider the following code:
class Foo:
def __mul__(self,other):
return other/0
x = Foo()
x.__mul__ = lambda other:other*0.5
print(x.__mul__(5))
print(x*5)
In Python2 (with from future import print), this outputs
2.5
2.5
In Python3, this outputs
2.5
---------------------------------------------------------------------------
ZeroDivisionError Traceback (most recent call last)
<ipython-input-1-36322c94fe3a> in <module>()
5 x.__mul__ = lambda other:other*0.5
6 print(x.__mul__(5))
----> 7 print(x*5)
<ipython-input-1-36322c94fe3a> in __mul__(self, other)
1 class Foo:
2 def __mul__(self,other):
----> 3 return other/0
4 x = Foo()
5 x.__mul__ = lambda other:other*0.5
ZeroDivisionError: division by zero
I ran into this situation when I was trying to implement a type that supported a subset of algebraic operations. For one instance, I needed to modify the multiplication function for laziness: some computation must be deferred until the instance is multiplied with another variable. The monkey patch worked in Python 2, but I noticed it failed in 3.
Why does this happen?
Is there any way to get more flexible operator overloading in Python3?
That is not a monkeypatch.
This would have been a monkeypatch:
class Foo:
def __mul__(self, other):
return other / 0
Foo.__mul__ = lambda self,other: other * 0.5
x = Foo()
x*9 # prints 4.5
What was done with x.__mul__ = lambda other:other*0.5 was creating a __mul__ attribute on the x instance.
Then, it was expected that x*5 would call x.__mul__(5). And it did, in Python 2.
In Python 3, it called Foo.__mul__(x, 5), so the attribute was not used.
Python 2 would have done the same as Python 3, but it did not because Foo was created as an old-style class.
This code would be equivalent for Python 2 and Python 3:
class Foo(object):
def __mul__(self,other):
return other/0
x = Foo()
x.__mul__ = lambda other:other*0.5
print(x.__mul__(5))
print(x*5)
That will raise an exception. Note the (object).
You can't override the special methods on instance level. Based on python's documentation:
For custom classes, implicit invocations of special methods are only guaranteed to work correctly if defined on an object’s type, not in the object’s instance dictionary.
The rationale behind this behaviour lies with a number of special methods such as __hash__() and __repr__() that are implemented by all objects, including type objects. If the implicit lookup of these methods used the conventional lookup process, they would fail when invoked on the type object itself:
>>> 1 .__hash__() == hash(1)
True
>>> int.__hash__() == hash(int)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: descriptor '__hash__' of 'int' object needs an argument
So one simple way is defining a regular function for your your monkey-patching aims, and assign your new method to it:
In [45]: class Foo:
def __init__(self, arg):
self.arg = arg
def __mul__(self,other):
return other * self.arg
def _mul(self, other):
return other/0
Demo:
In [47]: x = Foo(10)
In [48]: x * 3
Out[48]: 30
In [49]: my_func = lambda x: x * 0.5
In [50]: x._mul = my
my_func mypub/
In [50]: x._mul = my_func
In [51]: x._mul(4)
Out[51]: 2.0

Difference between generator expression and generator function

Is there any difference — performance or otherwise — between generator expressions and generator functions?
In [1]: def f():
...: yield from range(4)
...:
In [2]: def g():
...: return (i for i in range(4))
...:
In [3]: f()
Out[3]: <generator object f at 0x109902550>
In [4]: list(f())
Out[4]: [0, 1, 2, 3]
In [5]: list(g())
Out[5]: [0, 1, 2, 3]
In [6]: g()
Out[6]: <generator object <genexpr> at 0x1099056e0>
I'm asking because I want to decide how I should decide between using the two. Sometimes generator functions are clearer and then the choice is clear. I am asking about those times when code clarity does not make one choice obvious.
The functions you provided have completely different semantics in the general case.
The first one, with yield from, passes the control to the iterable. This means that calls to send() and throw() during the iteration will be handled by the iterable and not by the function you are defining.
The second function only iterates over the elements of the iterable, and it will handle all the calls to send() and throw(). To see the difference check this code:
In [8]: def action():
...: try:
...: for el in range(4):
...: yield el
...: except ValueError:
...: yield -1
...:
In [9]: def f():
...: yield from action()
...:
In [10]: def g():
...: return (el for el in action())
...:
In [11]: x = f()
In [12]: next(x)
Out[12]: 0
In [13]: x.throw(ValueError())
Out[13]: -1
In [14]: next(x)
---------------------------------------------------------------------------
StopIteration Traceback (most recent call last)
<ipython-input-14-5e4e57af3a97> in <module>()
----> 1 next(x)
StopIteration:
In [15]: x = g()
In [16]: next(x)
Out[16]: 0
In [17]: x.throw(ValueError())
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-17-1006c792356f> in <module>()
----> 1 x.throw(ValueError())
<ipython-input-10-f156e9011f2f> in <genexpr>(.0)
1 def g():
----> 2 return (el for el in action())
3
ValueError:
In fact, due to this reason, yield from probably has a higher overhead than the genexp, even though it is probably irrelevant.
Use yield from only when the above behaviour is what you want or if you are iterating over a simple iterable that is not a generator (so that yield from is equivalent to a loop + simple yields).
Stylistically speaking I'd prefer:
def h():
for el in range(4):
yield el
Instead of returning a genexp or using yield from when dealing with generators.
In fact the code used by the generator to perform the iteration is almost identical to the above function:
In [22]: dis.dis((i for i in range(4)).gi_code)
1 0 LOAD_FAST 0 (.0)
>> 3 FOR_ITER 11 (to 17)
6 STORE_FAST 1 (i)
9 LOAD_FAST 1 (i)
12 YIELD_VALUE
13 POP_TOP
14 JUMP_ABSOLUTE 3
>> 17 LOAD_CONST 0 (None)
20 RETURN_VALUE
As you can see it does a FOR_ITER + YIELD_VALUE. note that the argument (.0), is iter(range(4)). The bytecode of the function also contains the calls to LOAD_GLOBAL and GET_ITER that are required to lookup range and obtain its iterable. However this actions must be performed by the genexp too, just not inside its code but before calling it.
In addition to #Bakuriu's good point — that generator functions implement send(), throw(), and close() — there is another difference I've run into. Sometimes, you have some setup code that happens before the yield statement is reached. If that setup code can raise exceptions, then the generator-returning version might be preferable to the generator function because it will raise the exception sooner. E.g.,
def f(x):
if x < 0:
raise ValueError
for i in range(4):
yield i * i
def g(x):
if x < 0:
raise ValueError
return (i * i for i in range(x))
print(list(f(4)))
print(list(g(4)))
f(-1) # no exception until the iterator is consumed!
g(-1)
If one wants both behaviors, I think the following is best:
def f(count):
x = 0
for i in range(count):
x = yield i + (x or 0)
def protected_f(count):
if count < 0:
raise ValueError
return f(count)
it = protected_f(10)
try:
print(next(it))
x = 0
while True:
x = it.send(x)
print(x)
except StopIteration:
pass
it = protected_f(-1)

python passing positional arguments to another function call [duplicate]

This question already has answers here:
What does ** (double star/asterisk) and * (star/asterisk) do for parameters?
(25 answers)
What do *args and **kwargs mean? [duplicate]
(5 answers)
Closed 9 years ago.
Can someone please explain to me how the below works:
class Memoize:
def __init__(self, f):
self.f = f
self.memo = {}
def __call__(self, *args):
if not args in self.memo:
self.memo[args] = self.f(*args)
return self.memo[args]
Then:
def factorial(k):
if k < 2: return 1
return k * factorial(k - 1)
factorial = Memoize(factorial)
This is taken from this question. I would like to understand how does self.f(*args) work. args is a tuple of positional arguments. When I try to do:
*(1,2,3)
I get syntax error. I thought it's some kind on unpacking operator or something like that. Does asterisk with a tuple mean anything independently or does it exist only in the context of a function call? Thank you for any explanations.
It /is/ an unpacking operator. However, it doesn't always work that way when called directly in the interpreter. Check this out:
In [90]: def addThese(a,b,c):
....: print 'a', a
....: print 'b', b
....: print 'c', c
....: print 'a+b+c', a+b+c
....: return a+b+c
....:
In [91]: args = (1,3,5)
In [92]: addThese(1,3,5)
a 1
b 3
c 5
a+b+c 9
Out[92]: 9
In [93]: addThese(args)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-93-cc47562f352a> in <module>()
----> 1 addThese(args)
TypeError: addThese() takes exactly 3 arguments (1 given)
In [94]: addThese(*args)
a 1
b 3
c 5
a+b+c 9
Out[94]: 9

Categories