After asking this question, it received a comment about how you could do something like this:
>>> def a(n):
print(n)
return a
>>> b = a(3)(4)(5)
3
4
5
Is it possible to use this or similar concepts to make it possible to index lists like my_list(n) instead of my_list[n]?
You'd have to use a custom class, and give it a __call__ special method to make it callable. A subclass of list would do nicely here:
class CallableList(list):
def __call__(self, item):
return self[item]
You cannot use this to assign to an index, however, only item access works. Slicing would require you to use to create a slice() object:
a = CallableList([1, 2, 3])
a(2)
a(slice(None, 2, None))
nested = CallableList([1, 2, CallableList([4, 5, 6])])
nested(2)(-1)
For anything more, you'd have to create a custom Python syntax parser to build an AST, then compile to bytecode from there.
the parentheses in my_list() are treated as a function call. If you want, you could write your own class that wraps a list and overwrite the call method to index into the list.
class MyList(object):
def __init__(self, alist):
self._list = alist
def __call__(self, index):
return self._list[index]
>>> mylist = MyList(['a','b','c','d','e','f'])
>>> mylist(3)
'd'
>>> mylist(4)
'e'
You could create a function that returns a lambda function:
def make_callable(some_list):
return lambda x: some_list[x]
original_list = [ 1, 2, 3, 4 ]
callable_list = make_callable(original_list)
print(callable_list(1)) # Prints 2
Related
I am just checking where is the limit of changing python using python (without modifying interpreter and/or C code).
I know that I can basically monkey patch every builtin function like this:
import builtins
int(1)
# 1
def new_int(number):
return number + 1
builtins.int = new_int
int(1)
# 2
I know I can turn python classes upside down using special methods like __new__, __get__ etc. and that I can overload any operator.
but is there any way to monkey patch parentheses? so that instead of creating list, python will do something else with elements between [...] like creating tuple.
# normal
old = [1,2,3,4]
print(old)
# [1, 2, 3, 4]
type(old)
# <class 'list'>
# some strange code that monkeypatches [] so that instead of list, it creates tuple
def monkey_patch_list(values):
return tuple(values)
[] = monkey_patch_list # of course it is wrong
# new
new = [1,2,3,4]
print(new)
# (1, 2, 3, 4)
type(old)
# <class 'tuple'>
Probably there is no way to do it just in python but maybe somewhere hidden in python code there is a definition of handling [] so that I can mess it up. If there is anyone crazy like me and knows how to do it I will appreciate help.
Disclaimer:
Don't worry just for fun, thanks ;)
This can be done as a function decorator as long as the lists to be replaced as tuples are defined in a function.
To do that, use ast.NodeTransformer to replace any ast.List node with an equivalent ast.Tuple node in the function's AST:
import ast
import inspect
from textwrap import dedent
class ForceTuples(ast.NodeTransformer):
def visit_List(self, node):
return ast.Tuple(**vars(node))
# remove 'force_tuples' from the function's decorator list to avoid re-decorating during exec
def visit_FunctionDef(self, node):
node.decorator_list = [
decorator for decorator in node.decorator_list
if not isinstance(decorator, ast.Name) or decorator.id != 'force_tuples'
]
self.generic_visit(node)
return node
def force_tuples(func):
tree = ForceTuples().visit(ast.parse(dedent(inspect.getsource(func))))
ast.fix_missing_locations(tree)
scope = {}
exec(compile(tree, inspect.getfile(func), 'exec'), func.__globals__, scope)
return scope[func.__name__]
so that:
#force_tuples
def foo():
bar = [1, 2, 3, 4]
print(bar)
print(type(bar))
foo()
outputs:
(1, 2, 3, 4)
<class 'tuple'>
Demo: https://replit.com/#blhsing/MellowLinearSystemsanalysis
This question already has answers here:
Making objects iterable?
(3 answers)
Closed 2 years ago.
what should I do when I would like to return object from class and at the same time can print this result.
I did this:
class Engine:
def __init__(self, mylist):
self._mylist = list(mylist)
def __str__(self) -> str:
return ','.join(el.__str__() for el in self._mylist)
def __repr__(self):
return self._mylist
but when I do:
p = Engine(lst)
print(p)
for x in p:
print("Element", x)
I get:
element1, element2, element3
TypeError: 'Engine' object is not iterable
What should I do to have possibility to print this result and also iterate over it just like the object?
You can refer to one of the class variables for this to work. By default, you can't iterate over an entire object. The proper way would be to iterate through the list you passed, which is now an object variable. It works if you run:
lst = [1, 2, 3, 4, 5]
p = Engine(lst)
print(p)
for ix in p._mylist:
print("Element", ix)
But I'm guessing you're seeking another solution because you named the list _mylist, which generally means that this variable is not supposed to be accessed from the outside.
Therefore, you can instead turn the object itself into an iterable. For this, you need to define the __iter__ dunder method, and define it's behavior when you treat it as an iterator.
def __iter__(self):
return iter(self._mylist)
class Engine:
def __init__(self, mylist):
self._mylist = list(mylist)
def __str__(self) -> str:
return ','.join(el.__str__() for el in self._mylist)
def __repr__(self):
return self._mylist
def __iter__(self):
return iter(self._mylist)
lst = [1, 2, 3, 4, 5]
p = Engine(lst)
print(p)
for ix in p:
print("Element", ix)
1,2,3,4,5
Element 1
Element 2
Element 3
Element 4
Element 5
Short Version
Can I make a read-only list using Python's property system?
Long Version
I have created a Python class that has a list as a member. Internally, I would like it to do something every time the list is modified. If this were C++, I would create getters and setters that would allow me to do my bookkeeping whenever the setter was called, and I would have the getter return a const reference, so that the compiler would yell at me if I tried to do modify the list through the getter. In Python, we have the property system, so that writing vanilla getters and setters for every data member is (thankfully) no longer necessary.
However, consider the following script:
def main():
foo = Foo()
print('foo.myList:', foo.myList)
# Here, I'm modifying the list without doing any bookkeeping.
foo.myList.append(4)
print('foo.myList:', foo.myList)
# Here, I'm modifying my "read-only" list.
foo.readOnlyList.append(8)
print('foo.readOnlyList:', foo.readOnlyList)
class Foo:
def __init__(self):
self._myList = [1, 2, 3]
self._readOnlyList = [5, 6, 7]
#property
def myList(self):
return self._myList
#myList.setter
def myList(self, rhs):
print("Insert bookkeeping here")
self._myList = rhs
#property
def readOnlyList(self):
return self._readOnlyList
if __name__ == '__main__':
main()
Output:
foo.myList: [1, 2, 3]
# Note there's no "Insert bookkeeping here" message.
foo.myList: [1, 2, 3, 4]
foo.readOnlyList: [5, 6, 7, 8]
This illustrates that the absence of the concept of const in Python allows me to modify my list using the append() method, despite the fact that I've made it a property. This can bypass my bookkeeping mechanism (_myList), or it can be used to modify lists that one might like to be read-only (_readOnlyList).
One workaround would be to return a deep copy of the list in the getter method (i.e. return self._myList[:]). This could mean a lot of extra copying, if the list is large or if the copy is done in an inner loop. (But premature optimization is the root of all evil, anyway.) In addition, while a deep copy would prevent the bookkeeping mechanism from being bypassed, if someone were to call .myList.append() , their changes would be silently discarded, which could generate some painful debugging. It would be nice if an exception were raised, so that they'd know they were working against the class' design.
A fix for this last problem would be not to use the property system, and make "normal" getter and setter methods:
def myList(self):
# No property decorator.
return self._myList[:]
def setMyList(self, myList):
print('Insert bookkeeping here')
self._myList = myList
If the user tried to call append(), it would look like foo.myList().append(8), and those extra parentheses would clue them in that they might be getting a copy, rather than a reference to the internal list's data. The negative thing about this is that it is kind of un-Pythonic to write getters and setters like this, and if the class has other list members, I would have to either write getters and setters for those (eww), or make the interface inconsistent. (I think a slightly inconsistent interface might be the least of all evils.)
Is there another solution I am missing? Can one make a read-only list using Python's property system?
You could have method return a wrapper around your original list -- collections.Sequence might be of help for writing it. Or, you could return a tuple -- The overhead of copying a list into a tuple is often negligible.
Ultimately though, if a user wants to change the underlying list, they can and there's really nothing you can do to stop them. (After all, they have direct access to self._myList if they want it).
I think that the pythonic way to do something like this is to document that they shouldn't change the list and that if the do, then it's their fault when their program crashes and burns.
despite the fact that I've made it a property
It does not matter if it's a property, you are returning a pointer to the list so you can modify it.
I'd suggest creating a list subclass and overriding append and __add__ methods
The proposed solutions of returning a tuple or subclassing list for the return, seem like nice solutions, but i was wondering whether it wouldn't be easier to subclass the decorator instead? Not sure it this might be a stupid idea:
using this safe_property protects against accidental sending mixed API signals (internal immutable attributes are "protected" against all operations, while the for mutable attributes, some operations are still allowed with the normal property builtin)
advantage: easier to use than to implement custom return types everywhere -> easier to internalize
disadvantage: necessity to use different name
class FrozenList(list):
def _immutable(self, *args, **kws):
raise TypeError('cannot change object - object is immutable')
pop = _immutable
remove = _immutable
append = _immutable
clear = _immutable
extend = _immutable
insert = _immutable
reverse = _immutable
class FrozenDict(dict):
def _immutable(self, *args, **kws):
raise TypeError('cannot change object - object is immutable')
__setitem__ = _immutable
__delitem__ = _immutable
pop = _immutable
popitem = _immutable
clear = _immutable
update = _immutable
setdefault = _immutable
class safe_property(property):
def __get__(self, obj, objtype=None):
candidate = super().__get__(obj, objtype)
if isinstance(candidate, dict):
return FrozenDict(candidate)
elif isinstance(candidate, list):
return FrozenList(candidate)
elif isinstance(candidate, set):
return frozenset(candidate)
else:
return candidate
class Foo:
def __init__(self):
self._internal_lst = [1]
#property
def internal_lst(self):
return self._internal_lst
#safe_property
def internal_lst_safe(self):
return self._internal_lst
if __name__ == '__main__':
foo = Foo()
foo.internal_lst.append(2)
# foo._internal_lst is now [1, 2]
foo.internal_lst_safe.append(3)
# this throws an exception
Very much interested in other opinions on this as i haven't seen this implemented somewhere else.
Why is a list preferable to a tuple? Tuples are, for most intents and purposes, 'immutable lists' - so by nature they will act as read-only objects that can't be directly set or modified. At that point one simply needs to not write said setter for that property.
>>> class A(object):
... def __init__(self, list_data, tuple_data):
... self._list = list(list_data)
... self._tuple = tuple(tuple_data)
... #property
... def list(self):
... return self._list
... #list.setter
... def list(self, new_v):
... self._list.append(new_v)
... #property
... def tuple(self):
... return self._tuple
...
>>> Stick = A((1, 2, 3), (4, 5, 6))
>>> Stick.list
[1, 2, 3]
>>> Stick.tuple
(4, 5, 6)
>>> Stick.list = 4 ##this feels like a weird way to 'cover-up' list.append, but w/e
>>> Stick.list = "Hey"
>>> Stick.list
[1, 2, 3, 4, 'Hey']
>>> Stick.tuple = 4
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: can't set attribute
>>>
This can be accomplished by using the Sequence type hint, which unlike list is non-modifiable:
from typing import Sequence
def foo() -> Sequence[int]:
return []
result = foo()
result.append(10)
result[0] = 10
Both mypy and pyright will give an error when trying to modify a list that is hinted with Sequence:
$ pyright /tmp/foo.py
/tmp/foo.py:7:8 - error: Cannot access member "append" for type "Sequence[int]"
Member "append" is unknown (reportGeneralTypeIssues)
/tmp/foo.py:8:1 - error: "__setitem__" method not defined on type "Sequence[int]" (reportGeneralTypeIssues)
2 errors, 0 warnings, 0 informations
Python itself however ignores those hints, so a little must be taken to make sure that either of those type checkers are run regularly or part of the build process.
There is also a Final type hint, that acts similar to C++ const, it however only provides protection of the variable in which the list reference is stored, not for the list itself, so it's not useful here, but might be of use in other situations.
The two main suggestions seem to be either using a tuple as a read-only list, or subclassing list. I like both of those approaches.
Returning a tuple from the getter, or using a tuple in the first place, prevents one from using the += operator, which can be a useful operator and also triggers the bookkeeping mechanism by calling the setter. However, returning a tuple is a one-line change, which is nice if you would like to program defensively but judge that adding a whole other class to your script might be unnecessarily complicated.
Here is a script that illustrates both approaches:
import collections
def main():
foo = Foo()
print('foo.myList:', foo.myList)
try:
foo.myList.append(4)
except RuntimeError:
print('Appending prevented.')
# Note that this triggers the bookkeeping, as we would like.
foo.myList += [3.14]
print('foo.myList:', foo.myList)
try:
foo.readOnlySequence.append(8)
except AttributeError:
print('Appending prevented.')
print('foo.readOnlySequence:', foo.readOnlySequence)
class UnappendableList(collections.UserList):
def __init__(self, *args, **kwargs):
data = kwargs.pop('data')
super().__init__(self, *args, **kwargs)
self.data = data
def append(self, item):
raise RuntimeError('No appending allowed.')
class Foo:
def __init__(self):
self._myList = [1, 2, 3]
self._readOnlySequence = [5, 6, 7]
#property
def myList(self):
return UnappendableList(data=self._myList)
#myList.setter
def myList(self, rhs):
print('Insert bookkeeping here')
self._myList = rhs
#property
def readOnlySequence(self):
# or just use a tuple in the first place
return tuple(self._readOnlySequence)
if __name__ == '__main__':
main()
Output:
foo.myList: [1, 2, 3]
Appending prevented.
Insert bookkeeping here
foo.myList: [1, 2, 3, 3.14]
Appending prevented.
foo.readOnlySequence: (5, 6, 7)
This answer was posted as an edit to the question Python read-only lists using the property decorator by the OP ngb under CC BY-SA 3.0.
I would like to map a method over a list of objects instantiating different classes. All the objects inherit from the same base class and define a method of the desired name.
To make it clearer consider the following code:
class A:
def __init__(self, x):
self.x = x
def foo (self):
return self.x
class B(A):
def foo (self):
return self.x+1
class C(A):
def foo (self):
return self.x-1
Now consider a list of objects instantiating the classes B and C. I would like to do something like that:
result = []
for obj in [B(1), C(1)]:
result.append(obj.foo())
How would you proceed to map the method foo on each element of the list? Is it at all possible? The best I could come up with is something like that:
map(A.foo, [B(1), C(1)])
but clearly it doesn't return my desired result. How can I specify the method related to the object?
I hope I made myself clear.
NB: I work primarily with Python2.7, but I would equally be interested in solutions valid for "newer" versions.
Map(A.foo, [B(1), C(1)]) is basically doing A.foo(B(1)) and A.foo(C(1)) which isn't what you are looking for.
Using your classes from above, I would just do:
In: objs = [B(1), C(1)]
In: [x.foo() for x in objs]
Out: [2, 0]
Amjith has a pure map implementation if you'd prefer that.
>>> map(lambda x: x.foo(), [B(1), C(1)])
>>> [2, 0]
The lambda function will take each object in the list and call foo() on that object. Thus the resulting list will have the results returned by the corresponding object's foo().
For most practical purposes, I'd recommend #AlG's list comprehension, but you can do this with map as well:
>>> import operator
>>> map(operator.methodcaller("foo"), [B(1), C(1)])
[2, 0]
Suppose you have something like this:
class intlist:
def __init__(self,l = []):
self.l = l
def add(self,a):
self.l.append(a)
def appender(a):
obj = intlist()
obj.add(a)
print obj.l
if __name__ == "__main__":
for i in range(5):
appender(i)
A function creates an instance of intlist and calls on this fresh instance the method append on the instance attribute l.
How comes the output of this code is:
[0]
[0, 1]
[0, 1, 2]
[0, 1, 2, 3]
[0, 1, 2, 3, 4]
?
If i switch
obj = intlist()
with
obj = intlist(l=[])
I get the desired output
[0]
[1]
[2]
[3]
[4]
Why this happens?
Thanks
Ah, you've hit one of the common Python gotchas: default values are computed once, then re-used. So, every time __init__ is called, the same list is being used.
This is the Pythonic way of doing what you want:
def __init__(self, l=None):
self.l = [] if l is None else l
For a bit more information, check out the Python docs (especially about three paragraphs after that heading).
Edit: There is a much better description in another answer.
When you set the default value of l=[] in __init__, you're actually using the same list each time. Instead, you could try something like:
class intlist:
def __init__(self, l=None):
if l is None:
self.l = []
else:
self.l = l
The issue is that when you are saying
def __init__(self,l = []):
You are telling Python to use the same list, [], for each invocation of the constructor. So each time obj = intlist() is called the same list is appended to.
What you should do instead is set l to a default value of None, which is a scalar (so your code will work as expected if it is used multiple times). Then, if l is None, initialize a new class member as []. Otherwise just assign the member variable to l.
obj = intlist() calls your __init__() function which uses the same array for every instance of the class.
obj = intlist(l=[]) creates a new array for every instance.
For more information I suggest reading this: http://effbot.org/zone/default-values.htm
The behavior occurs because all calls to your __init__ method share the same default list.
Try:
class intlist:
def __init__(self, l):
self.l = l if (l is not None) else []
def add(self,a):
self.l.append(a)
EDIT: Use is not, per SilentGhost
Be careful with default parameters of types like lists and dicts. Each instance of intlist gets that same list object from the default parameter.