I have a (bad?) habit of displaying classes in Python like structures in Matlab, where each attribute is printed along with its value in a nice clean layout. This is done by implementing the __repr__ method in the class.
When working with objects inside of dictionaries or lists, this display style can be a bit distracting. In this case I'd like to do a more basic display.
Here's the envisioned pseudocode:
def __repr__(self):
if direct_call():
return do_complicated_printing(self)
else:
#something simple that isn't a ton of lines/characters
return type(self)
In this code direct_call() means that this isn't being called as part of another display call. Perhaps this might entail looking for repr in the stack? How would I implement direct call detection?
So I might have something like:
>>> data
<class my_class> with properties:
a: 1
cheese: 2
test: 'no testing'
But in a list I'd want a display like:
>>> data2 = [data, data, data, data]
>>> data2
[<class 'my_class'>,<class 'my_class',<class 'my_class'>,<class 'my_class'>]
I know it is possible for me to force this type of display by calling some function that does this, but I want my_class to be able to control this behavior, without extra work from the user in asking for it.
In other words, this is not a solution:
>>> print_like_I_want(data2)
This is a strange thing to want to do, and generally a function or method ought to do the same thing whoever is calling it. But in this case, __repr__ is only meant for the programmer's convenience, so convenience seems like a good enough reason to make it work the way you're asking for.
However, unfortunately what you want isn't actually possible, because for whatever reason, the list.__repr__ method isn't visible on the stack. I tested in Python 3.5.2 and Python 3.8.1:
>>> class ReprRaises:
... def __repr__(self):
... raise Exception()
...
>>> r = ReprRaises()
>>> r
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in __repr__
Exception
>>> [r]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in __repr__
Exception
As you can see, the stack is the same whether or not the object being repr'd is in a list. (The __repr__ frame on the stack belongs to the ReprRaises class, not list.)
I also tested using inspect.stack:
>>> import inspect
>>> class ReprPrints:
... def __repr__(self):
... print(*inspect.stack(), sep='\n')
... return 'foo'
>>> r = ReprPrints()
>>> r
FrameInfo(frame=<frame object at 0x7fcbe4a38588>, filename='<stdin>', lineno=3, function='__repr__', code_context=None, index=None)
FrameInfo(frame=<frame object at 0x7fcbe44fb388>, filename='<stdin>', lineno=1, function='<module>', code_context=None, index=None)
foo
>>> [r]
FrameInfo(frame=<frame object at 0x7fcbe4a38588>, filename='<stdin>', lineno=3, function='__repr__', code_context=None, index=None)
FrameInfo(frame=<frame object at 0x7fcbe44fb388>, filename='<stdin>', lineno=1, function='<module>', code_context=None, index=None)
[foo]
Again, there's no visible difference in the call stack between the object itself vs. the object in a list; so there's nothing for your __repr__ to check for.
So, the closest you can get is some kind of print_like_I_want function. This can at least be written in a way that lets each class define its own behaviour:
def pp(obj):
try:
_pp = obj._pp
except AttributeError:
print(repr(obj))
else:
print(_pp())
The only way I can think of to do it with fewer keypresses is by overloading a unary operator, like the usually-useless unary plus:
>>> class OverloadUnaryPlus:
... def __repr__(self):
... return 'foo'
... def __pos__(self):
... print('bar')
...
>>> obj = OverloadUnaryPlus()
>>> obj
foo
>>> +obj
bar
__repr__ is intended to provide a short, often programmatic display of an object. It's used as the method of choice for all built in containers to display elements. You should therefore override __repr__ to provide your short output.
__str__ is the function intended for the full fancy display of an object. It's what normally shows up when you print an object. You can also trigger it by calling str. You should put your long fancy output in __str__, not __repr__.
The only modification you will have to make is to explicitly call print(obj) or str(obj) rather than repr(obj) or just obj in the REPL. As #kaya3's excellent answer shows, stack inspection won't help you much, and in my opinion would not be a clean solution even if it did.
Related
Let's say you work with a wrapper object:
class IterOrNotIter:
def __init__(self):
self.f = open('/tmp/toto.txt')
def __getattr__(self, item):
try:
return self.__getattribute__(item)
except AttributeError:
return self.f.__getattribute__(item)
This object implements __iter__, because it passes any call to it to its member f, which implements it. Case in point:
>>> x = IterOrNotIter()
>>> x.__iter__().__next__()
'Whatever was in /tmp/toto.txt\n'
According to the documentation (https://docs.python.org/3/library/stdtypes.html#iterator-types), IterOrNotIter should thus be iterable.
However, the Python interpreter does not recognize an IterOrNotIter object as actually being iterable:
>>> x = IterOrNotIter()
>>> for l in x:
... print(l)
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'IterOrNotIter' object is not iterable
Whereas this works:
>>> x = IterOrNotIter()
>>> for l in x.f:
... print(l)
...
Whatever was in /tmp/toto.txt
I don't understand why.
Basically because your class just doesn't have a real __iter__ method:
>>> hasattr(IterOrNotIter, '__iter__')
False
So it doesn't qualify as iterator because the actual check for __iter__ checks for the existence instead of assuming it's implemented. So workarounds with __getattr__ or __getattribute__ (unfortunatly) don't work.
This is actually mentioned in the documentation for __getattribute__:
Note
This method may still be bypassed when looking up special methods as the result of implicit invocation via language syntax or built-in functions. See Special method lookup.
The latter section also explains the why:
Bypassing the __getattribute__() machinery in this fashion provides significant scope for speed optimisations within the interpreter, at the cost of some flexibility in the handling of special methods (the special method must be set on the class object itself in order to be consistently invoked by the interpreter).
Emphasis mine.
Given how dynamic Python is, I'll be shocked if this isn't somehow possible:
I would like to change the implementation of sys.stdout.write.
I got the idea from this answer to another question of mine: https://stackoverflow.com/a/24492990/901641
I tried to simply write this:
original_stdoutWrite = sys.stdout.write
def new_stdoutWrite(*a, **kw):
original_stdoutWrite("The new one was called! ")
original_stdoutWrite(*a, **kw)
sys.stdout.write = new_stdoutWrite
But it tells me AttributeError: 'file' object attribute 'write' is read-only.
This is a nice attempt to keep me from doing something potentially (probably) stupid, but I'd really like to go ahead and do it anyways. I suspect the interpreter has some kind of lookup table its using that I can modify, but I couldn't find anything like that on Google. __setattr__ didn't work, either - it returned the exact same error about the attribute being read-only.
I'm specifically looking for a Python 2.7 solution, if that's important, although there's no reason to resist throwing in answers that work for other versions since I suspect other people in the future will look here with similar questions regarding other versions.
Despite its dynamicity, Python does not allow monkey-patching built-in types such as file. It even prevents you to do so by modifying the __dict__ of such a type — the __dict__ property returns the dict wrapped in a read-only proxy, so both assignment to file.write and to file.__dict__['write'] fail. And for at least two good reasons:
the C code expects the file built-in type to correspond to the PyFile type structure, and file.write to the PyFile_Write() function used internally.
Python implements caching of attribute access on types to speed up method lookup and instance method creation. This cache would be broken if it were allowed to directly assign to type dicts.
Monkey-patching is of course allowed for classes implemented in Python which can handle dynamic modifications just fine.
However... if you really know what you are doing, you can use the low-level APIs such as ctypes to hook into the implementation and get to the type dict. For example:
# WARNING: do NOT attempt this in production code!
import ctypes
def magic_get_dict(o):
# find address of dict whose offset is stored in the type
dict_addr = id(o) + type(o).__dictoffset__
# retrieve the dict object itself
dict_ptr = ctypes.cast(dict_addr, ctypes.POINTER(ctypes.py_object))
return dict_ptr.contents.value
def magic_flush_mro_cache():
ctypes.PyDLL(None).PyType_Modified(ctypes.py_object(object))
# monkey-patch file.write
dct = magic_get_dict(file)
dct['write'] = lambda f, s, orig_write=file.write: orig_write(f, '42')
# flush the method cache for the monkey-patch to take effect
magic_flush_mro_cache()
# magic!
import sys
sys.stdout.write('hello world\n')
Despite Python mostly being a dynamic language, there are native objects types like str, file (including stdout), dict, and list that are actually implemented in low-level C and are completely static:
>>> a = []
>>> a.append = 'something else'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'list' object attribute 'append' is read-only
>>> a.hello = 3
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'list' object has no attribute 'hello'
>>> a.__dict__ # normal python classes would have this
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'list' object has no attribute '__dict__'
If your object is native C code, your only hope is to use an actual regular class. For your case, like already mentioned, you could do something like:
class NewOut(type(sys.stdout)):
def write(self, *args, **kwargs):
super(NewOut, self).write('The new one was called! ')
super(NewOut, self).write(*args, **kwargs)
sys.stdout = NewOut()
or, to do something similar to your original code:
original_stdoutWrite = sys.stdout.write
class MyClass(object):
pass
sys.stdout = MyClass()
def new_stdoutWrite(*a, **kw):
original_stdoutWrite("The new one was called! ")
original_stdoutWrite(*a, **kw)
sys.stdout.write = new_stdoutWrite
In a bit of my code I'm using the nice memoized class from the Python Decorator Library.
One of the libraries I'm using uses introspection on a function to get the number of arguments it takes, and fails on the decorated function. Specifically, it checks the co_argcount variable.
if (PyInt_AsLong(co_argcount) < 1) {
PyErr_SetString(PyExc_TypeError, "This function has no parameters to mini\
mize.");
It seems the argcount isn't being transferred to the memoized function.
>>> def f(x):
... return x
...
>>> f.func_code.co_argcount
1
>>> g = memoized(f)
>>> g.func_code.co_argcount
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'memoized' object has no attribute 'func_code'
How can I modify the memoized class so that my memoized functions look, taste, and smell like the original function?
You need to create a signature-preserving decorator. The easiest way to do that is to use the library http://pypi.python.org/pypi/decorator which takes care of preserving the signature for you.
The internals of the library are quite ugly (it uses exec!) but it encapsulates them quite well.
Add that to your memoized class
def __getattr__(self, name):
if name.startswith('func_'):
return getattr(self.func, name)
raise AttributeError
So it'll pass attribute lookup for func_... to the original function.
Maybe you will also want to write a __setattr__ function to deny writing these attributes, but it's not necessary if you know you won't try to change the values.
It's a thing that bugged me for a while. Why can't I do:
>>> a = ""
>>> a.foo = 2
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'str' object has no attribute 'foo'
...while I can do the following?
>>> class Bar():
... pass
...
>>> a = Bar()
>>> a.foo = 10 #ok!
What's the rule here? Could you please point me to some description?
You can add attributes to any object that has a __dict__.
x = object() doesn't have it, for example.
Strings and other simple builtin objects also don't have it.
Classes using __slots__ also do not have it.
Classes defined with class have it unless the previous statement applies.
If an object is using __slots__ / doesn't have a __dict__, it's usually to save space. For example, in a str it would be overkill to have a dict - imagine the amount of bloat for a very short string.
If you want to test if a given object has a __dict__, you can use hasattr(obj, '__dict__').
This might also be interesting to read:
Some objects, such as built-in types and their instances (lists, tuples, etc.) do not have a __dict__. Consequently user-defined attributes cannot be set on them.
Another interesting article about Python's data model including __dict__, __slots__, etc. is this from the python reference.
Is there a possibility to create any python object that will be not sortable? So that will be an exception when trying to sort a list of that objects?
I created a very simple class, didn't define any comparison methods, but still instances of this class are comparable and thus sortable. Maybe, my class inherits comparison methods from somewhere. But I don't want this behaviour.
You could define a __cmp__ method on the class and always raise an exception when it is called. That might do the trick.
Out of curiosity, why?
As Will McCutchen has mentioned, you can define a __cmp__ method that raises an exception to prevent garden variety sorting. Something like this:
class Foo(object):
def __cmp__(self, other):
raise Exception()
a = [Foo(), Foo(), Foo()]
a.sort()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in __cmp__
Exception
However, you cannot truly prevent a developer from sorting a list of your objects. Using the key or cmp argument with list.sort() or with the built-in standalone sorted() function , anyone can circumvent the __cmp__ method by using a custom comparison function or sorting key.
# continuing from above
>>> a = [Foo(), Foo(), Foo()]
>>> a
[<__main__.Foo object at 0x1004a3350>, <__main__.Foo object at 0x1004a3390>,
<__main__.Foo object at 0x1004a33d0>]
>>> a.sort(key=id, reverse=True)
>>> # or a.sort(cmp=lambda a, b: cmp(id(b), id(a)))
>>> # or sorted(a, key=id)
>>> # etc...
[<__main__.Foo object at 0x1004a33d0>, <__main__.Foo object at 0x1004a3390>,
<__main__.Foo object at 0x1004a3350>]
As others will point out, I'm not sure there's much value in trying to prevent someone from sorting an object. If this isn't just a curious itch you're trying to scratch, what's the use case for this?
The default list sorting uses the built-in cmp() function on its elements. The cmp() function checks if its arguments (2 elements from your list) have a __cmp__() method. If yes, this method is used for comparison. Otherwise, as in your case, the argument object IDs (return value of the built-in function id()) are used for comparison.
To let the sorting fail, you could define a comparison method which throws an Exception:
>>> class X(object):
... def __cmp__(self, other):
... raise StandardError # or whatever Exception you need
...
>>> l = [X(), X(), X()]
>>> l.sort()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in __cmp__
StandardError
For what it's worth, in Python 3 the default will be for new items to not be comparable (and hence not sortable). In Python 2, you have to explicitly create a __cmp__ or __lt__ method, as others have said.
Why don't you just write a class that contains a list object and provides methods to access the data inside? By doing that you would effectively hide the list and therefore prevent them from sorting it.
Sets don't have a total ordering
>>> s=set((1,2,3))
>>> t=set("abc")
>>> s<t
False
>>> t<s
False
>>>
But no exception is raise when you try to sort them
>>> sorted([s,t])
[set([1, 2, 3]), set(['a', 'c', 'b'])]
The python sort algorithms use the __lt__ special method. Keeping in mind that using the cmp and key arguments of the sorting function and methods, it is suggested that your class defines a method:
def __lt__(self, other):
raise NotImplementedError