Let me start by saying what I would like to do. I want to create a lazy wrapper for a variable, as in I record all the method calls and operator calls and evaluate them later when I specify the variable to call it on.
As such, I want to be able to intercept all the method calls and operator calls and special methods so that I can work on them. However, __getattr__ doesn't intercept operator calls or __str__ and such, so I want to know if there is a generic way to overload all method calls, or should I just dynamically create a class and duplicate the code for all of it (which I already did, but is ugly).
It can be done, but yes, it becomes "ugly" - I wrote a lazy decorator once, that turns any function into a "lazily computed function".
Basically, I found out that the only moment an object's value is actually used in Python is when one of the special "dunder" methods s called. For example, when you have a number, it's value is only used when you are either using it in another operation, or converting it to a string for IO (which also uses a "dunder" method)
So, my wrapper anotates the parameters to a function call, and returns an special object,
which has potentially all of the "dunder" methods. Just when one of those methods is called, the original function is called - and its return value is then cached for further usage.
The implementation is here:
https://bitbucket.org/jsbueno/metapython/src/510a7d125b24/lazy_decorator.py
Sorry for the text and most of the presentation being in Portuguese.
Related
Consider the following code:
num = 1 + 1j
print(num.imag)
As opposed to
word = "hey"
print(word.islower())
One requires parentheses, and the other doesn't. I know in Python when we call functions without parentheses, we get back only a reference to the function, but it doesn't really answer it. So 'imag' returns a reference? because it seems the method does get executed and returns the imag part.
imag is not a method. It's simply a number-valued attribute.
islower is a method. In order to call the method, you put parentheses after the name.
num.imag is not a function, it's an attribute. To call a function you need the parentheses, or the __call__ method.
Attributes (e.g. imag) are like variables inside the object so you don't use parentheses to access them. Methods (e.g. islower()) are like functions inside the object so they do require parentheses to accept zero or more parameters and perform some work.
Objects can also have 'properties' that are special functions that behave like attributes (i.e. no parentheses) but can perform calculation or additional work when they are referenced or assigned.
I need to create an algorithm which is executed as a sequence of functions chosen at run-time, a bit like the strategy pattern. I want to avoid creating many single method classes (100+), but use simple functions, and assemble them using a decorator.
actions = []
def act(specific_fn):
actions.append(specific_fn)
return specific_fn
#act
def specific_fn1():
print("specific_fn1")
#act
def specific_fn2():
print("specific_fn2")
def execute_strategy():
[f() for f in actions]
I have a couple of questions:
How can I modify the decorator function act to take the list actions as a parameter, so that it adds the decorated function into the list?
How do I use specific_fnX defined in another file? Currently, I just import the file inside the calling function - but that seems odd. Any other options?
Also, any other ideas on implementing this pattern?
This is a pretty good tutorial on using decorators, both with and without arguments. Others may know of others.
The trick is to remember that a decorator is a function call, but when applied to a function definition, it magically replaces the function with the returned result. If you want #my_decorator(my_list) to be a decorator, then my_decorator(my_list) must either return a function or an object with a __call__ method, and that function is then called on called on specific_fn1.
So yes, you need a function that returns a function that returns a function. Looking at some examples in a tutorial will make this clearer.
As to your second question, you can just call my_decorator(my_list)(specific_fn1) without using a decorator, and then ignorning the result.
def __remove_client(self, parameters):
try:
client = self.__client_service.remove_client_by_id(int(parameters[0]))
FunctionsManager.add_undo_operation([self.__client_service, self.__rental_service],
UndoHandler.delete_client_entry, [client[0], client[1]])
FunctionsManager.add_redo_operation(eval('self.__add_new_client(client[0].id,client[0].name)'))
And this gives me : 'UI' object has no attribute '__add_new_client'
What should I do? Or is there another way of adding that function to my repo() stack without calling the function while I am at it?
According to the docs on Private methods:
Notice that code passed to exec() or eval() does not consider the classname of the invoking class to be the current class; this is similar to the effect of the global statement, the effect of which is likewise restricted to code that is byte-compiled together. The same restriction applies to getattr(), setattr() and delattr(), as well as when referencing __dict__ directly.
As for why your eval() is pointless, this:
eval('self.__add_new_client(client[0].id,client[0].name)')
is exacty equivalent to if you just ran the code:
self.__add_new_client(client[0].id,client[0].name)
directly. It seems like maybe you were hoping for some kind of delayed lazy evaluation or something but that's not how it works. Perhaps you wanted to pass a partial evaluation of that method such as:
from functools import partial
FunctionsManager.add_redo_operation(partial(self.__add_new_client, client[0].id, client[0].name))
If this is your own code, you shouldn't actually use the __ methods unless you know exactly what you're doing. There is generally no good reason to use this (Guido has even I think regretted the feature in the past). It's mostly just useful in the special case described in the docs, where you might intend a subclass to override a special method, and you want to keep a "private" copy of that method that cannot be overridden.
Otherwise just use the single _ convention for internal attributes and methods.
After I learned about different data types I learned that once an object from a given type is created it has innate methods that can do 'things'.
Playing around, I noticed that, while some methods return a value, others make change to the original data stored.
Is there any specific term for these two types of methods and is there any intuition or logic as to which methods return a value and which make changes?
For example:
abc= "something"
defg= [12,34,11,45,132,1]
abc.capitalise() #this returns a value
defg.sort() #this changes the orignal list
Is there any specific term for these two types of methods
A method that changes an object's state (ie list.sort()) is usually called a "mutator" (it "mutates" the object). There's no general name for methods that return values - they could be "getters" (methods that take no arguments and return part of the object's state), alternative constructors (methods that are called on the class itself and provide an alternative way to construct an instance of the class), or just methods that take some arguments, do some computations based on both the arguments and the object's state and return a result, or actually just do anything (do some computation AND change the object's state AND return a value).
is there any intuition or logic as to which methods return a value and which make changes?
Some Python objects are immutable (strings, numerics, tuples etc) so when you're working on one of those types you know you won't have any mutator. Except for this special case, nope, you will have to check the doc. The only naming convention here is that methods whose name starts with "set_" and take one argument will change the object's state based on their argument (and most often return nothing) and that methods whose name starts with "get_" and take no arguments will return informations on the object's state and change nothing (you'll often see the formers named "setters" and the laters named "getters"), but like any convention it's only followed by those who follow it, IOW don't assume that because a method name starts with "get_" or "set_" it will indeed behave as expected.
Strings are immutable, so all libraries that do string manipulation will return a new string.
For the other types, you will have to refer to the library documentation.
In Python, len is a function to get the length of a collection by calling an object's __len__ method:
def len(x):
return x.__len__()
So I would expect direct call of __len__() to be at least as fast as len().
import timeit
setup = '''
'''
print (timeit.Timer('a="12345"; x=a.__len__()', setup=setup).repeat(10))
print (timeit.Timer('a="12345"; x=len(a)', setup=setup).repeat(10))
Demo link
But results of testing with the above code shows len() to be faster. Why?
The builtin len() function does not look up the .__len__ attribute. It looks up the tp_as_sequence pointer, which in turn has a sq_length attribute.
The .__len__ attribute on built-in objects is indirectly mapped to the same slot, and it is that indirection (plus the attribute lookup) that takes more time.
For Python-defined classes, the type object looks up the .__len__ method when the sq_length is requested.
From an excellent Python Object-Oriented Programming: Build robust and maintainable object-oriented Python applications and libraries, 4th Edition book by Steven F. Lott, and Dusty Phillips
You may wonder why these objects don't have a length property instead of having to call a function on them. Technically, they do. Most objects that len() will apply to have a method called __len__() that returns the same value. So len(myobj) seems to call myobj.__len__().
Why should we use the len() function instead of the __len__() method? Obviously, __len__() is a special double-underscore method, suggesting that we shouldn't call it directly. There must be an explanation for this. The Python developers don't make such design decisions lightly.
The main reason is efficiency. When we call the __len__() method of an object, the object has to look the method up in its namespace, and, if the special __getattribute__() method (which is called every time an attribute or method on an object is accessed) is defined on that object, it has to be called as well. Furthermore, the __getattribute__() method may have been written to do something clever, for example, refusing to give us access to special methods such as __len__()! The len() function doesn't encounter any of this. It actually calls the __len__() method on the underlying class, so len(myobj) maps to MyObj.__len__(myobj).
__len__ is slower than len(), because __len__
involves a dict lookup.