We Keep Coding

Prevent calling a function more than once if the parameters have been used before

I would like a way to limit the calling of a function to once per values of parameters.
For example
def unique_func(x):
return x
>>> unique_func([1])
[1]
>>> unique_func([1])
*** wont return anything ***
>>> unique_func([2])
[2]
Any suggestions? I've looked into using memoization but not established a solution just yet.
This is not solved by the suggested Prevent a function from being called twice in a row since that only solves when the previous func call had them parameters.

Memoization uses a mapping of arguments to return values. Here, you just want a mapping of arguments to None, which can be handled with a simple set.
def idempotize(f):
cache = set()
def _(x):
if x in cache:
return
cache.add(x)
return f(x)
return _
#idempotize
def unique_fun(x):
...
With some care, this can be generalized to handle functions with multiple arguments, as long as they are hashable.
def idempotize(f):
cache = set()
def _(*args, **kwargs):
k = (args, frozenset(kwargs.items()))
if k in cache:
return
return f(*args, **kwargs)
return _

Consider using the built-in functools.lru_cache() instead of rolling your own.
It won't return nothing on the second function call with the same arugments (it will return the same thing as the first function call) but maybe you can live with that. It would seem like a negligible price to pay, compared to the advantages of using something that's maintained as part of the standard library.
Requires your argument x to be hashable, so won't work with lists. Strings are fine.
from functools import lru_cache
#lru_cache()
def unique_fun(x):
...

I've built a function decorator to handle this scenario, that limits function calls to the same function in a given timeframe.
You can directly use it via PyPI with pip install ofunctions.threading or checkout the github sources.
Example: I want to limit calls to the same function with the same parameters to one call per 10 seconds:
from ofunctions.threading import no_flood
#no_flood(10)
def my_function():
print("It's me, the function")
for _ in range(0, 5):
my_function()
# Will print the text only once.
if after 10 seconds the function is called again, we'll allow a new execution, but will prevent any other execution for the next 10 seconds.
By default #no_flood will limit function calls with the same parameter, so that calling func(1) and func(2) are still allowed concurrently.
The #no_flood decorator can also limit all function calls to a given function regardless of it's parameters:
from ofunctions.threading import no_flood
#no_flood(10, False)
def my_function(var):
print("It's me, function number {}".format(var))
for i in range(0, 5):
my_function(i)
# Will only print function text once

How to force a function to exit after calling another function?

I have two functions each calling the other on their final line as a means of passing control. Is there an elegant way to force one function to exit before the other gets executed, so that you don't stack your program up on corpses?
Example:
def f1(x):
print("f1",x)
f2(x+1)
def f2(x):
print("f2",x)
f1(x+1)
f1(0)
The way it is now, after ~1000 calls it runs into the recursion limit. Of cause, a simple external control structure would get rid of the problem, but I'd rather have the control flow engraved in the functions.
Edit:
But isn't this rather hindering?
If Python strives to be a high-level language with a high level of abstraction, at some point of building modules upon modules, wouldn't Python collapse on its own behavior of not reducing its footprint?

If you want one function to exit before the other is called, you will have to actually exit the function instead of having it call the other one:
def f1(x):
print('f1', x)
return f2, (x+1,)
def f2(x):
print('f2', x)
return f1, (x+1,)
f, args = f1, (0,)
while True:
f, args = f(*args)
This will require that external control structure you wanted to avoid.
As an aside, the way you're trying to make function calls work is a lot like GOTO, and can end up having the same effects on program complexity and maintainability.

The easiest way would be to set a condition in your function in order to terminate/stop the recursion.
It could be something like this:
def f1(x):
if x == 30: #Set a limit in order to just print, without calling
print("f1", x)
else:
print("f1",x)
f2(x+1)
def f2(x):
print("f2",x)
f1(x+1)
f1(0)

Simple solution
You could just add an if statement:
def f1(x):
print("f1", x)
if x <= 30:
f2(x+1)
def f2(x):
print("f2",x)
if x <= 30:
f1(x+1)
Long answer
The answer to if you can force the function to quit before calling the next function is no-ish. Specifically you are asking for a tail-recursion optimization which python does not optimize.

As user2357112 has pointed out, I'm not sure why you would want to do this.
However, if you really have to do this then you can assign a boolean value to the function to indicate that it has been called already.
def f1(x):
print("f1",x)
f1.called = True
if not f2.called:
f2(x+1)
def f2(x):
print("f2",x)
f2.called = True
if not f1.called:
f1(x+1)
f1(0)

Alternative to exec

I'm currently trying to code a Python (3.4.4) GUI with tkinter which should allow to fit an arbitrary function to some datapoints. To start easy, I'd like to create some input-function and evaluate it. Later, I would like to plot and fit it using curve_fit from scipy.
In order to do so, I would like to create a dynamic (fitting) function from a user-input-string. I found and read about exec, but people say that (1) it is not safe to use and (2) there is always a better alternative (e.g. here and in many other places). So, I was wondering what would be the alternative in this case?
Here is some example code with two nested functions which works but it's not dynamic:
def buttonfit_press():
def f(x):
return x+1
return f
print(buttonfit_press()(4))
And here is some code that gives rise to NameError: name 'f' is not defined before I can even start to use xval:
def buttonfit_press2(xval):
actfitfunc = "f(x)=x+1"
execstr = "def {}:\n return {}\n".format(actfitfunc.split("=")[0], actfitfunc.split("=")[1])
exec(execstr)
return f
print(buttonfit_press2(4))
An alternative approach with types.FunctionType discussed here (10303248) wasn't successful either...
So, my question is: Is there a good alternative I could use for this scenario? Or if not, how can I make the code with exec run?
I hope it's understandable and not too vague. Thanks in advance for your ideas and input.
#Gábor Erdős:
Either I don't understand or I disagree. If I code the same segment in the mainloop, it recognizes f and I can execute the code segment from execstr:
actfitfunc = "f(x)=x+1"
execstr = "def {}:\n return {}\n".format(actfitfunc.split("=")[0], actfitfunc.split("=")[1])
exec(execstr)
print(f(4))
>>> 5
#Łukasz Rogalski:
Printing execstr seems fine to me:
def f(x):
return x+1
Indentation error is unlikely due to my editor, but I double-checked - it's fine.
Introducing my_locals, calling it in exec and printing in afterwards shows:
{'f': <function f at 0x000000000348D8C8>}
However, I still get NameError: name 'f' is not defined.
#user3691475:
Your example is very similar to my first example. But this is not "dynamic" in my understanding, i.e. one can not change the output of the function while the code is running.
#Dunes:
I think this is going in the right direction, thanks. However, I don't understand yet how I can evaluate and use this function in the next step? What I mean is: in order to be able to fit it, I have to extract fitting variables (i.e. a in f(x)=a*x+b) or evaluate the function at various x-values (i.e. print(f(3.14))).

The problem with exec/eval, is that they can execute arbitrary code. So to use exec or eval you need to either carefully parse the code fragment to ensure it doesn't contain malicious code (an incredibly hard task), or be sure that the source of the code can be trusted. If you're making a small program for personal use then that's fine. A big program that's responsible for sensitive data or money, definitely not. It would seem your use case counts as having a trusted source.
If all you want is to create an arbitrary function at runtime, then just use a combination of the lambda expression and eval. eg.
func_str = "lambda x: x + 1" # equates to f(x)=x+1
func = eval(func_str)
assert func(4) == 5
The reason why your attempt isn't working is that locals(), in the context of a function, creates a copy of the local namespace. Mutations to the resulting dictionary do not effect the current local namespace. You would need to do something like:
def g():
src = """
def f(x):
return x + 1
"""
exec_namespace = {} # exec will place the function f in this dictionary
exec(src, exec_namespace)
return exec_namespace['f'] # retrieve f

I'm not sure what exactly are you trying to do, i.e. what functions are allowed, what operations are permitted, etc.
Here is an example of a function generator with one dynamic parameter:
>>> def generator(n):
def f(x):
return x+n
return f
>>> plus_one=generator(1)
>>> print(plus_one(4))
5

Memoize a function so that it isn't reset when I rerun the file in Python

I often do interactive work in Python that involves some expensive operations that I don't want to repeat often. I'm generally running whatever Python file I'm working on frequently.
If I write:
import functools32
#functools32.lru_cache()
def square(x):
print "Squaring", x
return x*x
I get this behavior:
>>> square(10)
Squaring 10
100
>>> square(10)
100
>>> runfile(...)
>>> square(10)
Squaring 10
100
That is, rerunning the file clears the cache. This works:
try:
safe_square
except NameError:
#functools32.lru_cache()
def safe_square(x):
print "Squaring", x
return x*x
but when the function is long it feels strange to have its definition inside a try block. I can do this instead:
def _square(x):
print "Squaring", x
return x*x
try:
safe_square_2
except NameError:
safe_square_2 = functools32.lru_cache()(_square)
but it feels pretty contrived (for example, in calling the decorator without an '#' sign)
Is there a simple way to handle this, something like:
#non_resetting_lru_cache()
def square(x):
print "Squaring", x
return x*x
?

Writing a script to be executed repeatedly in the same session is an odd thing to do.
I can see why you'd want to do it, but it's still odd, and I don't think it's unreasonable for the code to expose that oddness by looking a little odd, and having a comment explaining it.
However, you've made things uglier than necessary.
First, you can just do this:
#functools32.lru_cache()
def _square(x):
print "Squaring", x
return x*x
try:
safe_square_2
except NameError:
safe_square_2 = _square
There is no harm in attaching a cache to the new _square definition. It won't waste any time, or more than a few bytes of storage, and, most importantly, it won't affect the cache on the previous _square definition. That's the whole point of closures.
There is a potential problem here with recursive functions. It's already inherent in the way you're working, and the cache doesn't add to it in any way, but you might only notice it because of the cache, so I'll explain it and show how to fix it. Consider this function:
#lru_cache()
def _fact(n):
if n < 2:
return 1
return _fact(n-1) * n
When you re-exec the script, even if you have a reference to the old _fact, it's going to end up calling the new _fact, because it's accessing _fact as a global name. It has nothing to do with the #lru_cache; remove that, and the old function will still end up calling the new _fact.
But if you're using the renaming trick above, you can just call the renamed version:
#lru_cache()
def _fact(n):
if n < 2:
return 1
return fact(n-1) * n
Now the old _fact will call fact, which is still the old _fact. Again, this works identically with or without the cache decorator.
Beyond that initial trick, you can factor that whole pattern out into a simple decorator. I'll explain step by step below, or see this blog post.
Anyway, even with the less-ugly version, it's still a bit ugly and verbose. And if you're doing this dozens of times, my "well, it should look a bit ugly" justification will wear thin pretty fast. So, you'll want to handle this the same way you always factor out ugliness: wrap it in a function.
You can't really pass names around as objects in Python. And you don't want to use a hideous frame hack just to deal with this. So you'll have to pass the names around as strings. ike this:
globals().setdefault('fact', _fact)
The globals function just returns the current scope's global dictionary. Which is a dict, which means it has the setdefault method, which means this will set the global name fact to the value _fact if it didn't already have a value, but do nothing if it did. Which is exactly what you wanted. (You could also use setattr on the current module, but I think this way emphasizes that the script is meant to be (repeatedly) executed in someone else's scope, not used as a module.)
So, here that is wrapped up in a function:
def new_bind(name, value):
globals().setdefault(name, value)
… which you can turn that into a decorator almost trivially:
def new_bind(name):
def wrap(func):
globals().setdefault(name, func)
return func
return wrap
Which you can use like this:
#new_bind('foo')
def _foo():
print(1)
But wait, there's more! The func that new_bind gets is going to have a __name__, right? If you stick to a naming convention, like that the "private" name must be the "public" name with a _ prefixed, we can do this:
def new_bind(func):
assert func.__name__[0] == '_'
globals().setdefault(func.__name__[1:], func)
return func
And you can see where this is going:
#new_bind
#lru_cache()
def _square(x):
print "Squaring", x
return x*x
There is one minor problem: if you use any other decorators that don't wrap the function properly, they will break your naming convention. So… just don't do that. :)
And I think this works exactly the way you want in every edge case. In particular, if you've edited the source and want to force the new definition with a new cache, you just del square before rerunning the file, and it works.
And of course if you want to merge those two decorators into one, it's trivial to do so, and call it non_resetting_lru_cache.
However, I'd keep them separate. I think it's more obvious what they do. And if you ever want to wrap another decorator around #lru_cache, you're probably still going to want #new_bind to be the outermost decorator, right?
What if you want to put new_bind into a module that you can import? Then it's not going to work, because it will be referring to the globals of that module, not the one you're currently writing.
You can fix that by explicitly passing your globals dict, or your module object, or your module name as an argument, like #new_bind(__name__), so it can find your globals instead of its. But that's ugly and repetitive.
You can also fix it with an ugly frame hack. At least in CPython, sys._getframe() can be used to get your caller's frame, and frame objects have a reference to their globals namespace, so:
def new_bind(func):
assert func.__name__[0] == '_'
g = sys._getframe(1).f_globals
g.setdefault(func.__name__[1:], func)
return func
Notice the big box in the docs that tells you this is an "implementation detail" that may only apply to CPython and is "for internal and specialized purposes only". Take this seriously. Whenever someone has a cool idea for the stdlib or builtins that could be implemented in pure Python, but only by using _getframe, it's generally treated almost the same as an idea that can't be implemented in pure Python at all. But if you know what you're doing, and you want to use this, and you only care about present-day versions of CPython, it will work.

There is no persistent_lru_cache in the stdlib. But you can build one pretty easily.
The functools source is linked directly from the docs, because this is one of those modules that's as useful as sample code as it is for using it directly.
As you can see, the cache is just a dict. If you replace that with, say, a shelf, it will become persistent automatically:
def persistent_lru_cache(filename, maxsize=128, typed=False):
"""new docstring explaining what dbpath does"""
# same code as before up to here
def decorating_function(user_function):
cache = shelve.open(filename)
# same code as before from here on.
Of course that only works if your arguments are strings. And it could be a little slow.
So, you might want to instead keep it as an in-memory dict, and just write code that pickles it to a file atexit, and restores it from a file if present at startup:
def decorating_function(user_function):
# ...
try:
with open(filename, 'rb') as f:
cache = pickle.load(f)
except:
cache = {}
def cache_save():
with lock:
with open(filename, 'wb') as f:
pickle.dump(cache, f)
atexit.register(cache_save)
# …
wrapper.cache_save = cache_save
wrapper.cache_filename = filename
Or, if you want it to write every N new values (so you don't lose the whole cache on, say, an _exit or a segfault or someone pulling the cord), add this to the second and third versions of wrapper, right after the misses += 1:
if misses % N == 0:
cache_save()
See here for a working version of everything up to this point (using save_every as the "N" argument, and defaulting to 1, which you probably don't want in real life).
If you want to be really clever, maybe copy the cache and save that in a background thread.
You might want to extend the cache_info to include something like number of cache writes, number of misses since last cache write, number of entries in the cache at startup, …
And there are probably other ways to improve this.
From a quick test, with save_every=1, this makes the cache on both get_pep and fib (from the functools docs) persistent, with no measurable slowdown to get_pep and a very small slowdown to fib the first time (note that fib(100) has 100097 hits vs. 101 misses…), and of course a large speedup to get_pep (but not fib) when you re-run it. So, just what you'd expect.

I can't say I won't just use #abarnert's "ugly frame hack", but here is the version that requires you to pass in the calling module's globals dict. I think it's worth posting given that decorator functions with arguments are tricky and meaningfully different from those without arguments.
def create_if_not_exists_2(my_globals):
def wrap(func):
if "_" != func.__name__[0]:
raise Exception("Function names used in cine must begin with'_'")
my_globals.setdefault(func.__name__[1:], func)
def wrapped(*args):
func(*args)
return wrapped
return wrap
Which you can then use in a different module like this:
from functools32 import lru_cache
from cine import create_if_not_exists_2
#create_if_not_exists_2(globals())
#lru_cache()
def _square(x):
print "Squaring", x
return x*x
assert "_square" in globals()
assert "square" in globals()

I've gained enough familiarity with decorators during this process that I was comfortable taking a swing at solving the problem another way:
from functools32 import lru_cache
try:
my_cine
except NameError:
class my_cine(object):
_reg_funcs = {}
#classmethod
def func_key (cls, f):
try:
name = f.func_name
except AttributeError:
name = f.__name__
return (f.__module__, name)
def __init__(self, f):
k = self.func_key(f)
self._f = self._reg_funcs.setdefault(k, f)
def __call__(self, *args, **kwargs):
return self._f(*args, **kwargs)
if __name__ == "__main__":
#my_cine
#lru_cache()
def fact_my_cine(n):
print "In fact_my_cine for", n
if n < 2:
return 1
return fact_my_cine(n-1) * n
x = fact_my_cine(10)
print "The answer is", x
#abarnert, if you are still watching, I'd be curious to hear your assessment of the downsides of this method. I know of two:
You have to know in advance what attributes to look in for a name to associate with the function. My first stab at it only looked at func_name which failed when passed an lru_cache object.
Resetting a function is painful: del my_cine._reg_funcs[('__main__', 'fact_my_cine')], and the swing I took at adding a __delitem__ was unsuccessful.

UnitTest Python mock only one function multiple call

I'm using Mock (http://www.voidspace.org.uk/python/mock/mock.html), and came across a particular mock case that I cant figure out the solution.
I have a function with multiple calls to some_function that is being Mocked.
def function():
some_function(1)
some_function(2)
some_function(3)
I only wanna mock the first and third call to some_function. The second call I wanna to be made to the real some_function.
I tried some alternatives with http://www.voidspace.org.uk/python/mock/mock.html#mock.Mock.mock_calls, but with no success.
Thanks in advance for the help.

It seems that the wraps argument could be what you want:
wraps: Item for the mock object to wrap. If wraps is not None then calling the
Mock will pass the call through to the wrapped object (returning the
real result and ignoring return_value).
However, since you only want the second call to not be mocked, I would suggest the use of mock.side_effect.
If side_effect is an iterable then each call to the mock will return
the next value from the iterable.
If you want to return a different value for each call, it's a perfect fit :
somefunction_mock.side_effect = [10, None, 10]
Only the first and third calls to somefunction will return 10.
However, if you do need to call the real function, but not the second time, you can also pass side_effect a callable, but I find it pretty ugly (there might be a smarter to do it):
class CustomMock(object):
calls = 0
def some_function(self, arg):
self.calls += 1
if self.calls != 2:
return my_real_function(arg)
else:
return DEFAULT
somefunction_mock.side_effect = CustomMock().some_function

Even simpler than creating a CustomMock class :
def side_effect(*args, **kwargs):
if side_effect.counter < 10:
side_effect.counter += 1
return my_real_function(arg)
else:
return DEFAULT
side_effect.counter = 0
mocked_method.side_effect = side_effect

I faced the same situation today. After some hesitation I found a different way to work around it.
At first, I have a function looks like this:
def reboot_and_balabala(args):
os.system('do some prepare here')
os.system('reboot')
sys.exit(0)
I want the first call to os.system be invoked, otherwise the local file is not generated, and I cannot verify it.
But I really do not want the second call to os.system be invoked, lol.
At first, I have an unittest similar to:
def test_reboot_and_balabala(self):
with patch.object(os, 'system') as mock_system:
# do some mock setup on mock_system, this is what I looked for
# but I do not found any easy and clear solution
with patch.object(sys, 'exit') as mock_exit:
my_lib.reboot_and_balabala(...)
# assert mock invoke args
# check generated files
But finally, I realized, after adjusting the code, I have a more better code structure, and unittests, by following way:
def reboot():
os.system('reboot')
sys.exit(0)
def reboot_and_balabala(args):
os.system('do some prepare here')
reboot()
And then we can test those code by:
def test_reboot(self):
with patch.object(os, 'system') as mock_system:
with patch.object(sys, 'exit') as mock_exit:
my_lib.reboot()
mock_system.assert_called_once_with('reboot')
mock_exit.assert_called_once_with(0)
def test_reboot_and_balabala(self):
with patch.object(my_lib, 'reboot') as mock_reboot:
my_lib.reboot_and_balabala(...)
# check generated files here
mock_reboot.assert_called_once()
This is not a direct answer to the question. But I think this is very inspiring.