Related
I would like to have a function AllTrue that takes three arguments:
List: a list of values
Function: a function to apply to all values
Condition: something to test against the function's output
and return a boolean of whether or not all values in the list match the criteria.
I can get this to work for basic conditions as follows:
def AllTrue(List, Function = "Boolean", Condition = True):
flag = True
condition = Condition
if Function == "Boolean"
for element in List:
if element != condition:
flag = False
break
else:
Map = map(Function, List)
for m in Map:
if m != condition:
flag = False
break
return flag
Since python doesn't have function meant for explicitly returning if something is True, I just make the default "Boolean". One could clean this up by defining TrueQ to return True if an element is True and then just mapping TrueQ on the List.
The else handles queries like:
l = [[0,1], [2,3,4,5], [6,7], [8,9],[10]]
AllTrue(l, len, 2)
#False
testing if all elements in the list are of length 2. However, it can't handle more complex conditions like >/< or compound conditions like len > 2 and element[0] == 15
How can one do this?
Cleaned up version
def TrueQ(item):
return item == True
def AllTrue(List, Function = TrueQ, Condition = True):
flag = True
condition = Condition
Map = map(Function, List)
for m in Map:
if m != condition:
flag = False
break
return flag
and then just call AllTrue(List,TrueQ)
Python already has built-in the machinery you are trying to build. For example to check if all numbers in a list are even the code could be:
if all(x%2==0 for x in L):
...
if you want to check that all values are "truthy" the code is even simpler:
if all(L):
...
Note that in the first version the code is also "short-circuited", in other words the evaluation stops as soon as the result is known. In:
if all(price(x) > 100 for x in stocks):
...
the function price will be called until the first stock is found with a lower or equal price value. At that point the search will stop because the result is known to be False.
To check that all lengths are 2 in the list L the code is simply:
if all(len(x) == 2 for x in L):
...
i.e. more or less a literal translation of the request. No need to write a function for that.
If this kind of test is a "filter" that you want to pass as a parameter to another function then a lambda may turn out useful:
def search_DB(test):
for record in database:
if test(record):
result.append(record)
...
search_DB(lambda rec: all(len(x) == 2 for x in rec.strings))
I want a function that takes a list, a function, and a condition, and tells me if every element in the list matches the condition. i.e. foo(List, Len, >2)
In Python >2 is written lambda x : x>2.
There is (unfortunately) no metaprogramming facility in Python that would allow to write just >2 or things like ยท>2 except using a string literal evaluation with eval and you don't want to do that. Even the standard Python library tried going down that path (see namedtuple implementation in collections) but it's really ugly.
I'm not saying that writing >2 would be a good idea, but that it would be nice to have a way to do that in case it was a good idea. Unfortunately to have decent metaprogramming abilities you need a homoiconic language representing code as data and therefore you would be programming in Lisp or another meta-language, not Python (programming in Lisp would indeed be a good idea, but for reasons unknown to me that approach is still unpopular).
Given that, the function foo to be called like
foo(L, len, lambda x : x > 2)
is just
def foo(L, f=lambda x : x, condition=lambda x: x):
return all(condition(f(x)) for x in L)
but no Python programmer would write such a function, because the original call to foo is actually more code and less clear than inlining it with:
all(len(x) > 2 for x in L)
and requires you to also learn about this thing foo (that does what all and a generator expression would do, just slower, with more code and more obfuscated).
You are reinventing the wheel. Just use something like this:
>>> l = [[0,1], [2,3,4,5], [6,7], [8,9],[10]]
>>> def all_true(iterable, f, condition):
... return all(condition(f(e)) for e in iterable)
...
>>> def cond(x): return x == 2
...
>>> all_true(l, len, cond)
False
You can define a different function to check a different condition:
>>> def cond(x): return x >= 1
...
>>> all_true(l, len, b)
True
>>>
And really, having your own function that does this seems like overkill. For example, to deal with your "complex condition" you could simply do something like:
>>> l = [[0,2],[0,1,2],[0,1,3,4]]
>>> all(len(sub) > 2 and sub[0] == 5 for sub in l)
False
>>> all(len(sub) > 1 and sub[0] == 0 for sub in l)
True
>>>
I think the ideal solution in this case may be:
def AllTrue(List, Test = lambda x:x):
all(Test(x) for x in List)
This thereby allows complex queries like:
l = [[0, 1], [1, 2, 3], [2, 5]]
AllTrue(l, lambda x: len(x) > 2 and x[0] == 1)
To adhere to Juanpa's suggestion, here it is in python naming conventions and an extension of what I posted in the question now with the ability to handle simple conditions like x > value.
from operator import *
all_true(a_list, a_function, an_operator, a_value):
a_map = map(a_function, a_list)
return all( an_operator(m, a_value) for m in a_map)
l = [[0,2],[0,1,2],[0,1,3,4]]
all_true(l, len, gt, 2)
#True
Note: this works for single conditions, but not for complex conditions like
len > 2 and element[0] == 5
I have a method that calls 4 other methods in sequence to check for specific conditions, and returns immediately (not checking the following ones) whenever one returns something Truthy.
def check_all_conditions():
x = check_size()
if x:
return x
x = check_color()
if x:
return x
x = check_tone()
if x:
return x
x = check_flavor()
if x:
return x
return None
This seems like a lot of baggage code. Instead of each 2-line if statement, I'd rather do something like:
x and return x
But that is invalid Python. Am I missing a simple, elegant solution here? Incidentally, in this situation, those four check methods may be expensive, so I do not want to call them multiple times.
Alternatively to Martijn's fine answer, you could chain or. This will return the first truthy value, or None if there's no truthy value:
def check_all_conditions():
return check_size() or check_color() or check_tone() or check_flavor() or None
Demo:
>>> x = [] or 0 or {} or -1 or None
>>> x
-1
>>> x = [] or 0 or {} or '' or None
>>> x is None
True
You could use a loop:
conditions = (check_size, check_color, check_tone, check_flavor)
for condition in conditions:
result = condition()
if result:
return result
This has the added advantage that you can now make the number of conditions variable.
You could use map() + filter() (the Python 3 versions, use the future_builtins versions in Python 2) to get the first such matching value:
try:
# Python 2
from future_builtins import map, filter
except ImportError:
# Python 3
pass
conditions = (check_size, check_color, check_tone, check_flavor)
return next(filter(None, map(lambda f: f(), conditions)), None)
but if this is more readable is debatable.
Another option is to use a generator expression:
conditions = (check_size, check_color, check_tone, check_flavor)
checks = (condition() for condition in conditions)
return next((check for check in checks if check), None)
Don't change it
There are other ways of doing this as the various other answers show. None are as clear as your original code.
In effectively the same answer as timgeb, but you could use parenthesis for nicer formatting:
def check_all_the_things():
return (
one()
or two()
or five()
or three()
or None
)
According to Curly's law, you can make this code more readable by splitting two concerns:
What things do I check?
Has one thing returned true?
into two functions:
def all_conditions():
yield check_size()
yield check_color()
yield check_tone()
yield check_flavor()
def check_all_conditions():
for condition in all_conditions():
if condition:
return condition
return None
This avoids:
complicated logical structures
really long lines
repetition
...while preserving a linear, easy to read flow.
You can probably also come up with even better function names, according to your particular circumstance, which make it even more readable.
This is a variant of Martijns first example. It also uses the "collection of callables"-style in order to allow short-circuiting.
Instead of a loop you can use the builtin any.
conditions = (check_size, check_color, check_tone, check_flavor)
return any(condition() for condition in conditions)
Note that any returns a boolean, so if you need the exact return value of the check, this solution will not work. any will not distinguish between 14, 'red', 'sharp', 'spicy' as return values, they will all be returned as True.
Have you considered just writing if x: return x all on one line?
def check_all_conditions():
x = check_size()
if x: return x
x = check_color()
if x: return x
x = check_tone()
if x: return x
x = check_flavor()
if x: return x
return None
This isn't any less repetitive than what you had, but IMNSHO it reads quite a bit smoother.
I'm quite surprised nobody mentioned the built-in any which is made for this purpose:
def check_all_conditions():
return any([
check_size(),
check_color(),
check_tone(),
check_flavor()
])
Note that although this implementation is probably the clearest, it evaluates all the checks even if the first one is True.
If you really need to stop at the first failed check, consider using reduce which is made to convert a list to a simple value:
def check_all_conditions():
checks = [check_size, check_color, check_tone, check_flavor]
return reduce(lambda a, f: a or f(), checks, False)
reduce(function, iterable[, initializer]) : Apply function of two
arguments cumulatively to the items of iterable, from left to right,
so as to reduce the iterable to a single value. The left argument, x,
is the accumulated value and the right argument, y, is the update
value from the iterable. If the optional initializer is present, it is
placed before the items of the iterable in the calculation
In your case:
lambda a, f: a or f() is the function that checks that either the accumulator a or the current check f() is True. Note that if a is True, f() won't be evaluated.
checks contains check functions (the f item from the lambda)
False is the initial value, otherwise no check would happen and the result would always be True
any and reduce are basic tools for functional programming. I strongly encourage you to train these out as well as map which is awesome too!
If you want the same code structure, you could use ternary statements!
def check_all_conditions():
x = check_size()
x = x if x else check_color()
x = x if x else check_tone()
x = x if x else check_flavor()
return x if x else None
I think this looks nice and clear if you look at it.
Demo:
For me, the best answer is that from #phil-frost, followed by #wayne-werner's.
What I find interesting is that no one has said anything about the fact that a function will be returning many different data types, which will make then mandatory to do checks on the type of x itself to do any further work.
So I would mix #PhilFrost's response with the idea of keeping a single type:
def all_conditions(x):
yield check_size(x)
yield check_color(x)
yield check_tone(x)
yield check_flavor(x)
def assessed_x(x,func=all_conditions):
for condition in func(x):
if condition:
return x
return None
Notice that x is passed as an argument, but also all_conditions is used as a passed generator of checking functions where all of them get an x to be checked, and return True or False. By using func with all_conditions as default value, you can use assessed_x(x), or you can pass a further personalised generator via func.
That way, you get x as soon as one check passes, but it will always be the same type.
Ideally, I would re-write the check_ functions to return True or False rather than a value. Your checks then become
if check_size(x):
return x
#etc
Assuming your x is not immutable, your function can still modify it (although they can't reassign it) - but a function called check shouldn't really be modifying it anyway.
I like #timgeb's. In the meantime I would like to add that expressing None in the return statement is not needed as the collection of or separated statements are evaluated and the first none-zero, none-empty, none-None is returned and if there isn't any then None is returned whether there is a None or not!
So my check_all_conditions() function looks like this:
def check_all_conditions():
return check_size() or check_color() or check_tone() or check_flavor()
Using timeit with number=10**7 I looked at the running time of a number of the suggestions. For the sake of comparison I just used the random.random() function to return a string or None based on random numbers. Here is the whole code:
import random
import timeit
def check_size():
if random.random() < 0.25: return "BIG"
def check_color():
if random.random() < 0.25: return "RED"
def check_tone():
if random.random() < 0.25: return "SOFT"
def check_flavor():
if random.random() < 0.25: return "SWEET"
def check_all_conditions_Bernard():
x = check_size()
if x:
return x
x = check_color()
if x:
return x
x = check_tone()
if x:
return x
x = check_flavor()
if x:
return x
return None
def check_all_Martijn_Pieters():
conditions = (check_size, check_color, check_tone, check_flavor)
for condition in conditions:
result = condition()
if result:
return result
def check_all_conditions_timgeb():
return check_size() or check_color() or check_tone() or check_flavor() or None
def check_all_conditions_Reza():
return check_size() or check_color() or check_tone() or check_flavor()
def check_all_conditions_Phinet():
x = check_size()
x = x if x else check_color()
x = x if x else check_tone()
x = x if x else check_flavor()
return x if x else None
def all_conditions():
yield check_size()
yield check_color()
yield check_tone()
yield check_flavor()
def check_all_conditions_Phil_Frost():
for condition in all_conditions():
if condition:
return condition
def main():
num = 10000000
random.seed(20)
print("Bernard:", timeit.timeit('check_all_conditions_Bernard()', 'from __main__ import check_all_conditions_Bernard', number=num))
random.seed(20)
print("Martijn Pieters:", timeit.timeit('check_all_Martijn_Pieters()', 'from __main__ import check_all_Martijn_Pieters', number=num))
random.seed(20)
print("timgeb:", timeit.timeit('check_all_conditions_timgeb()', 'from __main__ import check_all_conditions_timgeb', number=num))
random.seed(20)
print("Reza:", timeit.timeit('check_all_conditions_Reza()', 'from __main__ import check_all_conditions_Reza', number=num))
random.seed(20)
print("Phinet:", timeit.timeit('check_all_conditions_Phinet()', 'from __main__ import check_all_conditions_Phinet', number=num))
random.seed(20)
print("Phil Frost:", timeit.timeit('check_all_conditions_Phil_Frost()', 'from __main__ import check_all_conditions_Phil_Frost', number=num))
if __name__ == '__main__':
main()
And here are the results:
Bernard: 7.398444877040768
Martijn Pieters: 8.506569201346597
timgeb: 7.244275416364456
Reza: 6.982133448743038
Phinet: 7.925932800076634
Phil Frost: 11.924794811353031
A slight variation on Martijns first example above, that avoids the if inside the loop:
Status = None
for c in [check_size, check_color, check_tone, check_flavor]:
Status = Status or c();
return Status
This way is a little bit outside of the box, but I think the end result is simple, readable, and looks nice.
The basic idea is to raise an exception when one of the functions evaluates as truthy, and return the result. Here's how it might look:
def check_conditions():
try:
assertFalsey(
check_size,
check_color,
check_tone,
check_flavor)
except TruthyException as e:
return e.trigger
else:
return None
You'll need a assertFalsey function that raises an exception when one of the called function arguments evaluates as truthy:
def assertFalsey(*funcs):
for f in funcs:
o = f()
if o:
raise TruthyException(o)
The above could be modified so as to also provide arguments for the functions to be evaluated.
And of course you'll need the TruthyException itself. This exception provides the object that triggered the exception:
class TruthyException(Exception):
def __init__(self, obj, *args):
super().__init__(*args)
self.trigger = obj
You can turn the original function into something more general, of course:
def get_truthy_condition(*conditions):
try:
assertFalsey(*conditions)
except TruthyException as e:
return e.trigger
else:
return None
result = get_truthy_condition(check_size, check_color, check_tone, check_flavor)
This might be a bit slower because you are using both an if statement and handling an exception. However, the exception is only handled a maximum of one time, so the hit to performance should be minor unless you expect to run the check and get a True value many many thousands of times.
The pythonic way is either using reduce (as someone already mentioned) or itertools (as shown below), but it seems to me that simply using short circuiting of the or operator produces clearer code
from itertools import imap, dropwhile
def check_all_conditions():
conditions = (check_size,\
check_color,\
check_tone,\
check_flavor)
results_gen = dropwhile(lambda x:not x, imap(lambda check:check(), conditions))
try:
return results_gen.next()
except StopIteration:
return None
If you can require Python 3.8, you can use the new feature of "assignment expressions" to make the if-else chain somewhat less repetitive:
def check_all_conditions():
if (x := check_size()): return x
if (x := check_color()): return x
if (x := check_tone()): return x
if (x := check_flavor()): return x
return None
Or use max:
def check_all_conditions():
return max(check_size(), check_color(), check_tone(), check_flavor()) or None
I have seen some interesting implementations of switch/case statements with dicts in the past that led me to this answer. Using the example you've provided you would get the following. (It's madness using_complete_sentences_for_function_names, so check_all_conditions is renamed to status. See (1))
def status(k = 'a', s = {'a':'b','b':'c','c':'d','d':None}) :
select = lambda next, test : test if test else next
d = {'a': lambda : select(s['a'], check_size() ),
'b': lambda : select(s['b'], check_color() ),
'c': lambda : select(s['c'], check_tone() ),
'd': lambda : select(s['d'], check_flavor())}
while k in d : k = d[k]()
return k
The select function eliminates the need to call each check_FUNCTION twice i.e. you avoid check_FUNCTION() if check_FUNCTION() else next by adding another function layer. This is useful for long running functions. The lambdas in the dict delay execution of it's values until the while loop.
As a bonus you may modify the execution order and even skip some of the tests by altering k and s e.g. k='c',s={'c':'b','b':None} reduces the number of tests and reverses the original processing order.
The timeit fellows might haggle over the cost of adding an extra layer or two to the stack and the cost for the dict look up but you seem more concerned with the prettiness of the code.
Alternatively a simpler implementation might be the following :
def status(k=check_size) :
select = lambda next, test : test if test else next
d = {check_size : lambda : select(check_color, check_size() ),
check_color : lambda : select(check_tone, check_color() ),
check_tone : lambda : select(check_flavor, check_tone() ),
check_flavor: lambda : select(None, check_flavor())}
while k in d : k = d[k]()
return k
I mean this not in terms of pep8 but in terms of using one concise descriptive word in place of a sentence. Granted the OP may be following some coding convention, working one some existing code base or not care for terse terms in their codebase.
my code consists of me recreating the function 'filter()' and using it with a function to filter words longer than 5 characters. It worked with the actual function filter when I tried it btw...I'm using python 3+
def filter1(fn, a):
i = 0
while i != len(a):
u = i - 1
a[i] = fn(a[i], a[u])
i += 1
return a
def filter_long_words(l):
if len[l] > 5:
return [l]
listered = ['blue', 'hdfdhsf', 'dsfjbdsf', 'jole']
print(list(filter1(filter_long_words, listered)))
getting error
TypeError: filter_long_words() takes 1 positional argument but 2 were given
You are passing two parameters to fn (which refers to filter_long_words) here:
a[i] = fn(a[i], a[u])
But filter_long_words only accepts one parameter.
Notes:
You can loop through lists using for item in my_list, or if you want index as well for index, item in enumerate(my_list).
I think you might get an IndexError since u will be -1 in the first round of your loop.
The filter function can also be expressed as a list comprehension: (item for item in listered if filter_long_words(item))
My version of filter would look like this, if I have to use a for loop:
def my_filter(fn, sequence):
if fn is None:
fn = lambda x: x
for item in sequence:
if fn(item):
yield item
Since you have stated that you are using Python 3, this returns a generator instead of a list. If you want it to return a list:
def my_filter(fn, sequence):
if fn is None:
fn = lambda x: x
acc = []
for item in sequence:
if fn(item):
acc.append(item)
return acc
If you don't need to use a for loop:
def my_filter(fn, sequence):
if fn is None:
fn = lambda x: x
return (item for item in sequence if fn(item))
Your're calling fn with 2 parameters in filter1(fn, a), and since you've passed filter_long_words() to filter1 as fn, that triggers the error.
But there's more weird stuff:
I don't understand the magick of filter1 or what you were trying to
accomplish, but it seems to me that you don't have a clear idea what to do.
But if you want to mimic (somehow) how filter works, you have to return a
list which contains only items for which the fn function returns true. When
you know this, you can rewrite it - here are a few suggestions for rewrite
# explicit, inefficient and long, but straightforward version:
def filter1(fn, a):
new_list = []
for item in a:
if fn(item):
new_list.append(item):
return new_list
# shorter version using list comprehensions:
def filter1(fn, a):
return [item for item in a if fn(item)]
The filter_long_words function is wrong too - it should return True or
False. The only reason why it could work is because any non-empty list is
treated as True by python and default return value of a function is None,
which translates to False. But it's confusing and syntactically wrong to use
len[l] - the proper usage is len(l).
There are a few suggestions for rewrite, which all returns explicit boolean
values:
# unnecessary long, but self-explanatory:
def filter_long_words(l):
if len(l) > 5:
return True
else
return False
# short variant
def filter_long_words(l):
return len(l) > 5
You are calling "filter_long_words" with 2 parameter => fn(a[i], a[u]) also there is an error
def filter_long_words(l):
if **len[l]** > 5:
return [l]
len is builtin method it should be len(l)
To my understanding, adding elements to a set does not return anything.
eg:
>>> s=set([1,2,3])
>>> s.add(1)
>>> b = s.add(1)
>>> b
>>> b = s.add(5)
>>> b
>>> print b
None
>>> b = s.add(5)
>>> print b
None
but I am confused how this function to remove duplicates of a list while preserving order works:
def f7(seq):
seen = set()
seen_add = seen.add
return [ x for x in seq if x not in seen and not seen_add(x)]
not seen_add(x) return true always irrespective of if I add duplicates or not.
The only real check is if x not in seen, since if that fails, it will not perform not see_add(x).
Is my understanding here or am I missing something?
however, here, not seen_add(1) return true if I add duplicate or a unique element.
This is puzzling, so is the not here just do an empty check? but seen_add() is not returning a set to do an empty check. and why seen_add(2) does not return anything but not seen_add(1) returns the boolean True
>>> seen=set()
>>> seen_add=seen.add
>>> seen_add(1)
>>> seen
set([1])
>>> seen.add(2)
>>> seen
set([1, 2])
>>> not seen_add(1)
True
>>> not seen_add(4)
True
As you have discovered, seen_add(x) always returns None.
Now, not None is True:
>>> not None
True
This is explained in the documentation:
5.1. Truth Value Testing
Any object can be tested for truth value, for use in an if or while condition or as operand of the Boolean operations below. The following values are considered false:
None
False
...
Since not seen_add(x) is always True, it has no effect on the result of the if. It is only used for the side effect of adding x to seen.
Put another way, the following:
seen_add = seen.add
return [ x for x in seq if x not in seen and not seen_add(x)]
is a shorter (and likely more performant) way to write:
result = []
for x in seq:
if x not in seen:
result.append(x)
seen.add(x)
return result
As said by the others, not seen_add(x) always evaluates to True, so it is used only as a hack to add the addition statement into the list comprehension. Since the logic value of x and True is equal to x, this does not change the behavior of the if.
Another thing that this does, due to the short-circuit behavior of and, is that the right-hand argument to and is only executed if the left-hand argument is True. So the call to seen.add() is only done when the element is not yet in the set.
The equivalent python code would be:
def f7(seq):
seen = set()
result = []
for x in seq:
if x not in seen:
result.append(x)
seen.add(x)
return result
The solution with the list comprehension might be faster, but it is too cryptic to my taste.
This is because not None is True.
maybe this can help you
def add_ok(set, value):
return len(set) != (set.add(value), len(set))[1]
print(add_ok(a, 1))
print(add_ok(a, 1))
If not, what is the best way to do this?
Right now I'm doing (for a django project):
if not 'thing_for_purpose' in request.session:
request.session['thing_for_purpose'] = 5
but its pretty awkward. In Ruby it would be:
request.session['thing_for_purpose'] ||= 5
which is much nicer.
Jon-Eric's answer's is good for dicts, but the title seeks a general equivalent to ruby's ||= operator.
A common way to do something like ||= in Python is
x = x or new_value
Precise answer: No. Python does not have a single built-in operator op that can translate x = x or y into x op y.
But, it almost does. The bitwise or-equals operator (|=) will function as described above if both operands are being treated as booleans, with a caveat. (What's the caveat? Answer is below of course.)
First, the basic demonstration of functionality:
x = True
x
Out[141]: True
x |= True
x
Out[142]: True
x |= False
x
Out[143]: True
x &= False
x
Out[144]: False
x &= True
x
Out[145]: False
x |= False
x
Out[146]: False
x |= True
x
Out[147]: True
The caveat is due python not being strictly-typed, and thus even if the values are being treated as booleans in an expression they will not be short-circuited if given to a bitwise operator. For example, suppose we had a boolean function which clears a list and returns True iff there were elements deleted:
def my_clear_list(lst):
if not lst:
return False
else:
del lst[:]
return True
Now we can see the short-circuited behavior as so:
x = True
lst = [1, 2, 3]
x = x or my_clear_list(lst)
print(x, lst)
Output: True [1, 2, 3]
However, switching the or to a bitwise or (|) removes the short-circuit, so the function my_clear_list executes.
x = True
lst = [1, 2, 3]
x = x | my_clear_list(lst)
print(x, lst)
Output: True []
Above, x = x | my_clear_list(lst) is equivalent to x |= my_clear_list(lst).
dict has setdefault().
So if request.session is a dict:
request.session.setdefault('thing_for_purpose', 5)
Setting a default makes sense if you're doing it in a middleware or something, but if you need a default value in the context of one request:
request.session.get('thing_for_purpose', 5) # gets a default
bonus: here's how to really do an ||= in Python.
def test_function(self, d=None):
'a simple test function'
d = d or {}
# ... do things with d and return ...
In general, you can use dict[key] = dict.get(key, 0) + val.