We Keep Coding

'Force' Python to evaluate all boolean expressions [duplicate]

This question already has answers here:
How to prevent short-circuit evaluation?
(3 answers)
Closed 4 years ago.
Most languages have a way to force 'fail early and often' through forcing all booleans in an if to be evaluate.
Suppose I have the following if statement in Python:
if(age>100 or patient_is_brain_dead or patient_lacks_left_arm):
do something...
The problem is I may have forgotten to set either boolean for 'patient_is_brain_dead or 'has_no_left_arm'.
Since most alive people are under 100, tests 2 and 3 happen rarely since Python, seeing the 'or' coming ahead, stops evaluating if age<100 to save time.
My code is heavily compute-bound, so the overhead of testing all three cases will not degrade performance, but would catch a lot of potential bugs that would happen potentially in one-in-500 cases.
Is there a way to force Python to evaluate them all? Don't suggest reversing the order, because any case could be the rare one(s) depending on user input.

In my opinion, you shouldn't want to do this in production code. If a couple of variables should be Boolean and you need to check if either of them are True, you should use short-circuiting logic.
So what are your options if, as you say, you want to make sure your code doesn't fail on edge cases? You can use unit testing to test all viable scenarios. But this may be overkill for your use case. You can just use an assert statement:
assert all(isinstance(i, bool) for i in [patient_is_brain_dead, patient_lacks_left_arm])
if age > 100 or patient_is_brain_dead or patient_lacks_left_arm:
# do something...
The benefit of such a solution is it's possible to turn off assert statements via command line -O for production code. While, for testing purposes, you can be sure your variables are of the correct type.

No, you will need to explicitly check for None. It is not a "roundabout" way, that is just how the language works.
If you want the conditional to fail if any of the variables are not set, you can use all() to check that they aren't None:
if(all(i is not None for i in [age, patient_is_brain_dead, patient_lacks_left_arm]) and
(age > 100 or patient_is_brain_dead or patient_lacks_left_arm)):
do something...

Python lazy multiple conditions check

I have a lot of conditions to check, but condition evaluation is heavy (e.g. condition requires database access), so I have to check them lazily.
Normally, such check could be written in if clause:
if type in FOOD_PRIZES and Prize.objects.filter(type=type).exists():
pass
If the number of conditions are increasing then if clause becomes ugly.
I can make list of condition lambdas and use all method, but it looks ugly too:
conditions = [
lambda: type in FOOD_PRIZES,
lambda: Prize.objects.filter(type=type).exists()
]
if all(condition() for condition in conditions):
pass
Is there a better way to make code shorter? Is there another ways to make conditions lazy?

Your best bet is to continue what you're doing — but put your fastest conditions to check first.
all() will short-circuit, meaning that as soon as a condition evaluates to False, it will stop processing conditions, saving you the time you would expend by running the other queries.
As for what looks best — that's up to you. But it's far better to have effective, readable code than "attractive" code! And short code isn't always better. Verbosity often makes code more readable to others.
Just be careful. For example, if subsequent conditions are dependent on the first, using all can break. For example, given x='6.5', if isinstance(x, float) and x>5.5 would work but all((isinstance(x, float), x>5.5)) would error.

making python code block with loop faster

Is there a way I can implement the code block below using map or list comprehension or any other faster way, keeping it functionally the same?
def name_check(names, n_val):
lower_names = names.lower()
for item in set(n_val):
if item in lower_names:
return True
return False
Any help here is appreciated

A simple implementation would be
return any(character in names_lower for character in n_val)
A naive guess at the complexity would be O(K*2*N) where K is the number of characters in names and N is the number of characters in n_val. We need one "loop" for the call to lower*, one for the inner comprehension, and one for any. Since any is a built-in function and we're using a generator expression, I would expect it to be faster than your manual loop, but as always, profile to be sure.
To be clear, any short-circuits, so that behaviour is preserved
Notes on Your Implementation
On using a set: Your intuition to use a set to reduce the number of checks is a good one (you could add it to my form above, also), but it's a trade-off. In the case that the first element short circuits, the extra call to set is an additional N steps to produce the set expression. In the case where you wind up checking each item, it will save you some time. It depends on your expected inputs. If n_val was originally an iterable, you've lost that benefit and allocated all the memory up front. If you control the input to the function, why not just recommend it's called using lists that don't have duplicates (i.e., call set() on its input), and leave the function general?
* #Neopolitan pointed out that names_lower = names.lower() should be called out of the loop, as your original implementation called it, else it may (will?) be called repeatedly in the generator expression

Embedded if statements

Suppose I have a function like the following:
bigrams=[(k,v) for (k,v) in dict_bigrams.items()
if k[:pos_qu]==selection[:pos_qu]
and (k[pos_qu+1:]==selection[pos_qu+1:] if pos_qu!=1)
and k[pos_qu] not in alphabet.values()]
I want to make the second condition, namely k[pos_qu+1:]==selection[pos_qu+1:] dependent from another if statement, if pos_qu!=1. I tried (as shown above) by including the two together into parentheses but python flags a syntax error at the parentheses

If I understand your requirement correctly, you only want to check k[pos_qu+1:]==selection[pos_qu+1:] if the condition pos_qu!=1 is also met. You can rephrase that as the following condition:
pos_qu==1 or k[pos_qu+1:]==selection[pos_qu+1:]
Putting this into your comprehension:
bigrams=[(k,v) for (k,v) in dict_bigrams.items()
if k[:pos_qu]==selection[:pos_qu]
and (pos_qu==1 or k[pos_qu+1:]==selection[pos_qu+1:])
and k[pos_qu] not in alphabet.values()]

Whenever you find yourself with a complex list comprehension, trying to figure out how to do something complicated and not knowing how, the answer is usually to break things up. Expression syntax is inherently more limited than full statement (or multi-statement suite) syntax in Python, to prevent you from writing things that you won't be able to read later. Usually, that's a good thing—and, even when it isn't, you're better off going along with it than trying to fight it.
In this case, you've got a trivial comprehension, except for the if clause, which you don't know how to write as an expression. So, I'd turn the condition into a separate function:
def isMyKindOfKey(k):
… condition here
[(k,v) for (k,v) in dict_bigrams.items() if isMyKindOfKey(k)]
This lets you use full multi-statement syntax for the condition. It also lets you give the condition a name (hopefully something better than isMyKindOfKey); makes the parameters, local values captured by the closure, etc. more explicit; lets you test the function separately or reuse it; etc.
In cases where the loop itself is the non-trivial part (or there's just lots of nesting), it usually makes more sense to break up the entire comprehension into an explicit for loop and append, but I don't think that's necessary here.
It's worth noting that in this case—as in general—this doesn't magically solve your problem, it just gives you more flexibility in doing so. For example, you can use the same transformation from postfix if to infix or that F.J suggests, but you can also leave it as an if, e.g., like this:
def isMyKindOfKey(k):
retval = k[:pos_qu]==selection[:pos_qu]
if pos_qu!=1:
retval = retval and (k[pos_qu+1:]==selection[pos_qu+1:])
retval = retval and (k[pos_qu] not in alphabet.values())
return retval
That probably isn't actually the way I'd write this, but you can see how this is a trivial way to transform what's in your head into code, which would be very hard to do in an expression.

just change the order
bigrams=[(k,v) for (k,v) in dict_bigrams.items()
if k[:pos_qu]==selection[:pos_qu] #evaluated first
and pos_qu!=1 #if true continue and evaluate this next
and (k[pos_qu+1:]==selection[pos_qu+1:]) #if pos_qu != 1 lastly eval this
as the comment mentions this is not a very pythonic list comprehension and would be much more readable as a standard for loop..

Is it bad style to reassign long variables as a local abbreviation?

I prefer to use long identifiers to keep my code semantically clear, but in the case of repeated references to the same identifier, I'd like for it to "get out of the way" in the current scope. Take this example in Python:
def define_many_mappings_1(self):
self.define_bidirectional_parameter_mapping("status", "current_status")
self.define_bidirectional_parameter_mapping("id", "unique_id")
self.define_bidirectional_parameter_mapping("location", "coordinates")
#etc...
Let's assume that I really want to stick with this long method name, and that these arguments are always going to be hard-coded.
Implementation 1 feels wrong because most of each line is taken up with a repetition of characters. The lines are also rather long in general, and will exceed 80 characters easily when nested inside of a class definition and/or a try/except block, resulting in ugly line wrapping. Let's try using a for loop:
def define_many_mappings_2(self):
mappings = [("status", "current_status"),
("id", "unique_id"),
("location", "coordinates")]
for mapping in mappings:
self.define_parameter_mapping(*mapping)
I'm going to lump together all similar iterative techniques under the umbrella of Implementation 2, which has the improvement of separating the "unique" arguments from the "repeated" method name. However, I dislike that this has the effect of placing the arguments before the method they're being passed into, which is confusing. I would prefer to retain the "verb followed by direct object" syntax.
I've found myself using the following as a compromise:
def define_many_mappings_3(self):
d = self.define_bidirectional_parameter_mapping
d("status", "current_status")
d("id", "unique_id")
d("location", "coordinates")
In Implementation 3, the long method is aliased by an extremely short "abbreviation" variable. I like this approach because it is immediately recognizable as a set of repeated method calls on first glance while having less redundant characters and much shorter lines. The drawback is the usage of an extremely short and semantically unclear identifier "d".
What is the most readable solution? Is the usage of an "abbreviation variable" acceptable if it is explicitly assigned from an unabbreviated version in the local scope?

itertools to the rescue again! Try using starmap - here's a simple demo:
list(itertools.starmap(min,[(1,2),(2,2),(3,2)]))
prints
[1,2,2]
starmap is a generator, so to actually invoke the methods, you have to consume the generator with a list.
import itertools
def define_many_mappings_4(self):
list(itertools.starmap(
self.define_parameter_mapping,
[
("status", "current_status"),
("id", "unique_id"),
("location", "coordinates"),
] ))
Normally I'm not a fan of using a dummy list construction to invoke a sequence of functions, but this arrangement seems to address most of your concerns.
If define_parameter_mapping returns None, then you can replace list with any, and then all of the function calls will get made, and you won't have to construct that dummy list.

I would go with Implementation 2, but it is a close call.
I think #2 and #3 are equally readable. Imagine if you had 100s of mappings... Either way, I cannot tell what the code at the bottom is doing without scrolling to the top. In #2 you are giving a name to the data; in #3, you are giving a name to the function. It's basically a wash.
Changing the data is also a wash, since either way you just add one line in the same pattern as what is already there.
The difference comes if you want to change what you are doing to the data. For example, say you decide to add a debug message for each mapping you define. With #2, you add a statement to the loop, and it is still easy to read. With #3, you have to create a lambda or something. Nothing wrong with lambdas -- I love Lisp as much as anybody -- but I think I would still find #2 easier to read and modify.
But it is a close call, and your taste might be different.

I think #3 is not bad although I might pick a slightly longer identifier than d, but often this type of thing becomes data driven, so then you would find yourself using a variation of #2 where you are looping over the result of a database query or something from a config file

There's no right answer, so you'll get opinions on all sides here, but I would by far prefer to see #2 in any code I was responsible for maintaining.
#1 is verbose, repetitive, and difficult to change (e.g. say you need to call two methods on each pair or add logging -- then you must change every line). But this is often how code evolves, and it is a fairly familiar and harmless pattern.
#3 suffers the same problem as #1, but is slightly more concise at the cost of requiring what is basically a macro and thus new and slightly unfamiliar terms.
#2 is simple and clear. It lays out your mappings in data form, and then iterates them using basic language constructs. To add new mappings, you only need add a line to the array. You might end up loading your mappings from an external file or URL down the line, and that would be an easy change. To change what is done with them, you only need change the body of your for loop (which itself could be made into a separate function if the need arose).
Your complaint of #2 of "object before verb" doesn't bother me at all. In scanning that function, I would basically first assume the verb does what it's supposed to do and focus on the object, which is now clear and immediately visible and maintainable. Only if there were problems would I look at the verb, and it would be immediately evident what it is doing.