I am coding a kind of command line user interface where an arbitrary boolean expression can be given as input. This expression must be repeatedly evaluated on a dictionary that is changing after some update.
Here is the simplified code :
import traceback
def update_my_dict():
return {'a': 1}
my_dict = {'a': 0}
bool_exp = input()
# -- verify code can be executed in the current context
try:
result = eval(bool_exp, my_dict)
except Exception:
print(f'Expression cannot be evaluated, evaluation raise the following error:')
print(traceback.format_exc())
quit()
# -- verify code return a boolean
if not isinstance(result, bool):
print(f'Expression return {type(result)}, expect bool')
quit()
# -- go
while not eval(bool_exp, my_dict):
my_dict = update_my_dict()
Before running the last while loop I want to verify that the expression can be executed in the current context and ensuring that its return a boolean.
My problem is if the expression is, for example bool_exp = a == 1 and b == 2 the first test evaluation of the expression while return false but do not raise exception because off lazy evaluation. But when my_dict is updated then an error will be raised.
So is that possible to, some how, disable the lazy/short-circuit evaluation for the first test evaluation ? I searched some solution using ast, but it seems complicated since bool_exp can be arbitrary long and complex, like containing entangled boolean expressions and so on.
PS: I know eval() is unsafe in a general context but my code will not be usable externaly
PS2: I know its possible to catch the exception in the while loop but it looks a bit sub-optimized knowing my_dict keys will never change when updated, only their values. Also this question is more like if its possible to control the evaluation behavior
EDIT
"but you could use & and | instead of and and or !"
No. I cannot tell what will be entered as input. So the user can input whatever s.he want.
A Correct input expression:
Should return a boolean.
Should only involve dictionary keys in the tests.
"Correct" meaning it will be repeatedly evaluated in the ending while loop without raised any exception.
We assume that the keys in the dictionary will stay constant. i.e only the values of the dict will change during the update phase. In other words, the first try/except aim to verify that the expression is only doing some test on variables that are in my_dict.
I cannot explain further the global use of this or I'll need a very long post with lost of what seems irrelevant information to resolve the issue.
You could put the "verification" code inside the loop. This makes sense as your input is changing so you should verify it on each change. You already evaluate the expression every time the dict values change, so the only added logic compared to your current code is that the isinstance check is now done as well with every change of the dict:
import traceback
def update_my_dict():
return {'a': 1}
my_dict = {'a': 0}
bool_exp = input()
# -- go
while True:
# -- verify code can be executed in the current context
try:
result = eval(bool_exp, my_dict)
except Exception:
print(f'Expression cannot be evaluated, evaluation raise the following error:')
print(traceback.format_exc())
quit()
# -- verify code return a boolean
if not isinstance(result, bool):
print(f'Expression return {type(result)}, expect bool')
quit()
if result:
break
my_dict = update_my_dict()
On the same example input a == 1 and b == 2 this will output:
Expression cannot be evaluated, evaluation raise the following error:
Traceback (most recent call last):
File "main.py", line 14, in <module>
result = eval(bool_exp, my_dict)
File "<string>", line 1, in <module>
NameError: name 'b' is not defined
Related
# Python 3.8
import sys
print(sys.tracebacklimit)
Running the code results with:
AttributeError: module 'sys' has no attribute 'tracebacklimit'
But the code below works:
sys.tracebacklimit = 3
print(sys.tracebacklimit)
3
Question:
How to get the current tracebacklimit value from the code?
The documentation (search for tracebacklimit) states (with my added emphasis on the first word):
When this variable is set to an integer value, it determines the maximum number of levels of traceback information printed when an unhandled exception occurs. The default is 1000. When set to 0 or less, all traceback information is suppressed and only the exception type and value are printed.
In other words, if it's not set (or presumably also set to a non-integer), it will use 1000 as a default. So you can get the current value with something like:
try:
tbl = int(sys.tracebacklimit)
except:
tbl = 1000
You could also try the following, it is shorter but it doesn't specifically cover the case where it's set to a non-integer, so it may give you the wrong thing:
tbl = getattr(sys, "tracebacklimit", 1000)
As an aside, you could create a helper function which, given a string representation of the item in question, evaluated it for you:
def ensure(str_item, typ, deflt):
# Execute in "sandbox" context, catching problems.
local_dict = {}
try:
exec(f"myvar = {str_item}", globals(), local_dict)
except:
return deflt
# Extract, and check type of, value.
item = local_dict["myvar"]
if typ is not None and type(item) != typ:
return deflt
# Was valid, return it.
return item
# Test code.
import sys
#sys.tracebacklimit = 7 # Uncomment to test if set.
#sys.tracebacklimit = "hello" # Uncomment to test if set to non-integer.
print(ensure("sys.tracebacklimit", int, 1000))
This makes sure the item exists and that it is of the expected type (unless None is given for the type).
It's probably not needed for a simple case like this but I always like to add useful helper functions to my ever growing snippet library :-)
This is a little difficult to explain, so let's hope I'm expressing the problem coherently:
Say I have this list:
my_list = ["a string", 45, 0.5]
The critical point to understand in order to see where the question comes from is that my_list is generated by another function; I don't know ahead of time anything about my_list, specifically its length and the datatype of any of its members.
Next, say that every time <my_list> is generated, there is a number of predetermined operations I want to perform on it. For example, I want to:
my_text = my_list[1]+"hello"
some_var = my_list[10]
mini_list = my_list[0].split('s')[1]
my_sum = my_list[7]+2
etc. The important point here is that it's a large number of operations.
Obviously, some of these operations would succeed with any given my_list and some would fail and, importantly, those which fail will do so with an unpredictable Error type; but I need to run all of them on every generation of my_list.
One obvious solution would be to use try/except on each of these operations:
try:
my_text = my_list[1]+"hello"
except:
my_text = "None"
try:
some_var = my_list[10]
except:
some_var = "couldn't do it"
etc.
But with a large number of operations, this gets very cumbersome. I looked into the various questions about multiple try/excepts, but unless I'm missing something, they don't address this.
Based on someone's suggestion (sorry, lost the link), I tried to create a function with a built-in try/except, create another list of these operations, and send each operation to the function. Something along the lines of
def careful(op):
try:
return op
else:
return "None"
And use it with, for example, the first operation:
my_text = careful(my_list[1]+"hello")
The problem is python seems to evaluate the careful() argument before it's sent out to the function and the error is generated before it can be caught...
So I guess I'm looking for a form of a ternary operator that can do something like:
my text = my_list[1]+"hello" if (this doesn't cause any type of error) else "None"
But, if one exist, I couldn't find it...
Any ideas would be welcome and sorry for the long post.
Maybe something like this?
def careful(op, default):
ret = default
try:
ret = computation()
else:
pass
return ret
If you must do this, consider keeping a collection of the operations as strings and calling exec on them in a loop
actions = [
'my_text = my_list[1]+"hello"',
'some_var = my_list[10]',
'mini_list = my_list[0].split("s")[1]',
'my_sum = my_list[7]+2',
]
If you make this collection a dict, you may also assign a default
Note that if an action default (or part of an action string) is meant to be a string, it must be quoted twice. Consider using block-quotes for this if you already have complex escaping, like returning a raw strings or a string representing a regular expression
{
"foo = bar": r"""r'[\w]+baz.*'"""
}
complete example:
>>> actions_defaults = {
... 'my_text = my_list[1]+"hello"': '"None"',
... 'some_var = my_list[10]': '"couldn\'t do it"',
... 'mini_list = my_list[0].split("s")[1]': '"None"',
... 'my_sum = my_list[7]+2': '"None"',
... }
>>>
>>> for action, default in actions_defaults.items():
... try:
... exec(action)
... except Exception: # consider logging error
... exec("{} = {}".format(action.split("=")[0], default))
...
>>> my_text
'None'
>>> some_var
"couldn't do it"
Other notes
this is pretty evil
declaring your vars before running to be their default values is probably better/clearer (sufficient to pass in the except block, as the assignment will fail)
you may run into weird scoping and need to access some vars via locals()
This sounds like an XY Problem
If you can make changes to the source logic, returning a dict may be a much better solution. Then you can determine if a key exists before doing some action, and potentially also look up the action which should be taken if the key exists in another dict.
This question already has answers here:
Callable as the default argument to dict.get without it being called if the key exists
(6 answers)
Closed 6 years ago.
Seems like the fallback is called even if the key is present inside the dictionary. Is this an intended behaviour? How can workaround it?
>>> i = [1,2,3,4]
>>> c = {}
>>> c[0]= 0
>>> c.get(0, i.pop())
0
>>> c.get(0, i.pop())
0
>>> c.get(0, i.pop())
0
>>> c.get(0, i.pop())
0
>>> c.get(0, i.pop())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IndexError: pop from empty list
When doing c.get(0, i.pop()), the i.pop() part gets evaluated before the result it returns is passed to the c.get(...). That's the reason that the error appears if a list i is empty due to the previous .pop() calls on it.
To get around this, you should either check if the list is not empty before trying to pop an element from it, or just try it an catch a possible exception:
if not i:
# do not call do i.pop(), handle the case in some way
default_val = i.pop()
or
try:
c.get(0, i.pop())
except IndexError:
# gracefully handle the case in some way, e.g. by exiting
default_val = i.pop()
The first approach is called LBYL ("look before you leap"), while the second is referred to as EAFP ("easier to ask for forgiveness than permission"). The latter is usually preferred in Python and considered more Pythonic, because the code does not get cluttered with a lot of safeguarding checks, although the LBYL approach has its merits, too, and can be just as readable (use-case dependent).
This is the expected results because you're directly invoking i.pop() which gets called before c.get().
The default argument to dict.get does indeed get evaluated before the dictionary checks if the key is present or not. In fact, it's evaluated before the get method is even called! Your get calls are equivalent to this:
default = i.pop() # this happens unconditionally
c.get(0, default)
default = i.pop() # this one too
c.get(0, default)
#...
If you want to specify a callable that will be used only to fill in missing dictionary values, you might want to use a collections.defaultdict. It takes a callable which is used exactly that way:
c = defaultdict(i.pop) # note, no () after pop
c[0] = 0
c[0] # use regular indexing syntax, won't pop anything
Note that unlike a get call, the value returned by the callable will actually be stored in the dictionary afterwards, which might be undesirable.
There is no real way to workaround this except using if...else... !
In your case, this code would work:
c[0] if 0 in c else i.pop()
This is intended behavior because i.pop() is an expression that is evaluated before c.get(...) is. Imagine what would happen if that weren't the case. You might have something like this:
def myfunction(number):
print("Starting work")
# Do long, complicated setup
# Do long, complicated thing with number
myfunction(int('kkk'))
When would you have int('kkk') to be evaluated? Would it be as soon as myfunction() uses it (after the parameters)? It would then finally have the ValueError after the long, complicated setup. If you were to say x = int('kkk'), when would you expect the ValueError? The right side is evaluated first, and the ValueError occurs immediately. x does not get defined.
There are a couple possible workarounds:
c.get(0) or i.pop()
That will probably work in most cases, but won't work if c.get(0) might return a Falsey value that is not None. A safer way is a little longer:
try:
result = c[0]
except IndexError:
result = i.pop()
Of course, we like EAFP (Easier to Ask Forgiveness than Permission), but you could ask permission:
c[0] if 0 in c else i.pop()
(Credits to #soon)
Both the arguments are evaluated before calling the get function. Thus in all calls the size of list is decreased by 1 even if the key is present.
Try something like
if c.has_key(0):
print c[0]
else:
print i.pop()
I have the following dictionary:
dic = {"a": "first", "b": "second"}
and it's ok, when I do the following:
print dic.get("a")
print dic.get("a", "asd")
print dic.get("a", dic.get("c"))
but when I use this method like this:
print dic.get("a", dic.get("c").split(" ",1)[0])
I receive the following error:
Traceback (most recent call last):
File "<console>", line 1, in <module>
AttributeError: 'NoneType' object has no attribute 'split'
I dont't understand the last case. The second argument calculated (dic.get("c") should be None - it's ok), but there is a key "a" in dictionary and first argument shouldn't fire calculating of the second argument.
How I can fix this? And why it happened?
TIA!
The second argument is always evaluated whether or not it is used.
To fix it you can catch the exception that is raised when an item is not found instead of using get.
try:
result = dic['a']
except KeyError:
result = dic['c'].split(' ', 1)[0]
print result
Maybe you just have typo and meant
print dic.get("a", dic.get("c")).split(" ",1)[0]
That is you meant to split the result of the outer dic.get not the inner one.
As others have explained, Python (like most other languages outside the functional family) evaluates all arguments of a function before calling it. Thus, dic.get("c") is None when key "c" doesn't exist in the dictionary and None has no .split() method, and this evaluation happens regardless of whether (and in fact before) the get succeeds or fails.
Instead, use a short-circuiting Boolean operator or a conditional expression.
# if dic.get["a"] is always truthy when it exists
dic.get("a") or dic.get("c", "").split(" ", 1)[0]
# if dic["a"] could be non-truthy, e.g. empty string
dic["a"] if "a" in dic else dic.get("c", "").split(" ", 1)[0]
dic.get("c")
Your dictionary doesn't contain "c", so it returns None.
dic.get("c").split(" ",1)[0]
Since we know that dic.get("c") is None:
None.split(" ",1)[0]
None has no split method, so that's why you get that error.
The arguments are all evaluated before they're passed to the method.
I am still new to Python and have been reviewing the following code not written by me.
Could someone please explain how the first instance of the variable "clean" is able to be be called in the check_arguments function? It seems to me as though it is calling an as yet undefined variable. The code works but shouldn't that call to "clean" produce an error?
To be clear the bit I am referring to is this.
def check_arguments(ages):
clean, ages_list = parse_ages_argument(ages)
The full code is as follows...
def check_arguments(ages):
clean, ages_list = parse_ages_argument(ages)
if clean != True:
print('invalid ages: %s') % ages
return ages_list
def parse_ages_argument(ages):
clean = True
ages_list = []
ages_string_list = ages.split(',')
for age_string in ages_string_list:
if age_string.isdigit() != True:
clean = False
break
for age_string in ages_string_list:
try:
ages_list.append(int(age_string))
except ValueError:
clean = False
break
ages_list.sort(reverse=True)
return clean, ages_list
ages_list = check_arguments('1,2,3')
print(ages_list)
Python doesn't have a comma operator. What you are seeing is sequence unpacking.
>>> a, b = 1, 2
>>> print a, b
1 2
how the first instance of the variable "clean" is able to be be called in the check_arguments function?
This is a nonsensical thing to ask in the first place, since variables aren't called; functions are. Further, "instance" normally means "a value that is of some class type", not "occurrence of the thing in question in the code listing".
That said: the line of code in question does not use an undefined variable clean. It defines the variable clean (and ages_list at the same time). parse_ages_argument returns two values (as you can see by examining its return statement). The two returned values are assigned to the two variables, respectively.