How to properly interpret a single line of python code? - python

I need to execute a line of python code that is entered by the user.
If it is a statement I want to execute it, but if it is an expression, I want the result to be returned and do some fancy stuff with it.
The problem is that python has two different functions for that, namely exec and eval.
Currently I just try to evaluate the string that the user entered.
If that raises a SyntaxError, this may indicate that the string is an statement instead, so I try to execute it.
try:
result = eval(command, scope)
except SyntaxError:
# Probably command is a statement, not an expression
try:
exec(command, scope)
except Exception as e:
return command + ' : ' + str(e)
except Exception as e:
return command + ' : ' + str(e)
else:
pass # Some fancy stuff
This feels rather hacky. Is there a neater, more pythonic way to do this?

While I think your existing code is probably reasonably Pythonic (under the doctrine that it's "easier to ask forgiveness than permission"), I suspect the best alternative approach is to use the ast module to inspect the code in your string:
tree = ast.parse(some_input_string)
if len(tree.body) == 1 and isinstance(tree.body[0], ast.Expr):
result = eval(some_input_string, scope)
else:
exec(some_input_string, scope)
result = None
Note that some common statements are really "expression statements". So, an input string like 'do_stuff("args")' will use the eval branch of the code above, rather than the exec branch. I don't think this will have any adverse consequences, but you never know.
It is also possible to compile the tree that has been parsed and then pass the result into the eval or exec calls later. I found it rather fiddly to get right though (you need to wrap the ast.Expr's value attribute in an ast.Expression in the top branch) and so I went with the simpler (to read and understand) alternative of just passing in the string and letting Python parse it again.

You can refactor the try-except a bit. There's no real context in your example, but assuming you want to be able to execute a=1 and then evaluate a afterwards and get 1, then you could do something like...
from code import InteractiveConsole
interpreter = InteractiveConsole()
def run(code):
try: return eval(code.strip(), interpreter.locals)
except: pass
try: interpreter.runcode(code)
except Exception as error: return error
This should work for more than one line of code too.
Without knowing a bit more about your objective it's difficult to say how to do it best, but what you have is fine in principle, it just needs tidying up. This similar answer includes a minimal version of the same try-except logic again, with a focus on mimicking the interpreter more faithfully.

you missed one, actually there are three functions related to executing code, and the one you missed is compile().
compile() takes three required arguments, the code to be compiled, the name of the module being compiled, which will appear in tracebacks originating from that code, and a "mode". The mode argument should be one of "exec", for compiling whole modules, "eval" for compiling simple expressions, and "single", which should be a single line of interactive input!
In all three cases, you pass the returned code object to eval, with the desired context:
>>> c = compile("if 1 < 2:\n print(3)", "<string>", "single")
>>> eval(c)
3
>>>

Related

How to handle unexpected argument type in python

Background
I am new to python and I am writing a simple function but I am also interested in learning to do things the correct / pythonic way as I progress in my journey.
Lets consider the function below
def test_func(nested_lists,val):
return
I am expecting two arguments. One argument would be a list containing more lists. Something like this [[1,2,3,],[4,5,6,]...]. The second argument could be a value like 1.
If someone say for instance passes in a single value as the first argument and an array as the second argument. My code as it is currently returning the correct output which is 0 , However is there another way that i should be handle this?
For example should I be doing something like this
if(type(value) == list):
return 0
Or do i not need to do anything because my function is returning 0 anyway.
I know this maybe a very basic question so please forgive me but coming from a java background I am new to python so i am not sure how to handle such scenarios in python.
The other answer illustrates the proper way to check in advance for problems you can foresee. I'll provide a different approach.
The idiomatic solution in python is to "ask forgiveness, not permission". There are a lot of things that can go wrong, and where other languages might ask you to foresee all those problems and address them manually, python encourages just handling them as they happen. I would recommend doing:
def test_func(nested_lists, val):
try:
...
except TypeError:
# do whatever error-handling behavior you need to
# either throw a custom exception or return a specific value or whatever
return 0
and then designing your code in such a way that, if nested_lists and values are not compatible types, then they throw a TypeError (e.g. trying to iterate through nested_lists should fail if nested_lists is not a list. You can experiment with this behavior in a python console, but in general trying to do something to a variable that doesn't work because it's not the right type will produce a TypeError).
If your current code is working correctly, there is no pressing need to change anything. However, there are some reasons you might want to code more defensively;
If the code will seem to work correctly when you pass in bogus values, it would be better if it raised an exception instead of return a bogus value. The responsibility to call it correctly lies squarely with the caller, but enforcing it can help make sure the code is correct.
if not isinstance(nested_lists,list):
raise ValueError('Need a list, got {0!r}'.format(nested_lists))
This has the drawback that it hardcodes list as the type for the first argument; properly reusable code should work with any type, as long as it has the required methods and behaviors to remain compatible with your implementation. Perhaps instead check for a behavior:
try:
something involving nested_lists[0][0]
except (IndexError, AttributeError):
raise ValueError('Expected nested list but got {0!r}'.format(nested_lists))
(The try is not strictly necessary here; but see below.)
If you get a traceback when you call the code incorrectly, but it is opaque or misleading, it is more helpful to catch and explicitly point out the error earlier. #or example, the snippet above (without the try wrapper) would produce
Traceback (most recent call last):
module __main__ line 141
traceback.print_exc()
module <module> line 1
test_func(1,1)
module <module> line 2
AttributeError: 'int' object has no attribute '__getitem__'
which is somewhat unobvious to debug.
If the code will be used by third parties, both of the above considerations will be more important from a support point of view, too.
Notice how the code raises an exception when called incorrectly. This is generally better than silently returning some garbage value, and the caller can similarly trap the error with a try/except if this is well-defined (i.e. documented!) behavior.
Finally, since Python 3.5, you have the option to use type annotations:
def test_func(nested_lists: list, val: int) -> int:
...
As noted in the documentation, the core language does not (yet?) enforce these type checks, but they can help static code analysis tools point out possible errors.

Commented block indentation in python

Is there a way to avoid "expected indent block" errors in Python without adding pass to a function?
def exclamation(s):
# s += "!!!"
# return s
print exclamation("The indented horror")
The above code results in an error on line 5. Yes, remove the comments and the code works fine. However, in debugging stuff I often find myself in a similar situation. Is this just a hang-up of the off-side rule and something I need to get used to or are there ways around this?
There has to be something within your function definition to avoid a SyntaxError.
The issue is that the interpreter will effectively ignore comments during parsing, and so while to a human it might look like something is there, to the parser it is empty.
As jonrsharpe has pointed out in a comment, you can use docstrings to "comment out" your code and have it still work. This is because the docstring is, in effect, a normal string. As such this will be parsed and won't cause a SyntaxError. To take your example code it would look like:
def exclamation(s):
'''s += "!!!"
return s'''
# This should print None as nothing is now returned from the func
print(exclamation("The indented horror"))

Python: performing a function to check if the file has a suffix

I wrote this function to use from an other function, checking if the output has the correct suffix. Before I use it I have two questions:
Is TypeError the best exception to use in this case?
Is there some built-in function I could use instead of mine?
Code:
def suffix_NameCheck(inFile):
if os.path.splitext(inFile)[1] is None:
raise TypeError('"%s" has not a suffix' % inFile)
else:
return inFile
There is not a built-in function that does this, because
Not all file systems really use extensions.
It's probably more common to just use the if statement on its own, rather than throwing it in a function.
The reason is that you're probably going to have to catch that exception somewhere (unless you intentionally want it to end the program, in which case, there's probably a better way to do it than throwing an exception, e.g. sys.exit). The code to catch the exception would be at least as involved as the code to just do the if check itself and handle it.
I would not raise an exception from a function named suffix_NameCheck, but rather return True or False instead. The normal operation of suffix_NameCheck is not affected by wether or not there is a correct suffix on the value, it's job is to check for that suffix.
The function that uses suffix_NameCheck could still raise an exception; ValueError may be a better exception to use in that case.
I think this is more appropriately a ValueError. The problem you are attempting to report is that the user provided a file path/name argument as a string that has no file extension. That's a problem with the value of the string, not anything's type.
Other answerers have a point that this sort of functionality at first glance should really be returning a bool, but given that that use case is covered simply with bool(os.path.splitext(candidate)[1]), I think that if you really do want to raise an exception here, you are already working with a lower level function that returns a bool.

Raising and catching exceptions in an expression in Python 3

Let me start off by saying: this is to be used for esoteric purposes - not production code. I'm playing around with doing stuff in a single line of Python code, hence my need for expressions and not statements. (EDIT: I'm working on mechanically compiling code to single line of (mostly) equivalent Python code, BitBucket - onelinepython. Note it's very work in progress, hence my reticence in initially mentioning it)
I essentially want to do two things:
Call a function that raises an exception instance of my choosing something like:
raise_exception(WhateverException())
Run a function in an enclosed environment where I can get the exception instance that is raised, if one is raised, and, otherwise, the return value of the function that was called. E.g.:
has_exception, return_or_exception = check_exception(f, param1, param2, ...)
Ideally, I want to do this with some default library or built-in function (no matter how much I have to bastardise its intended use). I don't need functions that have the exact same signatures as the examples I provided, just something I can mangle into something close enough. I do have one restriction, though: no use of eval() or equivalent functions.
EDIT: I know I could define my own functions to do this, but then they would still have to follow the restriction that they are a single expression. So solutions that use raise and try inside a function definition are out. Function definitions, raise-statement and try-blocks are unfortunately statements and not expressions.
As for any solutions I've tried. The answer is none yet. The closest I have to an idea of how to solve this is by misusing unittest's assert functionality, but I think that is a dead-end.
EDIT 2: To make it clear, I'm fine with using a module or such that uses raise-statements or try-blocks somewhere in its code. My goal is to take some code and turn it into an equivalent single line of code (which includes any helper functions I may be using). But since I want this to work on a default installation of Python I want to only use default libraries.
To raise an exception:
>>> import warnings
>>> WV = type("WV", (Warning, ValueError), {})
>>> warnings.simplefilter("error", WV)
>>> warnings.warn("wv", WV)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
__main__.WV: wv
To catch an exception:
>>> import unittest
>>> res = type("TR", (unittest.TestResult, ), dict(addError=lambda self, test, err: setattr(self, '_last_err', err)))()
>>> unittest.FunctionTestCase(lambda: [][0])(res)
>>> res._last_err
(<type 'exceptions.IndexError'>, IndexError('list index out of range',), <traceback object at 0x2b4358e69950>)
Note that the warnings method will only work for exceptions that derive from Warning, but you should always be able to multiply-inherit; here's an example:
>>> WS = type("WS", (Warning, StopIteration), {})
>>> warnings.simplefilter("error", WS)
>>> list(type("R", (object,), dict(__init__=lambda self, stop: (setattr(self, 'stop', stop), setattr(self, 'i', 0), None)[-1], __iter__=lambda self: self, next=lambda self: (self.i, setattr(self, 'i', self.i + 1))[0] if self.i < self.stop else warnings.warn("Stop", WS)))(5))
[0, 1, 2, 3, 4]
You can define your own functions to do this:
def raise_exception(ex):
raise ex
def check_exception(f, *args, **kwargs):
try:
return False, f(*args, **kwargs)
except Exception as e:
return True, e
This answer suggests that catching exceptions with an expression is not possible in general. I'm also pretty sure that it's not possible to raise an arbitrary exception without using raise. (You can generate some particular exceptions with expressions like 1/0 or dict['keyThatWillNeverExist'], but not any arbitrary exception with arbitrary exception info.)
The language reference says:
The Python interpreter raises an exception when it detects a run-time error (such as division by zero). A Python program can also explicitly raise an exception with the raise statement. Exception handlers are specified with the try ... except statement.
Although this doesn't rule out the possibility that some dark corner of the language specification allows raising exceptions in other ways, the statement is pretty straightforward: you raise exceptions with raise and catch them with try/except.
Note that using unittest, or any other Python library, is unlikely to be a real solution in your sense, because unittest contains functions written in Python that use try/except. So if you're okay with using unittest, you ought to be okay with writing your own functions.
I imagine it might be possible to achieve your goal by "cheating" and writing a C extension that provides functions doing what you want. But that's not really converting it to equivalent Python code.
You are asking how to raise an exception without using raise and catch an exception without using except. Your reluctance to use these statements is because you can't use more than one statement in a single line of code, and you have the idea to compile Python modules into oneliners.
Short answer: Well, you can't.
And even if you could, why would you? It's a completely meaningless effort. The code is not faster or even significantly smaller because it is in one line. It goes against the idea of Python as well. And if you want to obfuscate it, there are much better ways, including compiling it to bytecode.
Longer answer:
You could implement your own exception system, independent of the Python exceptions, but that would be astonishingly slow, and this would still not catch the Python exceptions, so it's not useful in your case.
For the raise-statement, you could re-implementing the raise statement as a function in C, but this you seem to think is cheating, and I also don't see how it would be possible with other statements, such as except.
You could also move out some statements into functions in a separate module, but this is of course then no longer actually a one-liner module in any meaningful way, and not all statements are easily wrapped like this, except being the most relevant case here. You'd have to wrap the whole try/except block, but the resulting function would in turn also only take expressions as parameters, so you would have to extract the blocks into functions, and you'd end up needing to basically re-implement most of Python as a statement-less language, which is silly. And you'd end up with the helper functions in a separate module, which you don't want to.
So the answer to your question of how to raise an exception without using raise and catch an exception without using except is "You don't".

Can I be warned when I used a generator function by accident

I was working with generator functions and private functions of a class. I am wondering
Why when yielding (which in my one case was by accident) in __someFunc that this function just appears not to be called from within __someGenerator. Also what is the terminology I want to use when referring to these aspects of the language?
Can the python interpreter warn of such instances?
Below is an example snippet of my scenario.
class someClass():
def __init__(self):
pass
#Copy and paste mistake where yield ended up in a regular function
def __someFunc(self):
print "hello"
#yield True #if yielding in this function it isn't called
def __someGenerator (self):
for i in range(0, 10):
self.__someFunc()
yield True
yield False
def someMethod(self):
func = self.__someGenerator()
while func.next():
print "next"
sc = someClass()
sc.someMethod()
I got burned on this and spent some time trying to figure out why a function just wasn't getting called. I finally discovered I was yielding in function I didn't want to in.
A "generator" isn't so much a language feature, as a name for functions that "yield." Yielding is pretty much always legal. There's not really any way for Python to know that you didn't "mean" to yield from some function.
This PEP http://www.python.org/dev/peps/pep-0255/ talks about generators, and may help you understand the background better.
I sympathize with your experience, but compilers can't figure out what you "meant for them to do", only what you actually told them to do.
I'll try to answer the first of your questions.
A regular function, when called like this:
val = func()
executes its inside statements until it ends or a return statement is reached. Then the return value of the function is assigned to val.
If a compiler recognizes the function to actually be a generator and not a regular function (it does that by looking for yield statements inside the function -- if there's at least one, it's a generator), the scenario when calling it the same way as above has different consequences. Upon calling func(), no code inside the function is executed, and a special <generator> value is assigned to val. Then, the first time you call val.next(), the actual statements of func are being executed until a yield or return is encountered, upon which the execution of the function stops, value yielded is returned and generator waits for another call to val.next().
That's why, in your example, function __someFunc didn't print "hello" -- its statements were not executed, because you haven't called self.__someFunc().next(), but only self.__someFunc().
Unfortunately, I'm pretty sure there's no built-in warning mechanism for programming errors like yours.
Python doesn't know whether you want to create a generator object for later iteration or call a function. But python isn't your only tool for seeing what's going on with your code. If you're using an editor or IDE that allows customized syntax highlighting, you can tell it to give the yield keyword a different color, or even a bright background, which will help you find your errors more quickly, at least. In vim, for example, you might do:
:syntax keyword Yield yield
:highlight yield ctermbg=yellow guibg=yellow ctermfg=blue guifg=blue
Those are horrendous colors, by the way. I recommend picking something better. Another option, if your editor or IDE won't cooperate, is to set up a custom rule in a code checker like pylint. An example from pylint's source tarball:
from pylint.interfaces import IRawChecker
from pylint.checkers import BaseChecker
class MyRawChecker(BaseChecker):
"""check for line continuations with '\' instead of using triple
quoted string or parenthesis
"""
__implements__ = IRawChecker
name = 'custom_raw'
msgs = {'W9901': ('use \\ for line continuation',
('Used when a \\ is used for a line continuation instead'
' of using triple quoted string or parenthesis.')),
}
options = ()
def process_module(self, stream):
"""process a module
the module's content is accessible via the stream object
"""
for (lineno, line) in enumerate(stream):
if line.rstrip().endswith('\\'):
self.add_message('W9901', line=lineno)
def register(linter):
"""required method to auto register this checker"""
linter.register_checker(MyRawChecker(linter))
The pylint manual is available here: http://www.logilab.org/card/pylint_manual
And vim's syntax documentation is here: http://www.vim.org/htmldoc/syntax.html
Because the return keyword is applicable in both generator functions and regular functions, there's nothing you could possibly check (as #Christopher mentions). The return keyword in a generator indicates that a StopIteration exception should be raised.
If you try to return with a value from within a generator (which doesn't make sense, since return just means "stop iteration"), the compiler will complain at compile-time -- this may catch some copy-and-paste mistakes:
>>> def foo():
... yield 12
... return 15
...
File "<stdin>", line 3
SyntaxError: 'return' with argument inside generator
I personally just advise against copy and paste programming. :-)
From the PEP:
Note that return means "I'm done, and have nothing interesting to
return", for both generator functions and non-generator functions.
We do this.
Generators have names with "generate" or "gen" in their name. It will have a yield statement in the body. Pretty easy to check visually, since no method is much over 20 lines of code.
Other methods don't have "gen" in their name.
Also, we do not every use __ (double underscore) names under any circumstances. 32,000 lines of code. Non __ names.
The "generator vs. non-generator" method function is entirely a design question. What did the programmer "intend" to happen. The compiler can't easily validate your intent, it can only validate what you actually typed.

Categories