I am writing a platform for agent simulations in python 3.7. A user of the system implements the mind (decision making procedure, belief revision etc) in a python class. An agent attempts an action as follows:
import action
class MyMind:
def decide(self):
return action.move()
decide is called by the platform, the action is then validated and executed in an environment.
def agent_cycle(self):
action = self.mind.decide()
#validate actions
if action is None:
raise ActionException("blah blah")
self.actuator.attempt(action) #attempt in environment
The problem is that if an action is invalid, an exception is thrown, the stacktrace is not informative to the user - i.e. it doesn't provided any information about their decide implementation. decide can be very complex, without any information about it execution path it can be quite difficult to debug. I am wondering if there is a simple way to include decide in the stack trace. Of course I understand why it is not in the stack trace, but I want to 'fake' where the exception is raised (or something along those lines).
I have tried to accomplish this through the use of sys.settrace, to record stack frames in decide and using inspect to get the source from frames. I can get quite a nice summary this way similar to traceback However it is quite hacky and I am not sure how best to incorporate it into an exception. I feel like there must be an easier way, any suggestions would be greatly appreciated.
Related
I would like to ask how to solve such a situation according to the art of Python.
I have an extentions_and_directories module which includes this function, among others:
def create_dir_if_not_exits(url):
path = pathlib.Path(url)
path.mkdir(parents=True, exist_ok=True)
If I try to create a subfolder of a folder for which I do not have permission I get a PermissionError.
I could catch him at this stage, but I don't really know what to do with him then. Raise another exception?
Returning false - this is probably not a good solution in terms of correct encoding.
So I use this function, for example, in the Project class in the method of the create_project class.
This function in a nutshell creates a project file in the appropriate directory that, if necessary
must create. Simplifying it to what we are interested in, it looks like this:
def create_project(directory, filename):
#code[...]
create_dir_if_not_exits(url)
#code[...]
This class is only part of the modules that helps organize the work, so it doesn't pay off
no GUI or print messages. It is only supposed to be a tool that I will use
in other modules and classes. For example, I will use it like this (pseudo code here)
1. Open the dialog for the user to choose where to save the file.
2. create_project(directory, filename)
3. Save the file in the created directory.
How to properly behave in a situation?
Is it correct, if only at the last stage of the code (written as pseudo code in this case,
I will add a try and the appropriate behavior in the form of returning a message in the form of a GUI and ordering it again
file selection, or according to the art, should I handle this exception somehow, at an earlier stage?
Theoretically, a developer using the create_project method in his project won't get it this way
an exception in my create_project method, only in one of the lines of the create_dir_if_not_exits function,
which in turn is one line of create_project code. So it is not - as I understand it - full
this term means an exception to the create_project method. Is not this a problem and will not stay
it is perceived as improper practice, misleading the user of the method, or else
the previous function create_dir_if_not_exits, is this a correct practice and it should be so?
I'm writing a Python function which takes data from an online source and copies it into a local data dump. If there's a file present already on the intended path for said data dump, I'd like my program to stop abruptly, giving a short message to the user - I'm just building a tiny CLI-type program - explaining why what he or she was about to attempt could destroy valuable data.
Would it be appropriate to raise a FileExists error in the above circumstances? If so, I imagine my code would look something like this:
def make_data_dump():
if os.path.exists("path/to/dump"):
raise FileExistsError("Must not overwrite dump at path/to/dump.")
data = get_data_from_url()
write_to_path(data, "path/to/dump")
Apologies if this is a silly question, but I couldn't find any guidance on when to raise a FileExistsError manually, only on what to do if one's program raises such an exception unexpectedly - hence my asking if raising said exception manually is ever good practice.
The Python documentation explicitly states that this is allowed:
User code can raise built-in exceptions. This can be used to test an exception handler or to report an error condition “just like” the situation in which the interpreter raises the same exception; but beware that there is nothing to prevent user code from raising an inappropriate error.
However, your code example is wrong for a different reason. The problem with this code is that it's using the LBYL (Look Before You Leap) pattern, which might read to race conditions. (In the time between checking if the file exists and writing the file, another process could have created the file, which now would be overwritten). A better pattern for these type of scenarios is the EAFP (Easier to Ask for Forgiveness than Permission) pattern.
See What is the EAFP principle in Python? for more information and examples.
Having said that, I think that for most Python code, manually raising a FileExistsError isn't that common, since you would use the standard Python libraries that already throw this error when necessary. However, one valid reason I can think of is when you would write a wrapper for a low-level function (implemented in another language like C) and would want to translate the error to Python.
A hypothetical code example to demonstrate this:
def make_data_dump():
data = get_data_from_url()
# Assuming write_to_path() is a function in a C library, which returns an error code.
error = write_to_path(data, "path/to/dump")
if error == EEXIST:
raise FileExistsError("Must not overwrite dump at path/to/dump.")
Note that some other built-in exceptions are much more common to raise manually, for example a StopIteration, when implementing an iterator, or a ValueError, for example when your method gets an argument with the wrong value (example).
Some exceptions you will rarely use yourself, like a SyntaxError for example.
As per NobbyNobbs's comment above: if the programmer raises standard exception in his code, it's difficult to work out, during error handling, if a given exception was raised on the application or the system level. Therefore, it's a practice best avoided.
I wonder is if throwing exceptions is the best way to communicate something to the user, in case the user is another programmer.
I'm developing a small library to create text-based games (think Dwarf Fortress, but extremely more simple). Of course, I want stuff to move inside the map. It's very simple and the docstring is very complete, so it should read nicely.
def move(self, tile):
"""Move the object to a tile.
That means: unlink the piece from its current tile and link it
to the new tile.
Raise CharacterIsNotOnATileError if piece didn't already
have an associated tile, CharacterIsNotOnThisBoardError if
the destinity tile is not on the same board as the current tile,
OutOfBoard error if destinity tile is falsey (most probably
this means you're tring to move somewhere outside the map)
"""
if tile.piece is not None:
raise PositionOccupiedError(tile)
if not self.home_tile:
raise PieceIsNotOnATileError
if self.home_tile.board is not tile.board:
raise PieceIsNotOnThisBoardError
if not tile:
raise OutOfBoardError
self.home_tile.piece = None
tile.piece = self
Is this structure bad? I think it reads nicely: when the user tries to move a piece of his own, he can do:
try:
character.move(somewhere)
except PositionOcuppiedError:
character.punish # cause i'm in hardcore
except OutOfBoardError:
character.kill # cause we were in a floating world and the
# character just fell off
There's some more logic in the library implemented like this, where the code tries to do something but if it can't, it will throw an exception. Is this OK? Or maybe I should be returning error codes (as integers, for example).
This usage case seems to be all right for exceptions. Some people say that:
"Exceptions should be exceptional"
but in Python, use of exceptions to perform flow control is not regarded as a bad practice (read more "Is it a good practice to use try except else in python").
Moreover, be aware that throwing and checking exceptions in Python might have a negative performance impact, if placed in repeatedly invoked functions. Unless you are developing a fast-paced game with hundreds of players, it shouldn't be your primary concern. You can read more about performance related issues of catching exceptions in this topic on Stack Overflow.
Personally, as a programmer I would rather deal with exceptions (when programming in Python) than with error codes or something similar. On the other hand, it is not much harder to to handle wide range of returned statuses if they are given as error codes - you can still mimic switch-case construct, using e.g. dictionary with callable handlers as values.
In my opinion throwing exceptions are better way to inform that some error occurred. Especially if your function does not return any value. Imagine that developer after each call have to check type of returned value. It is much clearer.
Check this.
The documentation for GAE's ndp.put_multi is severely lacking. NDB Entities and Keys - Python — Google Cloud Platform shows that it returns a list of keys (list_of_keys = ndb.put_multi(list_of_entities)), but it says nothing about failures. NDB Functions doesn't provide much more information.
Spelunking through the code (below), shows me that, at least for now, put_multi just aggregates the Future.get_result()s returned from the async method, which itself delegates to the entities' put code. Now, the docs for the NDB Future Class indicate that a result will be returned or else an exception will be raised. I've been told, however, that the result will be None if a particular put failed (I can't find any authoritative documentation to that effect, but if it's anything like db.get then that would make sense).
So all of this boils down to some questions I can't find the answers to:
Clearly, the return value is a list - is it a list with some elements possibly None? Or are exceptions used instead?
When there is an error, what should be re-put? Can all entities be re-put (idempotent), or only those whose return value are None (if that's even how errors are communicated)?
How common are errors (One answer: 1/3000)? Do they show up in logs (because I haven't seen any)? Is there a way to reliably simulate an error for testing?
Usage of the function in an open source library implies that the operation is idempotent, but that's about it. (Other usages don't even bother checking the return value or catching exceptions.)
Handling Datastore Errors makes no mention of anything but exceptions.
I agree with your reading of the code: put_multi() reacts to an error the same way put_async().get_result() does. If put() would raise an exception, put_multi() will also, and will be unhelpful about which of the multiple calls failed. I'm not aware of a circumstance where put_multi() would return None for some entries in the key list.
You can re-put entities that have been put, assuming no other user has updated those entities since the last put attempt. Entities that are created with system-generated IDs have their in-memory keys updated, so re-putting these would overwrite the existing entities and not create new ones. I wouldn't call it idempotent exactly because the retry would overwrite any updates to those entities made by other processes.
Of course, if you need more control over the result, you can perform this update in a transaction, though all entities would need to be in the same entity group for this to work with a primitive transaction. (Cross-group transactions support up to five distinct entity groups.) A failure during a transaction would ensure that none of the entities are created or updated if any of the attempts fail.
I don't know a general error rate for update failures. Such failures are most likely to include contention errors or "hot tablets" (too many updates to nearby records in too short a time, resulting in a timeout), which would depend on app behavior. All such errors are reported by the API as exceptions.
The easiest way to test error handling call paths would be to wrap the Model class and override the methods with a test mode behavior. You could get fancier and dig into the stub API used by testbed, which may have a way to hook into low-level calls and simulate errors. (I don't think this is a feature of testbed directly but you could use a similar technique.)
I'm doing a bit of research for my final year project. It's mostly about creating a more convenient way of dealing with Exceptions thrown in programs. It does this by creating a custom handler for each type of Exception. I was wondering how often are builtin/standard library Exceptions are dealt with in comparison to Exception by you/3rd party software?
Why I'm asking is two fold:
I would like my demonstration as to more realistic. My project has the chance to be more help than just dealing with Exceptions so given the chance, I would rather work on giving the tool far more abilities. Given this, I would like my sample handlers to be bias in the "right" direction.
It will influence how detailed I can make the API to help create more detailed Exceptions and Exception Handlers.
Thanks for taking the time to read this crap.
EDIT:
I'll break it down because I dont think I'm explaining it properly.
The nice little stack trace you get when errors thrown about? I want to try and improve it and see if something before could indicate when it all started to go wrong(for some errors might need a different strategy for others and that's where defining handlers come in). I think I could do this. In order to do this, I need to divide my time accordingly. I want to know whether I should focus on tackling builtin errors or helping people define their handlers for their Exceptions(maybe this second is pointless but I can't know until I ask people). I'll do this by asking people about their experiences.
EDIT2:
I'm a dumbass, I mean errors not exceptions. I need sleep.
Regardless of what you're trying to do with the answer, I'll answer your specific question:
how often are builtin/standard library
Exceptions are dealt with in
comparison to Exception by you/3rd
party software?
It depends on the domain. Some areas lend themselves to defining specific exceptions (e.g. web programming), and others tend to rely on builtins (e.g. mathematical and scientific computing). The amount of exceptions handled probably leans more towards "standard" exceptions like TypeError or NameError, but harder errors usually aren't contained in the builtins (it's easy to fix an incorrect argument, invalid input, or a typo, which are common causes of exceptions like NameError or TypeError, but it's hard to fix an error that doesn't come from something so simple).
So, IMO, standard exceptions are more prevalent but the ones defined by users, frameworks, etc. are more useful/important (or whatever you want to call it) because they represent more complex and relevant issues.
You can always look at some popular 3rd party Python modules (Google code is a good place to look) and see how many of them define exceptions.