From time to time, I write a generic function in Python that gets called multiple times with different arguments. Usually, this is driven by a definition somewhere else.
For example:
def issue_sql_query(name, select_stmt):
...
QUERIES = [
"get_user_rows", "SELECT name, rowid FROM table WHERE type == 'USER';"
...
]
results = []
for name, select_stmt in QUERIES:
results.append((name, issue_sql_query(name, select_stmt)))
If there's an exception in the generic function (i.e., issue_sql_query or somewhere deeper), I have relatively little info in the traceback to identify which definition caused the error.
What I'd like to do is dynamically rename or augment the function name/stack frame so that tracebacks would include some identifying info.
What would be nice is something like this:
File "test.py", line 21, in <module>
results.append((name, issue_sql_query(select_stmt)))
File "test.py", line 11, in issue_sql_query(name="get_user_rows")
raise RuntimeError("Some error")
RuntimeError: Some error
I could, of course, stick exception handling at the generic points and rebuild the exception using traceback to have more context, which is pretty straightforward and likely the right choice. It gets a little tricky when you have multiple levels of generic functions, but that's certainly possible to handle.
Any other ideas on how to accomplish this? Did I miss some easy way to change the stack frame name?
Edit:
Adding an example traceback showing a relatively not useful traceback:
Traceback (most recent call last):
File "C:\Python27\lib\runpy.py", line 162, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "C:\Python27\lib\runpy.py", line 72, in _run_code
exec code in run_globals
File "c:\tmp\report.py", line 767, in <module>
main()
File "c:\tmp\report.py", line 750, in main
charts.append(report.get_chart(title, definition))
File "c:\tmp\report.py", line 614, in get_chart
return self.get_bar_chart(title, definition)
File "c:\tmp\report.py", line 689, in get_bar_chart
definition, cursor, **kwargs))
File "c:\tmp\report.py", line 627, in create_key_table
for row in cursor.execute(full_select_stmt):
sqlite3.OperationalError: near "==": syntax error
You could create wrapper functions for each named query so that you can control the exception thrown:
>>> fns = {}
>>> def issue_sql_query(name, stmt):
... if name not in fns:
... def f(name, stmt):
... # run query here
... raise Exception(name)
...
... fns[name] = f
... return fns[name](name, stmt)
...
>>>
>>> issue_sql_query('b', 'SQL')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 9, in issue_sql_query
File "<stdin>", line 5, in f
Exception: b
>>> issue_sql_query('a', 'SQL')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 9, in issue_sql_query
File "<stdin>", line 5, in f
Exception: a
>>>
Related
I'm trying to get this, https://github.com/joaquinlpereyra/twitterImgBot, to work
and it works and it seems ok.
But after some hours, it stops working and this error comes up:
*python3 twitterbot.py
Traceback (most recent call last):
File "/home/user/.local/lib/python3.7/site-packages/tweepy/binder.py", line 118, in build_path
value = quote(self.session.params[name])
KeyError: 'id'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "twitterbot.py", line 209, in <module>
main()
File "twitterbot.py", line 200, in main
orders()
File "twitterbot.py", line 118, in orders
timeline.delete_tweet_by_id(tweet.in_reply_to_status_id, api)
File "/home/user/Skrivebord/twitterboot/lo/bot/timeline.py", line 12, in delete_tweet_by_id
api.destroy_status(id_to_delete)
File "/home/user/.local/lib/python3.7/site-packages/tweepy/binder.py", line 245, in _call
method = APIMethod(args, kwargs)
File "/home/user/.local/lib/python3.7/site-packages/tweepy/binder.py", line 71, in __init__
self.build_path()
File "/home/user/.local/lib/python3.7/site-packages/tweepy/binder.py", line 120, in build_path
raise TweepError('No parameter value found for path variable: %s' % name)
tweepy.error.TweepError: No parameter value found for path variable: id*
It seems like the Python has some problem because if I make a new install on a another PC it works for some hours and then stops.
Strange.
This is likely because tweet is not in reply to a status, so has an in_reply_to_status_id attribute that's None, so API.destroy_status is called with an id of None.
I got an error for this code:
from pathos.multiprocessing import ProcessingPool
def diePlz(im):
print('Whoopdepoop!')
def caller():
im = 1
pool = ProcessingPool()
pool.map(diePlz,[im,im,im,im])
if __name__=='__main__':
caller()
when I ran it with the cProfiler: (python3 -m cProfile testProfiler.py)
multiprocess.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/home/rohit/.local/lib/python3.6/site-packages/multiprocess/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/home/rohit/.local/lib/python3.6/site-packages/multiprocess/pool.py", line 44, in mapstar
return list(map(*args))
File "/home/rohit/.local/lib/python3.6/site-packages/pathos/helpers/mp_helper.py", line 15, in <lambda>
func = lambda args: f(*args)
File "testProfiler.py", line 3, in diePlz
print('Whoopdepoop!')
NameError: name 'print' is not defined
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/lib/python3.6/cProfile.py", line 160, in <module>
main()
File "/usr/lib/python3.6/cProfile.py", line 153, in main
runctx(code, globs, None, options.outfile, options.sort)
File "/usr/lib/python3.6/cProfile.py", line 20, in runctx
filename, sort)
File "/usr/lib/python3.6/profile.py", line 64, in runctx
prof.runctx(statement, globals, locals)
File "/usr/lib/python3.6/cProfile.py", line 100, in runctx
exec(cmd, globals, locals)
File "testProfiler.py", line 11, in <module>
caller()
File "testProfiler.py", line 8, in caller
pool.map(diePlz,[im,im,im,im])
File "/home/rohit/.local/lib/python3.6/site-packages/pathos/multiprocessing.py", line 137, in map
return _pool.map(star(f), zip(*args)) # chunksize
File "/home/rohit/.local/lib/python3.6/site-packages/multiprocess/pool.py", line 266, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/home/rohit/.local/lib/python3.6/site-packages/multiprocess/pool.py", line 644, in get
raise self._value
NameError: name 'print' is not defined
But when I ran it without the cProfiler:
$ python3 testProfiler.py
Whoopdepoop!
Whoopdepoop!
Whoopdepoop!
Whoopdepoop!
The code that I've provided is a minimal working example for the problem. There is a much larger code that I want to debug, but am not able to do so because cProfiler keeps raising weird errors.
In this case, the point of importance is
NameError: name 'print' is not defined
which means python3 is not able to recognize print itself. In my code, it was not able to recognize range.
So, I realize this is a long time after the original post, but I have this exact same issue.
In my case I was getting the exact same error as the original post - python builtin functions such as print() or len() resulted in errors like this:
NameError: name 'len' is not defined
I'm currently running multiprocess version 0.70.11.1 and dill version 0.3.3 (components of pathos that make process based parallelism work).
Based on what I found in an issue comment: https://github.com/uqfoundation/pathos/issues/129#issuecomment-536081859 one of the package authors recommends trying:
import dill
dill.settings['recurse'] = True
At least in my case, the above fixed the error!
I am using following method isset(var) to determine if a variable exists.
def isset(variable):
try:
variable
except NameError:
return False
else:
return True
It returns True if variable exists. But if a variable doesn't exist I get following:
Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/home/lenovo/pyth/master/vzero/__main__.py", line 26, in <module>
ss.run()
File "vzero/ss.py", line 4, in run
snap()
File "vzero/ss.py", line 7, in snap
core.display()
File "vzero/core.py", line 77, in display
stdout(session(username()))
File "vzero/core.py", line 95, in session
if isset(ghi): #current_sessions[user]):
NameError: global name 'ghi' is not defined
I don't want all these errors. I just want it return False. No output. How can I do this?
Instead of writing a complex helper function isset and calling it
if not isset('variable_name'):
# handle the situation
in the place where you want to check the presence of the variable do:
try:
# some code with the variable in question
except NameError:
# handle the situation
I created a library with decorator. It allows to run seamlessly any function in a separate thread and get timeout exception even that function stuck in TASK_UNINTERRUPTIBLE due to IO to the broken NFS or other nasty problem.
Now I'm working with exception handling. I'm passing exception from the thread back to the caller thread and reraising it again.
Right now trace (if it happens inside decorated function) looks like this:
Traceback (most recent call last):
File "1.py", line 11, in <module>
foo()
File "1.py", line 9, in foo
boo()
File "thread_timeout/__init__.py", line 116, in inner_worker
result = wrapped(*args, **kwargs)
File "1.py", line 6, in boo
file('/fdf','r')
If I remove decorator it looks like this:
Traceback (most recent call last):
File "1.py", line 11, in <module>
foo()
File "1.py", line 9, in foo
boo()
File "1.py", line 6, in boo
file('/fdf','r')
IOError: [Errno 2] No such file or directory: '/fdf'
Question: should I try to hide decorator from the trace or not?
Library code is available here: https://github.com/amarao/thread_timeout/blob/master/thread_timeout/init.py
Thanks.
I am following the code in the readthedocs (http://django-permission.readthedocs.org/en/latest/). Difficulty starts at Apply permission logic section of the docs. All works fine as I cut&paste
art1 = Article.objects.create(
title="Article 1",
body="foobar hogehoge",
author=user1
)
The following traceback is generated
Traceback (most recent call last):
File "<console>", line 4, in <module>
File "C:\Django\test_permissions\lib\site-packages\django\db\models\manager.py", line 157, in create
return self.get_queryset().create(**kwargs)
File "C:\Django\test_permissions\lib\site-packages\django\db\models\query.py", line 320, in create
obj = self.model(**kwargs)
File "C:\Django\test_permissions\lib\site-packages\django\db\models\base.py", line 417, in __init__
raise TypeError("'%s' is an invalid keyword argument for this function" % list(kwargs)[0])
TypeError: 'author' is an invalid keyword argument for this function
If it is changed to
art1 = Project.objects.create(
rest of code is okay
It works okay. So I guess an error. Maybe, I'm unsure.
Anyway, still cut&paste into shell until I get to
>>> assert user1.has_perm('permission.change_article') == False
Traceback (most recent call last):
File "<console>", line 1, in <module>
AssertionError
So I try
>>> assert user1.has_perm('permission.change_article') == True
Works fine. I have to say at this stage I have no idea as to what is going on.
So next line is
assert user1.has_perm('permission.change_article', art1) == True
And now the traceback
>>> assert user1.has_perm('permission.change_article', art1) == True
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "c:\django\test_permissions\lib\site-packages\django\contrib\auth\models.py", line 336, in has_perm return _user_has_perm(self, perm, obj)
File "c:\django\test_permissions\lib\site-packages\django\contrib\auth\models.py", line 273, in _user_has_perm if backend.has_perm(user, perm, obj):
File "c:\django\test_permissions\lib\site-packages\permission\backends.py", line 71, in has_perm if handler.has_perm(user_obj, perm, obj=obj):
File "c:\django\test_permissions\lib\site-packages\permission\handlers.py", line 237, in has_perm if permission_logic.has_perm(user_obj, perm, obj):
File "c:\django\test_permissions\lib\site-packages\permission\logics\author.py", line 122, in has_perm author = field_lookup(obj, self.field_name)
File "c:\django\test_permissions\lib\site-packages\permission\utils\field_lookup.py", line 42, in field_lookup return field_lookup(field_lookup(obj, field_path[0]), field_path[1])
File "c:\django\test_permissions\lib\site-packages\permission\utils\field_lookup.py", line 41, in field_lookup return getattr(obj, field_path[0])
AttributeError: 'Project' object has no attribute 'project'
Have I done something wrong?
I have no idea what to do. I am as far out as Bishops Rock Lighthouse ;-)
I need to get permissions working for my project. Is this the app to go with?
BTW.
(test_permissions) c:\django\test_permissions\test_permissions>pip freeze
Django==1.6.5
Pillow==2.2.2
South==0.8.4
app-version==0.1.2
django-appconf==0.6
django-crispy-forms==1.4.0
django-permission==0.8.0
six==1.7.0
tolerance==0.1.1
Working in virtualenv on Win7
Tommy.
I'm the author of this library.
I'm really sorry but the example code was mis-leading. If you follow the Note section which said "From django-permission version 0.8.0, you can specify related object with field__name attribute like django queryset lookup. See the working example below:", the Article does not have author that's why django blame your code. It should be something like
prj1 = Project.objects.create(
title="Project 1",
body="hello",
author=user1,
)
art1 = Article.objects.create(
title="Article 1",
body="foobar hogehoge",
project=prj1,
)
Then the example should work.
I'll update the document as soon as possible.