We Keep Coding

What does 'File "<string>"' stands for on python debugger and traceback?

Eventually, when I am in a debug session, the current file is named "<string>". What I would expect to see is the name of the module where the debugger is currently on.
The same happens on the traceback:
Traceback (most recent call last):
...
File "~/project_x/app/services/contacts.py", line 23, in find_account
account = self._account_repo.get_by_id(id)
File "<string>", line 2, in get_by_id
File "~/project_x/lib/python3.7/site-packages/pony/orm/core.py", li
ne 528, in new_func
result = func(*args, **kwargs)

It means that the code came from something that was not a file, but a string. My guess would be an exec() call, but maybe there are other ways of getting the same effect. The previous stack frame might give some clues.

Can't use class constructor argument in Python

I can't get what I'm doing wrong here. I'm defining a class with the following code:
import sqlite3 as lite
import sys
import os.path
class database:
def __init__(self,dbfname):
if os.path.isfile(dbfname):
self.con = lite.connect(dbfname)
else:
self.con = self.createDBfile(dbfname)
#Other methods...
Then, when I try to make an instance of the class
base = database("mydb.db")
I get an error saying that there is no "global" variable named dbfname .
Traceback (most recent call last):
File "testdb.py", line 67, in <module>
base = database("mydb.db")
File "testdb.py", line 13, in __init__
self.con = self.createDBfile(dbfname)
File "testdb.py", line 15, in createDBfile
if os.path.isfile(dbfname):
NameError: global name 'dbfname' is not defined
What would be the correct way to use the argument variable dbfname ?

This code looks fine. The error isn't in the code you posted though; it's in testdb.py on line 15 in the createDBfile() method (not in the __init__()).
How do I know this? Well, lets carefully look at the traceback Python gives us:
Traceback (most recent call last):
File "testdb.py", line 67, in <module>
base = database("mydb.db")
File "testdb.py", line 13, in __init__
self.con = self.createDBfile(dbfname)
File "testdb.py", line 15, in createDBfile
if os.path.isfile(dbfname):
NameError: global name 'dbfname' is not defined
Like the first line says, the most recent call is last. So you read a traceback from down to up (and not from up to down).
The very last line is the actual error, but just before that is:
File "testdb.py", line 15, in createDBfile
if os.path.isfile(dbfname):
So it says that in the file testdb.py, on line 15, in the method createDBfile() an error occurred. Python also prints out the contents of this line 15.
Above that is the call to the createDBfile() method in your __init__() function, and above that the call to the __init__() function (by creating the class instance).
You didn't post the contents of this createDBfile() method, so I can't tell you where exactly the error is. I suspect you did something wrong with the function arguments (perhaps as simple as a typo?)

Should I hide decorator from the trace of the decorated function? (python)

I created a library with decorator. It allows to run seamlessly any function in a separate thread and get timeout exception even that function stuck in TASK_UNINTERRUPTIBLE due to IO to the broken NFS or other nasty problem.
Now I'm working with exception handling. I'm passing exception from the thread back to the caller thread and reraising it again.
Right now trace (if it happens inside decorated function) looks like this:
Traceback (most recent call last):
File "1.py", line 11, in <module>
foo()
File "1.py", line 9, in foo
boo()
File "thread_timeout/__init__.py", line 116, in inner_worker
result = wrapped(*args, **kwargs)
File "1.py", line 6, in boo
file('/fdf','r')
If I remove decorator it looks like this:
Traceback (most recent call last):
File "1.py", line 11, in <module>
foo()
File "1.py", line 9, in foo
boo()
File "1.py", line 6, in boo
file('/fdf','r')
IOError: [Errno 2] No such file or directory: '/fdf'
Question: should I try to hide decorator from the trace or not?
Library code is available here: https://github.com/amarao/thread_timeout/blob/master/thread_timeout/init.py
Thanks.

Raise an exception with traceback starting from caller

I'm trying to make an automated test framework for a side-project and could use some help creating the assertion checks.
Running this in python...
assert(False)
Gives you this...
Traceback (most recent call last):
File "test.py", line 1, in <module>
assert(False)
AssertionError
As you can see the traceback lowest level is assert(False). So I made my custom assert that prints when the assert succeeds.
def custom_assert(condition):
if condition:
print("Yay! It werks!")
else:
raise Exception("Nay, it don't werks...")
custom_assert(False)
But instead of what assert gives, custom_assert gives me this.
Traceback (most recent call last):
File "test.py", line 14, in <module>
custom_assert(False)
File "test.py", line 12, in custom_assert
raise Exception("Nay, it don't werks...")
Exception: Nay, it don't werks...
Which is of course the default behavior. Perfectly useful 99.9999% of the time, but this is that one time it could be improved. It's not useful to know that the method I called to raise an error when the condition is false raised the error.
How can I make my custom_assert raise an exception with a traceback starting from the caller, the same way assert does?
P.S.: I don't want to print it, I want the exception to have properly modified traceback so it works properly with debuggers and other tools too!
Edit
To clarify, the traceback I want would be like this.
Traceback (most recent call last):
File "test.py", line 14, in <module>
custom_assert(False)
Exception: Nay, it don't werks...

Essentially what you want to do is something similar to this:
tb = None
try:
raise Exception('foo')
except Exception:
tb = sys.exc_info()[2]
tb.tb_frame = tb.tb_frame.f_back # This line doesn't work
raise Exception('Nay it doesnt werks').with_traceback(tb)
but you can't assign tb_frame, and from mucking around in the CPython code, this is C-generated data structures (not python) (see sys._getframe())
So your only option left is to mock the entire machinery and then convince python to use your stack. This looks like what jinja2 is doing. If that's what you choose to do, good luck! (It's out of my scope at that point)

Python: eliminating stack traces into library code?

When I get a runtime exception from the standard library, it's almost always a problem in my code and not in the library code. Is there a way to truncate the exception stack trace so that it doesn't show the guts of the library package?
For example, I would like to get this:
Traceback (most recent call last):
File "./lmd3-mkhead.py", line 71, in <module>
main()
File "./lmd3-mkhead.py", line 66, in main
create()
File "./lmd3-mkhead.py", line 41, in create
headver1[depotFile]=rev
TypeError: Data values must be of type string or None.
and not this:
Traceback (most recent call last):
File "./lmd3-mkhead.py", line 71, in <module>
main()
File "./lmd3-mkhead.py", line 66, in main
create()
File "./lmd3-mkhead.py", line 41, in create
headver1[depotFile]=rev
File "/usr/anim/modsquad/oses/fc11/lib/python2.6/bsddb/__init__.py", line 276, in __setitem__
_DeadlockWrap(wrapF) # self.db[key] = value
File "/usr/anim/modsquad/oses/fc11/lib/python2.6/bsddb/dbutils.py", line 68, in DeadlockWrap
return function(*_args, **_kwargs)
File "/usr/anim/modsquad/oses/fc11/lib/python2.6/bsddb/__init__.py", line 275, in wrapF
self.db[key] = value
TypeError: Data values must be of type string or None.
update: added an answer with the code, thanks to the pointer from Alex.

The traceback module in Python's standard library lets you emit error tracebacks in a way that accords to your liking, while an exception is propagating. You can use this power either in the except leg of a try/except statement, or in a function you've installed as sys.excepthook, which gets called if and when an exception propagates all the way; quoting the docs:
In an interactive session this happens
just before control is returned to the
prompt; in a Python program this
happens just before the program exits.
The handling of such top-level
exceptions can be customized by
assigning another three-argument
function to sys.excepthook.
Here's a simple, artificial example:
>>> import sys
>>> import traceback
>>> def f(n):
... if n<=0: raise ZeroDivisionError
... f(n-1)
...
>>> def excepthook(type, value, tb):
... traceback.print_exception(type, value, tb, 3)
...
>>> sys.excepthook = excepthook
>>> f(8)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in f
File "<stdin>", line 3, in f
ZeroDivisionError
as you see, without needing a try/except, you can easily limit the traceback to (for example) the first three levels -- even though we know by design that there were 9 nested levels when the exception was raised.
You want something more sophisticated than a simple limit on levels, so you'll need to call traceback.format_exception, which gives you a list of lines rather than printing it, then "prune" from that list the lines that are about modules you never want to see in your tracebacks, and finally emit the remaining lines (typically to sys.stderr, but, whatever!-).

Thanks to the pointer from Alex, here's teh codez:
def trimmedexceptions(type, value, tb, pylibdir=None, lev=None):
"""trim system packages from the exception printout"""
if pylibdir is None:
import traceback, distutils.sysconfig
pylibdir = distutils.sysconfig.get_python_lib(1,1)
nlev = trimmedexceptions(type, value, tb, pylibdir, 0)
traceback.print_exception(type, value, tb, nlev)
else:
fn = tb.tb_frame.f_code.co_filename
if tb.tb_next is None or fn.startswith(pylibdir):
return lev
else:
return trimmedexceptions(type, value, tb.tb_next, pylibdir, lev+1)
import sys
sys.excepthook=trimmedexceptions
# --- test code ---
def f1(): f2()
def f2(): f3()
def f3():
import xmlrpclib
proxy = xmlrpclib.ServerProxy('http://nosuchserver')
proxy.f()
f1()
Which yields this stack trace:
Traceback (most recent call last):
File "./tsttraceback.py", line 47, in <module>
f1()
File "./tsttraceback.py", line 40, in f1
def f1(): f2()
File "./tsttraceback.py", line 41, in f2
def f2(): f3()
File "./tsttraceback.py", line 45, in f3
proxy.f()
gaierror: [Errno -2] Name or service not known

The Traceback library is probably what you want. Here's one example that might help:
import traceback
try:
your_main()
except:
lines = traceback.format_exc()
print lines[:lines.find('File "/usr')]
(This obviously won't work if there's an exception outside the library, and might not exactly fit your needs, but it's one way of using the traceback library)

Put an unqualified try...except at the top of your code (ie: in your "main") or set sys.excepthook. You can then format the stack trace however you'd like.