I am just getting started with Python/Jython and the SAX parser (xml.sax). I wrote a simple content handler as a test.
from __future__ import with_statement
from xml.sax import make_parser, handler
from xml.sax.handler import ContentHandler
class CountingHandler(ContentHandler):
def __init__(self):
self.counter = 0
def startElement(self, name, attrs):
self.counter += 1
def main(argv=sys.argv):
parser = make_parser()
h = CountingHandler()
parser.setContentHandler(h)
with open(argv[1], "r") as input:
parser.parse(input)
When I run this on some documents (not all), I get an error:
Traceback (most recent call last):
File "src/sciencenetworks/xmltools.py", line 93, in <module>
sys.exit(main())
File "src/sciencenetworks/xmltools.py", line 88, in main
parser.parse(input)
File "/amd.home/home/staudt/workspace/jython/Lib/xml/sax/drivers2/drv_javasax.py", line 141, in parse
self._parser.parse(JyInputSourceWrapper(source))
File "/amd.home/home/staudt/workspace/jython/Lib/xml/sax/drivers2/drv_javasax.py", line 90, in resolveEntity
return JyInputSourceWrapper(self._resolver.resolveEntity(pubId, sysId))
File "/amd.home/home/staudt/workspace/jython/Lib/xml/sax/drivers2/drv_javasax.py", line 75, in __init__
if source.getByteStream():
AttributeError: 'unicode' object has no attribute 'getByteStream'
When I look into the source code of drv_javasax.py, it seems like input is not recognized as a file like object, which it is.
Any ideas on how to fix this?
I think it's this bug: http://bugs.jython.com/issue1488. Fixed in Jython 2.5.2-b1: http://www.jython.org/latest.html
When you insert print type(input) after your with statement, what do you see?
When you revert to old-style "try/finally" code instead of "with", does it work for all files?
What is different between files that work and files that don't work?
What happens if you change the name input to something that doesn't shadow a built-in function?
Related
I am getting an error when trying to run a python script in Linux. I am trying to run a Firewall configuration converter I have dowloaded from here:
https://github.com/glennake/DirectFire_Converter
(converter.py file)
First I was getting an error related to logger, but after running a pip3 install logger it has been fixed. Now I am getting NameError: name 'logging' is not defined, despite of code looking good as far I know. For some reason, it looks like this line is not working:
logger = logging.getLogger(__name__)
But I am seeing that all modules are being imported properly. Any idea about what is bringing the issue? Thanks.
full error traceback:
Traceback (most recent call last):
File "converter1.py", line 257, in <module>
main(src_format=args.source, dst_format=args.destination, routing_info=args.routing)
File "/home/ubuntu/.local/lib/python3.8/site-packages/traceback_with_variables/print.py", line 98, in wrapper
return func(*args, **kwargs)
File "converter1.py", line 233, in main
parsed_data = parse(
File "converter1.py", line 107, in parse
from DirectFire.Converter.parsers.ciscoasa_pre83 import parse
File "/home/ubuntu/DirectFire_Converter/DirectFire/Converter/parsers/ciscoasa_pre83.py", line 23, in <module>
logger = logging.getLogger(__name__)
NameError: name 'logging' is not defined
You need to import, add import logging at the top of your file
I was going through this question : How do I return the definition of a class in python?
But I am unable to display the class definition. I am getting the below error:
>>> class A:
... pass
...
>>> import inspect
>>> source_text = inspect.getsource(A)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\**\Python\Python36\lib\inspect.py", line 968, in getsource
lines, lnum = getsourcelines(object)
File "C:\Users\**\Python\Python36\lib\inspect.py", line 955, in getsourcelines
lines, lnum = findsource(object)
File "C:\Users\**\Python\Python36\lib\inspect.py", line 768, in findsource
file = getsourcefile(object)
File "C:\Users\**\Python\Python36\lib\inspect.py", line 684, in getsourcefile
filename = getfile(object)
File "C:\Users\**\Python\Python36\lib\inspect.py", line 654, in getfile
raise TypeError('{!r} is a built-in class'.format(object))
TypeError: <module '__main__' (<_frozen_importlib_external.SourceFileLoader object at 0x0000026A79293F60>)> is a built-in class
>>>
Can someone please advise what am I doing wrong here? Thanks.
The inspect.getsource() function only works if there is a text file available to load the source code.
You typed the definition of the class into the interactive interpreter, which doesn't keep the original source around when compiling that source into in-memory class and code objects.
Put your class definition into a module, import the module, and then use inspect.getsource().
inspect.getsource() works by first finding the module for a given object (for classes, by looking at the ClassObj.__module__ attribute for the module name, then getting the module via sys.modules[modulename]) then seeing if the module has a __file__ attribute from which a readable source file can be determined. If there is such a filename and it can be read, then the inspect module reads that file to then search for the class ClassName: line and giving you all lines from that point on with same or deeper indentation. The interactive interpreter executes everything in the __main__ module and there is no __file__ attribute for the interpreter, so any attempts at loading source code for objects defined there will simply fail.
If you just wanted to know what members the class defines, use dir() or help() on the object instead. You don't need to see the full source code for that information.
I am struggling trying to understand how to write tests in Flask.
I've inherited an app that already has a bunch of tests that hit routes like /login and test that the response is what's expected.
I have a substantially more complicated situation. I need to test a route/method that depending on the circumstances, hits an external api, figures out whether a path exists in the container the app itself is running in, starts a process that takes 10+ minutes to run on a different machine -- all manner of things. So I can't just hit the route and see if I got what I wanted; I need mocking and patching to mimic the effects of various external world states.
Right now I have a route defined like so in brain_db/views.py:
#app.route('/label_view/<int:scan_number>')
#login_required
def label_view(scan_number):
<so much complicated logic>
The first route defined in that same file, brain_db/views.py, is
#app.route('/surface_test')
def surface_test():
<some code>
Here is a simplified version of the file that's throwing the error:
import unittest
from mock import MagicMock, patch
from flask_brain_db.test_helpers import set_up, tear_down
from flask_brain_db.brain_db.models import Scan
from brain_db.views import label_view
class BrainDBTest(unittest.TestCase):
def setUp(self):
app, db = set_up()
scan = Scan(1, '000001', '000001_MR1', 'scan.nii.gz', scan_number=1)
db.session.add(scan)
scan = Scan.query.filter(Scan.scan_number == 1).first()
db.session.commit()
def tearDown(self):
tear_down()
def mock_volume_views_setup(self)
scan = Scan.query.filter(Scan.scan_number == 1).first()
container_file_path = '/path/to/file/in/container'
return scan, container_file_path
def mock_os_path_exists(self, arg):
return True
#patch('brain_db_helpers.volume_views_setup', mock_volume_views_setup)
#patch('os.path.exists', mock_os_path_exists)
def test_label_view(self):
rv = label_view(1)
assert(True) # I'll actually write tests when I figure out that I can!
print rv
Here is the error:
======================================================================
ERROR: brain_db.tests.test (unittest.loader.ModuleImportFailure)
----------------------------------------------------------------------
ImportError: Failed to import test module: brain_db.tests.test
Traceback (most recent call last):
File "/usr/local/lib/python2.7/unittest/loader.py", line 254, in _find_tests
module = self._get_module_from_name(name)
File "/usr/local/lib/python2.7/unittest/loader.py", line 232, in _get_module_from_name
__import__(name)
File "/usr/src/app/flask_brain_db/brain_db/tests/test.py", line 7, in <module>
from brain_db.views import label_view
File "/usr/src/app/flask_brain_db/brain_db/views.py", line 36, in <module>
#app.route('/surface_test')
File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1250, in decorator
self.add_url_rule(rule, endpoint, f, **options)
File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 66, in wrapper_func
return f(self, *args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1221, in add_url_rule
'existing endpoint function: %s' % endpoint)
AssertionError: View function mapping is overwriting an existing endpoint function: surface_test
What I have done to try to solve my problem: I've read a bunch of the posts on SO that quote that same AssertionError. E.g. 1, 2. I can see that the general shape of the problem is that my routes have already been defined, and
from brain_db.views import label_view
is executing the views module again, thus redefining the routes, so I get an error thrown.
What I don't understand is how exactly I should avoid this. I need to be able to import a method into another file to be able to test it. Are all the routes supposed to be wrapped in if __name__ == main? I am brand new to Flask development and haven't yet seen example code where this is the case; I'm dubious that this is the correct solution; it's just the only thing that's offered when you try to search for preventing code from being executed on import.
The way I'm running my tests right now is via the file manage.py in the top level of my application. It contains the following method:
#manager.command
def test():
"""Runs the tests without coverage"""
tests = unittest.TestLoader().discover(start_dir='.', pattern='test*.py')
res = unittest.TextTestRunner(verbosity=2).run(tests)
sys.exit(not res.wasSuccessful())
I run python manage.py test at the command line.
It also might be relevant that while I've put the test that's failing in a submodule within brain_db, several tests run before it that hit routes defined in the app and test for the expected result. However, commenting out those tests has no effect on the way my test is failing.
Finally, I was initially getting an error at the line from flask_brain_db.brain_db.models import Scan:
ERROR: brain_db.tests.test (unittest.loader.ModuleImportFailure)
----------------------------------------------------------------------
ImportError: Failed to import test module: brain_db.tests.test
Traceback (most recent call last):
File "/usr/local/lib/python2.7/unittest/loader.py", line 254, in _find_tests
module = self._get_module_from_name(name)
File "/usr/local/lib/python2.7/unittest/loader.py", line 232, in _get_module_from_name
__import__(name)
File "/usr/src/app/flask_brain_db/brain_db/tests/test.py", line 5, in <module>
from flask_brain_db.brain_db.models import Scan
File "/usr/src/app/flask_brain_db/brain_db/models.py", line 6, in <module>
class Scan(db.Model):
File "/usr/local/lib/python2.7/site-packages/flask_sqlalchemy/model.py", line 67, in __init__
super(NameMetaMixin, cls).__init__(name, bases, d)
File "/usr/local/lib/python2.7/site-packages/flask_sqlalchemy/model.py", line 121, in __init__
super(BindMetaMixin, cls).__init__(name, bases, d)
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/ext/declarative/api.py", line 65, in __init__
_as_declarative(cls, classname, cls.__dict__)
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/ext/declarative/base.py", line 116, in _as_declarative
_MapperConfig.setup_mapping(cls, classname, dict_)
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/ext/declarative/base.py", line 144, in setup_mapping
cfg_cls(cls_, classname, dict_)
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/ext/declarative/base.py", line 172, in __init__
self._setup_table()
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/ext/declarative/base.py", line 465, in _setup_table
**table_kw)
File "/usr/local/lib/python2.7/site-packages/flask_sqlalchemy/model.py", line 90, in __table_cls__
return sa.Table(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/sqlalchemy/sql/schema.py", line 439, in __new__
"existing Table object." % key)
InvalidRequestError: Table 'scan' is already defined for this MetaData instance. Specify 'extend_existing=True' to redefine options and columns on an existing Table object.
I made it go away by including
__table_args__ = {'extend_existing': True}
In the model definition, but I don't know if I should have done that and I suspect I was just postponing the same problem I have now. It seems like the fundamental problem is that I don't know how to write tests without redefining a bunch of things that have been already been defined.
What is the correct way to approach this? Please let me know if I need to provide any other information.
You should be able to access your all view functions from your app object. Try removing the line "from brain_db.views import label_view", and instead define your label_view method after running set_up() using the below:
label_view = app.view_functions["label_view"]
How to get module source code by a python 'object' of that module?
class TestClass(object):
def __init__(self):
pass
def testMethod(self):
print 'abc'
return 'abc'
It's well-known that
print inspect.getsource(TestClass)
can be used to get source code of 'TestClass'.
However, the result of
ob = TestClass()
print inspect.getsource(ob)
as below, is not as expected.
Traceback (most recent call last):
File "D:\Workspaces\WS1\SomeProject\src\python\utils\ModuleUtils.py", line 154, in <module>
print inspect.getsource(ob)
File "C:\SciSoft\WinPython-64bit-2.7.10.3\python-2.7.10.amd64\lib\inspect.py", line 701, in getsource
lines, lnum = getsourcelines(object)
File "C:\SciSoft\WinPython-64bit-2.7.10.3\python-2.7.10.amd64\lib\inspect.py", line 690, in getsourcelines
lines, lnum = findsource(object)
File "C:\SciSoft\WinPython-64bit-2.7.10.3\python-2.7.10.amd64\lib\inspect.py", line 526, in findsource
file = getfile(object)
File "C:\SciSoft\WinPython-64bit-2.7.10.3\python-2.7.10.amd64\lib\inspect.py", line 420, in getfile
'function, traceback, frame, or code object'.format(object))
TypeError: <utils.TestClass.TestClass object at 0x0000000003C337B8> is not a module, class, method, function, traceback, frame, or code object
The question is:
If there is an object like 'ob' above, how to check the module source code of ob, or 'TestClass', via a method that takes 'ob' itself as a parameter?
In short, implement the following module
def getSource(obj):
###returns the result which is exactly identical to inspect.getsource(TestClass)
ob = TestClass()
###prints the result which is exactly identical to inspect.getsource(TestClass)
print getSource(ob)
(The scenario of a method like this is much more common than inspect.getsource(); for example, check the source code of an unknown, unpickled object.)
Instances don't have source code.
Use:
print inspect.getsource(type(ob))
or:
print inspect.getsource(ob.__class__)
I get the following error when our program is run on Windows XP using ActivePerl 2.6 (although it runs fine on linux). Any ideas of what is going on? Is this a bug in the module or a problem with our code?
Error log:
File "C:\Python26\lib\site-packages\Scientific\IO\PDB.py", line 1163, in __ini
t__
self.parseFile(PDBFile(file_or_filename))
File "C:\Python26\lib\site-packages\Scientific\IO\PDB.py", line 1371, in parse
File
type, data = file.readLine()
File "C:\Python26\lib\site-packages\Scientific\IO\PDB.py", line 200, in readLi
ne
line = self.file.readline()
AttributeError: 'unicode' object has no attribute 'readline'
Exception AttributeError: "'unicode' object has no attribute 'close'" in <bound
method PDBFile.__del__ of <Scientific.IO.PDB.PDBFile instance at 0x00FD0440>> ig
nored
UPDATE: Following Joe's suggestion below, I get the following error instead:
Traceback (most recent call last):
File "NewParseV1_00.py", line 47, in <module>
PromptUser()
File "NewParseV1_00.py", line 45, in PromptUser
parseGroupFile(grpfilepath, outputPPYfilepath, sorcePDBfilepath)
File "NewParseV1_00.py", line 39, in parseGroupFile
return GeneratePPY(LL,GRPNAME,SRCPDB,OUTPPY)
File "NewParseV1_00.py", line 10, in __init__
self.PDBStruct = Scientific.IO.PDB.Structure(SRCPDB)
File "C:\Python26\lib\site-packages\Scientific\IO\PDB.py", line 1163, in __ini
t__
self.parseFile(PDBFile(file_or_filename))
File "C:\Python26\lib\site-packages\Scientific\IO\PDB.py", line 161, in __init
__
if isinstance(file_or_filename, basestr):
NameError: global name 'basestr' is not defined
Exception AttributeError: "PDBFile instance has no attribute 'open'" in <bound m
ethod PDBFile.__del__ of <Scientific.IO.PDB.PDBFile instance at 0x00FDB418>> ign
ored
I'm guessing, but it looks like a bug in Scientific.IO.PDB where they do something like:
if type(file_or_filename) == str:
file = open(file_or_filename)
else:
file = file_or_filename
rather than doing:
if isinstance(file_or_filename, basestr):
...
For whatever reason, you appear to be passing in a unicode filename on the windows side, and a raw string on the linux side.
Again, though, I'm just taking a guess. I'm assuming Scientific here is this module? http://dirac.cnrs-orleans.fr/plone/software/scientificpython/ (I'm not familiar with it...)
Edit: Yep!
Here's the relevant code from Scientific.IO.PDB:
class PDBFile:
"""
X{PDB} file with access at the record level
The low-level file access is handled by the module
L{Scientific.IO.TextFile}, therefore compressed files and URLs
(for reading) can be used as well.
"""
def __init__(self, file_or_filename, mode = 'r', subformat = None):
"""
#param file_or_filename: the name of the PDB file, or a file object
#type file_or_filename: C{str} or C{file}
#param mode: the file access mode, 'r' (read) or 'w' (write)
#type mode: C{str}
#param subformat: indicates a specific dialect of the PDB format.
Subformats are defined in
L{Scientific.IO.PDBExportFilters}; they are used
only when writing.
#type subformat: C{str} or C{NoneType}
"""
if isinstance(file_or_filename, str):
self.file = TextFile(file_or_filename, mode)
else:
self.file = file_or_filename
<snip>
It should be as simple as changing isinstance(file_or_filename, str) to isinstance(file_or_filename, basestr). You might want to file a bug report...
Changing the code in Scientific.IO.PDB as explained by Joe needs one final modification to work:
On line 161, change if isinstance(file_or_filename, str): to if isinstance(file_or_filename, basestring):
(Note that Joe wrote that the second argument should be basestr but that gives the error in the update in the original question.)