I'm trying to use pycparser to parse this C code:
https://github.com/nbeaver/mx-trunk/blob/0b80678773582babcd56fe959d5cfbb776cc0004/libMx/d_adsc_two_theta.c
A repo with a minimal example and Makefile is here:
https://github.com/nbeaver/pycparser-problem
Using pycparser v2.14 (from pip) and gcc 4.9.2 on Debian Jessie.
Things I have tried:
Pass the -nostdinc flag to gcc and including the fake_libc_include folder.
Use -D'__attribute__(x)=' to take out GCC extensions
Use fake headers for e.g. <sys/param.h>
Use the -std=c99 in case the code is not C99 compatible.
Reproduce the redis example in case there is something weird with my machine.
This is what the traceback looks like:
Traceback (most recent call last):
File "just_parse.py", line 21, in <module>
parse(path)
File "just_parse.py", line 9, in parse
ast = pycparser.parse_file(filename)
File "/home/nathaniel/.local/lib/python2.7/site-packages/pycparser/__init__.py", line 93, in parse_file
return parser.parse(text, filename)
File "/home/nathaniel/.local/lib/python2.7/site-packages/pycparser/c_parser.py", line 146, in parse
debug=debuglevel)
File "/home/nathaniel/.local/lib/python2.7/site-packages/pycparser/ply/yacc.py", line 265, in parse
return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc)
File "/home/nathaniel/.local/lib/python2.7/site-packages/pycparser/ply/yacc.py", line 1047, in parseopt_notrack
tok = self.errorfunc(errtoken)
File "/home/nathaniel/.local/lib/python2.7/site-packages/pycparser/c_parser.py", line 1680, in p_error
column=self.clex.find_tok_column(p)))
File "/home/nathaniel/.local/lib/python2.7/site-packages/pycparser/plyparser.py", line 55, in _parse_error
raise ParseError("%s: %s" % (coord, msg))
pycparser.plyparser.ParseError: in/d_adsc_two_theta.c:63:82: before: .
The traceback points to this line:
https://github.com/nbeaver/mx-trunk/blob/0b80678773582babcd56fe959d5cfbb776cc0004/libMx/d_adsc_two_theta.c#L63
Which in turn points to this #define macro:
https://github.com/nbeaver/mx-trunk/blob/0b80678773582babcd56fe959d5cfbb776cc0004/libMx/mx_motor.h#L484
The cause appears to be the offsetof() function. The minimal working examples are fixed by recent commits, however:
https://github.com/eliben/pycparser/issues/87
Related
I am trying to creat AST usinfg pyCparser,
The following error printed:
Traceback (most recent call last):
File "C:\Work\RE\Tools\VarsExporter\BuildExportedDb.py", line 1076, in
main()
File "C:\Work\RE\Tools\VarsExporter\BuildExportedDb.py", line 1032, in main
ast = parse_file(i_file)
File "C:\Python27\lib\site-packages\pycparser_init_.py", line 93, in
parse_file
return parser.parse(text, filename)
File "C:\Python27\lib\site-packages\pycparser\c_parser.py", line 152, in
parse
debug=debuglevel)
File "C:\Python27\lib\site-packages\pycparser\ply\yacc.py", line 331, in
parse
return self.parseopt_notrack(input, lexer, debug, tracking, tokenfunc)
File "C:\Python27\lib\site-packages\pycparser\ply\yacc.py", line 1199, in
parseopt_notrack
tok = call_errorfunc(self.errorfunc, errtoken, self)
File "C:\Python27\lib\site-packages\pycparser\ply\yacc.py", line 193, in
call_errorfunc
r = errorfunc(token)
File "C:\Python27\lib\site-packages\pycparser\c_parser.py", line 1761, in
p_error
column=self.clex.find_tok_column(p)))
File "C:\Python27\lib\site-packages\pycparser\plyparser.py", line 67, in _
parse_error
raise ParseError("%s: %s" % (coord, msg))
ParseError: Objectffly\SerDb.i:43:18: before: __loff_t
What causes the above issue? How Can I handle it?
Any suggestions how can I debug it, and find out what's going on?
From pyCparser git FAQ:
C code almost always #includes various header files from the standard C library, like stdio.h. While (with some effort) pycparser can be made to parse the standard headers from any C compiler, it's much simpler to use the provided "fake" standard includes in utils/fake_libc_include. These are standard C header files that contain only the bare necessities to allow valid parsing of the files that use them.
To solve the issue I have successfully used the method described here.
I am trying to convert a JaCoCo coverage report to Cobertura format (since Shippable only supports Cobertura). This guy claims to have a tool to convert JaCoCo to Cobertura, however when running his script I get the following error:
Traceback (most recent call last):
File "cover2cover.py", line 151, in <module>
jacoco2cobertura(filename, source_root)
File "cover2cover.py", line 139, in jacoco2cobertura
convert_root(root, into, source_root)
File "cover2cover.py", line 127, in convert_root
packages.append(convert_package(package))
File "cover2cover.py", line 113, in convert_package
c_classes.append(convert_class(j_class, j_package))
File "cover2cover.py", line 100, in convert_class
c_methods.append(convert_method(j_method, j_method_lines))
File "cover2cover.py", line 85, in convert_method
convert_lines(j_lines, c_method)
File "cover2cover.py", line 33, in convert_lines
for jline in j_lines:
File "cover2cover.py", line 23, in method_lines
larger = list(int(jm.attrib['line']) for jm in jmethods if int(jm.attrib['line']) > start_line)
File "cover2cover.py", line 23, in <genexpr>
larger = list(int(jm.attrib['line']) for jm in jmethods if int(jm.attrib['line']) > start_line)
KeyError: 'line'
I know nothing about python, so any help would be appreciated.
I don't know python either, but I know that python 2 and python 3 have significant differences. Perhaps you ran into that?
I was able to run the script ok with this version:
$> python --version
Python 2.7.11
To ensure I got the script without any download or browser or line-ending type issues, I did clone the git repo:
$> git clone https://github.com/rix0rrr/cover2cover.git
Then the script ran first try on my jacoco XML file.
I have an issue when I am trying to decompile a .pyc file.
The traceback is the following:
Traceback (most recent call last):
File "my.py", line 4, in <module>
uncompyle2.uncompyle_file("/home/user/Downloads/asd.pyc", fileobj)
File "/home/user/Desktop/uncompyle2/uncompyle2/__init__.py", line 130, in uncompyle_file
uncompyle(version, co, outstream, showasm, showast, deob)
File "/home/user/Desktop/uncompyle2/uncompyle2/__init__.py", line 93, in uncompyle
tokens, customize = scanner.disassemble(co, deob=deob)
File "/home/user/Desktop/uncompyle2/uncompyle2/Scanner.py", line 214, in disassemble
cf = self.find_jump_targets(code)
File "/home/user/Desktop/uncompyle2/uncompyle2/Scanner.py", line 926, in find_jump_targets
self.detect_structure(i, op)
File "/home/user/Desktop/uncompyle2/uncompyle2/Scanner.py", line 737, in detect_structure
if int(self.code[jmp]) == RETURN_VALUE:
**TypeError: array indices must be integers**
Any ideas about this ?
I'm using Python 2.7.6 on a Ubuntu machine.
The command that I'm running to have this work is the following:
uncompyle2 asd.pyc
//EDIT: As far as I can tell, this happens only on a specific file(asd.py). It works on other files.
Any workaround ?
The .pyc file that you're trying to decompile is probably obfuscated. It's not uncompyle2s job to also deofuscate the file.
Try something else like pyc2py. Maybe it will work.
According to PEP440, it seems the following format below is a valid version string:
X.Y.postN # Post-release
But when I use something like: 1.0.post1, I get the following exception in pbr.
(I'm using the pbr version 0.11.0)
ERROR:root:Error parsing
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/pbr/core.py", line 104, in pbr
attrs = util.cfg_to_args(path)
File "/usr/local/lib/python2.7/dist-packages/pbr/util.py", line 238, in cfg_to_args
pbr.hooks.setup_hook(config)
File "/usr/local/lib/python2.7/dist-packages/pbr/hooks/__init__.py", line 27, in setup_hook
metadata_config.run()
File "/usr/local/lib/python2.7/dist-packages/pbr/hooks/base.py", line 29, in run
self.hook()
File "/usr/local/lib/python2.7/dist-packages/pbr/hooks/metadata.py", line 28, in hook
self.config['name'], self.config.get('version', None))
File "/usr/local/lib/python2.7/dist-packages/pbr/packaging.py", line 1009, in get_version
version = _get_version_from_git(pre_version)
File "/usr/local/lib/python2.7/dist-packages/pbr/packaging.py", line 947, in _get_version_from_gitpre_version)
File "/usr/local/lib/python2.7/dist-packages/pbr/version.py", line 236, in from_pip_string
% (remainder, version_string))
ValueError: Unknown remainder ['post1'] in '1.0.post1'
error in setup command: Error parsing /home/admin/mypackage/setup.cfg:
ValueError: Unknown remainder ['post1'] in '1.0.post1'
Anybody knows why that format is not supported in PBR?
Ref the docs: http://docs.openstack.org/developer/pbr/#version
The versions should be compatible with "Linux Compatible Semantic Versioning 3.0.0": http://docs.openstack.org/developer/pbr/semver.html
I am trouble getting the Maltparser to work with Python NLTK.
Here is my code so far:
import nltk
os.environ["MALT_PARSER"] = "C:/Python34/maltparser-1.8.1"
os.environ["MALTPARSERHOME"] = "C:/Python34/maltparser-1.8.1"
parser8 = nltk.parse.malt.MaltParser(
... working_dir="C:/Python34/maltparser-1.8.1", mco="engmalt.poly-1.7",
... additional_java_args=['-Xmx512m'])
txt = "This is a test sentence"
parser8.raw_parse(txt)
I have downloaded and selected to used a pre-trained model.
This is the response I get:
runfile('C:/Anaconda/Lib/site-packages/nltk/malt2.py', wdir='C:/Anaconda/Lib/site-packages/nltk')
Traceback (most recent call last):
File "<ipython-input-38-73069e4ee673>", line 1, in <module>
runfile('C:/Anaconda/Lib/site-packages/nltk/malt2.py', wdir='C:/Anaconda/Lib/site-packages/nltk')
File "C:\Anaconda\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 580, in runfile
execfile(filename, namespace)
File "C:/Anaconda/Lib/site-packages/nltk/malt2.py", line 14, in <module>
parser8.raw_parse(txt)
File "C:\Anaconda\lib\site-packages\nltk\parse\malt.py", line 139, in raw_parse
return self.parse(words, verbose)
File "C:\Anaconda\lib\site-packages\nltk\parse\malt.py", line 126, in parse
return self.parse_sents([sentence], verbose)[0]
File "C:\Anaconda\lib\site-packages\nltk\parse\malt.py", line 114, in parse_sents
return self.tagged_parse_sents(tagged_sentences, verbose)
File "C:\Anaconda\lib\site-packages\nltk\parse\malt.py", line 194, in tagged_parse_sents
"code %d" % (' '.join(cmd), ret))
Exception: MaltParser parsing (java -Xmx512m -jar C:/Python34/maltparser-1.8.1\malt.jar -w C:/Python34/maltparser-1.8.1 -c engmalt.poly-1.7.mco -i C:\Python34\maltparser-1.8.1\malt_input.conllqgpbye -o C:\Python34\maltparser-1.8.1\malt_output.conllib1nx0 -m parse) failed with exit code 2
I have followed all the advice on this post How to use malt parser in python nltk.
Specifically:
-I downloaded the latest version of MaltParser.
-Using Pip, I uninstalled and re installed NLTK to get the latest version, which includes the addition in malt/py that allows 'additional_java_args' to be added as a parameter.
-I renamed the jar file to 'malt.jar'.
-I set an environmental variable pointing both MALT_PARSER and MALTPARSERHOME to the working directory.
-I've tried both the linear and poly pre-trained models.
-The code for malt.py can be found here http://www.nltk.org/_modules/nltk/parse/malt.html
If there isn't an apparent solution, how can I continue to debug this myself?
It seems that there's some slash (/ ) inconsistency with the exception raised. Nothing I do seems to fix it though.