I'm having issues getting going with a Python target in ANTLR4. There seems to be very few examples available and going to the corresponding Java code doesn't seem relevant.
I'm using the standard Hello.g4 grammar:
// Define a grammar called Hello
grammar Hello;
r : 'hello' ID ; // match keyword hello followed by an identifier
ID : [a-z]+ ; // match lower-case identifiers
WS : [ \t\r\n]+ -> skip ; // skip spaces, tabs, newlines
The example (built from the standard Hello.g4 example):
input_ = antlr4.FileStream(_FILENAME)
lexer = HelloLexer.HelloLexer(input_)
stream = antlr4.CommonTokenStream(lexer)
parser = HelloParser.HelloParser(stream)
rule_name = 'r'
tree = getattr(parser, rule_name)()
I also wrote a listener. To assert/verify that this is correct, I'll repeat it here:
class HelloListener(antlr4.ParseTreeListener):
def enterR(self, ctx):
print("enterR")
def exitR(self, ctx):
print("exitR")
def enterId(self, ctx):
print("enterId")
def exitId(self, ctx):
print("exitId")
So, first, I can't guarantee that the string I'm giving it is valid because I'm not getting any screen output. How do I tell from the tree object if anything was matched? How do I extract the matching rules/tokens?
A Python example would be great, if possible.
I hear you, having the same issues right now. The Python documentation for v4 is useless and v3 differs to much to be usable. I'm thinking about switching back to Java to implement my stuff.
Regarding your code: I think your own custom listener has to inherit from the generated HelloListener. You can do the printing there.
Also try parsing invalid input to see if the parser starts at all. I'm not sure about the line with getattr(parser, rule_name)() though. I followed the steps in the (unfortunately very short) documentation for the Antlr4 Python target: https://theantlrguy.atlassian.net/wiki/display/ANTLR4/Python+Target
You can also find some documentation about the listener stuff there. Hope it helps.
This question seems to be old, but I also had the same problems and found out how to deal with it. When using strings in python, you have to use the function antlr4.InputStream as pointed out here
So, in the end, you could get a working example with this sort of code (based on Alan's answer and an example from dzone)
from antlr4 import *
from grammar.HelloListener import HelloListener
from grammar.HelloLexer import HelloLexer
from grammar.HelloParser import HelloParser
import sys
class HelloPrintListener(HelloListener):
def enterHi(self, ctx):
print("Hello: %s" % ctx.ID())
def main():
giveMeInput = input ("say hello XXXX\n")
print("giveMeInput is {0}".format(giveMeInput))
# https://www.programcreek.com/python/example/93166/antlr4.InputStream
# https://groups.google.com/forum/#!msg/antlr-discussion/-9VJ5H9NcDs/OukVNCTQCAAJ
i_stream = InputStream(giveMeInput)
lexer = HelloLexer(i_stream)
t_stream = CommonTokenStream(lexer)
parser = HelloParser(t_stream)
tree = parser.hi()
printer = HelloPrintListener()
walker = ParseTreeWalker()
walker.walk(printer, tree)
if __name__ == '__main__':
main()
I've created an example for Python 2 using the Hello grammar.
Here's the relevant code:
from antlr4 import *
from HelloLexer import HelloLexer
from HelloListener import HelloListener
from HelloParser import HelloParser
import sys
class HelloPrintListener(HelloListener):
def enterHi(self, ctx):
print("Hello: %s" % ctx.ID())
def main():
lexer = HelloLexer(StdinStream())
stream = CommonTokenStream(lexer)
parser = HelloParser(stream)
tree = parser.hi()
printer = HelloPrintListener()
walker = ParseTreeWalker()
walker.walk(printer, tree)
if __name__ == '__main__':
main()
As fabs said, the key is to inherit from the generated HelloListener. There seems to be some pickiness on this issue, as you can see if you modify my HelloPrintListener to inherit directly from ANTLR's ParseTreeListener. I imagined that would work since the generated HelloListener just has empty methods, but I saw the same behavior you saw (listener methods never being called).
Even though the documentation for Python listeners are lacking, the available methods are similar to Java.
The antlr documentation has been updated to document the support for python 3 and python 4 targets. The examples from the antlr book converted to python3 can be found here, they are sufficient enough to get anyone started.
Related
I'm using the ANTLRv4 Python3 grammar from here:
https://github.com/antlr/grammars-v4/blob/master/python3/Python3.g4
and running:
java -jar antlr-4.6-complete.jar -Dlanguage=Python2 Python3.g4
to generate Python3Lexer.py + some other files.
However, Python3Lexer.py contains code which is not python! For eg.
def __init__(self, input=None):
super(Python3Lexer, self).__init__(input)
self.checkVersion("4.6")
self._interp = LexerATNSimulator(self, self.atn, self.decisionsToDFA, PredictionContextCache())
self._actions = None
self._predicates = None
// A queue where extra tokens are pushed on (see the NEWLINE lexer rule).
private java.util.LinkedList<Token> tokens = new java.util.LinkedList<>();
// The stack that keeps track of the indentation level.
private java.util.Stack<Integer> indents = new java.util.Stack<>();
Its unusable because of this. Does anyone know why this is happening and how I can fix it? Thanks!
This grammar is full of action code written in Java to deal with specialities of Python. You have to port that manually to python to make the grammar usuable for you. This is why grammar writers are encouraged to move out action code into base classes or listener code.
I'm following this example here to use MS Message Queues with IronPython.
The example works to create a message text string without any properties.
import clr
clr.AddReference('System.Messaging')
from System.Messaging import MessageQueue
ourQueue = '.\\private$\\myqueue'
queue = MessageQueue(ourQueue)
queue.Send('Hello from IronPython')
I am trying to create an empty message and then add properties (like label, a reply queue and a binary message body) and then send that complete message.
How can I do this in IronPython?
The documentation of the message class is here, but obviously has no python sample code. I have never used .net code with python and just installed IronPython to connect to an existing MSMQ environment, so I'm a bit stuck in how to proceed.
Any help?
update
See answer below, I managed to guess the systax to create a message.
The solution seems a bit hacky so I'll leave this open for a few days
I don't think that this works with IronPython classes, because serialize and deserialize them does not work like it does for c#/.net classes.
The only thing to get this work, will be to get IronPython classes serialize-able and deserialize-able. I think deserialization will be the hard part. But you may proof me wrong.
I've started guessing the syntax by examining c# examples and
have hacked a solution to the problem. The following code
delivers a message with a user defined label and response queue and a body message.
import clr
from System import Array
from System import Byte
clr.AddReference('System.Messaging')
from System.Messaging import MessageQueue
from System.Messaging import Message
ourQueue = '.\\private$\python_in'
ourOutQueue = '.\\private$\python_out'
if not MessageQueue.Exists(ourQueue):
queue = MessageQueue.Create(ourQueue)
else:
queue = MessageQueue(ourQueue)
if not MessageQueue.Exists(ourOutQueue):
out_queue = MessageQueue.Create(ourOutQueue)
else:
out_queue = MessageQueue(ourOutQueue)
mymessage = Message()
mymessage.Label = 'MyLabel'
mymessage.ResponseQueue = out_queue
mystring = 'hello world'
mybytearray = bytearray(mystring)
# this is very hacky
mymessage.BodyStream.Write(Array[Byte](mybytearray),0,len(mybytearray))
queue.Send(mymessage)
I am writing an external Python/comtypes script (in PythonWin) that needs to get a reference to the current ArcGIS 10.0 ArcMap session (through the ArcObjects COM). Because the script is outside the application boundary, I am getting the application reference through the AppROT (running object table). The first code snippet below is the main Python driver module. In it is a function GetApp() to grab an application reference from the AppROT. This code works fine and returns IApplication on the singleton AppRef object. Makes sense, and that's what the ArcObjects reference seems to indicate. Now, my main goal is to get to an IMxDocument. In the main loop, I get to an IDocument successfully and can print the title. The next line, though, a Query Interface, throws an error - NameError: name 'esriArcMapUI' is not defined. The error occurs consistently, even after closing PythonWin and reopening (which you always want to try before you conclude that you have a problem). [BTW, the second code snippet is the CType() function for QI, defined in and imported from the SignHelpers.py module.] So, here are my questions:
(1) What COM object is the IDocument on?
(2) How do I get from my IDocument to the intended IMxDocument? Do I need to create a new MxDocument object first? [Sorry. Two questions there.]
(3) If I don't have to create a new object, then how do I do the QI?
I did a lot of ArcObjects work in VB6 quite a few years ago, so explicit QI's and namespaces are putting the screws to me at the moment. Once I can get to an IMxDocument I will be home free. I would appreciate any help anyone can give me with this.
Also, I apologize for the formatting of the code below. I could not figure out how to get Python code to format correctly. Indentation doesn't work, and some of the Python code is interpreted as formatting characters.
=== code: main py module ===
import sys, os
sys.path.append('C:\GISdata\E_drive\AirportData\ATL\Scripts')
import comtypes
from SignHelpers import *
def GetApp(app):
"""Get a hook into the current session of ArcMap; \n\
Execute GetDesktopModules() and GetStandaloneModules() first"""
print "GetApp called" ####
import comtypes.gen.esriFramework as esriFramework
import comtypes.gen.esriArcMapUI as esriArcMapUI
import comtypes.gen.esriCarto as esriCarto
pAppROT = NewObj(esriFramework.AppROT, esriFramework.IAppROT)
iCount = pAppROT.Count
print "appROT count = ", iCount ####
if iCount == 0:
print 'No ArcGIS application currently running. Terminating ...'
return None
for i in range(iCount):
pApp = pAppROT.Item(i) #returns IApplication on AppRef
if pApp.Name == app:
print "Application: ", pApp.Name ####
return pApp
print 'No ArcMap session is running at this time.'
return None
if __name__ == "__main__":
#Wrap needed ArcObjects type libraries (.olb)...
... code omitted ...
GetDesktopModules(dtOLB) #These force comtypes to generate
GetStandaloneModules(saOLB) #the Python wrappers for .olb's
#Connect to ArcMap
pApp = GetApp("ArcMap")
pDoc = pApp.Document #IDocument on current Document object
print pDoc.Title
pMxDoc = CType(pDoc, esriArcMapUI.IMxDocument) #QI to IMxDocument on MxDocument
=== code for CType() ===
def CType(obj, interface):
try:
newobj = obj.QueryInterface(interface)
return newobj
except:
return None
Scoping error (per the comments):
The import comtypes.gen.esriArcMapUI as esriArcMapUI statement needed to define the esriArcMapUI namespace was being run within the GetApp() function, so the namespace was local to the function.
I'm relatively new to programming/python, so I'd appreciate any help I can get. I want to save an excel file as a specific format using Excel through COM. Here is the code:
import win32com.client as win32
def excel():
app = 'Excel'
x1 = win32.gencache.EnsureDispatch('%s.Application' % app)
ss = x1.Workbooks.Add()
sh = ss.ActiveSheet
x1.Visible = True
sh.Cells(1,1).Value = 'test write'
ss.SaveAs(Filename="temp.xls", FileFormat=56)
x1.Application.Quit()
if __name__=='__main__':
excel()
My question is how do I specify the FileFormat if I don't explicitly know the code for it? Browsing through the documentation I find the reference at about a FileFormat object. I'm clueless on how to access the XlFileFormat object and import it in a way that I can find the enumeration value for it.
Thanks!
This question is a bit stale, but for those reaching this page from Google (as I did) my solution was accessing the constants via the win32com.client.constants object instead of on the application object itself as suggested by Eric. This lets you use enum constants just like in the VBE:
>>> import win32com.client
>>> xl = win32com.client.gencache.EnsureDispatch('Excel.Application')
>>> C = win32com.client.constants
>>> C.xlWorkbookNormal
-4143
>>> C.xlCSV
6
>>> C.xlErrValue
2015
>>> C.xlThemeColorAccent1
5
Also, unless you've manually run the makepy utility, the constants may not be available if initializing the application with the regular win32com.client.Dispatch(..) method, which was another issue I was having. Using win32com.client.gencache.EnsureDispatch(..) (as the questioner does) checks for and generates the Python bindings at runtime if required.
I found this ActiveState page to be helpful.
When I used COM to access quickbooks, I could reach the constants defined under a constants member of the object. The code looked something like this (you'll be intersted in the third line):
self._session_manager.OpenConnection2("",
application_name,
QBFC8Lib.constants.ctLocalQBD)
I'm not sure if this will work, but try this:
import win32com.client as win32
def excel():
app = 'Excel'
x1 = win32.gencache.EnsureDispatch('%s.Application' % app)
ss = x1.Workbooks.Add()
sh = ss.ActiveSheet
x1.Visible = True
sh.Cells(1,1).Value = 'test write'
ss.SaveAs(Filename="temp.xls", FileFormat=x1.constants.xlWorkbookNormal)
x1.Application.Quit()
if __name__=='__main__':
excel()
Replace xlWorkbookNormal with whatever format your trying to choose in the X1FileFormat web page you posted in your question.
All of the file format constants are documented here
As a general rule I find it really useful to pre-record any code in the VBA IDE in Excel. This way you can find out all the values of constants etc that you need to use within your python code. You can also make sure stuff will work from within a more controlled environment.
Basically, something similar to System.Xml.XmlWriter - A streaming XML Writer that doesn't incur much of a memory overhead. So that rules out xml.dom and xml.dom.minidom. Suggestions?
I think you'll find XMLGenerator from xml.sax.saxutils is the closest thing to what you want.
import time
from xml.sax.saxutils import XMLGenerator
from xml.sax.xmlreader import AttributesNSImpl
LOG_LEVELS = ['DEBUG', 'WARNING', 'ERROR']
class xml_logger:
def __init__(self, output, encoding):
"""
Set up a logger object, which takes SAX events and outputs
an XML log file
"""
logger = XMLGenerator(output, encoding)
logger.startDocument()
attrs = AttributesNSImpl({}, {})
logger.startElementNS((None, u'log'), u'log', attrs)
self._logger = logger
self._output = output
self._encoding = encoding
return
def write_entry(self, level, msg):
"""
Write a log entry to the logger
level - the level of the entry
msg - the text of the entry. Must be a Unicode object
"""
#Note: in a real application, I would use ISO 8601 for the date
#asctime used here for simplicity
now = time.asctime(time.localtime())
attr_vals = {
(None, u'date'): now,
(None, u'level'): LOG_LEVELS[level],
}
attr_qnames = {
(None, u'date'): u'date',
(None, u'level'): u'level',
}
attrs = AttributesNSImpl(attr_vals, attr_qnames)
self._logger.startElementNS((None, u'entry'), u'entry', attrs)
self._logger.characters(msg)
self._logger.endElementNS((None, u'entry'), u'entry')
return
def close(self):
"""
Clean up the logger object
"""
self._logger.endElementNS((None, u'log'), u'log')
self._logger.endDocument()
return
if __name__ == "__main__":
#Test it out
import sys
xl = xml_logger(sys.stdout, 'utf-8')
xl.write_entry(2, u"Vanilla log entry")
xl.close()
You'll probably want to look at the rest of the article I got that from at http://www.xml.com/pub/a/2003/03/12/py-xml.html.
I think I have your poison :
http://sourceforge.net/projects/xmlite
Cheers
Some years ago I used MarkupWriter from 4suite
General-purpose utility class for generating XML (may eventually be
expanded to produce more output types)
Sample usage:
from Ft.Xml import MarkupWriter
writer = MarkupWriter(indent=u"yes")
writer.startDocument()
writer.startElement(u'xsa')
writer.startElement(u'vendor')
#Element with simple text (#PCDATA) content
writer.simpleElement(u'name', content=u'Centigrade systems')
#Note writer.text(content) still works
writer.simpleElement(u'email', content=u"info#centigrade.bogus")
writer.endElement(u'vendor')
#Element with an attribute
writer.startElement(u'product', attributes={u'id': u"100\u00B0"})
#Note writer.attribute(name, value, namespace=None) still works
writer.simpleElement(u'name', content=u"100\u00B0 Server")
#XML fragment
writer.xmlFragment('<version>1.0</version><last-release>20030401</last-release>')
#Empty element
writer.simpleElement(u'changes')
writer.endElement(u'product')
writer.endElement(u'xsa')
writer.endDocument()
Note on the difference between 4Suite writers and printers
Writer - module that exposes a broad public API for building output
bit by bit
Printer - module that simply takes a DOM and creates output from it
as a whole, within one API invokation
Recently i hear a lot about how lxml is great, but I don't have first-hand experience, and I had some fun working with gnosis.
I've always had good results with lxml. It's a pain to install, as it's mostly a wrapper around libxml2, but lxml.etree tree objects have a .write() method that takes a file-like object to stream to.
from lxml.etree import XML
tree = XML('<root><a><b/></a></root>')
tree.write(your_file_object)
Second vote for ElementTree (cElementTree is a C implementation that is a little faster, like cPickle vs pickle). There's some short example code here that you can look at to give you an idea of how it works: http://effbot.org/zone/element-index.htm
(this is Fredrik Lundh, who wrote the module in the first place. It's so good it got drafted into the standard library with 2.5 :-) )
xml.etree.cElementTree, included in the default distribution of CPython since 2.5. Lightning fast for both reading and writing XML.