We Keep Coding

eval(parse()) not having same effect as repl in edited version of norvig's lispy.py interpreter

I'm trying to edit Norvig's lispy.py scheme-to-python interpreter for a racket program in which, in addition to the procedures he includes, I use string->jsexpr and string-replace.
To do this, I edited his add_globals function as follows.
def add_globals(self):
"Add some Scheme standard procedures."
import math, cmath, operator as op
self.update(vars(math))
self.update(vars(cmath))
self.update({
'+':op.add, '-':op.sub, '*':op.mul, '/':op.div, 'not':op.not_,
'>':op.gt, '<':op.lt, '>=':op.ge, '<=':op.le, '=':op.eq,
'equal?':op.eq, 'eq?':op.is_, 'length':len, 'cons':cons,
'car':lambda x:x[0], 'cdr':lambda x:x[1:], 'append':op.add,
'list':lambda *x:list(x), 'list?': lambda x:isa(x,list),
'null?':lambda x:x==[], 'symbol?':lambda x: isa(x, Symbol),
'boolean?':lambda x: isa(x, bool), 'pair?':is_pair,
'port?': lambda x:isa(x,file), 'apply':lambda proc,l: proc(*l),
'eval':lambda x: eval(expand(x)), 'load':lambda fn: load(fn), 'call/cc':callcc,
'open-input-file':open,'close-input-port':lambda p: p.file.close(),
'open-output-file':lambda f:open(f,'w'), 'close-output-port':lambda p: p.close(),
'eof-object?':lambda x:x is eof_object, 'read-char':readchar,
# Additions below
'reverse':lambda x: x[::-1],
'string->jsexpr':lambda x: json.loads(x),
'string-replace':lambda strng,x,y: strng.replace(x, y),
# Additions above
'read':read, 'write':lambda x,port=sys.stdout:port.write(to_string(x)),
'display':lambda x,port=sys.stdout:port.write(x if isa(x,str) else to_string(x))})
return self
In the lispy repl, string-replace seems to succeed:
% python lispy.py
Lispy version 2.0
lispy> (string-replace "[['Noth', '<b>ing</b>'], ' in']" "\'" "\"")
"[[\"Noth\", \"<b>ing</b>\"], \" in\"]"
lispy>
However when, at the end of lispy.py, I run
if __name__ == '__main__':
x = eval(parse("""(begin
(string-replace "[['Noth', '<b>ing</b>'], ' in']" "\'" "\"")
)
"""
))
print x
instead of running this (which is how the file came),
if __name__ == '__main__':
repl()
the program hangs — and when I trace it, I see that there's an infinite loop.
Any ideas?

Got it. I needed to add an extra backslash to escape the escape in the triple quoted section, like this
if __name__ == '__main__':
x = eval(parse("""(begin
(string-replace "[['Noth', '<b>ing</b>'], ' in']" "\\'" "\\"")
)
"""
))
print x

Why am I having "Invalid syntax in Python?"

In the following code sample, the last line (print buildConnectionString(myParams)) is throwing the following error:
Invalid syntax
def buildConnectionString(params):
return ";".join(["%s=%s" % (k, v) for k, v in params.items()])
if __name__ == "__main__":
myParams = {"server":"mpilgrim", \
"database":"master", \
"uid":"sa", \
"pwd":"secret"
}
print buildConnectionString(myParams)

Assuming that you're on Python 3, print is a function and needs to be wrapped in parentheses:
print(buildConnectionString(myParams))

VIM command to insert multiline text with argument

new VIM user. I'm trying to make creating python properties easier for my class definitions. What I would like for say I type
:pyp x
then VIM will autofill where my cursor is
#property
def x(self):
return self.x
#property.setter
def x(self,val):
self._x = val
or more abstractly I type
:pyp <property_name>
and VIM fills
#property
def <property_name>(self):
return self.<property_name>
#property.setter
def <property_name>(self,val):
self._<property_name> = val
I've looked at a few posts and the wikis on functions, macros but I'm very unsure of how to go about it or what to even look up as I am brand new VIM user, less than a week old.
I tried using [this][1] as an example, in my .vimrc but I couldn't even get that to work.
Edit:
So the code I am currently trying is
function! PythonProperty(prop_name)
let cur_line = line('.')
let num_spaces = indent('.')
let spaces = repeat(' ',num_spaces)
let lines = [ spaces."#property",
\ spaces."def ".prop_name."(self):",
\ spaces." return self.".property,
\ spaces."#property.setter",
\ spaces."def".prop_name."(self,val)",
\ spaces." self._".prop_name." = val" ]
call append(cur_line,lines)
endfunction
and I am getting the errors
E121: Undefined variable: prop_name
I am typing
`:call PythonProperty("x")`
[1]: https://vi.stackexchange.com/questions/9644/how-to-use-a-variable-in-the-expression-of-a-normal-command

E121: Undefined variable: prop_name
In VimScript variables have scopes. The scope for function arguments is a:, while the default inside a function is l: (local variable). So the error means that l:prop_name was not yet defined.
Now how I do this:
function! s:insert_pyp(property)
let l:indent = repeat(' ', indent('.'))
let l:text = [
\ '#property',
\ 'def <TMPL>(self):',
\ ' return self.<TMPL>',
\ '#property.setter',
\ ' def <TMPL>(self,val):',
\ ' self._<TMPL> = val'
\ ]
call map(l:text, {k, v -> l:indent . substitute(v, '\C<TMPL>', a:property, 'g')})
call append('.', l:text)
endfunction
command! -nargs=1 Pyp :call <SID>insert_pyp(<q-args>)
Alternatively, we can simulate actual key presses (note that we don't need to put indents in the template anymore; hopefully, the current buffer has set ft=python):
function! s:insert_pyp2(property)
let l:text = [
\ '#property',
\ 'def <TMPL>(self):',
\ 'return self.<TMPL>',
\ '#property.setter',
\ 'def <TMPL>(self,val):',
\ 'self._<TMPL> = val'
\ ]
execute "normal! o" . substitute(join(l:text, "\n"), '\C<TMPL>', a:property, 'g') . "\<Esc>"
endfunction
command! -nargs=1 Pyp2 :call <SID>insert_pyp2(<q-args>)
its very very difficult if not impossible to get pluggins
I suggest you to watch this video on youtube. In fact, many of Vim plugins are just overkill.

using libclang, callback function does not traverse recursively and does not visit all functions

I am following an example which shows limitation of python bindings, from site http://eli.thegreenplace.net/2011/07/03/parsing-c-in-python-with-clang/
It uses "libclang visitation API directly".
import sys
import clang.cindex
def callexpr_visitor(node, parent, userdata):
if node.kind == clang.cindex.CursorKind.CALL_EXPR:
print 'Found %s [line=%s, col=%s]' % (
node.spelling, node.location.line, node.location.column)
return 2 # means continue visiting recursively
index = clang.cindex.Index.create()
tu = index.parse(sys.argv[1])
clang.cindex.Cursor_visit(
tu.cursor,
clang.cindex.Cursor_visit_callback(callexpr_visitor),
None)
The output shows all functions called along with their line numbers.
Found foo [line=8, col=5]
Found foo [line=10, col=9]
Found bar [line=15, col=5]
Found foo [line=16, col=9]
Found bar [line=17, col=9]
When I run the same code, I only get output
Found bar [line=15, col=5]
The version I use is llvm3.1 with windows (with the changes suggested in the link).
I feel, returning 2 is not calling the callback function again.
I have even tried using 'get_children' on node and traversing without callback, I get the same result.
import sys
import clang.cindex
#def callexpr_visitor(node, parent, userdata):
def callexpr_visitor(node):
if node.kind == clang.cindex.CursorKind.CALL_EXPR:
print 'Found %s [line=%s, col=%s]' % (
clang.cindex.Cursor_displayname(node), node.location.line, node.location.column)
for c in node.get_children():
callexpr_visitor(c)
#return 2 # means continue visiting recursively
index = clang.cindex.Index.create()
tu = index.parse(sys.argv[1])
#clang.cindex.Cursor_visit(
# tu.cursor,
# clang.cindex.Cursor_visit_callback(callexpr_visitor),
# None)
callexpr_visitor(tu.cursor)
I could not get the reason for this behavior after much search and trials.
Can anyone explain this please ?
Regards.

I think I found the reason for this.
If I change the return type of function 'foo', from 'bool' to 'int', I get the expected result.
This may be as 'bool' is not a keyword in 'c'.
That was simple.

How to parse nagios status.dat file?

I'd like to parse status.dat file for nagios3 and output as xml with a python script.
The xml part is the easy one but how do I go about parsing the file? Use multi line regex?
It's possible the file will be large as many hosts and services are monitored, will loading the whole file in memory be wise?
I only need to extract services that have critical state and host they belong to.
Any help and pointing in the right direction will be highly appreciated.
LE Here's how the file looks:
########################################
# NAGIOS STATUS FILE
#
# THIS FILE IS AUTOMATICALLY GENERATED
# BY NAGIOS. DO NOT MODIFY THIS FILE!
########################################
info {
created=1233491098
version=2.11
}
program {
modified_host_attributes=0
modified_service_attributes=0
nagios_pid=15015
daemon_mode=1
program_start=1233490393
last_command_check=0
last_log_rotation=0
enable_notifications=1
active_service_checks_enabled=1
passive_service_checks_enabled=1
active_host_checks_enabled=1
passive_host_checks_enabled=1
enable_event_handlers=1
obsess_over_services=0
obsess_over_hosts=0
check_service_freshness=1
check_host_freshness=0
enable_flap_detection=0
enable_failure_prediction=1
process_performance_data=0
global_host_event_handler=
global_service_event_handler=
total_external_command_buffer_slots=4096
used_external_command_buffer_slots=0
high_external_command_buffer_slots=0
total_check_result_buffer_slots=4096
used_check_result_buffer_slots=0
high_check_result_buffer_slots=2
}
host {
host_name=localhost
modified_attributes=0
check_command=check-host-alive
event_handler=
has_been_checked=1
should_be_scheduled=0
check_execution_time=0.019
check_latency=0.000
check_type=0
current_state=0
last_hard_state=0
plugin_output=PING OK - Packet loss = 0%, RTA = 3.57 ms
performance_data=
last_check=1233490883
next_check=0
current_attempt=1
max_attempts=10
state_type=1
last_state_change=1233489475
last_hard_state_change=1233489475
last_time_up=1233490883
last_time_down=0
last_time_unreachable=0
last_notification=0
next_notification=0
no_more_notifications=0
current_notification_number=0
notifications_enabled=1
problem_has_been_acknowledged=0
acknowledgement_type=0
active_checks_enabled=1
passive_checks_enabled=1
event_handler_enabled=1
flap_detection_enabled=1
failure_prediction_enabled=1
process_performance_data=1
obsess_over_host=1
last_update=1233491098
is_flapping=0
percent_state_change=0.00
scheduled_downtime_depth=0
}
service {
host_name=gateway
service_description=PING
modified_attributes=0
check_command=check_ping!100.0,20%!500.0,60%
event_handler=
has_been_checked=1
should_be_scheduled=1
check_execution_time=4.017
check_latency=0.210
check_type=0
current_state=0
last_hard_state=0
current_attempt=1
max_attempts=4
state_type=1
last_state_change=1233489432
last_hard_state_change=1233489432
last_time_ok=1233491078
last_time_warning=0
last_time_unknown=0
last_time_critical=0
plugin_output=PING OK - Packet loss = 0%, RTA = 2.98 ms
performance_data=
last_check=1233491078
next_check=1233491378
current_notification_number=0
last_notification=0
next_notification=0
no_more_notifications=0
notifications_enabled=1
active_checks_enabled=1
passive_checks_enabled=1
event_handler_enabled=1
problem_has_been_acknowledged=0
acknowledgement_type=0
flap_detection_enabled=1
failure_prediction_enabled=1
process_performance_data=1
obsess_over_service=1
last_update=1233491098
is_flapping=0
percent_state_change=0.00
scheduled_downtime_depth=0
}
It can have any number of hosts and a host can have any number of services.

Pfft, get yerself mk_livestatus. http://mathias-kettner.de/checkmk_livestatus.html

Nagiosity does exactly what you want:
http://code.google.com/p/nagiosity/

Having shamelessly stolen from the above examples,
Here's a version build for Python 2.4 that returns a dict containing arrays of nagios sections.
def parseConf(source):
conf = {}
patID=re.compile(r"(?:\s*define)?\s*(\w+)\s+{")
patAttr=re.compile(r"\s*(\w+)(?:=|\s+)(.*)")
patEndID=re.compile(r"\s*}")
for line in source.splitlines():
line=line.strip()
matchID = patID.match(line)
matchAttr = patAttr.match(line)
matchEndID = patEndID.match( line)
if len(line) == 0 or line[0]=='#':
pass
elif matchID:
identifier = matchID.group(1)
cur = [identifier, {}]
elif matchAttr:
attribute = matchAttr.group(1)
value = matchAttr.group(2).strip()
cur[1][attribute] = value
elif matchEndID and cur:
conf.setdefault(cur[0],[]).append(cur[1])
del cur
return conf
To get all Names your Host which have contactgroups beginning with 'devops':
nagcfg=parseConf(stringcontaingcompleteconfig)
hostlist=[host['host_name'] for host in nagcfg['host']
if host['contact_groups'].startswith('devops')]

Don't know nagios and its config file, but the structure seems pretty simple:
# comment
identifier {
attribute=
attribute=value
}
which can simply be translated to
<identifier>
<attribute name="attribute-name">attribute-value</attribute>
</identifier>
all contained inside a root-level <nagios> tag.
I don't see line breaks in the values. Does nagios have multi-line values?
You need to take care of equal signs within attribute values, so set your regex to non-greedy.

You can do something like this:
def parseConf(filename):
conf = []
with open(filename, 'r') as f:
for i in f.readlines():
if i[0] == '#': continue
matchID = re.search(r"([\w]+) {", i)
matchAttr = re.search(r"[ ]*([\w]+)=([\w\d]*)", i)
matchEndID = re.search(r"[ ]*}", i)
if matchID:
identifier = matchID.group(1)
cur = [identifier, {}]
elif matchAttr:
attribute = matchAttr.group(1)
value = matchAttr.group(2)
cur[1][attribute] = value
elif matchEndID:
conf.append(cur)
return conf
def conf2xml(filename):
conf = parseConf(filename)
xml = ''
for ID in conf:
xml += '<%s>\n' % ID[0]
for attr in ID[1]:
xml += '\t<attribute name="%s">%s</attribute>\n' % \
(attr, ID[1][attr])
xml += '</%s>\n' % ID[0]
return xml
Then try to do:
print conf2xml('conf.dat')

If you slightly tweak Andrea's solution you can use that code to parse both the status.dat as well as the objects.cache
def parseConf(source):
conf = []
for line in source.splitlines():
line=line.strip()
matchID = re.match(r"(?:\s*define)?\s*(\w+)\s+{", line)
matchAttr = re.match(r"\s*(\w+)(?:=|\s+)(.*)", line)
matchEndID = re.match(r"\s*}", line)
if len(line) == 0 or line[0]=='#':
pass
elif matchID:
identifier = matchID.group(1)
cur = [identifier, {}]
elif matchAttr:
attribute = matchAttr.group(1)
value = matchAttr.group(2).strip()
cur[1][attribute] = value
elif matchEndID and cur:
conf.append(cur)
del cur
return conf
It is a little puzzling why nagios chose to use two different formats for these files, but once you've parsed them both into some usable python objects you can do quite a bit of magic through the external command file.
If anybody has a solution for getting this into a a real xml dom that'd be awesome.

For the last several months I've written and released a tool that that parses the Nagios status.dat and objects.cache and builds a model that allows for some really useful manipulation of Nagios data. We use it to drive an internal operations dashboard that is a simplified 'mini' Nagios. Its under continual development and I've neglected testing and documentation but the code isn't too crazy and I feel fairly easy to follow.
Let me know what you think...
https://github.com/zebpalmer/NagParser