We Keep Coding

Make a comparison and print out what is not in the list, how can I do that?

import re
Hosts = ["gvff", "etc"]
def list():
with open('C:/Users/user/Desktop/scripts/devices.txt', 'r') as file:
for row in file:
device = row.rsplit('\n')
devices = re.search("^(.*?)\,(.*?)\,(.*?)\,", device[0])
if devices != None:
device = devices.group(1)
hardwaretype = devices.group(2)
level = devices.group(3)
for host in Hosts:
if device.startswith(host):
if level == 'Production' or level == 'Production-Secondary':
print(f'| {device} | {hardwaretype} | {level} |')
list()
Hey I'm not getting anywhere at the moment. I want to compare the hosts list with the devices.txt. It currently prints me everything it has matched, but I need the items that could not be matched. How could I do that here?
Current Output: | blabla.bla.bla | physical | Production | this host is in the device.txt and in the Hosts list, so its the matching output
Excepted Output: looks like above, but only the hosts which are in devices.txt but not in Hosts List.
Many greetings

Just add the not keyword before device.startswith(host), it will make the condition true if the device not starts with 'host' otherwise false.
change
if device.startswith(host):
...
to
if not device.startswith(host):
...

Python 2.7 path replace string

I need to replace some string by others. I'm using function pathlib to do that it's working fine but I have a problem when there is two same string in my file and I need to change just one
My file (wireless.temp) is like this example
config 'conf'
option disabled '0'
option hidden '1'
option ssid 'confSSID'
option encryption 'psk2'
option key 'qwerty'
config 'client'
option disabled '0'
option hidden '0'
option ssid 'clientSSID'
option encryption 'psk2'
option key 'qwerty'
For example, I need to change string like 'disabled', 'hidden', 'ssid', 'key', in config station and/or config device. Right now I'm using this code
f1=open('wireless.temp', 'r').read()
f2=open('wireless.temp','w')
#checkbox from QT interface
if self.chkWifiEnable.isChecked():
newWifiEnable = "0"
else:
newWifiEnable = "1"
start = f1.find("config 'client'")
print start
end = f1.find("config", start + 1)
print end
if end < 0:
end = len(f1)
station = f1[start:end]
print station
print f1.find("disabled")
print f1.find("'")
actualValue = f1[(station.find("disabled")+10):station.find("'")]
print actualValue
station = station.replace("disabled '" + actualValue, "disabled '" + newWifiEnable)
print station
m = f1[:start] + station + f1[end:]
f2.write(m)
I have a problem with this code, first when I execute my output is
config 'conf'
option device 'radio0'
option ifname 'conf'
option network 'conf'
option mode 'ap'
option disabled '0'
option hidden '1'
option isolate '1'
option ssid 'Conf-2640'
option encryption 'psk2'
option key '12345678'
config 'client'
option device 'radio0'
option ifname 'ra0'
option network 'lan'
option mode 'ap'
option disabled '00' <---- problem
option hidden '0'
option ssid 'FW-2640'
option encryption 'psk2'
option key '12345678'
option disabled line in config 'client' section, my program add another 0 all time also I want to lighten my code because I need to do that for many others string.
Does anyone have an idea?
Thanks

The Path and pathlib2 is a red-herring. You are using that to find and read a file into a string. The issue is replacing text in a narrow section of the whole text. Specifically, between 'config device' and the next 'config ...' item
You can use .find() to find the beginning of the correct config section, and again to find the start of the next config section. Ensure you treat -1, for not found, as the end of the section. The text within that range can be modified, and then combine the resulting modification with the unmodified parts that came before and after it.
wirelessF = """
config device
.....
option key 'qwerty'
.....
config station
.....
option key 'qwerty'
.....
config otherthing
.....
option key 'qwerty'
.....
"""
actualWifiPass = 'qwerty'
newWifiPass = 's3cr3t'
start = wirelessF.find("config station")
end = wirelessF.find("config ", start+1)
if end < 0:
end = len(wirelessF)
station = wirelessF[start:end]
station = station.replace("key '"+actualWifiPass, "key '"+newWifiPass)
wirelessF = wirelessF[:start] + station + wirelessF[end:]
print(wirelessF)
Outputs:
config device
.....
option key 'qwerty'
.....
config station
.....
option key 's3cr3t'
.....
config otherthing
.....
option key 'qwerty'
.....

How to use index in python list?

I am trying to execute the code of pyethereum but when I was analyzing the code in
pyethereum/Ethereum/hybrid_casper/consenus.py
I can't understand where the 'NULL_SENDER' value is defined and how this state.config['NULL_SENDER] will execute.
key, account = state.config['NULL_SENDER'], privtoaddr(state.config['NULL_SENDER'])

Let's look through all the code.
from ethereum import utils, transactions
from ethereum.common import update_block_env_variables
from ethereum.messages import apply_transaction
from ethereum.hybrid_casper import casper_utils
from ethereum.utils import privtoaddr
# Block initialization state transition
def initialize(state, block=None):
config = state.config
state.txindex = 0
state.gas_used = 0
state.bloom = 0
state.receipts = []
if block is not None:
update_block_env_variables(state, block)
# Initalize the next epoch in the Casper contract
if state.block_number % state.config['EPOCH_LENGTH'] == 0 and state.block_number != 0:
key, account = state.config['NULL_SENDER'], privtoaddr(state.config['NULL_SENDER'])
data = casper_utils.casper_translator.encode('initialize_epoch', [state.block_number // state.config['EPOCH_LENGTH']])
transaction = transactions.Transaction(state.get_nonce(account), 0, 3141592,
state.config['CASPER_ADDRESS'], 0, data).sign(key)
success, output = apply_transaction(state, transaction)
assert success
if state.is_DAO(at_fork_height=True):
for acct in state.config['CHILD_DAO_LIST']:
state.transfer_value(
acct,
state.config['DAO_WITHDRAWER'],
state.get_balance(acct))
if state.is_METROPOLIS(at_fork_height=True):
state.set_code(utils.normalize_address(
config["METROPOLIS_STATEROOT_STORE"]), config["METROPOLIS_GETTER_CODE"])
state.set_code(utils.normalize_address(
config["METROPOLIS_BLOCKHASH_STORE"]), config["METROPOLIS_GETTER_CODE"])
We can see that this code is most likely being imported by multiple programs, which it is! https://github.com/ethereum/pyethereum/blob/develop/ethereum/hybrid_casper/chain.py
If we see the usage of the function in chain.py, the state parameter is being fulfilled with self.state, it is self.state = self.mk_poststate_of_blockhash(self.db.get(b'head_hash')).
This function returns a State object, made in ethereum.state, which can be turned into a dictionary. This most likely means that it is getting the value that corresponds to the key called 'NULL_SENDER'.

How to color specific part of the text with some separator

I'm trying to color some specific part of the text, i have tried to say:
if word.strip().startswith(":"):
self.setAttributesForRange(NSColor.greenColor(), None, highlightOffset, len(word))
When someone types the sign : it gets colored green. That is good, but it keeps coloring the word after it like this:
:Hello Hello :Hello <---- this all gets colored green, but I want something like:
:Hello Hello :Hello <---- where everything get colored except the middle "hello" because it doesn't start with the sign : , please help me out
from Foundation import *
from AppKit import *
import objc
class PyObjC_HighlightAppDelegate(NSObject):
# The connection to our NSTextView in the UI
highlightedText = objc.IBOutlet()
# Default font size to use when highlighting
fontSize = 12
def applicationDidFinishLaunching_(self, sender):
NSLog("Application did finish launching.")
def textDidChange_(self, notification):
"""
Delegate method called by the NSTextView whenever the contents of the
text view have changed. This is called after the text has changed and
been committed to the view. See the Cocoa reference documents:
http://developer.apple.com/documentation/Cocoa/Reference/ApplicationKit/Classes/NSText_Class/Reference/Reference.html
http://developer.apple.com/documentation/Cocoa/Reference/ApplicationKit/Classes/NSTextView_Class/Reference/Reference.html
Specifically the sections on Delegate Methods for information on additional
delegate methods relating to text control is NSTextView objects.
"""
# Retrieve the current contents of the document and start highlighting
content = self.highlightedText.string()
self.highlightText(content)
def setAttributesForRange(self, color, font, rangeStart, rangeLength):
"""
Set the visual attributes for a range of characters in the NSTextView. If
values for the color and font are None, defaults will be used.
The rangeStart is an index into the contents of the NSTextView, and
rangeLength is used in combination with this index to create an NSRange
structure, which is passed to the NSTextView methods for setting
text attributes. If either of these values are None, defaults will
be provided.
The "font" parameter is used as an key for the "fontMap", which contains
the associated NSFont objects for each font style.
"""
fontMap = {
"normal" : NSFont.systemFontOfSize_(self.fontSize),
"bold" : NSFont.boldSystemFontOfSize_(self.fontSize)
}
# Setup sane defaults for the color, font and range if no values
# are provided
if color is None:
color = NSColor.blackColor()
if font is None:
font = "normal"
if font not in fontMap:
font = "normal"
displayFont = fontMap[font]
if rangeStart is None:
rangeStart = 0
if rangeLength is None:
rangeLength = len(self.highlightedText.string()) - rangeStart
# Set the attributes for the specified character range
range = NSRange(rangeStart, rangeLength)
self.highlightedText.setTextColor_range_(color, range)
self.highlightedText.setFont_range_(displayFont, range)
def highlightText(self, content):
"""
Apply our customized highlighting to the provided content. It is assumed that
this content was extracted from the NSTextView.
"""
# Calling the setAttributesForRange with no values creates
# a default that "resets" the formatting on all of the content
self.setAttributesForRange(None, None, None, None)
# We'll highlight the content by breaking it down into lines, and
# processing each line one by one. By storing how many characters
# have been processed we can maintain an "offset" into the overall
# content that we use to specify the range of text that is currently
# being highlighted.
contentLines = content.split("\n")
highlightOffset = 0
for line in contentLines:
if line.strip().startswith("#"):
# Comment - we want to highlight the whole comment line
self.setAttributesForRange(NSColor.greenColor(), None, highlightOffset, len(line))
elif line.find(":") > -1:
# Tag - we only want to highlight the tag, not the colon or the remainder of the line
startOfLine = line[0: line.find(":")]
yamlTag = startOfLine.strip("\t ")
yamlTagStart = line.find(yamlTag)
self.setAttributesForRange(NSColor.blueColor(), "bold", highlightOffset + yamlTagStart, len(yamlTag))
elif line.strip().startswith("-"):
# List item - we only want to highlight the dash
listIndex = line.find("-")
self.setAttributesForRange(NSColor.redColor(), None, highlightOffset + listIndex, 1)
# Add the processed line to our offset, as well as the newline that terminated the line
highlightOffset += len(line) + 1

It all depends on what word is.
In [6]: word = ':Hello Hello :Hello'
In [7]: word.strip().startswith(':')
Out[7]: True
In [8]: len(word)
Out[8]: 19
Compare:
In [1]: line = ':Hello Hello :Hello'.split()
In [2]: line
Out[2]: [':Hello', 'Hello', ':Hello']
In [3]: for word in line:
print word.strip().startswith(':')
print len(word)
...:
True
6
False
5
True
6
Notice the difference in len(word), which I suspect is causing your problem.

How to parse nagios status.dat file?

I'd like to parse status.dat file for nagios3 and output as xml with a python script.
The xml part is the easy one but how do I go about parsing the file? Use multi line regex?
It's possible the file will be large as many hosts and services are monitored, will loading the whole file in memory be wise?
I only need to extract services that have critical state and host they belong to.
Any help and pointing in the right direction will be highly appreciated.
LE Here's how the file looks:
########################################
# NAGIOS STATUS FILE
#
# THIS FILE IS AUTOMATICALLY GENERATED
# BY NAGIOS. DO NOT MODIFY THIS FILE!
########################################
info {
created=1233491098
version=2.11
}
program {
modified_host_attributes=0
modified_service_attributes=0
nagios_pid=15015
daemon_mode=1
program_start=1233490393
last_command_check=0
last_log_rotation=0
enable_notifications=1
active_service_checks_enabled=1
passive_service_checks_enabled=1
active_host_checks_enabled=1
passive_host_checks_enabled=1
enable_event_handlers=1
obsess_over_services=0
obsess_over_hosts=0
check_service_freshness=1
check_host_freshness=0
enable_flap_detection=0
enable_failure_prediction=1
process_performance_data=0
global_host_event_handler=
global_service_event_handler=
total_external_command_buffer_slots=4096
used_external_command_buffer_slots=0
high_external_command_buffer_slots=0
total_check_result_buffer_slots=4096
used_check_result_buffer_slots=0
high_check_result_buffer_slots=2
}
host {
host_name=localhost
modified_attributes=0
check_command=check-host-alive
event_handler=
has_been_checked=1
should_be_scheduled=0
check_execution_time=0.019
check_latency=0.000
check_type=0
current_state=0
last_hard_state=0
plugin_output=PING OK - Packet loss = 0%, RTA = 3.57 ms
performance_data=
last_check=1233490883
next_check=0
current_attempt=1
max_attempts=10
state_type=1
last_state_change=1233489475
last_hard_state_change=1233489475
last_time_up=1233490883
last_time_down=0
last_time_unreachable=0
last_notification=0
next_notification=0
no_more_notifications=0
current_notification_number=0
notifications_enabled=1
problem_has_been_acknowledged=0
acknowledgement_type=0
active_checks_enabled=1
passive_checks_enabled=1
event_handler_enabled=1
flap_detection_enabled=1
failure_prediction_enabled=1
process_performance_data=1
obsess_over_host=1
last_update=1233491098
is_flapping=0
percent_state_change=0.00
scheduled_downtime_depth=0
}
service {
host_name=gateway
service_description=PING
modified_attributes=0
check_command=check_ping!100.0,20%!500.0,60%
event_handler=
has_been_checked=1
should_be_scheduled=1
check_execution_time=4.017
check_latency=0.210
check_type=0
current_state=0
last_hard_state=0
current_attempt=1
max_attempts=4
state_type=1
last_state_change=1233489432
last_hard_state_change=1233489432
last_time_ok=1233491078
last_time_warning=0
last_time_unknown=0
last_time_critical=0
plugin_output=PING OK - Packet loss = 0%, RTA = 2.98 ms
performance_data=
last_check=1233491078
next_check=1233491378
current_notification_number=0
last_notification=0
next_notification=0
no_more_notifications=0
notifications_enabled=1
active_checks_enabled=1
passive_checks_enabled=1
event_handler_enabled=1
problem_has_been_acknowledged=0
acknowledgement_type=0
flap_detection_enabled=1
failure_prediction_enabled=1
process_performance_data=1
obsess_over_service=1
last_update=1233491098
is_flapping=0
percent_state_change=0.00
scheduled_downtime_depth=0
}
It can have any number of hosts and a host can have any number of services.

Pfft, get yerself mk_livestatus. http://mathias-kettner.de/checkmk_livestatus.html

Nagiosity does exactly what you want:
http://code.google.com/p/nagiosity/

Having shamelessly stolen from the above examples,
Here's a version build for Python 2.4 that returns a dict containing arrays of nagios sections.
def parseConf(source):
conf = {}
patID=re.compile(r"(?:\s*define)?\s*(\w+)\s+{")
patAttr=re.compile(r"\s*(\w+)(?:=|\s+)(.*)")
patEndID=re.compile(r"\s*}")
for line in source.splitlines():
line=line.strip()
matchID = patID.match(line)
matchAttr = patAttr.match(line)
matchEndID = patEndID.match( line)
if len(line) == 0 or line[0]=='#':
pass
elif matchID:
identifier = matchID.group(1)
cur = [identifier, {}]
elif matchAttr:
attribute = matchAttr.group(1)
value = matchAttr.group(2).strip()
cur[1][attribute] = value
elif matchEndID and cur:
conf.setdefault(cur[0],[]).append(cur[1])
del cur
return conf
To get all Names your Host which have contactgroups beginning with 'devops':
nagcfg=parseConf(stringcontaingcompleteconfig)
hostlist=[host['host_name'] for host in nagcfg['host']
if host['contact_groups'].startswith('devops')]

Don't know nagios and its config file, but the structure seems pretty simple:
# comment
identifier {
attribute=
attribute=value
}
which can simply be translated to
<identifier>
<attribute name="attribute-name">attribute-value</attribute>
</identifier>
all contained inside a root-level <nagios> tag.
I don't see line breaks in the values. Does nagios have multi-line values?
You need to take care of equal signs within attribute values, so set your regex to non-greedy.

You can do something like this:
def parseConf(filename):
conf = []
with open(filename, 'r') as f:
for i in f.readlines():
if i[0] == '#': continue
matchID = re.search(r"([\w]+) {", i)
matchAttr = re.search(r"[ ]*([\w]+)=([\w\d]*)", i)
matchEndID = re.search(r"[ ]*}", i)
if matchID:
identifier = matchID.group(1)
cur = [identifier, {}]
elif matchAttr:
attribute = matchAttr.group(1)
value = matchAttr.group(2)
cur[1][attribute] = value
elif matchEndID:
conf.append(cur)
return conf
def conf2xml(filename):
conf = parseConf(filename)
xml = ''
for ID in conf:
xml += '<%s>\n' % ID[0]
for attr in ID[1]:
xml += '\t<attribute name="%s">%s</attribute>\n' % \
(attr, ID[1][attr])
xml += '</%s>\n' % ID[0]
return xml
Then try to do:
print conf2xml('conf.dat')

If you slightly tweak Andrea's solution you can use that code to parse both the status.dat as well as the objects.cache
def parseConf(source):
conf = []
for line in source.splitlines():
line=line.strip()
matchID = re.match(r"(?:\s*define)?\s*(\w+)\s+{", line)
matchAttr = re.match(r"\s*(\w+)(?:=|\s+)(.*)", line)
matchEndID = re.match(r"\s*}", line)
if len(line) == 0 or line[0]=='#':
pass
elif matchID:
identifier = matchID.group(1)
cur = [identifier, {}]
elif matchAttr:
attribute = matchAttr.group(1)
value = matchAttr.group(2).strip()
cur[1][attribute] = value
elif matchEndID and cur:
conf.append(cur)
del cur
return conf
It is a little puzzling why nagios chose to use two different formats for these files, but once you've parsed them both into some usable python objects you can do quite a bit of magic through the external command file.
If anybody has a solution for getting this into a a real xml dom that'd be awesome.

For the last several months I've written and released a tool that that parses the Nagios status.dat and objects.cache and builds a model that allows for some really useful manipulation of Nagios data. We use it to drive an internal operations dashboard that is a simplified 'mini' Nagios. Its under continual development and I've neglected testing and documentation but the code isn't too crazy and I feel fairly easy to follow.
Let me know what you think...
https://github.com/zebpalmer/NagParser