We Keep Coding

Tab/Indent Error in Python Command Prompt

New to Python & taking a course where teacher provided a code snippet to work on. The provided snippet is giving me a TabError as below, but I cannot seem to figure out why. I was hoping someone here could point out the issue.
Error Received:
Error Received
Code where error occurs (Updated to have code lines instead of screen shot):
def filterResultsAcc(self, qcoverage=0.6, scoverage=0.6, evalue=0.01, best_taxon=False):
results = []
hits_by_taxon = {}
for(qstart,qend,sacc,staxid,slen,sstart,send,evalu) in self.blasthits:
qcov = ((qend-qstart) + 1.0) / float(self.querylen)
scov = ((send-sstart) + 1.0 / float(slen)
if qcov >= qcoverage and scov >= scoverage and evalu <= evalue:
if best_taxon:
if staxid in hits_by_taxon.keys():
(sid,e,qc,sc) = hits_by_taxon[staxid]
if evalu < e:
hits_by_taxon[staxid] = (sacc,evalu,qcov,scov)
elif evalu == e:
if qcov > qc:
hits_by_taxon[staxid] = (sacc,evalu,qcov,scov)
elif qvoc == qc:
if scov > sc:
hits_by_taxon[staxid] = (sacc,evalu,qcov,scov)
else:
hits_by_taxon[staxid] = (sacc,evalu,qcov,scov)
else:
results.append(sacc)
if best_taxon:
for taxid in hits_by_taxon.keys():
(sac,e,qc,sc) = hits_by_taxon[taxid]
results.append(sacc)
results.sort()
return results

The error shows that you've mixed tab and spaces, just as it says.
If your editor supports it – turn something like "show unprintable characters" to see what your whitespaces really is.
Another way is to select all code, usually editors will distinguish between spaces and tabs.

Python on Visual Studio Error

Can somebody help me identify the error here? I am using Python on Visual Studio, and I have copied the code from another source. When I paste the code into a playground like Ideone.com, the program runs without issue. Only when I paste into visual studio do I get an error. The error is in the last two lines of code that begin with "print". This is probably a huge noob question, but I am a beginner. Help!
import hashlib as hasher
import datetime as date
# Define what a Snakecoin block is
class Block:
def __init__(self, index, timestamp, data, previous_hash):
self.index = index
self.timestamp = timestamp
self.data = data
self.previous_hash = previous_hash
self.hash = self.hash_block()
def hash_block(self):
sha = hasher.sha256()
sha.update(str(self.index) + str(self.timestamp) + str(self.data) + str(self.previous_hash))
return sha.hexdigest()
# Generate genesis block
def create_genesis_block():
# Manually construct a block with
# index zero and arbitrary previous hash
return Block(0, date.datetime.now(), "Genesis Block", "0")
# Generate all later blocks in the blockchain
def next_block(last_block):
this_index = last_block.index + 1
this_timestamp = date.datetime.now()
this_data = "Hey! I'm block " + str(this_index)
this_hash = last_block.hash
return Block(this_index, this_timestamp, this_data, this_hash)
# Create the blockchain and add the genesis block
blockchain = [create_genesis_block()]
previous_block = blockchain[0]
# How many blocks should we add to the chain
# after the genesis block
num_of_blocks_to_add = 20
# Add blocks to the chain
for i in range(0, num_of_blocks_to_add):
block_to_add = next_block(previous_block)
blockchain.append(block_to_add)
previous_block = block_to_add
# Tell everyone about it!
print "Block #{} has been added to the blockchain!".format(block_to_add.index)
print "Hash: {}\n".format(block_to_add.hash)

In visual studio and most other editors, they are updated to the most recent python, also meaning that you have to place parentheses after the print statement.
For example:
print("hello")

Visual Studio runs Python 3.x.x, so, you have to write the print sentences on this way:
print('String to print')
Because in Python 3 print is a function, not a reserved word like in Python 2.

Generating consecutive bitmaps with wxPython trouble

Well, while I don't consider myself an experienced programmer, especially when it comes to multimedia applications, I had to do this image viewer sort of a program that displays images fullscreen, and the images change when the users press <- or -> arrows.
The general idea was to make a listbox where all the images contained in a certain folder (imgs) are shown, and when someone does double click or hits RETURN a new, fullscreen frame is generated containing, at first, the image selected in the listbox but after the user hits the arrows the images change in conecutive order, just like in a regular image viewer.
At the beginning, when the first 15 - 20 images are generated, everything goes well, after that, although the program still works an intermediary, very unpleasant and highly unwanted, effect appears between image generations, where basically some images that got generated previously are displayed quickly on the screen and after this the right, consecutive image appears. On the first apparitions the effect is barelly noticeble, but after a while it takes longer and longer between generations.
Here's the code that runs when someone does double click on a listbox entry:
def lbclick(self, eve):
frm = wx.Frame(None, -1, '')
frm.ShowFullScreen(True)
self.sel = self.lstb.GetSelection() # getting the selection from the listbox
def pressk(eve):
keys = eve.GetKeyCode()
if keys == wx.WXK_LEFT:
self.sel = self.sel - 1
if self.sel < 0:
self.sel = len(self.chk) - 1
imgs() # invocking the function made for displaying fullscreen images when left arrow key is pressed
elif keys == wx.WXK_RIGHT:
self.sel = self.sel + 1
if self.sel > len(self.chk) - 1:
self.sel = 0
imgs() # doing the same for the right arrow
elif keys == wx.WXK_ESCAPE:
frm.Destroy()
eve.Skip()
frm.Bind(wx.EVT_CHAR_HOOK, pressk)
def imgs(): # building the function
imgsl = self.chk[self.sel]
itm = wx.Image(str('imgs/{0}'.format(imgsl)), wx.BITMAP_TYPE_JPEG).ConvertToBitmap() # obtaining the name of the image stored in the list self.chk
mar = itm.Size # Because not all images are landscaped I had to figure a method to rescale them after height dimension, which is common to all images
frsz = frm.GetSize()
marx = float(mar[0])
mary = float(mar[1])
val = frsz[1] / mary
vsize = int(mary * val)
hsize = int(marx * val)
panl = wx.Panel(frm, -1, size = (hsize, vsize), pos = (frsz[0] / 2 - hsize / 2, 0)) # making a panel container
panl.SetBackgroundColour('#000000')
imag = wx.Image(str('imgs/{0}'.format(imgsl)), wx.BITMAP_TYPE_JPEG).Scale(hsize, vsize, quality = wx.IMAGE_QUALITY_NORMAL).ConvertToBitmap()
def destr(eve): # unprofessionaly trying to destroy the panel container when a new image is generated hopeing the unvanted effect, with previous generated images will disappear. But it doesn't.
keycd = eve.GetKeyCode()
if keycd == wx.WXK_LEFT or keycd == wx.WXK_RIGHT:
try:
panl.Destroy()
except:
pass
eve.Skip()
panl.Bind(wx.EVT_CHAR_HOOK, destr) # the end of my futile tries
if vsize > hsize: # if the image is portrait instead of landscaped I have to put a black image as a container, otherwise in the background the previous image will remain, even if I use Refresh() on the container (the black bitmap named fundal)
intermed = wx.Image('./res/null.jpg', wx.BITMAP_TYPE_JPEG).Scale(frsz[0], frsz[1]).ConvertToBitmap()
fundal = wx.StaticBitmap(frm, 101, intermed)
stimag = wx.StaticBitmap(fundal, -1, imag, size = (hsize, vsize), pos = (frsz[0] / 2 - hsize / 2, 0))
fundal.Refresh()
stimag.SetToolTip(wx.ToolTip('Esc = exits fullscreen\n<- -> arrows = quick navigation'))
def destr(eve): # the same lame attempt to destroy the container in the portarit images situation
keycd = eve.GetKeyCode()
if keycd == wx.WXK_LEFT or keycd == wx.WXK_RIGHT:
try:
fundal.Destroy()
except:
pass
eve.Skip()
frm.Bind(wx.EVT_CHAR_HOOK, destr)
else: # the case when the images are landscape
stimag = wx.StaticBitmap(panl, -1, imag)
stimag.Refresh()
stimag.SetToolTip(wx.ToolTip('Esc = exits fullscreen\n<- -> arrows = quick navigation'))
imgs() # invocking the function imgs for the situation when someone does double click
frm.Center()
frm.Show(True)
Thanks for any advice in advance.
Added later:
The catch is I'm trying to do an autorun presentation for a DVD with lots of images on it. Anyway it's not necessarely to make the above piece of code work properly if there are any other options. I've already tried to use windows image viewer, but strangely enough it doesn't recognizes relative paths and when I do this
path = os.getcwd() # getting the path of the current working directory
sel = listbox.GetSelection() # geting the value of the current selection from the list box
imgname = memlist[sel] # retrieving the name of the images stored in a list, using the listbox selection, that uses as choices the same list
os.popen(str('rundll32.exe C:\WINDOWS\System32\shimgvw.dll,ImageView_Fullscreen {0}/imgsdir/{1}'.format(path, imgname))) # oepning the images up with bloody windows image viewer
it opens the images only when my program is on the hard disk, if it's on a CD / image drive it doesn't do anything.

There's an image viewer included with the wxPython demo package. I also wrote a really simple one here. Either one should help you in your journey. I suspect that you're not reusing your panels/frames and instead you're seeing old ones that were never properly destroyed.

Crawling web pages with Python

I have a seed file of 250 URLs of IMDB's top 250 movies.
I need to crawl each one of them and get some info from it.
I've created a function that gets a URL of a movie and returns the info I need. It works great. My problem is when I'm trying to run this function on all of the 250 URLs.
After a certian amount (not constant!) of URLs that were crawled successfully, the program stops its run. The python.exe process takes 0% CPU and the memory consumption doesn't change. After some debugging, I figured that the problem is with the parsing, it just stops working and I have no idea why (stuck on a find command).
I'm using urllib2 to get the HTML content of the URL, than parse it as a string and then continue to the next URL (I'm going only once on each of these strings, linear time for all the checks and extractions).
Any idea what can cause this kind of behavior?
EDIT:
I'm attaching one of the problematic functions' code (got 1 more, but I'm guessing it's the same problem)
def getActors(html,actorsDictionary):
counter = 0
actorsLeft = 3
actorFlag = 0
imdbURL = "http://www.imdb.com"
for line in html:
# we have 3 actors, stop
if (actorsLeft == 0):
break
# current line contains actor information
if (actorFlag == 1):
endTag = str(line).find('/" >')
endTagA = str(line).find('</a>')
if (actorsLeft == 3):
actorList = str(line)[endTag+7:endTagA]
else:
actorList += ", " + str(line)[endTag+7:endTagA]
actorURL = imdbURL + str(line)[str(line).find('href=')+6:endTag]
actorFlag = 0
actorsLeft -= 1
actorsDictionary[actorURL] = str(line)[endTag+7:endTagA]
# check if next line contains actor information
if (str(line).find('<td class="name">') > -1 ):
actorFlag = 1
# convert commas and clean \n
actorList = actorList.replace(",",", ")
actorList = actorList.replace("\n","")
return actorList
I'm calling the function this way:
for url in seedFile:
moviePage = urllib.request.urlopen(url)
print(getTitleAndYear(moviePage),",",movieURL,",",getPlot(moviePage),getActors(moviePage,actorsDictionary))
This works great without the getActors function
There is no exception raised here (I removed the try and catch for now)
and it's getting stuck in the for loop after some iterations
EDIT 2: if I run only the getActors function, it works well and finishes all the URLs in the seed file (250)

How to parse nagios status.dat file?

I'd like to parse status.dat file for nagios3 and output as xml with a python script.
The xml part is the easy one but how do I go about parsing the file? Use multi line regex?
It's possible the file will be large as many hosts and services are monitored, will loading the whole file in memory be wise?
I only need to extract services that have critical state and host they belong to.
Any help and pointing in the right direction will be highly appreciated.
LE Here's how the file looks:
########################################
# NAGIOS STATUS FILE
#
# THIS FILE IS AUTOMATICALLY GENERATED
# BY NAGIOS. DO NOT MODIFY THIS FILE!
########################################
info {
created=1233491098
version=2.11
}
program {
modified_host_attributes=0
modified_service_attributes=0
nagios_pid=15015
daemon_mode=1
program_start=1233490393
last_command_check=0
last_log_rotation=0
enable_notifications=1
active_service_checks_enabled=1
passive_service_checks_enabled=1
active_host_checks_enabled=1
passive_host_checks_enabled=1
enable_event_handlers=1
obsess_over_services=0
obsess_over_hosts=0
check_service_freshness=1
check_host_freshness=0
enable_flap_detection=0
enable_failure_prediction=1
process_performance_data=0
global_host_event_handler=
global_service_event_handler=
total_external_command_buffer_slots=4096
used_external_command_buffer_slots=0
high_external_command_buffer_slots=0
total_check_result_buffer_slots=4096
used_check_result_buffer_slots=0
high_check_result_buffer_slots=2
}
host {
host_name=localhost
modified_attributes=0
check_command=check-host-alive
event_handler=
has_been_checked=1
should_be_scheduled=0
check_execution_time=0.019
check_latency=0.000
check_type=0
current_state=0
last_hard_state=0
plugin_output=PING OK - Packet loss = 0%, RTA = 3.57 ms
performance_data=
last_check=1233490883
next_check=0
current_attempt=1
max_attempts=10
state_type=1
last_state_change=1233489475
last_hard_state_change=1233489475
last_time_up=1233490883
last_time_down=0
last_time_unreachable=0
last_notification=0
next_notification=0
no_more_notifications=0
current_notification_number=0
notifications_enabled=1
problem_has_been_acknowledged=0
acknowledgement_type=0
active_checks_enabled=1
passive_checks_enabled=1
event_handler_enabled=1
flap_detection_enabled=1
failure_prediction_enabled=1
process_performance_data=1
obsess_over_host=1
last_update=1233491098
is_flapping=0
percent_state_change=0.00
scheduled_downtime_depth=0
}
service {
host_name=gateway
service_description=PING
modified_attributes=0
check_command=check_ping!100.0,20%!500.0,60%
event_handler=
has_been_checked=1
should_be_scheduled=1
check_execution_time=4.017
check_latency=0.210
check_type=0
current_state=0
last_hard_state=0
current_attempt=1
max_attempts=4
state_type=1
last_state_change=1233489432
last_hard_state_change=1233489432
last_time_ok=1233491078
last_time_warning=0
last_time_unknown=0
last_time_critical=0
plugin_output=PING OK - Packet loss = 0%, RTA = 2.98 ms
performance_data=
last_check=1233491078
next_check=1233491378
current_notification_number=0
last_notification=0
next_notification=0
no_more_notifications=0
notifications_enabled=1
active_checks_enabled=1
passive_checks_enabled=1
event_handler_enabled=1
problem_has_been_acknowledged=0
acknowledgement_type=0
flap_detection_enabled=1
failure_prediction_enabled=1
process_performance_data=1
obsess_over_service=1
last_update=1233491098
is_flapping=0
percent_state_change=0.00
scheduled_downtime_depth=0
}
It can have any number of hosts and a host can have any number of services.

Pfft, get yerself mk_livestatus. http://mathias-kettner.de/checkmk_livestatus.html

Nagiosity does exactly what you want:
http://code.google.com/p/nagiosity/

Having shamelessly stolen from the above examples,
Here's a version build for Python 2.4 that returns a dict containing arrays of nagios sections.
def parseConf(source):
conf = {}
patID=re.compile(r"(?:\s*define)?\s*(\w+)\s+{")
patAttr=re.compile(r"\s*(\w+)(?:=|\s+)(.*)")
patEndID=re.compile(r"\s*}")
for line in source.splitlines():
line=line.strip()
matchID = patID.match(line)
matchAttr = patAttr.match(line)
matchEndID = patEndID.match( line)
if len(line) == 0 or line[0]=='#':
pass
elif matchID:
identifier = matchID.group(1)
cur = [identifier, {}]
elif matchAttr:
attribute = matchAttr.group(1)
value = matchAttr.group(2).strip()
cur[1][attribute] = value
elif matchEndID and cur:
conf.setdefault(cur[0],[]).append(cur[1])
del cur
return conf
To get all Names your Host which have contactgroups beginning with 'devops':
nagcfg=parseConf(stringcontaingcompleteconfig)
hostlist=[host['host_name'] for host in nagcfg['host']
if host['contact_groups'].startswith('devops')]

Don't know nagios and its config file, but the structure seems pretty simple:
# comment
identifier {
attribute=
attribute=value
}
which can simply be translated to
<identifier>
<attribute name="attribute-name">attribute-value</attribute>
</identifier>
all contained inside a root-level <nagios> tag.
I don't see line breaks in the values. Does nagios have multi-line values?
You need to take care of equal signs within attribute values, so set your regex to non-greedy.

You can do something like this:
def parseConf(filename):
conf = []
with open(filename, 'r') as f:
for i in f.readlines():
if i[0] == '#': continue
matchID = re.search(r"([\w]+) {", i)
matchAttr = re.search(r"[ ]*([\w]+)=([\w\d]*)", i)
matchEndID = re.search(r"[ ]*}", i)
if matchID:
identifier = matchID.group(1)
cur = [identifier, {}]
elif matchAttr:
attribute = matchAttr.group(1)
value = matchAttr.group(2)
cur[1][attribute] = value
elif matchEndID:
conf.append(cur)
return conf
def conf2xml(filename):
conf = parseConf(filename)
xml = ''
for ID in conf:
xml += '<%s>\n' % ID[0]
for attr in ID[1]:
xml += '\t<attribute name="%s">%s</attribute>\n' % \
(attr, ID[1][attr])
xml += '</%s>\n' % ID[0]
return xml
Then try to do:
print conf2xml('conf.dat')

If you slightly tweak Andrea's solution you can use that code to parse both the status.dat as well as the objects.cache
def parseConf(source):
conf = []
for line in source.splitlines():
line=line.strip()
matchID = re.match(r"(?:\s*define)?\s*(\w+)\s+{", line)
matchAttr = re.match(r"\s*(\w+)(?:=|\s+)(.*)", line)
matchEndID = re.match(r"\s*}", line)
if len(line) == 0 or line[0]=='#':
pass
elif matchID:
identifier = matchID.group(1)
cur = [identifier, {}]
elif matchAttr:
attribute = matchAttr.group(1)
value = matchAttr.group(2).strip()
cur[1][attribute] = value
elif matchEndID and cur:
conf.append(cur)
del cur
return conf
It is a little puzzling why nagios chose to use two different formats for these files, but once you've parsed them both into some usable python objects you can do quite a bit of magic through the external command file.
If anybody has a solution for getting this into a a real xml dom that'd be awesome.

For the last several months I've written and released a tool that that parses the Nagios status.dat and objects.cache and builds a model that allows for some really useful manipulation of Nagios data. We use it to drive an internal operations dashboard that is a simplified 'mini' Nagios. Its under continual development and I've neglected testing and documentation but the code isn't too crazy and I feel fairly easy to follow.
Let me know what you think...
https://github.com/zebpalmer/NagParser