python and redis, unclear syntax - python

This is copy of the code in mining the social web book.
I am a new in this field and with redis too. I want to understand what does $ mean in this context. Also the print with %s, What does it mean?
This is the source code below (from: https://github.com/ptwobrussell/Mining-the-Social-Web):
import sys
import redis
from twitter__util import getRedisIdByScreenName
# A pretty-print function for numbers
from twitter__util import pp
r = redis.Redis()
screen_names=['user1','user2']
def friendsFollowersInCommon(screen_names):
r.sinterstore('temp$friends_in_common',
[getRedisIdByScreenName(screen_name, 'friend_ids')
for screen_name in screen_names]
)
r.sinterstore('temp$followers_in_common',
[getRedisIdByScreenName(screen_name, 'follower_ids')
for screen_name in screen_names]
)
print 'Friends in common for %s: %s' % (', '.join(screen_names),
pp(r.scard('temp$friends_in_common')))
print 'Followers in common for %s: %s' % (', '.join(screen_names),
pp(r.scard('temp$followers_in_common')))
# Clean up scratch workspace
r.delete('temp$friends_in_common')
r.delete('temp$followers_in_common')
if __name__ == "__main__":
if len(screen_names) < 2:
print >> sys.stderr, "Please supply at least two screen names."
sys.exit(1)
friendsFollowersInCommon(screen_names[1:])

$ symbol is just a part of key name. It separates name parts. I usually use : for the same purpose (e.g. users:123)
%s part is python's string formatting.

Related

Python script to check bond status for Linux

I have the Below python code which I'm using to determine the Linux bond/team status. This code works just fine. I am not good at aligning the output formatting thus getting little hiccup.
I wanted the Printing Format into a Certain format, would appreciate any help on the same.
Below is the code exercise:
#!/usr/bin/python
# Using below file to process the data
# cat /proc/net/bonding/bond0
import sys
import re
def usage():
print '''USAGE: %s [options] [bond_interface]
Options:
--help, -h This usage document
Arguments:
bond_interface The bonding interface to query, eg. 'bond0'. Default is 'bond0'.
''' % (sys.argv[0])
sys.exit(1)
# Parse arguments
try:
iface = sys.argv[1]
if iface in ('--help', '-h'):
usage()
except IndexError:
iface = 'bond0'
# Grab the inf0z from /proc
try:
bond = open('/proc/net/bonding/%s' % iface).read()
except IOError:
print "ERROR: Invalid interface %s\n" % iface
usage()
# Parse and output
active = 'NONE'
Link = 'NONE'
slaves = ''
state = 'OK'
links = ''
bond_status = ''
for line in bond.splitlines():
m = re.match('^Currently Active Slave: (.*)', line)
if m:
active = m.groups()[0]
m = re.match('^Slave Interface: (.*)', line)
if m:
s = m.groups()[0]
slaves += ', %s' % s
m = re.match('^Link Failure Count: (.*)', line)
if m:
l = m.groups()[0]
links += ', %s' % l
m = re.match('^MII Status: (.*)', line)
if m:
s = m.groups()[0]
if slaves == '':
bond_status = s
else:
slaves += ' %s' % s
if s != 'up':
state = 'FAULT'
print "%s %s (%s) %s %s %s" % (iface, state, bond_status, active, slaves, links)
Result:
$ ./bondCheck.py
bond0 OK (up) ens3f0 , ens3f0 up, ens3f1 up , 0, 0
Expected:
bond0: OK (up), Active Slave: ens3f0 , PriSlave: ens3f0(up), SecSlave: ens3f1(up) , LinkFailCountOnPriInt: 0, LinkFailCountOnSecInt: 0
I tried to format in a very basic way as shown below :
print "%s: %s (%s), Active Slave: %s, PriSlave: %s (%s), SecSlave: %s (%s), LinkFailCountOnPriInt: %s, LinkFailCountOnSecInt: %s" % (iface, state, bond_status, active, slaves.split(',')[1].split()[0], slaves.split(',')[1].split()[1], slaves.split(',')[2].split()[0], slaves.split(',')[2].split()[1], links.split(',')[1], links.split(',')[2])
RESULT:
bond0: OK (up), Active Slave: ens3f0, PriSlave: ens3f0 (up), SecSlave: ens3f1 (up), LinkFailCountOnPriInt: 1, LinkFailCountOnSecInt: 1
However, I would suggest to get the values into variables prior and then use them in the print statement so as to avoid "out of index" issues during print() , as in rare cases like bond with only one interface will report indexing error while splitting hence good to get the values in variable and suppress the out of index into exception for those cases.
Do not use the way with "/proc/net/bonding/%s' for querying bond status. It could trigger system panic. Try to use "/sys/class/net/bondX/bonding", it is more safe.

HSM using label of key object in PKCS11

This block of code is loading a cryptoki.so library and retrieving slot info. This is getting a list of objects in slot num. 0. I don't need to access all the keys to perform a few functions, just a specific key pair. Is there a way to get a single desired token by using the label name, object ID, or handle?
pkcs11 = PyKCS11.PyKCS11Lib()
pkcs11.load(lib)
pkcs11.initialize()
info = pkcs11.getInfo()
i = pkcs11.getSlotInfo(0)
pkcs11.openSession(0)
print "Library manufacturerID: " + info.manufacturerID
slots = pkcs11.getSlotList()
print "Available Slots:", len(slots)
for s in slots:
try:
i = pkcs11.getSlotInfo(s)
print "Slot no:", s
print format_normal % ("slotDescription", i.slotDescription.strip())
print format_normal % ("manufacturerID", i.manufacturerID.strip())
t = pkcs11.getTokenInfo(s)
print "TokenInfo"
print format_normal % ("label", t.label.strip())
print format_normal % ("manufacturerID", t.manufacturerID.strip())
print format_normal % ("model", t.model.strip())
session = pkcs11.openSession(s)
print "Opened session 0x%08X" % session.session.value()
if pin_available:
try:
session.login(pin=pin)
except:
print "login failed, exception:", str(sys.exc_info()[1])
objects = session.findObjects()
print
print "Found %d objects: %s" % (len(objects), [x.value() for x in objects])
The specific script i'm running only has a few commands defined, such as -pin --sign --decrypt --lib do i need to define a common pkcs11-tool such as --init-token or --token-label to pass it as an argument when executing my script ? Or can i directly assign a variable to the desired LabelName within the python script?
So from command line i'm running
$./Test.py --pin=pass and getting the following
Library manufacturerID: Safenet, Inc.
Available Slots: 4
Slot no: 0
slotDescription: ProtectServer K5E:00045
manufacturerID: SafeNet Inc.
TokenInfo
label: CKM
manufacturerID: SafeNet Inc.
model: K5E:PL25
Opened session 0x00000002
Found 52 objects: [5021, 5022, 5014, 5016, 4, 5, 6, 7, 8, 9, 16, 18, 23, 24, 26, 27, 29, 30, 32, 33, 35, 36, 38, 39, 5313, 5314, 4982, 5325, 5326, 5328, 5329, 5331, 5332, 5335, 5018, 4962, 5020, 4963, 5357, 5358, 5360, 5361, 5363, 5364, 5366, 5367, 5369, 5370, 5372, 5373, 5375, 5376]
I'm ultimately trying to get only one of these objects to run some tests.For instance objectID = 201603040001 contains a private/cert file. I want to specify this particular handle. The actual label is something like 000103...3A0. How can I define this so I don't get the rest of the objects inside the library.
Here's the a list of just a couple of the HSM objects
HANDLE LABEL TYPE OBJECT-ID
5314 00000103000003A1 X509PublicKeyCertificate 201603040001
5313 00000103000003A1 RSAPrivateKey 201603040001
I'm trying to pull just one of the Labels.
Here's the defined usage
def usage():
print "Usage:", sys.argv[0],
print "[-p pin][--pin=pin]",
print "[-s slot][--slot=slot]",
print "[-c lib][--lib=lib]",
print "[-h][--help]",
print "[-o][--opensession]"
try:
opts, args = getopt.getopt(sys.argv[1:], "p:c:Sd:h:s", ["pin=", "lib=", "sign", "decrypt", "help","slot="])
except getopt.GetoptError:
# print help information and exit:
usage()
sys.exit(2)
I'm not aware of how I can add an argument so I can use --sign with just a specific label. End game I want to use $./Test.py --pin=pass --sign --label "00000103000003A4" or by handle $./Test.py --pin=pass --sign --handle=5313
UPDATED from suggested comments below. still having trouble getting attributes for rsa private key and cert. Using the specific token has worked but those objects inside of it are returning bad attribute types
t = pkcs11.getTokenInfo(s)
print "TokenInfo"
if 'CKM' == t.label.decode('ascii').strip():
tokenInfo = pkcs11.getTokenInfo(slot)
if '00000103000003A1' == tokenInfo.label.decode('ascii').strip():
print format_normal % ("label", t.label.strip())
print format_normal % ("manufacturerID", t.manufacturerID.strip())
print format_normal % ("model", t.model.strip())
session = pkcs11.openSession(s)
print("Opened session 0x%08X" % session.session.value())
if pin_available:
try:
if (pin is None) and \
(PyKCS11.CKF_PROTECTED_AUTHENTICATION_PATH & t.flags):
print("\nEnter your PIN for %s on the pinpad" % t.label.strip())
session.login(pin=pin)
except:
print("login failed, exception:", str(sys.exc_info()[1]))
break
objects = session.findObjects([(CKA_LABEL, "00000103000003A4")])
print()
print("Found %d objects: %s" % (len(objects), [x.value() for x in objects]))
all_attributes = list(PyKCS11.CKA.keys())
# only use the integer values and not the strings like 'CKM_RSA_PKCS'
all_attributes.remove(PyKCS11.CKA_PRIVATE_EXPONENT)
all_attributes.remove(PyKCS11.CKA_PRIME_1)
all_attributes.remove(PyKCS11.CKA_PRIME_2)
all_attributes.remove(PyKCS11.CKA_EXPONENT_1)
all_attributes.remove(PyKCS11.CKA_EXPONENT_2)
all_attributes.remove(PyKCS11.CKA_COEFFICIENT)
all_attributes = [e for e in all_attributes if isinstance(e, int)]
n_obj = 1
for o in objects:
print()
print((red + "==================== Object: %d ====================" + normal) % o.value())
n_obj += 1
try:
attributes = session.getAttributeValue(o, all_attributes)
except PyKCS11.PyKCS11Error as e:
continue
attrDict = dict(zip(all_attributes, attributes))
if attrDict[PyKCS11.CKA_CLASS] == PyKCS11.CKO_PRIVATE_KEY \
and attrDict[PyKCS11.CKA_KEY_TYPE] == PyKCS11.CKK_RSA:
m = attrDict[PyKCS11.CKA_MODULUS]
e = attrDict[PyKCS11.CKA_PUBLIC_EXPONENT]
if m and e:
mx = eval(b'0x' + ''.join("%02X" %c for c in m))
ex = eval(b'0x' + ''.join("%02X" %c for c in e))
if sign:
try:
toSign = "12345678901234567890123456789012" # 32 bytes, SHA256 digest
print("* Signing with object 0x%08X following data: %s" % (o.value(), toSign))
signature = session.sign(o, toSign)
sx = eval(b'0x' + ''.join("%02X" % c for c in signature))
print("Signature:")
print(dump(''.join(map(chr, signature))))
if m and e:
print("Verifying using following public key:")
print("Modulus:")
print(dump(''.join(map(chr, m))))
print("Exponent:")
print(dump(''.join(map(chr, e))))
decrypted = pow(sx, ex, mx) # RSA
print("Decrypted:")
d = binascii.unhexlify(hexx(decrypted))
print(dump(d))
if toSign == d[-20:]:
print("*** signature VERIFIED!\n")
the following is what prints. nothing seems to be working using the specific objects, there are no error messages
Slot no: 0
slotDescription: ProtectServer K5E:00045
manufacturerID: SafeNet Inc.
TokenInfo
Opened session 0x00000002
Found 2 objects: [5328, 5329]
==================== Object: 5328 ====================
==================== Object: 5329 ====================
You can work only with one token by checking its label before use, e.g.:
tokenInfo = pkcs11.getTokenInfo(slot)
if 'DesiredTokenLabel' == tokenInfo.label.decode('ascii').strip():
# Start working with this particular token
session = pkcs11.openSession(s)
You can enumerate only specific object using a template argument for the findObjects call, e.g.:
# get objects labelled "PRIV"
objects = session.findObjects([(CKA_LABEL, "PRIV")])
# get all private key objects
objects = session.findObjects([(CKA_CLASS, CKO_PRIVATE_KEY)])
# get all private key objects labelled "PRIV"
objects = session.findObjects([(CKA_CLASS, CKO_PRIVATE_KEY),(CKA_LABEL, "PRIV")])
# get all RSA private key objects labelled "PRIV"
objects = session.findObjects([(CKA_CLASS, CKO_PRIVATE_KEY),(CKA_KEY_TYPE, CKK_RSA),(CKA_LABEL, "PRIV")])
Below is an example code with hard-coded parameters:
from PyKCS11 import *
pkcs11 = PyKCS11.PyKCS11Lib()
pkcs11.load("your module path...")
slots = pkcs11.getSlotList()
for s in slots:
t = pkcs11.getTokenInfo(s)
if 'CKM' == t.label.decode('ascii').strip():
session = pkcs11.openSession(s)
objects = session.findObjects([(CKA_LABEL, "00000103000003A1")])
print ("Found %d objects: %s" % (len(objects), [x.value() for x in objects]))
Please note that it is Python 3 as I can't use PyKCS11 in Python 2.x right now.
Soma additional (random) notes:
do not rely on handles -- they may (and will) be different for different program runs
your program's command line arguments are up to you -- you must decide yourself if your program needs arguments like --token-label
Disclaimer: I am not that into python so please do validate my thoughts
Good luck!
EDIT(Regarding you recent edit)>
No error is (most probably) shown because the exception is caught and ignored:
try:
attributes = session.getAttributeValue(o, all_attributes)
except PyKCS11.PyKCS11Error as e:
continue

handling python networkx keyerror using graphml

I have a problem with my networkx code. I'm trying to list the attributes of my edges but I keep getting KeyError or when I try to catch the error it skips the entire code.
Background:
I have a graph that has the edges representing firewall rules with custom properties for the protocol, source port, destination port and the edge labels contains the action(allow or deny) while the nodes between the edges are the source and destination address...
The default value for the protocol, source and destination port is IP, any, any...
What Im trying to do:
Im trying to list all the edges (rules) and their respective properties one by one for example:
Rule has PROTOCOL: tcp ACTION: allow sPORT: 6667 dPORT: 6667
Problem: I use default values for protocol, source port and destination ports in some instances and hence the values do no show even if I try
for u,v,n in G.edges_iter(data=True):
print ('%s %s %s' % (u,v,n))
and when I try
print ('Rule has PROTOCOL: %s ACTION: %s sPORT: %s dPORT: %s' % (n['protocol'],n['label'], n['s_port'], n['d_port']))
I get a KeyError raised because in some edges I do not have either protocol or s_port or d_port declared. And when i try using try and except(continue) on the KeyError it just breaks out of the for loop and goes on to execute the remaining section of my code.
Question: how do I handle the KeyError or enter something like "any" or"None" or "Not declared" when an edge attribute is not available, that i want my code to print something like:
Rule has PROTOCOL: tcp ACTION: allow sPORT: any dPORT: any
When Im using default values for source and destination ports instead of skipping the code
This is what my code looks like:
import os
import sys
import string
import networkx as nx
G = nx.read_graphml("fwha.graphml")
count = 0
edges = G.number_of_edges()
nodes = G.number_of_nodes()
print "Number of edges in diagram is: ", edges
for u,v,n in G.edges_iter(data=True):
try:
print ('Rule has PROTOCOL: %s ACTION: %s sPORT: %s dPORT: %s' % (n['protocol'], n['label'], n['s_port'], n['d_port']))
except KeyError:
continue
print "Number of nodes in diagram is: ", nodes
for m,k in G.nodes_iter(data=True):
print('%s %s' % (k,m))
It's just a regular python dictionary... you can do a series of things like
n.get('protocol', 'any'),
Which is a lot of typing, so you can do it all at once after the graph is initialized:
for v in G.node:
G.node[v].setdefault('protocol', 'any')
etc. then walk as normal.
Well I tried this and even though its not too neat and can use functions to tidy it up, it works for now so :)
for u,v,n in G.edges_iter(data=True):
try:
if not n['s_port']:
print "No Problem here"
except KeyError:
n['s_port'] = 'any'
for u,v,n in G.edges_iter(data=True):
try:
if not n['d_port']:
print "No Problem here"
except KeyError:
n['d_port'] = 'any'
for u,v,n in G.edges_iter(data=True):
try:
count = count + 1
print ('Rule %d - Protocol: %s Action: %s SourcePort: %s Destination Port: %s' % (count,n['protocol'], n['label'], n['s_port'], n['d_port']))
except KeyError, e:
count = count+1
print ('Number %s: It didnt execute and the error is %s' % (count, str(e)))

Python IRC Bot: Set variables from channel reading

I'm working on a simple IRC bot and I'm trying to create a timer highlight function.
When I enter the following:
!hl
10
I want to assign 10 (or whatever might be there in it's place) as a variable called 'var_time' in minutes.
var_time= 0 #External statement
def timer_commands(nick,channel,message):
global var_time
if message.find("!hl", var_time )!=-1:
ircsock.send('PRIVMSG %s :%s I will highlight you in %s minutes!\r\n' % (channel,nick, var_time))
time.sleep(float(var_time)) #The delay here is in seconds
ircsock.send('PRIVMSG %s :%s, You asked me %s minutes ago to highlight you.!\r\n' % (channel,nick,var_time))
I know var_time is not taking the value 10, that is precisely my question, how can I make that happen?
Here is how the function is called:
while 1:
ircmsg = ircsock.recv(2048) # receive data from the server
ircmsg = ircmsg.strip('\n\r') # removing any unnecessary linebreaks.
ircraw = ircmsg.split(' ')
print(ircmsg) # Here we print what's coming from the server
if ircmsg.find(' PRIVMSG ')!=-1:
nick=ircmsg.split('!')[0][1:]
channel=ircmsg.split(' PRIVMSG ')[-1].split(':')[0]
timer_commands(nick,channel,ircmsg)
Thanks in advance.
Solution:
def timer_commands(nick,channel,message):
if ircraw[3] == ':!hl':
var_time = float(ircraw[4])
ircsock.send('PRIVMSG %s :I will highlight you in %s minutes!\r\n' % (channel, nick, ircraw[4]))
time.sleep(var_time*60) #Delay in Minutes
ircsock.send('PRIVMSG %s :%s pYou asked me %s minutes ago to highlight you.! \r\n' % (channel, nick, ircraw[4]))
Thanks Anonymous
try regular expressions:
>>> re.match('!hl ([0-9]+)$', '!hl 91445569').groups()[0]
'91445569'
>>> re.match('!hl ([0-9]+)$', '!hl 1').groups()[0]
'1'
or, in your code:
import re
m = re.match('!hl ([0-9]+)$', ircmsg)
if m is not None:
var_time = int(re.groups()[0])

How check application verision information in PySide

I created EXE file with Python (PySide) + PyInstaller. Once I try to use
print QtGui.QApplication.applicationVersion()
I don't see valid version in x.x.x.x format of the application.
Are there any built-in functions in PySide instead of this, or maybe should I use other library for it?
PS. I don't believe that Python doesn't have any methods to extract information about EXE :)
Have a look here.
There's everything you're looking for !
Hope this helps.
Found one way to do it:
import win32api
def get_version_info():
try:
filename = APP_FILENAME
return get_file_properties(filename)['FileVersion']
except BaseException, err:
log_error(err.message)
return '0.0.0.0'
def get_file_properties(filename):
"""
Read all properties of the given file return them as a dictionary.
"""
propNames = ('Comments', 'InternalName', 'ProductName',
'CompanyName', 'LegalCopyright', 'ProductVersion',
'FileDescription', 'LegalTrademarks', 'PrivateBuild',
'FileVersion', 'OriginalFilename', 'SpecialBuild')
props = {'FixedFileInfo': None, 'StringFileInfo': None, 'FileVersion': '0.0.0.0'}
try:
# backslash as parm returns dictionary of numeric info corresponding to VS_FIXEDFILEINFO struc
print filename
fixedInfo = win32api.GetFileVersionInfo(filename, '\\')
props['FixedFileInfo'] = fixedInfo
props['FileVersion'] = "%d.%d.%d.%d" % (fixedInfo['FileVersionMS'] / 65536,
fixedInfo['FileVersionMS'] % 65536, fixedInfo['FileVersionLS'] / 65536,
fixedInfo['FileVersionLS'] % 65536)
# \VarFileInfo\Translation returns list of available (language, codepage)
# pairs that can be used to retreive string info. We are using only the first pair.
lang, codepage = win32api.GetFileVersionInfo(filename, '\\VarFileInfo\\Translation')[0]
# any other must be of the form \StringfileInfo\%04X%04X\parm_name, middle
# two are language/codepage pair returned from above
strInfo = {}
for propName in propNames:
strInfoPath = u'\\StringFileInfo\\%04X%04X\\%s' % (lang, codepage, propName)
## print str_info
strInfo[propName] = win32api.GetFileVersionInfo(filename, strInfoPath)
props['StringFileInfo'] = strInfo
except BaseException, err:
log_error(err.message)
return props

Categories