We Keep Coding

Creating Custom varbinds dictionary using pysnmp

I am using sendVarbinds from pysnmp to send traps with custom MIBs
e.g.
def sendTrap(self, trapName, trapObjects):
mibViewController = view.MibViewController(self.snmpEngine.getMibBuilder())
ntfOrg = ntforg.NotificationOriginator()
ntfOrg.snmpContext = self.snmpContext
ntfOrg.sendVarBinds(
self.snmpEngine,
'my-notification', # notification targets
None, '', # contextEngineId, contextName
rfc1902.NotificationType(
trapName,
objects=trapObjects # <-- how to construct this?
).resolveWithMib(mibViewController)
)
The NotificationType requires a dictionary of varbinds , having object key value pair .
I would like to populate the object dictionary key and value from a json object .Right now I am doing it like this for each trap.
def function1():
trapName = rfc1902.ObjectIdentity('MY-MIB', 'ServerPart')
trapObjects = {
('MY-MIB', 'ServerIP'): event.get_ServerIP(),
('MY-MIB', 'ServerPartName'): event.get_ServerPartName(),
('MY-MIB', 'ServerState'): event.get_ServerState()
}
def function2():
trapName = rfc1902.ObjectIdentity('MY-MIB', 'ServerMemory')
trapObjects = {
('MY-MIB', 'ServerIP'): event.get_ServerIP(),
('MY-MIB', 'ServerMemoryAv'): event.get_ServerMemoryAv()
}
But I want to have a single function to create the traps without checking the varbinds and fill them using the JSON object? Say I run a loop on dictionary items().
My flow is like this.
Client --- event JSON object ----> Trap daemon ---- traps -----> Manager
This is an example of the JSON object which comes from client
my_event('ServerPart', '10.22.1.1', '44', '1', host, port)
my_event('ServerMemory', '10.22.1.1', '100MB', host, port)
Is there any example which can be followed?

Each notification type requires its own set of managed objects (trapOpjects) to be included with the notification. I suppose the mapping of event name -> notification type is 1-to-1 and you can re-constract event field getter from field name.
Then what's remaining is to map event and its parameters to notification type and managed objects:
eventParamsToManagedObjectsMap = {
'ServerPart': { # <- notification type
'ServerIP': 'get_ServerIP', # <- trap objects
# e.g. to populate ServerIP,
# call get_serverIP()
# ...
}
...
}
Finally, to build such map you could try something along these lines:
notificationType = 'ServerMemory' # on input you should have all
# event or notification types
trapObjectIdentity = rfc1902.ObjectIdentity('MY-MIB', notificationType)
trapObjectIdentiry.resolveWithMib(mibViewController)
trapObjectNames = trapObjectIdentity.getMibNode().getObjects()
eventParamsToManagedObjectsMap = defaultdict(dict)
for mibName, mibObjectName in trapObjectNames:
eventParamsToManagedObjectsMap[notificationType][mibObjectName] = 'get_' + mibObjectName
Once you have this map, you should be able to build notifications out of events uniformly within a single function.

I guess you can try something like this (untested):
def get_notification(trapName, **kwargs):
return rfc1902.NotificationType(
trapName,
objects={('MY-MIB', k): v for k, v in kwargs.items()}
}
Assuming json_data is already a Python dict (if it is a string call json.loads(json_data) in order to parse it) you can call it like this:
ntfOrg.sendVarBinds(
self.snmpEngine,
'my-notification', # notification targets
None, '', # contextEngineId, contextName
get_notification(**json_data).resolveWithMib(mibViewController)
)

Bulk inserting on table with foreign key field

Given the following models,
class Server(BaseModel):
name = peewee.CharField(unique=True)
class Member(BaseModel):
name = peewee.CharField(unique=True)
server = peewee.ForeignKeyField(Server, related_name='members')
and a dictionary with keys being Server names and values being tuples of Member names,
data = {
'Server01': ('Laurence', 'Rose'),
'Server02': ('Rose', 'Chris'),
'Server03': ('Isaac',)
}
what is the fastest way of bulk inserting Members using peewee? It seems like one should use Model.insert_many() here, but since Member.server expects a Server or Server.id, that would require iterating over data.items() and selecting a Server for each name.
for server_name, member_names in data.items():
server = Server.select().where(Server.name == server_name)
member_data = []
for name in member_names:
member_data.append({'name': name, 'server': server})
with db.atomic():
Member.insert_many(member_data)
Needless to say, this is terribly inefficient. Is there a better way of doing it?

Well, if you don't know ahead of time which servers are present in the DB it seems like your problem is the data-structure you're using. Keeping server_name -> member_names in a dict like that and trying to insert it all in one go is not how relational databases work.
Try this:
server_to_id = {}
for server_name in data:
if server_name not in server_to_id:
server = Server.create(name=server_name)
server_to_id[server_name] = server.id
for server_name, member_names in data.items():
server_id = server_to_id[server_name]
member_data = [{'name': name, 'server': server_id} for name in member_names]
Member.insert_many(member_data).execute()
Note: don't forget to call .execute() when using insert() or insert_many().

Extracting BIND parameters to build a JSON query

I have a file which was exported from BIND containing TSIG values for about 500 domain names. I need to repurpose the data into JSON for a REST API query. The BIND data is formatted like so:
// secondary-example.com.
key "2000000000000.key." {
algorithm hmac-md5;
secret "ahashedvalue=";
};
zone "secondary-example.com." {
type slave;
file "sec/secondary-example.com.";
allow-transfer { 1.1.1.1;
1.1.2.2;
};
also-notify { 1.1.1.1;
2.2.2.2;
};
masters {
1.2.3.4 key 2000000000000.key.;
};
};
From this I need to extract the key, zone and secret. Here's an example API request.
{
"properties":{
"name":"secondary-example.com.",
"accountName":"example",
"type":"SECONDARY"
},
"secondaryCreateInfo":{
"primaryNameServers":{
"nameServerIpList":{
"nameServerIp1":{
"ip":"1.2.3.4",
"tsigKey":"2000000000000.key.",
"tsigKeyValue":"ahashedvalue="
}
}
}
}
}
I'm having difficulty crafting a regular expression appropriate for the scenario. I'm looking construct the JSON in a python script and send the request through Postman.

I spent a couple days reading up on regex and figured out a solution. So, each of those "zones" began with a comment... e.g. "secondary-example.com"... and each set of BIND info was 17 lines long exactly. This solution is hackey and always assumes data is correct, but it managed to work.
Separate the zones into chunks of text.
zones = []
cur_zone = ''
f = open(bind_file).readlines()
for line in f:
if line[0:2] == '//':
zones.append(cur_zone)
cur_zone = ''
else:
cur_zone = cur_zone + line
zones.pop(0) # Drop the first list item, it's empty
Iterate through those chunks and match the needed parameters.
for z in zones:
z_lines = z.splitlines()
# Regex patterns to make the required parameters
key = re.findall('\"(.*)\"', z_lines[0])[0]
secret = re.findall('\"(.*)\"', z_lines[2])[0]
name = re.findall('\"(.*)\"', z_lines[5])[0]
master = re.findall('\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}', z_lines[15])[0]

Python organzing default values as objects

I have a django application that is utilizing a third party API and needs to receive several arguments such as client_id, user_id etc. I currently have these values labeled at the top of my file as variables, but I'd like to store them in an object instead.
My current set up looks something like this:
user_id = 'ID HERE'
client_id = 'ID HERE'
api_key = 'ID HERE'
class Social(LayoutView, TemplateView):
def grab_data(self):
authenticate_user = AuthenticateService(client_id, user_id)
I want the default values set up as an object
SERVICE_CONFIG = {
'user_id': 'ID HERE',
'client_id': 'ID HERE'
}
So that I can access them in my classes like so:
authenticate_user = AuthenticateService(SERVICE_CONFIG.client_id, SERVICE_CONFIG.user_id)
I've tried SERVICE_CONFIG.client_id, and SERVICE_CONFIG['client_id'], as well as setting up the values as a mixin but I can't figure out how to access them any other way.

Python is not Javascript. That's a dictionary, not an object. You access dictionaries using the item syntax, not the attribute syntax:
AuthenticateService(SERVICE_CONFIG['client_id'], SERVICE_CONFIG['user_id'])

You can use a class, an instance, or a function object to store data as properties:
class ServiceConfig:
user_id = 1
client_id = 2
ServiceConfig.user_id # => 1
service_config = ServiceConfig()
service_config.user_id # => 1
service_config = lambda:0
service_config.user_id = 1
service_config.client_id = 2
service_config.user_id # => 1
Normally using a dict is the simplest way to store data, but in some cases higher readability of property access can be preferred, then you can use the examples above. Using a lambda is the easiest way but more confusing for someone reading your code, therefore the first two approaches are preferable.

Django parsing a json request that has a dictionary result in error

I'm passing the following NSMutableDictionary as parameters of AFNetworking from iPhone application (IOS 6.1) to a Django server
NSMutableDictionary *parameters = [[NSMutableDictionary alloc] init];
[parameters setObject:[self getProductDictionary] forKey:#"prodDic"];
[parameters setObject:ApplicationDelegate.userUniqueId forKey:#"userID"];
[manager POST:BaseURLString parameters:parameters success:^(AFHTTPRequestOperation *operation, id responseObject) {...}
-(void) getProductDictionary
{
NSMutableDictionary *products = [[NSMutableDictionary alloc] init];
[products setObject:#"aaa" forKey:#"prod1"];
[products setObject:#"vvv" forKey:#"prod2"];
[products setObject:#"bbb" forKey:#"prod3"];
}
in the server i have the following view:
import json
def foo(request):
if request.method == 'POST':
user_id = request.POST['userID']
products=json.loads(request.POST['prodDic'])
return HttpResponse("Done")
the following line couses the error products=json.loads(request.POST['prodDic']) and the error is: raise MultiValueDictKeyError(repr(key)) MultiValueDictKeyError: "'prodDic'"
for debug I removed the line and added this code:
#for debug
for key in request.POST:
value = request.POST[key]
print value
the key looks like this: prodDic[prod1]
how can I receive the dictionary into products so i can run over it like so:
for key in products:
value = products[key]

Request objects GET and POST can contain multiple values of same key. As some HTML form elements, notably , pass multiple values for the same key.
You are passing the request.POST object to JSON serializer which doesn't allow multiple keys with same name. So please inspect your request.POST data and see if you are getting multiple keys with same name or not. If yes then you have to process this data before passing to JSON load function.
I hope it will give you some pointers to solve the problem

This is what I finally found out... the problem started in objective c, passing the dictionary as is to AFNetworking caused the funny looking keys on the server side so
I've converted the NSDictionary to NSData
Converted the NSData to NSString
created a new dictionary with a "root" dummy key and the converted string as value
Here is my updated working code:
NSMutableDictionary *parameters = [[NSMutableDictionary alloc] init];
[parameters setObject:[self getProductDictionary] forKey:#"prodDic"];
[parameters setObject:ApplicationDelegate.userUniqueId forKey:#"userID"];
NSData *jsonData = [NSJSONSerialization dataWithJSONObject:parameters options:NSJSONWritingPrettyPrinted error:&error];
NSString *jsonString = [[NSString alloc] initWithData:jsonData encoding:NSUTF8StringEncoding];
[manager POST:BaseURLString parameters:[[NSDictionary alloc] initWithObjectsAndKeys:jsonString, #"root",nil] success:^(AFHTTPRequestOperation *operation, id responseObject) {...}
then in the server side:
import json
def foo(request):
if request.method == 'POST':
request_data=json.loads(request.POST['root'])
products=request_data['prodDic']
for key in products:
value=products[key]
#do my stuff
return HttpResponse("Done")
hope this save someone time & agony :)