Python Struct Arrays - python

Currently working on some Python scripts to interact with the Spacwalk\Satellite API. I'm able to return one piece of an array I'm looking for, but not the rest. Below is the API call I'm making in my script. (key) is the session key to authenticate with the server.
duplicates = client.system.listDuplicatesByHostname(key)
Running my script will produce the following kind of output:
print duplicates
[{'hostname': 'host01', 'systems': [{'last_checkin': <DateTime '20131231T14:06:54' at 192a908>, 'systemName': 'host01.example.com', 'systemId': 1000011017}
I can pull out the 'hostname' field using something like this:
for duplicate in duplicates:
print 'Hostname: %s' % ( duplicate.get('hostname')
But I can't retrieve any of the other items. "systems" is apparently a separate array (nested?) within the first array. I'm unsure of how to reference that second "systems" array. The API reference says the output will be in this format:
Returns:
array:
struct - Duplicate Group
string "hostname"
array "systems"
struct - system
int "systemId"
string "systemName"
dateTime.iso8601 "last_checkin" - Last time server successfully checked in
I'm not sure how to pull out the other values such as systemID, systemName. Is this considered a tuple? How would I go about retrieving these values? (I'm very new to Python, I've read about "structs" but haven't found any examples that really made sense to me.) Not necessarily looking for an answer to this exact question, but anywhere someone could point me to examples that clearly explain how to work with these kinds of arrays would be most helpful!!

Inside of the for loop you will have a dictionary called duplicate that contains the keys 'hostname' and 'systems', so duplicate['hostname'] will get the hostname (a string) and duplicate['systems'] will get the systems array.
You can then access an individual element from that systesm array using indexing, for example duplicate['systems'][0] would get the first system. However what you probably want to be doing instead is create a loop like for system in duplicate['systems'], that way you can iterate over each system in order.
Each system you get will be a dictionary that has the keys 'systemId', 'systemName', and 'last_checkin'.
Here is what I imagine the full code might look like:
for duplicate in duplicates:
print 'Hostname: ' + duplicate['hostname']
for system in duplicate['systems']:
print 'System ID: ' + system['systemId']
print 'System Name: ' + system['systemName']
print 'Last Checkin: ' + system['last_checkin']
I would suggest taking a look at the data structures tutorial.

Thanks guys, the input provided helped me figure this out. I got the output I needed using the following:
for duplicate in duplicates:
print 'IP: ' + duplicate['ip']
for system in duplicate['systems']:
print 'SystemID: ', system['systemId'], 'Name: ', system['systemName']

Related

Gremlin for Python returns a list of traversal commands instead of data

I am trying to use an AWS Lambda function w/ Python 3.7 to access my Neptune DB. For a very simple test, I have the following code in my lambda.
g = graph.traversal().withRemote(DriverRemoteConnection('ws://[endpoint]:8182/gremlin','g'))
g.addV('student').property('name', 'Jeffery').property('GPA', 4.0)
students = g.V('student').values('name')
print(numVert)
After trying many different traversals the only value I get from the print statement is
[['V', 'student'], ['values', 'name']] or some similar list representation of the commands I want to execute instead of the data itself (like Jeffrey).
Am I missing some obvious error? I have tried specifying how I want my result with toList which does not help. Thanks!
When using Gremlin from code you need to always end your query with a terminal step such as toList, next or iterate etc. What you are seeing is just the byte code "to string" form of your query/traversal as the query was not actually executed due to the lack of a terminal step. You also need to use hasLabel when you search for the students. The V() step takes an optional list of one or more IDs not a label.
g.addV('student').property('name', 'Jeffery').property('GPA', 4.0).next()
students = g.V().hasLabel('student').values('name').toList()
print(students)
Here is your query run using Gremlin Python
>>> g.addV('student').property('name', 'Jeffery').property('GPA', 4.0).next()
v[9eb98696-d979-c492-ab2d-a36a219bac6c]
>>> students = g.V().hasLabel('student').values('name').toList()
>>> print(students)
['Jeffery']

Python- Insert new values into 'nested' list?

What I'm trying to do isn't a huge problem in php, but I can't find much assistance for Python.
In simple terms, from a list which produces output as follows:
{"marketId":"1.130856098","totalAvailable":null,"isMarketDataDelayed":null,"lastMatchTime":null,"betDelay":0,"version":2576584033,"complete":true,"runnersVoidable":false,"totalMatched":null,"status":"OPEN","bspReconciled":false,"crossMatching":false,"inplay":false,"numberOfWinners":1,"numberOfRunners":10,"numberOfActiveRunners":8,"runners":[{"status":"ACTIVE","ex":{"tradedVolume":[],"availableToBack":[{"price":2.8,"size":34.16},{"price":2.76,"size":200},{"price":2.5,"size":237.85}],"availableToLay":[{"price":2.94,"size":6.03},{"price":2.96,"size":10.82},{"price":3,"size":33.45}]},"sp":{"nearPrice":null,"farPrice":null,"backStakeTaken":[],"layLiabilityTaken":[],"actualSP":null},"adjustmentFactor":null,"removalDate":null,"lastPriceTraded":null,"handicap":0,"totalMatched":null,"selectionId":12832765}...
All I want to do is add in an extra field, containing the 'runner name' in the data set below, into each of the 'runners' sub lists from the initial data set, based on selection_id=selectionId.
So initially I iterate through the full dataset, and then create a separate list to get the runner name from the runner id (I should point out that runnerId===selectionId===selection_id, no idea why there are multiple names are used), this works fine and the code is shown below:
for market_book in market_books:
market_catalogues = trading.betting.list_market_catalogue(
market_projection=["RUNNER_DESCRIPTION", "RUNNER_METADATA", "COMPETITION", "EVENT", "EVENT_TYPE", "MARKET_DESCRIPTION", "MARKET_START_TIME"],
filter=betfairlightweight.filters.market_filter(
market_ids=[market_book.market_id],
),
max_results=100)
data = []
for market_catalogue in market_catalogues:
for runner in market_catalogue.runners:
data.append(
(runner.selection_id, runner.runner_name)
)
So as you can see I have the data in data[], but what I need to do is add it to the initial data set, based on the selection_id.
I'm more comfortable with Php or Javascript, so apologies if this seems a bit simplistic, but the code snippets I've found on-line only seem to assist with very simple Python lists and nothing 'nested' (to me the structure seems similar to a nested array).
As per the request below, here is the full list:
{"marketId":"1.130856098","totalAvailable":null,"isMarketDataDelayed":null,"lastMatchTime":null,"betDelay":0,"version":2576584033,"complete":true,"runnersVoidable":false,"totalMatched":null,"status":"OPEN","bspReconciled":false,"crossMatching":false,"inplay":false,"numberOfWinners":1,"numberOfRunners":10,"numberOfActiveRunners":8,"runners":[{"status":"ACTIVE","ex":{"tradedVolume":[],"availableToBack":[{"price":2.8,"size":34.16},{"price":2.76,"size":200},{"price":2.5,"size":237.85}],"availableToLay":[{"price":2.94,"size":6.03},{"price":2.96,"size":10.82},{"price":3,"size":33.45}]},"sp":{"nearPrice":null,"farPrice":null,"backStakeTaken":[],"layLiabilityTaken":[],"actualSP":null},"adjustmentFactor":null,"removalDate":null,"lastPriceTraded":null,"handicap":0,"totalMatched":null,"selectionId":12832765},{"status":"ACTIVE","ex":{"tradedVolume":[],"availableToBack":[{"price":20,"size":3},{"price":19.5,"size":26.36},{"price":19,"size":2}],"availableToLay":[{"price":21,"size":13},{"price":22,"size":2},{"price":23,"size":2}]},"sp":{"nearPrice":null,"farPrice":null,"backStakeTaken":[],"layLiabilityTaken":[],"actualSP":null},"adjustmentFactor":null,"removalDate":null,"lastPriceTraded":null,"handicap":0,"totalMatched":null,"selectionId":12832767},{"status":"ACTIVE","ex":{"tradedVolume":[],"availableToBack":[{"price":11,"size":9.75},{"price":10.5,"size":3},{"price":10,"size":28.18}],"availableToLay":[{"price":11.5,"size":12},{"price":13.5,"size":2},{"price":14,"size":7.75}]},"sp":{"nearPrice":null,"farPrice":null,"backStakeTaken":[],"layLiabilityTaken":[],"actualSP":null},"adjustmentFactor":null,"removalDate":null,"lastPriceTraded":null,"handicap":0,"totalMatched":null,"selectionId":12832766},{"status":"ACTIVE","ex":{"tradedVolume":[],"availableToBack":[{"price":48,"size":2},{"price":46,"size":5},{"price":42,"size":5}],"availableToLay":[{"price":60,"size":7},{"price":70,"size":5},{"price":75,"size":10}]},"sp":{"nearPrice":null,"farPrice":null,"backStakeTaken":[],"layLiabilityTaken":[],"actualSP":null},"adjustmentFactor":null,"removalDate":null,"lastPriceTraded":null,"handicap":0,"totalMatched":null,"selectionId":12832769},{"status":"ACTIVE","ex":{"tradedVolume":[],"availableToBack":[{"price":18.5,"size":28.94},{"price":18,"size":5},{"price":17.5,"size":3}],"availableToLay":[{"price":21,"size":20},{"price":23,"size":2},{"price":24,"size":2}]},"sp":{"nearPrice":null,"farPrice":null,"backStakeTaken":[],"layLiabilityTaken":[],"actualSP":null},"adjustmentFactor":null,"removalDate":null,"lastPriceTraded":null,"handicap":0,"totalMatched":null,"selectionId":12832768},{"status":"ACTIVE","ex":{"tradedVolume":[],"availableToBack":[{"price":4.3,"size":9},{"price":4.2,"size":257.98},{"price":4.1,"size":51.1}],"availableToLay":[{"price":4.4,"size":20.97},{"price":4.5,"size":30},{"price":4.6,"size":16}]},"sp":{"nearPrice":null,"farPrice":null,"backStakeTaken":[],"layLiabilityTaken":[],"actualSP":null},"adjustmentFactor":null,"removalDate":null,"lastPriceTraded":null,"handicap":0,"totalMatched":null,"selectionId":12832771},{"status":"ACTIVE","ex":{"tradedVolume":[],"availableToBack":[{"price":24,"size":6.75},{"price":23,"size":2},{"price":22,"size":2}],"availableToLay":[{"price":26,"size":2},{"price":27,"size":2},{"price":28,"size":2}]},"sp":{"nearPrice":null,"farPrice":null,"backStakeTaken":[],"layLiabilityTaken":[],"actualSP":null},"adjustmentFactor":null,"removalDate":null,"lastPriceTraded":null,"handicap":0,"totalMatched":null,"selectionId":12832770},{"status":"ACTIVE","ex":{"tradedVolume":[],"availableToBack":[{"price":5.7,"size":149.33},{"price":5.5,"size":29.41},{"price":5.4,"size":5}],"availableToLay":[{"price":6,"size":85},{"price":6.6,"size":5},{"price":6.8,"size":5}]},"sp":{"nearPrice":null,"farPrice":null,"backStakeTaken":[],"layLiabilityTaken":[],"actualSP":null},"adjustmentFactor":null,"removalDate":null,"lastPriceTraded":null,"handicap":0,"totalMatched":null,"selectionId":10064909}],"publishTime":1551612312125,"priceLadderDefinition":{"type":"CLASSIC"},"keyLineDescription":null,"marketDefinition":{"bspMarket":false,"turnInPlayEnabled":false,"persistenceEnabled":false,"marketBaseRate":5,"eventId":"28180290","eventTypeId":"2378961","numberOfWinners":1,"bettingType":"ODDS","marketType":"NONSPORT","marketTime":"2019-03-29T00:00:00.000Z","suspendTime":"2019-03-29T00:00:00.000Z","bspReconciled":false,"complete":true,"inPlay":false,"crossMatching":false,"runnersVoidable":false,"numberOfActiveRunners":8,"betDelay":0,"status":"OPEN","runners":[{"status":"ACTIVE","sortPriority":1,"id":10064909},{"status":"ACTIVE","sortPriority":2,"id":12832765},{"status":"ACTIVE","sortPriority":3,"id":12832766},{"status":"ACTIVE","sortPriority":4,"id":12832767},{"status":"ACTIVE","sortPriority":5,"id":12832768},{"status":"ACTIVE","sortPriority":6,"id":12832770},{"status":"ACTIVE","sortPriority":7,"id":12832769},{"status":"ACTIVE","sortPriority":8,"id":12832771},{"status":"LOSER","sortPriority":9,"id":10317013},{"status":"LOSER","sortPriority":10,"id":10317010}],"regulators":["MR_INT"],"countryCode":"GB","discountAllowed":true,"timezone":"Europe\/London","openDate":"2019-03-29T00:00:00.000Z","version":2576584033,"priceLadderDefinition":{"type":"CLASSIC"}}}
i think i understand what you are trying to do now
first hold your data as a python object (you gave us a json object)
import json
my_data = json.loads(my_json_string)
for item in my_data['runners']:
item['selectionId'] = [item['selectionId'], my_name_here]
the thing is that my_data['runners'][i]['selectionId'] is a string, unless you want to concat the name and the id together, you should turn it into a list or even a dictionary
each item is a dicitonary so you can always also a new keys to it
item['new_key'] = my_value
So, essentially this works...with one exception...I can see from the print(...) in the loop that the attribute is updated, however what I can't seem to do is then see this update outside the loop.
mkt_runners = []
for market_catalogue in market_catalogues:
for r in market_catalogue.runners:
mkt_runners.append((r.selection_id, r.runner_name))
for market_book in market_books:
for runner in market_book.runners:
for x in mkt_runners:
if runner.selection_id in x:
setattr(runner, 'x', x[1])
print(market_book.market_id, runner.x, runner.selection_id)
print(market_book.json())
So the print(market_book.market_id.... displays as expected, but when I print the whole list it shows the un-updated version. I can't seem to find an obvious solution, which is odd, as it seems like a really simple thing (I tried messing around with indents, in case that was the problem, but it doesn't seem to be, its like its not refreshing the market_book list post update of the runners sub list)!

Naming DataFrames iteritively using items from a List + a string

I have a list of names of countries.. and I have a large dataframe where one of the columns is ' COUNTRY ' (yes it has a space before and after the word country) I want to be create smaller DataFrames based on country names
cleaned_df[cleaned_df[' COUNTRY ']==asia_country_list[1]]
seems too long a command to achieve this? It does work though.
Now,
str("%s_data" % (asia_country_list[1]))
gives
'Taiwan_data'
but when I combine the above two:
str("%s_data" % (asia_country_list[1])) = cleaned_df[cleaned_df[' COUNTRY ']==asia_country_list[1]]
I get:
SyntaxError: can't assign to function call
happy to learn other ways as well to achieve this pls.. Thanks vm
I don't think you should do this, but if you really need it :
exec(str("%s_data" % (asia_country_list[1])) +"= cleaned_df[cleaned_df[' COUNTRY ']==asia_country_list[1]]")
should work.
Using a dictionary is likely to solve your problem
D={}
D["%s_data" % (asia_country_list[1]))]=cleaned_df[cleaned_df[' COUNTRY ']==asia_country_list[1]]]
EDIT : the first solution is a bad idea : exec is a dangerous command, if one column is named "del cleaned_df" you will actually execute it, it can get destructive. Typically I am guessing spaces are a problem in your case. It's a bit like SQL injections...

Getting address from gdb.Value

I'm creating a command for gdb using python integration and trying to get the address of a variable. I've got the value already using:
v = gdb.parse_and_eval("var_name")
But that v can serialise in a number of ways depending on the type. For example if it's a char*, it will be displayed as:
>>> print v
0x7f06a2cfd320 "x"
>>> print v.address
None
Ok, so this doesn't work as expected. I tried first dereferencing, then getting the address, but that gives me back the original value:
>>> print v.dereference().address
0x7f06a2cfd320 "x"
How can I get the address part only? The only way I could find so far looks a bit long:
>>> vptr = gdb.lookup_type("void").pointer()
>>> print v.cast(vptr)
0x7f06a2cfd320
Is there some more direct way?
It isn't totally clear to me what you are asking.
In the first part it sounds like you want the address of the variable "var_name". In this case, you want ".address" as you wrote; however, note that not all variables have addresses. For example the variable could be put into a register by the compiler. Given that this attribute's value is None, I would guess this is what happened. (You can use other gdb commands to find out for sure...)
In the second part, though, it sounds like maybe you want the value of the pointer. You can either just use the value directly (things like + work fine on Value); or you can cast it directly to a Python "long" to get the value as a plain old scalar:
number = long(v)

Python splitting values from urllib in string

I'm trying to get IP location and other stuff from ipinfodb.com, but I'm stuck.
I want to split all of the values into new strings that I can format how I want later. What I wrote so far is:
resp = urllib2.urlopen('http://api.ipinfodb.com/v3/ip-city/?key=mykey&ip=someip').read()
out = resp.replace(";", " ")
print out
Before I replaced the string into new one the output was:
OK;;someip;somecountry;somecountrycode;somecity;somecity;-;42.1975;23.3342;+05:00
So I made it show only
OK someip somecountry somecountrycode somecity somecity - 42.1975;23.3342 +05:00
But the problem is that this is pretty stupid, because I want to use them not in one string, but in more, because what I do now is print out and it outputs this, I want to change it like print country, print city and it outputs the country,city etc. I tried checking in their site, there's some class for that but it's for different api version so I can't use it (v2, mine is v3). Does anyone have an idea how to do that?
PS. Sorry if the answer is obvious or I'm mistaken, I'm new with Python :s
You need to split the resp text by ;:
out = resp.split(';')
Now out is a list of values instead, use indexes to access various items:
print 'Country: {}'.format(out[3])
Alternatively, add format=json to your query string and receive a JSON response from that API:
import json
resp = urllib2.urlopen('http://api.ipinfodb.com/v3/ip-city/?format=json&key=mykey&ip=someip')
data = json.load(resp)
print data['countryName']

Categories