I'm using the pyvmomi module to export data from our vCenter. I'm very close to getting the desired output, but my script is only iterating through once. Why?
If I print(d) in the for loop before updating the dictionary, it will print all of the data.
Script Summary:
top_dict = {"data": []}
def get_obj(content, vimtype, name=None):
return [item for item in content.viewManager.CreateContainerView(
content.rootFolder, [vimtype], recursive=True).view]
## MAIN ##
...
content = si.RetrieveContent()
d = {}
idnum = 0
for dc in get_obj(content, vim.Datacenter):
for cluster in get_obj(content, vim.ClusterComputeResource):
for h in cluster.host:
host_wwpn1 = ''
for adaptr in h.config.storageDevice.hostBusAdapter:
if hasattr(adaptr, 'portWorldWideName'):
if host_wwpn1 == '':
host_wwpn1 = (hex(adaptr.portWorldWideName)[2:18])
else:
host_wwpn2 = (hex(adaptr.portWorldWideName)[2:18])
d['id'] = idnum
d['datacenter'] = dc.name
d['cluster'] = cluster.name
d['host'] = h.name
d['pwwn_F1'] = host_wwpn1
d['pwwn_F2'] = host_wwpn2
idnum = idnum + 1
top_dict.update({"data": [d]})
Current Output:
{
"data": [
{
"id": 0,
"datacenter": "MY_DATACENTER",
"cluster": "MY_CLUSTER",
"host": "MY_HOSTNAME",
"pwwn_F1": "XXXXXXXXXXXXXXXX",
"pwwn_F2": "XXXXXXXXXXXXXXXX"
}
]
}
I'm pretty sure your issue is on the very last line of the code you've shown. That line replaces the entire contents of the top_dict with new values. I'm pretty sure you want to be adding your new dictionary d to the list that's inside of top_dict.
Instead, I think you want to be doing top_dict["data"].append(d). You will also need to move the initialization of d at the same level as the append call (so probably between the first two loops, if you keep the last line indented as it is now).
I'm not sure if that last line is indented the correct amount or not (since I don't actually know what your code is supposed to do). Currently, you might set the values in d several times before using them. You may want to indent the last line to be at the same level as the lines setting values in d (the initialization of d should then also be at that level too).
Related
I am trying to get the pagination results of two pages but return is exiting loop and displays only one result from a page.
Is there a way to store or merge them?
def incidents():
m = True
limit = 50
offset = 0
while m == True:
url = f"{URL}/incidents"
params = {
"statuses[]": "resolved",
"include[]" : 'channel',
"limit" : limit,
"offset" : offset,
"total" : "true",
}
r = requests.get(url, headers=headers, params=params)
data = r.json()
offset += 50
print(offset, (r.text))
more = False # Set deliberately for easier understanding
return data
The offset, (r.text) output looks like -
50 {"incidents":[{"incident_number":1,"title":"crit server is on fire" ....
100 {"incidents":[{"incident_number":54,"title":"ghdg","description":"ghdg",....
Return only displays below and not the other one. There should be a way like use a generator for example? So we can merge them both and store in data variable so data can be returned?
100 {"incidents":[{"incident_number":54,....
I believe you could store the results in your own list:
incidents = []
and then
data = r.json()
for element in data['incidents']:
incidents.append(element)
Edited for clarity - that way you're gathering all incidents in a single object.
I'm not sure because you just gave the very start of r.text (is there more than 'incidents' within the result?), but i expect the previous answer to be a bit short; i'd suggest something like
results = []
(before the while) and at the end
data = r.json()
results += data['incidents']
return results
(btw: in your original post, each run through while just set the var "data", so no wonder the return can only deal with the very last part retrieved. But i guess that is just an artifact of you simplification, like the "more=False" would even prevent getting a second page)
I have two dictionaries of data which needs to be compared and fetch the respective data from one dictionary to another:
netname = []
netstatus = []
Dict1:
data1: {
"node1":["id1",["net1","net2"]],
"node2":["id2",["net3","net4"]],
"node3":["id3",["net5","net1"]],
"node4":["id4",["net2","net5"]],
....
....
....
}
Dict2:
data2: {
"detail1":["net1","id1","netone","available"],
"detail2":["net2","id2","nettwo","available"],
"detail3":["net1","id3","netthree","not available"],
"detail4":["net4","id4","netfour","not available"],
"detail5":["net5","id4","netfive","available"],
"detail6":["net2","id2","netsix","available"],
....
....
}
I am trying to get the complete details of each of every node in a tabular format using prettytable:
The code I am trying here is:
for node,values in data1.items():
id = values[0]
networks = values[1]
for network in networks:
if any(any(network in x for x in netlist) for netlist in data2.values()):
if any((network in y for y in data2.values() if y[0] == network and y[1] == id)):
for val in data2.values():
if (val[0] == network and val[1] == id):
nwinfo = netname.append(val[2])
nwstatus = netstatus.append(val[3])
else:
print("node id",id,"is not registered in network",network)
else:
print("Node is not registered in any networks..")
when I executed this code, I am getting false values. Do the any(condition) correct here or do I need to add anything to display the correct values after comparing data1 with data2.
First any() condition in the above script is to check if id of data1 dict is present in entire dict data2
second any() condition in the above script is to check if the id is connected to respective network or not
In the above case, I want to check if the id and the net(n) should be compared properly in dict1 and dict2 and display the respective values.
One way forward would be to reshape your inputs to make the join easier. I would do that with comprehensions based around a new "key" that would be the combination of "id" and "net".
You can uncomment the print statements to get a better look at what the reshaping is doing if you like.
data1 = {
"node1":["id1",["net1","net2"]],
"node2":["id2",["net3","net4"]],
"node3":["id3",["net5","net1"]],
"node4":["id4",["net2","net5"]],
}
data2 = {
"detail1":["net1","id1","netone","available"],
"detail2":["net2","id2","nettwo","available"],
"detail3":["net1","id3","netthree","not available"],
"detail4":["net4","id4","netfour","not available"],
"detail5":["net5","id4","netfive","available"],
"detail6":["net2","id2","netsix","available"],
}
data1_reshaped = [
f"{cell}:{row[0]}"
for row in data1.values() for cell in row[1]
]
#print(data1_reshaped)
data2_reshaped = {
f"{x[0]}:{x[1]}": {"name": x[2], "status": x[3]}
for x in data2.values()
}
#print(data2_reshaped)
netname = []
netstatus = []
## now it is a simple lookup based on an array of keys
for key in data1_reshaped:
match = data2_reshaped.get(key)
if not match:
continue
netname.append(match["name"])
netstatus.append(match["status"])
print(netname)
print(netstatus)
This should give you:
['netone', 'netthree', 'netfive']
['available', 'not available', 'available']
If in the end you want something more like:
{
'netone': 'available',
'netthree': 'not available',
'netfive': 'available'
}
The join step even simpler.
I am trying to grab this data and print into a string of text i am having the worst! issues getting this to work.
Here is the source i am working with to get a better understanding i am working on an envirmental controller and my sonoff switch combined
https://github.com/FirstCypress/LiV/blob/master/software/liv/iotConnectors/sonoff/sonoff.py this code works for two pages once completed so ignore the keys for tempature etc
m = json.loads(content)
co2 = m["Value"]
I need the value of "Value" under the "TaskValues" it should be either a 1 or a 0 in almost any case how would i pulled that key in the right form?
"Sensors":[
{
"TaskValues": [
{"ValueNumber":1,
"Name":"Switch",
"NrDecimals":0,
"Value":0
}],
"DataAcquisition": [
{"Controller":1,
"IDX":0,
"Enabled":"false"
},
{"Controller":2,
"IDX":0,
"Enabled":"false"
},
{"Controller":3,
"IDX":0,
"Enabled":"false"
}],
"TaskInterval":0,
"Type":"Switch input - Switch",
"TaskName":"relias",
"TaskEnabled":"true",
"TaskNumber":1
}
],
"TTL":60000
}
You can get it by
m['Sensors'][0]['TaskValues'][0]['Value']
"Value" is nested in your json, as you've mentioned. To get what you want, you'll need to traverse the parent data structures:
m = json.loads(content)
# This is a list
a = m.get('Sensors')
# This is a dictionary
sensor = a[0]
# This is a list
taskvalue = sensor.get('TaskValues')
# Your answer
value = taskvalue[0].get('Value')
I have a python script that has the following output stored in a variable called jvmData:
Stats name=jvmRuntimeModule, type=jvmRuntimeModule#
{
name=HeapSize, ID=1, description=The total memory (in KBytes) in the Java virtual machine run time., unit=KILOBYTE, type=BoundedRangeStatistic, lowWaterMark=1048576, highWaterMark=1048576, current=1048576, integral=0.0, lowerBound=1048576, upperBound=2097152
name=FreeMemory, ID=2, description=The free memory (in KBytes) in the Java virtual machine run time., unit=KILOBYTE, type=CountStatistic, count=348466
name=UsedMemory, ID=3, description=The amount of used memory (in KBytes) in the Java virtual machine run time., unit=KILOBYTE, type=CountStatistic, count=700109
name=UpTime, ID=4, description=The amount of time (in seconds) that the Java virtual machine has been running., unit=SECOND, type=CountStatistic, count=3706565
name=ProcessCpuUsage, ID=5, description=The CPU Usage (in percent) of the Java virtual machine., unit=N/A, type=CountStatistic, count=0
}
What I would like to do is simply print out name/value pairs for the important parts, which in this case would simply be:
HeapSize=1048576
FreeMemory=348466
UsedMemory=700109
UpTime=3706565
ProcessCpuUsage=0
Im not at all good with python :) The only solution in my head seems very long-winded? Split the lines, throw away first, second and last lines, then loop through each line with different cases (sometimes current, sometimes count) for finding the length of string, etc etc
Perhaps (well definitely) I am missing something some nice function I can use to put these into the equivalent of a java hashmap or something?
The "equivalent of a java HashMap" would be called a dictionary in python. As for how to parse this, just iterate over the lines that contain the data, make a dict of all key/value pairs in the line and have a special case for the HeapSize:
jvmData = "..." #the string holding the data
jvmLines = jvmData.split("\n") #a list of the lines in the string
lines = [l.strip() for l in jvmLines if "name=" in l] #filter all data lines
result = {}
for line in lines:
data = dict(s.split("=") for s in line.split(", "))
#the value is in "current" for HeapSize or in "count" otherwise
value = data["current"] if data["name"] == "HeapSize" else data["count"]
result[data["name"]] = value
As you seem to be stuck on Jython2.1, here's a version that should work with it (obviously untested). Basically the same as above, but with the list comprehension and generator expression replaced by filter and map respectively, and without using the ternary if/else operator:
jvmData = "..." #the string holding the data
jvmLines = jvmData.split("\n") #a list of the lines in the string
lines = filter(lambda x: "name=" in x, jvmLines) #filter all data lines
result = {}
for line in lines:
data = dict(map(lambda x: x.split("="), line.split(", ")))
if data["name"] == "HeapSize":
result[data["name"]] = data["current"]
else:
result[data["name"]] = data["count"]
Try something using find function and small re:
import re
final_map = {}
NAME= 'name='
COUNT= 'count='
HIGHWATERMARK= "highWaterMark="
def main():
with open(r'<file_location>','r') as file:
lines = [line for line in file if re.search(r'^name', line)]
for line in lines:
sub = COUNT if line.find(COUNT) != -1 else HIGHWATERMARK
final_map[line[line.find(NAME)+5:line.find(',')]] = line[line.find(sub)+len(sub):].split(',')[0].strip()
print line[line.find(NAME)+5:line.find(',')]+'='+final_map[line[line.find(NAME)+5:line.find(',')]]
if __name__ == '__main__':
main()
Output:
HeapSize=1048576
FreeMemory=348466
UsedMemory=700109
UpTime=3706565
ProcessCpuUsage=0
I've got a diff file and I want to handle adds/deletions/modifications to update an SQL database.
+NameA|InfoA1|InfoA2
-NameB|InfoB1|InfoB2
+NameB|InfoB3|InfoB2
-NameC|InfoC1|InfoC2
-NameD|InfoD1|InfoD2
-NameE|InfoE1|InfoE2
+NameD|InfoD1|InfoD3
+NameE|InfoE3|InfoE2
With a Python script, I first detect two following lines with a regular expressions to handle modifications like B.
re.compile(r"^-(.+?)\|(.*?)\|(.+?)\n\+(.+?)\|(.*?)\|(.+?)(?:\n|\Z)", re.MULTILINE)
I delete all the matching lines, and then rescan my file and then handle all of them like additions/deletions.
My problem is with lines like D & E. For the moment I treat them like two deletions, then two additions, and I've got consequences of CASCADE DELETE in my SQL database, as I should treat them as modifications.
How can I handle such modifications D & E?
The diff file is generated by a bash script before, I could handle it differently if needed.
Try this:
>>> a = '''
+NameA|InfoA1|InfoA2
-NameB|InfoB1|InfoB2
+NameB|InfoB3|InfoB2
-NameC|InfoC1|InfoC2
-NameD|InfoD1|InfoD2
-NameE|InfoE1|InfoE2
+NameD|InfoD1|InfoD3
+NameE|InfoE3|InfoE2
'''
>>> diff = {}
>>> for row in a.splitlines():
if not row:
continue
s = row.split('|')
name = s[0][1:]
data = s[1:]
if row.startswith('+'):
change = diff.get(name, {'rows': []})
change['rows'].append(row)
change['status'] = 'modified' if change.has_key('status') else 'added'
else:
change = diff.get(name, {'rows': []})
change['rows'].append(row)
change['status'] = 'modified' if change.has_key('status') else 'removed'
diff[name] = change
>>> def print_by_status(status=None):
for item, value in diff.items():
if status is not None and status == value['status'] or status is None:
print '\nStatus: %s\n%s' % (value['status'], '\n'.join(value['rows']))
>>> print_by_status(status='added')
Status: added
+NameA|InfoA1|InfoA2
>>> print_by_status(status='modified')
Status: modified
-NameD|InfoD1|InfoD2
+NameD|InfoD1|InfoD3
Status: modified
-NameE|InfoE1|InfoE2
+NameE|InfoE3|InfoE2
Status: modified,
-NameB|InfoB1|InfoB2
+NameB|InfoB3|InfoB2
In this case you will have dictionary with all collected data with diff status and rows. You can do with current dict whatever you want.