We Keep Coding

Parse JSON value into key-value attributes

I got the following output from my API:
{
'Type': 'Notification',
'MessageId': 'xxx',
'TopicArn': 'xxx',
'Subject': 'xxx',
'Message': 'EventType=Delete, FriendlyType=was deleted, '
'Timestamp=2021-11-08T15:30:45Z, UserId=1111, UserName=me#me.com, '
'IPAddr=(empty), AccountId=22222, AccountName=test-account, '
'ProjectId=test-project',
'Timestamp': '2021-11-08T15:30:46.214Z',
'SignatureVersion': '1'
}
Now I want to access the "Message" variable - once I am in, and get the following output (as already visible in the previous mentioned JSON):
EventType=Delete, FriendlyType=was deleted, Timestamp=2021-11-08T15:30:45Z, UserId=1111, UserName=me#me.com, IPAddr=(empty), AccountId=22222, AccountName=test-account, ProjectId=test-project
How can I now access the keys like EventType, FriendlyType, etc.? I assume that I have to convert this output at first to a valid JSON, but I am currently baffled.

In case that you are not able to receive the Message data as a JSON, a way to handle the situation is convert the message string into a dict <key>:<value>:
message_as_dict = dict(map(lambda var: var.strip().split("=") ,message.split(",")))
NOTICE the .strip() in order to remove the spaces on the beginning of the key.
That shoud create a dictionary with the following structure:
{'EventType': 'Delete', 'FriendlyType': 'was deleted', 'Timestamp': '2021-11-08T15:30:45Z', 'UserId': '1111', 'UserName': 'me#me.com', 'IPAddr': '(empty)', 'AccountId': '22222', 'AccountName': 'test-account', 'ProjectId': 'test-project'}
Then you can access to the values with, for example:
print(message_as_dict["UserName"])
> me#me.com

you can parse your string spliting and then use it to create a dict. Maybe it's not the best solution but it's a simple one.
response = {
'Type': 'Notification',
'MessageId': 'xxx',
'TopicArn': 'xxx',
'Subject': 'xxx',
'Message': 'EventType=Delete, FriendlyType=was deleted, Timestamp=2021-11-08T15:30:45Z, UserId=1111, UserName=me#me.com, IPAddr=(empty), AccountId=22222, AccountName=test-account, ProjectId=test-project',
'Timestamp': '2021-11-08T15:30:46.214Z',
'SignatureVersion': '1'
}
keyVals = [el.split('=') for el in response['Message'].split(', ')]
subdict = {}
for key,val in keyVals:
subdict[key] = val

As mentioned in one of the answer, you can parse your Message string, but I too feel it won't be the best solution. What I noticed is that your JSON is not in proper format. See below for proper JSON you should be getting from your API.
{
"Type": "Notification",
"MessageId": "xxx",
"TopicArn": "xxx",
"Subject": "xxx",
"Message": {
"EventType": "Delete",
"FriendlyType": "was deleted",
"Timestamp": "2021-11-08T15:30:45Z",
"UserId": "1111",
"UserName": "me#me.com",
"IPAddr": "(empty)",
"AccountId": "22222",
"AccountName": "test-account",
"ProjectId": "test-project"
},
"SignatureVersion": "1"
}
Once you are able to get this output, you may further access nested objects. For example, to access FriendlyType from Message, you can simply say, body.Message.FriendlyType. body here means your entire JSON object.

You could do it by splitting the 'Message' string up into (key, value) pairs and constructing a dictionary from them:
from pprint import pprint
output = {'Type': 'Notification',
'MessageId': 'xxx',
'TopicArn': 'xxx',
'Subject': 'xxx',
'Message': 'EventType=Delete, FriendlyType=was deleted, '
'Timestamp=2021-11-08T15:30:45Z, UserId=1111, UserName=me#me.com, '
'IPAddr=(empty), AccountId=22222, AccountName=test-account, '
'ProjectId=test-project',
'Timestamp': '2021-11-08T15:30:46.214Z',
'SignatureVersion': '1'}
msg_dict = dict(pair.split('=') for pair in output['Message'].split(', '))
pprint(msg_dict, sort_dicts=False)
Output:
{'EventType': 'Delete',
'FriendlyType': 'was deleted',
'Timestamp': '2021-11-08T15:30:45Z',
'UserId': '1111',
'UserName': 'me#me.com',
'IPAddr': '(empty)',
'AccountId': '22222',
'AccountName': 'test-account',
'ProjectId': 'test-project'}

How do I read a yaml file into a Jupyter notebook?

I have a file from an Open API Spec that I have been trying to access in a Jupyter notebook. It is a .yaml file. I was able to upload it into Jupyter and put it in the same folder as the notebook I'd like to use to access it. I am new to Jupyter and Python, so I'm sorry if this is a basic question. I found a forum that suggested this code to read the data (in my file: "openapi.yaml"):
import yaml
with open("openapi.yaml", 'r') as stream:
try:
print(yaml.safe_load(stream))
except yaml.YAMLError as exc:
print(exc)
This seems to bring the data in, but it is a completely unstructured stream like so:
{'openapi': '3.0.0', 'info': {'title': 'XY Tracking API', 'version': '2.0', 'contact': {'name': 'Narrativa', 'url': 'http://link, 'email': '}, 'description': 'The XY Tracking Project collects information from different data sources to provide comprehensive data for the XYs, X-Y. Contact Support:'}, 'servers': [{'url': 'link'}], 'paths': {'/api': {'get': {'summary': 'Data by date range', 'tags': [], 'responses': {'200': {'description': 'OK', 'content': {'application/json': {'schema': {'$ref': '#/components/schemas/covidtata'}}}}}, 'operationId': 'get-api', 'parameters': [{'schema': {'type': 'string', 'format': 'date'}, 'in': 'query', 'name': 'date_from', 'description': 'Date range beginig (YYYY-DD-MM)', 'required': True}, {'schema': {'type': 'string', 'format': 'date'}, 'in': 'query', 'name': 'date_to', 'description': 'Date range ending (YYYY-DD-MM)'}], 'description': 'Returns the data for a specific date range.'}}, '/api/{date}': {'parameters': [{'schema': {'type': 'string', 'format': 'date'}, 'name': 'date', 'in': 'path', 'required': True}], 'get': {'summary': 'Data by date', 'tags': [], 'responses': {'200': {'description': 'OK', 'content': {'application/json': {'schema': {'$ref': '#/components/schemas/data'}}}}}, 'operationId': 'get-api-date', 'description': 'Returns the data for a specific day.'}}, '/api/country/{country}': {'parameters': [{'schema': {'type': 'string', 'example': 'spain'}, 'name': 'country', 'in': 'path', 'required': True, 'example': 'spain'}, {'schema': {'type': 'strin
...etc.
I'd like to work through the data for analysis but can't seem to access it correctly. Any help would be extremely appreciated!!! Thank you so much for reading.

What you're seeing in the output is JSON. This is in a machine-readable format which doesn't need human-readable newlines or indentation. You should be able to work with this data just fine in your code.
Alternatively, you may want to consider another parser/emitter such as ruamel.yaml which can make dealing with YAML files considerably easier than the package you're currently importing. Print statements with this package can preserve lines and indentation for better readability.

Process JSON Responses, Editing and Sending

I am working with an API. I get a response from the API which looks like this:
from oauthlib.oauth2 import BackendApplicationClient
from requests.auth import HTTPBasicAuth
from requests_oauthlib import OAuth2Session
auth = HTTPBasicAuth(client_id, client_secret)
client = BackendApplicationClient(client_id=client_id)
oauth = OAuth2Session(client=client)
token = oauth.fetch_token(token_url=token_url, auth=auth)
client = OAuth2Session(client_id, token=token, auto_refresh_url=token_url,token_updater=token_saver)
token_saver = []
device_policy = client.get('{URL}/v1?ids='+ids)
I get this response
[{'id': '',
'name': 'A Name',
'description': '',
'platform_name': 'Windows',
'groups': [],
'enabled': True,
'created_by': 'An Email',
'created_timestamp': '2019-03-28T12:51:30.989736386Z',
'modified_by': 'An Email ,
'modified_timestamp': '2019-11-19T21:14:53.0189419Z',
'settings': {'enforcement_mode': 'MONITOR_ENFORCE',
'end_user_notification': 'SILENT',
'classes': [{'id': 'ANY', 'action': 'FULL_ACCESS', 'exceptions': []},
{'id': 'IMAGING', 'action': 'FULL_ACCESS', 'exceptions': []},
{'id': 'MASS_STORAGE', 'action': 'BLOCK_ALL', 'exceptions': []},
{'id': 'MOBILE', 'action': 'BLOCK_ALL', 'exceptions': []},
{'id': 'PRINTER', 'action': 'FULL_ACCESS', 'exceptions': []},
{'id': 'WIRELESS', 'action': 'BLOCK_ALL', 'exceptions': []}]}}]
In each class there is list for hold exceptions. The API accepts a patch (not really a patch) that if this data is resubmitted with the exception field holding the contents of this function then an exception is accepted.
`
file_info = {
"class": "ANY",
"vendor_name": "",
"product_name": "",
"serial_number": serial_number,
"combined_id": "",
"action": "FULL_ACCESS",
"match_method": "VID_PID_SERIAL"
}
`
The challenge I have is accepting the first document and then adding the exception material to create a this patch request. I can "walk" the document but cannot work how to create a new body text to send. I think I want to do something like this but not using append as this throws an error.
new_walk_json = walk_json.append(['classes'][0]['exceptions']['Test'])

Realised .update function can used.

Send location ( lat & long ) to API AI

I'm using API AI to make a Facebook bot. After sharing my location in Facebook chat bot. I got a JSON like this:
{'message': {'attachments': [{'payload': {'coordinates': {'lat': 52.335001190772,'long': 4.8887078680234}},'title': 'Holiday Inn','type': 'location','url': 'https://www.facebook.com/l.php?u=https%3A%2F%2Fwww.bing.com%2Fmaps%2Fdefault.aspx%3Fv%3D2%26pc%3DFACEBK%26mid%3D8100%26where1%3DDe%2BBoelelaan%2B2%252C%2B1083%2BHJ%2BAmsterdam%252C%2BNetherlands%26FORM%3DFBKPL1%26mkt%3Den-US&h=mAQEt4NIX&s=1&enc=AZN97DQxpVq5xpkZqvgi3bMq2OVJNwWBOXOiIOW4FHx1-kgYHxTPKfFwRkUsl0ibr0K5GAquaEltxBMLGvjxmUbCa1AmptlN85rg4jLhDH6K0g'}],
According to that JSON, I have lat and long value:
payload = message['message']['attachments'][0]['payload']
lat = payload['coordinates']['lat']
long = payload['coordinates']['long']
What I want is to send those values to parameters in API AI. So I wrote a method to post that:
def post_location(self, text, lat, long):
return self.get(
params={
'query': text,
'parameters': {
'latitude': lat,
'longitude': long,
},
'lang': 'en'
}
)
And this is how my get looks like:
def get(self, params):
"""
Get connection with api ai
:rtype: object
"""
print(params)
request = requests.get(
url=self.url, params=params, headers=self.headers)
content = request.content.decode('utf-8')
try:
return json.loads(content)
except ValueError:
print('Invalid JSON')
Finally I call post_location method where I handle the facebook message and give it the values but when I run, only the text ( query ) is sent to API AI.
def _post_location_to_api(message, lat, long):
ai_api = AIApi()
return ai_api.post_location(message, lat, long)
location = _post_location_to_api(message['message']['attachments'][0]['type'], latitude, longitude)
print(location) gives me this:
{'id': 'd4374511-86ce-4ccb-b7b3-e813011a0998', 'sessionId': '00000000-0000-0000-0000-000000000000', 'timestamp': '2016-10-04T11:26:11.613Z', 'result': {'action': 'PlaceSearch', 'actionIncomplete': False, 'score': 0.51, 'contexts': [], 'fulfillment': {'speech': ''}, 'parameters': {'latitude': '', 'longitude': ''}, 'source': 'agent', 'resolvedQuery': 'location', 'metadata': {'intentName': 'I want to eat in Amsterdam', 'intentId': '89f515a6-f723-4df5-96b2-9e0f784747c6', 'webhookUsed': 'true', 'warning': 'IMPORTANT: Please use unique session ID for each client. Otherwise you may experience unstable behaviour.'}}, 'status': {'errorDetails': 'Webhook call failed. Status code 404. Error:404 Not Found', 'errorType': 'partial_content', 'code': 206}}
What did I do wrong?

TypeError: list indices must be integers, not str in Flask App

I am getting error list indices must be integers, not str during the iteration of list meters_info. Why I am getting such error?
Thanks Guys pointing me out, Original application is in django and I am converting it to flask app. In django they are datastructures.SortedDict. How I use same functionality in flask
See this link
from django.utils import datastructures
meters_info = datastructures.SortedDict([
("instance", {
'label': '',
'description': _("Existence of instance"),
}),
("instance:<type>", {
'label': '',
'description': _("Existence of instance <type> "
"(openstack types)"),
}),
("memory", {
'label': '',
'description': _("Volume of RAM"),
}),])
Flask App
def _get_nova_meters_info(self):
meters_info= [
("instance", {
'label': '',
'description': "Existence of instance",
}),
("instance:<type>", {
'label': '',
'description': "Existence of instance <type> (openstack types)",
}),
("memory", {
'label': '',
'description': "Volume of RAM",
}),]
Getting message in console
File "/home/vagrant/api/ceilometer.py", line 137, in _get_nova_meters_info
meters_info[name]=dict(meters_info["instance:<type>"])
TypeError: list indices must be integers, not str
Please see the following method where I am getting error
for flavor in self.get_flavor_names():
name='instance:%s' %flavor
meters_info[name]=dict(meters_info["instance:<type>"])
meters_info[name]['description']= (
'Duration of instance type %s (openstack flavor)' % flavor)
return meters_info
def get_flavor_names(self):
return ['m1.tiny', 'm1.small', 'm1.medium', 'm1.large', 'm1.nano','m1.xlarge', 'm1.micro']

You have a list of tuples containing dictionaries. That can't be accessed with strings, you'd need to use integers. The original Django app used a special type of Django dictionary called SortedDict, thankfully Python has had an equivalent called OrderedDict since 2.7. You just need to import it like this
from collections import OrderedDict
And then adjust your syntax to use OrderedDict and remove the _() around the values:
meters_info = OrderedDict([
("instance", {
'label': '',
'description': "Existence of instance",
}),
("instance:<type>", {
'label': '',
'description': "Existence of instance <type> "
"(openstack types)",
}),
("memory", {
'label': '',
'description': "Volume of RAM",
}),])

To use data in such way you need to rebuild meter_info to dictionary:
meters_info= {
"instance" : {
'label': '',
'description': "Existence of instance",
},
"instance:<type>" : {
'label': '',
'description': "Existence of instance <type> (openstack types)",
},
"memory" : {
'label': '',
'description': "Volume of RAM",
},}
also it's is better if you have same structure to use classes. For example
class Meters_info_param_stucture(object):
def __init__(self, label = None, description = None):
self.label = label
self.description = description
meters_info= {
"instance" : Meters_info_param_stucture(description = "Existence of instance"),
"instance:<type>" : Meters_info_param_stucture(description = "Existence of instance <type> (openstack types)"),
"memory" : Meters_info_param_stucture(description = "Volume of RAM"),}
>>>meters_info["memory"].description
'Volume of RAM'

The problem is that your array is a list of tuples, and a tuple has a str as the first element, and dict as the second. So you should access it like this:
meters_info[0][0][<some_key>]
For example:
meters_info[0][0]["instance"]
But I do not understand the structure and the reason you keep it like that. You should rather consider reorganising meters_info into a dictionary, like Dmitry.Samborskyi suggests.