So I run a GET to system A and get this output:
'{
"comment": "A good object",
"number": "1.1",
"extra_stuff": {
"extra_id": {"value": 100},
"extra_name": {"value": "Test"},
}
}'
Before I POST to system B, I need to translate (and omit) certain keys (but keep the value). What I want from this GET and translate is ONLY the nested dictionary keys (keys in extra_stuff) and only one of them.
Translation:
mapping_dict = { 'extra_id': 'id' }
dict_to_post = { '#KEY extra_ID': #VALUE FROM extra_ID# :, 'name' : Test, 'other' : True }
I got a function for translating the dictionary keys from another question put here:
def update_dict_keys(obj, mapping_dict):
if isinstance(obj, dict):
return {mapping_dict[k]: update_dict_keys(v, mapping_dict) for k, v in obj.iteritems()}
else:
return obj
But I am unsure how to send only the nested dictionary into that function, I only seem to put the "main" dictionary in there. Not sure what the best way about it is. I would further need to handle information from the "main" dictionary as well in another post.
Are you sure, there is a , behind {"value": "Test"} bacause this is no valid json and your get response looks like json.
If this is only a mistake in your post here, you could
use json.loads() to get a dictionary from your response string:
import json
res = '{
"comment": "A good object",
"number": "1.1",
"extra_stuff": {
"extra_id": {"value": 100},
"extra_name": {"value": "Test"}
}
}'
# convert to dict
res_dict = json.loads(res)
# do your mapping
dict_to_post = { 'extra_id': res_dict['extra_stuff']['extra_id']['value']}
# dict_to_post is {'extra_id': 100} now
Furthermore I don't understand what you need the mapping_dict for. Do you want to have {'id': 100} as result?
So I figured out that the response I get is already decoded! Problem I had was that the decoded response was a list with a dictionary within a dictionary.
Related
I'm using requests and JSON to pull some data from an API, and I'm struggling with using a nested dict.
Here is the JSON data:
{"data": [
{
"ContactId": "123",
"EmailAddress": "abc#xyz.com",
"FirstName": null,
"LastName": null,
"ClickDate": "6/6/1966",
"Clicks": "5",
"IPAddress": "1.1.1.1.1",
"UserAgent": "IE8.0",
"UniqueLinksClicked": [
{
"LinkURL": "http://link1.com",
"LinkURL": "http://link2.com",
"LinkURL": "http://link3.com"
}
]
}
]}
I'm able to access all of the ContactID and other 1st level stuff fine, but I can't figure out how to traverse the "LinkURL" stuff.
Here is my python...
result = requests.get(requesturl, headers=headers)
jdata = json.loads(result.content)
for result in jdata["data"]:
contactID = str([(result["ContactId"])])
for result in jdata["data"]["UniqueLinksClicked"]: #I'm doing this wrong, but I'm not sure how.
print(ContactID + " " + str([(result["LinkURL"])]))
The line marked with a comment above generates a TypeError indicating it's a list, where I expected it to be a dict:
list indices must be integers or slices, not str
If instead I drop the ["data"] dereference and try to access "UniqueLinksClicked" on jdata:
for link in jdata["UniqueLinksClicked"]:
I get a key error because the ["UniqueLinksClicked"] is an item inside of the ["data"] dict.
How do I do this correctly?
You can iterate over the links in a nested loop. Do not use the same variable name result in two nested loops! Use a different variable name in the inner loop.
for link in result["UniqueLinksClicked"]:
print(ContactID, link["LinkURL"])
(Moved from question.)
[OP] was confused about the variable naming in the for variable1 in variable2["dict"]: portion. After some help from Håken Lid, [they] figured it out.
It should look like this...
for item in jdata["data"]:
contactID = str([(item["ContactId"])])
print(contactID)
for link in item["UniqueLinksClicked"]:
print(link["LinkURL"])
I've got a json file that I've pulled from a web service and am trying to parse it. I see that this question has been asked a whole bunch, and I've read whatever I could find, but the json data in each example appears to be very simplistic in nature. Likewise, the json example data in the python docs is very simple and does not reflect what I'm trying to work with. Here is what the json looks like:
{"RecordResponse": {
"Id": blah
"Status": {
"state": "complete",
"datetime": "2016-01-01 01:00"
},
"Results": {
"resultNumber": "500",
"Summary": [
{
"Type": "blah",
"Size": "10000000000",
"OtherStuff": {
"valueOne": "first",
"valueTwo": "second"
},
"fieldIWant": "value i want is here"
The code block in question is:
jsonFile = r'C:\Temp\results.json'
with open(jsonFile, 'w') as dataFile:
json_obj = json.load(dataFile)
for i in json_obj["Summary"]:
print(i["fieldIWant"])
Not only am I not getting into the field I want, but I'm also getting a key error on trying to suss out "Summary".
I don't know how the indices work within the array; once I even get into the "Summary" field, do I have to issue an index manually to return the value from the field I need?
The example you posted is not valid JSON (no commas after object fields), so it's hard to dig in much. If it's straight from the web service, something's messed up. If you did fix it with proper commas, the "Summary" key is within the "Results" object, so you'd need to change your loop to
with open(jsonFile, 'w') as dataFile:
json_obj = json.load(dataFile)
for i in json_obj["Results"]["Summary"]:
print(i["fieldIWant"])
If you don't know the structure at all, you could look through the resulting object recursively:
def findfieldsiwant(obj, keyname="Summary", fieldname="fieldIWant"):
try:
for key,val in obj.items():
if key == keyname:
return [ d[fieldname] for d in val ]
else:
sub = findfieldsiwant(val)
if sub:
return sub
except AttributeError: #obj is not a dict
pass
#keyname not found
return None
I have a json object which I would like to filter for misspelled key name. So for the example below, I would like to have a json object without the misspelled test_name key. What is the easiest way to do this?
json_data = """{
"my_test": [{
"group_name": "group-A",
"results": [{
"test_name": "test1",
"time": "8.556",
"status": "pass"
}, {
"test_name": "test2",
"time": "45.909",
"status": "pass"
}, {
"test_nameZASSD": "test3",
"time": "9.383",
"status": "fail"
}]
}]
}"""
This is an online test, and looks like i'm not allowed to use jsonSchema.
So far my code looks like this:
if 'test_suites' in data:
for suites in data["test_suites"]:
if 'results' in suites and 'suite_name' in suites:
for result in suites["results"]:
if 'test_name' not in result or 'time' not in result or 'status' not in result:
result.clear()
else:
....
else:
print("Check 'suite_name' and/or 'results'")
else:
print("Check 'test_suites'")
It kind of works, but result.clear() leaves a empty {}, which get annoying later. What can I do here?
It looks like your data have a consistent schema. So I would try using json schema to solve your problem. With that you can set up a schema and only allow objects with certain key names.
If you just want to check if a certain key is in the dictionary and make sure that you only get the ones that are according to spec you could do something like this:
passed = []
for item in result:
if 'test_name' in item.keys():
passed.append(item)
But if you have a lot of different keys you need to check for it will become unwieldy. So for bigger projects I would say that json schema is the way to go.
Why does this code give a KeyError?
output_format = """
{
"File": "{filename}",
"Success": {success},
"ErrorMessage": "{error_msg}",
"LogIdentifier": "{log_identifier}"
}
"""
print output_format.format(filename='My_file_name',
success=True,
error_msg='',
log_identifier='123')
Error message:
KeyError: ' "File"'
You need to double the outer braces; otherwise Python thinks { "File".. is a reference too:
output_format = '{{ "File": "{filename}", "Success": {success}, "ErrorMessage": "{error_msg}", "LogIdentifier": "{log_identifier}" }}'
Result:
>>> print output_format.format(filename='My_file_name',
... success=True,
... error_msg='',
... log_identifier='123')
{ "File": "My_file_name", "Success": True, "ErrorMessage": "", "LogIdentifier": "123" }
If, indicentally, you are producing JSON output, you'd be better off using the json module:
>>> import json
>>> print json.dumps({'File': 'My_file_name',
... 'Success': True,
... 'ErrorMessage': '',
... 'LogIdentifier': '123'})
{"LogIdentifier": "123", "ErrorMessage": "", "Success": true, "File": "My_file_name"}
Note the lowercase true in the output, as required by the JSON standard.
As mentioned by Tudor in a comment to another answer, the Template class was the solution that worked best for me. I'm dealing with nested dictionaries or list of dictionaries and handling those were not as straightforward.
Using Template though the solution is quite simple.
I start with a dictionary that is converted into a string. I then replace all instances of { with ${ which is the Template identifier to substitute a placeholder.
The key point of getting this to work is using the Template method safe_substitute. It will replace all valid placeholders like ${user_id} but ignore any invalid ones that are part of the dictionary structure, like ${'name': 'John', ....
After the substitution is done I remove any leftovers $ and convert the string back to a dictionary.
In the code bellow, resolve_placeholders returns a dictionary where each key matches a placeholder in the payload string and the value is substituted by the Template class.
from string import Template
.
.
.
payload = json.dumps(payload)
payload = payload.replace('{', '${')
replace_values = self.resolve_placeholders(payload)
if replace_values:
string_template = Template(payload)
payload = string_template.safe_substitute(replace_values)
payload = payload.replace('${', '{')
payload = json.loads(payload)
To extend on Martijn Pieters answer and comment:
According to MArtijn' comment, escaping the {..} pairs that are not placeholders is they way to go with nested dictionaries. I haven't succeded in doing that, so I suggest the following method.
For nested dictionaries I tried doubling up on any { and } of the nested dictionaries.
a='{{"names":{{"a":"{name}"}}}}'
a.format(name=123) output:
output: '{"names":{"a":"123"}}'
But this makes using format to change values inside a json string, a over-complex method, so I use a twist on the format command.
I replace ${param_name} in a json string. For example:
My predefined JSON looks like this:
my_json_dict = {
'parameter': [
{
'name': 'product',
'value': '${product}'
},
{
'name': 'suites',
'value': '${suites}'
},
{
'name': 'markers',
'value': '${markers}'
}
]
}
I provide this dictionary as values to replace instead of the parameters
parameters = {
'product': 'spam',
'suites': 'ham',
'markers': 'eggs'
}
And use this code to do the replacment
json_str = json.dumps(my_json_dict)
for parameter_name, parameter_value in parameters.iteritems():
parameter_name = '${'+parameter_name+'}'
json_str = json_str.replace(parameter_name, parameter_value)
json_dict = json.loads(json_str)
I have an array of dictionaries like so:
myDict[0] = {'date':'today', 'status': 'ok'}
myDict[1] = {'date':'yesterday', 'status': 'bad'}
and I'm trying to export this array to a json file where each dictionary is its own entry. The problem is when I try to run:
dump(myDict, open("test.json", "w"))
It outputs a json file with a number prefix before each entry
{"0": {"date": "today", "status": "ok"}, "1": {"date": "yesterday", "status": "bad"} }
which apparently isn't legal json since my json parser (protovis) is giving me error messages
Any ideas?
Thanks
Use a list instead of a dictionary; you probably used:
myDict = {}
myDict[0] = {...}
You should use:
myList = []
myList.append({...}
P.S.: It seems valid json to me anyways, but it is an object and not a list; maybe this is the reason why your parser is complaining
You should use a JSON serializer...
Also, an array of dictionaries would better serialize to something like this:
[
{
"date": "today",
"status": "ok"
},
{
"date": "yesterday",
"status": "bad"
}
]
That is, you should just use a JavaScript array.