Why does this code give a KeyError?
output_format = """
{
"File": "{filename}",
"Success": {success},
"ErrorMessage": "{error_msg}",
"LogIdentifier": "{log_identifier}"
}
"""
print output_format.format(filename='My_file_name',
success=True,
error_msg='',
log_identifier='123')
Error message:
KeyError: ' "File"'
You need to double the outer braces; otherwise Python thinks { "File".. is a reference too:
output_format = '{{ "File": "{filename}", "Success": {success}, "ErrorMessage": "{error_msg}", "LogIdentifier": "{log_identifier}" }}'
Result:
>>> print output_format.format(filename='My_file_name',
... success=True,
... error_msg='',
... log_identifier='123')
{ "File": "My_file_name", "Success": True, "ErrorMessage": "", "LogIdentifier": "123" }
If, indicentally, you are producing JSON output, you'd be better off using the json module:
>>> import json
>>> print json.dumps({'File': 'My_file_name',
... 'Success': True,
... 'ErrorMessage': '',
... 'LogIdentifier': '123'})
{"LogIdentifier": "123", "ErrorMessage": "", "Success": true, "File": "My_file_name"}
Note the lowercase true in the output, as required by the JSON standard.
As mentioned by Tudor in a comment to another answer, the Template class was the solution that worked best for me. I'm dealing with nested dictionaries or list of dictionaries and handling those were not as straightforward.
Using Template though the solution is quite simple.
I start with a dictionary that is converted into a string. I then replace all instances of { with ${ which is the Template identifier to substitute a placeholder.
The key point of getting this to work is using the Template method safe_substitute. It will replace all valid placeholders like ${user_id} but ignore any invalid ones that are part of the dictionary structure, like ${'name': 'John', ....
After the substitution is done I remove any leftovers $ and convert the string back to a dictionary.
In the code bellow, resolve_placeholders returns a dictionary where each key matches a placeholder in the payload string and the value is substituted by the Template class.
from string import Template
.
.
.
payload = json.dumps(payload)
payload = payload.replace('{', '${')
replace_values = self.resolve_placeholders(payload)
if replace_values:
string_template = Template(payload)
payload = string_template.safe_substitute(replace_values)
payload = payload.replace('${', '{')
payload = json.loads(payload)
To extend on Martijn Pieters answer and comment:
According to MArtijn' comment, escaping the {..} pairs that are not placeholders is they way to go with nested dictionaries. I haven't succeded in doing that, so I suggest the following method.
For nested dictionaries I tried doubling up on any { and } of the nested dictionaries.
a='{{"names":{{"a":"{name}"}}}}'
a.format(name=123) output:
output: '{"names":{"a":"123"}}'
But this makes using format to change values inside a json string, a over-complex method, so I use a twist on the format command.
I replace ${param_name} in a json string. For example:
My predefined JSON looks like this:
my_json_dict = {
'parameter': [
{
'name': 'product',
'value': '${product}'
},
{
'name': 'suites',
'value': '${suites}'
},
{
'name': 'markers',
'value': '${markers}'
}
]
}
I provide this dictionary as values to replace instead of the parameters
parameters = {
'product': 'spam',
'suites': 'ham',
'markers': 'eggs'
}
And use this code to do the replacment
json_str = json.dumps(my_json_dict)
for parameter_name, parameter_value in parameters.iteritems():
parameter_name = '${'+parameter_name+'}'
json_str = json_str.replace(parameter_name, parameter_value)
json_dict = json.loads(json_str)
Related
I have a variable group_array like this
groups_array=[{"group": "18652_PDR"}, {"group": "11262_PDR"}, {"group": "3787_PDR"}, {"group": "4204_PDR"}]
I want to put the groups_array variable inside below string so I tried like this using the f method
data = f'{"request":{"streaming_type":"quote", "data":{"groups": {groups_array}}, "request_type":"subscribe", "response_format":"json"}}'
But am getting error deeply nested error
I want the string to be in below format after adding the variable
data = '{"request":{"streaming_type":"quote", "data":{"groups": [{"group": "18652_PDR"}, {"group": "11262_PDR"}, {"group": "3787_PDR"}, {"group": "4204_PDR"}]}, "request_type":"subscribe", "response_format":"json"}}'
Could someone tell me how can I properly do it in this case as there are too many "" and {}?
You can just use the built-in json package. You would create a dictionary of your data and pass it to json.dumps to receive the json string.
import json
groups_array = [{"group": "18652_PDR"}, {"group": "11262_PDR"}, {"group": "3787_PDR"}, {"group": "4204_PDR"}]
data = {"request":
{"streaming_type": "quote",
"data": {
"groups": groups_array
},
"request_type": "subscribe",
"response_format": "json"
}
}
data_json = json.dumps(data)
print(data)
# Output:
{'request': {'streaming_type': 'quote', 'data': {'groups': [{'group': '18652_PDR'}, {'group': '11262_PDR'}, {'group': '3787_PDR'}, {'group': '4204_PDR'}]}, 'request_type': 'subscribe', 'response_format': 'json'}}
You can assign directly to dict and create a json (stringify it) if your request needs string:
import json
data = json.dumps({"request":{"streaming_type":"quote", "data":{"groups": groups_array}, "request_type":"subscribe", "response_format":"json"}})
Or if dictionary is also an option - pass it without json.dumps
print(keystr)
print(addstr10)
dict[keystr] = addstr10
json.dump(dict, json_file, indent=2)
This yields for keystr = nested_object
and for addstr10 it yields
{"another_nested_object": { "another_another_nested_object": "ymwjkxpr", "integervalue" : 11, "floatvalue" : 4.703165912962773, "anothernamer" : True, "another_nested_object_1": { "another_key": "vbiofdhz" } } }
which look right but the json file has extra backslashes in the value pair and I want to get rid of it
{
"nested_object": "{\"another_nested_object\": { \"another_another_nested_object\": \"ymwjkxpr\", \"integervalue\" : 11, \"floatvalue\" : 4.703165912962773, \"anothernamer\" : True, \"another_nested_object_1\": { \"another_key\": \"vbiofdhz\" } } }"
}
The reason is that addstr is already encoded as JSON. So you're encoding it a second time when you include it as a value in the dictionary.
You should decode it first, so change
dict[keystr] = addstr10
to
dict[keystr] = json.loads(addstr10)
Howevever, the nested object contains invalid JSON. JSON uses true and false for booleans, but your string has True, which won't parse properly.
You could try using ast.literal_eval() instead of json.loads().
import ast
dict[keystr] = ast.literal_eval(addstr10)
Need to create a function to extract values from the json. after that i need to parse the so many similar type of json and save in to file.
json is below
j = '''{
"action": "post",
"status": "completed",
"result": {
"country": "usa",
"is_allow": 'true',
"advance details": {
"value": 'true'
}
}
}'''
convert to json
k = eval(json.dumps(m))
{'action': 'post',
'status': 'completed',
'result': {'country': 'usa',
'is_allow': 'true',
'advance details': {'value': 'true'}}}
Expected out
{'is_allow': 'true','value': 'true'}
pseudo code
Disclaimer : please don't do m['result']['is_allow'] because i need as function to pass so many json
When working with json in python, it should be deserialized into a python object, like a dict, or list; it's weird think to create a function that is 'generic' to any key you want to get, it's just the __getitem__ of dict, if you think to export thing into function, be more specific like this:
def get_is_allow_and_value(my_data):
result = {}
result['is_allow'] = my_data['result']['is_allow']
result['value'] = my_data['result']['advance details']['value']
return result
Also don't use eval you can simple use json.loads for this
I have a dict which I am encoding as a string as such:
import json
template = json.dumps({
'_index': '{0}',
'_type': '{1}',
'_id': '{2}',
'_source': {
'doc': '{3}',
'doc_as_upsert': True
}
})
Now, I try to format it as per the new python conventions mentioned here: https://pyformat.info/
print template.format('one','two','three','four')
However, I get an error as so
Traceback (most recent call last): File "python", line 1, in
KeyError: '"_type"'
What am I doing wrong here?
The problem stems from the curly braces in your JSON - you need to double-escape them in order for str.format() to work, e.g.:
import json
template = json.dumps({
'_index': '{0}',
'_type': '{1}',
'_id': '{2}',
'_source': {
'doc': '{3}',
'doc_as_upsert': True
}
})
template = template.replace("{", "{{").replace("}", "}}")
print(template.format('one','two','three','four'))
It will no longer err, but it will also escape your parameter curly braces so they wont get replaced by str.format(), so you'll have to invent your own 'parameter' escape as well (make sure it doesn't appear as the markup code for JSON, tho, like curly braces do), for example using < and > instead:
import json
template = json.dumps({
'_index': '<0>',
'_type': '<1>',
'_id': '<2>',
'_source': {
'doc': '<3>',
'doc_as_upsert': True
}
})
template = template.replace("{", "{{").replace("}", "}}").replace("<", "{").replace(">", "}")
print(template.format('one', 'two', 'three', 'four'))
But it's much better idea to directly replace your data before turning it onto JSON. You can call str.format() on each (str) value in your dict individually, passing a dict with all the parameters to it and using named parameters (i.e. {one}) to pick up the needed argument from expanded keys.
UPDATE: You don't even need to recurse through your data for the last one as the json serializer is going to recurse through it anyway, but unfortunately the json module doesn't make it easy to swap the default behavior of string serializing so you'll have to do some monkey-patching:
from json import dumps, encoder
def template_json(data, args, **kwargs):
json_s1, json_s2 = encoder.encode_basestring, encoder.encode_basestring_ascii
encoder.encode_basestring = lambda s: json_s1(s.format(**args))
encoder.encode_basestring_ascii = lambda s: json_s2(s.format(**args))
try:
return dumps(data, **kwargs)
finally:
encoder.encode_basestring, encoder.encode_basestring_ascii = json_s1, json_s2
It essentially temporarily wraps the internal JSON string building methods with the ones that apply formatting first and then reverts back everything so that other functions that may depend on the json module don't get unexpected behavior (although there is a little danger here, too - this is not thread-safe). Since it will be reading the elements one by one we cannot really use positional formatting so this uses named formatting as suggested above. You can test it as:
data = {
'_index': '{one}',
'_type': '{two}',
'_id': '{three}',
'_source': {
'doc': '{four}',
'doc_as_upsert': True,
}
}
template_vars = {"one": "value 1", "two": "value 2", "three": "value 3", "four": "value 4"}
print(template_json(data, template_vars, indent=4))
Resulting in:
{
"_source": {
"doc": "value 4",
"doc_as_upsert": true
},
"_index": "value 1",
"_id": "value 3",
"_type": "value 2"
}
But, generally, if you have to hack around your system to achieve what you want - you might want to reconsider if that's the right approach in the first place and can your objective be achieved in a simpler manner?
I'm trying to access certain fields of information in JSON dict. My code is set up as the following:
Views.py
def viewIssues(request):
r = requests.get(bucket_url)
issue_payload = r.json()
issue = json.loads(str(issue_payload))
context = {
"issue_title": issue['issues']['title'],
"issue_content": issue['issues']['content'],
"title": "View Issues",
}
return render(request, "view_issues.html", context)
str(issue_payload) gives me this:
{
'search':None,
'count':1,
'filter':{
},
'issues':[
{
'priority':'major',
'comment_count':0,
'utc_created_on':'2016-11-12 01:48:16+00:00',
'utc_last_updated':'2016-11-12 01:48:16+00:00',
'status':'new',
'title':'example issue',
'reported_by':{
'is_staff':False,
'display_name':'display name',
'is_team':False,
'resource_uri':'/1.0/users/username',
'avatar':'https://bitbucket.org/account/username/avatar/32/?ts=1479493904',
'first_name':'firstname',
'username':'username',
'last_name':'lastname'
},
'is_spam':False,
'content':'blah blah',
'metadata':{
'milestone':None,
'component':None,
'version':None,
'kind':'bug'
},
'local_id':1,
'created_on':'2016-11-12T02:48:16.052',
'resource_uri':'/1.0/repositories/username/supportal2016test/issues/1',
'follower_count':1
}
]
}
However when I try to use the json.loads and indices ['issues']['title'] and ['issues']['title'] I get an error:
JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)
I'm wondering if it's because the converted payload has quotations on each field (i.e. 'issues'). Any help would be much appreciated.
The .json() call already parses the JSON result and returns a Python structure in this case a dictionary. Then your call
issue = json.loads(str(issue_payload))
forces the dictionary into a string and tries to parse it again. But the dictionary string representation contains ' around strings and not " as required in JSON.
To cut the long story short: issue_payload is what you want already.