Need to create a function to extract values from the json. after that i need to parse the so many similar type of json and save in to file.
json is below
j = '''{
"action": "post",
"status": "completed",
"result": {
"country": "usa",
"is_allow": 'true',
"advance details": {
"value": 'true'
}
}
}'''
convert to json
k = eval(json.dumps(m))
{'action': 'post',
'status': 'completed',
'result': {'country': 'usa',
'is_allow': 'true',
'advance details': {'value': 'true'}}}
Expected out
{'is_allow': 'true','value': 'true'}
pseudo code
Disclaimer : please don't do m['result']['is_allow'] because i need as function to pass so many json
When working with json in python, it should be deserialized into a python object, like a dict, or list; it's weird think to create a function that is 'generic' to any key you want to get, it's just the __getitem__ of dict, if you think to export thing into function, be more specific like this:
def get_is_allow_and_value(my_data):
result = {}
result['is_allow'] = my_data['result']['is_allow']
result['value'] = my_data['result']['advance details']['value']
return result
Also don't use eval you can simple use json.loads for this
Related
I am using the Python requests library to get some data from an API. However, within the output dictionary, I would like to have a list. I want to have a list within the data_providers key where the first entry for name, url and api_url to be hard coded and the second entry to be gotten from the response.
Currently I get the error TypeError: unhashable type: 'dict', and my code look like this
output_dict = {
'data_providers' : {
{
'name': 'NBN Atlas',
'url': 'https://registry.nbnatlas.org/datasets',
'api_url': 'https://registry.nbnatlas.org/ws/dataResource',
},
{
'name': owner_name,
'url': owner_url,
'api_url': owner_api_url,
},
}
}
After running the code and dumping it into a JSON file my desired response should look something like this:
[
{
"data_providers": [
{
"name": "NBN Atlas",
"url": "https://registry.nbnatlas.org/datasets",
"api_url": "https://registry.nbnatlas.org/ws/dataResource"
},
{
"name": "Marine Biological Association",
"url": "https://registry.nbnatlas.org/public/show/dp80",
"api_url": "https://registry.nbnatlas.org/ws/dataProvider/dp80"
}
]
}
]
Any way to go about this?
If I understood your question correctly, you wish to fix the unhashable type error and display valid JSON. If so, you can correct the error by changing data_providers dictionary to a list since it contains multiple nested dictionaries.
output_dict = {
'data_providers': [
{
'name': 'NBN Atlas',
'url': 'https://registry.nbnatlas.org/datasets',
'api_url': 'https://registry.nbnatlas.org/ws/dataResource'
},
{
'name': owner_name,
'url': owner_url,
'api_url': owner_api_url
}
]
}
You can now use a statement similar to print(json.dumps(output_dict)) to display the result as JSON.
So I run a GET to system A and get this output:
'{
"comment": "A good object",
"number": "1.1",
"extra_stuff": {
"extra_id": {"value": 100},
"extra_name": {"value": "Test"},
}
}'
Before I POST to system B, I need to translate (and omit) certain keys (but keep the value). What I want from this GET and translate is ONLY the nested dictionary keys (keys in extra_stuff) and only one of them.
Translation:
mapping_dict = { 'extra_id': 'id' }
dict_to_post = { '#KEY extra_ID': #VALUE FROM extra_ID# :, 'name' : Test, 'other' : True }
I got a function for translating the dictionary keys from another question put here:
def update_dict_keys(obj, mapping_dict):
if isinstance(obj, dict):
return {mapping_dict[k]: update_dict_keys(v, mapping_dict) for k, v in obj.iteritems()}
else:
return obj
But I am unsure how to send only the nested dictionary into that function, I only seem to put the "main" dictionary in there. Not sure what the best way about it is. I would further need to handle information from the "main" dictionary as well in another post.
Are you sure, there is a , behind {"value": "Test"} bacause this is no valid json and your get response looks like json.
If this is only a mistake in your post here, you could
use json.loads() to get a dictionary from your response string:
import json
res = '{
"comment": "A good object",
"number": "1.1",
"extra_stuff": {
"extra_id": {"value": 100},
"extra_name": {"value": "Test"}
}
}'
# convert to dict
res_dict = json.loads(res)
# do your mapping
dict_to_post = { 'extra_id': res_dict['extra_stuff']['extra_id']['value']}
# dict_to_post is {'extra_id': 100} now
Furthermore I don't understand what you need the mapping_dict for. Do you want to have {'id': 100} as result?
So I figured out that the response I get is already decoded! Problem I had was that the decoded response was a list with a dictionary within a dictionary.
I am trying to make api call which has json as value to one of the keys. It is failing because of extra double quotes in inner json formed.
variable = request.GET.get('name', '{}')
This value of variable is json that is passed from webpage.
Sample data in variable will be like:-
variable= {'name': 'ABC', 'Id': '1'}
Now when I try to form my payload to be sent in post call,
payload = {
'var1': var1,
'variable': variable,
}
Now this "variable" which is added to payload is treated as String and append with double quotes, like this:-
{'var1': '130', 'variable': "{'name': 'ABC', 'Id': '1'}"}
But I want it to be like this:-
{'var1': '130', 'variable': {'name': 'ABC', 'Id': '1'}}
Any suggestions how to make it possible?
this is the code to make the post call with data:-
r = requests.post("URL",data= json.dumps(payload),headers={'Authorization': obj.header, 'Content-type': 'application/json'}, proxies=getProxyDict())
When you get variable from the request it isn't just 'treated' as a string... is is a string.
So you need to:
json.loads(variable)
before adding it to the dict which is later serialized to json.
i.e.
variable = json.loads(request.GET.get('name', '{}'))
payload = {
'var1': var1,
'variable': variable,
}
r = requests.post("URL",data= json.dumps(payload),headers={'Authorization': obj.header, 'Content-type': 'application/json'}, proxies=getProxyDict())
You have presumably already dumped variable to a JSON string. Don't do that; leave it as a dict, and dump the whole thing in one go.
import json
variable = {'name': 'ABC', 'Id': '1'}
payload = {
'var1': 'foo',
'variable': variable,
}
print json.dumps(payload, indent=4)
results to
{
"variable": {
"name": "ABC",
"Id": "1"
},
"var1": "foo"
}
which is perfectly json. But, you may need to set your headers according to the API you're talking to.
request = requests.get('myurl', headers={'accept': 'application/json'})
json_data = request.json()
# manipulate json_data
request = requests.post('myurl', headers={'content-type': 'application/json'}, json=json_data)
Few things I would like to point here.
variable= "{'name': 'ABC', 'Id': '1'}"
This is NOT a valid json string.
variable= '{"name": "ABC", "Id": "1"}'
This is a valid json string.
So first before attaching this variable to payload dict you need to convert it to a python dict like this.
variable = json.loads(variable)
This will create variable something like this.
{u'name': u'ABC', u'Id': u'1'}
Now you can go ahead and add this to your payload dict and continue what you are doing.
Why does this code give a KeyError?
output_format = """
{
"File": "{filename}",
"Success": {success},
"ErrorMessage": "{error_msg}",
"LogIdentifier": "{log_identifier}"
}
"""
print output_format.format(filename='My_file_name',
success=True,
error_msg='',
log_identifier='123')
Error message:
KeyError: ' "File"'
You need to double the outer braces; otherwise Python thinks { "File".. is a reference too:
output_format = '{{ "File": "{filename}", "Success": {success}, "ErrorMessage": "{error_msg}", "LogIdentifier": "{log_identifier}" }}'
Result:
>>> print output_format.format(filename='My_file_name',
... success=True,
... error_msg='',
... log_identifier='123')
{ "File": "My_file_name", "Success": True, "ErrorMessage": "", "LogIdentifier": "123" }
If, indicentally, you are producing JSON output, you'd be better off using the json module:
>>> import json
>>> print json.dumps({'File': 'My_file_name',
... 'Success': True,
... 'ErrorMessage': '',
... 'LogIdentifier': '123'})
{"LogIdentifier": "123", "ErrorMessage": "", "Success": true, "File": "My_file_name"}
Note the lowercase true in the output, as required by the JSON standard.
As mentioned by Tudor in a comment to another answer, the Template class was the solution that worked best for me. I'm dealing with nested dictionaries or list of dictionaries and handling those were not as straightforward.
Using Template though the solution is quite simple.
I start with a dictionary that is converted into a string. I then replace all instances of { with ${ which is the Template identifier to substitute a placeholder.
The key point of getting this to work is using the Template method safe_substitute. It will replace all valid placeholders like ${user_id} but ignore any invalid ones that are part of the dictionary structure, like ${'name': 'John', ....
After the substitution is done I remove any leftovers $ and convert the string back to a dictionary.
In the code bellow, resolve_placeholders returns a dictionary where each key matches a placeholder in the payload string and the value is substituted by the Template class.
from string import Template
.
.
.
payload = json.dumps(payload)
payload = payload.replace('{', '${')
replace_values = self.resolve_placeholders(payload)
if replace_values:
string_template = Template(payload)
payload = string_template.safe_substitute(replace_values)
payload = payload.replace('${', '{')
payload = json.loads(payload)
To extend on Martijn Pieters answer and comment:
According to MArtijn' comment, escaping the {..} pairs that are not placeholders is they way to go with nested dictionaries. I haven't succeded in doing that, so I suggest the following method.
For nested dictionaries I tried doubling up on any { and } of the nested dictionaries.
a='{{"names":{{"a":"{name}"}}}}'
a.format(name=123) output:
output: '{"names":{"a":"123"}}'
But this makes using format to change values inside a json string, a over-complex method, so I use a twist on the format command.
I replace ${param_name} in a json string. For example:
My predefined JSON looks like this:
my_json_dict = {
'parameter': [
{
'name': 'product',
'value': '${product}'
},
{
'name': 'suites',
'value': '${suites}'
},
{
'name': 'markers',
'value': '${markers}'
}
]
}
I provide this dictionary as values to replace instead of the parameters
parameters = {
'product': 'spam',
'suites': 'ham',
'markers': 'eggs'
}
And use this code to do the replacment
json_str = json.dumps(my_json_dict)
for parameter_name, parameter_value in parameters.iteritems():
parameter_name = '${'+parameter_name+'}'
json_str = json_str.replace(parameter_name, parameter_value)
json_dict = json.loads(json_str)
I have an array of dictionaries like so:
myDict[0] = {'date':'today', 'status': 'ok'}
myDict[1] = {'date':'yesterday', 'status': 'bad'}
and I'm trying to export this array to a json file where each dictionary is its own entry. The problem is when I try to run:
dump(myDict, open("test.json", "w"))
It outputs a json file with a number prefix before each entry
{"0": {"date": "today", "status": "ok"}, "1": {"date": "yesterday", "status": "bad"} }
which apparently isn't legal json since my json parser (protovis) is giving me error messages
Any ideas?
Thanks
Use a list instead of a dictionary; you probably used:
myDict = {}
myDict[0] = {...}
You should use:
myList = []
myList.append({...}
P.S.: It seems valid json to me anyways, but it is an object and not a list; maybe this is the reason why your parser is complaining
You should use a JSON serializer...
Also, an array of dictionaries would better serialize to something like this:
[
{
"date": "today",
"status": "ok"
},
{
"date": "yesterday",
"status": "bad"
}
]
That is, you should just use a JavaScript array.