I have testdata read from excel and want it used to format a nested json string for request post body used, details as below:
import json
kwargs = {"name": "testname", "device": {"plt": "dsk"}}
payload = """
{{
"name": "{name}",
"device": "{device}",
}}
"""
payload = json.loads(payload.format(**kwargs))
And the expected payload should be:
{
"name": "testname",
"device": {"plt": "dsk"}
}
But there's error with json.loads(payload.format(**kwargs)), then how to format nested json string with **dict?
error message
You should do the operation in steps to figure out what goes wrong.
kwargs = {"name": "testname", "device": {"plt": "dsk"}}
payload = """
{{
"name": "{name}",
"device": "{device}",
}}
"""
json_string = payload.format(**kwargs)
print(json_string)
gives
{
"name": "testname",
"device": "{'plt': 'dsk'}",
}
Notice the dictionary {'plt': 'dsk'} was stringified python-style before being interpolated into the payload string. This is not valid json -- the apostrophes need to be replaced with quotation marks.
To get around this, I suggest you create a python dict that looks like payload and then convert that to a json string.
# Some preexisting dictionary
payload_dict = {"oldvalue": 0}
# Add values from kwargs
for key, value in kwargs.items():
payload_dict[key] = value
# Dump it to json
payload = json.dumps(payload_dict, indent=4) # indent=4 is only for pretty-print
print(payload)
which gives the output:
{
"oldvalue": 0,
"name": "testname",
"device": {
"plt": "dsk"
}
}
If payload_dict is identical to kwargs, then all you need to do is
payload = json.dumps(kwargs)
print(payload)
{
"name": "testname",
"device": {
"plt": "dsk"
}
}
Related
resp = {
"Name": "test",
"os": "windows",
"Agent": {
"id": "2",
"status": [
{
"code": "123",
"level": "Info",
"displayStatus": "Ready",
"message": "running",
"time": "2022-01-18T09:51:08+00:00"
}
]
}
I am trying to get the time value from the JSON.
I tried the below code but faced error with dict
resp1 = json.loads(resp)
resp2 = resp1.values()
creation_time = resp2.get("Agent").get("status")
val= creation_time["time"]
print(val) ## Thrwoing error as dict_values has no 'get'
Any suggestion on python how to take this time values
Few problems I noticed
You are trying to load a Dict type using the json's loads function which is supposed to get a string in json format (ex: '{ "name":"John", "age":30, "city":"New York"}')
You tried to access resp2 before declaration (I guessed you meant "resp1?")
You're using resp3 without declaration.
You are missing }
You don't need the .value() function because it will return a list.
Also creation time is a list with one object, so you need to access it too.
Considering all this, you can change it as follows:
import json
resp = '{ "Name": "test", "os": "windows","Agent": {"id": "2","status": [{"code": "123","level": "Info","displayStatus": "Ready","message": "running","time": "2022-01-18T09:51:08+00:00"}]}}'
resp1 = json.loads(resp)
creation_time = resp1.get("Agent").get("status")
val= creation_time[0]["time"]
print(val)
You just need to access the dicts using [] like so:
resp = {"Name": "test", "os": "windows", "Agent": {"id": "2","status": [{"code": "123","level": "Info","displayStatus": "Ready","message": "running","time": "2022-01-18T09:51:08+00:00"}]}}
creation_time = resp["Agent"]["status"]
val= creation_time[0]["time"]
print(val)
Output:
2022-01-18T09:51:08+00:00
I am using python 3.6.3a. I would like to generate payload for each of the json records. I am using each variable to access the record. How to assign variable value (each in this case) in payload? I tried {each} and other methods but didn't work.
code snippet below.
json_records = [{"description":"<p>This is scenario1<\/p>","owner":"deb",
"priority":"high"},
{"description":"<p>This is scenario2<\/p>","owner":"deb",
"priority":"medium"}]
json_object = json.loads(json_records)
for each in json_object:
payload = """
{
"subject": "test",
"fieldValues": [
{each}
]
}
"""
There are two ways to approach this problem.
One way could be creating a dict() object and inserting keys as you wish, then json.dumps(object) to convert into string payload as in:
import json
json_records = [{"description":"This is scenario1</p>","owner":"deb","priority":"high"}
,{"description":"This is scenario2</p>","owner":"deb","priority":"medium"}]
for obj in json_records:
payload = dict()
payload['subject'] = 'test'
for key,value in obj.items():
payload['fieldName'] = {
key:value
}
print(json.dumps(payload))
#{"subject": "test", "fieldName": {"priority": "high"}}
#{"subject": "test", "fieldName": {"priority": "medium"}}
Second way is to create a textual payload from string as in, however if you need a valid JSON at the end, this would require a post-step of validation (something like try json.loads(payload) - So I'd just use the first method. I would use this method only if I have a specific requirements to generate the payload in a certain way.
import json
json_records = [{"description":"This is scenario1</p>","owner":"deb","priority":"high"}
,{"description":"This is scenario2</p>","owner":"deb","priority":"medium"}]
# json_object = json.loads(json_records) # json.loads works only on byte-like strings. your object is already in python in this case.
for obj in json_records:
payload = """
{
"subject": "test",
"fieldValues": [
%s
]
}
""" % (obj["priority"])
print(payload)
#{
# "subject": "test",
# "fieldValues": [
# high
# ]
# }
#
#
# {
# "subject": "test",
# "fieldValues": [
# medium
# ]
# }
You could make payload a Template string and use it to put the data in each JSON record into the format you want. Bracket {} characters have not special meaning in Templates, which is what makes using them easy.
Doing that will create a valid string representation of a dictionary containing everything. You can turn this into an actual Python dictionary data-structure using the ast.literal_eval() function, and then convert that into JSON string format — which I think is the final format you're after.
rom ast import literal_eval
import json
from string import Template
from textwrap import dedent
json_records = '''[{"description":"<p>This is scenario1<\/p>","owner":"deb",
"priority":"high"},
{"description":"<p>This is scenario2<\/p>","owner":"deb",
"priority":"medium"}]'''
json_object = json.loads(json_records)
payload = Template(dedent("""
{
"subject": "test",
"fieldValues": [
$each
]
}""")
)
for each in json_object:
obj = literal_eval(payload.substitute(dict(each=each)))
print(json.dumps(obj, indent=2))
Output:
{
"subject": "test",
"fieldValues": [
{
"description": "<p>This is scenario1</p>",
"owner": "deb",
"priority": "high"
}
]
}
{
"subject": "test",
"fieldValues": [
{
"description": "<p>This is scenario2</p>",
"owner": "deb",
"priority": "medium"
}
]
}
I'm calling an API that will require dynamic variables as parameters. However, I do not know how to format the string to include variables when it is surrounded by triple quotes and a backslash escape character at the beginning of the string literal.
I've tried varying the amount of quotes and using the ".format()" function.
Here is code formatted in a way that gets a successful result:
payload = "{\n\t\"firm\": \"myfirm\",\n\t\"id\": \"f87987562\",\n\t\"data\": {\n\t\t\"tracking_preference\": 2\n\t} \n}\n"
Here is my attempt at trying to format the string in a cleaner way while also including variables:
payload = \
"""{
"firm": {0},
"id": {1},
"data": {
"tracking_preference": {2}
}
}
""".format('myfirm', "f87987562", 2)
This is the error that I am receiving:
19 }
20 }
---> 21 """.format('myfirm', "f87987562", 2)
22
23 apikey = "secret_key"
KeyError: '\n "firm"'
I suspect it has something to do with the backslash, but its implementation seems necessary. Any help and insight on the intuition behind this string formatting is greatly appreciated.
I am trying to pass the string literal into the request function:
response = requests.request("POST", url, data=payload, headers=headers)
In a format string, { and } are special. To embed a literal parenthesis, use {{ or }}.
payload = """{{
"firm": "{0}",
"id": "{1}",
"data": {{
"tracking_preference": {2}
}}
}}
""".format('myfirm', "f87987562", 2)
print(payload)
Output:
{
"firm": "myfirm",
"id": "f87987562",
"data": {
"tracking_preference": 2
}
}
In Python 3.6+, f-strings can make this simpler:
firm = 'myfirm'
id = 'f87987562'
tracking = 2
payload = f'''{{
"firm": "{firm}",
"id": "{id}",
"data": {{
"tracking_preference": {tracking}
}}
}}'''
Finally, the json module is ideal for this specific scenario:
import json
firm = 'myfirm'
id = 'f87987562'
tracking = 2
data = {'firm':firm,'id':id,'data':{'tracking_preference':tracking}}
payload = json.dumps(data,indent=2) # indent is optional for easy reading
print(payload)
Output:
{
"firm": "myfirm",
"id": "f87987562",
"data": {
"tracking_preference": 2
}
}
If you're using python 3.6+, you could use an f-string:
payload = \
f"""{
"firm": {"myfirm"},
"id": {"f87987562"},
"data": {
"tracking_preference": {2}
}
}
"""
If not, you might be better off with a string template:
from string import Template
payload_t = Template(
"""{
"firm": ${firm},
"id": ${id},
"data": {
"tracking_preference": ${tracking}
}
}
""")
payload = payload_t.substitute(firm="myfirm", id="f87987562", tracking=2)
With the following simple Python script:
import json
file = 'toy.json'
data = json.loads(file)
print(data['gas']) # example
My data generates the error ...is not JSON serializable.
With this, slightly more sophisticated, Python script:
import json
import sys
#load the data into an element
data = open('transactions000000000029.json', 'r')
#dumps the json object into an element
json_str = json.dumps(data)
#load the json to a string
resp = json.loads(json_str)
#extract an element in the response
print(resp['gas'])
The same.
What I'd like to do is extract all the values of a particular index, so ideally I'd like to render the input like so:
...
"hash": "0xf2b5b8fb173e371cbb427625b0339f6023f8b4ec3701b7a5c691fa9cef9daf63",
"gasUsed": "21000",
"hash": "0xf8f2a397b0f7bb1ff212b6bcc57e4a56ce3e27eb9f5839fef3e193c0252fab26"
"gasUsed": "21000"
...
The data looks like this:
{
"blockNumber": "1941794",
"blockHash": "0x41ee74e34cbf9ef4116febea958dbc260e2da3a6bf6f601bfaeb2cd9ab944a29",
"hash": "0xf2b5b8fb173e371cbb427625b0339f6023f8b4ec3701b7a5c691fa9cef9daf63",
"from": "0x3c0cbb196e3847d40cb4d77d7dd3b386222998d9",
"to": "0x2ba24c66cbff0bda0e3053ea07325479b3ed1393",
"gas": "121000",
"gasUsed": "21000",
"gasPrice": "20000000000",
"input": "",
"logs": [],
"nonce": "14",
"value": "0x24406420d09ce7440000",
"timestamp": "2016-07-24 20:28:11 UTC"
}
{
"blockNumber": "1941716",
"blockHash": "0x75e1602cad967a781f4a2ea9e19c97405fe1acaa8b9ad333fb7288d98f7b49e3",
"hash": "0xf8f2a397b0f7bb1ff212b6bcc57e4a56ce3e27eb9f5839fef3e193c0252fab26",
"from": "0xa0480c6f402b036e33e46f993d9c7b93913e7461",
"to": "0xb2ea1f1f997365d1036dd6f00c51b361e9a3f351",
"gas": "121000",
"gasUsed": "21000",
"gasPrice": "20000000000",
"input": "",
"logs": [],
"nonce": "1",
"value": "0xde0b6b3a7640000",
"timestamp": "2016-07-24 20:12:17 UTC"
}
What would be the best way to achieve that?
I've been thinking that perhaps the best way would be to reformat it as valid json?
Or maybe to just treat it like regex?
Your json file is not valid. This data should be a list of dictionaries. You should then separate each dictionary with a comma, Like this:
[
{
"blockNumber":"1941794",
"blockHash": "0x41ee74bf9ef411d9ab944a29",
"hash":"0xf2ef9daf63",
"from":"0x3c0cbb196e3847d40cb4d77d7dd3b386222998d9",
"to":"0x2ba24c66cbff0bda0e3053ea07325479b3ed1393",
"gas":"121000",
"gasUsed":"21000",
"gasPrice":"20000000000",
"input":"",
"logs":[
],
"nonce":"14",
"value":"0x24406420d09ce7440000",
"timestamp":"2016-07-24 20:28:11 UTC"
},
{
"blockNumber":"1941716",
"blockHash":"0x75e1602ca8d98f7b49e3",
"hash":"0xf8f2a397b0f7bb1ff212e193c0252fab26",
"from":"0xa0480c6f402b036e33e46f993d9c7b93913e7461",
"to":"0xb2ea1f1f997365d1036dd6f00c51b361e9a3f351",
"gas":"121000",
"gasUsed":"21000",
"gasPrice":"20000000000",
"input":"",
"logs":[
],
"nonce":"1",
"value":"0xde0b6b3a7640000",
"timestamp":"2016-07-24 20:12:17 UTC"
}
]
Then use this to open the file:
with open('toy.json') as data_file:
data = json.load(data_file)
You can then render the desired output like:
for item in data:
print item['hash']
print item['gasUsed']
If each block is valid JSON data you can parse them seperatly:
data = []
with open('transactions000000000029.json') as inpt:
lines = []
for line in inpt:
if line.startswith('{'): # block starts
lines = [line]
else:
lines.append(line)
if line.startswith('}'): # block ends
data.append(json.loads(''.join(lines)))
for block in data:
print("hash: {}".format(block['hash']))
print("gasUsed: {}".format(block['gasUsed']))
i want to append below json with the data
meta = [{
"output_metadata": {
"api_URL": apiURL,
"query_execution_time": queryExecTime,
"api_execution_time": apiExecTime,
}
}]
jsondata = json.dumps([dict(ix) for ix in Data], default=str)
json data:
{"data": [{"id": "1234", "name": "jhon", "dept": "APA"}]}
meta.append(jsondata)
expected result:
{"output_metadata": {"api_url": "xxxxx", "query_execution_time":"xxxxx", "api_execution_time":"xxxxx"}},{"data": "[{"id": "1234", "name": "jhon", "dept": "APA"}]}
output:
{"output_metadata": {"api_url": "XXXXXX", "query_execution_time": "XXXXXX", "api_execution_time":"XXXXXX" }},{"data": "[{"\id": "1234\", "\name": "\jhon", "\dept": "\APA"}]}
How to remove \ from the final output?
If this thing you wrote above is python the meta variable you create is invalid because before every " you should use an escape character and every time you go in a new line. For example you should write:
meta = ["{\
\"output_metadata\": {\
\"api_URL\": apiURL,\
\"query_execution_time\": queryExecTime,\
\"api_execution_time\": apiExecTime, \
}\
}"]
data = ["{\"data\": {\"id\": \"1234\", \"name\": \"jhon\", \"dept\": \"APA\"}]}"]
meta.append(data)
Where you handle the json's as strings and then append them in one list. Is this what you want?
EDIT: if you run something like
data = [{"id": 1234, "name": "jhon", "dept": "APA" }]
jdata= json.dumps([dict(ix) for ix in data], default=str)
apiURL = 'url'
queryExecTime = 1
apiExecTime = 1
meta = [{ "output_metadata": { "api_url": apiURL,
"query_execution_time": queryExecTime,
"api_execution_time": apiExecTime, } }]
jdata = { "data": jdata }
meta.append(jdata)
res = json.dumps(meta)
print(res)
the result will be:
'[{"output_metadata": {"api_url": "url", "query_execution_time": 1, "api_execution_time": 1}}, {"data": "[{\\"id\\": 1234, \\"name\\": \\"jhon\\", \\"dept\\": \\"APA\\"}]"}]'
The \ are used as escape characters for the ". You see the result as a literal string.