The following code works but only uploads the one result at the index as per the below, see ['result'][2]["text"] - which uploads the 'text' result at index 2 perfectly:
with open("test.json") as json_file:
data = json.load(json_file)
connection.request('POST', '/1/batch', json.dumps({
"requests": [{
"method": "POST",
"path": "/1/classes/test",
"body": {
"playerName": data['results'][2]["text"],
"Url": data['results'][2]["Url"],
"Amount": data['results'][2]["Amount"]
}
}]
})
How can I loop through all of the json results here ([0] to eof) rather than having to change ['results'][0]["text"], press enter, ['results'][1]["text"] press enter, ['results'][2]["text"] press enter etc?
A simple for will solve it:
with open("test.json") as json_file:
data = json.load(json_file)
for entry in data['results']:
connection.request('POST', '/1/batch', json.dumps({
"requests": [{
"method": "POST",
"path": "/1/classes/test",
"body": {
"playerName": entry["text"],
"Url": entry["Url"],
"Amount": entry["Amount"]
}
}]
})
If you really need the index number, you can use the enumerate() function:
for index, entry in enumerate(data["results"]):
# ...
Edit:
If by '/1/batch' you mean '/2/batch' (eg. typo), then you will need to use enumerate(). Eg:
with open("test.json") as json_file:
data = json.load(json_file)
for index, entry in enumerate(data['results']):
connection.request('POST', '/%d/batch' % (index), json.dumps({
"requests": [{
"method": "POST",
"path": "/%d/classes/test" % (index),
"body": {
"playerName": entry["text"],
"Url": entry["Url"],
"Amount": entry["Amount"]
}
}]
})
Related
Hi guys I am trying to extract the same key but with different values over a long JSON response, but i keep getting :
KeyError: 'id'
Not sure what i am doing wrong, but i am accessing it using REST API:
This is what i have as a script :
from requests.auth import HTTPBasicAuth
import requests
import json
import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
def countries():
data = requests.get("https://10.24.21.4:8543/api/netim/v1/countries/", verify=False, auth=HTTPBasicAuth("admin", "admin"))
rep = data.json()
for cid in rep:
cid = rep["id"]
print(cid)
countries()
The response is rather long, but it is like this, you will see "id", and i need the respective values :
{
"items": [
{
"name": "Afghanistan",
"displayName": "Afghanistan",
"meta": {
"type": "COUNTRY"
},
"id": "AF",
"links": {
"self": {
"path": "/api/netim/v1/countries/AF"
}
}
},
{
"name": "Albania",
"displayName": "Albania",
"meta": {
"type": "COUNTRY"
},
"id": "AL",
"links": {
"self": {
"path": "/api/netim/v1/countries/AL"
}
}
},
{
"name": "Algeria",
"displayName": "Algeria",
"meta": {
"type": "COUNTRY"
},
"id": "DZ",
"links": {
"self": {
"path": "/api/netim/v1/countries/DZ"
}
}
},
{
"name": "American Samoa",
"displayName": "American Samoa",
"meta": {
"type
I rewrote your functions a little, You should now be able to get all teh IDs from the JSON response. I suggest you look into teh basics of Dictionaries and Lists in Python
from requests.auth import HTTPBasicAuth
import requests
import json
import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
def countries():
data = requests.get("https://10.24.21.4:8543/api/netim/v1/countries/", verify=False, auth=HTTPBasicAuth("admin", "admin"))
rep = data.json()
return [elem.get("id","") for elem in rep['items']]
countries()
Update:
If you wish to extract the value of the "path" key and simultaneously the value of the "id" key, You would need a list of dictionaries where every dictionary corresponds to a single record from the json.
the modified function is as follows:
def countries():
data = requests.get("https://10.24.21.4:8543/api/netim/v1/countries/", verify=False, auth=HTTPBasicAuth("admin", "admin"))
rep = data.json()
return [{"id":elem.get("id",""),"path":elem["links"]["self"]["path"]} for elem in rep['items']]
the get() returns a default value in case the key is absent in the dictionary. The function, new as well as the previous one, would not fail in case the values were not returned in the JSON response for the id and path keys
If you are sure that the value of links will always be available you can use the above function directly else you will have to write a custom function that would parse the key links and return an empty string if it is empty in the json
The response is not an array, it's a dictionary.
You want the "items" element of that dictionary:
for cid in rep['items']:
after making post call of API I want to extract specific key/value and than save it onto a text file.
What have been done so far:-
(1)Rest api call and return list
import requests
import json
#API details
url = "http://192.168.1.100:9792/api/scan"
body = json.dumps({"service":"scan", "user_id":"1", "action":"read_all", "code":"0"})
headers = {'Content-Type': 'application/json'}
#Making http post request
response = requests.post(url, headers=headers, data=body, verify=False)
#Decode response.json() method to a python dictionary for data process utilization
dictData = response.json()
#Json string
json_str = json.dumps(dictData)
print(json_str)
print(json_str) output as below
{
"node": [
{
"id": "123",
"ip": "10.101.10.1",
"model": "md1",
"type": "basic",
"name": "aaa"
},
{
"id": "456",
"ip": "10.101.10.2",
"model": "sp3",
"type": "extra",
"name": "bbb"
},
{
"id": "789",
"ip": "1.1.1.1",
"model": "md1",
"type": "basic",
"name": "ccc"
},
{
"id": "101",
"ip": "2.2.2.2",
"model": "advance",
"type": "sw1",
"name": "ddd"
}
],
"status": "success"
}
(2)Extract specific key/value, This is where I'm getting error to get the key/value from the list
for i in json_str["node"]:
if i["type"]=="basic" or i["type"]=="sw1" :
print(i["name"],i["ip"], i["type"])
I'm getting error
for i in json_str["node"]:
TypeError: string indices must be integers, not str
I tried change to json_str[0] but it still doesn't return the key/value that i want.
Please assist further. thanks
Just use back dictData as it already in dictionary
for i in dictData["node"]
You dumped the json into str
Again to work with dict first load the json and then try
json_str = json.dumps(dictData)
json_dict = json.loads(json_str)
print(json_dict)
I am trying to convert my json data into a dictionary with key the id of the json data. for example lets say i have the following json:
{
"id": "1",
"name": "John",
"surname": "Smith"
},
{
"id": "2",
"name": "Steve",
"surname": "Ger"
}
And i want to construct a new dictionary which includes the id as a key and save it into a file so i wrote the following code:
json_dict = []
request = requests.get('http://example.com/...')
with open("data.json", "w") as out:
loaded_data = json.loads(request.text)
for list_item in loaded_data:
json_dict.append({"id": list_item["id"], "data": list_item })
out.write(json.dumps(json_dict))
In the file i get the following output:
[{"data": {"id":"1",
"name":"John",
"Surname":"Smith"
}
},
{"data": {"id":"2",
"name":"Steve",
"Surname":"Ger"
}
},
]
Why the id is not included in my dict before data ?
I'm pretty sure you're looking at a ghost here. You probably tested wrong. It will go away when you try to create a minimal, complete, and verifiable example for us (i. e. with a fixed string as input instead of a request call, with a print instead of an out.write, etc).
This is my test in which I could not reproduce the problem:
entries = [
{
"id": "1",
"name": "John",
"surname": "Smith"
},
{
"id": "2",
"name": "Steve",
"surname": "Ger"
}
]
json_dict = []
for i in entries:
json_dict.append({"id":i["id"], "data": i})
json.dumps(json_dict, indent=2)
This prints as expected:
[
{
"id": "1",
"data": {
"id": "1",
"surname": "Smith",
"name": "John"
}
},
{
"id": "2",
"data": {
"id": "2",
"surname": "Ger",
"name": "Steve"
}
}
]
You could try this instead:
json_dict = []
with open('data.json', 'w) as f:
loaded_data = json.loads(request.text)
for list_item in loaded_data:
json_dict.append({list_item['id'] : list_item})
I am trying to interact with an API and running into issues accessing nested objects. Below is sample json output that I am working with.
{
"results": [
{
"task_id": "22774853-2b2c-49f4-b044-2d053141b635",
"params": {
"type": "host",
"target": "54.243.80.16",
"source": "malware_analysis"
},
"v": "2.0.2",
"status": "success",
"time": 227,
"data": {
"details": {
"as_owner": "Amazon.com, Inc.",
"asn": "14618",
"country": "US",
"detected_urls": [],
"resolutions": [
{
"hostname": "bumbleride.com",
"last_resolved": "2016-09-15 00:00:00"
},
{
"hostname": "chilitechnology.com",
"last_resolved": "2016-09-16 00:00:00"
}
],
"response_code": 1,
"verbose_msg": "IP address in dataset"
},
"match": true
}
}
]
}
The deepest I am able to access is the data portion which returns too much.... ideally I am just trying access as_owner,asn,country,detected_urls,resolutions
When I try to access details / response code ... etc I will get a KeyError. My nested json goes deeper then other Q's mentioned and I have tried that logic.
Below is my current code snippet and any help is appreciated!
import requests
import json
headers = {
'Content-Type': 'application/json',
}
params = (
('wait', 'true'),
)
data = '{"target":{"one":{"type": "ip","target": "54.243.80.16", "sources": ["xxx","xxxxx"]}}}'
r=requests.post('https://fakewebsite:8000/api/services/intel/lookup/jobs', headers=headers, params=params, data=data, auth=('apikey', ''))
parsed_json = json.loads(r.text)
#results = parsed_json["results"]
for item in parsed_json["results"]:
print(item['data'])
You just need to index correctly into the converted JSON. Then you can easily loop over a list of the keys you want to fetch, since they are all in the "details" dictionary.
import json
raw = '''\
{
"results": [
{
"task_id": "22774853-2b2c-49f4-b044-2d053141b635",
"params": {
"type": "host",
"target": "54.243.80.16",
"source": "malware_analysis"
},
"v": "2.0.2",
"status": "success",
"time": 227,
"data": {
"details": {
"as_owner": "Amazon.com, Inc.",
"asn": "14618",
"country": "US",
"detected_urls": [],
"resolutions": [
{
"hostname": "bumbleride.com",
"last_resolved": "2016-09-15 00:00:00"
},
{
"hostname": "chilitechnology.com",
"last_resolved": "2016-09-16 00:00:00"
}
],
"response_code": 1,
"verbose_msg": "IP address in dataset"
},
"match": true
}
}
]
}
'''
parsed_json = json.loads(raw)
wanted = ['as_owner', 'asn', 'country', 'detected_urls', 'resolutions']
for item in parsed_json["results"]:
details = item['data']['details']
for key in wanted:
print(key, ':', json.dumps(details[key], indent=4))
# Put a blank line at the end of the details for each item
print()
output
as_owner : "Amazon.com, Inc."
asn : "14618"
country : "US"
detected_urls : []
resolutions : [
{
"hostname": "bumbleride.com",
"last_resolved": "2016-09-15 00:00:00"
},
{
"hostname": "chilitechnology.com",
"last_resolved": "2016-09-16 00:00:00"
}
]
BTW, when you fetch JSON data using requests there's no need to use json.loads: you can access the converted JSON using the .json method of the returned request object instead of using its .text attribute.
Here's a more robust version of the main loop of the above code. It simply ignores any missing keys. I didn't post this code earlier because the extra if tests make it slightly less efficient, and I didn't know that keys could be missing.
for item in parsed_json["results"]:
if not 'data' in item:
continue
data = item['data']
if not 'details' in data:
continue
details = data['details']
for key in wanted:
if key in details:
print(key, ':', json.dumps(details[key], indent=4))
# Put a blank line at the end of the details for each item
print()
I am now able to upload a file and create an initial version for the file but am unable to update the version. Using the code in Step 6 of this tutorial does not work even after replacing versions:autodesk.core:File with versions:autodesk.bim360:File. From the error message, Request contains 0 includes instead of 1, it seems I need part of the included block from the example in Step 5 but I am not sure what it should look like; given the outer relationships blocks do not match in format between Steps 5 and 6, I am assuming the inner content varies as well.
def create_version_for_file(self, project_name, file_name, folder_id, object_id, storage_id):
project_id = self.get_project_id_by_name(project_name)
existing_object_id = self.get_version_id_for_file_in_folder(project_name, folder_id, file_name)
url = '{}data/v1/projects/{}/items'.format(self.DOMAIN, project_id)
logger.info('Starting version create at %s for file_name %s, folder %s, object %s',
url, file_name, folder_id, object_id)
if existing_object_id:
logger.info('Creating version for existing object')
data = self._get_version_data_for_existing_file(file_name, object_id, storage_id)
else:
logger.info('Creating version for new object')
data = self._get_version_json_for_new_file(file_name, folder_id, object_id)
response = self.session.post(url, json=data, headers={
'content-type': 'application/vnd.api+json',
'accept': 'application/vnd.api+json'
})
if response.status_code != status.HTTP_201_CREATED:
logger.warn('Version create for %s failed with status %s: %s', file_name, response.status_code,
response.content)
return None
return json.loads(response.content)
def _get_version_json_for_new_file(self, file_name, folder_id, object_id):
return {
"jsonapi": {"version": "1.0"},
"data": {
"type": "items",
"attributes": {
"displayName": file_name,
"extension": {
"type": "items:autodesk.bim360:File",
"version": "1.0"
}
},
"relationships": {
"tip": {
"data": {
"type": "versions",
"id": "1"
}
},
"parent": {
"data": {
"type": "folders",
"id": folder_id
}
}
}
},
"included": [
{
"type": "versions",
"id": "1",
"attributes": {
"name": file_name,
"extension": {
"type": "versions:autodesk.bim360:File",
"version": "1.0"
}
},
"relationships": {
"storage": {
"data": {
"type": "objects",
"id": object_id
}
}
}
}
]
}
def _get_version_data_for_existing_file(self, file_name, object_id, storage_id):
return {
"jsonapi": {
"version": "1.0"
},
"data": {
"type": "versions",
"attributes": {
"name": file_name,
"extension": {
"type": "versions:autodesk.bim360:File",
"version": "1.0"
}
},
"relationships": {
"item": {
"data": {
"type": "items",
"id": object_id
}
},
"storage": {
"data": {
"type": "objects",
"id": storage_id
}
}
}
}
}
Your payloads are correct. To create the second version, you have to send your request to the CreateVersion endpoint.
url = '{}data/v1/projects/{}/versions'.format(self.DOMAIN, project_id)
You must be missing a little detail that makes it not working, but since you provide so little description nobody can tell based on that ... In order to create a new version, you first need to create a new storage location, upload the file to that location, and post a new version, referencing the existing item id.
Alternatively take a look my that node js sample to upload a file and create a new item or version if the item exists.
The data for your updating the version seems incorrect to me, Try the following steps.
Create a new Storage location on the same folder where V1 of your file is located
upload v2 of your file to the new storage location created on the step below.
Get the item ID of the resource from the manifest result of your successful upload of V1 or call the following end point to obtain the ID https://developer.autodesk.com/en/docs/data/v2/reference/http/projects-project_id-folders-folder_id-contents-GET
Once you have all this information your data to pass in will look like this.
{
"jsonapi": { "version": "1.0" },
"data": {
"type": "versions",
"attributes": {
"name": "v2File.rvt",
"extension": { "type": "versions:autodesk.bim360:File", "version": "1.0"}
},
"relationships": {
"item": { "data": { "type": "items", "id": "urn:adsk.wipprod:dm.lineage:8jehs1iSQGu_wXpZ977bjA" } },
"storage": { "data": { "type": "objects", "id": "urn:adsk.objects:os.object:wip.dm.prod/a7fde689-62cf-49c1-9273-8df976568996.rvt" } }
}
}
}
The only difference I can think of is your objectId is not correct. Image below is proof of both versions uploaded with the API.