I usually use Powershell and have parsed JSONs from HTTP requests, successfully, before. I am now using Python and using the 'Requests' library. I have successfully got the JSON from the API. Here is the format it came through in (I removed some information and other fields).:
{'content': [
{
'ContactCompany': Star,
'ContactEmail': test#company.star,
'ContactPhoneNumber': 123-456-7894,
'assignedGroup': 'TR_Hospital',
'assignedGroupId': 'SGP000000132297',
'serviceClass': None, 'serviceReconId': None
}
]
}
I'm having trouble getting the values inside of the 'content.' With my Powershell experience in the past, I've tried:
tickets_json = requests.get(request_url, headers=api_header).json()
Tickets_Info = tickets_json.content
for tickets in tickets_info:
tickets.assignedGroup
How do I parse the JSON to get the information inside of 'Content' in Python?
tickets_json = requests.get(request_url, headers=api_header).json()
tickets_info = tickets_json['content']
for tickets in tickets_info:
print(tickets['assignedGroup'])
Related
I have a specific number of documents from the azure search index to be deleted and I need a solution in python for the same.
I created an index in azure search already and the format of the index is given below
> {
> "#odata.context": "https://{name}.search.windows.net/indexes({'index name'})/$metadata#docs(*)",
> "value": [
> {
> "#search.score": ,
> "content": "",
> "metadata_storage_name": "",
> "metadata_storage_path": "",
> "metadata_storage_file_extension": "",}]}
metadata_storage_path is the unique key for each document in azure search index.
I got 2 ways to go about the problem using azure-python SDK and python request module but both the methods are throwing me an error which is listed below.
method - 1 (using python request module)
I got the reference from azure documentation
https://learn.microsoft.com/en-us/rest/api/searchservice/addupdate-or-delete-documents
import json
import requests
api_key = "B346FEAB56E6D5*******"
headers = {
'api-key': f'{api_key}',
'Content-Type': 'application/json'
}
doc_idx = "Index name"
doc_url = f"https://{name}.search.windows.net/indexes/{doc_idx}-index/docs/search?api-version=2020-06-30-Preview"
payload = json.dumps({
"#search.action": "delete",
"key_field_name":({"metadata_storage_path": "aHR0cHM6Ly9mc2NvZ******"})
},
)
response = json.loads(requests.request("POST", doc_url, headers=headers, data=payload).text)
print(response)
I am getting the following error.
{'error': {'code': '',
'message': "The request is invalid.## Heading ## Details: parameters : The parameter 'key_field_name' in the request payload is not a valid parameter for the operation 'search'.\r\n"}}
I also tried manipulating the code but I am not able to make it work please let me know weather i am making some mistake in the code or is there some issues with the python request module and azure search.
Method - 2 (using azure python SDK)
I got the Reference from azure documentation.
https://learn.microsoft.com/en-us/python/api/azure-search-documents/azure.search.documents.searchclient?view=azure-python
I tried to delete one document inside the azure search index with azure python SDK and the code is given below.
from azure.core.credentials import AzureKeyCredential
from azure.search.documents.indexes import SearchIndexClient
from azure.search.documents import SearchClient
key = AzureKeyCredential('B346FEAB******')
doc_idx = "index name"
service_endpoint = f"https://{name}.search.windows.net/indexes/{doc_idx}-index/docs/search?api-version=2020-06-30-Preview"
search_client = SearchClient(service_endpoint, doc_idx , key,)
# result = search_client.delete_documents(documents=[DOCUMENT])
result = search_client.delete_documents(documents=[{"metadata_storage_name": "XYZ.jpg"}])
print("deletion of document succeeded: {}".format(result[0].succeeded))
I am getting the following error.
ResourceNotFoundError Traceback (most recent call last)
<ipython-input-7-88beecc15663> in <module>
13 # result = search_client.upload_documents(documents=[DOCUMENT])
14
---> 15 result = search_client.delete_documents(documents=[{"metadata_storage_name": "XYZ.jpg"}])----------------------------------------------
ResourceNotFoundError: Operation returned an invalid status 'Not Found'
I also tried using metadata_storage_path instead of metadata_storage_name and I got the same error.
please check the code and let me know where I am making mistake and also if there is any other method for deleting a specific document in azure search index.
You have not defined a variable for name.
service_endpoint = f"https://{name}.search.windows.net/indexes/{doc_idx}-index/docs/search?api-version=2020-06-30-Preview"
Becomes
https://.search.windows.net/indexes/Index%20name-index/docs/search?api-version=2020-06-30-Preview
So I am trying to locate and acquire data from an api, I am fine with actually getting the data which is in json format from the api into my python program, however I am having troubles searching through the json for the specific data I want.
Here is a basic idea of what the json file from the api looks like:
{
"data": {
"inventory": {
"object_name": {
"amount": 8,
},
(Obviously the } close, I just didn't copy them)
I am trying to locate the amount within the json file of a specific object.
So far, here is the code I have, however, I have run into the error json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)
I have researched the error and it appears to be caused usually by a faulty json file, however as I have imported the json file from an api, it having issues is not the case and must be an issue with some of the converting to strings and such I have done in my code.
data = requests.get('[api]',
headers={
"[api key name]" : "[api key]"
})
dataJson = data.json()
dataStr = str(dataJson)
amt = json.loads(dataStr)['data'].get('inventory').get('object_name').get('amount')
As stated previously, the main issue I have is actually collecting the data I need from the json endpoint, everything is fine with getting the data into the python script.
dataJson = data.json() is already python dict no need to json.loads(dataStr) just use
data = requests.get('[api]',
headers={
"[api key name]" : "[api key]"
})
dataJson = data.json()
amt = dataStr['data'].get('inventory').get('object_name').get('amount')
I am attempting to write out some JSON output into a csv file but first i am trying to understand how the data is structured. I am working from a sample script which connects to an API and pulls down data based a query specified.
The json is returned from the server with this query:
response = api_client.get_search_results(search_id, 'application/json')
body = response.read().decode('utf-8')
body_json = json.loads(body)
If i perform a
print(body_json.keys())
i get the following output:
dict_keys(['events'])
So from this is it right to assume that the entries i am really interested in are another dictionary inside the events dictionary?
If so how can i 'access' them?
Sample JSON data the search query returns to the variable above
{
"events":[
{
"source_ip":"10.143.223.172",
"dest_ip":"104.20.251.41",
"domain":"www.theregister.co.uk",
"Domain Path":"NULL",
"Domain Query":"NULL",
"Http Method":"GET",
"Protocol":"HTTP",
"Category":"NULL",
"FullURL":"http://www.theregister.co.uk"
},
{
"source_ip":"10.143.223.172",
"dest_ip":"104.20.251.41",
"domain":"www.theregister.co.uk",
"Domain Path":"/2017/05/25/windows_is_now_built_on_git/",
"Domain Query":"NULL",
"Http Method":"GET",
"Protocol":"HTTP",
"Category":"NULL",
"FullURL":"http://www.theregister.co.uk/2017/05/25/windows_is_now_built_on_git/"
},
]
}
Any help would be greatly appreciated.
Json.keys() only returns the keys associated with json.
Here is the code:
for key in json_data.keys():
for i in range(len(json_data[key])):
key2 = json_data[key][i].keys()
for k in key2:
print k + ":" + json_data[key][i][k]
Output:
Http Method:GET
Category:NULL
domain:www.theregister.co.uk
Protocol:HTTP
Domain Query:NULL
Domain Path:NULL
source_ip:10.143.223.172
FullURL:http://www.theregister.co.uk
dest_ip:104.20.251.41
Http Method:GET
Category:NULL
domain:www.theregister.co.uk
Protocol:HTTP
Domain Query:NULL
Domain Path:/2017/05/25/windows_is_now_built_on_git/
source_ip:10.143.223.172
FullURL:http://www.theregister.co.uk/2017/05/25/windows_is_now_built_on_git/
dest_ip:104.20.251.41
To answer your question: yes. Your body_json has returned a dictionary with a key of "events" which contains a list of dictionaries.
The best way to 'access' them would be to iterate over them.
A very rudimentary example:
for i in body_json['events']:
print(i)
Of course, during the iteration you could access the specific data that you needed by replacing print(i) with print(i['FullURL'])and saving it to a variable and so on.
It's important to note that whenever you're working with API's that return a JSON response, you're simply working with dictionaries and Python data structures.
Best of luck.
Following is my code in view:
RESULTS= {}
for k in RESULTS_LIST[0].iterkeys():
RESULTS[k] = list(results[k] for results in RESULTS_LIST)
RESULTS.pop('_id',None)
html_table = json2html.convert(json=RESULTS)
return render(request,'homepage.html',{'html_table':html_table})
here I am arranging a data fetched from Mongo Db in a JSON named as RESULTS and by using JSON2HTML package, it is successfully generated html table for data given in JSON. To embedd the html table code in a blank division at html page, I am doing:
<div>{{html_table}}</div>
But it failed to display the table on page. I have tried numerous ways to make it but didn't succeed. Please help me to resolve this issue. if any relevant example, you have done before , then please guide me in a right direction.
JS code is:
angular.module("homeapp",[])
.controller("homecontroller",['$http','$scope',function($http,$scope){
$scope.search= {
'keyword': null,
'option': null,
'startDate':null,
'lastDate': null
};
$scope.search_info=function(){
var search_req = {
url:'/home',
method:'POST',
data:$scope.search
}
$http(search_req) //in case I dont want any response back or
$http(search_req).success(function(response){
window.alert(response)
}) //in case I need to check response
}
}]);
I have got solution to this problem in an easy way via using js, which is given below and it worked.
$http(search_req).success(function(response){
angular.element(".search_results").append(response);
})
The JSON syntax definition say that
html/xml tags (like the <script>...</script> part) are not part of
valid json, see the description at http://json.org.
A number of browsers and tools ignore these things silently, but python does
not.
I'd like to insert the javascript code (google analytics) to get info about the users using this service (place, browsers, OS ...).
What do you suggest to do?
I should solve the problem on [browser output][^1] or [python script][^2]?
thanks,
Antonio
[^1]: Browser output
<script>...</script>
[{"key": "value"}]
[^2]: python script
#!/usr/bin/env python
import urllib2, urllib, json
url="http://.........."
params = {}
url = url + '?' + urllib.urlencode(params, doseq=True)
req = urllib2.Request(url)
headers = {'Accept':'application/json;text/json'}
for key, val in headers.items():
req.add_header(key, val)
data = urllib2.urlopen(req)
print json.load(data)
These sound like two different kinds of services--one is a user-oriented web view of some data, with visualizations, formatting, etc., and one is a machine-oriented data service. I would keep these separate, and maybe build the user view as an extension to the data service.