Here is the sample string i am receiving from one of the web services,
body=%7B%22type%22%3A%22change%22%2C%22url%22%3A%22http%3A%2F%2Fapi.pachube.com%2Fv2%2Ftriggers%2F4100%22%2C%22environment%22%3A%7B%22feed%22%3A%22http%3A%2F%2Fapi.pachube.com%2Fv2%2Ffeeds%2F36133%22%2C%22title%22%3A%22Current+Cost+Bridge%22%2C%22description%22%3Anull%2C%22id%22%3A36133%7D%2C%22threshold_value%22%3Anull%2C%22timestamp%22%3A%222012-01-05T09%3A27%3A01Z%22%2C%22triggering_datastream%22%3A%7B%22url%22%3A%22http%3A%2F%2Fapi.pachube.com%2Fv2%2Ffeeds%2F36133%2Fdatastreams%2F1%22%2C%22value%22%3A%7B%22value%22%3A%22523%22%2C%22max_value%22%3A1269.0%2C%22min_value%22%3A0.0%7D%2C%22id%22%3A%221%22%2C%22units%22%3A%7B%22symbol%22%3A%22W%22%2C%22type%22%3A%22derivedUnits%22%2C%22label%22%3A%22watts%22%7D%7D%2C%22id%22%3A4100%7D
Here is the code,
class Feeds():
def GET(self):
print "Get request is accepted."
return render.index(None)
def POST(self):
print "Post request is accepted."
print (web.data())
Now when that web-service posts the above given data, how will i convert it to readable format? Then, i need to convert it to JSON object and use further. So, how will i convert it?
When i try this code,
json_data = json.loads(web.data())
print json_data['body']
return render.index(json_data['body'])
It gives me an error,
enter code Traceback (most recent call last):
File "/usr/local/lib/python2.6/dist-packages/web/application.py", line 237, in process
return self.handle()
File "/usr/local/lib/python2.6/dist-packages/web/application.py", line 228, in handle
return self._delegate(fn, self.fvars, args)
File "/usr/local/lib/python2.6/dist-packages/web/application.py", line 409, in _delegate
return handle_class(cls)
File "/usr/local/lib/python2.6/dist-packages/web/application.py", line 385, in handle_class
return tocall(*args)
File "/home/ubuntu/pachubeConsumer/src/controllers/feeds.py", line 17, in POST
json_data = json.loads(web.data())
File "/usr/lib/python2.6/json/__init__.py", line 307, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.6/json/decoder.py", line 319, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.6/json/decoder.py", line 338, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded here
Where am i going wrong ??
Thanks in advance.
First you need to unquote the body
import urllib
body="%7B%22type%22%3A%22change%22%2C%22url%22%3A%22http%3A%2F%2Fapi.pachube.com%2Fv2%2Ftriggers%2F4100%22%2C%22environment%22%3A%7B%22feed%22%3A%22http%3A%2F%2Fapi.pachube.com%2Fv2%2Ffeeds%2F36133%22%2C%22title%22%3A%22Current+Cost+Bridge%22%2C%22description%22%3Anull%2C%22id%22%3A36133%7D%2C%22threshold_value%22%3Anull%2C%22timestamp%22%3A%222012-01-05T09%3A27%3A01Z%22%2C%22triggering_datastream%22%3A%7B%22url%22%3A%22http%3A%2F%2Fapi.pachube.com%2Fv2%2Ffeeds%2F36133%2Fdatastreams%2F1%22%2C%22value%22%3A%7B%22value%22%3A%22523%22%2C%22max_value%22%3A1269.0%2C%22min_value%22%3A0.0%7D%2C%22id%22%3A%221%22%2C%22units%22%3A%7B%22symbol%22%3A%22W%22%2C%22type%22%3A%22derivedUnits%22%2C%22label%22%3A%22watts%22%7D%7D%2C%22id%22%3A4100%7D"
unquoted = urllib.unquote(body)
Then just load the JSON like normal
import json
pythonDict = json.loads(unquoted)
you need to unescape the query string first using urllib.unquote()
import urllib
unescaped = urllib.unquote(web.data())
then you may use json.loads to convert it into json.
json_data = json.loads(unescaped)
Python 3 urllib unquote move to parse:
from urllib import parse
parse.unquote(web.data())
Related
I am getting the error while executing the following python script.
The issue is with converting html output to json . It will be very helpful if you help me to solve the problem.
import urllib2
import json`
url ='http://localhost:40102/cq/etc/replication/agents.author/publish-
u11cmspu1.html'
username='admin'
password='admin'
passman = urllib2.HTTPPasswordMgrWithDefaultRealm()
passman.add_password(None, url, username, password)``
authhandler = urllib2.HTTPBasicAuthHandler(passman)
opener = urllib2.build_opener(authhandler)
urllib2.install_opener(opener)
pagehandle = urllib2.urlopen(url)
json_response=json.loads(pagehandle.read())
# output=json.load(pagehandle.read())
print json_response
ERROR :
Traceback (most recent call last):
File "./repQstatus.py", line 18, in <module>
json_response=json.loads(pagehandle.read())
File "/usr/lib64/python2.6/json/__init__.py", line 307, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python2.6/json/decoder.py", line 319, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python2.6/json/decoder.py", line 338, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
What is going on here is the return value of pagehandle.read() is returning an string that is not identifiable as JSON by the python json library.
There could definitely still be JSON inside the response of pagehandle.read(), but something is keeping it from being properly evaluated.
I am trying to get JSON and print VIA python scirpt. I got response from server 200, not sure why i can not able print the file. Please help me on this!!
The Code used is:
Import Requests
response = requests.get("https://Site1/rest/settings/all-server-status", params={'serverId': '56cd7e4d2d0edcace915e674'}, verify=False)
json_data = json.loads(response.text)
I got below error:
Traceback (most recent call last):
File "<pyshell#51>", line 1, in <module>
json_data = json.loads(response.text)
File "C:\Python27\lib\json\__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "C:\Python27\lib\json\decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Python27\lib\json\decoder.py", line 384, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
I tried below statements also but no response:
json_data = json.loads(response)
json_data = json.loads(response.read())
The Json Sample output expecting is:
[{"id":"56cd7e4d2d0edcace915e674","protocol":"https","hostName":"x.x.x.x","port":443,"serverName":"Site1","status":"connected","connected":true}]
Thanks in advance!
The POST request you are making is not returning anything, however the Response 200 indicates the connection was successful(No SSL error etc).
The problem is here: params={'serverId': '56cd7e4d2d0edcace915e674'}.
I would suggest debugging your request.get().
Start by checking if the https://hostname.com/key=value is valid.
I'm trying to get some data from this website. I can enter 'text' and 'longest_only' parameters but when I pass 'ontologies' param, it says No JSON object could be decoded. Here's the complete URL http://data.bioontology.org/annotator?text=lung cancer,bone marrow&ontologies=NCIT&longest_only=true
I'm using Python 2.7
The argument is ontologies[], since you can specify more than one. Your request should be similar to the one that the online search uses:
text=lung+cancer%2Cbone+marrow&ontologies%5B%5D=NCIT&longest_only=true&raw=true
Simply execute the same search there, and use the developer tools option of your favorite browser to check what is the actual payload being sent.
This is not an answer, but the only place I can show the error that I see when executing the sample code. I placed the code in a new module in main and run it in Python 3.4.
import requests
if __name__ == '__main__':
url = 'http://bioportal.bioontology.org/annotator'
params = {
'text': 'lung cancer,bone marrow',
'ontologies': 'NCIT',
'longest_only': 'true'
}
session = requests.Session()
session.get(url)
response = session.post(url, data=params)
data = response.json()
# get the annotations
for annotation in data['annotations']:
print (annotation['annotatedClass']['prefLabel'])
I receive the following error.
Traceback (most recent call last):
File "/Users/.../Sandbox/Ontology.py", line 21, in <module>
data = response.json()
File "/Users/erwin/anaconda/lib/python3.4/site-packages/requests/models.py", line 799, in json
return json.loads(self.text, **kwargs)
File "/Users/erwin/anaconda/lib/python3.4/json/__init__.py", line 318, in loads
return _default_decoder.decode(s)
File "/Users/erwin/anaconda/lib/python3.4/json/decoder.py", line 343, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/Users/erwin/anaconda/lib/python3.4/json/decoder.py", line 361, in raw_decode
raise ValueError(errmsg("Expecting value", s, err.value)) from None
ValueError: Expecting value: line 1 column 1 (char 0)
I use the following code:
import traceback
errors = open('ERROR(S).txt', 'a')
try:
execfile("testing.py", {})
except Exception:
errors.write(traceback.format_exc() + '\n')
errors.write("\n")
errors.close()
For example I might receive
Traceback (most recent call last):
File "C:\starter.py", line 87, in execute_subscripts
execfile("test.py", {})
File "test.py", line 85, in <module>
JSONDATA = json.loads(JSONDATA)
File "C:\Python27\lib\json\__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "C:\Python27\lib\json\decoder.py", line 365, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Python27\lib\json\decoder.py", line 383, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
How can I also make it retrieve the the data inside the variable JSONDATA as well as this traceback?
You could just use the logging library and log the JSONDATA either to stderr with a logging.StreamHandler or to a file with a logging.FileHandler.
Example:
l = logging.getLogger("test")
l.addHandler(logging.StreamHandler())
l.setLevel(logging.DEBUG)
...
l.debug("JSON DATA: %s", JSONDATA)
...
I would think this is the proper way to find out what went wrong. (Alternatively you could use simple print or if you want to go into detail use pdb)
I want to get elevation data from Google Earth according to latitude and longitude, but I am not able to do this. I'm not sure what I'm doing wrong but my code is shown below.
def getElevation(locations,sensor="true", **elvtn_args):
elvtn_args.update({
'locations': locations,
'sensor': sensor
})
url = ELEVATION_BASE_URL
params = urllib.parse.urlencode(elvtn_args)
baseurl = url +"?"+ params;
req = urllib.request.urlopen(str(baseurl));
response = simplejson.load(req);
And the error I get is :
Traceback (most recent call last):
File "D:\GIS\Arctools\ElevationChart - Copy.py", line 85, in <module>
getElevation(pathStr)
File "D:\GIS\Arctools\ElevationChart - Copy.py", line 45, in getElevation
response = simplejson.load(req);
File "C:\Python32\lib\json\__init__.py", line 262, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "C:\Python32\lib\json\__init__.py", line 307, in loads
return _default_decoder.decode(s)
File "C:\Python32\lib\json\decoder.py", line 351, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
TypeError: can't use a string pattern on a bytes-like object
Any help appreciated.
Post is a little late but recently ran into the same problem. The solution below worked for me. Basically what Lennart said.
from urllib import request
import json
req = request.urlopen('https://someurl.net/api')
encoding = req.headers.get_content_charset()
obj = json.loads(req.read().decode(encoding))
In Python 3, binary data, such as the raw response of a http request, is stored in bytes objects. json/simplejson expects strings. The solution is to decode the bytes data to string data with the appropriate encoding, which you can find in the header.
You find the encoding with:
encoding = req.headers.get_content_charset()
You then make the content a string by:
body = req.readall().decode(encoding)
This body you then can pass to the json loader.
(Also, please stop calling the response "req". It's confusing, and makes it sounds like it is a request, which it isn't.)