We Keep Coding

Import JSON in Python error BadStatusLine

I'm trying to import the Json data generated by an Impinj R420 reader.
The code i use is:
# import urllib library
import urllib.request
from urllib.request import urlopen
# import json
import json
# store the URL in url as
# parameter for urlopen
url = "http://10.234.92.19:14150"
# store the response of URL
response = urllib.request.urlopen(url)
# storing the JSON response
# from url in data
data_json = json.loads(response())
# print the json response
print(data_json)
When i execute the programm it gives the following error:
Traceback (most recent call last):
File "C:\Users\V\PycharmProjects\Stapelcontrole\main.py", line 13, in <module>
response = urllib.request.urlopen(url)
File "C:\Users\V\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 216, in urlopen
return opener.open(url, data, timeout)
File "C:\Users\V\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 519, in open
response = self._open(req, data)
File "C:\Users\V\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 536, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "C:\Users\V\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 496, in _call_chain
result = func(*args)
File "C:\Users\V\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 1377, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "C:\Users\V\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 1352, in do_open
r = h.getresponse()
File "C:\Users\V\AppData\Local\Programs\Python\Python310\lib\http\client.py", line 1374, in getresponse
response.begin()
File "C:\Users\V\AppData\Local\Programs\Python\Python310\lib\http\client.py", line 318, in begin
version, status, reason = self._read_status()
File "C:\Users\V\AppData\Local\Programs\Python\Python310\lib\http\client.py", line 300, in _read_status
raise BadStatusLine(line)
http.client.BadStatusLine: {"epc":"3035307B2831B383E019E8EA","firstSeenTimestamp":"2022-04-11T11:24:23.592434Z","isHeartBeat":false}
Process finished with exit code 1
I know this is an error in the response where it gets a faulty HTTP status code.
Yet i don't know how to fix the error.
Could you advice me how to fix this?
The {"epc":"3035307B2831B383E019E8EA","firstSeenTimestamp":"2022-04-11T11:24:23.592434Z","isHeartBeat":false} is an answer i expect.
Thanks in advance
Edit:
Even with
with urllib.request.urlopen(url) as f:
data_json = json.load(f)`
i get the same BadStatusLine error.
I can't setup the reader any different, it can only sent a JSON response trough the IP-adress of the device. Is there a way to import the data without the HTTP Protocol?

# store the response of URL
response = urllib.request.urlopen(url)
# storing the JSON response
# from url in data
data_json = json.loads(response())
Here you are actually calling response, I do not know what you want to achieve by that, but examples in urllib.request docs suggest that urllib.request.urlopen should be treated akin to local file handle, thus please replace above using
with urllib.request.urlopen(url) as f:
data_json = json.load(f)
Observe that I used json.load not json.loads
EDIT: After Is there a way to import the data without the HTTP Protocol? I conclude more low-level solution is needed, hopefully socket will you allow to get what you want, using echo client example as starting point I prepared following code
import socket
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.connect(("10.234.92.19",14150))
s.sendall(b'GET / HTTP/1.1\r\n')
data = s.recv(1024)
print(data)
If everything will work as intended you should get printed 1024 first bytes of answer. If it so change 1024 to value which will be always bigger than number of bytes of response and use json.dumps(data) to get parsed response

Can't retrieve the data from API

I'm trying to retrieve the price data from the following API but I keep getting this error message when I execute the code.
def upbitask_xrp_krw_get():
result = requests.get("https://crix-api-endpoint.upbit.com/v1/crix/candles/minutes/1?code=CRIX.UPBIT.KRW-XRP&count=1").json()
return result["highPrice"]
And the error message I get is
Traceback (most recent call last):
File "test.py", line 194, in <module>
main()
File "test.py", line 130, in main
upbitask_xrp_krw = Decimal(upbitask_xrp_krw_get())
File "test.py", line 10, in upbitask_xrp_krw_get
"https://crix-api-endpoint.upbit.com/v1/crix/candles/minutes/1?code=CRIX.UPBIT.KRW-XRP&count=1").json()
File "/usr/lib/python3/dist-packages/requests/models.py", line 793, in json
return json.loads(self.text, **kwargs)
File "/usr/lib/python3.4/json/__init__.py", line 318, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.4/json/decoder.py", line 343, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.4/json/decoder.py", line 361, in raw_decode
raise ValueError(errmsg("Expecting value", s, err.value)) from None
ValueError: Expecting value: line 1 column 1 (char 0)
I've successfully accessed the data from other API which only has {} brackets whereas this one has [] brackets which I assume is the problem here?

In order to use the requests module you need to import it in your script first. this is true for any Python module you wish to use.
It seems the object returned by the API endpoint is a list of JSON and not simply a JSON object. Try this:
import requests
def upbitask_xrp_krw_get():
result = requests.get("https://crix-api-endpoint.upbit.com/v1/crix/candles/minutes/1?code=CRIX.UPBIT.KRW-XRP&count=1").json()
return result[0]["highPrice"]
The [0] indicates to return the first item of the list.
Here is the result:
Update
It seems there is a rate limite that prevents to query the API to often. When you reach the limit you get a 403 response code. Here is an example how you could manage it:
import requests
def upbitask_xrp_krw_get():
result = requests.get("https://crix-api-endpoint.upbit.com/v1/crix/candles/minutes/1?code=CRIX.UPBIT.KRW-XRP&count=1")
if result.status_code == 200: # Http code 200 means success
json = result.json()
high_price = json[0]["highPrice"]
else:
high_price = None
return high_price

I'm not exactly sure why you're getting that error, but you can bypass it by just calling json.loads() on the text of the request:
result = requests.get("https://crix-api-endpoint.upbit.com/v1/crix/candles/minutes/1?code=CRIX.UPBIT.KRW-XRP&count=1")
result_text = result.text
json_data = json.loads(result_text)
return json_data[0]["highPrice"]
You'll need to import json however. Also, you'll need to get the dictionary out of the list (hence the [0]).
EDIT: This is actually really bizarre: sometimes it's printing out a result and other times it's raising an error. It could have to do with the request being different over a very short period of time.

I believe this may work better:
def upbitask_xrp_krw_get():
result = requests.get("https://crix-api-endpoint.upbit.com/v1/crix/candles/minutes/1?code=CRIX.UPBIT.KRW-XRP&count=1").json()
return result[0]["highPrice"]
Your result is a list of json data. Just need to select the index you want in the return value.
EDIT: oh neat, there are different results after subsequent requests. There must be some kind of rate limiting, the first time I hit the API I get the list, the next time i hit it I just get a 403 response

I m just a beginner in python i did some research and this worked for me
import json
import requests
url ='https://crix-api-endpoint.upbit.com/v1/crix/candles/minutes/1?code=CRIX.UPBIT.KRW-XRP&count=1'
response = requests.get(url)
data = response.json()
print(data[0]['code'])
This worked for me
Thanks,

How to add dictionary to json object

In my construct below, I am trying to pass a JSON object through my web service. As a new requirement I have to pass a dictionary object which is sent in the code below. Can you please guide me how to add the dictionary to JSON object?
if plain_text is not None:
blob = TextBlob(plain_text)
sentiment = TextBlob(plain_text)
sent = {}
for sentence in blob.sentences:
sent[sentence] =sentence.sentiment.polarity
print sent
return json.dumps(
{'input' : plain_text,
'Polarity': sentiment.polarity,
#'sent': json.dumps(sent) # this is where I am stuck as this 'sent' is a dict
},
indent=4)
If I uncomment the line I get the below error:
Exception:
TypeError('keys must be a string',)
Traceback:
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\bottle-0.12.7-py2.7.egg\bottle.py", line 862, in _handle
return route.call(**args)
File "C:\Python27\lib\site-packages\bottle-0.12.7-py2.7.egg\bottle.py", line 1729, in wrapper
rv = callback(*a, **ka)
File "C:\Users\hp\Desktop\RealPy\WebServices\bottle\server_bckup.py", line 53, in sentimentEngine
'sent': json.dumps(sent),
File "C:\Python27\lib\json\__init__.py", line 231, in dumps
return _default_encoder.encode(obj)
File "C:\Python27\lib\json\encoder.py", line 201, in encode
chunks = self.iterencode(o, _one_shot=True)
File "C:\Python27\lib\json\encoder.py", line 264, in iterencode
return _iterencode(o, 0)
TypeError: keys must be a string

In JSON, dictionary keys must be strings. You have a Python dictionary sent, that you want to serialize into JSON. This fails, as your dictionary sent has keys that are not strings, but textblob.blob.Sentence instances.
If it makes sense, you can change your code to read:
for sentence in blob.sentences:
sent[str(sentence)] = sentence.sentiment.polarity
Or, you can customize the Python JSON encoder to know how to serialize TextBlob Sentences.

How to convert a string data to a JSON object in python?

Here is the sample string i am receiving from one of the web services,
body=%7B%22type%22%3A%22change%22%2C%22url%22%3A%22http%3A%2F%2Fapi.pachube.com%2Fv2%2Ftriggers%2F4100%22%2C%22environment%22%3A%7B%22feed%22%3A%22http%3A%2F%2Fapi.pachube.com%2Fv2%2Ffeeds%2F36133%22%2C%22title%22%3A%22Current+Cost+Bridge%22%2C%22description%22%3Anull%2C%22id%22%3A36133%7D%2C%22threshold_value%22%3Anull%2C%22timestamp%22%3A%222012-01-05T09%3A27%3A01Z%22%2C%22triggering_datastream%22%3A%7B%22url%22%3A%22http%3A%2F%2Fapi.pachube.com%2Fv2%2Ffeeds%2F36133%2Fdatastreams%2F1%22%2C%22value%22%3A%7B%22value%22%3A%22523%22%2C%22max_value%22%3A1269.0%2C%22min_value%22%3A0.0%7D%2C%22id%22%3A%221%22%2C%22units%22%3A%7B%22symbol%22%3A%22W%22%2C%22type%22%3A%22derivedUnits%22%2C%22label%22%3A%22watts%22%7D%7D%2C%22id%22%3A4100%7D
Here is the code,
class Feeds():
def GET(self):
print "Get request is accepted."
return render.index(None)
def POST(self):
print "Post request is accepted."
print (web.data())
Now when that web-service posts the above given data, how will i convert it to readable format? Then, i need to convert it to JSON object and use further. So, how will i convert it?
When i try this code,
json_data = json.loads(web.data())
print json_data['body']
return render.index(json_data['body'])
It gives me an error,
enter code Traceback (most recent call last):
File "/usr/local/lib/python2.6/dist-packages/web/application.py", line 237, in process
return self.handle()
File "/usr/local/lib/python2.6/dist-packages/web/application.py", line 228, in handle
return self._delegate(fn, self.fvars, args)
File "/usr/local/lib/python2.6/dist-packages/web/application.py", line 409, in _delegate
return handle_class(cls)
File "/usr/local/lib/python2.6/dist-packages/web/application.py", line 385, in handle_class
return tocall(*args)
File "/home/ubuntu/pachubeConsumer/src/controllers/feeds.py", line 17, in POST
json_data = json.loads(web.data())
File "/usr/lib/python2.6/json/__init__.py", line 307, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.6/json/decoder.py", line 319, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.6/json/decoder.py", line 338, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded here
Where am i going wrong ??
Thanks in advance.

First you need to unquote the body
import urllib
body="%7B%22type%22%3A%22change%22%2C%22url%22%3A%22http%3A%2F%2Fapi.pachube.com%2Fv2%2Ftriggers%2F4100%22%2C%22environment%22%3A%7B%22feed%22%3A%22http%3A%2F%2Fapi.pachube.com%2Fv2%2Ffeeds%2F36133%22%2C%22title%22%3A%22Current+Cost+Bridge%22%2C%22description%22%3Anull%2C%22id%22%3A36133%7D%2C%22threshold_value%22%3Anull%2C%22timestamp%22%3A%222012-01-05T09%3A27%3A01Z%22%2C%22triggering_datastream%22%3A%7B%22url%22%3A%22http%3A%2F%2Fapi.pachube.com%2Fv2%2Ffeeds%2F36133%2Fdatastreams%2F1%22%2C%22value%22%3A%7B%22value%22%3A%22523%22%2C%22max_value%22%3A1269.0%2C%22min_value%22%3A0.0%7D%2C%22id%22%3A%221%22%2C%22units%22%3A%7B%22symbol%22%3A%22W%22%2C%22type%22%3A%22derivedUnits%22%2C%22label%22%3A%22watts%22%7D%7D%2C%22id%22%3A4100%7D"
unquoted = urllib.unquote(body)
Then just load the JSON like normal
import json
pythonDict = json.loads(unquoted)

you need to unescape the query string first using urllib.unquote()
import urllib
unescaped = urllib.unquote(web.data())
then you may use json.loads to convert it into json.
json_data = json.loads(unescaped)

Python 3 urllib unquote move to parse:
from urllib import parse
parse.unquote(web.data())

Error when parsing JSON data

I want to get elevation data from Google Earth according to latitude and longitude, but I am not able to do this. I'm not sure what I'm doing wrong but my code is shown below.
def getElevation(locations,sensor="true", **elvtn_args):
elvtn_args.update({
'locations': locations,
'sensor': sensor
})
url = ELEVATION_BASE_URL
params = urllib.parse.urlencode(elvtn_args)
baseurl = url +"?"+ params;
req = urllib.request.urlopen(str(baseurl));
response = simplejson.load(req);
And the error I get is :
Traceback (most recent call last):
File "D:\GIS\Arctools\ElevationChart - Copy.py", line 85, in <module>
getElevation(pathStr)
File "D:\GIS\Arctools\ElevationChart - Copy.py", line 45, in getElevation
response = simplejson.load(req);
File "C:\Python32\lib\json\__init__.py", line 262, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "C:\Python32\lib\json\__init__.py", line 307, in loads
return _default_decoder.decode(s)
File "C:\Python32\lib\json\decoder.py", line 351, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
TypeError: can't use a string pattern on a bytes-like object
Any help appreciated.

Post is a little late but recently ran into the same problem. The solution below worked for me. Basically what Lennart said.
from urllib import request
import json
req = request.urlopen('https://someurl.net/api')
encoding = req.headers.get_content_charset()
obj = json.loads(req.read().decode(encoding))

In Python 3, binary data, such as the raw response of a http request, is stored in bytes objects. json/simplejson expects strings. The solution is to decode the bytes data to string data with the appropriate encoding, which you can find in the header.
You find the encoding with:
encoding = req.headers.get_content_charset()
You then make the content a string by:
body = req.readall().decode(encoding)
This body you then can pass to the json loader.
(Also, please stop calling the response "req". It's confusing, and makes it sounds like it is a request, which it isn't.)