We Keep Coding

VK_API: Problems with music, json decoder

I'm trying to make a special parser for VK, which downloads all music from page of some user, but there's some problem with vk_api, which allows to access audiofiles. I'm trying to call method get() to get list of all tracks, but launch of this program:
session = vk_api.VkApi(token=tkn)
vk = session.get_api()
vk_audio = audio.VkAudio(session)
def get_list_audio():
dct = vk_audio.get(owner_id=owner_id, album_id=None, access_hash=None)
return dct
print(get_list_audio())
shows nothing but an error, connected with json decoder:
Traceback (most recent call last):
File "C:\Users\ann\PycharmProjects\vk-parser\main.py", line 44, in <module>
print(get_list_audio())
File "C:\Users\ann\PycharmProjects\vk-parser\main.py", line 38, in get_list_audio
dct = vk_audio.get(owner_id=owner_id, album_id=None, access_hash=None)
File "D:\PycharmProjects\vk-parser\lib\site-packages\vk_api\audio.py", line 158, in get
return list(self.get_iter(owner_id, album_id, access_hash))
File "D:\PycharmProjects\vk-parser\lib\site-packages\vk_api\audio.py", line 107, in get_iter
response = self._vk.http.post(
File "D:\PycharmProjects\vk-parser\lib\site-packages\requests\models.py", line 900, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Python 3.9\lib\json\__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "C:\Python 3.9\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Python 3.9\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Please, give some hints, what can I do?

vk_api/audio.py makes just POST request to m.vk.com/audio:
and the error occures when it tries to convert the response to json (line 119). If we run these lines manually (or you can use debugger anytime to see what happens), we see 302 http error with empty response. json() can't get json from this empty response, so you see the exception. You have to choose another library (to work with vk api) or to write these actions yourself. Or to fix this lib code :)
Probably, this vk_api library does not support the changes that have occurred in vk.com. You can take this code from the library and make your own method for getting audio based on it.

Python Rejects Valid JSON

I'm trying to process this JSON using python3:
http://www.bom.gov.au/fwo/IDV60701/IDV60701.94857.json
But I'm getting the following error:
Traceback (most recent call last):
File "./weath.py", line 41, in <module>
data1 = response.json()
File "/home/dz/anaconda3/lib/python3.8/site-packages/requests/models.py", line 898, in json
return complexjson.loads(self.text, **kwargs)
File "/home/dz/anaconda3/lib/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/home/dz/anaconda3/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/home/dz/anaconda3/lib/python3.8/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
https://jsonlint.com/
Validates the JSON as valid, so I'm not sure why this is failing.
Here is the python code:
url = "http://www.bom.gov.au/fwo/IDV60701/IDV60701.94857.json"
response = requests.get(url)
data1 = response.json()
This was working 2 week ago.
How can I fix this?

This is an issue related to the specific service endpoint you're using. They have disabled web scraping through some mechanism.
If you look at your response object, you'll see it's a 403 (forbidden) with the following message:
Potential automated request detected! We are making changes to our website therefore web scraping is no longer supported. Please contact us by filling in the details at http://reg.bom.gov.au/screenscraper/screenscraper_enquiry_form/ and we will get in touch with you.
You can verify this for yourself:
response = requests.get("http://www.bom.gov.au/fwo/IDV60701/IDV60701.94857.json")
print(response.status_code) # 403
print(response.text) # above quote

When I ran this code:
import requests
url = "http://www.bom.gov.au/fwo/IDV60701/IDV60701.94857.json"
response = requests.get(url).text
print(response)
The print returned
Potential automated request detected! We are making changes to our website therefore web scraping is no longer supported. Please contact us by filling in the details at http://reg.bom.gov.au/screenscraper/screenscraper_enquiry_form/ and we will get in touch with you.
Seems like that website has disabled web-scraping or something

I got this error simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

I hope you're doing good.
I'm trying to get the solar radiation values from this website 'solcast.com.au' .. I have went to their API documentation and followed it here ' https://docs.solcast.com.au/#forecasts-by-location' and I have applied the code:
import requests
url = 'https://api.solcast.com.au/world_radiation/forecasts?latitude= -33.865143&longitude=151.209900&api_key=MYAPI'
res = requests.get(url)
data = res.json()
forecast = data["forecasts"]["ghi"]
print('forecastss: {} dgree'.format(forecast))
So when I run the code I'm getting this error:
Traceback (most recent call last):
File "/home/pi/Desktop/solcastoo.py", line 5, in <module>
data = res.json()
File "/usr/lib/python3/dist-packages/requests/models.py", line 897, in json
return complexjson.loads(self.text, **kwargs)
File "/usr/lib/python3/dist-packages/simplejson/__init__.py", line 518, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 370, in decode
obj, end = self.raw_decode(s)
File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 400, in raw_decode
return self.scan_once(s, idx=_w(s, idx).end())
simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Would really appreciate your help.

As John mentioned, you need to specify in your request the format you're willing to receive.
You can do it by adding headers to your request:
import requests
url = 'https://api.solcast.com.au/world_radiation/forecasts?latitude= -33.865143&longitude=151.209900&api_key=API_KEY'
res = requests.get(url, headers={'Content-Type': 'application/json'})
data = res.json()
forecast = data["forecasts"][0]["ghi"]
print('forecastss: {} dgree'.format(forecast))
In their documentation they give you two additional options:
“Accepts” HTTP request header, eg “​application/json​” for ​JSON
“format” query string, eg “​format=json​” for ​JSON
Endpoint suffix file extension, eg “​forecasts.json​” for ​JSON
The second option doesn't work, at least for this specific request. The third option works, but it's a bit odd.
The first option is more commonly used in APIs, but be prepared to use other options too.
PS they say in the documentation that `headers={'Accepts': 'application/json'}
should give the desired result, so I'd assume it also could be a possibility in other endpoints.
Good luck

No JSON object could be decoded Python

I'm trying to get some data from this website. I can enter 'text' and 'longest_only' parameters but when I pass 'ontologies' param, it says No JSON object could be decoded. Here's the complete URL http://data.bioontology.org/annotator?text=lung cancer,bone marrow&ontologies=NCIT&longest_only=true
I'm using Python 2.7

The argument is ontologies[], since you can specify more than one. Your request should be similar to the one that the online search uses:
text=lung+cancer%2Cbone+marrow&ontologies%5B%5D=NCIT&longest_only=true&raw=true
Simply execute the same search there, and use the developer tools option of your favorite browser to check what is the actual payload being sent.

This is not an answer, but the only place I can show the error that I see when executing the sample code. I placed the code in a new module in main and run it in Python 3.4.
import requests
if __name__ == '__main__':
url = 'http://bioportal.bioontology.org/annotator'
params = {
'text': 'lung cancer,bone marrow',
'ontologies': 'NCIT',
'longest_only': 'true'
}
session = requests.Session()
session.get(url)
response = session.post(url, data=params)
data = response.json()
# get the annotations
for annotation in data['annotations']:
print (annotation['annotatedClass']['prefLabel'])
I receive the following error.
Traceback (most recent call last):
File "/Users/.../Sandbox/Ontology.py", line 21, in <module>
data = response.json()
File "/Users/erwin/anaconda/lib/python3.4/site-packages/requests/models.py", line 799, in json
return json.loads(self.text, **kwargs)
File "/Users/erwin/anaconda/lib/python3.4/json/__init__.py", line 318, in loads
return _default_decoder.decode(s)
File "/Users/erwin/anaconda/lib/python3.4/json/decoder.py", line 343, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/Users/erwin/anaconda/lib/python3.4/json/decoder.py", line 361, in raw_decode
raise ValueError(errmsg("Expecting value", s, err.value)) from None
ValueError: Expecting value: line 1 column 1 (char 0)

Error when parsing JSON data

I want to get elevation data from Google Earth according to latitude and longitude, but I am not able to do this. I'm not sure what I'm doing wrong but my code is shown below.
def getElevation(locations,sensor="true", **elvtn_args):
elvtn_args.update({
'locations': locations,
'sensor': sensor
})
url = ELEVATION_BASE_URL
params = urllib.parse.urlencode(elvtn_args)
baseurl = url +"?"+ params;
req = urllib.request.urlopen(str(baseurl));
response = simplejson.load(req);
And the error I get is :
Traceback (most recent call last):
File "D:\GIS\Arctools\ElevationChart - Copy.py", line 85, in <module>
getElevation(pathStr)
File "D:\GIS\Arctools\ElevationChart - Copy.py", line 45, in getElevation
response = simplejson.load(req);
File "C:\Python32\lib\json\__init__.py", line 262, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "C:\Python32\lib\json\__init__.py", line 307, in loads
return _default_decoder.decode(s)
File "C:\Python32\lib\json\decoder.py", line 351, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
TypeError: can't use a string pattern on a bytes-like object
Any help appreciated.

Post is a little late but recently ran into the same problem. The solution below worked for me. Basically what Lennart said.
from urllib import request
import json
req = request.urlopen('https://someurl.net/api')
encoding = req.headers.get_content_charset()
obj = json.loads(req.read().decode(encoding))

In Python 3, binary data, such as the raw response of a http request, is stored in bytes objects. json/simplejson expects strings. The solution is to decode the bytes data to string data with the appropriate encoding, which you can find in the header.
You find the encoding with:
encoding = req.headers.get_content_charset()
You then make the content a string by:
body = req.readall().decode(encoding)
This body you then can pass to the json loader.
(Also, please stop calling the response "req". It's confusing, and makes it sounds like it is a request, which it isn't.)