I'm run my code to extract required data from RNA central database based RNAcentral accession number.
Sometimes in the middle, the program stops and start showing this error:
Error Obtained
File "C:\Users\soura\Desktop\Thesis\Saurav Data\Data_Extraction_RNA_Central.py", line 53, in <module>
data = page.json()['results']
File "D:\programs\anaconda3\lib\site-packages\requests\models.py", line 900, in json
return complexjson.loads(self.text, **kwargs)
File "D:\programs\anaconda3\lib\json\__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "D:\programs\anaconda3\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "D:\programs\anaconda3\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
JSONDecodeError: Expecting value
This is the part of my code where I'm having an error
import requests
import time
ids = ['URS0000D57BCE', 'URS0000D57BCE', 'URS0000EEF870', 'URS00009C33DE']
for id in range(len(ids)):
# Accessing the database cross-reference url with exception handling and extracting info
page = ''
while page == '':
try:
page = requests.get('http://rnacentral.org/api/v1/rna/{}/xrefs.json'.format(ids[id]))
break
except:
print('Connection refused by the server at id {} and position {}'.format(ids[id], ids.index(ids[id])))
print('Lets me sleep for 5 seconds')
time.sleep(5)
continue
# Extracting json content from above url
data = page.json()['results']
Later I modified my code like this but still, I'm getting the same error:
I Changed page = '' to page = None
I thought since the error is for the None value of the page. I wrote a while loop in such a way until there is the None value of page, the code will re-run again and again.
import requests
import time
ids = ['URS0000D57BCE', 'URS0000D57BCE', 'URS0000EEF870', 'URS00009C33DE']
for id in range(len(ids)):
# Accessing the database cross-reference url with exception handling and extracting info
page = None
while page == None:
try:
page = requests.get('http://rnacentral.org/api/v1/rna/{}/xrefs.json'.format(ids[id]))
break
except:
print('Connection refused by the server at id {} and position {}'.format(ids[id], ids.index(ids[id])))
print('Lets me sleep for 5 seconds')
time.sleep(5)
continue
# Extracting json content from above url
data = page.json()['results']
Now, I'm getting this error:
Traceback (most recent call last):
File "Data_Extraction_RNA_Central.py", line 61, in <module>
data = page.json()['results']
File "C:\Users\soura\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\models.py", line 898, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Users\soura\AppData\Local\Programs\Python\Python37\lib\json\__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "C:\Users\soura\AppData\Local\Programs\Python\Python37\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\soura\AppData\Local\Programs\Python\Python37\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Please if anyone can help, it will be a great help for me. :-)
Related
here is the full error message for when i run the code that is under this error message:
PS C:\Users\Admin> & C:/Users/Admin/AppData/Local/Programs/Python/Python39/python.exe c:/Users/Admin/Desktop/dadjoke.py
Traceback (most recent call last):
File "c:\Users\Admin\Desktop\dadjoke.py", line 9, in <module>
data = json.loads(str(question.text ))
File "C:\Users\Admin\AppData\Local\Programs\Python\Python39\lib\json\__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "C:\Users\Admin\AppData\Local\Programs\Python\Python39\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\Admin\AppData\Local\Programs\Python\Python39\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
PS C:\Users\Admin>
here is my code:
import requests
import json
params = {"q":"joke"}
url = "https://icanhazdadjoke.com"
question = requests.get(url, params)
if question.status_code == 200:
data = json.loads(str(question.text ))
print(data)
I've just take a look at the API docs of the website.
Since you didn't specify your Accept header, the website will return a HTML response as default
Solution:
headers = {
"Accept": "application/json"
}
url = "https://icanhazdadjoke.com"
r = requests.get(url, headers = headers)
I am getting an error:
raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Code is as follows:
import requests
import json
search_terms = input('Name of university: ')
search_year_start = input('Search start year (YYYY): ')
search_end_start = input('Search end year (YYYY): ')
parameters = {"affiliation": search_terms}
response = requests.get("https://api.ror.org/organizations", params=parameters)
ror_response = response.json()
name_variations = []
for organisation in ror_response['items']:
if organisation['chosen']==True:
name_variations.append(organisation['organization']['name'])
name_variations.extend(organisation['organization']['aliases'])
for relationship in organisation['organization']['relationships']:
if relationship['type'] == 'Child':
name_variations.append(relationship['label'])
seed_dois=[]
for variation in name_variations:
response = requests.get(url=f"https://api.crossref.org/works?filter=from-pub-date:{search_year_start},until-pub-date:{search_end_start},has-affiliation:true&query.affiliation={variation}&sample=1&select=DOI")
response_data =response.json()
seed_dois.append(response_data['message']['items'])
for doi in seed_dois:
response=requests.get(url=f"https://opencitations.net/index/coci/api/v1/citations/{doi}")
response_data = response.json()
citing_dois=[]
for doi in response_data:
cited_by=doi['citing']
citing_dois.append(cited_by)
The error is related to the seed_dois[] as far as I can tell and when I am trying to put this in the requests.get. If I hard code the URL everything works fine after that point, and everything works fine up until that point.
Why am I getting this error and what do I do to resolve it?
Edit
Full traceback
File "c:\Users\timc0\Desktop\py4e\patent_app\ror.py", line 43, in <module>
response_data = response.json()
File "C:\Users\timc0\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\models.py", line 910, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Users\timc0\AppData\Local\Programs\Python\Python39\lib\json\__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "C:\Users\timc0\AppData\Local\Programs\Python\Python39\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\timc0\AppData\Local\Programs\Python\Python39\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)````
I am using requests module to run a curl command, not sure why its prefixing ')]}\' in the front of the output, it makes r.json() fail as shown below.
When I paste the URL in a browser and execute, it downloads a .json file, and that file has same characters in the front, Am I missing some option? I need to process the output as json.
>>> r = requests.get('https://gerrit-review-server/a/changes/?q=status:open%20project:myproj/test/a_proj%20change:1510&o=CURRENT_REVISION', verify=False, auth=HTTPBasicAuth('user','pass'),
>>>
>>> r.text
')]}\'\n[{"id":"myproj%2Ftest%2Fa_proj~master~I15790ba05690e0a9984cb05bce06574645274966","project":"myproj/test/a_proj","branch":"master","hashtags":[],"change_id":"I15790ba05690e0a9984cb05bce06574645274966","subject":"Test","status":"NEW","created":"2021-01-27 19:38:57.000000000","updated":"2021-03-21 14:19:42.000000000","submit_type":"MERGE_IF_NECESSARY","mergeable":true,"insertions":1,"deletions":0,"total_comment_count":0,"unresolved_comment_count":0,"has_review_started":true,"_number":1510,"owner":{"_account_id":10008339},"current_revision":"fe3cc60cc66ad6f20c631ee818ccd91955c69d37",<..deleted..>,"description":"Rebase"}},"requirements":[]}]\n'
>>> r.json()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.6/site-packages/requests/models.py", line 900, in json
return complexjson.loads(self.text, **kwargs)
File "/usr/lib64/python3.6/json/__init__.py", line 354, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
>>>
As Carcigenicate says, it seems that the json is wrongly formatted. You can try the loads, and if it fails try to correct it:
r = requests.get(some_url)
try:
data = r.json()
except json.JSONDecodeError:
# cut everything in front of the first "\n"
raw_data = r.text.split("\n", maxsplit=1)[1]
# cut everything behind the last "\n"
raw_data = raw_data.rsplit("\n", maxsplit=1)[0]
# try to load again the json
# If it fails it will raise the exception again
data = json.loads(raw_data)
I'm new to python and APIs and I'm using this tutorial to learn with last-fm. I'm getting these errors:
Traceback (most recent call last):
File "/Users/vikram03/Desktop/Python_learning/fm.py", line 89, in <module>
total_pages = int(response.json()['artists']['#attr']['totalPages'])
File "/Applications/anaconda3/lib/python3.8/site-packages/requests/models.py", line 900, in json
return complexjson.loads(self.text, **kwargs)
File "/Applications/anaconda3/lib/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/Applications/anaconda3/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/Applications/anaconda3/lib/python3.8/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Here's the code in question:
#initializes reponses list
responses = []
page = 1
total_pages = 70
while page <= total_pages:
payload = {
'method': 'chart.gettopartists',
'limit': 500,
'page': page
}
#print
print("Requesting page {}/{}".format(page, total_pages))
#clear clear_output
clear_output(wait=True)
#make the API call
response = lastfm_get(payload)
#error check
if response.status_code != 200:
print(reponse.text)
break
#extract pagination info
page = int(response.json()['artists']['#attr']['page'])
total_pages = int(response.json()['artists']['#attr']['totalPages'])
# append
responses.append(response)
#if its not cached, sleep
if not getattr(response, 'from_cache', False):
time.sleep(0.25)
#increment
page += 1
This function: lastfm_get just does requests.get with the appropriate params and this url 'http://ws.audioscrobbler.com/2.0/'
From other posts I get the impression the program is getting non-JSON objects, but when I print
total_pages = int(response.json()['artists']['#attr']['totalPages']) I get the correct number.
edit:
this is the function:M
def lastfm_get(payload):
#define headers and url
headers = {'user-agent': USER_AGENT}
url = 'http://ws.audioscrobbler.com/2.0/'
#add API key and format
payload['api_key'] = API_KEY
payload['format'] = 'json'
response = requests.get(url, headers = headers, params = payload)
return response
edit2:
I added this code to check if I was getting a JSON object, and am getting a type error after about 200 pages:
#check object
try:
json_test = json.loads(lastfm_get(payload))
except TypeError:
print("not a json object")
Error:
Requesting page 256/7743
Requesting page 259/7743
Requesting page 275/7743
Requesting page 294/7743
Traceback (most recent call last):
File "/Users/vikram03/Desktop/Python_learning/fm.py", line 94, in <module>
page = int(response.json()['artists']['#attr']['page'])
File "/Applications/anaconda3/lib/python3.8/site-packages/requests/models.py", line 900, in json
return complexjson.loads(self.text, **kwargs)
File "/Applications/anaconda3/lib/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/Applications/anaconda3/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/Applications/anaconda3/lib/python3.8/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
I got a type error when I tried with except ValueError:
Traceback (most recent call last):
File "/Users/vikram03/Desktop/Python_learning/fm.py", line 84, in <module>
json_test = json.loads(lastfm_get(payload))
File "/Applications/anaconda3/lib/python3.8/json/__init__.py", line 341, in loads
raise TypeError(f'the JSON object must be str, bytes or bytearray, '
TypeError: the JSON object must be str, bytes or bytearray, not Response
EDIT3: I get this error when testing the output of lastfm_get:
TypeError: Object of type Response is not JSON serializable
Unsure how to resolve this.
I'm not sure why the page numbers are skipping either
I am creating a website using Django .
Below is my code:
views.py:
for i in range(0,len(userdata)):
json_hall = requests.get(
"https://www.onebookingsystem.com/API/Admin/booking_list_id.php?id=%s" % userdata[i]['bookid'])
r = json_hall.json()
hall_data = json.loads(json_hall.text)
id_data[i]['bookid'] = hall_data[0]['bookid']
When i run i am getting the error like this.
File "D:\KarthikWorkSpace\Projects\Python\obs_admin\Application\obs_app\views.py", line 1968, in bookings_owner
hall_data = json.loads(json_hall.text)
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Any help would be appreciated.
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
This usually means that the string you're trying to decode is empty, eg:
>>> import json
>>> json.loads('')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.6/json/__init__.py", line 354, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Please, make sure the request you are sending is correct.
You could print or inspect response.text to confirm that.
When using requests, the .json() method on a response object already gives you a Python object (decoded from the literal response)
You don't need to parse again the data with json.loads. By the way, here's a more Pythonic way to iterate over your data:
for (idx, user) in enumerate(userdata):
response = requests.get(
"https://www.onebookingsystem.com/API/Admin/booking_list_id.php?id=%s" % user['bookid'])
hall_data = response.json()
id_data[idx]['bookid'] = hall_data[0]['bookid']