403 error when trying to make requests to pypi api - python

My code:
import requests
x = requests.get('http://pypi.python.org/pypi/urllib3/json')
print(x)
Output:
<Response [403]>
Why am I getting this response even though it works fine in the browser?
I tried to print the json:
import requests
x = requests.get('http://pypi.python.org/pypi/urllib3/json')
print(x.json())
But it gave me an error:
Traceback (most recent call last):
File "C:\Users\HP\Desktop\temp.py", line 5, in <module>
print(x.json())
File "C:\Users\HP\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests\models.py", line 900, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.1776.0_x64__qbz5n2kfra8p0\lib\json\__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.1776.0_x64__qbz5n2kfra8p0\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.1776.0_x64__qbz5n2kfra8p0\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

It should be https and not http.
This will work.
import requests
x = requests.get('https://pypi.python.org/pypi/urllib3/json')
print(x)
json_str = x.json()
print(json_str)
<Response [200]>
{'info': {'author': 'Andrey Petrov', 'author_email': 'andrey.petrov#shazow.net', 'bugtrack_url': None,....}

Use https://pypi.python.org/pypi/urllib3/json instead, which works fine on my computer.
from requests import get
url = 'https://pypi.python.org/pypi/urllib3/json'
result = get(url).json()

Related

TD Ameritrade API 'Get Option Chain' giving <Response [200]> but I'm getting JSONDecodeError: Expecting value?

After generating the access_token (which works when I use it on TD Ameritrade's API website) I'm trying to get option chains for a stock. I can get it to work on TD Ameritrade's API website, and I get an 'OKAY' response when I run my code, but no JSON data attached, any idea why? My relevant code is below.
content = requests.get(url = https://api.tdameritrade.com/v1/marketdata/chains, params = params_dictionary, headers = access_token)
print(content)
print(repr(content.text))
data = content.json()
print(data)
but for my output I get
<Response [200]>
''
Traceback (most recent call last):
File "C:\Users\USER\Documents\GitHub\pythonfiles\TD Ameritrade API tests.py", line 98, in <module>
data = content.json()
File "C:\Users\USER\Anaconda3\lib\site-packages\requests\models.py", line 900, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Users\USER\Anaconda3\lib\json\__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "C:\Users\USER\Anaconda3\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\USER\Anaconda3\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
JSONDecodeError: Expecting value
It was a mistake on my part unshown in my question. I was accidentally using 'https://api.tdameritrade.com/v1/marketdata/https://api.tdameritrade.com/v1/marketdata/chains' for my URL. I'm unsure how this didn't break things, but this was my issue.

I am getting error json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

I get the correct result using the postman for a post request with the same params and headers. I get an error when I try to use the request module in python. You may see my postman screenshot for more info.
I see a json response in Postman which I try to get the same result from my code.
here is my code.
headers = {'Accept':'application/json, text/javascript, */*; q=0.01',
'Content-Type':'application/json; charset=UTF-8' ,
'Accept-Language':'en-US,en;q=0.9,fa-AF;q=0.8,fa;q=0.7,ru;q=0.6'}
payload = {"oppId":329656}
r = requests.post('https://www.grants.gov/grantsws/rest/opportunity/details', params=payload , headers=headers )
response = r.json()
print(response)
the error I get is :
Traceback (most recent call last):
File "requestDemo.py", line 120, in <module>
response = r.json()
File "E:\anaconda\envs\AHG_web\lib\site-packages\requests\models.py", line 898, in json
return complexjson.loads(self.text, **kwargs)
File "E:\anaconda\envs\AHG_web\lib\json\__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "E:\anaconda\envs\AHG_web\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "E:\anaconda\envs\AHG_web\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
if you wish to check the URL, here is the website, click on the first row of the table and check the network tab of the enter link description here dev tool.
I want to recieve the correct json response.

python-twitter stream raises JSONDecodeError

When calling for the next tweet of the stream with next(), the following error occurs. When I try calling other functions with the same twitter API object, such as GetFriends(), it works fine. Any help is much appreciated!
Here is the error:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/twitter/api.py", line 4897, in _ParseAndCheckTwitter
data = json.loads(json_data)
File "/usr/local/lib/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/usr/local/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/lib/python3.8/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
And here is the code:
import twitter
class Topic:
def __init__(self, topic):
self.api = self.get_twitter_api()
self.stream = self.api.GetStreamFilter(track=[topic])
self.count_tweets()
def get_twitter_api(self):
with open('/twitter_credentials.json') as f:
return twitter.Api(**eval(f.read()))
def count_tweets(self):
while next(self.stream):
print("tweet")
The oldest trick in the book worked: restarting the PC.

Can't get a valid response from a webpage containing json data

I've written a script in python to get response from a webpage. The data in that webpage are in json format. However, when I try like below I get an error. Can somebody give me any workaround as to how I can get a valid response?
Here is my failure attempt:
import requests
import json
URL = "https://www.sandicormls.com/agent/AgentSearch?officeSortOption=name&_=1516998894917&_keywordsAll=&officeLocation=&sortOption=rndsrt&_keywordsAgentName=&page=&typeIn=Realtor%2CBroker%2COwner%2COffice+Manager%2CAppraiser&searchMode=agent&officeName="
res = requests.get(URL,headers={'User-Agent':'Mozilla/5.0'})
print(res.json())
This is the traceback:
Traceback (most recent call last):
File "C:\Users\AppData\Local\Programs\Python\Python35-32\new_line_one.py", line 34, in <module>
print(res.json())
File "C:\Users\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\models.py", line 892, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Users\AppData\Local\Programs\Python\Python35-32\lib\json\__init__.py", line 319, in loads
return _default_decoder.decode(s)
File "C:\Users\AppData\Local\Programs\Python\Python35-32\lib\json\decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\AppData\Local\Programs\Python\Python35-32\lib\json\decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
I just gave it a try and confirm that your code is fine as it works on my environment. Though, I run it on a Linux (Ubuntu 16.04) virtual machine.
You could check what data you are getting back
print(res.headers[‘content-type’])
or examine the content such as
print(res.text)

Python: json.decoder.JSONDecodeError

I want to get a json response from server
Here is my code:
import requests
from requests_ntlm import HttpNtlmAuth
r = requests.get('http://helpbook.com/Home/UserInfo?empId=00458974',
auth=HttpNtlmAuth('domain\\login','password'))
print(r.json())
But each time I get the error:
Traceback (most recent call last):
File "C:\Users\user\Desktop\SM\PYTHON_SCRIPTS\Send json query\iso.py", line 6, in <module>
print(r.json())
File "C:\Program Files\Python36\lib\site-packages\requests\models.py", line 894, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Program Files\Python36\lib\json\__init__.py", line 354, in loads
return _default_decoder.decode(s)
File "C:\Program Files\Python36\lib\json\decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Program Files\Python36\lib\json\decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
How to fix it?
I do not have enough reputation to post a comment.
requests cannot decode the JSON response, because there is no JSON response. You should check the return code first:
import requests
from requests_ntlm import HttpNtlmAuth
r = requests.get('http://helpbook.com/Home/UserInfo?empId=00458974',
auth=HttpNtlmAuth('domain\login', 'password'))
if r.status_code == 200:
print(r.json())
else:
print("Request failed: {} - {}".format(r.status_code, r.reason))
And you would get the result:
Request failed: 404 - Not Found
With that information, you can debug you HTTP API and check why you receive 404 error in the first place.

Categories