We Keep Coding

Python request returning 403

I am trying to use requests to make an api call that this page is making https://www.betonline.ag/sportsbook/martial-arts/mma.
requests.post(
url='https://api.betonline.ag/offering/api/offering/sports/offering-by-league',
headers={'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.0 Safari/605.1.15'}
json={"Sport":"martial-arts","League":"mma","ScheduleText":None,"Period":0}
)
I have also tried including all the headers I see in the image but am still unable to get a 200 and get the response.
What am I missing?

Is there a way to find a browser's user-agent with cmd or python?

I am writing a code, where I have to use headless browser, but to access a specific website, I need to send user-agent as well. I am currently doing it by sending the following snippet of code(Python/Selenium/ChromeDriver).
opts = Options()
opts.add_argument("--headless")
opts.add_argument("--no-sandbox")
opts.add_argument("user-agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36")
But I wanted to make the user-agent genuine, instead of same for every browser/device where the code runs, thus I want to know the user-agent of browser on user's device.
So is there any way to find a browser's user-agent by using Python/Selenium code or command prompt?

httpagentparser extracts os, browser etc... information from http user agent string
so try this
import httpagentparser as agent
s = "user-agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
print(agent.detect(s))

Unable to read website intermittently with requests

I tried to read a website using Python requests.
However, it sometimes succeeds but sometimes fails.
Here is my code:
import requests
headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36'}
url = 'https://hn.house.ifeng.com/homedetail/25762.shtml'
res = requests.get(url, headers = headers, timeout = 10, verify=False)
What are the reasons and how to solve it?
Thank you very much.

Timeout during fetching https website using Python

I have trouble fetching zomato.com website using Python and requests library.
import requests
r = requests.get('https://www.zomato.com/san-antonio')
print r.status_code
I run this script and get no response. I'm guessing that the problem is https, but I tried it with some other https websites and it worked liked a charm, and 200 was printed to the console.
Am I missing something here?

You'll need to pretend you're coming from an actual browser:
import requests
r = requests.get('https://www.zomato.com/san-antonio', headers={"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36"})
print(r.status_code)
# returns: 200

Can't access this webpage

I am trying to write a python parser for this website 'http://www.topuniversities.com/university-rankings/world-university-rankings/2015#sorting=rank+region=+country=+faculty=+stars=false+search='
Every time I do the regular urlopen and print it, it says
'Access denied | www.topuniversities.com used CloudFlare to restrict access'.
After I tried this method
class MyOpener(FancyURLopener):
version = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36'
url = 'http://www.topuniversities.com/university-rankings/world-university rankings/2015#sorting=rank+region=+country=+faculty=+stars=false+search='
myopener = MyOpener()
page = myopener.open(url).read()
print page
But this prints out something other then what my chrome's inspect elements shows. I need to parse the names of the universities their rankings and the url that leads to their page.
What do I do? Please Help