For some reason python requests does not do rePOST after encountered redirect header
import requests
proxies = {'http': 'http://127.0.0.1:8888',}
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.87 Safari/537.36',
'content-type': 'application/x-www-form-urlencoded'}
r = requests.post(url, data, headers=headers, timeout=timeout, proxies=proxies, allow_redirects=True,)
html = r.text
So it means I can't login to any form that is behind redirect. How can I solve this issue? Thank you!
Related
I'm trying to use a python script to access on a website
But it tells me CSRF verification failed. Request aborted. The python-requests code is as followed:
import requests
payload = {
'inUserName': 'NAN',
'inUserPass': 'NAN'
}
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like
Gecko) Chrome/103.0.0.0 Safari/537.36',
'From': 'https://pruebita.gob.pe/accounts/login/',
}
with requests.Session() as s:
p = s.post('https://pruebita.gob.pe/accounts/login/', data=payload, headers=headers)
print (p.text)
r = s.get('A protected web page url')
print (r.text)
This question already has an answer here:
Reading value from HTML page - nseindia
(1 answer)
Closed 2 years ago.
import requests
from bs4 import BeautifulSoup
url = "https://www.nseindia.com/api/option-chain-indices?symbol=BANKNIFTY"
headers={'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) chrome/85.0.4183.102 Safari/537.36','Accept-Language': 'en-US,en;q=0.9,hi;q=0.8,es;q=0.7','Accept-Encoding': 'gzip, deflate, br'}
response.raise_for_status()
df = response.json()
This used to work but now this error comes up HTTPError: 401 Client Error: Unauthorized for url: https://www.nseindia.com/api/option-chain-indices?symbol=BANKNIFTY.
any idea what should i do to access the web site in python ?
Well in that case, we should visit the main page to obtain the Cookies and then contact the API:
import requests
params = {
'symbol': 'BANKNIFTY'
}
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:81.0) Gecko/20100101 Firefox/81.0'
}
def main(url):
with requests.Session() as req:
req.headers.update(headers)
r = req.get("https://www.nseindia.com")
r = req.get(url, params=params).json()
print(r)
main("https://www.nseindia.com/api/option-chain-indices")
import requests
x = requests.get('https://www1.nseindia.com/live_market/dynaContent/live_watch/equities_stock_watch.htm' )
print(x.status_code)
print(x.content)
Giving connection error. Please help how to correct it.
Try this:
import requests
url = "https://www1.nseindia.com/live_market/dynaContent/live_watch/equities_stock_watch.htm"
headers = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, '
'like Gecko) '
'Chrome/80.0.3987.149 Safari/537.36',
'accept-language': 'en,gu;q=0.9,hi;q=0.8', 'accept-encoding': 'gzip, deflate, br'}
session = requests.Session()
request = session.get(url, headers=headers, timeout=5)
cookies = dict(request.cookies)
response = session.get(url, headers=headers, timeout=5, cookies=cookies)
print(response.status_code)
print(response.content)
This code for the first time you try to access the website in your program, If your accessing the site multiple times then use this:
response = session.get(url, headers=headers, timeout=5, cookies=cookies) everytime you try to access again.
Tell me if this works
Try add user agent to header:
import requests
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.162 Safari/537.36'}
r = requests.get('https://www1.nseindia.com/live_market/dynaContent/live_watch/equities_stock_watch.htm', headers=headers)
print(r.content)
I'm pretty new in python requests, and I have made a simple program, just to login, in netflix.
Here's my code.
url = 'https://www.netflix.com/login'
headers = {
'User-Agent' : 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/80.0.3987.149 Safari/537.36'
}
r = requests.post(url, data={'userLoginId':'my-email', 'password': 'my-password'}, headers=headers)
print(r.status_code)
The output of status code is 200, so it's right
You could use a Session to check the results of your login.
s = requests.Session()
r = s.post(url, data={'userLoginId':'my-email', 'password': 'my-password'}, headers=headers)
print(r.status_code)
Then you could check the cookies for your session with:
s.cookies.getdict()
Another example in this question:
Using Python Requests: Sessions, Cookies, and POST
so ive been trying to figure out how to do the 'follow' thing using python codes on imvu.com, but it always returns the message "invalid arguments" error in $: failed reading: not a valid json value"
import requests
headers = {
"Origin": "https://secure.imvu.com/",
"Referer": "https://secure.imvu.com/next/av/Sammy165/",
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.109 Safari/537.36",
"X-IMVU-SAUCE": "" #removed sauce for account safety
}
url = "https://api.imvu.com/profile/profile-user-696969696/subscriptions"
data = {"id": "https://api.imvu.com/profile/profile-user-175389029"}
req = requests.post(url=url, headers=headers, data=data)
print(req.text)
Have you tried
requests.post(url=url, headers=headers, json=data)
?
You have to do json.dumps(data). See code below
import requests
import json
headers = {
"Origin": "https://secure.imvu.com/",
"Referer": "https://secure.imvu.com/next/av/Sammy165/",
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.109 Safari/537.36",
"X-IMVU-SAUCE": "" #removed sauce for account safety
}
url = "https://api.imvu.com/profile/profile-user-696969696/subscriptions"
data = {"id": "https://api.imvu.com/profile/profile-user-175389029"}
req = requests.post(url=url, headers=headers, data=json.dumps(data))
print(req.text)
Output:
{"status":"failure","error":"ERROR-GENERIC-001","message":"Permission Denied: You are not allowed to modify this subscription set."}