I'm trying to run a search on a website for the word 'Adrian'. I already understand that first I have to send a request to the website, in the response I will have an XSRF-token that I need to use for the second request. As I understand, if I'm using session.get(), it keeps the cookies automatically for the second request, too.
I run the first request, get a 200 OK response, I print out the cookies, the token is there. I run the second request, I get back a 400 error but if I print out the header of the second request, the token is there. I don't know where it is going wrong.
Why do I get 400 for the second one?
import requests
session = requests.Session()
response = session.get('https://www.racebets.com/en/horse-racing/formguide')
print(response)
cookies = session.cookies.get_dict()
print(cookies)
XSRFtoken = cookies['XSRF-TOKEN']
print(XSRFtoken)
response = session.get('https://www.racebets.com/ajax/formguide/search?s=Adrian')
print(response)
print(response.request.headers)
I also tried to skip session and use requests.get() in the second request and add the token to the header by myself but the result is the same:
import requests
session = requests.Session()
response = session.get('https://www.racebets.com/en/horse-racing/formguide')
print(response)
cookies = session.cookies.get_dict()
print(cookies)
XSRFtoken = cookies['XSRF-TOKEN']
print(XSRFtoken)
headers = {'XSRF-TOKEN': XSRFtoken}
response = session.get('https://www.racebets.com/ajax/formguide/search?s=Adrian', headers=headers)
print(response)
print(response.request.headers)
As Paul said:
The API you're trying to make an HTTP GET request to cares about two request headers: cookie and x-xsrf-token. Log your browser's network traffic to see what they're composed of.
The header needs to be named x-xsrf-token. Try this:
token = session.cookies.get('XSRF-TOKEN')
headers = {'X-XSRF-TOKEN': token}
response = session.get('https://www.racebets.com/ajax/formguide/search?s=Adrian', headers=headers)
Related
I am trying to do a python request get command using python. I have:
endpoint = https://eventsapi.uatxyzreports.com/api/v1/Events/OR/12345
headers = {"Authorization":"abcxyz12345"}
response = requests.get(endpoint, headers = headers)
the response code is ("401")
but when I put in the same token I got a response code of 200. Where did it go wrong?
i'm trying to add more than 2 values in cookies python script.
Cookies values are supposed to be this way in the header
Cookie: AWSALB=Wpq++dLpIbyAkxZlCDLyZb5R4aJw0G5J9pYeW2jEQtCpDjpbt7aCTPi2Xo+gJd834M3t9BSv/GoEzgu3NiAtPmN3wXixRxiQnpduvfC5wb0wK14XO5crqMTss4Ou; PHPSESSID=ttsrnob9ir39bgqqf6qtjmtrq2;
it shows like this when i try to run the script
{'error': True, 'errorType': 'sessionExpired'}
This is my attempt
headers = {"Connection":"close"}
k = requests.get('site.com/api/test', headers=headers)
session_id = k.cookies['AWSALB'], k.cookies['PHPSESSID']
headers['Cookie'] = 'AWSALB={};','PHPSESSID={};'.format(session_id)
print(session_id)
Thanks.
You need to refresh you cookies periodically, probably by doing a login request first and then storing them from the response received.
After receiving new cookies, you can define them separately:
cookies = {
"AWSALB": k.cookies['AWSALB'],
"PHPSESSID": k.cookies['PHPSESSID'],
}
and send in your request:
k = requests.get(URL, headers=headers, cookies=cookies)
I'm simply testing out the capabilities of Zendesk API with the intention of creating a ticket-based Slack application but I cannot seem to get past the simple act of authenticating myself.
I am using a token for authentication and the requests library as instructed on Zendesk's help desk but I still seem to be getting the same
Status: 401 Problem with the request. Exiting.
error message.
I currently have two versions of a Python3 code that have the same output each time.
Passing the Authorization header
import requests
import base64
# Set the request parameters
url = 'https://domain.zendesk.com/api/v2/tickets.json'
user = 'email'
pwd = 'token'
# Create the basic auth header
auth = user + '/token:' + pwd
auth = auth.encode("utf-8")
auth = base64.b64encode(auth)
auth = auth.decode("utf-8")
headers = {'Authorization': 'Basic ' + auth}
# Do the HTTP get request
response = requests.get(url, headers=headers)
# Check for HTTP codes other than 200
if response.status_code != 200:
print('Status:', response.status_code, 'Problem with the request. Exiting.')
exit()
# Decode the JSON response into a dictionary and use the data
data = response.json()
print(data.text)
Using auth tuple in the requests.get function
import requests
# Set the request parameters
url = 'https://domain.zendesk.com/api/v2/tickets.json'
user = 'email' + '/token'
pwd = 'token'
# Do the HTTP get request
response = requests.get(url, auth=(user, pwd))
# Check for HTTP codes other than 200
if response.status_code != 200:
print('Status:', response.status_code, 'Problem with the request. Exiting.')
exit()
# Decode the JSON response into a dictionary and use the data
data = response.json()
print(data.text)
I have also tried cURL which seems to be working without any hiccups. My guess is that it has something to do with how requests library is coded, however, I could not seem to find the answer anywhere.
Any help would be much appreciated.
According to the docs you need to enable password and email as auth
https://developer.zendesk.com/documentation/ticketing/getting-started/zendesk-api-quick-start/#preparation
I am writing a python program which will send a post request with a password, if the password is correct, the server will return a special cookie "BDCLND".
I did this in Postman first. You can see the url, headers, the password I used and the response cookies in the snapshots below.
The response cookie didn't have the "BDCLND" cookie because the password 'ssss' was wrong. However, the server did send a 'BAIDUID' cookie back, now, if I sent another post request with the 'BAIDUID' cookie and the correct password 'v0vb', the "BDCLND" cookie would show up in the response. Like this:
Then I wrote the python program like this:
import requests
import string
import re
import sys
def main():
url = "https://pan.baidu.com/share/verify?surl=pK753kf&t=1508812575130&bdstoken=null&channel=chunlei&clienttype=0&web=1&app_id=250528&logid=MTUwODgxMjU3NTEzMTAuMzM2MTI4Njk5ODczMDUxNw=="
headers = {
"Content-Type":"application/x-www-form-urlencoded; charset=UTF-8",
"Referer":"https://pan.baidu.com/share/init?surl=pK753kf"
}
password={'pwd': 'v0vb'}
response = requests.post(url=url, data=password, headers=headers)
cookieJar = response.cookies
for cookie in cookieJar:
print(cookie.name)
response = requests.post(url=url, data=password, headers=headers, cookies=cookieJar)
cookieJar = response.cookies
for cookie in cookieJar:
print(cookie.name)
main()
When I run this, the first forloop did print up "BAIDUID", so that part is good, However, the second forloop printed nothing, it turned out the second cookiejar was just empty. I am not sure what I did wrong here. Please help.
Your second response has no cookies because you set the request cookies manually in the cookies parameter, so the server won't send a 'Set-Cookie' header.
Passing cookies across requests with the cookies parameter is not a good idea, use a Session object instead.
import requests
def main():
ses = requests.Session()
ses.headers['User-Agent'] = 'Mozilla/5'
url = "https://pan.baidu.com/share/verify?surl=pK753kf&t=1508812575130&bdstoken=null&channel=chunlei&clienttype=0&web=1&app_id=250528&logid=MTUwODgxMjU3NTEzMTAuMzM2MTI4Njk5ODczMDUxNw=="
ref = "https://pan.baidu.com/share/init?surl=pK753kf"
headers = {"Referer":ref}
password={'pwd': 'v0vb'}
response = ses.get(ref)
cookieJar = ses.cookies
for cookie in cookieJar:
print(cookie.name)
response = ses.post(url, data=password, headers=headers)
cookieJar = ses.cookies
for cookie in cookieJar:
print(cookie.name)
main()
I would like to access a web page from a python program.
I have to set up cookies to load the page.
I used the httplib2 library, but I didn't find how add my own cookie
resp_headers, content = h.request("http://www.theURL.com", "GET")
How can I create cookies with the right name and value, add it to the function and then load the page?
Thanks
From http://code.google.com/p/httplib2/wiki/Examples hope will help )
Cookies
When automating something, you often need to "login" to maintain some sort of session/state with the server. Sometimes this is achieved with form-based authentication and cookies. You post a form to the server, and it responds with a cookie in the incoming HTTP header. You need to pass this cookie back to the server in subsequent requests to maintain state or to keep a session alive.
Here is an example of how to deal with cookies when doing your HTTP Post.
First, lets import the modules we will use:
import urllib
import httplib2
Now, lets define the data we will need. In this case, we are doing a form post with 2 fields representing a username and a password.
url = 'http://www.example.com/login'
body = {'USERNAME': 'foo', 'PASSWORD': 'bar'}
headers = {'Content-type': 'application/x-www-form-urlencoded'}
Now we can send the HTTP request:
http = httplib2.Http()
response, content = http.request(url, 'POST', headers=headers, body=urllib.urlencode(body))
At this point, our "response" variable contains a dictionary of HTTP header fields that were returned by the server. If a cookie was returned, you would see a "set-cookie" field containing the cookie value. We want to take this value and put it into the outgoing HTTP header for our subsequent requests:
headers['Cookie'] = response['set-cookie']
Now we can send a request using this header and it will contain the cookie, so the server can recognize us.
So... here is the whole thing in a script. We login to a site and then make another request using the cookie we received:
#!/usr/bin/env python
import urllib
import httplib2
http = httplib2.Http()
url = 'http://www.example.com/login'
body = {'USERNAME': 'foo', 'PASSWORD': 'bar'}
headers = {'Content-type': 'application/x-www-form-urlencoded'}
response, content = http.request(url, 'POST', headers=headers, body=urllib.urlencode(body))
headers = {'Cookie': response['set-cookie']}
url = 'http://www.example.com/home'
response, content = http.request(url, 'GET', headers=headers)
http = httplib2.Http()
# get cookie_value here
headers = {'Cookie':cookie_value}
response, content = http.request("http://www.theURL.com", 'GET', headers=headers)
You may want to add another header parameters to specify another HTTP request parameters.
You can also use just urllib2 library
import urllib2
opener = urllib2.build_opener()
opener.addheaders.append(('Cookie', 'cookie1=value1;cookie2=value2'))
f = opener.open("http://www.example.com/")
the_page_html = f.read()