I have problem with simple authorization and upload API script.
When authorized, client receives several cookies, including PHPSESSID cookie (in browser).
I use requests.post method with form data for authorization:
r = requests.post(url, headers = self.headers, data = formData)
self.cookies = requests.utils.dict_from_cookieja(r.cookies)
Headers are used for custom User-Agent only.
Authorization is 100% fine (there is a logout link on the page).
Later, i try to upload data using the authorized session cookies:
r = requests.post(url, files = files, data = formData, headers = self.headers, cookies = self.cookies)
But site rejects the request. If we compare the requests from script and google chrome (using Wireshark), there is no differences in request body.
Only difference is that 2 cookies sent by requests class, while google chrome sends 7.
Update: Double checked, first request receives 7 cookies. post method just ignore half...
My mistake in code was that i was assigning cookies from each next API request to the session cookies dictionary. On each request since logged in, cookies was 'reset' by upcoming response cookies, that's was the problem. As auth cookies are assigned only at login request, they were lost at the next request.
After each authorized request i use update(), not assigning.
self.cookies.update( requests.utils.dict_from_cookiejar(r.cookies) )
Solves my issue, upload works fine!
Related
I am trying to write an azure function to manage SSO between two services. The first one will host the link to the HTTP triggered Azure Function which then should respond with the formatted SAML Response which then gets sent to the consumption URL as a POST, but I can only make GET requests with the azure.functions.HttpResponse method needed to parse outputs for Azure Functions (unless I'm wrong).
Alternatively I've tried to set the cookie that I get as a response from sending the SAML Response with the python requests method, but the consumption URL doesn't seem to care that the cookie is there and just brings me back to the login page.
The SP in this situation is Absorb LMS and I can confirm that the SAML Response is formatted correctly because submitting it from an HTTP form works fine (which I've also tried returning as the body of the azure.functions.HttpResponse, but I just get HTTP errors which I can't make heads or tails of).
import requests
import azure.functions as func
headers = {
'Content-Type': 'application/x-www-form-urlencoded'
}
body = {"SAMLResponse": *b64 encoded saml response and assertion*}
response = requests.post(url=*acs url*, headers=headers, data=body)
headers = response.headers
headers['Location'] = *acs url*
return func.HttpResponse(headers=headers, status_code=302)
There is a parser of products from the STEPN marketplace. To receive a JSON response, you need to send a session with an authorized account in cookies.
# how the parser works
cookies = {'SESSIONIDD2': '7951767220820838781:1658220355588:1400231'} # cookies received from the developer tools in the browser
r = request.get('https://api.stepn.com/run/orderlist?order=2001&chain=103&refresh=true&page=0&type=600&gType=&quality=&level=0&bread=0', cookies=cookies)
# get a JSON response with the necessary data
But after some time, the session is logged out in cookies and you need to log in to the browser again and log in
I tried to log in via request.session (passed all the headers, cookies), but received an 'Incorrect username/password' in response
with requests.Session() as session:
r = session.get('https://m.stepn.com/')
r = session.get('https://api.stepn.com/run/login?account={email}&password={password}&type=3') # I also got the string for the request in the developer tools
# get {"code":201003,"msg":"Incorrect username/password"}
I've recently reversed Stepn web authentication(email and password encryption). Here is my solution in rust: https://github.com/Numenorean/stepn-password, you can remake it as a library using python(or C) ffi, and then call needed function just from your code, so after that you only need to send correct auth request
In my application, I have my API that is in localhost:8000/api/v0.1/save_with_post.
I've also made a Python Script in order to do a Post Request on such Api.
### My script
import requests
url = 'localhost:8000/api/v0.1/save_with_post'
myobj = {'key': 'value'}
x = requests.post(url, data = myobj)
Is it possible to view headers and body of the request in Chrome rather than debugging my application code?
You want Postman.
With Postman you can either generate a request to your service from Postman itself, or set up Postman as a proxy so you can see the requests that your API client is generating and the responses from the server.
If you want to view the response headers from the post request, have you tried:
>>> x.headers
Or you could just add headers yourself to your POST request as so:
h = {"Content-Type": "application/xml", ("etc")}
x = requests.post(url, data = myobj, headers = h)
well, I don't know if there is a way you could view the request in Chrome DevTools directly (I don't think there is) however, I know there are two alternatives for seeing the request body and response:
1 - use selenium with a chrome webdriver
this will allow you to run chrome automated by python. then you can open a test page and run javascript in it to do your post request,
see this for more info on how to do this:
1 https://selenium-python.readthedocs.io/getting-started.html
2 Getting the return value of Javascript code in Selenium
you will need to use Selenium-requests library to use requests library with selenium
https://pypi.org/project/selenium-requests/3
2 - use Wireshark
this program will allow you to see all the traffic that is going on your network card and therefore you will be able to monitor all the requests going back and forth. however, Wireshark will throw all the traffic that you network card send or receives it may be hard to see the specific request you want
I have a quick question regarding HTTP Basic Authentication after a redirect.
I am trying to login to a website which, for operational reasons, immediately redirects me to a central login site using an HTTP 302 response. In my testing, it appears that the Requests module does not send my credentials to the central login site after the redirect. As seen in the code snippet below, I am forced to extract the redirect URL from the response object and attempt the login again.
My question is simply this:
is there a way to force Requests to re-send login credentials after a redirect off-host?
For portability reasons, I would prefer not to use a .netrc file. Also, the provider of the website has made url_login static but has made no such claim about url_redirect.
Thanks for your time!
CODE SNIPPET
import requests
url_login = '<url_login>'
myauth = ('<username>', '<password')
login1 = requests.request('get', url_login, auth=myauth)
# this login fails; response object contains the login form information
url_redirect = login1.url
login2 = requests.request('get', url_redirect, auth=myauth)
# this login succeeds; response object contains a welcome message
UPDATE
Here is a more specific version of the general code above.
The first request() returns an HTTP 200 response and has the form information in its text field.
The second request() returns an HTTP 401 response with 'HTTP Basic: Access denied.' in its text field.
(Of course, the login succeeds when provided with valid credentials.)
Again, I am wondering whether I can achieve my desired login with only one call to requests.request().
import requests
url_login = 'http://cddis-basin.gsfc.nasa.gov/CDDIS_FileUpload/login'
myauth = ('<username>', '<password>')
with requests.session() as s:
login1 = s.request('get', url_login, auth=myauth)
url_earthdata = login1.url
login2 = s.request('get', url_earthdata, auth=myauth)
My solution to this would be use of "Session". Here is how you can implement Session.
import requests
s = requests.session()
url_login = "<loginUrl>"
payload = {
"username": "<user>",
"password": "<pass>"
}
req1 = s.post(url_login, data=payload)
# Now to make sure you do not get the "Access denied", use the same session variable for the request.
req2 = s.get(url_earthdata)
This should solve your problem.
This isn't possible with Requests, by design. The issue stems from a security vulnerability, where if an attacker modifies the redirect URL and the credentials are automatically sent to the redirect URL, then the credentials are compromised. So, credentials are stripped from redirect calls.
There's a thread about this on github:
https://github.com/psf/requests/issues/2949
I'm just studying the requests library(http://docs.python-requests.org/en/latest/),
and got a problem on how to fetch a page with cookies using requests.
for example:
url2= 'https://passport.baidu.com'
parsedCookies={'PTOKEN': '412f...', 'BDUSS': 'hnN2...', ...} #Sorry that the cookies value is replaced by ... for instance of privacy
req = requests.get(url2, cookies=parsedCookies)
text=req.text.encode('utf-8','ignore')
f=open('before.html','w')
f.write(text)
f.close()
req.close()
when I use the codes above to fetch the page, it just saves the login page to 'before.html' instead of logined page, it refers that actually I haven't logged in successfully.
But if I use URLlib2 to fetch the page, it works properly as expected.
parsedCookies="PTOKEN=412f...;BDUSS=hnN2...;..." #Different format but same content with the aboved cookies
req = urllib2.Request(url2)
req.add_header('Cookie', parsedCookies)
ret = urllib2.urlopen(req)
f=open('before_urllib2.html','w')
f.write(ret.read())
f.close()
ret.close()
When I use these codes, it saves the logined page in before_urllib2.html.
--
Are there any mistakes in my code?
Any reply would be grateful.
You can use Session object to get what you desire:
url2='http://passport.baidu.com'
session = requests.Session() # create a Session object
cookie = requests.utils.cookiejar_from_dict(parsedCookies)
session.cookies.update(cookie) # set the cookies of the Session object
req = session.get(url2, headers=headers,allow_redirects=True)
If you use the requests.get function, it doesn't send cookies for the redirected page. Instead, if you use the Session().get function, it will maintain and send cookies for all http requests, this is what the concept "session" exactly means.
Let me try to elaborate to you what happens here:
When I sent cookies to http://passport.baidu.com/center and set the parameter allow_redirects as false, the returned status code is 302 and one of the headers of the response is 'location': '/center?_t=1380462657' (This is a dynamic value generated by server, you can replace it with what you get from server):
url2= 'http://passport.baidu.com/center'
req = requests.get(url2, cookies=parsedCookies, allow_redirects=False)
print req.status_code # output 302
print req.headers
But when I set the parameter allow_redirects as True, it still doesn't redirect to the page (http://passport.baidu.com/center?_t=1380462657) and the server return the login page. The reason is that the requests.get doesn't send cookies for the redirected page, here is http://passport.baidu.com/center?_t=1380462657, so we can login successfully. That is why we need the Session object.
If I set url2 = http://passport.baidu.com/center?_t=1380462657, it will return the page you want. One solution is use the above code to get the dynamic location value and form a path to you account like http://passport.baidu.com/center?_t=1380462657 , then you can get the desired page.
url2= 'http://passport.baidu.com' + req.headers.get('location')
req = session.get(url2, cookies=parsedCookies, allow_redirects=True )
But this is cumbersome, so when dealing with cookies, Session object do excellent job for us!