I'm currently writing a short script using the Request API and I need to access my e-mail box in order to retrieve some information.
So I wrote this code in order to authenticate:
#! /usr/bin/python3
import requests
payload = {'horde_user': '',
'horde_pass': '',
'login_post': '1'}
r = requests.post('http://webmail.technion.ac.il/login.php', data=payload)
print(r.status_code)
print(r.text)
Obviously I replaced horde_user by my username and horde_pass by my password in the payload dictionary.
But the output is the source code of the initial page, while I expected it to be the content of my mail box.
And the status code is 200 while it is supposed to be 302. Could somebody explain to me what I'm doing wrong.
Thanks a lot for helping!
Try requesting the page first and then pulling out the session ids from that first request to be used in the login attempt. I bet it is failing because you're sending old session ids and the website is rejecting them and sending you back to the login page with a new session id. The session ids would be Horde and horde_secret_key in the cookies.
Related
I'am trying to scrape this website: https://gb4.typewriter.at/. I'am struggling with logging in. The weird thing is: the form is being filled out but it is not sent, and therefore I don't get any cookies.
Here is what I tried:
import requests
url = "https://gb4.typewriter.at/"
s = requests.Session()
s.get(url)
payload = {
"LoginForm[username]": "some_username",
"LoginForm[pw]": "some_password",
}
r = self.session.post(
f"{url}/index.php?r=site/login",
data=payload,
)
print(r.content) # this prints the html with both forms filled
print(r.cookies) # gives me a PHPSESSID cookie, not the cookie I need, the site uses another one (some random chrachters) as authentication cookie.
I also tried to send a post request with Postman, which worked completely fine and logged me in.
I would really appreciate some help!
I'm trying to get data from
https://www.biman-airlines.com/bookings/flight_selection.aspx
For example, when I choose flight from Dhaka(DAC) to Sylhet(ZYL), it goes to
https://www.biman-airlines.com/bookings/flight_selection.aspx?TT=RT&SS=&RT=&FL=on&DC=DAC&AC=ZYL&AM=2018-01&AD=09&DC=&AC=&AM=&AD=&DC=&AC=&AM=&AD=&DC=&AC=&AM=&AD=&RM=2018-01&RD=10&PA=1&PT=&PC=&PI=&CC=&NS=&CD=&FS=B4B9631
and shows the flight information
but when I'm trying to perform such get request using python, it shows no info
here is my code:
import requests
print(requests.get('https://www.biman-airlines.com/bookings/flight_selection.aspx?TT=RT&SS=&RT=&FL=on&DC=DAC&AC=ZYL&AM=2018-01&AD=09&DC=&AC=&AM=&AD=&DC=&AC=&AM=&AD=&DC=&AC=&AM=&AD=&RM=2018-01&RD=10&PA=1&PT=&PC=&PI=&CC=&NS=&CD=&FS=').text)
What am I doing wrong?
thanks in advance for any help
but when I'm trying to perform such get request using python, it shows no info. What am I doing wrong?
The request result shows no info because there is no cookie data in the python HTTP request.
If you check the HTTP request in browser debug window, you can see there is cookie along with the request -- the cookie identifies who the client is and tells server "Hi, server, I'm a valid user":
With reasonable guess, in this biman-airlines.com case, the server would check the cookie and return result only if the cookie is valid.
Thus, you need to add your Cookie header in the python code:
# The cookie below is just for example, you would get your own cookie once visiting the website.
headers = {
'Cookie': 'chocolateChip=nbixfy44dvziejjdxd2wmzs3; BNI_bg_zapways=0000000000000000000000009301a8c000005000; ASPSESSIONIDSQDCSSDT=PFJPADACFOGBDMONPBHPMFJN'
}
print(requests.get('https://www.biman-airlines.com/bookings/flight_selection.aspx?TT=RT&SS=&RT=&FL=on&DC=DAC&AC=ZYL&AM=2018-01&AD=09&DC=&AC=&AM=&AD=&DC=&AC=&AM=&AD=&DC=&AC=&AM=&AD=&RM=2018-01&RD=10&PA=1&PT=&PC=&PI=&CC=&NS=&CD=&FS=B4B9631', headers=headers).text)
I have a script to post a form from a distant web page.
This webpage sends a cookie, and I like to get this and insert it in the header to post the form.
How could I do that ?
Thank you
If you want to get cookies from GET request and use it in POST, you can try to use session() as below:
session = requests.session()
response = session.get(URL) # send GET to get cookies
cookies = response.cookies.get_dict() # Grab cookies
post = session.post(URL, cookies=cookies) # Send POST with grabbed cookies
Thanks a lot,
When I go to the website from a web browser I have this :
Cookie: __qca=P0-1275531178-1484650046149; _ga=GA1.2.1669629019.1484650088; optimizelyEndUserId=oeu1486719631164r0.37280650070167387; optimizelySegments=%7B%224541596561%22%3A%22ff%22%2C%224547081444%22%3A%22none%22%2C%224502945149%22%3A%22direct%22%2C%224547981473%22%3A%22false%22%7D; optimizelyBuckets=%7B%7D; IRF_1322=%7Bvisits%3A1%2Cuser%3A%7Btime%3A1486719631639%2Cref%3A%22direct%22%2Cpv%3A1%2Ccap%3A%7B%7D%2Cv%3A%7B%7D%7D%2Cvisit%3A%7Btime%3A1486719631639%2Cref%3A%22direct%22%2Cpv%3A1%2Ccap%3A%7B%7D%2Cv%3A%7B%7D%7D%2Clp%3A%22http%3A%2F%2Flastingmemoriesbylisa.zenfolio.com%2F%22%2Cdebug%3A0%2Ca%3A1486719631639%7D; zf_5y_visitor=s7fL-1Wogsx-E1mBu-WkZbl-5gcAAAAATer6DQw9WNnr; zf_10y_tz=120; zf_pat=256887569$labellevuephotography$$396676290$492530544; zf_lsc=NPUPdGc1+bbM3gpqJ/i3nH4o...0
but when I use my script I online have :
zf_5y_visitor=s7fL-1Wogsx-E1mBu-WkZbl-5gcAAAAATer6DQw9WNnr; zf_10y_tz=120;
Is that possible to get the other parts ? At least the one starting with "zf"
I am trying to send a post request using the nice Requests library in Python. I am sending the payload, as shown in the code, however, the r.text print statement shows the html dump of the myaccount.nytimes.com page, which is not what I want. Any one knows what's happening?
payload = {
'userid': 'myemail',
'password': 'mypass'
}
s = requests.session()
r = s.post('https://myaccount.nytimes.com/auth/login/?URI=http://www.nytimes.com/2014/09/13/opinion/on-long-island-a-worthy-plan-for-coastal-flooding.html?partner=rss', data=payload)
print(r.text)
There are a couple of hidden <input> fields that you are omitting from your form:
is_continue
expires
token
token looks like it would be required, maybe the others aren't.
And possibly remember which is the "remember me" tickbox at the bottom of the form.
Starting with token try incrementally adding fields until it works.
Edit from comment: Token is provided to you when you first access the login page. Thus you need to do an initial GET to https://myaccount.nytimes.com/auth/login/, parse the HTML (BeautifulSoup?) to get the token (and other fields), then POST back to the server. Or you could use mechanize to handle this more easily.
My intention is to log into a site and then access a protected image from a python script. I have both legal and working access from a browser.
This is what I have now.
import requests
s = requests.Session()
s.get('*domain*')
r_login =s.post('*domain*/user.php', headers={'cmd': 'login', 'loginname': '***', 'password': '***' })
print (s.cookies)
print (r_login.status_code)
r_img = s.get('*domain*/*protectedimage*.jpg')
print (r_img.status_code)
print (r.cookies)
print (s.cookies['PHPSESSID'])
Output:
<<class 'requests.cookies.RequestsCookieJar'>[<Cookie PHPSESSID=664b0842085b847a04d415a22e013ad8 for *domain*/>]>
200
403
<<class 'requests.cookies.RequestsCookieJar'>[]>
664b0842085b847a04d415a22e013ad8
I am sure I can successfully log in, because I have once downloaded the html file after doing so, and it was in a form of being logged in. But my problem is that it seems to me that my PHPSESSID cookie does not pass so I get a 403 error back. But I clearly have it in my session. I have also tried adding the cookie manually to my "r_img" line, and it made no difference, I still get an empty CookieJar and a 403 error back. Would this be not possible with only the requests modul? Did I overlook something? Excuse me for being not quite familiar with HTTP requests.
I'm using Python 3.4 just for sake of clarity.
You are passing in your form data as HTTP headers. A POST login form should send form elements as the data parameter instead:
r_login = s.post('*domain*/user.php',
data={'cmd': 'login', 'loginname': '***', 'password': '***' })
Do inspect the returned body, not just the status code. Your POST request was accepted by the server (200 OK) but since no login information was posted, the body will most likely tell you something like "login incorrect, please try again".
The server most likely cleared the cookie again seeing as it was not a valid login session when you requested the image. The 403 response probably contains a Set-Cookie header for PHPSESSID with a date in the past to clear it.
Try doing it like this:
As per python-requests docs:
payload = {'cmd': 'login', 'loginname': '***', 'password': '***'}
url = '*domain*/user.php'
s.post(url, data=payload)