I am trying to login to a page using Python requests (2.10.0).
Code:
payload = {
'user_login': 'amitg.ind#gmail.com',
'user_pass': 'xxx',
'rememberme': '1'
}
with requests.session() as sess:
resp = sess.post(URL_LOGIN, data=payload)
print resp.cookies
This should have returned multiple cookies. When I see in Chrome developer tools, I see the following:
However, when I print it on console, I see only the last cookie. Rest all are lost. The same login password works in browser. Due to this, subsequent request from same session fails.
RequestsCookieJar [Cookie wordpress_test_cookie=WP+Cookie+check for some.domain].
Why are the other cookies not preserved in session. All other subsequent requests doesn't recognize the login and I suspect its because of the missing cookies.
Could anyone help please?
Thanks #Padraic - I was using the 'id' attribute but 'name' was to be used. Thanks for that. wp-submit was not required. Thanks again.
Related
I'm relatively new to Python so excuse any errors or misconceptions I may have. I've done hours and hours of research and have hit a stopping point.
I'm using the Requests library to pull data from a website that requires a login. I was initially successful logging in through through a session.post,(payload)/session.get. I had a [200] response. Once I tried to view the JSON data that was beyond the login, I hit a [403] response. Long story short, I can make it work by logging in through a browser and inspecting the web elements to find the current session cookie and then defining the headers in requests to pass along that exact cookie with session.get
My questions is...is it possible to set/generate/find this cookie through python after logging in? After logging in and out a few times, I can see that some of the components of the cookie remain the same but others do not. The website I'm using is garmin connect.
Any and all help is appreciated.
If your issue is about login purposes, then you can use a session object. It stores the corresponding cookies so you can make requests, and it generally handles the cookies for you. Here is an example:
s = requests.Session()
# all cookies received will be stored in the session object
s.post('http://www...',data=payload)
s.get('http://www...')
Furthermore, with the requests library, you can get a cookie from a response, like this:
url = 'http://example.com/some/cookie/setting/url'
r = requests.get(url)
r.cookies
But you can also give cookie back to the server on subsequent requests, like this:
url = 'http://httpbin.org/cookies'
cookies = dict(cookies_are='working')
r = requests.get(url, cookies=cookies)
I hope this helps!
Reference: How to use cookies in Python Requests
I am trying to use python to login to a website (kicktipp.de). This is the following code I came up with by studying other stackoverflow question. Unfortunantelly for some reasons that I don't understand yet, it is not working. Can you help me? Thanks in advance!
import requests
payload = {
'kennung': 'name',
'passwort': 'pw'
}
with requests.Session() as s:
p = s.post('https://www.kicktipp.de/alternativlos/profil/login',
data=payload)
#print(p.text)
r = s.get('https://www.kicktipp.de/alternativlos/tippuebersicht')
print(r.text)
Before posting, you should get the page once, so that you get the session cookies, and re-transmit them.
Also you post to login, but you should post to loginaction
https://www.kicktipp.de/alternativlos/profil/loginaction
And finally, you are missing the charset post parameter.
As a word of advice, when you try to do that, open the console in "network" tab.
Check "preserve log" (on chrome), and then do log in as you would normally.
In the console, you'll see every request being made.
The first one is the POST that you are trying to do, copy it as much as possible.
See :
import requests
payload = {
'kennung': 'name',
'passwort': 'pw',
'_charset_' : 'UTF-8'
}
with requests.Session() as s:
s.get('https://www.kicktipp.de/alternativlos/profil/login') # get session cookie
p = s.post('https://www.kicktipp.de/alternativlos/profil/loginaction',
data=payload) # login
#print(p.text)
r = s.get('https://www.kicktipp.de/alternativlos/tippuebersicht')
print(r.text)
The url to post to when you check the form action in the browser inspector is /alternativlos/profil/loginaction.
...
p = s.post('https://www.kicktipp.de/alternativlos/profil/loginaction',
...
I'm trying to access a site (for which I have a login) through a .get(url) request. However, I tried passing the cookies that should authenticate my request but I keep getting a 401 error. I tried passing the cookies in the .get argument like so
requests.post('http://eventregistry.org/json/article?action=getArticles&articlesConceptLang=eng&articlesCount=25&articlesIncludeArticleConcepts=true&articlesIncludeArticleImage=true&articlesIncludeArticleSocialScore=true&articlesPage=1&articlesSortBy=date&ignoreKeywords=&keywords=soybean&resultType=articles', data = {"connect.sid': "long cookie found on chrome settings")
(Scroll over to see how cookies were used. Apologies for super long URL)
Am I approaching the cookie situation the wrong way? Should I login in with my username or password instead of passing the cookies? Or did I misinterpret my Chrome's cookie?
Thanks!
Solved:
import requests
payload = {
'email': '####gmail.com', #find the right name for the forms from HTML of site
'pass': '###'}
# Use 'with' to ensure the session context is closed after use.
with requests.Session() as s:
p = s.post('loginURL')
r = s.get('restrictedURL')
print(r) #etc
I just wanted to let you know that we've updated the package to access the Event Registry data so now you can actually make requests without using the cookies. Instead you can just append the parameter apiKey=XXXX in the url. You can find details on the documentation page:
http://eventregistry.org/documentation
I am trying to log in with a post request using the python requests module on a MediaWiki page:
import requests
s = requests.Session()
s.auth = ('....', '....')
url = '.....'
values = {'wpName' : '....',
'wpPassword' : '.....'}
req = s.post(url, values)
print(req.content)
I can't tell from the return value of the post request whether the login attempt was succesful. Is there something I can do to check this? Thanks.
Under normal circumstances i would advise you to go the mechanize way and make things way too easy for yourself but since you insist on requests, then let us use that.
YOu obviously have got the values right but i personally don't use the auth() function. So, try this instead.
import requests
url = 'https://example.com/wiki/index.php?title=Special:UserLogin'
values = {
'wpName': 'myc00lusername',
'wpPassword': 'Myl33tPassw0rd12'
}
session = requests.session()
r = session.post(url, data=values)
print r.cookies
This is what I used to solve this.
After getting a successful login, I read the texts from
response.text
and compared it to the text I got when submitting incorrect information.
The reason I did this is that validation is done on the server side and Requests will get a 200 OK response whether it was successful or not.
So I ended up adding this line.
logged_in = True if("Incorrect Email or password" in session.text) else False
Typically such an authentication mechanism is implemented using HTTP cookies. You might be able to check for the existence of a session cookie after you've authenticated successfully. You find the cookie in the HTTP response header or the sessions cookie attribute s.cookies.
I am trying to scrape some selling data using the StubHub API. An example of this data seen here:
https://sell.stubhub.com/sellapi/event/4236070/section/null/seatmapdata
You'll notice that if you try and visit that url without logging into stubhub.com, it won't work. You will need to login first.
Once I've signed in via my web browser, I open the URL which I want to scrape in a new tab, then use the following command to retrieve the scraped data:
r = requests.get('https://sell.stubhub.com/sellapi/event/4236070/section/null/seatmapdata')
However, once the browser session expires after ten minutes, I get this error:
<FormErrors>
<FormField>User Auth Check</FormField>
<ErrorMessage>
Either is not active or the session might have expired. Please login again.
</ErrorMessage>
I think that I need to implement the session ID via cookie to keep my authentication alive and well.
The Requests library documentation is pretty terrible for someone who has never done this sort of thing before, so I was hoping you folks might be able to help.
The example provided by Requests is:
s = requests.Session()
s.get('http://httpbin.org/cookies/set/sessioncookie/123456789')
r = s.get("http://httpbin.org/cookies")
print r.text
# '{"cookies": {"sessioncookie": "123456789"}}'
I honestly can't make heads or tails of that. How do I preserve cookies between POST requests?
I don't know how stubhub's api works, but generally it should look like this:
s = requests.Session()
data = {"login":"my_login", "password":"my_password"}
url = "http://example.net/login"
r = s.post(url, data=data)
Now your session contains cookies provided by login form. To access cookies of this session simply use
s.cookies
Any further actions like another requests will have this cookie