Im trying to login to a website using python requests. My code is as follows:
url = "https://program.uffiliates.com/en/Auth/Login"
payload = {
'uerName': '',
'pasword': ''}
try:
with requests.Session() as s:
r = s.post(url, data=payload)
print(r.text)
except requests.exceptions.RequestException as e:
print(e)
return
This doesnt work, it prints out the html-code of the login page and not what i should see when im logged in. I assume im using the wrong names of the formular. In the html code i find the following:
<form action="/en/Auth/LogInAction" method="post">
<input id="hiddenUrl" name="hiddenUrl" type="hidden" value="" />
<input id="SST_ID" name="SST_ID" type="hidden" value="2" />
<input id="serial" name="serial" type="hidden" value="" />
<input id="referer" name="referer" type="hidden" value="" />
This confuses me. Can someone tell me what names i have to use for the "payload" to put in my username and password.
And after logging in how can i navigate through the backend? Should i just use requests.get with the specific url since im already logged in, or should i somehow use requests to click on buttons/links to navigate?
Thanks alot!
If I understand your question correctly, you need to POST your username and password via requests and be logged in.
first of all, the url you are posting the data to should be the form's action. not the login url.
secondly, there are a few hidden inputs in the URL, which I suspect the server will look at to confirm that the request is coming from an HTML that it recognises. You will need the inputs in your requests too.
try
url = "https://program.uffiliates.com/en/Auth/LogInAction"
payload = {
'uerName': '',
'pasword': '',
'hiddenUrl':'',
'SST_ID' : '',
'serial':'',
'referer':''}
try:
with requests.Session() as s:
r = s.post(url, data=payload)
print(r.text)
except requests.exceptions.RequestException as e:
print(e)
return
Related
I'm trying to get the HTML content of a password protected site using Ghost.py.
The web server I have to access, has the following HTML code (I cut it just to the important parts):
URL: http://192.168.1.60/PAGE.htm
<html>
<head>
<script language="JavaScript">
function DoHash()
{
var psw = document.getElementById('psw_id');
var hpsw = document.getElementById('hpsw_id');
var nonce = hpsw.value;
hpsw.value = MD5(nonce.concat(psw.value));
psw.value = '';
return true;
}
</script>
</head>
<body>
<form action="PAGE.HTM" name="" method="post" onsubmit="DoHash();">
Access code <input id="psw_id" type="password" maxlength="15" size="20" name="q" value="">
<br>
<input type="submit" value="" name="q" class="w_bok">
<br>
<input id="hpsw_id" type="hidden" name="pA" value="180864D635AD2347">
</form>
</body>
</html>
The value of "#hpsw_id" changes every time you load the page.
On a normal browser, once you type the correct password and press enter or click the "submit" button, you land on the same page but now with the real contents.
URL: http://192.168.1.60/PAGE.htm
<html>
<head>
<!–– javascript is gone ––>
</head>
<body>
Welcome to PAGE.htm content
</body>
</html>
First I tried with mechanize but failed, as I need javascript. So now I´m trying to solve it using Ghost.py
My code so far:
import ghost
g = ghost.Ghost()
with g.start(wait_timeout=20) as session:
page, extra_resources = session.open("http://192.168.1.60/PAGE.htm")
if page.http_status == 200:
print("Good!")
session.evaluate("document.getElementById('psw_id').value='MySecretPassword';")
session.evaluate("document.getElementsByClassName('w_bok')[0].click();", expect_loading=True)
print session.content
This code is not loading the contents correctly, in the console I get:
Traceback (most recent call last): File "", line 8, in
File
"/usr/local/lib/python2.7/dist-packages/ghost/ghost.py", line 181, in
wrapper
timeout=kwargs.pop('timeout', None)) File "/usr/local/lib/python2.7/dist-packages/ghost/ghost.py", line 1196, in
wait_for_page_loaded
'Unable to load requested page', timeout) File "/usr/local/lib/python2.7/dist-packages/ghost/ghost.py", line 1174, in
wait_for
raise TimeoutError(timeout_message) ghost.ghost.TimeoutError: Unable to load requested page
Two questions...
1) How can I successfully login to the password protected site and get the real content of PAGE.htm?
2) Is this direction the best way to go? Or I'm missing something completely which will make things work more efficiently?
I'm using Ubuntu Mate.
This is not the answer I was looking for, just a work-around to make it work (in case someone else has a similar issue in the future).
To skip the javascript part (which was stopping me to use python's request), I decided to do the expected hash on python (and not on web) and send the hash as the normal web form would do.
So the Javascript basically concatenates the hidden hpsw_id value and the password, and makes a md5 from it.
The python now looks like this:
import requests
from hashlib import md5
from re import search
url = "http://192.168.1.60/PAGE.htm"
with requests.Session() as s:
# Get hpsw_id number from website
r = s.get(url)
hpsw_id = search('name="pA" value="([A-Z0-9]*)"', r.text)
hpsw_id = hpsw_id.group(1)
# Make hash of ID and password
m = md5()
m.update(hpsw_id + 'MySecretPassword')
pA = m.hexdigest()
# Post to website to login
r = s.post(url, data=[('q', ''), ('q', ''), ('pA', pA)])
print r.content
Note: the q, q and pA are the elements that the form (q=&q=&pA=f08b97e5e3f472fdde4280a9aa408aaa) is sending when I login normally using internet browser.
If someone however knows the answer of my original question I would be very appreciated if you post it here.
I am trying to login to a local webpage using username, password, and domain. I have for user:
<input name="j_username" class="formStyle" onchange="loadDomainListForADLogin(this)" id="username" type="text">
For Password:
<input name="j_password" class="formStyle" id="password" type="password">
And for Domain:
<select name="domain" class="formStyle" onchange="checkLocalAuth(this)">
<option>-- Choose --</option>
<option value="1"> MYDOMAIN.NET </option>
<option value="Local Authentication">Local Authentication</option>
</select>
I followed some instruction and tried to replicate similar technique by:
import requests, lxml
r = requests.get('http://MyWebPage:8008')
payload = {
"j_username": "user",
"j_password": "passw00rd",
"domain": "MYDOMAIN.NET"
}
requests.post(r, data=payload)
But something is not right, and code doesn't print the layout behind the login
Error I get:
Traceback (most recent call last):
---SNIP---
File "test.py", line 11, in <module>
requests.post(r, data=payload)
requests.exceptions.MissingSchema: Invalid URL '<Response [200]>': No schema supplied. Perhaps you meant http://<Response [200]>?
It might be the way you handle the url, as the request.get is triggered and the post is referencing a response obtained from the get value.
If the website needs a session to work properly you could wrap the request block as a session:
import requests, lxml
url = 'http://MyWebPage:8008'
s = requests.Session()
payload = {
"j_username": "user",
"j_password": "passw00rd",
"domain": "MYDOMAIN.NET"
}
s.post(url, data=payload)
I would like to submit a form on a webpage.
The page has however several forms :
<form method="post" action="https://mywebsite.com/pageA" id="order" class="order ajaxForm">
<input type="text" class="decimal" name="value" id="fieldA" value="0" />
</label>
</form>
<form method="post" action="https://mywebsite.com/pageB" id="previousorder" class="order ajaxForm">
<input type="text" class="decimal" name="value" id="fieldB" value="0" />
</label>
</form>
Is there an easy way to trigger a specific form using python & request ?
I'd go with some more advanced tools like mechanize or MechanicalSoup. The latter is actually based on requests internally (I assume you meant requests package by "request"). Both of these tools allow to "select a desired form" and then submit it specifying the required parameters.
For instance, submitting the order form with MechanicalSoup would look something like this:
import mechanicalsoup
browser = mechanicalsoup.StatefulBrowser()
browser.open("https://yourwebsite.com")
# Fill-in the order form
browser.select_form('#order')
browser["value"] = "100"
browser.submit_selected()
You have to look at the DevTools Network tab while posting a form.
Every form will have different request url and post parameters. Generally, what you will need to do with requests is something like that:
req = requests.post('https://mywebsite.com/pageB',
data = {'fieldB':'value_you_want_to_submit'})
But better first investigate it with DevTools.
Try something like this: (prob need to make some modifications but it will be close to what you want this example is for login form):
install lxml
import requests
from lxml import html
payload = {
"username": "<USER NAME>",
"password": "<PASSWORD>",
"csrfmiddlewaretoken": "<CSRF_TOKEN>"
}
sessionReq = requests.session()
login_url = "https://example.be/account/login.php"
result = sessionReq.get(login_url)
tree = html.fromstring(result.text)
authenticity_token = list(set(tree.xpath("//input[#name='csrfmiddlewaretoken']/#value")))[0]
result = sessionReq.post(login_url,data = payload, headers = dict(referer = login_url)
url = 'https://bitbucket.org/dashboard/overview'
I hope this helps you :)
I use Requests (2.2.1) to login a url http://tx3.netease.com/logging.php?action=login, but the login logic of this url is different from Django's csrf token mechanism, that is:
When you GET this url, there is two import values formhash and sts in html text, both of which will be used in a js function do_encrypt (in file http://tx3.netease.com/forumdata/cache/rsa/rsa_min.js). This is fine, I can easily grab them via re.
The key part of html text is:
<form method="post" name="login" id="loginform" class="s_clear" onsubmit="do_encrypt('ori_password','password');pwdclear = 1;" action="logging.php?action=login&loginsubmit=yes">
<input type="hidden" name="formhash" value="91e54489" />
<input type="hidden" name="referer" value="http://tx3.netease.com/" />
<input type="hidden" name="sts" id="sts" value="1409414053" />
<input type="hidden" name="password" id="password" />
...
<input type="password" id="ori_password" name="ori_password" onfocus="clearpwd()" onkeypress="detectCapsLock(event, this)" size="36" class="txt" tabindex="1" autocomplete="off" />
...
</form>
2. After entering email and original password ori_password, clicking submit button will call do_encrypt, which will use formhash, sts and ori_password to set the real password password for the post dict. Problem comes out -- There seems no way to get password string directly. (For contrast, you can directly get csrfmiddlewaretoken from session_client.cookies['csrftoken'] in Django case)
This is the code:
import requests
import json
import re
loginUrl = "http://tx3.netease.com/logging.php?action=login"
client = requests.session()
r = client.get(loginUrl)
r.encoding='gb18030'
stsPat = re.compile('<input type="hidden" name="sts" id="sts" value="(\d+?)" />')
formhashPat = re.compile('<input type="hidden" name="formhash" value="([\d\w]+?)" />')
sts = stsPat.search(r.text).groups()[0]
formhash = formhashPat.search(r.text).groups()[0]
loginData={
'username' : "smaller9#163.com",
'password' : ..., # Set by js function do_encrypt
'referer':'/',
'loginfield':'username',
'ori_password':'', # it's `111111`, but `do_encrypt` will set it to empty.
'loginsubmit':'true',
'sts':sts,
'formhash':formhash,
}
# r = client.post(url=loginUrl,data=loginData)
Assuming you have permission to do so, try logging in with selenium as i think that will be more inline with what you are ultimately trying to do.
from selenium import webdriver
USERNAME = "foo#bar.com"
PASSWORD = "superelite"
# create a driver
driver = webdriver.Firefox()
# get the homepage
driver.get("http://tx3.netease.com/logging.php?action=login")
un_elm = driver.find_element_by_id("username")
pw_elm = driver.find_element_by_id("ori_password")
submit = driver.find_element_by_css_selector("[name=loginsubmit]")
un_elm.send_keys(USERNAME)
pw_elm.send_keys(PASSWORD)
# click submit
submit.click()
# get the PHPSESSID cookie as that has your login data, if you want to use
# it elsewhere
# print driver.get_cookies():
# do something else ...
I have a critical issue. I would like integrate my application with another much older application. This service is simply a web form, probably behind a framework (I think ASP Classic maybe). I have an action URL, and I have the HTML code for replicating this service.
This is a piece of the old service (the HTML page):
<FORM method="POST"
url="https://host/path1/path2/AdapterHTTP?action_name=myactionWebAction&NEW_SESSION=true"
enctype="multipart/form-data">
<INPUT type="text" name="AAAWebView-FormAAA-field1" />
<INPUT type="hidden" name="AAAWebView-FormAAA-field2" value="" />
<INPUT type="submit" name="NAV__BUTTON__press__AAAWebView-FormAAA-enter" value="enter" />
</FORM>
My application should simulate form submission of this old application from code-behind with Python. For now, I didn't have so much luck.
For now I do this
import requests
payload = {'AAAWebView-FormAAA-field1': field1Value, \
'AAAWebView-FormAAA-field2': field2Value, \
'NAV__BUTTON__press__AAAWebView-FormAAA-enter': "enter"
}
url="https://host/path1/path2/AdapterHTTP?action_name=myactionWebAction&NEW_SESSION=true"
headers = {'content-type': 'multipart/form-data'}
r = requests.post(url, data=payload, headers=headers)
print r.status_code
I receive a 200 HTTP response code, but if I click on submit button on the HTML page, the action saves the values, but my code does not do the same. How do I fix this problem?
The owner of an old application sent me this Java exception log. Any ideas?
org.apache.commons.fileupload.FileUploadException: the request was rejected because no multipart boundary was found
Try passing an empty dictionary as files with requests.post. This will properly construct a request with multipart boundary I think.
r = requests.post(url, data=payload, headers=headers, files={})