So, im trying to make a program that can click a button from multiple links, links that i will get from a list, but first i need to understand how can I do this with only one link. They all have the same HTML structure, but I dont know how to do this.
HTML:
<div class="_55wr">
<form method="post">
<input type="hidden" name="fb_dtsg" value="AQG7lSxYN2mb:AQFMMcWJcZtZ" autocomplete="off">
<input type="hidden" name="jazoest" value="22090" autocomplete="off">
<table class="btnBar">
<tbody>
<tr>
<td>
<button type="submit" value="Bloquear" class="_54k8 _52jg _56bs _26vk _56b_ _56bu" name="confirmed" data-sigil="touchable"><span class="_55sr">Bloquear</span></button>
</td>
<td>
<button type="submit" value="Cancelar" class="_54k8 _52jg _56bs _26vk _56b_ _56bt" name="canceled" data-sigil="touchable"><span class="_55sr">Cancelar</span></button>
</td>
</tr>
</tbody>
</table>
</form>
</div>
The idea is to click the first button ('<button type="submit" value="Bloquear"...').
Current code:
import requests
auth = ('email#email.com', 'pass')
payload = {}
url = 'https://www.example.com'
s = requests.Session()
res = s.get('https://www.example.com')
cookies = res.cookies
r = requests.post(url, cookies = cookies, auth = auth, verify = False, payload = payload)
I searched for similar questions, but every question was using some "id" ({'id':'value'}), which I don't have here. So, what value should i use in payload?
The requests library makes HTTP requests which means that it does not render the JS and it can not click buttons. Monitor your network behavior using google dev tools to know what data are sent to the server when you click a button, and then make a POST request sending the same data by the params keyword in the request. For example
data = {'button' : 'clicked1'}
r.requests.post('your_url.com', params = data)
For clicking buttons, I would personally use the selenium library which emulates the browser and provides its automation
I would use Selenium.
The code would be like so:
driver = webdriver.Firefox()
button = driver.find_element_by_xpath("//button[#value = \'Bloquear\']")
button.click()
Related
I want to Automate Fill Form using Playwright Python version. On Playwright NodeJS, I've been successfully to do the desired. Only when Porting the JS Script into Playwright Python API, it's kinda different.
The Web has Multiple Button (Submit and Random) with different Name. I can achieve both Button via get_by_role but I have no idea to get into Specific Submit Button:
#HTML
<form method="post" action="?query=MyAPI" enctype="multipart/form-data">
<table style="width: 100%">
<tbody><tr>
<td style="width: 146px">URL:</td>
<td><input name="url" type="text" style="width: 100%"></td></tr>
<tr><td style="width: 146px"> </td>
<td><br>
<input name="Random" type="random" value="Random" style="width: 97px">
<span>
<input name="Submit" type="submit" value="Submit" style="width: 157px">
</span>
</td></tr>
</tbody></table>
</form>
#JS Version
await page.getByRole('button', { name: 'Submit' }).click();
It seems the Python Version don't have { name: 'Submit'} feature, so I can only do ('button'). I'm not sure if I left something from the API, Already tried the following things but to not avail.
page.get_by_role('button').get_by_text('Submit').click()
page.get_by_role('button', {'name': 'Submit'}).click()
EDIT2
It's a Public UR, Here's The Submit Button XPath: /html/body/div[1]/div[2]/div[2]/article[1]/form/table/tbody/tr[2]/td[2]/span/input
EDIT1:
The Button is a part of Multipart Forms, it has no ID so I can't find any Alternative solution.
Try with:
await page.locator('//input[#value="Submit"]').click();
Or
await page.locator('//input[#type="submit"]').click();
Or
await page.locator('//input[#name="Submit1"]').click();
Then you are trying with Xpath, which should be enough if that is the only submit value/name/type of the page.
Basically with .locator you are locating the element and then with .click() you are making click on that element previously located
Fully working example
from playwright.sync_api import sync_playwright
import time
def run(playwright):
chrome = playwright.chromium
browser = chrome.launch(headless=False)
context = browser.new_context()
page = context.new_page()
page.goto("https://suip.biz/?act=iscloudflare")
page.locator("//input[#name='url']").fill("google.com")
page.locator("//input[#value='Submit']").click()
time.sleep(10)
browser.close()
with sync_playwright() as playwright:
run(playwright)
FIXED! updated with working code
I have been going about trying to make this auto login thing working for me.
Do note I'm still a Python novice at this point.
The following is the html code I found when I inspected the relevant form:
<form action="/cgi-bin/netlogin.pl" method="post" name="netlogin">
<tr>
<td><div align="right">Intranet userid:</div></td>
<td><input type="text" size="20" maxlength="50" name="uid" id="uid" class="formField" /></td>
</tr>
<tr>
<td><div align="right">Wachtwoord:</div></td>
<td><input type="password" size="20" maxlength="50" name="pwd29296" class="formField" autocomplete="off"/></td>
</tr>
<tr>
<td> </td>
<td><input type="submit" name="submit" value="Login" /></td>
</tr>
</form>
I start off with getting the portal html content to find the right "pwdXXXX" variable. As with every refresh, the ID for the password form changes. To my understanding this, and the regex works.
It does go wrong when trying to pass the password. Which makes me think I got it wrong with the form its keys? I have absolutely no clue. I also tried using the urllib2 approach instead of using mechanize. No result either.
Working code:
url = "https://netlogin.kuleuven.be/cgi-bin/wayf2.pl?inst=kuleuven&lang=nl&submit=Ga+verder+%2F+Continue"
br = mechanize.Browser()
br.set_handle_robots(False)
br.open(url)
br.select_form(name = "netlogin")
form = str(br.form)
uID = "uid"
dynamic_pwID = re.findall(r"(pwd\d+)", form) #pwID changes when page is refreshed
pwID = dynamic_pwID[0]
br[uID] = "xxxx"
br[pwID]= "xxxx"
res = br.submit()
A part of your problem may well be that since you close the socket that you read the page with urllib with your mechanize session will have a different ID and so require a new token.
You will need to keep a single connection open for the duration of the session. So I think that you will need to parse the contents of the reply to br.read() to find your value for pwID.
Comment From OP:
I left out the urllib part and it's working now. I used str(br.form) instead of br.read() though.
I'm trying to get to grips with web2py/python. I want to get the user to fill in a search form, the term they search for is sent to my python script which should send the query to the blekko API and output the results to them in a new HTML page. I've implemented the following code but instead of my normal index page appearing, I'm getting the html response directly from blekko with '%(query)' /html appearing in it's search bar. Really need some help with this!
HTML form on the default/index.html page
<body>
<div id="MainArea">
<p align="center">MY SEARCH ENGINE</p>
<form name="form1" method="get" action="">
<label for="SearchBar"></label>
<div align="center">
<input name="SearchBar" type="text" id="SearchBar" value="" size = "100px"><br />
<input name="submit" type="submit" value="Search">
</div>
</form>
<p align="center"> </p>
Python code on the default.py controller
import urllib2
def index():
import urllib2
address = "http://www.blekko.com/?q='%(query)'+/html&auth=<mykey>"
query = request.vars.query
response = urllib2.urlopen(address)
html=response.read()
return html
I think you are misunderstanding how string formatting works. You need to put the address and query together still:
address = "http://www.blekko.com/?q='%(query)s'+/html&auth=<mykey>" % dict(query=request.vars.query)
Add a hidden field to your form, call it "submitted". Then reformat your controller function as such:
import urllib2
def index():
if request.vars.submitted:
address = "http://www.blekko.com/?q='%(query)'+/html&auth=<mykey>"
query = request.vars.query
response = urllib2.urlopen(address)
html=response.read()
return html
else:
return dict()
This will show your index page unless the form was submitted and the page received the "submitted" form variable.
The /html doesn't do anything. Glad your question got answered. There is python client code for the blekko search api here: https://github.com/sampsyo/python-blekko
I'm looking to grab the form data that needs to be passed along to a specific website and submit it. Below is the html(form only) that I need to simulate. I've been working on this for a few hours, but can't seem to get anything to work. I want this to work in Google App Engine. Any help would be nice.
<form method="post" action="/member/index.bv">
<table cellspacing="0" cellpadding="0" border="0" width="100%">
<tr>
<td align="left">
<h3>member login</h3><input type="hidden" name="submit" value="login" /><br />
</td>
</tr>
<tr>
<td align="left" style="color: #8b6c46;">
email:<br />
<input type="text" name="email" style="width: 140px;" />
</td>
</tr>
<tr>
<td align="left" style="color: #8b6c46;">
password:<br />
<input type="password" name="password" style="width: 140px;" />
</td>
</t>
<tr>
<td>
<input type="image" class="formElementImageButton" src="/resources/default/images/btnLogin.gif" style="width: 46px; height: 17px;" />
</td>
</tr>
<tr>
<td align="left">
<div style="line-height: 1.5em;">
join<br />
forgot password?<input type="hidden" name="lastplace" value="%2F"><br />
having trouble logging on, click here for help
</div>
</td>
</tr>
</table>
</form>
currently I'm trying to use this code to access it, but it's not working. I'm pretty new to this, so maybe I'm just missing it.
import urllib2, urllib
url = 'http://blah.com/member/index.bv'
values = {'email' : 'someemail#gmail.com',
'password' : 'somepassword'}
data = urllib.urlencode(values)
req = urllib2.Request(url, data)
response = urllib2.urlopen(req)
the_page = response.read()
Is this login page for a 3rd party site? If so, there may be more to it than simply posting the form inputs.
For example, I just tried this with the login page on one of my own sites. A simple post request won't work in my case, and this may be the same with the login page you are accessing as well.
For starters the login form may have a hidden csrf token value that you have to send when posting your login request. This means you'd have to first get the login page and parse the resulting html for the csrf token value. The server may also require its session cookie in the login request.
I'm using the requests module to handle the get/post and beautifulsoup to parse the data.
import requests
import zlib
from BeautifulSoup import BeautifulSoup
# first get the login page
response = requests.get('https://www.site.com')
# if content is zipped, then you'll need to unzip it
html = zlib.decompress(response.read(), 16+zlib.MAX_WBITS)
# parse the html for the csrf token
soup = BeautifulSoup(html)
csrf_token = soup.find(name='input', id='csrf_token')['value']
# now, submit the login data, including csrf token and the original cookie data
response = requests.post('https://www.site.com/login',
{'csrf_token': csrf_token,
'username': 'username',
'password': 'ckrit'},
cookies=response.cookies)
login_result = zlib.decompress(response.read(), 16+zlib.MAX_WBITS)
print login_result
I cannot say if GAE will allow any of this or not, but at least it might be helpful in figuring out what you may require in your particular case. Also, as Carl points out, if a submit input is used to trigger the post you'd have to include it. In my particular example, this isn't required.
You're missing the hidden submit=login argument. Have you tried:
import urllib2, urllib
url = 'http://blah.com/member/index.bv'
values = {'submit':'login',
'email' : 'someemail#gmail.com',
'password' : 'somepassword'}
data = urllib.urlencode(values)
req = urllib2.Request(url, data)
response = urllib2.urlopen(req)
the_page = response.read()
I'm having issues submitting the result of a form submission (I can submit a form, but I can't submit the form on the page that follows the first).
I have:
browser = mechanize.Browser()
browser.set_handle_robots(False)
browser.open('https://www.example.com/login')
browser.select_form(nr=0)
browser.form['j_username'] = 'username'
browser.form['j_password'] = 'password'
req = browser.submit()
This works, as print req results in
`
<body onload="document.forms[0].submit()">
<noscript>
<p>
<strong>Note:</strong> Since your browser does not support JavaScript,
you must press the Continue button once to proceed.
</p>
</noscript>
<form action="https://www.example.com/Shibboleth.sso/SAML2/POST" method="post">
<div>
<input type="hidden" name="RelayState" value="cookie:95ca495c"/>
<input type="hidden" name="SAMLResponse" value="really long encoded value"/>
</div>
<noscript>
<div>
<input type="submit" value="Continue"/>
</div>
</noscript>
</form>
</body>
`
But I get errors when I try to use req.select_form(nr=0)
I assume this is probably from something along the lines of how mechanize returns objects from submit() and that I'm going about this the wrong way.
Any input or guidance would be appreciated :)
try again browser.select_form(nr=0) instead of req.select_form(nr=0). (after submitting or clicking a link or so, the new response is considered as an actual browser page - like in a browser :) )