Cannot find element with Selenium Webdriver and Python - python

I'm using Selenium Webdriver with Python and hit a problem where I cannot find an element. I'm trying to select the h2 element (or its containing div) in the table:
<table class="TabStrip" cellspacing="0" cellpadding="0" border="0" style="padding-top: 0px;">
<tbody>
<tr>
<td style="white-space: nowrap;vertical-align:bottom;">
<td style="white-space: nowrap;vertical-align:bottom;">
<td style="white-space: nowrap;vertical-align:bottom;">
<td style="white-space: nowrap;vertical-align:bottom;">
<td style="white-space: nowrap;vertical-align:bottom;">
<td style="white-space: nowrap;vertical-align:bottom;">
<td style="white-space: nowrap;vertical-align:bottom;">
<td style="white-space: nowrap;vertical-align:bottom;">
<td style="white-space: nowrap;vertical-align:bottom;">
<td style="white-space: nowrap;vertical-align:bottom;">
<td style="white-space: nowrap;vertical-align:bottom;">
<td style="white-space: nowrap;vertical-align:bottom;">
<td style="white-space: nowrap;vertical-align:bottom;">
<td style="white-space: nowrap;vertical-align:bottom;">
<td style="white-space: nowrap;vertical-align:bottom;">
<div class="TabInactive"> </div>
<div class="TabNotSelected" onmouseout="previousSibling.className='TabInactive'" onmouseover="previousSibling.className='TabActiveRIA'" tabindex="10" onblur="blurTab(event);" onfocus="focusTab(event);" onkeydown="postBackOnEnter(event, 'b$_tab-flexiKB01P112', null, '1', 'TAB|b$_tab-flexiKB01P112');" onclick="postBackOnEnter(event, 'b$_tab-flexiKB01P112', null, '0', 'TAB|b$_tab-flexiKB01P112');" data-name="b$_tab-flexiKB01P112" style="white-space: nowrap" role="tab" aria -expanded="false">
<h2 class="PageTabTitleNotSelected">Workflow Approval</h2>
</div>
</td>
<td class="TabFiller" style="width:100%;"> </td>
</tr>
</tbody>
</table>
If I use:
row = wait.until(ec.presence_of_element_located(
(By.XPATH, '//table[#class="TabStrip"]/tbody/tr[1]')))
to select the row, everything works fine. But as soon as I try to add the cell:
cell = wait.until(ec.presence_of_element_located(
(By.XPATH, '//table[#class="TabStrip"]/tbody/tr[1]/td[15]')))
or anything within it, I get a timeout and element not found.
The content I'm after is plainly visible in the browser. Any ideas on where I might be going wrong?
EDIT
Everything works fine within the Selenium IDE using:
<tr>
<td>clickAndWait</td>
<td>//td[15]/div[2]</td>
<td></td>
</tr>

Related

Element changes from display: none to display:block selenium python

I am trying to fill a form online using selenium and at some point I have to fill a date. I can't use send_keys() since it is not allowed by the page. Instead, when I click on the date field, it pops up a datepicker window that prompts to select the year, and I can do this successfully.
After picking the year, the previous window is removed and a new one that prompts to select the month is displayed. This is done by setting the style from display: none to display: block and to the previous year window the style is set from display: block to display: none.
The problem is that even if the new window is_displayed() and is_enabled() methods return True, the elements of the second window, when using is_displayed() on them returns False, even if the is_enabled() method returns True.
I think that I should refresh the dom elements of my driver, but driver.refresh() puts me back in step 0, where I have to pick the year again.
This is my code:
# Code for selecting year (Works)
dateWindow = driver.find_element_by_xpath('/html/body/div[9]/div[3]/table')
rows = dateWindow.find_elements_by_tag_name("tr")
rows[1].find_element_by_xpath('//span[text()="%s"]' % str_year).click()
# Code for selecting month (Does not work)
dateWindow = driver.find_element_by_xpath('/html/body/div[9]/div[2]/table')
rows = dateWindow.find_elements_by_tag_name("tr")
rows[1].find_element_by_xpath('//span[text()="%s"]' % str_month).click()
In the last line, I get this error:
selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable
This is the html of the page before selecting the year:
<div class="datepicker-days" style="display: none;">
<table class=" table-condensed">
<thead>
<tr>
<th class="prev" style="visibility: visible;">«</th>
<th colspan="5" class="datepicker-switch">June 1993</th>
<th class="next" style="visibility: visible;">»</th>
</tr>
<tr>
<th class="dow">Su</th>
<th class="dow">Mo</th>
<th class="dow">Tu</th>
<th class="dow">We</th>
<th class="dow">Th</th>
<th class="dow">Fr</th>
<th class="dow">Sa</th>
</tr>
</thead>
<tbody>
<tr>
<td class="old day">30</td>
<td class="old day">31</td>
<td class="day">1</td>
<td class="day">2</td>
<td class="day">3</td>
<td class="day">4</td>
...
<td class="day">29</td>
<td class="day">30</td>
<td class="new day">1</td>
<td class="new day">2</td>
<td class="new day">3</td>
</tr>
<tr>
<td class="new day">4</td>
<td class="new day">5</td>
<td class="new day">6</td>
<td class="new day">7</td>
<td class="new day">8</td>
<td class="new day">9</td>
<td class="new day">10</td>
</tr>
</tbody>
<tfoot>
<tr>
<th colspan="7" class="today" style="display: none;">Today</th>
</tr>
<tr>
<th colspan="7" class="clear" style="display: none;">Clear</th>
</tr>
</tfoot>
</table>
</div>
<div class="datepicker-months" style="display: none;">
<table class="table-condensed">
<thead>
<tr>
<th class="prev" style="visibility: visible;">«</th>
<th colspan="5" class="datepicker-switch">1993</th>
<th class="next" style="visibility: visible;">»</th>
</tr>
</thead>
<tbody>
<tr>
<td colspan="7">
<span class="month">Jan</span>
<span class="month">Feb</span>
<span class="month">Mar</span>
<span class="month">Apr</span>
<span class="month">May</span>
<span class="month">Jun</span>
<span class="month">Jul</span>
<span class="month">Aug</span>
<span class="month">Sep</span>
<span class="month">Oct</span>
<span class="month">Nov</span>
<span class="month">Dec</span>
</td>
</tr>
</tbody>
<tfoot>
<tr>
<th colspan="7" class="today" style="display: none;">Today</th>
</tr>
<tr>
<th colspan="7" class="clear" style="display: none;">Clear</th>
</tr>
</tfoot>
</table>
</div>
<div class="datepicker-years" style="display: block;">
<table class="table-condensed">
<thead>
<tr>
<th class="prev" style="visibility: visible;">«</th>
<th colspan="5" class="datepicker-switch">1990-1999</th>
<th class="next" style="visibility: visible;">»</th>
</tr>
</thead>
<tbody>
<tr>
<td colspan="7">
<span class="year old">1989</span>
<span class="year">1990</span>
<span class="year">1991</span>
<span class="year">1992</span>
<span class="year">1993</span>
<span class="year active">1994</span>
<span class="year">1995</span>
<span class="year">1996</span>
<span class="year">1997</span>
<span class="year">1998</span>
<span class="year">1999</span>
<span class="year new">2000</span>
</td>
</tr>
</tbody>
<tfoot>
<tr>
<th colspan="7" class="today" style="display: none;">Today</th>
</tr>
<tr>
<th colspan="7" class="clear" style="display: none;">Clear</th>
</tr>
</tfoot>
</table>
</div>
This is the html of the page before selecting the month and after selecting the year:
<div class="datepicker-days" style="display: none;">
<table class=" table-condensed">
<thead>
<tr>
<th class="prev" style="visibility: visible;">«</th>
<th colspan="5" class="datepicker-switch">June 1993</th>
<th class="next" style="visibility: visible;">»</th>
</tr>
<tr>
<th class="dow">Su</th>
<th class="dow">Mo</th>
<th class="dow">Tu</th>
<th class="dow">We</th>
<th class="dow">Th</th>
<th class="dow">Fr</th>
<th class="dow">Sa</th>
</tr>
</thead>
<tbody>
<tr>
<td class="old day">30</td>
<td class="old day">31</td>
<td class="day">1</td>
<td class="day">2</td>
<td class="day">3</td>
<td class="day">4</td>
...
<td class="day">29</td>
<td class="day">30</td>
<td class="new day">1</td>
<td class="new day">2</td>
<td class="new day">3</td>
</tr>
<tr>
<td class="new day">4</td>
<td class="new day">5</td>
<td class="new day">6</td>
<td class="new day">7</td>
<td class="new day">8</td>
<td class="new day">9</td>
<td class="new day">10</td>
</tr>
</tbody>
<tfoot>
<tr>
<th colspan="7" class="today" style="display: none;">Today</th>
</tr>
<tr>
<th colspan="7" class="clear" style="display: none;">Clear</th>
</tr>
</tfoot>
</table>
</div>
<div class="datepicker-months" style="display: block;">
<table class="table-condensed">
<thead>
<tr>
<th class="prev" style="visibility: visible;">«</th>
<th colspan="5" class="datepicker-switch">1993</th>
<th class="next" style="visibility: visible;">»</th>
</tr>
</thead>
<tbody>
<tr>
<td colspan="7">
<span class="month">Jan</span>
<span class="month">Feb</span>
<span class="month">Mar</span>
<span class="month">Apr</span>
<span class="month">May</span>
<span class="month">Jun</span>
<span class="month">Jul</span>
<span class="month">Aug</span>
<span class="month">Sep</span>
<span class="month">Oct</span>
<span class="month">Nov</span>
<span class="month">Dec</span>
</td>
</tr>
</tbody>
<tfoot>
<tr>
<th colspan="7" class="today" style="display: none;">Today</th>
</tr>
<tr>
<th colspan="7" class="clear" style="display: none;">Clear</th>
</tr>
</tfoot>
</table>
</div>
<div class="datepicker-years" style="display: none;">
<table class="table-condensed">
<thead>
<tr>
<th class="prev" style="visibility: visible;">«</th>
<th colspan="5" class="datepicker-switch">1990-1999</th>
<th class="next" style="visibility: visible;">»</th>
</tr>
</thead>
<tbody>
<tr>
<td colspan="7">
<span class="year old">1989</span>
<span class="year">1990</span>
<span class="year">1991</span>
<span class="year">1992</span>
<span class="year">1993</span>
<span class="year active">1994</span>
<span class="year">1995</span>
<span class="year">1996</span>
<span class="year">1997</span>
<span class="year">1998</span>
<span class="year">1999</span>
<span class="year new">2000</span>
</td>
</tr>
</tbody>
<tfoot>
<tr>
<th colspan="7" class="today" style="display: none;">Today</th>
</tr>
<tr>
<th colspan="7" class="clear" style="display: none;">Clear</th>
</tr>
</tfoot>
</table>
</div>
Any ideas? Thanks in advance
The desired element is an dynamic element so while selecting the Month you have to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:
Using XPATH:
dateWindow = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "/html/body/div[9]/div[2]/table")))
rows = dateWindow.find_elements_by_tag_name("tr")
rows[1].find_element_by_xpath('//span[text()="%s"]' % str_month).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

How to click on date in zebra datepicker in html?

After taking suggestions from people here, I think I reached somewhere. I again got stuck. I have to scrape data from a website using datepicker. html says that the website is using Zebra datapicker. I have to click on date and then scrape the data. I am struggling to click on date. Can someone help me?
I have to click on 1/06/19. I am not able to figure out a way to do this. For reference: http://apsdps.ap.gov.in/des_rainfall.html This is the website.
link = "http://apsdps.ap.gov.in/des_rainfall.html"
webdriver.Chrome(executable_path=r"chromedriver.exe")
driver = webdriver.Chrome()
driver.get(link)
#driver.switch_to_frame(driver.find_element_by_xpath("//iframe"))
input_date = driver.find_element_by_name("date1")
input_date.click()
HTML:
<div class="Zebra_DatePicker" style="left: 222.375px; display: none; top: 0px;">
<table class="dp_header" style="width: 218px;">
<tbody>
<tr>
<td class="dp_previous">«</td>
<td class="dp_caption">June, 2019</td>
<td class="dp_next">»</td>
</tr>
</tbody>
</table>
<table class="dp_daypicker">
<tbody>
<tr>
<th>Mo</th>
<th>Tu</th>
<th>We</th>
<th>Th</th>
<th>Fr</th>
<th>Sa</th>
<th>Su</th>
</tr>
<tr>
<td class="dp_not_in_month">27</td>
<td class="dp_not_in_month">28</td>
<td class="dp_not_in_month">29</td>
<td class="dp_not_in_month">30</td>
<td class="dp_not_in_month">31</td>
<td class="dp_weekend dp_selected">1</td>
<td class="dp_weekend">2</td>
</tr>
<tr>
<td>3</td>
<td class="">4</td>
<td>5</td>
<td>6</td>
<td>7</td>
<td class="dp_weekend">8</td>
<td class="dp_weekend">9</td>
</tr>
<tr>
<td>10</td>
<td>11</td>
<td class="">12</td>
<td>13</td>
<td>14</td>
<td class="dp_weekend">15</td>
<td class="dp_weekend">16</td>
</tr>
<tr>
<td>17</td>
<td>18</td>
<td class="">19</td>
<td>20</td>
<td>21</td>
<td class="dp_weekend">22</td>
<td class="dp_weekend">23</td>
</tr>
<tr>
<td>24</td>
<td>25</td>
<td class="">26</td>
<td>27</td>
<td>28</td>
<td class="dp_weekend">29</td>
<td class="dp_weekend">30</td>
</tr>
<tr>
<td class="dp_not_in_month">1</td>
<td class="dp_not_in_month">2</td>
<td class="dp_not_in_month">3</td>
<td class="dp_not_in_month">4</td>
<td class="dp_not_in_month">5</td>
<td class="dp_not_in_month">6</td>
<td class="dp_not_in_month">7</td>
</tr>
</tbody>
</table>
<table class="dp_monthpicker" style="width: 218px; height: 190px; display: none;"></table>
<table class="dp_yearpicker" style="width: 218px; height: 190px; display: none;"></table>
<table class="dp_footer" style="width: 218px;">
<tbody>
<tr>
<td class="dp_today" style="width: 50%;">Today</td>
<td class="dp_clear" style="width: 50%; display: table-cell;">Clear date</td>
</tr>
</tbody>
</table>
</div>
To send a date to the datepicker you can use the following Locator Strategies:
Code Block:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("start-maximized")
driver = webdriver.Chrome(options=chrome_options)
driver.get("http://apsdps.ap.gov.in/des_rainfall.html")
driver.execute_script("arguments[0].removeAttribute('readonly')", WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "input#datepicker-example6"))))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input#datepicker-example6"))).send_keys("07/06/2019")
Browser Snapshot:

Extract 2 pieces of information from html in python

I need help figuring out how to extract Grab and the number following data-b. There are many <tr> in the complete unmodified webpage and I need to filter using the "Need" just before </a>. I've been trying to do this with beautiful soup, though it looks like lxml might work better. I can get either all of the <tr>s or only the < a>...< /a> lines that contain Need but not just the <tr>s that contain need in that <a> line.
<tr >
<td>3</td>
<td>Leave</td><td>Useless</td>
<td class="text-right"> <span class="float2" data-a="24608000.0" data-b="518" data-n="818">Garbage</span></td>
<td class="text-right"> <span class="Float" data-a="3019" data-b="0.0635664" data-n="283">Garbage2</span></td>
<td class="text-right">7.38%</td>
<td class="text-right " >Recently</td>
</tr>
<tr >
<td>4</td>
<td>Grab</td><td>Need</td>
<td class="text-right"> <span class="bloat2" data="22435000.0" data-b="512" data-n="74491.2">More junk</span></td>
<td class="text-right"> <span class="bloat" data-a="301.177" data-b="35.848" data-n="0.5848">More junk2</span></td>
<td class="text-right">Some more</td>
<td class="text-right " >Recently</td>
</tr>
Thanks for any help!
from bs4 import BeautifulSoup
data = '''<tr>
<td>3</td>
<td>Leave</td><td>Useless</td>
<td class="text-right"> <span class="float2" data-a="24608000.0" data-b="518" data-n="818">Garbage</span></td>
<td class="text-right"> <span class="Float" data-a="3019" data-b="0.0635664" data-n="283">Garbage2</span></td>
<td class="text-right">7.38%</td>
<td class="text-right " >Recently</td>
</tr>
<tr>
<td>4</td>
<td>Grab</td><td>Need</td>
<td class="text-right"> <span class="bloat2" data="22435000.0" data-b="512" data-n="74491.2">More junk</span></td>
<td class="text-right"> <span class="bloat" data-a="301.177" data-b="35.848" data-n="0.5848">More junk2</span></td>
<td class="text-right">Some more</td>
<td class="text-right " >Recently</td>
</tr>
'''
soup = BeautifulSoup(data)
print(soup.findAll('a',{"href":"/local" })[0].text)
for a in soup.findAll('span',{"class":["bloat","bloat2"]}):
print(a['data-b'])

Signing In and Scraping Webpages with Scrapy

Hoping some one is able to help me with this. I am using Scrapy to login and scrape data. This particular code works for one website. But I have another website that Scrapy cannot log on to because of the following issue:
There are no form elements on the page.
The username and password fields, and the submit button are not in a form element, instead they are in a table element, which makes it a bit confusing. How can I make Scrapy log into the webpage using the table/tr elements....and not the form element?
Any help would be appreciated!
class LoginSpider(BaseSpider):
name = 'project'
allowed_domains = ["domain.com"]
start_urls = ["theloginURL"]
#this function will look for the form element and login with the username and password
def parse(self, response):
return [FormRequest.from_response(response,
formdata={'user_name': ' username123', 'Password': ' psd123'},
formxpath='//*[#name="Form1"]',callback=self.after_login)]
This is the HTML code that is on the log in page, in case you guys need it:
<table height="260px" id="loginMainTable" width="100%" cellspacing="0" cellpadding="0">
<tbody><tr>
<td>
<table align="center" class="blueBorder" cellspacing="0" cellpadding="0">
<tbody><tr class="HeaderFooterHide">
<td id="companyTD" width="100%" colspan="3" style="position:relative;">
<span class="header1 header1pos">
Application and Network Access Portal
</span>
<table width="100%" cellpadding="0" cellspacing="0">
<tbody><tr>
<td width="32px">
<img src="/uniquesig6d7a33b352f4c09846f8a6563bae192b/uniquesig0/InternalSite/images/headertopl.gif" align="absmiddle">
</td>
<td style="background-image: url('/uniquesig6d7a33b352f4c09846f8a6563bae192b/uniquesig0/InternalSite/images/headertopm.gif'); background-repeat: repeat-x">
</td>
<td width="520px" style="background-image: url('/uniquesig6d7a33b352f4c09846f8a6563bae192b/uniquesig0/InternalSite/images/headertopr.gif');">
</td>
</tr>
</tbody></table>
</td>
</tr>
<tr class="HeaderFooterHide">
<td width="100%" colspan="3" style="position:relative;">
<span style="position:absolute;margin-left:20px;">
</span>
<table width="100%" cellpadding="0" cellspacing="0">
<tbody><tr>
<td width="30px">
<img src="/uniquesig6d7a33b352f4c09846f8a6563bae192b/uniquesig0/InternalSite/images/headerbottoml.gif" align="absmiddle">
</td>
<td style="background-image: url('/uniquesig6d7a33b352f4c09846f8a6563bae192b/uniquesig0/InternalSite/images/headerbottomm.gif'); background-repeat: repeat-x">
</td>
<td width="30px" style="background-image: url('/uniquesig6d7a33b352f4c09846f8a6563bae192b/uniquesig0/InternalSite/images/headerbottomr.gif');">
</td>
</tr>
</tbody></table>
</td>
</tr>
<tr>
<td class="contentleft">
</td>
<td align="center" class="internalTD">
<table width="100%" height="100%" cellspacing="0" cellpadding="0" align="center">
<form id="form1" name="form1" autocomplete="off" method="post" action="/uniquesig6d7a33b352f4c09846f8a6563bae192b/uniquesig0/InternalSite/Validate.asp" onsubmit="return(SubmitForm());"></form>
<tbody><tr>
<td valign="top" height="250px" align="center">
<table border="0" cellspacing="0" cellpadding="0" class="content">
<tbody><tr>
<td class="msgText">Log On</td>
</tr>
</tbody></table>
<table border="0" cellspacing="0" cellpadding="0" class="content">
<tbody><tr>
<td class="paramText">Username</td>
<td><input class="paramTextbox" type="text" id="user_name" name="user_name" maxlength="50" size="11"></td>
</tr>
<tr>
<td class="paramText">Password</td>
<td><input class="paramTextbox" type="password" id="password" name="password" maxlength="20" onkeypress="capsDetect(arguments[0]);" size="11"></td>
</tr>
<input type="hidden" id="repository" name="repository" value="ADLDS" size="11">
<tr height="0px">
<td colspan="2" id="capsLockTD" height="0px"> </td>
</tr>
<tr>
<td> </td>
<td class="EzBiz_Text1"><input name="chkUsername" type="CHECKBOX" onclick="saveUserName()" id="chkUsername">Remember my User ID</td>
</tr>
<tr>
<td colspan="2" align="right">
<input border="0" class="button" type="submit" id="submit_button" value="Log On">
</td>
<td></td>
</tr>
</tbody></table>
<div class="EzBiz_Loginbutton" style="float: left;">
<input border="0" class="EzBiz_Button" type="image" id="submit_button" src="/uniquesig6d7a33b352f4c09846f8a6563bae192b/uniquesig0/InternalSite/images/CustomUpdate/login_submit.jpg">
</div>
<div class="EzBiz_PasswordForget">
<img src="/uniquesig6d7a33b352f4c09846f8a6563bae192b/uniquesig0/InternalSite/images/CustomUpdate/arrow.gif">
Did you forget your password?
</div>
</td>
</tr>
<input type="hidden" name="site_name" id="site_name" value="ezbizportal">
<input type="hidden" name="secure" id="secure" value="1">
<input type="hidden" name="resource_id" id="resource_id" value="49203789E2E14C4A92CEC904C24909CE">
<input type="hidden" name="login_type" id="login_type" value="2">
<tr>
<td>
<table cellspacing="0" cellpadding="0" class="content" width="100%">
<tbody><tr>
<td id="openerExistsTD" class="notification">
For security purposes, when you finish working with this site do one of the following:<li>Click the Logoff button to log off from the site.</li><li>Close all browser windows (including applications that are open in other windows).</li>
</td>
</tr>
<!-- Windows XP Service Pack 2 / 2003 / Vista Message - Start -->
<tr>
<td class="notification">
This site is intended for authorized users only.<br>
If you experience access problems contact the site administrator.
</td>
</tr>
</tbody></table>
</td>
</tr>
</tbody></table>
</td>
<td class="contentright">
</td>
</tr>
<tr class="HeaderFooterHide">
<td width="100%" colspan="3" style="position:relative;">
<span class="bottomText bottomTextPos">
© 2010 Microsoft Corporation. All rights reserved. Terms and Conditions.
</span>
<table width="100%" cellpadding="0" cellspacing="0">
<tbody><tr>
<td width="47px">
<img src="/uniquesig6d7a33b352f4c09846f8a6563bae192b/uniquesig0/InternalSite/images/footerbgl.gif" align="absmiddle">
</td>
<td style="background-image: url('/uniquesig6d7a33b352f4c09846f8a6563bae192b/uniquesig0/InternalSite/images/footerbgm.gif'); background-repeat: repeat-x">
</td>
<td width="47px" style="background-image: url('/uniquesig6d7a33b352f4c09846f8a6563bae192b/uniquesig0/InternalSite/images/footerbgr.gif');">
</td>
</tr>
</tbody></table>
</td>
</tr>
</tbody></table>
</td>
</tr>
</tbody></table>
A generic login module would be more useful. Try scrapy loginform library:
Install
pip install -i https://pypi.binstar.org/pypi/simple loginform
Code
from loginform import fill_login_form
import requests
url = "https://example.com/login"
r = requests.get(url)
result=fill_login_form(url, text, 'username', 'password')
About
loginform is a library for filling HTML login forms given the login
url, username & password. Which form and fields to fill are inferred
automatically.

Python 2.7.8 script to login to my netgear smart switch and do some configurations

I need help on writing a script to log my Netgear GS108Tv2 smart switch's web interface and to do some configurations on mac filtering on my school's window 7 pc. I tried to use the mechanize code and some examples i found on the net, but the web interface's code format seems a little different from the example in the mechanize folder i downloaded.
Below is the code i came up with after using the example. For now i would be very glad if i am able to just log in to the web interface as i believe that is pretty much all i can do. The rest might prove a little tough but i can work on them after i manage to log in.
I am very new to code writing and totally fresh to python. Hope i can get all the help i could. Thank you very much.
import mechanize
browser = mechanize.Browser()
browser.open("http://172.16.164.23/")
browser.select_form(nr = 0)
browser.form["pwd"] = "password"
browser.submit()
browser.open("http://172.16.164.23/base/web_main.html")
well that is pretty much all could come up with, there are no errors but it does not seem to work.
Below is the web interface's code.
<html><head>
<link rel="stylesheet" href="/base/style.css" type="text/css">
<meta http-equiv="Pragma" content="no-cache">
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> <!-- Style Sheet link and Meta data -->
<title>NETGEAR GS108T</title> <!-- Netgear Page Title -->
<script src="/base/js/tabs_Layer2.js" type="text/javascript"></script>
<script src="/base/js/rollover.js" type="text/javascript"></script>
<script src="/base/js/browser.js" type="text/javascript"></script>
<script language="javascript">
var a1, a2, a3, a4, a5, a6, a7, a8;
a1 = new Image(130,29);
a1.src = "/base/images/tab_Login_off.gif";
a2 = new Image(130,29);
a2.src = "/base/images/tab_Login_on.gif";
a3 = new Image(130,29);
a3.src = "/base/images/tab_Login_ro.gif";
a4 = new Image(130,29);
a4.src = "/base/images/tab_Help_off.gif";
a5 = new Image(130,29);
a5.src = "/base/images/tab_Help_on.gif";
a6 = new Image(130,29);
a6.src = "/base/images/tab_Help_ro.gif";
a7 = new Image(130,29);
a7.src = "/base/images/onlinehelp_off.gif";
a8 = new Image(130,29);
a8.src = "/base/images/onlinehelp_on.gif";
</script>
<script type="text/javascript" language="JavaScript">
//******************************************************************************
//* Purpose: Display an input error message.
//* Return: none
//******************************************************************************
function DisplayErrorMsg()
{
alert(document.forms[0].err_msg.value);
}
function CheckError()
{
if (document.forms[0].err_flag.value == 1)
{
DisplayErrorMsg();
}
document.forms[0].pwd.focus();
}
</script>
<script language="javascript">
<!--
if (top != self)
{
top.location.href = "/base/web_main.html";
}
if (self.opener != undefined)
{
self.close();
self.opener.location.reload();
}
-->
</script>
</head>
<body onload="CheckError()">
<form method="POST" action="/base/main_login.html">
<table class="tableStyle" height="100%">
<tbody><tr class="topAlign">
<td class="leftEdge"> </td>
<td valign="top">
<table class="tableStyle">
<tbody><tr>
<td class="leftInside"><img src="/base/images/clear.gif" width="8"></td>
<td colspan="2">
<table class="tableStyle">
<tbody><tr>
<td class="logoNetGear space50Percent topAlign"><img src="/base/images/clear.gif" width="149" height="62"></td>
<td class="gs108tImage spacer50Percent topAlign rightHAlign"><img src="/base/images/clear.gif" width="190" height="60"></td> <!-- Used to get the top right logo image with help URL -->
</tr>
<tr>
<td colspan="2" class="bottomAlign">
<table class="tableStyle">
<tbody><tr>
<td class="spacer20Percent bottomAlign"><table border="0" cellpadding="0" cellspacing="0">
<tbody><tr id="tabss" class="spacer100Percent">
<td class="navTopCell"><img src="/base/images/tab_Login_on.gif" border="0"></td>
<td class="navTopCell"><img src="/base/images/tab_Help_off.gif" border="0"></td>
</tr>
</tbody></table>
</td>
<td class="loginActionCell spacer80Percent rightHAlign padding5LeftRight"></td>
</tr>
</tbody></table>
</td>
</tr>
</tbody></table>
</td>
<td class="rightHAlign"><img src="/base/images/clear.gif" width="2"></td>
</tr>
<tr class="background-blue">
<td><img src="/base/images/clear.gif" width="8"></td>
<td colspan="2" class="padding7Top spacer100Percent">
<div id="primaryNav">
<table border="0" cellpadding="0" cellspacing="0">
<tbody><tr>
<td id="cloneTd" class="navCell"> </td>
</tr>
<tr id="subTubs"></tr>
</tbody></table>
</div>
</td>
<td></td>
</tr>
</tbody></table>
</td>
<td class="rightEdge"> </td>
</tr>
<tr class="topAlign">
<td valign="top" class="leftBodyNotch topAlign"> </td>
<td>
<table class="tableStyle">
<tbody><tr class="topAlign">
<td class="leftNextBodyNotch"><img src="/base/images/clear.gif" width="11" height="16"></td>
<td class="middleBodyNotch spacer100Percent"> </td>
<td class="rightNextBodyNotch"><img src="/base/images/clear.gif" width="11"></td>
</tr>
</tbody></table>
</td>
<td class="rightBodyNotch"> </td>
</tr>
<tr height="100%">
<td rowspan="2" class="leftEdge"> </td>
<td valign="top">
<table class="tableStyle" height="100%">
<tbody><tr>
<td class="leftInside"><img src="/base/images/clear.gif" width="8"></td>
<td class="spacer100Percent loginTable topAlign">
<table class="loginBox">
<tbody><tr>
<td colspan="3">
<table class="tableStyle">
<tbody><tr>
<td colspan="2" class="subSectionTabTopLeft spacer80Percent font12BoldBlue">Login</td>
<td class="subSectionTabTopRight spacer20Percent"><img src="/base/images/help_icon.gif" width="12" height="12" title="Click for help"></td></tr><tr><td colspan="3" class="subSectionTabTopShadow"> </td>
</tr>
</tbody></table>
</td>
</tr>
<tr>
<td class="subSectionBodyDot"> </td>
<td class="paddingsubSectionBodyNone">
<table class="tableStyle">
<tbody><tr>
<td class="font10Bold padding4Top">Password</td>
<td class="padding4Top"><input class="input" type="PASSWORD" name="pwd" maxlength="20" value=""></td>
</tr>
<tr>
<td colspan="2" class="padding5TopBottom10Right"><input type="IMAGE" name="login" src="/base/images/login_on.gif"></td>
</tr>
</tbody></table>
</td>
<td class="subSectionBodyDotRight"> </td>
</tr>
<tr>
<td colspan="3" class="subSectionBottom"> </td>
</tr>
</tbody></table>
</td>
</tr>
</tbody></table>
</td>
<td rowspan="2" class="rightEdge"> </td>
</tr>
<tr>
<td>
<table class="tableStyle">
<tbody><tr>
<td colspan="3" class="topBottomDivider" id="cloneTd"><img src="/base/images/clear.gif" height="3"></td>
</tr>
<tr>
<td colspan="3" class="footerBody">
<table class="tableStyle rightHAlign" align="right">
<tbody><tr>
<td id="ButtonsDiv"></td>
</tr>
</tbody></table>
</td>
</tr>
</tbody></table>
</td>
</tr>
<tr>
<td class="leftEdgeFooter"><img src="/base/images/clear.gif" width="11" height="9"></td>
<td>
<table class="tableStyle">
<tbody><tr>
<td class="leftBottomDivider"><img src="/base/images/clear.gif" width="11" height="9"></td>
<td class="middleBottomDivider spacer100Percent"><img src="/base/images/clear.gif" height="9"></td>
<td class="rightBottomDivider spacer1Percent"><img src="/base/images/clear.gif" height="9"></td>
</tr>
</tbody></table>
</td>
<td class="rightEdgeFooter"><img src="/base/images/clear.gif" width="11" height="9"></td>
</tr>
<tr>
<td class="leftCopyrightFooter"><img src="/base/images/clear.gif" width="11" height="9"></td>
<td class="middleCopyrightDivider">
<table class="blue10 tableStyle">
<tbody><tr class="topAlign">
<td>Copyright © 1996-2013 NETGEAR ®</td>
</tr>
</tbody></table>
</td>
<td class="rightCopyrightFooter"><img src="/base/images/clear.gif" width="11" height="9"></td>
</tr>
</tbody></table>
<input type="hidden" name="err_flag" size="16" maxlength="15" value="0">
<input type="hidden" name="err_msg" size="128" maxlength="512" value="">
</form>
</body></html>

Categories