I am trying to scrape information from a website but am having trouble navigating it using Selenium. The site uses ng-click to update a table so I must activate different tabs on the page to get the information I want. This is the html that generates the tabs:
<ul class="tabs swiper-wrapper" ng-class="{'swiper-wrapper' : swiperActive }">
<li ng-repeat="category in Report.Winners track by $index" ng-click="updateCategory(category.key)" ng-class="{'active' : category.key == activeCategory, 'swiper-slide' : swiperActive }" class="ng-scope active">
<p class="category text-small ng-binding">Category 1</p>
<p class="winner">
</p>
</li><li ng-repeat="category in Report.Winners track by $index" ng-click="updateCategory(category.key)" ng-class="{'active' : category.key == activeCategory, 'swiper-slide' : swiperActive }" class="ng-scope">
<p class="category text-small ng-binding">Category 2</p>
<p class="winner">
</p>
</li><li ng-repeat="category in Report.Winners track by $index" ng-click="updateCategory(category.key)" ng-class="{'active' : category.key == activeCategory, 'swiper-slide' : swiperActive }" class="ng-scope">
<p class="category text-small ng-binding">Category 3</p>
<p class="winner">
</p>
</li>
</ul>
I have figured out how to scrape the information from "Category 1" since it loads by default. How do I navigate to "Category 2" and "Category 3" so I can scrape those as well? Thanks!
Update:
I ended up using this to find the links for each category:
available_categories_links = browser.find_elements_by_css_selector("ul > [ng-click*=updateCategory]")
And then I loop through them like this:
for x in range(len(available_categories_links)):
available_categories_links[x].click()
Doing it this way doesn't let me access different tabs by name like I had originally hoped to do, and it's probably not the most efficient or very robust, but it gets the job done in my particular case.
I assume your tabs load on runtime.
So, to activate the tab, you need to use FindElement(By) to locate the tab and click it before you could access the information from that tab.
According to your code, no Id found to use FindElement(By.Id). So I suggest you use either FindElement(By.CSSSelector) or FindElement(By.Xpath) which you could copy the Locator string from browser's development tool, i.e. in Google Chrome right click==> inspect==>Copy==>Copy Selector or Copy Xpath.
After you got the Tab Element, i.e.
IWebElement tab = driver.FindElement(By.CSSSelector);
tab.Click();//Tab activated
//...Do your thing afterwards.
Hope this helps.
Related
I am trying to find a clickable (I think - see code below) h2 element.
Here is a snippet of the html:
<div class="center-block flex-y v-center h-fill">
<div class="flex-y v-center h-start">
<h2 class="section-name ng-binding" ng-attr-tabindex="{{ appbar.service.disableProductSectionClick ? -1 : 0 }}" ng-click="appbar.clickProductSection()" ng-keydown="appbar.clickProductSection($event)" ng-class="{ 'inactive': appbar.service.disableProductSectionClick || !appbar.homeState }" tabindex="0">Password Safe</h2>
<app-bar-breadcrumbs class="ng-isolate-scope">
<ul class="flex-x h-start v-center">
<!-- ngRepeat: link in crumbs.service.list track by $index -->.
<li ng-repeat="link in crumbs.service.list track by $index" ng-click="crumbs.navTo($index)" ng-keydown="crumbs.navTo($index, $event)" data-name="Requests" data-collapse="false" ng-attr-tabindex="{{ (crumbs.service.list.length === $index + 1 || link.method == 'stateless') ? '-1' : '0' }}" ng-hide="crumbs.service.hidden.indexOf($index) > -1" ng-class="{ 'stateless': link.method == 'stateless' }" class="flex-x static h-start v-center ng-binding ng-scope" tabindex="-1">
</li>
<!-- end ngRepeat: link in crumbs.service.list track by $index -->
</ul>
</app-bar-breadcrumbs>
</div>
</div>
I have tried to find the h2 element using xpath, sometimes with text. Code always returns TimeOutException. The element is not inside a frame but it is inside a bt-app-bar.
Full xpath
homepage_link = wait.until(EC.element_to_be_clickable((By.XPATH, '/html/body/bt-app-chrome/bt-app-bar/div[2]/div[3]/div/h2')))
xpath with tail and contains
homepage_link = wait.until(EC.element_to_be_clickable((By.XPATH, '//h2[contains(., "Password Safe")]')))
xpath with text
homepage_link = wait.until(EC.element_to_be_clickable((By.XPATH, '//*[text()="Password Safe"')))
another xpath with text
homepage_link = wait.until(EC.element_to_be_clickable((By.XPATH,
'/div/h2[contains(text(),"Password Safe")]')))
full css - a little verbose to reproduce here
I have also looked at the posts here: Locate an element in selenium using text inside h2 tag
and nothing there works for me. I just need to locate it and click on it.
Here is something simpler that may be easier to diagnose.
I click to a page which I have attached a screenshot of above. It has an app bar across the top as you can see. On top right is a menu that has "Requests" in text on a clickable element. When my app clicks on it, the app-bar now shows "Accounts" where it showed "Requests" before. I am unable to detect any other change in the app-bar. I have a time.sleep(5) and a wait-until-page-loads after clicking on "Requests". When I watch the html source code change with this click, all that seems to happen (within the context of the app-bar) is that the text "Requests" is now instructed to change to "Accounts". See html source code below.
I am able to locate the "Requests" element perfectly using this:
requests_link = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, 'app-bar-quicklinks > ul > li:nth-child(1)')))
When app-bar changes to "Accounts", CSS for the element seems to be identical to the one I use above for "Requests". Yet, it times out. How come?
<app-bar-quicklinks class="ng-isolate-scope" style="opacity: 1;">
<ul class="flex-x h-start v-center">
<!-- ngRepeat: link in qlinks.service.list track by $index -->
<li ng-repeat="link in qlinks.service.list track by $index" ng-click="qlinks.navTo($index)" ng-keydown="qlinks.navTo($index, $event)" tabindex="0" class="flex-x h-end v-center ng-binding ng-scope">Requests
</li>
<!-- end ngRepeat: link in qlinks.service.list track by $index --> .
<li ng-repeat="link in qlinks.service.list track by $index" ng-click="qlinks.navTo($index)" ng-keydown="qlinks.navTo($index, $event)" tabindex="-1" class="flex-x h-end v-center ng-binding ng-scope">Admin Sessions
</li>
<!-- end ngRepeat: link in qlinks.service.list track by $index -->
</ul>
</app-bar-quicklinks>
I switched to a frame after clicking on 'Requests' and needed to switch out of it to find the 'Accounts' link. Resolved now.
I can't focus on upload button to upload resume.
<div id="uploadCredential" campaignid="6ecdba81-5485-11e8-8fbf-bc764e1156ea" class=" ">
<a href="#">
<i class="icon-laptop"></i>
Upload Resume
</a>
</div>
URL: Website Link
As per the HTML you have shared the element with text as Upload Resume is within the <a> tag and to click on the element you can use either of the following Locator Strategies :
link_text :
driver.find_elements_by_link_text("Upload Resume").click()
css_selector :
driver.find_element_by_css_selector("div#uploadCredential > a").click()
xpath :
driver.find_element_by_xpath("//div[#id='uploadCredential']/a").click()
Use this code :
driver.find_elements_by_link_text("Upload Resume").click()
I have my script to login to a site. i then need to click on another link which is contained in an
<a href> </a>
I have tried multiple methods without success. The link I need "Available Deployments" only appears after clicking a dropdown box called "Job Board".
The site code looks like this:
<li class="">
<a aria-expanded="false" class="dropdown-toggle" data-toggle="dropdown" href="portalPost?s=a1W390000045MxAEAU&p=a1V39000003y7e1EAA" role="button">Job Board <span class="caret"></span>
</a>
<ul class="dropdown-menu" role="menu">
<li>
<a href="portalPage?s=a1W390000045MxAEAU&p=a1V39000003y7dbEAA">Available Deployments
</a>
</li>
i've tried a couple of versions, without success:
-SNIP-
driver.find_element_by_name("logmein").click()
driver.find_element_by_linkText("Job Board").click()
driver.find_element_by_linkText("Available Deployments").click()
and
-SNIP-
driver.find_element_by_name("logmein").click()
driver.find_element_by_xpath(u'//a[text()="Job Board"]').click()
driver.find_element_by_xpath(u'//a[text()="Available Deployments"]').click()
The errors I get typically look like:
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//a[text()="Available Deployments"]"}
It looks like there is a whitespace in text of the element.
You have to use normalize-space to trim text. See an example below.
driver.find_element_by_xpath(u'//a[text()="Job Board"]').click()
waitForPresence = WebDriverWait(driver, 10)
wait.until(EC.presence_of_element_located((By.XPATH,'//a[normalize-space(text())="Available Deployments"]')))
driver.find_element_by_xpath('//a[normalize-space(text())="Available Deployments"]').click()
I solved this with the following:
driver.find_element_by_link_text("Job Board").click()
driver.find_element_by_link_text("Available Deployments").click()
I am trying to use Python Selenium Firefox Webdriver to grab the h2 content 'My Data Title' from this HTML
<div class="box">
<ul class="navigation">
<li class="live">
<span>
Section Details
</span>
</li>
</ul>
</div>
<div class="box">
<h2>
My Data Title
</h2>
</div>
<div class="box">
<ul class="navigation">
<li class="live">
<span>
Another Section
</span>
</li>
</ul>
</div>
<div class="box">
<h2>
Another Title
</h2>
</div>
Each div has a class of box so I can't easily identify the one I want. Is there a way to tell Selenium to grab the h2 in the box class that comes after the one that has the span called 'Section Details'?
If you want grab the h2 in the box class that comes after the one that has the span with text Section Details try below xpath using preceding :-
(//h2[preceding::span[normalize-space(text()) = 'Section Details']])[1]
or using following :
(//span[normalize-space(text()) = 'Section Details']/following::h2)[1]
and for Another Section just change the span text in xpath as:-
(//h2[preceding::span[normalize-space(text()) = 'Another Section']])[1]
or
(//span[normalize-space(text()) = 'Another Section']/following::h2)[1]
Here is an XPath to select the title following the text "Section Details":
//div[#class='box'][normalize-space(.)='Section Details']/following::h2
yeah, you need to do some complicated xpath searching:
referenceElementList = driver.find_elements_by_xpath("//span")
for eachElement in referenceElementList:
if eachElement.get_attribute("innerHTML") == 'Section Details':
elementYouWant = eachElement.find_element_by_xpath("../../../following-sibling::div/h2")
elementYouWant.get_attribute("innerHTML") should give you "My Data Title"
My code reads:
find all span elements regardless of where they are in HTML and store them in a list called referenceElementList;
iterate all span elements in referenceElementList one by one, looking for a span whose innerHTML attribute is 'Section Details'.
if there is a match, we have found the span, and we navigate backwards three levels to locate the enclosing div[#class='box'], and find this div element next sibling, which is the second div element,
Lastly, we locate the h2 element from its parent.
Can you please tell me if my code works? I might have gone wrong somewhere navigating backwards.
There is potential difficulty you may encounter, the innerHTML attribute may contain tab, new line and space characters, in that case, you need regex to do some filtering first.
I'm trying to figure out the answer to a problem I've recently discovered. I have some HTML as indicated below and what I want to do, is send an HTML snipped to selenium so it can click on the checkbox. However, I don't want to lookup by id because as you can see, the id is ugly. Instead, I am looking for an alternative method.
HTML:
<li class= "zone odd open night">...</li>
<li class= "zone even open night">...</li>
<li class= "zone odd open night">...</li>
<label for="srr-2-1397538000">Room 226 1:00 AM</label>
<input type="checkbox" name="srr-2-1397538000" id="srr-2-1397538000" value="Y" class="interactive">
<span class-"drag-handle">...</span>
</li>
<li class="zone even open night">...</li>
As you can see, I'm trying to lookup that specific checkbox to click but I don't want to do it by looking up via id. I would rather look up by "Room" or something more generic.
Does anyone have any suggestions as to how to do this? With using selenium, or some other webdriver?
Also, I've tried the classic:
element = driver.find_element_by_id("srr-2-1397538000")
element.click()
But as I said, this is not what I want.
Thank You
Can you use the tag name?
element = driver.find_element_by_tag_name('input')
element.click()
If you want to look it up by Room, you can use xpath:
element = driver.find_element_by_xpath("//label[contains(text(),'Room')]/following-sibling::input")
element.click()