I'd like to click on the following link but it's not working:
<ul class="pagination pagination-large">
<li style="display:inline;">
<a name="Next" href="jamm/flavours/page=2" class="next">
<span class="icon-navigate_next"></span>
</a>
</li>
</ul>
My code
items = browser.find_elements_by_xpath("//ul[#class = 'pagination pagination-large']//li[#style ='display:inline;']")
print items
for k in items:
print k
k.click()
print("clicked")
k.send_keys(webdriver.common.keys.Keys.SPACE)
I think the problem is the xpath you are using is not finding the element you need, in fact, it targets the list element, not the anchor.
Maybe you can try to identify the link by using the css class next instead:
items = browser.find_elements_by_class_name('next')
for item in items:
item.click()
If it works now, you can either just use that if you don't have any other elements using it or you can fix your xpath.
Try:
browser.find_elements_by_name('Next')
Related
I have a bellow HTML:
<div class="row">
<div><span class="checkbox"></span></div>
<a title="something"></div>
</div>
<div class="row">
<div><span class="checkbox"></span></div>
<a title="somethingelse"></div>
</div>
I would like to click on the checkbox if the a's title is something. I do not know how many rows are there so I need to check every row and if there the title == 'something' click the previous checkbox.
I tried to implement this in Python and Selenium, but so far could not succeed. I can check for a like:
driver.find_element_by_xpath(f"//div[contains(#class, 'row')]//a[#title='something']")
But how I can select the previous element?
Does anybody have a recommendation?
I am using get_attribute and trying to extract title and then putting a if clause that if title attribute text matches with your expected string then click on the above check box.
try this :
actual_text = driver.find_element_by_xpath(f"//div[contains(#class, 'row')]//a[#title='something']")
if actual_text.get_attribute('title') == 'your expected string here':
driver.find_element(By.XPATH, "//div[contains(#class, 'row')]//a[#title='something']/../child::span")
Try this:
driver.find_element_by_xpath("//div[#class='row']/a[#title='something']/..//span[#class='checkbox']").click()
With Python Scrapy, I am trying to get contents in a webpage whose nodes look like this:
<div id="title">Title</div>
<ul>
<li>
<span>blahblah</span>
<div>blahblah</div>
<p>CONTENT TO EXTRACT</p>
</li>
<li>
<span>blahblah</span>
<div>blahblah</div>
<p>CONTENT TO EXTRACT</p>
</li>
...
</ul>
I'm a newbie with XPath and couldn't get it for now. My last try was something like:
contents = response.xpath('[#id="title"]/following-sibling::ul[1]//li//p.text()')
... but it seems I cannot use /following-sibling after [#id="title"].
Any idea?
Try this XPath
contents = response.xpath('//div[#id="title"]/following-sibling::ul[1]/li/p/text()')
It selects both "CONTENT TO EXTRACT" text nodes.
One XPath would be:
response.xpath('//*[#id="title"]/following-sibling::ul[1]//p/text()).getall()
which get text from every <p> tag child or grand child of nearest <ul> tag to node with id = "title".
XPath syntax
Try this using css selector.
response.css('#title ::text).extract()
guys,
I have a question, scrapy, selector, XPath
I would like to choose the link in the "a" tag in the last "li" tag in HTML, and how to write the query for XPath
I did that, but I believe there are simpler ways to do that, such as using XPath queries, not using list fragmentation, but I don't know how to write
from scrapy import Selector
sel = Selector(text=html)
print sel.xpath('(//ul/li)').xpath('a/#href').extract()[-1]
'''
html
'''
</ul>
<li>
<a href="/info/page/" rel="follow">
<span class="page-numbers">
35
</span>
</a>
</li>
<li>
<a href="/info/page/" rel="follow">
<span class="next">
next page.
</span>
</a>
</li>
</ul>
I am assuming you want specifically the link to the "next" page. If this is the case, you can locate an a element checking the child span to the "next" class:
//a[span/#class = "next"]/#href
I am trying to use Python Selenium Firefox Webdriver to grab the h2 content 'My Data Title' from this HTML
<div class="box">
<ul class="navigation">
<li class="live">
<span>
Section Details
</span>
</li>
</ul>
</div>
<div class="box">
<h2>
My Data Title
</h2>
</div>
<div class="box">
<ul class="navigation">
<li class="live">
<span>
Another Section
</span>
</li>
</ul>
</div>
<div class="box">
<h2>
Another Title
</h2>
</div>
Each div has a class of box so I can't easily identify the one I want. Is there a way to tell Selenium to grab the h2 in the box class that comes after the one that has the span called 'Section Details'?
If you want grab the h2 in the box class that comes after the one that has the span with text Section Details try below xpath using preceding :-
(//h2[preceding::span[normalize-space(text()) = 'Section Details']])[1]
or using following :
(//span[normalize-space(text()) = 'Section Details']/following::h2)[1]
and for Another Section just change the span text in xpath as:-
(//h2[preceding::span[normalize-space(text()) = 'Another Section']])[1]
or
(//span[normalize-space(text()) = 'Another Section']/following::h2)[1]
Here is an XPath to select the title following the text "Section Details":
//div[#class='box'][normalize-space(.)='Section Details']/following::h2
yeah, you need to do some complicated xpath searching:
referenceElementList = driver.find_elements_by_xpath("//span")
for eachElement in referenceElementList:
if eachElement.get_attribute("innerHTML") == 'Section Details':
elementYouWant = eachElement.find_element_by_xpath("../../../following-sibling::div/h2")
elementYouWant.get_attribute("innerHTML") should give you "My Data Title"
My code reads:
find all span elements regardless of where they are in HTML and store them in a list called referenceElementList;
iterate all span elements in referenceElementList one by one, looking for a span whose innerHTML attribute is 'Section Details'.
if there is a match, we have found the span, and we navigate backwards three levels to locate the enclosing div[#class='box'], and find this div element next sibling, which is the second div element,
Lastly, we locate the h2 element from its parent.
Can you please tell me if my code works? I might have gone wrong somewhere navigating backwards.
There is potential difficulty you may encounter, the innerHTML attribute may contain tab, new line and space characters, in that case, you need regex to do some filtering first.
I am using Python Selenium to look through some HTML and find elements. I have the following HTML saved into Python...
<section id="categories">
<ul id="category_list">
<li id="category84">
Sample Category
</li>
<li id="category984">
Another Category
</li>
<li id="category22">
My Sample Category
</li>
</ul>
</section>
I can find the categories section easy enough but now I would like to loop through each list item and save it's name and href link into an array.
Anyone got a similar example I can see?
Sure, let's use a CSS selector locator and a list comprehension calling .get_attribute("href") to get the link and .text to get the link text:
categories = driver.find_elements_by_css_selector("#categories #category_list li[id^=category] a")
result = [{"link": category.get_attribute("href"), "text": category.text}
for category in categories]
print(result)