I was trying to scrape a website, and I need to select only the ul element inside the div with a class "Slider__SliderWrapper-sc-143uniy-0 jrPmnS", however, since there are many div tags with the same class, the only way I have to select just the ul I need is by looking at the href of the a tag, the one inside the h2.
I can't use xpath, because div tags always change position.
<div>
<h2><a class="slider-components__SectionLink-sc-1r2bduf-3 jchpWs" href="rightOne">Right!</a></h2>
<div class="Slider__SliderWrapper-sc-143uniy-0 jrPmnS">
<ul class="Slider__List-sc-143uniy-1 MTYOL">
the right ul
</ul>
</div>
</div>
<div>
<h2><a class="slider-components__SectionLink-sc-1r2bduf-3 jchpWs" href="wrongOne">Something else</a></h2>
<div class="Slider__SliderWrapper-sc-143uniy-0 jrPmnS">
<ul class="Slider__List-sc-143uniy-1 MTYOL">
the wrong ul
</ul>
</div>
</div>
I thought about using css selector but I don't know how to, any help?
You definitely CAN use xpath to access the href attribute AND it's contents:
//a[contains(#href,'rightOne')]
and for the ul:
//h2/a[contains(#href,'rightOne')]/../following-sibling::div/ul
try xpath
//a[#href='rightOne']/../following-sibling::div/ul
Explanation :
You cannot use css_selector or any other locator since you are depending on a tag and you have to traverse upwards in DOM first, we are using /.. for that, alternatively you can use /parent::h2 and the next following-sibling using /following-sibling::div and then finally ul child
You cannot get a parent element with css selector, as it's not possible. Check here Is there a CSS parent selector?
In your case you would need to get the parent of a[href=rightOne] and get the ul of the following sibling.
With css you could use one of these locators:
div:nth-child(1) .Slider__SliderWrapper-sc-143uniy-0.jrPmnS>.Slider__List-sc-143uniy-1.MTYOL
Or
div:nth-child(1) .Slider__SliderWrapper-sc-143uniy-0.jrPmnS>ul
I would select any of XPaths proposed in other two answers if there are not restrictions on selectors.
But, if you are using such libraries as BeautfulSoup, you will have to use css selectors, as it does not support XPath. So, use the ones I proposed.
Related
I have HTML:
<div class="value">
<div style="float: right;">100.00</div>
<span class="yellow">Frequency:</span>
</div>
With selenium By.XPATH '//*[text()="Frequency:"]' I'm able to locate the <span> element. My goal is to get the innerText of the previous <div>.
Can I do it with selenium or should I use bs4 for this task?
I tried selenium.parent with documentation, but unable to do.
PS: I can't find the parent element or the div I need directly.
To print the text from the <div> tag i.e. 100.00 wrt the <span> you can use either of the following locator strategies:
Using xpath and get_attribute("innerHTML"):
print(driver.find_element(By.XPATH, "//span[text()='Frequency:']//preceding::div[1]").get_attribute("innerHTML"))
Using xpath and text attribute:
print(driver.find_element(By.XPATH, "//span[text()='Frequency:']//preceding::div[1]").text)
How can I get the text "950" from the div that has neither a ID nor a Class with python selenium?
<div class="player-hover-box" style="display: none;">
<div class="ps-price-hover">
<div><img class="price-platform-img-hover"></div>
<div>950</div>
</div>
I dont know how I could access this div and its text.
In case player-hover-box is an unique class name you can use the following command
price = driver.find_element_by_xpath('//div[#class="player-hover-box"]/div/div[2]').text
In case there are more products on that page with the similar HTML structure your XPath locator should contain some unique relation to some other element.
I have the below HTML snippet.
<div class="header">Planets</div>
<div class="event">Jupiter</div>
<div class="event">Mars</div>
<div class="header">Stars</div>
<div class="event">Acturus</div>
<div class="event">Pleaides</div>
Using driver.find_elements_by_class_name("event"), I am able to retrieve all the div tags with class "event".
I would want to navigate to the previous sibling and retrieve the div tag with class "header" for each WebElement.
Switch to by find_elements_by_xpath
driver.find_elements_by_xpath("//div[#class='event']/preceding-sibling::div[#class='header']")
I'm trying to have click a button in the browser with Selenium and Python.
The button is within the following
<div id="generate">
<i class="fa fa-bolt"></i>
<span>Download Slides</span>
<div class="clear"></div>
</div>
Chrome's dev console tells me the button is within <span> but I have no idea how to reference the button for a .click().
Well, if you just want to click on an element without an id or name, I'd suggest three ways to do it:
use xpath:
driver.find_element_by_xpath('//*[#id="generate"]/span')
use CSS selector:
driver.find_element_by_css_selector('#generate > span')
Just try .find_element_by_tag_name() like:
driver.find_element_by_id('generate').find_elements_by_tag_name('span')[0]
Note that this way first try to get the generate <div> element by it's id, and then finds all the <span> elements under that <div>.
Finally, gets the first <span> element use [0].
so i have some html like this
<div class="content">
<div class="infobox">
<p> text </p>
<p> more text </p>
</div>
<p> text again </p>
<p> even more text </p>
</div>
And i am using this selector '.content p::text' i thought this would only get me the immediate children, so i wanted it to extract "text again" and "even more text" but it's also getting the text from the paragraphs inside the other div, how can i prevent this from happening, i only want text from the paragraphs that are the immediate children of the div with the class .content
Scrapy uses an extended set of CSS selectors and XPath selectors. In your case, you're using CSS selectors. The CSS relationship selector you want is > denoting a parent/child relationship, as in: .content > p::text. Scrapy's selectors are described in the section titled "Selectors" in its documentation.
to get the child: div>p ( text, more text )
In your case to get what you need: div+p
http://www.w3schools.com/cssref/css_selectors.asp
Worth reading