I want to search class name with starts-with in specific Webelement but it search in entire page. I do not know what is wrong.
This returns list
muidatagrid_rows = driver.find_elements(by=By.CLASS_NAME, value='MuiDataGrid-row')
one_row = muidatagrid_rows[0]
This HTML piece in WebElement (one_row)
<div class="market-watcher-title_os_button_container__4-yG+">
<div class="market-watcher-title_tags_container__F37og"></div>
<div>
<a href="#" target="blank" rel="noreferrer" data-testid="ios download button for 1628080370">
<img class="apple-badge-icon-image"></a>
</div>
<div></div>
</div>
If a search with full class name like this:
tags_and_marketplace_section = one_row.find_element(by=By.CLASS_NAME, value="market-watcher-title_os_button_container__4-yG+")
It gives error:
selenium.common.exceptions.InvalidSelectorException: Message: Given css selector expression ".market-watcher-title_os_button_container__4-yG+" is invalid: InvalidSelectorError: Element.querySelector: '.market-watcher-title_os_button_container__4-yG+' is not a valid selector: ".market-watcher-title_os_button_container__4-yG+"
So i want to search with starts-with method but i can not get what i want.
This should returns only two Webelements but it returns 20
tags_and_marketplace_section = one_row.find_element(by=By.XPATH, value='//div[starts-with(#class, "market-watcher-")]')
print(len(tags_and_marketplace_section))
>>> 20
Without seeing the codebase you are scraping from it's difficult to help fully, however what I've found is that "Chaining" values can help to narrow down the returned results. Also, using the "By.CSS_SELECTOR" method works best for me.
For example, if what you want is inside a div and p, then you would do something like this;
driver.find_elements(by=By.CSS_SELECTOR, value="div #MuiDataGrid-row p")
Then you can work with the elements that are returned as you described. You maybe able to use other methods/selectors but this is my favourite route so far.
Related
I've defined css selectors within the script to get the text within span elements and I'm getting them accordingly. However, the way I tried is definitely messy. I just seperated different css selectors using comma to let the script understand I'm after this or that.
If I opt for xpath I could have used 'div//span[.="Featured" or .="Sponsored"]' but in case of css selector I could not find anything similar to serve the same purpose. I know using 'span:contains("Featured"),span:contains("Sponsored")' I can get the text but there is the comma in between as usual.
What is the ideal way to locate the elements (within different ids) using css selectors except for comma?
My try so far with:
from lxml.html import fromstring
html = """
<div class="rest-list-information">
<a class="restaurant-header" href="/madison-wi/restaurants/pizza-hut">
Pizza Hut
</a>
<div id="featured other-dynamic-ids">
<span>Sponsored</span>
</div>
</div>
<div class="rest-list-information">
<a class="restaurant-header" href="/madison-wi/restaurants/salads-up">
Salads UP
</a>
<div id="other-dynamic-ids border">
<span>Featured</span>
</div>
</div>
"""
root = fromstring(html)
for item in root.cssselect("[id~='featured'] span,[id~='border'] span"):
print(item.text)
You can do:
.rest-list-information div span
But I think it's a bad idea to consider the comma messy. You won't find many stylesheets that don't have commas.
If you are just looking to get all 'span' text from the HTML then the following should suffice:
root_spans = root.xpath('//span')
for i, root_spans in enumerate(root_spans):
span_text = root_spans.xpath('.//text()')[0]
print(span_text)
<div class="turbolink_scroller" id="container">
<article><div class="inner- article">
<a style="height:81px;" href="LINK TO EXTRACT">
<img width="81" height="81" src="//image.jpg" alt="code" />
Hello! I'm pretty new to selenium and I've been playing around with how to get sources for my webdriver. So far, I'm trying to extract a href link given an alt code as above and I'm not sure if the documentation has a means to do this. I'm feeling that the answer is find_by_xpath but I'm not entirely sure. Thank you for any tips!
The way is as follows
href = driver.find_element_by_tag_name('a').get_attribute('href')
of course, you may have a lot of 'a' tags in a page, so you may make the path to your respective tag,
e.g
div = driver.find_element_by_id('container')
a = div.find_element_by_tag_name('a')
href = a.get_attribute('href')
How to identify the link, I have inspected the elements which are as below :
<div class="vmKOT" role="navigation">
<a class="Ml68il" href="https://www.google.com" aria-label="Search" data-track-as="Welcome Header Search"></a>
<a class="WaidDw" href="https://mail.google.com" aria-label="Mail" data-track-as="Welcome Header Mail"></a>
<a class="a4KP9d" href="https://maps.google.com" aria-label="Maps" data-track-as="Welcome Header Maps"></a>
<a class="QJOPee" href="https://www.youtube.com" aria-label="YouTube" data-track-as="Welcome Header YouTube"></a>
</div>
I want to identify the class WaidDw or href and click it using python.
You can try
driver.find_element_by_class_name('WaidDw').click()
or
driver.find_element_by_xpath('//a[#href="https://mail.google.com" and #aria-label="Mail"]').click()
In your provided HTML all attribute's values are unique, you can locate easily that element by using their attribute value.
As your question points to locate this <a class="WaidDw" href="https://mail.google.com" aria-label="Mail" data-track-as="Welcome Header Mail"></a> element. I'm providing you multiple cssSelectors which can work easily to identify the same element as below :-
a.WaidDw
a.WaidDw[href='https://mail.google.com']
a.WaidDw[aria-label='Mail']
a.WaidDw[data-track-as='Welcome Header Mail']
a.WaidDw[href='https://mail.google.com'][aria-label='Mail']
a.WaidDw[href='https://mail.google.com'][aria-label='Mail'][data-track-as='Welcome Header Mail']
Note :- Keep in practice (priority) to use cssSelector instead xpath if possible, because cssSelectors perform far better than xpath
Locating Element by CSS Selectors using python :-
element = driver.find_element_by_css_selector('use any one of the given above css selector')
Clicks the element :-
element.click()
Reference link :-
https://www.w3schools.com/cssref/css_selectors.asp
https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Selectors
I'm trying to find one of these elements (vote up or vote down):
<div class="votingWrapper ">
<span class="voteBtn voteUp " data-postid="c4fcff79-7f73-493c-b6d8-f05ff1962897"></span>
<span class="postRank">
<span class="rankWrapper wrapper_up" data-points="1320">+1.3k</span>
<span class="rankWrapper wrapper_down" data-points="512">-512</span>
</span>
<span class="voteBtn voteDown " data-postid="c4fcff79-7f73-493c-b6d8-f05ff1962897"></span>
</div>
I tried to locate it by class, id, xpath (a lot of types) but I got NoSuchElementException for all attempts.
element = driver.find_element_by_id("c4fcff79-7f73-493c-b6d8-f05ff1962897")
element = driver.find_element_by_class_name("voteBtn voteDown ")
element = driver.find_element_by_xpath('//*[#id="c4fcff79-7f73-493c-b6d8-f05ff1962897"]/div[1]/span[3]')
I searched for 3 hours how to do that but I failed. It's impossible.
Can anyone give me some tip?
You have pass single class at a time
element_up = driver.find_element_by_class_name("voteUp")
element_down = driver.find_element_by_class_name("voteDown")
Using space in html tags is usually deprecated but your issue can be resolve easily.
In CSS, you can access a div with a class name with a name separate with a space using a dot.
Example :
<span class="voteBtn voteUp " data-postid="c4fcff79-7f73-493c-b6d8-f05ff1962897"></span>
can be access in CSS file by :
.voteBtn.voteUp {}
You can use the same technic with your selenium and as a one liner :
element = driver.find_element_by_css_selector("span.voteBtn.voteDown")
try the following XPaths (in case they are not in a frame):
to find voteUp:
//span[contains(#class,"voteBtn voteUp")]
to find voteDown:
//span[contains(#class,"voteBtn voteDown")]
If they are inside a frame (check whether the elements you are trying to find, are child elements of an iframe tag), then first find the frame and then switch to it. then use the XPath to find elements.
CSS for vote up: div.votingWrapper > span
or
span[class*='voteUp']
CSS for vote down: div.votingWrapper > span:nth-child(3)
or
span[class*='voteDown']
I'm trying to get Selenium to click on View All Companies button, but i'm not sure what am I doing wrong. It returns no element found
html code
<div class="screener-toggles">
<div class="buttons">
<span class="button selected" data-name="advanced-screener">Search by Screener<span data-name="advanced-screener" class="arrow selected"></span></span>
<span class="button" data-name="alpha-factors">Search by Alpha Factors<span data-name="alpha-factors" class="arrow"></span></span>
<span class="button" data-name="all-companies">View All Companies<span data-name="all-companies" class="arrow"></span></span>
</div>
</div>
python code I wrote
element1 = driver.find_elements_by_class_name('View All Companies')
element1.click()
# I have tried all-companies instead of View All Companies as well. But still doesn't work
Should I not be using find_elements_by_class_name?
Any advice on what I am doing wrong is greatly appreciated!
try xpath: "//span[contains(text(),'View All Companies')]"
View All Companies is text, not the class. Try looking by text with css_selector or xpath
element1 = find_element_by_css_selector('span:contains("View All Companies")')
element1 = find_element_by_xpath('//span[contains(text(), "View All Companies")]')
Or by the data-name attribute which contains all-companies
element1 = find_element_by_css_selector('span[data-name*="all-companies"]')
Yes, you should not use the find_elements_by_class_name instead of use find_element_by_class_name.
find_elements_by_class_name is used when your expecting your locator to return more than 1 element. for a specific element use only find_element_by_class_name.
Another thing is I am not able to see any class name as View All Companies in your HTML code. Please look into your HTML and select classname or other locator carefully
Hope it will help you