I'm writing a Python program that uses Selenium to navigate to and enter information into search boxes on an advanced search page. This website uses Javascript, and the IDs and Names for each search box change slightly each time the website is loaded, but the Class Names remain consistent. Class names are frequently reused though, so my goal is to use find_elements_by_class_name(classname) and then index through that list.
One box, for example, has the class name x-form-text x-form-field x-form-num-field x-form-empty-field, but I can't use this because selenium considers it a compound class name and throws an error. If I use just a portion of it, such as x-form-text, it can't find the element. My hope is to either find a way to allow the spaces or, if that can't be done, find a way to search for all elements whose class name contains a section of text without spaces, such as x-form-text.
Any help or thoughts would be greatly appreciated!
Edit:
I tried this code:
quantminclass = 'x-form-text.x-form-field.x-form-num-field.x-form-empty-field'
quantmin = '25'
browser.find_elements_by_css_selector(quantminclass)[0].send_keys(quantmin)
But got an error that the list index was out of range, implying that it can't find anything. I inspected the element and that is definitely its class name, so I'm not sure how to proceed.
Those are multiple classes, not a single class with spaces, just use all the classes together.
driver.find_element_by_css_selector('.class1.class2.class3')
In CSS selector a dot . is a class, you can concatenate any number class names
Try converting class name to a CSS selector.
With a CSS selector, a class named x-form-text x-form-field x-form-num-field
turns into .x-form-text.x-form-field.x-form-num-field
So basically just replace spaces with dots and you're good to go.
Since Selenium 4 find_element_by_* is depricated, so you need to use
find_element() [Selenium-doc]
from selenium.webdriver.common.by import By
# By CLASS_NAME
driver.find_element(By.CLASS_NAME, "x-form-text.x-form-field.x-form-num-field.x-form-empty-field")
# By CSS_SELECTOR
driver.find_element(By.CSS_SELECTOR, ".x-form-text.x-form-field.x-form-num-field.x-form-empty-field")
# By XPATH
driver.find_element(By.XPATH, "//*[#class='x-form-text x-form-field x-form-num-field x-form-empty-field']")
If you have class name (or another attrs) with spaces, for example:
<div class="target with space or maybe another-long-text">Test 123</div>
This will work:
driver.find_element_by_xpath("//div[#class='target with space or maybe another-long-text']")
Related
If I am implementing string locators, such as:
continue_button: str = "button:has-text(\"Continue\")"
If there are multiple buttons on the same page that say continue, but are for different paths, how do I select the correct continue... is there a way to add an index to that string locator?
There is several good practices for creating locators/selectors.
Using playwright there is official documentation for each common and unique selector on how-to and what-is doing.
More information in https://playwright.dev/docs/selectors#text-selector
About your case, i would suggest always to use an parent selector for locating an element.
When there is a button, try to find its unique parent.
By id
By unique class
Something else unique.
Example:
<dv id=test>
<button id=continue-test>Continue</button>
</div>
In this case you can use the unique id of the button and not the text.
Selector css: #continue-test
But if you, don't have an unique identifier for the button you can use the parent and go down to the button.
Selector css: #test > button
Matching text using css is not possible, but with XPATH can look like this:
//button[text()="Continue"]
This selector MATCHES the text using "equals".
Using playwright:
button:has-text("Continue")
Using has-text and quotes - matches the text using equals.
If you are using another selector for example text=Continue, this will match all elements that CONTAINS the text "Continue"
All this is explained with example in the official documentation for playwright selectors.
That does not mean to not use XPATH to achieve the goals.
CSS selectors are fast but kind of restricted to work with text.
Xpath is quite slower but much more powerful to work in text/parent/child elements etc.
I would suggest always to use an parent element with unique identifier and go down to reach your actual element, which will receive the interaction.
The fact that I love Playwright is because of scenarios like this and how easily it can be handled.
If you have a string named abc and there are multiple occurrences of that string on a single page, then you can use the nth-match criteria to pick the nth element.
For eg ,
await page.locator(':nth-match(:text("abc"), 3)').click();
will select the 3rd occurrence of the word abc. Similarly, in your case, if you want to select the first or second or third, you can simply do
await page.locator(':nth-match(:text("Continue"), 1)').click();
await page.locator(':nth-match(:text("Continue"), 2)').click();
await page.locator(':nth-match(:text("Continue"), 3)').click();
Please refer to the Selectors documentation for Playwright -> Selectors
This is different than the nth-child concept as mentioned
Unlike :nth-child(), elements do not have to be siblings, they could
be anywhere on the page. In the snippet above, all three buttons match
:text("Buy") selector, and :nth-match() selects the third button.
I'm trying to locate a specific HTML element via class name but it's not finding anything. I've looked into it for a while but I'm unable to identify the issue
HTML snippet: div's class name is fl-l score
My Python Code:
# Throws Timeout Exception because element unable to be located.
WebDriverWait(self.driver, 5).until(ec.presence_of_element_located((By.CLASS_NAME, "fl-l score")))
score_block = self.driver.find_element_by_class_name("fl-l score")
I should also note that I tried finding it using XPath, which worked, but it's not a viable solution because the website is dynamic.
Instead of finding element by class name, you can change the locator by css selector for multiple class names:
WebDriverWait(self.driver, 5).until(ec.presence_of_element_located((By.CSS_SELECTOR, ".fl-l.score")))
score_block = self.driver.find_element_by_css_selector(".fl-l.score")
The above code hits the page twice, to be more efficient you should compress it into one line, like so:
score_block = WebDriverWait(self.driver, 5).until(ec.presence_of_element_located((By.CSS_SELECTOR, ".fl-l.score")))
In Selenium we do not have support for multiple class name, so basically we are left with only css and xpath in this kinda situation.
Since xpath is not viable solution in your case. try css.
You can combine multiple classes by removing the spaces in between, and simply putting . in that place.
Also, WebdriverWait will always return a common exception, and it's hard to figure out what exactly went wrong, so try using driver.find_element in case you want to have the proper error stack trace.
But for just that matter you should not remove WebdriverWait.
Try this :
score_block = self.driver.find_element_by_class_name("div.fl-l.score")
should work for you. if it works then replace with WebDriverWait
The picture attached show the structure of the HTML page that I am trying to scrape:
First I retrieve the element league-item and then I am looking for the i item with class name : 'ds-icon-material league-toggle-icon'
Selenium is telling me that it cannot find any item with such name.
Here is my code:
path = r"""chromedriver.exe"""
driver = webdriver.Chrome(executable_path=path)
driver.get(_1bet)
time.sleep(5)
#a = driver.find_element_by_class_name('box-content.box-bordered.box-stick.box-bordered-last')
league1 = driver.find_elements_by_class_name('league-list')[0]
league1.find_element_by_class_name("ds-icon-material league-toggle-icon")
Can you please help me? I dont understand why it isn't working.
Thanks
NB: The website I'm scraping is: https://1bet.com/ca/sports/tennis?time_range=all
I can't access that web page so I can only guess what is going there.
I can figure 2 problems here:
To select element inside element it's better to use XPath starting with a dot .
The element you trying to access having 2 class names. You should use css selector or XPath to locate element according to multiple class names.
So I suggest you trying this:
league1 = driver.find_elements_by_class_name('league-list')[0]
league1.find_element_by_xpath(".//i[#class='ds-icon-material league-toggle-icon']")
Selenium expects single class name - and it adds dot at the beginning to create CSS selector.
But "ds-icon-material league-toggle-icon" is two classes and it will add dot befor first class but not before second class and this makes proble.
You may use directly css selector with all dots
.find_element_by_css_selctor(".ds-icon-material.league-toggle-icon")
or you have to trick Selenium and add missing dots between classes
.find_element_by_class_name("ds-icon-material.league-toggle-icon")
I can't connect with this page to confirm that this is all.
I have a number of links in rows, in a web page, whose class-names are the same. Like this:
I am able to click the first link occurrence using XPATH,
"(//span[#class='odds black'])"
However, I want to scan through the particular row and click on each odds (if it is present).
Any help on how to achieve this ?
Note: I cannot find the element using other attributes, as it will change dynamically as per the data.
Image of reference source code:
Instead of using the XPATH in this format:
"(//span[#class='odds black'])"
could you use it in this format shown just above your red box:
/html/body/div[2]/div[3]/div[1]/div[1]/div[2]table/tbody[31]/tr[1]/td[5]/a/span[2]/span/span
(you can get this format easily by selecting an element in firebug, right clicking it's code and selecting copy XPATH).
I have found in many instances I can add a counter for a tr[1] or some other path attribute in order to move down rows quite accurately. I can't really see your site to compare the xpath below but I imagine it would be something like:
/html/body/div[2]/div[3]/div[1]/div[1]/div[2]table/tbody[31]/tr[1]/td[5]/a/span[2]/span/span
/html/body/div[2]/div[3]/div[1]/div[1]/div[2]table/tbody[31]/tr[2]/td[5]/a/span[2]/span/span
/html/body/div[2]/div[3]/div[1]/div[1]/div[2]table/tbody[31]/tr[3]/td[5]/a/span[2]/span/span
then you can add a counter like "i"
so you would iterate the counter in the loop and set it to something along the lines of:
"/html/body/div[2]/div[3]/div[1]/div[1]/div[2]table/tbody[31]/tr["+str(i)+"]/td[5]/a/span[2]/span/span"
Assuming that class name will be always 'odds some color' you can use xpath's contains() function. Xpath like this:
"//span[contains(#class,'odds')]"
will return all spans that contain string 'odds' in classname.
CSS selectors are class aware so it would make more sense to me to use;
span.odds
Xpath treats class as a simple string so forces you to use "contains" where as CSS allows you to treat classes separately
I'm trying to make a test for my site. Having troubles on some user form. The trick is, that the number of text fields in the form varies depending in user options (disabled ones are present in the code, but have a style <displayed: none;> tag), so I'm trying to find more flexible approach than locating every element one-by-one and filling the forms with try/except blocks.
I'm using an xpath locator
text_fields = driver.find_elements_by_xpath("//div[#class='form-line']/div[#class='form-inputs']/input[#type='text' and not(ancestor::div[#style='display: none;'])]")
The trouble is that firebug locates only needed elements, but when I use it my selenium script, printing the list of text_fields gives me all the elements, even without a <displayed: none;> tag
How can I get only visible elements?
PS Sorry for my bad English ^_^
You can get all the form elements the usual way, then iterate on the list and remove those elements that do not return true on is_displayed().
Try the contains() method:
text_fields = driver.find_elements_by_xpath(
"//div[#class='form-line']/div[#class='form-inputs']/input[#type='text' and
not(ancestor::div[contains(#style, 'display: none;')])]")
The important part is:
div[contains(#style, 'display: none;')]
Note, that if the style contains the string display:none; or display:none, the selector won't match.
I use the following and it works great.
self.assertTrue(driver.find_element_by_xpath("//div[#id='game_icons']/div/div[2]/div/a/img"))
This is for Selenium and Python of course.