We Keep Coding

How to find element by XPATH using variable Selenium

I'm trying to parse some elements using XPATH
This is my python code:
driver.get("http://watir.com/examples/shadow_dom.html")
shadow_host = driver.find_element(By.XPATH, '//*[#id="shadow_host"]')
shadow_root1 = shadow_host.shadow_root
shadow_host1 = shadow_root1.find_element(By.XPATH, '/span')
selenium.common.exceptions.InvalidArgumentException: Message: invalid argument: invalid locator
I know that I can use CSS Selectors, but I want to use it in case there aren't any CSS Selectors. Can you help me, how to write shadow_host1 correctly using XPATH? Thanks

You cannot use XPATH locators with shadow root. You can use CSS LOCATORS instead (when on a Chrome/chromedriver setup), or alternatively (when using a Firefox/geckodriver setup) use javascript selectors (querySelector).

InvalidSelectorException Error while trying to get text from div class in Selenium Python

I'm trying to get text using Selenium WebDriver and here is my code. Please note that I don't want to use XPath, because in my case the ID gets changed on every relaunch of the web page.
My code:
driver.find_element_by_class_name("05uR6d").text
HTML:
<div class="O5uR6d">to fasten stuff</div>
Error:
selenium.common.exceptions.InvalidSelectorException: Message: invalid selector: An invalid or illegal selector was specified (Session info: chrome=88.0.4324.150)
Error is specific to the line of code I mentioned above.
How can I fix this?

Use this xpath:
driver.find_element_by_xpath("//div[contains(text(),'to fasten stuff')]")
Or this CSS:
driver.find_element_by_css_selector(".O5uR6d")
If both won't work, improve your question by adding more data of HTML you are looking at.

It can be done using multiple ways let me try to explain most of them.
Get element by class name.
this is the most easiest solution to get any element by class name you can simply do is:
driver.find_element_by_class_selector('foo');
Get Element by xpath
This is a bit tricky one, you can apply xpath either the class name, title, id or whatever remains same. it also works even if there's a text inside your div. For example:
driver.find_element_by_xpath("//tagname[#attribute='value']")
or in your case:
driver.find_element_by_xpath("//div['class='O5uR6d']")
or you can do something like #vitaliis said
driver.find_element_by_xpath("//div[contains(text(),'to fasten stuff')]")
You can read more about xpath and how to find it on this link
Get Elements by ID:
You can also get the element from id if there's any that's static:
driver.find_element_by_id('baz')
Get Elements by Name:
Get Elements by name using the following syntax:
driver.find_element_by_name('bazz')
Using CSS Selectors:
You can also use the css selectors to find the elements. Consider a following tag that has some attributes:
<p class="content">Site content goes here.</p>
You can get this element by:
driver.find_element_by_css_selector('p.content')
You can read more about it over here

How to locate the dropdown select element and select an option within a complex website using Selenium and Python

I have a very complex website which I am trying to test with Selenium. But when I try to get the XPath, I get like this for example.
//*[#id="datatable1595356931082"]/div[1]/div[2]/table/tbody/tr[3]/td[2]/div/select/option[8]
Absolute XPath:
/html/body/div[4]/div[2]/div[2]/div/div[2]/div[2]/div/div/div[1]/div[2]/table/tbody/tr[3]/td[2]/div/select/option[8]
in Selenium I tried with absolute path like
driver.find_element_by_xpath("/html/body/div[4]/div[2]/div[2]/div/div[2]/div[2]/div/div/div[1]/div[2]/")
and it tries to error out saying it is unable to find XPath. The datatable seems to create a dynamic number during runtime.
What here I am trying to do is to select a drop down which looks like this
<div role="columnheader" class="webix_hcell webix_ss_filter">
<select>
<option value=""></option>
<option value="A">A</option>
<option value="B">B</option>
.
.
.
</select>
</div>
I have also tried
driver.find_element_by_class_name('webix_hcell webix_ss_filter')
But that too errors out
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":".webix_hcell webix_ss_filter"}
For the above one
selenium.common.exceptions.InvalidSelectorException: Message: invalid selector: Unable to locate an element with the xpath expression /html/body/div[4]/div[2]/div[2]/div/div[2]/div[2]/div/div/div[1]/div[2]/ because of the following error:
SyntaxError: Failed to execute 'evaluate' on 'Document': The string '/html/body/div[4]/div[2]/div[2]/div/div[2]/div[2]/div/div/div[1]/div[2]/' is not a valid XPath expression.
(Session info: chrome=84.0.4147.89)

This won't work, because *_by_class_name is only for single class name:
driver.find_element_by_class_name('webix_hcell webix_ss_filter')
Try using *_by_css_selector:
element = driver.find_element_by_css_selector('div.webix_hcell.webix_ss_filter select')

There are a couple of things you need to consider as follows:
From your first attempt the WebElement appears as a dynamic element due to presence of the id attribute value as datatable1595356931082, which possibly will change on every access or periodically:
//*[#id="datatable1595356931082"]/div[1]/div[2]/table/tbody/tr[3]/td[2]/div/select/option[8]
In your second attempt you have used an absolute xpath. As the website is dynamic, elements will be repositioned on every access or periodically:
/html/body/div[4]/div[2]/div[2]/div/div[2]/div[2]/div/div/div[1]/div[2]/table/tbody/tr[3]/td[2]/div/select/option[8]
In your third attempt the xpath ends with / which isn't desired. Hence you face InvalidSelectorException
In your forth attempt you have passed multiple classes through driver.find_element_by_class_name('webix_hcell webix_ss_filter') where as driver.find_element_by_class_name() accepts only one classname as an argument.
You can find a detailed discussion in Invalid selector: Compound class names not permitted error using Selenium
Finally, it's a <select> node, so you need to use the Select class.
You can find a detailed discussion in How to select a drop-down menu value with Selenium using Python?
Solution
The relevant text based HTML would have helped us to construct a canonical answer. However as per the HTML provided to click on the option with text as A you can use the following either of the following Locator Strategies:
Using XPATH and select_by_visible_text():
select = Select(WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[#class='webix_hcell webix_ss_filter' and #role='columnheader']//following::select[1]"))))
select.select_by_visible_text('A')
Using CSS_SELECTOR and select_by_value():
select = Select(WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "div.webix_hcell.webix_ss_filter[role='columnheader'] +select"))))
select.select_by_value('A')
Note : You have to add the following imports :
from selenium.webdriver.support.ui import Select
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Unable to locate element error retrieving text value of class variable

I am trying to grab the value of the version info, 1.0, in this case.
If I right click and Inspect Element, I see the following:
<span class="ng-binding" ng-bind="$ctrl.Version">1.0</span>
If I copy CSS Selector in Firefox, it shows me the following:
.version-numbers > div:nth-child(1) > span:nth-child(2)
I am trying the following Python code:
print(driver.find_element_by_css_selector(".version-numbers[value='$ctrl.backendVersion']").text)
I haven't been able to get it to work with XPath either. I get an Unable to locate Element error.
Any suggestions?

As per the given HTML to retrieve the text 1.0 you need to induce WebDriverWait with expected_conditions clause as visibility_of_element_located and you can use the following line of code :
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//span[#class='ng-binding']"))).get_attribute("innerHTML"))

I used CSS Selector and it appears to be working now. Thank you for your suggestions.

Using XPath Selector 'following-sibling::text()' in Selenium (Python)

I'm trying to use Selenium (in Python) to extract some information from a website. I've been selecting elements with XPaths but am having trouble using the following-sibling selector. The HTML is as follows:
<span class="metadata">
<strong>Photographer's Name: </strong>
Ansel Adams
</span>
I can select "Photographer's Name" with
In [172]: metaData = driver.find_element_by_class_name('metadata')
In [173]: metaData.find_element_by_xpath('strong').text
Out[173]: u"Photographer's Name:"
I'm trying to select the section of text after the tag ('Ansel Adams' in the example). I assumed I could use the following-sibling selector but I receive the following error:
In [174]: metaData.find_element_by_xpath('strong/following-sibling::text()')
ERROR: An unexpected error occurred while tokenizing input
The following traceback may be corrupted or invalid
The error message is: ('EOF in multi-line statement', (328, 0))
... [NOTE: Omitted the traceback for brevity] ...
InvalidSelectiorException: Message: u'The given selector strong/following-sibling::text() is either invalid or does not result in a WebElement. The following error occurred:\n[InvalidSelectorError] The result of the xpath expression "strong/following-sibling::text()" is: [object Text]. It should be an element.'
Any ideas as to why this isn't working?

#RossPatterson is correct. The trouble is that the text 'Ansel Adams' is not a WebElement, so you cannot use find_element or find_elements. If you change your HTML to
<span class="metadata">
<strong>Photographer's Name: </strong>
<strong>Ansel Adams</strong>
</span>
then find_element_by_xpath('strong/following-sibling::*[1]').text returns 'Ansel Adams'.

This is documented in this Selenium bug report:
http://code.google.com/p/selenium/issues/detail?id=5459
"Your xpath doesn't return an element; it returns a text node. While this might have been perfectly acceptable in Selenium RC (and by extension, Selenium IDE), the methods on the WebDriver WebElement interface require an element object, not just any DOM node object. WebDriver is working as intended. To fix the issue, you'd need to change the HTML markup to wrap the text node inside an element, like a ."

To get the text "Ansel Adams", just use metaData.get_text(). I don't believe find_element_by_* will allow you to find a text node.