Unable to find element that looks like it's dynamically generated - python

I'm trying to find the email address text located in the below URL. You can clearly see the email address but I believe the text is generated dynamically through Javascript/React. When I copy the XPATH or CSS Selector and try to find the element like I would any other element I just get an error saying the element cannot be found.
I've tried time.sleep(30) to give the page time to fully load but that's not the issue.
I've tried:
driver.find_element(By.XPATH, '//*[#id="mount_0_0_D8"]/div/div[1]/div/div[5]/div/div/div[3]/div/div/div[1]/div[1]/div/div/div[4]/div[2]/div/div[1]/div[2]/div/div[1]/div/div/div/div/div[2]/div[2]/div/ul/div[2]/div[2]/div/div/span')
You can see from the snippet below that the email is visible but is between some ::before and ::after text I've not seen before.
https://www.facebook.com/homieestudio
Any ideas on how to consistently pull back the email address here? I'm using Chromedriver.

The clue is, the Email Address will always have the # sign.
Solution
To extract the text Info#homieestudio.co.uk ideally you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following locator strategies:
Using XPATH and text attribute:
driver.get('https://www.facebook.com/homieestudio')
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//span[contains(., '#')]"))).text)
Using XPATH and get_attribute("textContent"):
driver.get('https://www.facebook.com/homieestudio')
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//span[contains(., '#')]"))).get_attribute("textContent"))
Console output:
Info#homieestudio.co.uk
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python

You can find the email address using the below
wait = WebDriverWait(driver, 20)
element = wait.until(EC.visibility_of_element_located((By.XPATH, "//span[#dir='auto'][contains(text(), 'info#homieestudio.co.uk')]")))
OR
wait = WebDriverWait(driver, 20)
element = wait.until(EC.presence_of_element_located((By.XPATH, "//span[#dir='auto'][contains(text(), 'info#homieestudio.co.uk')]")))
IMPORT
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

Related

Selenium/Python: no such element with: driver.find_element(By.ID)

I try to locate elements on a website which are in a form to then send keys to login but I get an error, see below:
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="username"]"}
Line of code is:
driver.find_element(By.ID, "username").send_keys("test123")
For HTML code see:
HTML code
I use latest version of Python/Selenium and PyCharm.
I hope someone can help.
I don't see anything wrong in that line of code. Try one of the below
Try applying implicit wait as below:
driver = webdriver.Chrome()
driver.maximize_window()
driver.implicitly_wait(20)
Check if there is a frame/ iframe covering this element, if yes, try below code before locating the element
iframe = driver.find_element_by_xpath("<XPath expression of the frame>")
driver.switch_to.frame(iframe)
To switch back to the main DOM:
driver.switch_to.default_content()
If the frame doesn't exist, then try inducing explicit wait as below:
element = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.ID, "username"))
Below imports you will require:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
Given the HTML:
the element is a <input> element which would accept text input using send_keys() method.
Solution
Ideally to send a character sequence to an <input> element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.input#username[name='username']"))).send_keys("Coding_89")
Using XPATH:
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//input[#class='input' and #id='username'][#name='username']"))).send_keys("Coding_89")
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

How to read alert info text from a div node with Python Selenium?

I am trying to read the webpage where after selecting the dropdown menu, the message is displayed. When the message appears it has class
<div class="alert alert-info border-0 rounded-0"> No update currently available </div>
I wrote the following code to read the text but always getting an exception
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
alert = WebDriverWait(driver, 5).until(EC.alert_is_present)
print(alert.text)
at the line with webdriver wait, I am getting the below error
__init__() takes 1 positional argument but 2 were given
My goal is to read the text in the alert class and validate it further. Any help will be appreciated.
Though the WebElement's class attribute contains the value alert still it isn't a JavaScript enabled Alert and you have to deal with it as a regular WebElement.
To print the text No update currently available you can use either of the following Locator Strategies:
Using css_selector and get_attribute("innerHTML"):
print(driver.find_element_by_css_selector("div.alert.alert-info").get_attribute("innerHTML"))
Using xpath and text attribute:
print(driver.find_element_by_xpath("//div[contains(#class, 'alert') and contains(#class, 'alert-info')]").text)
Ideally you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR and text attribute:
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.alert.alert-info"))).text)
Using XPATH and get_attribute():
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[contains(#class, 'alert') and contains(#class, 'alert-info')]"))).get_attribute("innerHTML"))
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python
References
Link to useful documentation:
get_attribute() method Gets the given attribute or property of the element.
text attribute returns The text of the element.
Difference between text and innerHTML using Selenium

How to locate the button element using Selenium through Python

I'm trying to find the element and click for the button "Not Now". I've tried with with css_selector, xpath, but I"m unable at all to find the proper way.
HTML:
To locate and click() on the element with text as Not Now you can use the following Locator Strategy:
Using xpath:
driver.find_element_by_xpath("//button[text()='Not Now']").click()
However, the element looks dynamic to me so you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategy:
Using XPATH:
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//div//button[text()='Not Now']"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Reference
You can find a couple of relevant discussions in:
What does contains(., 'some text') refers to within xpath used in Selenium
While fetching all links,Ignore logout link from the loop and continue navigation in selenium java
How does dot(.) in xpath to take multiple form in identifying an element and matching a text

selenium using python: how to correctly click() an element?

while learning how to use selenium, Im trying to click an element but nothing happens and Im unable to reach the next page. this is the relevant page: http://buyme.co.il and Im trying to click: הרשמה
I managed to print the desired element (הרשמה) so I guess Im reaching the correct place in the page. but 'click()' doesnt work.
the second span <span>הרשמה</span> is what i want to click:
<li data-ember-action="636">
<a>
<span class="seperator-link">כניסה</span>
<span>הרשמה</span>
</a>
</li>
for elem in driver.find_elements_by_xpath('//* [#id="ember591"]/div/ul[1]/li[3]/a/span[2]'):
print (elem.text)
elem.click()
also tried this:
driver.find_element_by_xpath('//*[#id="ember591"]/div/ul[1]/li[3]/a').click()
I expected to get to the "lightbox" which contain the registration fields.
Any thoughts on the best way to accomplish this?
Explicit Waits - An explicit wait is a code you define to wait for a certain condition to occur before proceeding further in the code.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
browser = webdriver.Chrome()
browser.get("https://buyme.co.il/")
WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.ID, 'ember591')))
elm = browser.find_elements_by_xpath('//div[#id="ember591"]/div/ul[1]/li[3]/a')
elm[0].click()
Update:
WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.CLASS_NAME, 'login')))
email = browser.find_elements_by_xpath("//form[#id='ember1005']/div[1]/label/input")
email[0].send_keys("abc#gmail.com")
password = browser.find_elements_by_xpath("//form[#id='ember1005']/div[2]/label/input")
password[0].send_keys("test1234567")
login = browser.find_elements_by_xpath('//form[#id="ember1005"]/button')
login[0].click()
The desired element is an Ember.js enabled element so to locate the element you have to induce WebDriverWait for the element to be clickable and you can use the following Locator Strategy:
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//span[text()='הרשמה']"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Python Selenium: can't figure out xpath to clickable link element in span tag

I'm using selenium python and have tried more than a dozen ways to find a clickable link element from a span tag but have been unsuccessful. I've tried using xpath, by link text, partial link text, and other suggestions researched.
The last 3 attempts using:
browser.find_element_by_link_text("Web Time Clock-Eastern").click()
element = browser.find_element_by_partial_link_text('Clock-Eastern')
browser.wait.until(EC.element_to_be_clickable((By.XPATH, '//a[#span]/html/body/div[6]/div[1]/div[1]/div[1]/div/div/div[4]/div[2]button'))).click()
I've provided image of the inspected element html below:
I expect to locate an element I can pass click method to open corresponding web page.
To click() the element you have to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following solutions:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div.cardLinks a[href*='php/timeclock/WEB'] > span"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[#class='cardLinks']//a[contains(#href, 'php/timeclock/WEB')]/span[text()='Web Time Clock-Eastern']"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Try the below xpath.
//span[normalize-space(.)='Web Time Clock-Eastern']/parent::a
One of the main reasons for using XPath is when you don’t have a suitable id or name attribute for the element you wish to locate.
browser.find_element_by_xpath("//div[#class='cardLinks']/div/a").click()

Categories