Unable to click on element even after finding it- Python seleneium - python

I have been pained by a load more button for a while. I'm looking to create a loop where I click "load more" on the skills section of Linkedin pages. However, this button is just not consistently being clicked.
I was under the impression that the issue was that the element was not visible on the page. So, I have a segmented scroll, which continues moving down the page until the element is found. But what's baffling is that even though the page is now moving to the right place, the element is not being clicked. No error is being thrown.
I've tried nearly every version of the element location (xpath, class name, css selector, full xpath). Why would the button not be clicked, if it is visible on the page?
Relevant Code:
##log into Linkedin
linkedin_urls=['https://www.linkedin.com/in/julie-migliacci-revent/']
ChromeOptions = webdriver.ChromeOptions()
driver = webdriver.Chrome('C:\\Users\\Root\\Downloads\\chromedriver.exe')
driver.get('https://www.linkedin.com/login?fromSignIn=true&trk=guest_homepage-basic_nav-header-signin')
driver.maximize_window()
WebDriverWait(driver, 10).until(
EC.element_to_be_clickable((By.NAME, "session_key"))).send_keys("EMAIL")
WebDriverWait(driver, 10).until(
EC.element_to_be_clickable((By.NAME, "session_password"))).send_keys("PASSWORD")
WebDriverWait(driver, 10).until(
EC.element_to_be_clickable((By.XPATH, "//button[#class='btn__primary--large from__button--floating']"))).click()
linkedin_urls=['https://www.linkedin.com/in/julie-migliacci-revent/', 'https://www.linkedin.com/in/kathleen-meyers-126a7931']
for linkedin_url in linkedin_urls:
driver.get(linkedin_url)
looking_for_element = True
ldmore = ''
while looking_for_element:
elements = driver.find_elements_by_xpath('/html/body/div[7]/div[3]/div/div/div/div/div[2]/main/div[2]/div[6]/div/section/div[2]/button/span[1]')
if len(elements) > 0:
ldmore = elements[0]
ldmore.click()
looking_for_element = False
else:
global_copyright = driver.find_elements_by_css_selector('#globalfooter-copyright')
if len(global_copyright) > 0:
looking_for_element = False
else:
body = driver.find_element_by_css_selector('body')
sleep(5)
body.send_keys(Keys.PAGE_DOWN)
I've not seen a discussion on SO about element issues when the underlying solution is not visibility. The code is designed to stop once the element is located -- and is performing this correctly. But it just isn't clicking on the element. I'm not sure why this is.
Locations I've tried:
absolute xpath:
driver.find_element_by_xpath('/html/body/div[7]/div[3]/div/div/div/div/div[2]/main/div[2]/div[6]/div/section/div[2]/button/span[1]').click()
relative xpath:
//span[contains(text(),'Show more')]
class name:
pv-profile-section__card-action-bar pv-skills-section__additional-skills artdeco-container-card-action-bar artdeco-button artdeco-button--tertiary artdeco-button--3 artdeco-button--fluid" aria-controls="skill-categories-expanded
css:
body.render-mode-BIGPIPE.nav-v2.theme.theme--classic.ember-application.boot-complete.icons-loaded:nth-child(2) div.application-outlet:nth-child(77) div.authentication-outlet:nth-child(3) div.extended div.body div.pv-profile-wrapper.pv-profile-wrapper--below-nav div.self-focused.ember-view div.pv-content.profile-view-grid.neptune-grid.two-column.ghost-animate-in main.core-rail div.profile-detail div.pv-deferred-area.ember-view:nth-child(6) div.pv-deferred-area__content section.pv-profile-section.pv-skill-categories-section.artdeco-container-card.ember-view div.ember-view > button.pv-profile-section__card-action-bar.pv-skills-section__additional-skills.artdeco-container-card-action-bar.artdeco-button.artdeco-button--tertiary.artdeco-button--3.artdeco-button--fluid
UPDATE:
Tried the JS force, it clicked! But threw up the error: selenium.common.exceptions.WebDriverException: Message: unknown error: Cannot read property 'click' of null
if len(elements) > 0:
ldmore = elements[0]
ldmorebtn = driver.find_element_by_xpath('/html/body/div[7]/div[3]/div/div/div/div/div[2]/main/div[2]/div[6]/div/section/div[2]/button/span[1]').click()
#driver.execute_script("arguments[0].checked = true;", ldmore)
driver.execute_script("arguments[0].click();", ldmore)

The suggestion by #Datanovice to use javascript to force a click worked like a charm. Initially, when I had tried to adapt the solution, I received the error selenium.common.exceptions.WebDriverException: Message: unknown error: Cannot read property 'click' of null.
This error was because I had been using EC.element_to_be_clickable. Instead, when I paired the java method with EC.visibility_of_element_located, the click consistently worked.
The code:
ldmore = WebDriverWait(driver, 30).until(EC.visibility_of_element_located((By.XPATH,'xpath')))
driver.execute_script("arguments[0].click();", ldmore)

Related

Element is found but not clickable

I'm trying to find an element by it's id, click on it and download a file.
driver.get(url);
driver.implicitly_wait(60);
time.sleep(3)
element = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.ID, "ContentPlaceHolder1_a1")))
href = element.get_attribute('href')
value = href.split('/')[-1]
print(value);
element.click(); # Error
Error
element click intercepted: Element is not clickable at point (110, 1003)
I've tried Xpath, and CSS path too. All give the same error. If I check for the visibility then it times out. But I can manually see that the element is visible
element = WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH, "//a[contains(text(), 'text of the link')]")))
At last, I tried this code.
element = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.ID, "ContentPlaceHolder1_a1")))
ActionChains(driver).move_to_element(element).click().perform()
But it gives error
selenium.common.exceptions.MoveTargetOutOfBoundsException: Message: move target out of bounds
While identified the element, page is still loading, may be the reason it is clicking on somewhere else.
Scroll down to the element and then wait for 1 to 2 seconds and then click.
It is working perfectly for me.
element = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.ID, "ContentPlaceHolder1_a1")))
element.location_once_scrolled_into_view #scroll down to element
time.sleep(2) # wait for 2 seconds , 1 sec working as well
element.click()

Selenium EC.element_to_be_clickable still returns stale element error

I am not sure what is wrong with this. Am I using EC.element_to_be_clickable() right? I am getting the error message: "selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document". I am pretty sure the XPATH is valid and have even tried with another that designates the same element.
My code:
driver.get("https://browzine.com/libraries/1374/subjects")
parent_url = "https://browzine.com/libraries/1374/subjects"
wait = WebDriverWait(driver, 10)
subjects_avail = driver.find_elements(By.XPATH, "//span[#class='subjects-list-subject-name']")
subjects = 0
for sub in subjects_avail:
WebDriverWait(driver, 5).until(EC.element_to_be_clickable(
(By.XPATH, "//span[#class='subjects-list-subject-name']")))
ActionChains(driver).move_to_element(sub).click(sub).perform()
subjects = +1
driver.get(parent_url)
Each time you clicking the sub element the driver is navigating to the new page.
Then you are opening the main page again with
driver.get(parent_url)
But all the web elements in subjects_avail list became Stale since driver already left the original main page.
To make your code working you have to get the subjects_avail list again each time you getting back to the main page and then select the correct sub title element.
Something like this:
url = "https://browzine.com/libraries/1374/subjects"
subj_list_xpath = "//span[#class='subjects-list-subject-name']"
driver.get(url)
wait = WebDriverWait(driver, 10)
subjects_avail = driver.find_elements(By.XPATH, subj_list_xpath)
for i in range(len(subjects_avail)):
WebDriverWait(driver, 5).until(EC.visibility_of_element_located((By.XPATH, subj_list_xpath)))
subjects_avail = driver.find_elements(By.XPATH, subj_list_xpath)
ActionChains(driver).move_to_element(subjects_avail[i]).click(subjects_avail[i]).perform()
driver.get(url)

PYTHON - StaleElementReferenceException for enabled element

I am trying to click on the "Next page" in Python-Selenium. The element and its path are seen, the buttom is being clicked but after clicking an error is shown:
"StaleElementReferenceException:stale element reference: element is not attached to the page document"
My code so far:
element = WebDriverWait(driver, 20).until(EC.presence_of_element_located(\
(By.XPATH, butn)))
print (element.is_enabled())
while True and element.is_enabled()==True:
driver.find_element_by_xpath(butn).click()
The error is one element.is_enabled()==True after clicking
Can someone help?
When you search elements in Selenium then it doesn't keep full objects but only references to objects in DOM in browser. And when you click then browser creates new DOM and old references are incorrect and you have to find elements again.
Something like this
# find first time
element = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.XPATH, butn)))
print(element.is_enabled())
while element.is_enabled():
driver.find_element_by_xpath(butn).click()
# find again after changes in DOM
element = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.XPATH, butn)))
print(element.is_enabled())

How to Fix Python Selenium Can't Click Target Element?

My Python Selenium can't click target element, instead seem click element behind target element?
I try to click, or enter text in 'drop-down meun', but I find that I am result in clicking element behind this 'drop-down. I know this is element behind it, because there is advistering area behind, and the result top-up showing the same advistering material. Here is my code:
# info for login
my_email = 'my_email'
my_passcode = 'my_passcode'
email_url = r'https://www.gmx.com/#.1559516-header-navlogin2-1'
# start driver and open url
driver = webdriver.Chrome(chrome_path)
driver.get(email_url)
# input email account
xpath = r'//*[#id="login-email"]'
WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, xpath)))
target = driver.find_element_by_xpath(xpath)
actions = ActionChains(driver)
actions.move_to_element(target).perform()
actions.click().send_keys(my_email).perform()
# input passcode and hit 'enter' to login
xpath = r'//input[#id="login-password"]'
WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, xpath)))
target = driver.find_element_by_xpath(xpath)
actions = ActionChains(driver)
actions.move_to_element(target).perform()
actions.click().send_keys(my_passcode).send_keys(Keys.ENTER).perform()
It happens to me for some other site when the site appear to have 'two layers' (not sure if I am using the right word). I can process anything on top layer, and only result in activate anything behind it. Many thank when provide solution!!
You don't need to use this ActionChains class, all you need to do can be done using WebElement.send_keys() and WebElement.click()
You don't need to re-find the element after using WebDriverWait as it returns the WebElement in case of success
I fail to see clickign on login button anywhere in your script, you can locate the relevant button using XPath contains() function like:
xpath = r'//button[contains(#class,"login-submit")]'
and then just call click() function on the resulting variable
Example suggested code:
You don't need to use this ActionChains class, all you need to do can be done using WebElement.send_keys() and WebElement.click()
You don't need to re-find the element after using WebDriverWait as it returns the WebElement in case of success
I fail to see clickign on login button anywhere in your script, you can locate the relevant button using XPath contains() function like:
xpath = r'//button[contains(#class,"login-submit")]'
and then just call click() function on the resulting variable
Example suggested code:
# input email account
xpath = r'//*[#id="login-email"]'
target = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, xpath)))
target.send_keys(my_email)
# input passcode and hit 'enter' to login
xpath = r'//input[#id="login-password"]'
target = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, xpath)))
target.send_keys(my_passcode)
xpath = r'//button[contains(#class,"login-submit")]'
target = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, xpath)))
target.click()

Selenium explicit wait doesnt work

I have a code that tell Selenium to wait until an element is clickable but for some reason, Selenium doesnt wait but instead, click that element and raise a Not clickable at point (x, y) immediately. Any idea how to fix this ?
x = '//*[#id="arrow-r"]/i'
driver = webdriver.Chrome(path)
driver.get('https://www.inc.com/inc5000/list/2017')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, x)))
driver.find_element_by_xpath(x).click()
EC.element_to_be_clickable check if element is visible and enabled. In terms of visibility it doesn't cover scenario when element is behind other. Maybe your page use something like blockUI widget and click() occurs before the cover disappears. You can check if element is truly clickable by enriching EC.element_to_be_clickable((By.XPATH, x)) check with assertion that ensure element is not behind by other. In my projects I use implementation as below:
static bool IsElementClickable(this RemoteWebDriver driver, IWebElement element)
{
return (bool)driver.ExecuteScript(#"
(function(element){
var rec = element.getBoundingClientRect();
var elementAtPosition = document.elementFromPoint(rec.left+rec.width/2, rec.top+rec.height/2);
return element == elementAtPosition || element.contains(elementAtPosition);
})(arguments[0]);
", element);
}
This code is in C# but I'm sure you can easily translate into your programming language of choice.
UPDATE:
I wrote a blog post about problems related to clicking with selenium framework https://cezarypiatek.github.io/post/why-click-with-selenium-so-hard/
Here is a link to the 'waiting' section for the Python Selenium docs: Click here
Wait will be like:
element = WebDriverWait(driver, 10).until(
EC.visibility_of((By.XPATH, "Your xpath"))
)
element.click();

Categories