I am working on a program that automates logs into to a certain webpage and clicks certain buttons & links to reach a final destination to enter certain values and submit them. I have managed to navigate through the webpages but one of the webpages has a hyperlink button that I need Selenium to click, however, after trying multiple different methods, I cannot get it to work.
I have tried finding the element with By.XPATH, By.LINK_TEXT, By.PARTIAL_LINK_TEXT and none of these worked. I thought my issue might be that since it is clicking onto a totally new URL, so I load the new URL towards the bottom of my code to then move forward with my program.
The hyperlink button:
Button
The chunk of code to the hyperlink button I am trying to click on:
The XPath itself is : /html/body/div[2]/table/tbody/tr/td[2]/p/span/a[2]
driver = webdriver.Chrome(executable_path='C:\chromedriver.exe')
driver.get('')
'''
username_input = '//*[#id="userNameInput"]'
password_input = '//*[#id="passwordInput"]'
submit_button = '//*[#id="submitButton"]'
send_push = '//*[#id="auth_methods"]/fieldset/div[1]/button'
'''
# enters username and password into fields
driver.find_element("xpath", '//*[#id="userNameInput"]').click()
driver.find_element("xpath", '//*[#id="userNameInput"]').send_keys(username)
driver.find_element("xpath", '//*[#id="passwordInput"]').click()
driver.find_element("xpath", '//*[#id="passwordInput"]').send_keys(password)
driver.find_element("xpath", '//*[#id="submitButton"]').click()
# clicks 'send me a push' button on duo mobile screen
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[#id='duo_iframe']")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable(("xpath", "//button[normalize-space()='Send Me a Push']"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable(("xpath", '//*[#id="p_p_id_56_INSTANCE_xWhKj4tIFYvm_"]/div/div/div[1]/a[5]'))).click()
# loads next url which has the link on its webpage that needs to be clicked
driver.get('')
# attempts to click on link
driver.find_element("xpath", '/html/body/div[2]/table/tbody/tr/td[2]/p/span/a[2]').click()
I have removed the URLs in driver.get('') as they contain sensitive URLs
My last line of code is my attempt to click the hyperlink using the XPath
Any help is appreciated!
EDIT:
After further investigation, I now see that 2 buttons both have the same class name in their elements:
The SITE MAP button: SITE MAP
The Student button: Student
Considering the HTML:
The element with text as SITE MAP is a <a> tag and a dynamic element. To click on the clickable element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:
Using PARTIAL_LINK_TEXT:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, "SITE MAP"))).click()
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a.submenulinktext2"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[#class='submenulinktext2' and contains(., 'SITE MAP')]"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
I assume find_element does not return the correct element hence cannot click. Or you can verify it by changing .click() to .text. If it returns SITE MAP, then it returns a correct element.
I have another solution if you can try. It's using xpath and assumes class='pageheaderlink' is a single element in the HTML body.
list_subcontent = driver.find_elements_by_xpath(".//span[#class='pageheaderlink']")
for item in list_subcontent:
if item.text == 'SITE MAP':
item.click()
Related
I'm trying to access an input field of username (of login page) with Selenium. The page is JavaScript based.
driver.get() wait by default to load the complete page. In my case, it is unable to load that.
I can inspect the element on browser (firefox) and I get this.
<input type="text" autocomplete="username" name="username">
I tried to wait for that specific element with EC.presence_of_element_located.
Code trials:
driver = webdriver.Firefox()
driver.get(url)
delay = 10 # seconds
try:
myElem = WebDriverWait(driver, delay).until(EC.presence_of_element_located((By.NAME, 'username')))
print("Page is ready!")
except TimeoutException:
print("Loading took too much time!")
print(driver.page_source)
I get Loading took too much time!. Even though the element is there as I can inspect it in the browser. I also tried EC.presence_of_element_located((By.TAG_NAME, 'input'))) but It also can't find the tag.
Update from the comments: url='https://drive.inditex.com/drfrcomr/login'
To interact a clickable element instead of presence_of_element_located() you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:
Using NAME:
myElem = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.NAME, "username")))
Using CSS_SELECTOR:
myElem = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input[name='username']")))
Using XPATH:
myElem = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#name='username']")))
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
However on accessing the website I find the username field implemented through Windows Authentication
Solution
Incase of Basic Authentication you need to embed the username and password within the url as follows:
driver.get("http://username:password#drive.inditex.com/drfrcomr/login")
Update
Additionally the username field is within nested #shadow-root (open)
To send a character sequence to the username field you have to use shadowRoot.querySelector() and you can use the following Locator Strategy:
Code Block:
driver.get("http://username:password#drive.inditex.com/drfrcomr/login")
time.sleep(10)
item = driver.execute_script('''return document.querySelector('glyph-splash.hydrated').shadowRoot.querySelector('glyph-login-form.hydrated').shadowRoot.querySelector('input[name="username"]')''')
item.send_keys("ShaidaMuhammad")
You can do as stated here, so your code would looks like
url='https://user:password#drive.inditex.com/drfrcomr/login'
driver.get(url)
I tried with Chrome and it worked for me!
If you can get around that popup you may try:
myElem = WebDriverWait(driver, delay).until(EC.presence_of_element_located((By.XPATH, "//*[.='Username']")))
I am currently working on a program that logs into an account for me and navigates through some webpages
My current issue is that on one of the webpages, I cannot click ANY hyperlink buttons to get to the next page I need to get to. I have tried finding the element waiting to see if it is clickable and using:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.LINK_TEXT, ""))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, ""))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CLASS_NAME, ""))).click()
and
driver.find_element(By.XPATH, "").click()
(Along with the other methods)
However, I have had no success when I invoke a .click() after each of those.
After further investigation, I saw that the website uses the same class name for 2 of the elements. I am not sure if this plays any role into not being able to click on the one I need to.
Element which I am trying to click:
Student
The XPATH: /html/body/div[4]/table[1]/tbody/tr[2]/td[2]/a
Element with the same class name as the one above:
SITE MAP
The XPATH: /html/body/div[2]/table/tbody/tr/td[2]/p/span/a[2]
After seeing them both having the same class name, I tried:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a.submenulinktext2"))).click()
A thought I was that since the automation is going from one website URL and navigates to another page with a new URL to begin more navigation, that it did not know what URL we were on so I tried:
driver.get("NEW URL")
And with that in place there has been no success so far, unfortunately. Any guidance on how I can go about this would be appreciated!
Considering the HTML of the element:
SITE MAP
The element is a <a> tag with innertext as Student.
Solution
However the desired element is a dynamic element. So to click on the clickable element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:
Using LINK_TEXT:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.LINK_TEXT, "Student"))).click()
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[onmouseover*='Student'][onfocus*='Student']"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[contains(#onmouseover, 'Student') and contains(#onfocus, 'Student')][text()='Student']"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
I'm building a scraper with selenium, and I got it up and running except for one data field I'm missing. I need to first push a button, then get the value. But I haven't been able to push the button, because I can't seem to build the correct xpath expression to get it with selenium,
The url is this one: https://www.ikea.com/mx/es/p/kallax-estante-blanco-80275887/
And I need to click the button "Medidas" (scroll halfway through the page, right between "Detalles del producto" and "Productos similares") to open a side panel and get the info I need.
But so far, I haven't been able to set the correct XPATH expression for the button.
I tried with
driver.find_element_by_xpath(
"//button[#class='range-revamp-chunky-header']"
).click()
But this way it clicked the first button ("Detalles del producto")
Also tried with something like
driver.find_element_by_xpath(
"//button[#class='range-revamp-chunky-header' AND #text='Medidas']"
).click()
But I haven't been able to make it work, I just got this error:
Message: no such element: Unable to locate element: {"method":"xpath","selector":"//button[#class='range-revamp-chunky-header' and #text='Medidas']"}
By the way, the list of buttons is dynamic... sometimes, there can be 1, 2, 3 or 4 buttons with the same class, only difference I can see is the text of the button, so getting always the 2nd button with that class (like in the URL provided) won't always work.
To click on the element with text as Medidas you can use the following Locator Strategy:
Using xpath:
driver.find_element(By.XPATH, "//span[text()='Medidas']").click()
Ideally you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategy:
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//*[contains(., 'login or register')]"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
I've been trying to scrape this website Link I'm interested in clicking the first DONWLOAD button. However whenever I try to find any element that is a button, I can't find any.
Here is the code :
url = 'https://ember-climate.org/data/carbon-price-viewer/'
webdriver = create_driver()
with webdriver as driver:
driver.maximize_window()
driver.get(url)
wait = WebDriverWait(driver, 30)
try:
wait.until(EC.element_to_be_clickable((By.XPATH, '//button')))
except:
pass
ids = driver.find_elements_by_xpath('//button')
for ii in ids:
print (ii.tag_name, ii.get_attribute('class'))
Is there anything wrong with the XPath or is it an issue with the website itself?
Your xpath is wrong, Download button is wrapped inside an span tag not button tag.
try this instead :
//span[contains(text(),'DOWNLOAD')]
also, I see it's in iframe, which can be located via
iframe[name='ETS']
CSS_SELECTOR, and we need to switch also to this iframe.
so in sequence the explanation would be :
You would have to click on cookies button.
Download button is in an iframe, we need to switch to iframe first and then we can interact with download button.
download button is a part of span tag not button tag.
Use Explicit waits.
Prefer id, css over xpath. (if they are unique in nature)
Launch browser in full screen mode.
Code :
driver = webdriver.Chrome(driver_path)
driver.maximize_window()
#driver.implicitly_wait(30)
wait = WebDriverWait(driver, 50)
driver.get("https://ember-climate.org/data/carbon-price-viewer/")
wait.until(EC.element_to_be_clickable((By.ID, "cn-accept-cookie"))).click()
wait.until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR, "iframe[name='ETS']")))
wait.until(EC.element_to_be_clickable((By.XPATH, "//span[contains(text(),'DOWNLOAD')]"))).click()
Imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
My Python Selenium can't click target element, instead seem click element behind target element?
I try to click, or enter text in 'drop-down meun', but I find that I am result in clicking element behind this 'drop-down. I know this is element behind it, because there is advistering area behind, and the result top-up showing the same advistering material. Here is my code:
# info for login
my_email = 'my_email'
my_passcode = 'my_passcode'
email_url = r'https://www.gmx.com/#.1559516-header-navlogin2-1'
# start driver and open url
driver = webdriver.Chrome(chrome_path)
driver.get(email_url)
# input email account
xpath = r'//*[#id="login-email"]'
WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, xpath)))
target = driver.find_element_by_xpath(xpath)
actions = ActionChains(driver)
actions.move_to_element(target).perform()
actions.click().send_keys(my_email).perform()
# input passcode and hit 'enter' to login
xpath = r'//input[#id="login-password"]'
WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, xpath)))
target = driver.find_element_by_xpath(xpath)
actions = ActionChains(driver)
actions.move_to_element(target).perform()
actions.click().send_keys(my_passcode).send_keys(Keys.ENTER).perform()
It happens to me for some other site when the site appear to have 'two layers' (not sure if I am using the right word). I can process anything on top layer, and only result in activate anything behind it. Many thank when provide solution!!
You don't need to use this ActionChains class, all you need to do can be done using WebElement.send_keys() and WebElement.click()
You don't need to re-find the element after using WebDriverWait as it returns the WebElement in case of success
I fail to see clickign on login button anywhere in your script, you can locate the relevant button using XPath contains() function like:
xpath = r'//button[contains(#class,"login-submit")]'
and then just call click() function on the resulting variable
Example suggested code:
You don't need to use this ActionChains class, all you need to do can be done using WebElement.send_keys() and WebElement.click()
You don't need to re-find the element after using WebDriverWait as it returns the WebElement in case of success
I fail to see clickign on login button anywhere in your script, you can locate the relevant button using XPath contains() function like:
xpath = r'//button[contains(#class,"login-submit")]'
and then just call click() function on the resulting variable
Example suggested code:
# input email account
xpath = r'//*[#id="login-email"]'
target = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, xpath)))
target.send_keys(my_email)
# input passcode and hit 'enter' to login
xpath = r'//input[#id="login-password"]'
target = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, xpath)))
target.send_keys(my_passcode)
xpath = r'//button[contains(#class,"login-submit")]'
target = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, xpath)))
target.click()