Trying to scrape a website, I created a loop and was able to locate all the elements. My problem is, that the next button id changes on every page. So I can not use the id as a locator.
This is the next button on page 1:
<a rel="nofollow" id="f_c7" href="#" class="nextLink jasty-link"></a>
And this is the next button on page 2:
<a rel="nofollow" id="f_c9" href="#" class="nextLink jasty-link"></a>
Idea:
next_button = browser.find_elements_by_class_name("nextLink jasty-link")
next_button.click
I get this error message:
Message: no such element: Unable to locate element
The problem here might be that there are two next buttons on the page.
So I tried to create a list but the list is empty.
next_buttons = browser.find_elements_by_class_name("nextLink jasty-link")
print(next_buttons)
Any idea on how to solve my problem? Would really appreciate it.
This is the website:
https://fazarchiv.faz.net/faz-portal/faz-archiv?q=Kryptow%C3%A4hrungen&source=&max=10&sort=&offset=0&_ts=1657629187558#hitlist
There are two issues in my opinion:
Depending from where you try to access the site there is a cookie banner that will get the click, so you may have to accept it first:
browser.find_element_by_class_name('cb-enable').click()
To locate a single element, one of the both next buttons, it doeas not matter, use browser.find_element() instead of browser.find_elements().
Selecting your element by multiple class names use xpath:
next_button = browser.find_element(By.XPATH, '//a[contains(#class, "nextLink jasty-link")]')
or css selectors:
next_button = browser.find_element(By.CSS_SELECTOR, '.nextLink.jasty-link')
Note: To avoid DeprecationWarning: find_element_by_* commands are deprecated. Please use find_element() import in addition from selenium.webdriver.common.by import By
You can't get elements by multiple class names. So, you can use find_elements_by_css_selector instead.
next_buttons = browser.find_elements_by_css_selector(".nextLink.jasty-link")
print(next_buttons)
You can then loop through the list and click the buttons:
next_buttons = browser.find_elements_by_css_selector(".nextLink.jasty-link")
for button in next_buttons:
button.click()
Try below xPath
//a[contains(#class, 'step jasty-link')]/following-sibling::a
Related
I am trying to locate the "Confirm Booking" button in selenium. My driver is on the current webpage and we can see in the html that the button is of class "menubutton". Yet when I iterate through all elements of that class it only locates the "Return to timetable" button. As you can see in the terminal output.
My code is as follows:
#confirms booking
elems = driver.find_elements(By.CLASS_NAME, 'menubutton')
print(len(elems))
for e in elems:
print(e.text)
I have also tried locating by XPATH but it only finds the "Return to Class Timetable" button.
Any help would be appreciated!
Situation
I'm using Selenium and Python to extract info from a page
Here is the div I want to extract from:
I want to extract the "Registre-se" and the "Login" text.
My code
from selenium import webdriver
url = 'https://www.bet365.com/#/AVR/B146/R^1'
driver = webdriver.Chrome()
driver.get(url.format(q=''))
elements = driver.find_elements_by_class_name('hm-MainHeaderRHSLoggedOutNarrow_Join ')
for e in elements:
print(e.text)
elements = driver.find_elements_by_class_name('hm-MainHeaderRHSLoggedOutNarrow_Login ')
for e in elements:
print(e.text)
Problem
My code don't send any output.
HTML
<div class="hm-MainHeaderRHSLoggedOutNarrow_Join ">Registre-se</div>
<div class="hm-MainHeaderRHSLoggedOutNarrow_Login " style="">Login</div>
By looking this HTML
<div class="hm-MainHeaderRHSLoggedOutNarrow_Join ">Registre-se</div>
<div class="hm-MainHeaderRHSLoggedOutNarrow_Login " style="">Login</div>
and your code, which looks okay to me, except that part you are using find_elements for a single web element.
and by reading this comment
The class name "hm-MainHeaderRHSLoggedOutMed_Login " only appear in
the inspect of the website, but not in the page source. What it's
supposed to do now?
It is clear that the element is in either iframe or shadow root.
Cause page_source does not look for iframe.
Please check if it is in iframe, then you'd have to switch to iframe first and then you can use the code that you have.
switch it like this :
driver.switch_to.frame(driver.find_element_by_xpath('xpath here'))
On this website, I'm trying to find an element based on its XPATH, but the XPATH keeps changing. What's the next best alternative?
Snippet from website
<button class="KnkXXg vHcWfw T1alpA kiPMng AvEAGQ vM2UTA DM1_6g _-kwXsw Mqe1NA SDIrVw edrpZg" type="button" aria-expanded="true"><span class="nW7nAQ"><div class="VpIG5Q"></div></span></button>
XPATH:
//*[#id="__id15"]/div/div/div[1]/div[2]/div
#Sometimes id is a number between 15-18
//*[#id="__id23"]/div/div/div[1]/div[2]/div
#Sometimes id is a number between 13-23
Here's how I use the code:
element = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, """//*[#id="__id3"]/div/div/div[1]/div[2]/div/div/div/button"""))).click()
I've tried clicking the element by finding the button class, but for whatever reason it won't do anything.
element = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CLASS_NAME, "KnkXXg vHcWfw T1alpA kiPMng AvEAGQ vM2UTA DM1_6g _-kwXsw Mqe1NA SDIrVw edrpZg"))).click()
If Part of the text is keep changing you can use contains in the xpath.
//*[contains(#id,"__id"]/div/div/div[1]/div[2]/div
I have these following elements in a page and I need to select the element with the value 18 inside using Python and selenium script. That is the second link. Here is the HTML code of the page
<a class="ui-state-default ui-state-highlight ui-state-active" href="#">17</a>
<a class="ui-state-default ui-state-highlight ui-state-active" href="#">18</a>
<a class="ui-state-default ui-state-highlight ui-state-active" href="#">19</a>
I am trying to use the following Python and Selenium code to click
elem = driver.find_element_by_xpath('//a[#class="ui-state-default"]').click()
But that does not work. How do I fix it?
elem = driver.find_element_by_link_text("18")
elem.click()
That will find the first element matching that XPath, which is the one with the value 17. To select the element whose value is 18, try this:
driver.find_element_by_link_text('18').click()
PS: You don't need to set the click event to a variable. Only do this if you need to multiple actions on the element (send_keys(), click(), text, etc.)
I am parsing a web page with an organization like this:
<nav class="sidebar-main">
<div class="sidebar">Found 3 targets</div>
<ul><li><span>target1</span></li>
<li><a href="#target2" ><span>target2</span></a></li>
<li><span>target3</span></li></ul>
</nav>
My goal is to loop through each list element, clicking each one in the process:
sidebar = browser.find_element_by_class_name('sidebar-main')
elementList = sidebar.find_elements_by_tag_name("li")
for sample in elementList:
browser.implicitly_wait(5)
run_test1 = WebDriverWait(browser, 5).until(
EC.presence_of_element_located((By.CLASS_NAME, 'sidebar-main'))
)
sample.click()
I keep getting the error:
Message: The element reference of <li> stale either the element is no
longer attached to the DOM or the page has been refreshed.
Right now only one link is clicked, obviously selenium cannot locate subsequent elements upon page refresh, how do I get around this?
Once you click on the first link, either navigation to new page happens or the page is refreshed. You need to keep track of the element list, find the list elements again and then click on the required element. If page is changed, then you need to navigate back to the original page as well.
You can try something like below
sidebar = browser.find_element_by_class_name('sidebar-main')
elementList = sidebar.find_elements_by_tag_name("li")
for i in range(len(elementList)):
element = browser.find_element_by_class_name('sidebar-main').find_elements_by_tag_name("li")[i]
element.click()