How to scroll through an element using Selenium?
It's been a while since I've used selenium, but doing something like this should scroll until the desired element is in view using JavaScript.
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("your-site.com")
# Find element by ID or some other method
element = driver.find_element_by_id("id_of_element")
# Run JavaScript to scroll until the element is in view
driver.execute_script("arguments[0].scrollIntoView(true);", element);
Related
how to scroll a certain page element, not the page itself, but the element, there is a list that is updated dynamically and to get all its elements I need to scroll it to the end
You can use
org.openqa.selenium.interactions.Actions
from selenium.webdriver.common.action_chains import ActionChains
element = driver.find_element_by_id("my-id")
actions = ActionChains(driver)
actions.move_to_element(element).perform()
OR
driver.execute_script("arguments[0].scrollIntoView();", element)
If you wan to scroll to the end of the page then the easiest way is to select a label and then send:
label.sendKeys(Keys.PAGE_DOWN);
OR
label.send_keys(Keys.END)
Hope this answer
I am trying to get search results from yahoo search using python - selenium and bs4. I have been able to get the links successfuly but I am not able to click the button at the bottom to go to the next page. I tried one way, but it could't identify after the second page.
Here is the link:
https://in.search.yahoo.com/search;_ylt=AwrwSY6ratRgKEcA0Bm6HAx.;_ylc=X1MDMjExNDcyMzAwMgRfcgMyBGZyAwRmcjIDc2ItdG9wLXNlYXJjaARncHJpZANidkhMeWFsMlJuLnZFX1ZVRk15LlBBBG5fcnNsdAMwBG5fc3VnZwMxMARvcmlnaW4DaW4uc2VhcmNoLnlhaG9vLmNvbQRwb3MDMARwcXN0cgMEcHFzdHJsAzAEcXN0cmwDMTQEcXVlcnkDc3RhY2slMjBvdmVyZmxvdwR0X3N0bXADMTYyNDUzMzY3OA--?p=stack+overflow&fr=sfp&iscqry=&fr2=sb-top-search
This is what im doing to get data from page but need to put in a loop which changes pages:
page = BeautifulSoup(driver.page_source, 'lxml')
lnks = page.find('div', {'id': 'web'}).find_all('a', href = True)
for i in lnks:
print(i['href'])
You don't need to scroll down to the bottom. The next button is accessible without scrolling. Suppose you want to navigate 10 pages. The python script can be like this:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
driver=webdriver.Chrome()
driver.get('Yahoo Search URL')
# Let's create a loop containing the XPath for next button
# As well as waiting for the next button to be clickable.
for i in range(10):
WebDriverWait(driver, 5).until(EC.element_to_be_clickable(By.XPATH, '//a[#class="next"]'))
navigate = driver.find_element_by_xpath('//a[#class="next"]').click()
The next page button is on the bottom of the page so you first need to scroll to that element and then click it. Like this:
from selenium.webdriver.common.action_chains import ActionChains
actions = ActionChains(driver)
next_page_btn = driver.find_element_by_css_selector("a.next")
actions.move_to_element(next_page_btn).build().perform()
time.sleep(0.5)
next_page_btn.click()
I am trying to click on the "Next page" in Python-Selenium. The element and its path are seen, the buttom is being clicked but after clicking an error is shown:
"StaleElementReferenceException:stale element reference: element is not attached to the page document"
My code so far:
element = WebDriverWait(driver, 20).until(EC.presence_of_element_located(\
(By.XPATH, butn)))
print (element.is_enabled())
while True and element.is_enabled()==True:
driver.find_element_by_xpath(butn).click()
The error is one element.is_enabled()==True after clicking
Can someone help?
When you search elements in Selenium then it doesn't keep full objects but only references to objects in DOM in browser. And when you click then browser creates new DOM and old references are incorrect and you have to find elements again.
Something like this
# find first time
element = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.XPATH, butn)))
print(element.is_enabled())
while element.is_enabled():
driver.find_element_by_xpath(butn).click()
# find again after changes in DOM
element = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.XPATH, butn)))
print(element.is_enabled())
How can I scroll down and up inside a div using Selenium? I looked everywhere on the internet. Only solutions for pages.
element = driver.find_elements_by_xpath('//*[#id="root"]/div/main/div/div[2]/div[1]/div[1]/div/div[2]/nav/div[4]/div/div[2]/div/span')
element.execute_script("arguments[0].scrollIntoView();", element )
The actions class is capable of scrolling.
This import:
from selenium.webdriver.common.action_chains import ActionChains
This function:
def ScrollIntoView(element):
actions = ActionChains(driver)
actions.move_to_element(element).perform()
Assuming your element exists and is ready on the page you can Call it as such:
element = driver.find_elements_by_xpath('//*[#id="root"]/div/main/div/div[2]/div[1]/div[1]/div/div[2]/nav/div[4]/div/div[2]/div/span')
ScrollIntoView(element)
I'm learning how to use selenium and I'm stuck on figuring out how to scroll down in a website to verify an element exists.
I tried using the methods that was found in this question
Scrolling to element using webdriver?
but selenium won't scroll down the page. Instead it'll give me an error
"selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: element"
Heres the codes I am using
moveToElement:
element = driver.find_element_by_xpath('xpath')
actions = ActionChains(driver)
actions.move_to_element(element).perform()
Scrolling into View
element = driver.find_element_by_xpath('xpath')
driver.execute_script("arguments[1].scrollIntoView();", element)
The whole code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.action_chains import ActionChains
driver = webdriver.Firefox()
driver.get("https://www.linkedin.com/")
element =
driver.find_element_by_xpath('/html/body/div/main/div/div[1]/div/h1/img')
element = driver.find_element_by_xpath('//*[#id="login-email"]')
element.send_keys('')
element = driver.find_element_by_xpath('//*[#id="login-password"]')
element.send_keys('')
element = driver.find_element_by_xpath('//*[#id="login-submit"]')
element.click();
element = driver.find_element_by_xpath('')
actions = ActionChains(driver)
actions.move_to_element(element).perform()
There are two aspects to your is problem
Whether the element exist?
Whether the element displayed on the page?
Whether the element exist?
It may happen that the element exists on the page[i.e. it is part of the DOM] but it is not available right away for further selenium action becuase it is not visible[hidden], it's only gets visible after some actions or it may get displayed on scroll down the page.
Your code is throwing an exception here-
element = driver.find_element_by_xpath('xpath')
as WebDiver not able to find the element using mentioned xpath. Once you fix this, you can moe forward the next part.
Whether the element displayed on the page?
Once you fix the above issue, you should check whether the element is being displayed or not. If it's not displayed and avialble on the scroll then you can use code like
if !element.is_displayed():
driver.execute_script("arguments[1].scrollIntoView();", element)
Perfer using Action Class for very specific mouse action.
Update:-
If you are application using lazy loading and the element you are trying to find is available on scroll the you can try something like this -
You have to import exception like -
from selenium.common.exceptions import NoSuchElementException
and the create new recursion function which would scroll if element not found, something like this -
def search_element():
try:
elem = driver.find_element_by_xpath("your")
return elem
except NosSuchElementException:
driver.execute_script("window.scrollTo(0,Math.max(document.documentElement.scrollHeight,document.body.scrollHeight,document.documentElement.clientHeight));")
search_element()
I am not sure that having recurion for finding element here is good idea, also I have naver worked on the python so you need to lookout for the syntax
Amm may be this might help.Just send page down key and if you are sure that element exists definitely then this would work
from selenium.webdriver.common.keys import Keys
from selenium.webdriver import ActionChains
import time
while True:
ActionChains(driver).send_keys(Keys.PAGE_DOWN).perform()
time.sleep(2) #2 seconds
try:
element = driver.find_element_by_xpath("your_element_xpath")
break
except:
continue