We Keep Coding

How to scroll element Selenium Python

How to scroll through an element using Selenium?

It's been a while since I've used selenium, but doing something like this should scroll until the desired element is in view using JavaScript.
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("your-site.com")
# Find element by ID or some other method
element = driver.find_element_by_id("id_of_element")
# Run JavaScript to scroll until the element is in view
driver.execute_script("arguments[0].scrollIntoView(true);", element);

Unable to identify what to 'click' for next page using selenium

I am trying to get search results from yahoo search using python - selenium and bs4. I have been able to get the links successfuly but I am not able to click the button at the bottom to go to the next page. I tried one way, but it could't identify after the second page.
Here is the link:
https://in.search.yahoo.com/search;_ylt=AwrwSY6ratRgKEcA0Bm6HAx.;_ylc=X1MDMjExNDcyMzAwMgRfcgMyBGZyAwRmcjIDc2ItdG9wLXNlYXJjaARncHJpZANidkhMeWFsMlJuLnZFX1ZVRk15LlBBBG5fcnNsdAMwBG5fc3VnZwMxMARvcmlnaW4DaW4uc2VhcmNoLnlhaG9vLmNvbQRwb3MDMARwcXN0cgMEcHFzdHJsAzAEcXN0cmwDMTQEcXVlcnkDc3RhY2slMjBvdmVyZmxvdwR0X3N0bXADMTYyNDUzMzY3OA--?p=stack+overflow&fr=sfp&iscqry=&fr2=sb-top-search
This is what im doing to get data from page but need to put in a loop which changes pages:
page = BeautifulSoup(driver.page_source, 'lxml')
lnks = page.find('div', {'id': 'web'}).find_all('a', href = True)
for i in lnks:
print(i['href'])

You don't need to scroll down to the bottom. The next button is accessible without scrolling. Suppose you want to navigate 10 pages. The python script can be like this:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
driver=webdriver.Chrome()
driver.get('Yahoo Search URL')
# Let's create a loop containing the XPath for next button
# As well as waiting for the next button to be clickable.
for i in range(10):
WebDriverWait(driver, 5).until(EC.element_to_be_clickable(By.XPATH, '//a[#class="next"]'))
navigate = driver.find_element_by_xpath('//a[#class="next"]').click()

The next page button is on the bottom of the page so you first need to scroll to that element and then click it. Like this:
from selenium.webdriver.common.action_chains import ActionChains
actions = ActionChains(driver)
next_page_btn = driver.find_element_by_css_selector("a.next")
actions.move_to_element(next_page_btn).build().perform()
time.sleep(0.5)
next_page_btn.click()

PYTHON - StaleElementReferenceException for enabled element

I am trying to click on the "Next page" in Python-Selenium. The element and its path are seen, the buttom is being clicked but after clicking an error is shown:
"StaleElementReferenceException:stale element reference: element is not attached to the page document"
My code so far:
element = WebDriverWait(driver, 20).until(EC.presence_of_element_located(\
(By.XPATH, butn)))
print (element.is_enabled())
while True and element.is_enabled()==True:
driver.find_element_by_xpath(butn).click()
The error is one element.is_enabled()==True after clicking
Can someone help?

When you search elements in Selenium then it doesn't keep full objects but only references to objects in DOM in browser. And when you click then browser creates new DOM and old references are incorrect and you have to find elements again.
Something like this
# find first time
element = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.XPATH, butn)))
print(element.is_enabled())
while element.is_enabled():
driver.find_element_by_xpath(butn).click()
# find again after changes in DOM
element = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.XPATH, butn)))
print(element.is_enabled())

Cannot click on an xpath selected object Selenium (Python)

I am trying to click to an object that I select with Xpath, but there seems to be problem that I could not located the element. I am trying to click accept on the page's "Terms of Use" button. The code I have written is as
driver.get(link)
accept_button = driver.find_element_by_xpath('//*[#id="terms-ok"]')
accept_button.click()
prov = driver.find_element_by_id("province-region")
prov.click()
Here is the HTML code I have:
And I am getting a "NoSuchElementException". My goal is to click this "Kabul Ediyorum" button at the bottom of the HTML code. I started to think that we have some restrictions on the tasks we could do on that page. Any ideas ?

Not really sure what the issue might be.
But you could try the following:
Try to locate the element by its visible text
accept_button = driver.find_element_by_xpath("//*[text()='Kabul Ediyorum']").click()
Try with ActionChains
For that you need to import ActionChains
from selenium.webdriver.common.action_chains import ActionChains
accept_button = driver.find_element_by_xpath("//*[text()='Kabul Ediyorum']")
actions = ActionChains(driver)
actions.click(on_element=accept_button).perform()
Also make sure you have implicitly wait
# implicitly wait
driver.implicitly_wait(10)
Or explicit wait
element = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//*[text()='Kabul Ediyorum']"))).click()
Hope this helped!

Selenium to scroll down a div

How can I scroll down and up inside a div using Selenium? I looked everywhere on the internet. Only solutions for pages.
element = driver.find_elements_by_xpath('//*[#id="root"]/div/main/div/div[2]/div[1]/div[1]/div/div[2]/nav/div[4]/div/div[2]/div/span')
element.execute_script("arguments[0].scrollIntoView();", element )

The actions class is capable of scrolling.
This import:
from selenium.webdriver.common.action_chains import ActionChains
This function:
def ScrollIntoView(element):
actions = ActionChains(driver)
actions.move_to_element(element).perform()
Assuming your element exists and is ready on the page you can Call it as such:
element = driver.find_elements_by_xpath('//*[#id="root"]/div/main/div/div[2]/div[1]/div[1]/div/div[2]/nav/div[4]/div/div[2]/div/span')
ScrollIntoView(element)