I'm learning how to use selenium and I'm stuck on figuring out how to scroll down in a website to verify an element exists.
I tried using the methods that was found in this question
Scrolling to element using webdriver?
but selenium won't scroll down the page. Instead it'll give me an error
"selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: element"
Heres the codes I am using
moveToElement:
element = driver.find_element_by_xpath('xpath')
actions = ActionChains(driver)
actions.move_to_element(element).perform()
Scrolling into View
element = driver.find_element_by_xpath('xpath')
driver.execute_script("arguments[1].scrollIntoView();", element)
The whole code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.action_chains import ActionChains
driver = webdriver.Firefox()
driver.get("https://www.linkedin.com/")
element =
driver.find_element_by_xpath('/html/body/div/main/div/div[1]/div/h1/img')
element = driver.find_element_by_xpath('//*[#id="login-email"]')
element.send_keys('')
element = driver.find_element_by_xpath('//*[#id="login-password"]')
element.send_keys('')
element = driver.find_element_by_xpath('//*[#id="login-submit"]')
element.click();
element = driver.find_element_by_xpath('')
actions = ActionChains(driver)
actions.move_to_element(element).perform()
There are two aspects to your is problem
Whether the element exist?
Whether the element displayed on the page?
Whether the element exist?
It may happen that the element exists on the page[i.e. it is part of the DOM] but it is not available right away for further selenium action becuase it is not visible[hidden], it's only gets visible after some actions or it may get displayed on scroll down the page.
Your code is throwing an exception here-
element = driver.find_element_by_xpath('xpath')
as WebDiver not able to find the element using mentioned xpath. Once you fix this, you can moe forward the next part.
Whether the element displayed on the page?
Once you fix the above issue, you should check whether the element is being displayed or not. If it's not displayed and avialble on the scroll then you can use code like
if !element.is_displayed():
driver.execute_script("arguments[1].scrollIntoView();", element)
Perfer using Action Class for very specific mouse action.
Update:-
If you are application using lazy loading and the element you are trying to find is available on scroll the you can try something like this -
You have to import exception like -
from selenium.common.exceptions import NoSuchElementException
and the create new recursion function which would scroll if element not found, something like this -
def search_element():
try:
elem = driver.find_element_by_xpath("your")
return elem
except NosSuchElementException:
driver.execute_script("window.scrollTo(0,Math.max(document.documentElement.scrollHeight,document.body.scrollHeight,document.documentElement.clientHeight));")
search_element()
I am not sure that having recurion for finding element here is good idea, also I have naver worked on the python so you need to lookout for the syntax
Amm may be this might help.Just send page down key and if you are sure that element exists definitely then this would work
from selenium.webdriver.common.keys import Keys
from selenium.webdriver import ActionChains
import time
while True:
ActionChains(driver).send_keys(Keys.PAGE_DOWN).perform()
time.sleep(2) #2 seconds
try:
element = driver.find_element_by_xpath("your_element_xpath")
break
except:
continue
Related
i'm trying to save usernames who liked my post in Instagram
here is my code :
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.action_chains import ActionChains
from time import *
chrome_options = Options()
chrome_options.add_argument("--user-data-dir=chrome-data")
browser = webdriver.Chrome(options=chrome_options)
browser.get('https://www.instagram.com/p/CMAVjR5CmOx/')
continue_link = browser.find_element_by_partial_link_text('others')
continue_link.click()
elems = browser.find_elements_by_css_selector(".FPmhX.notranslate.MBL3Z")
links = [elem.get_attribute('title') for elem in elems]
WebElement = browser.find_element_by_xpath('//*[ contains (text(), ‘#’ ) ]')
WebElement.click()
browser.execute_script("arguments[0].scrollIntoView();", WebElement)
the problem is, it just save 11 first username and it can't scroll down to load all of them
i tried some codes to scroll down but it scroll main page while i want "followers list" to be scrolled
main screen
can anyone help me with this scrolling down?
i tried for "send.key" and "browser.execute_script("arguments[0].scrollIntoView();",Element)" but nothing usefull
Try this solution. I'm using it right now so pretty sure it work.
# try to find the locator of the div element which contain the slider
element = browser.find_element_by_xpath('//div/parent::ul/parent::div')
# increase the distance from the top element each time one pixel
verical_ordinate = 100
for i in range(0, 50):
print(verical_ordinate)
browser.execute_script("arguments[0].scrollTop = arguments[1]", element,
verical_ordinate)
verical_ordinate += 100
time.sleep(1)
For scrolling, I usually use this code and it always works.
driver.execute_script("arguments[0].scrollIntoView();",driver.find_element_by_xpath("your_xpath_selector"))
First, rename WebElement to your custom name, for example to web_element as your name is confusing.
Second, make sure this locator is correct. Unfortunately, I do not have an Instagram account and cannot verify this locator.
I am trying to go to the drought monitor website and tell it to select to show county data. I am able to get my code to navigate to the website, and it clicks the dropdown, but I cannot get it to type in "county". My code gets to the last line and then give the error: "Cannot focus element".
Any help would be greatly appreciated as I'm very new to Selenium.
from selenium import webdriver
from selenium.webdriver.support.ui import Select
from selenium.webdriver.common.keys import Keys
browser = webdriver.Chrome()
browser.get('http://droughtmonitor.unl.edu/Data/DataDownload/ComprehensiveStatistics.aspx')
browser.maximize_window()
dropdown = browser.find_element_by_xpath("""//*
[#id="dnn_ctr1009_USDMservice_CompStats_2017_aoiType_chosen"]""")
dropdown.click()
dropdown.send_keys('county')
dropdown.submit()
print("I'm done")
You're sending keys to the <div> that contains the search <input>, rather than to the <input> element itself. You'll need to find the <input> and send it the keys.
(Note: You also don't need to use XPath for something as simple as a lookup by id.)
dropdown = browser.find_element_by_id("dnn_ctr1009_USDMservice_CompStats_2017_aoiType_chosen")
dropdown.click()
search = dropdown.find_element_by_tag_name("input")
search.send_keys("county", Keys.ENTER)
I am trying to run a script in selenium webdriver python. Where I am trying to click on search field, but its always showing exception of "An element could not be located on the page using the given search parameters."
Here is script:
from selenium import webdriver
from selenium.webdriver.common.by import By
class Exercise:
def safari(self):
class Exercise:
def safari(self):
driver = webdriver.Safari()
driver.maximize_window()
url= "https://www.airbnb.com"
driver.implicitly_wait(15)
Title = driver.title
driver.get(url)
CurrentURL = driver.current_url
print("Current URL is "+CurrentURL)
SearchButton =driver.find_element(By.XPATH, "//*[#id='GeocompleteController-via-SearchBarV2-SearchBarV2']")
SearchButton.click()
note= Exercise()
note.safari()
Please Tell me, where I am wrong?
There appears to be two matching cases:
The one that matches the search bar is actually the second one. So you'd edit your XPath as follows:
SearchButton = driver.find_element(By.XPATH, "(//*[#id='GeocompleteController-via-SearchBarV2-SearchBarV2'])[2]")
Or simply:
SearchButton = driver.find_element_by_xpath("(//*[#id='GeocompleteController-via-SearchBarV2-SearchBarV2'])[2]")
You can paste your XPath in Chrome's Inspector tool (as seen above) by loading the same website in Google Chrome and hitting F12 (or just right click anywhere and click "Inspect"). This gives you the matching elements. If you scroll to 2 of 2 it highlights the search bar. Therefore, we want the second result. XPath indices start at 1 unlike most languages (which usually have indices start at 0), so to get the second index, encapsulate the entire original XPath in parentheses and then add [2] next to it.
Basically, I am trying to create a bot that can find an element on a page with a given raw_input text, finding the button, and clicking it. The purpose of this is to click on links (buttons) that may not be on the web page yet, but will appear after the site refreshes. Because of this, I cannot find elements by XPATH, because the XPATH will be unknown until the second the link becomes available. So, my question is: is there a way for Selenium to find a button based on text and click it? Here is some of my code:
key1 = raw_input('Enter the first keyword: ')
key2 = raw_input('Enter the second keyword: ')
key3 = raw_input('Enter the third keyword: ')
...
elem1 =driver.find_elements_by_xpath("//*[contains(text(), key1)]")
if elem1.is_displayed():
elem1.click()
else:
print "Cannot find element to click. Retrying..."
driver.refresh()
I need the program to find the text based on the keywords, and then click the button / link with these keywords in it. Is this possible? or only possible with XPATHs? The example code i gave with giving an elem has not been working.
thanks in advance
Ok first of all, the only one of selecting an element by text is sadly xpath, but do not worry, there is a workaround you could use waits
#Libraries (those comes with selenium, you don't have to install anything)
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
#Way
elem1 = WebDriverWait(driver, 10).until(
EC.element_to_be_clickable((By.XPATH, '//*[contains(text(),
{})]'.format(key1))
)
elem1.click()
this will make it wait for the element to be clickable if the element is not clickable after 10 seconds it will raise an error if it gets clickable before that time it will return the element and break the wait automatically.
I just want to write a simple log in code for one website. However, I think the log in page was written in JS. It's really hard to locate the elements with selenium.
The web page I am going to play with is:
"https://www.nike.com/snkrs/login?returnUrl=%2F"
This is how the page looks like and how the inspect element page look like:
I was trying to locate the element by following code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
driver = webdriver.Firefox()
driver.get("https://www.nike.com/snkrs/login?returnUrl=%2Fthread%2Fe9680e08e7e3cd76b8832684037a58a369cad5ed")
time.sleep(5)
driver.switch_to.frame(driver.find_element_by_tag_name("iframe"))
elem =driver.find_element_by_xpath("//*[#id='ce3feab5-6156-441a-970e-23544473a623']")
elem.send_keys("pycon")
elem.send_keys(Keys.RETURN)
driver.close()
This code return the error that the element could not find by [#id='ce3feab5-6156-441a-970e-23544473a623'.
I tried playing with frames, it seems does not work. If I went to "view web source" page, it is full of JS code.
Is there a good way to play with such a web page with selenium?
Try changing the code :
elem =driver.find_element_by_xpath("//*[#id='ce3feab5-6156-441a-970e-23544473a623']")
to
elem =driver.find_element_by_xpath("//*[#type='email']")
My guess (and observation) is that the id changes each time you visit the page. The id looks auto-generated, and when I go to the page multiple times, the id is different each time.
You'll need to search for something that doesn't change. For example, you can search for the name attribute, which has the seemingly static value "emailAddress"
element = driver.find_element_by_name("emailAddress")
You could also use an xpath expression to search for other attributes, such as data-componentname:
element = driver.find_element_by_xpath("//input[#data-componentname='emailAddress']")
Also, instead of a hard-coded sleep, you can simply wait for the element to be visible:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Firefox()
driver.get("https://www.nike.com/snkrs/login")
element = WebDriverWait(driver, 10).until(
EC.visibility_of_element_located((By.NAME, "emailAddress"))
)