Having trouble getting selenium to send keys to username field. I have tried "find_element_by_css_selector" and all the other find elements. Keep getting no such element exception.
This is the code I have so far:
import webbrowser
from selenium import webdriver
browser = webdriver.Chrome()
webbrowser.open('https://www.kraken.com/en-us/login')
browser.implicitly_wait(10)
browser.find_element_by_name('username').send_keys('your-username')
Good Question.This problem is caused because Selenium didn't find the element.TO fix this you need to replace the implicity_wait with 'time.sleep(in seconds)'.This will make selenium wait for the entire website to load.Some elements of the website take more time to load.Implicity wait won't work here.
Related
I have gone through existing questions and google results of a similar nature, every solution posed has not worked for me within the particular website I am currently scraping.
https://dutchie.com/embedded-menu/revolutionary-clinics-somerville/menu
I am sending page down keys to the body element, which loads each item to be scraped. I have two issues with this, first I am unable to detect when the scrolling has stopped. Second, I have to manually click the browser window as it opens to allow the keys to be sent. I am not sure how to mimic this same focus giving behavior via code.
elem = driver.find_element_by_tag_name("body")
elem.send_keys(Keys.PAGE_DOWN)
I have tried the following, in many different iterations and the number printed never charged regardless of how far down the page I am or if I used innerHeight, or body instead of documentElement.
height = driver.execute_script("return document.documentElement.scrollHeight")
If I attempt to scroll down the page using a similar approach, this page does not move.
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
I am unsure if this has to do with iframes or if I am simply misunderstanding the best approach.
Still have been unable to find a way to reliably detect the end of the page.
Thank you!
After importing the required imports
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
you can validate the page button is reached when the following element is visible:
element = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.XPATH, "//p[contains(text(),'License')]"))
As about the second issue, try clicking on the following element with Selenium:
driver.find_element_by_id("products-container").click
I have no environment to debug this, but I guess this will work
I'm using selenium and I have a problem handling the accept cookies pop-up.
from selenium import webdriver
from selenium.webdriver.support.ui import Select
import time
web = webdriver.Chrome("C:/chromedriver")
web.get('https://www.spaargids.be/sparen/schuldsaldo-berekenen.html')
time.sleep(2)
akkoord = web.find_element_by_xpath('/html/body/div/div[2]/div[3]/button[1]')
When I inspect the element, I can easily find it. But selenium is unable to do it.
Does anybody know how to fix this?
The problem is that the element is inside an iframe, you need to switch to the respective iframe.
Try this before akkoord variable:
web.switch_to.frame(web.find_element_by_xpath('//iframe[#title="SP Consent Message"]'))
That should make it work, then use the same approach but updating the XPATH to switch to the iframe where the content is.
I am trying to access PenFed in order to get my current outstanding amount. I have done quite a bit of research and unfortunately, I am still stumped. I am using Python Selenium and I am trying to click on the initial login button on the side in order to see the username field. This is the element's HTML code:
Login
When I try to run the following code:
driver.find_element_by_id("mobile-login").click()
I get the following error:
selenium.common.exceptions.ElementNotVisibleException: Message: element not interactable
Even when I try to use WebDriver Wait functions such as these:
try:
WebDriverWait(driver, 5).until(EC.visibility_of_element_located((By.ID, "mobile-login"))).click()
except ElementNotVisibleException:
WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.ID, "mobile-login"))).click()
No matter how long I make them wait, I get a timeout message:
raise TimeoutException(message, screen, stacktrace) selenium.common.exceptions.TimeoutException: Message:
All of my research says that invoking a wait function should fix it but it doesn't work for me. I also read that there might be an image overlay on top of the element that I would have to invoke before clicking on the button, but I didn't see anything in the website code as well. If I am testing it out, the only way that I am able to click on the button through code is if I physically click on it first, so I am unaware of anything else I can use. Thank you in advance for the help!
UPDATE: I have discovered that the following code works for me:
element = driver.find_element_by_id("mobile-login")
driver.execute_script("$(arguments[0]).click();", element)
But I do not know what the execute_script actually does. Can someone explain that piece of code works or if any other alternatives work for them?
The code you specified is JQuery. execute_script(p1, p2) runs a js script, where p1 is the script (in your case a JQuery line that clicks the element) and p2 is the desired element. It seems like you shouldn't need p2 if arguments[0] is equal to "element," but I'm not totally sure.
One potential fix is to use a counter for the number of times you clicked the element. If the counter reaches a certain number and the page doesn't change (you can check by finding a unique element/value on your current page), then you know it's not clickable.
Good luck!
The desired element is a dynamic element so to locate the element you have to induce WebDriverWait for the element to be clickable and you can use the following solution:
Code Block:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
options = Options()
options.add_argument('start-maximized')
options.add_argument('--disable-extensions')
driver = webdriver.Chrome(chrome_options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get('https://www.penfed.org/')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button.pfui-button.login-slide-button.pfui-button-login.dtm-global-nav[data-id='Open Log In Drawer']"))).click()
Browser Snapshot:
The link you are attempting to click is for the mobile site and is not visible if you are viewing the site at desktop resolutions. If you shrink your browser down until it changes layout, you will see the LOGIN button appear that corresponds to that link. That's why you are getting the ElementNotVisibleException.
To your second question, the reason that using .execute_script() works is that it executes JS directly and can click on anything... hidden or not. Selenium was designed to interact with the page as a user would so it won't let you click invisible elements, etc.
If you are intending for your script to act like a user, you will want to avoid using .execute_script() because it allows you to do things on the page that a user cannot do.
If you want to log in like a desktop user would, you need to click the "LOG IN" button using the CSS selector below
button[data-id='Open Log In Drawer']
That will open a side panel where you can enter your username, etc. and log in. FYI... you will likely need a wait to give the panel a chance to open before continuing the log in process.
I am writing a bot using Python with Selenium module.When I open a webpage with my bot, since the webpage contains too many external sources than dom, it takes a lot to get all of the page loaded. I used the explicit and implicit waits to eliminate this problem since I just wanted a specific element to be loaded and not all of the webpage, it didn't work. The problem is If i run the following statement:
driver = webdriver.Firefox()
driver.get('somewebpage')
elm = WebDriverWait(driver, 5).until(ExpectedConditions.presence_of_element_located((By.ID, 'someelementID'))
elm.click()
It doesn't work since the Selenium waits for the driver.get() to fully retrieve the webpage and then, it proceeds further. Now I want to write a code that sets a timeout for driver.get(), Like:
driver.get('somewebpage').timeout(5)
Where the driver.get() stops loading the page after 5 secs and the program flow proceeds, whether the driver.get() fully loaded webpage or not.
By the way, I have searched about the feature that I said above, and came across there:
Selenium WebDriver go to page without waiting for page load
But the problem is that the answer in the above link does not say anything about the Python equivalent code.
How do I accomplish the future that I am searching for?
python equivalent code for the question mentioned in the current question (Selenium WebDriver go to page without waiting for page load):
from selenium import webdriver
profile = webdriver.FirefoxProfile()
profile.set_preference('webdriver.load.strategy', 'unstable')
driver = webdriver.Firefox(profile)
and:
driver.set_page_load_timeout(5)
There is a ton of questions on this, here is an example. Here is an example that waits until all jquery ajax calls have completed or a 5 second timeout.
from selenium.webdriver.support.ui import WebDriverWait
WebDriverWait(driver, 5).until(lambda s: s.execute_script("return jQuery.active == 0"))
It was a really tedious issue to solve. I just did the following and the problem got resolved:
driver= webdriver.Firefox()
driver.set_page_load_timeout(5)
driver.get('somewebpage')
It worked for me using Firefox driver (and Chrome driver as well).
I am getting some very odd behavior on a project. Essentially this automation creates an article on a web page using selenium and verifies it's existence by clicking/opening the article. However when i run the program without debug it is failing to "click()" on the web element. So I investigate and start to debug. I notice that the web element ("target_element") is "None". I continue past for just debugging and it fails to click "target_element" as expected.
When i rerun the program and instead of continuing on i open the class in the debug view. I see that "target_element" exists, so i continue on and target_element.click() successfully loads the article on the webpage. Does anyone have any suggestions on how to fix this problem?
TLDR; article.target_element does not exist when running the program UNLESS i set a breakpoint and look at "article" in the debugger. Then if i continue in the program article.target_element suddenly exists.
Possible reason is the element load time.
When you are running the test Selenium runs faster than the time needed by the element to load properly and eventually throws the exception. On the other hand, during debug you are stepping in and giving enough time for the element to load and Selenium finds the element without any issue.
If, the problem is exactly what it is mentioned above the use of explicit wait should be the way to go.
Example taken from the Selenium Python doc
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Firefox()
driver.get("http://somedomain/url_that_delays_loading")
try:
element = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.ID, "myDynamicElement"))
)
finally:
driver.quit()