I'm attempting to create an automated login script to netflix website: https://www.netflix.com/it/
That's the code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Firefox()
while True:
driver.get("https://www.netflix.com/it/")
login = driver.find_element_by_css_selector(".authLinks.signupBasicHeader")
login.click()
element = driver.find_element_by_name("email")
element.send_keys("test1#email.com")
submit = driver.find_element_by_css_selector(".btn.login-button.btn-submit.btn-small")
submit.click()
element2 = driver.find_element_by_name("password")
element2.send_keys("test1")
submit.click()
But sometimes it works and sometimes it doesn't and it raises this exception:
Traceback (most recent call last):
File "netflix.py", line 35, in <module>
submit.click()
File "/home/user/.local/lib/python3.5/site-packages/selenium/webdriver/remote/webelement.py", line 72, in click
self._execute(Command.CLICK_ELEMENT)
File "/home/user/.local/lib/python3.5/site-packages/selenium/webdriver/remote/webelement.py", line 461, in _execute
return self._parent.execute(command, params)
File "/home/user/.local/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 236, in execute
self.error_handler.check_response(response)
File "/home/user/.local/lib/python3.5/site-packages/selenium/webdriver/remote/errorhandler.py", line 192, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotVisibleException: Message: Element is not currently visible and so may not be interacted with
Stacktrace:
at fxdriver.preconditions.visible (file:///tmp/tmp7otgskq8/extensions/fxdriver#googlecode.com/components/command-processor.js:10092)
at DelayedCommand.prototype.checkPreconditions_ (file:///tmp/tmp7otgskq8/extensions/fxdriver#googlecode.com/components/command-processor.js:12644)
at DelayedCommand.prototype.executeInternal_/h (file:///tmp/tmp7otgskq8/extensions/fxdriver#googlecode.com/components/command-processor.js:12661)
at DelayedCommand.prototype.executeInternal_ (file:///tmp/tmp7otgskq8/extensions/fxdriver#googlecode.com/components/command-processor.js:12666)
at DelayedCommand.prototype.execute/< (file:///tmp/tmp7otgskq8/extensions/fxdriver#googlecode.com/components/command-processor.js:12608)
The exception says that a part of the web page is invisible (even if that part IS in the page actually)... It's a sort of bug.
How can I bypass this?
No need to put the test code inside while loop, since login form should be submitted once. I guess when it works successfully, next iteration gives error since there is no form.
You are also clicking the submit button before and after password entry. It is better to click after form is fully filled, if possible and not testing empty form.
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("https://www.netflix.com/it/")
login = driver.find_element_by_css_selector(".authLinks.signupBasicHeader")
login.click()
element = driver.find_element_by_name("email")
element.send_keys("test1#email.com")
submit = driver.find_element_by_css_selector(".btn.login-button.btn-submit.btn-small")
submit.click()
element2 = driver.find_element_by_name("password")
element2.send_keys("test1")
submit = driver.find_element_by_css_selector(".btn.login-button.btn-submit.btn-small")
submit.click()
Related
I am trying to scrape data from this File-tuning dynamic website which is loading data through javascript (ajax) requests.
what is want to do is that It selects cars from type and then selects make, model, engine iteratively, and then I want to scrape data for each make, model, and engine.
Here is the code I write to select cars from type
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support.select import Select
PATH = "C:\SeleniumDrivers\geckodriver.exe"
driver = webdriver.Firefox(executable_path=PATH)
driver.get("https://file-tuning.com/chiptuning")
type_element_select = driver.find_element_by_id("type")
action = ActionChains(driver)
action.move_to_element(type_element_select)
action.click(type_element_select)
action.perform()
action.move_to_element(Select(type_element_select).select_by_value("cars"))
action.click(Select(type_element_select).select_by_value("cars"))
action.perform()
The error I get:
Traceback (most recent call last):
File "D:\Python\selenium\test.py", line 27, in <module>
action.move_to_element(Select(type_element_select).select_by_value("cars"))
File "C:\Users\Umair\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\support\select.py", line 82, in select_by_value
self._setSelected(opt)
File "C:\Users\Umair\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\support\select.py", line 212, in _setSelected
option.click()
File "C:\Users\Umair\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\remote\webelement.py", line 80, in click
self._execute(Command.CLICK_ELEMENT)
File "C:\Users\Umair\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\remote\webelement.py", line 633, in _execute
return self._parent.execute(command, params)
File "C:\Users\Umair\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\Umair\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotInteractableException: Message: Element <option> could not be scrolled into view
I also try this
types_element = driver.find_element_by_id("type")
types_object = Select(types_element)
types_object.select_by_visible_text("Cars")
but it also gives me the same exception.
how I can scrape through this site?
already have seen and tried other StackOverflow related question but didn't work out for me.
You need to use browser in full screen mode :
driver.maximize_window()
so that all web elements would be visible to Selenium.
Sample code :
PATH = "C:\SeleniumDrivers\geckodriver.exe"
driver = webdriver.Firefox(executable_path=PATH)
driver.maximize_window()
driver.implicitly_wait(30)
driver.get("https://file-tuning.com/chiptuning")
wait = WebDriverWait(driver, 10)
type_element_select = driver.find_element_by_id("type")
action = ActionChains(driver)
action.move_to_element(type_element_select)
action.click(type_element_select)
action.perform()
action.move_to_element(Select(type_element_select).select_by_value("cars"))
action.click(Select(type_element_select).select_by_value("cars"))
action.perform()
This is my current code, but whenever I run it I get an error on the last line
stale element reference: element is not attached to the page document
from selenium import webdriver
url = "https://otctransparency.finra.org/otctransparency/OtcDownload"
driver.get(url)
driver.maximize_window()
driver.implicitly_wait(5)
agree = driver.find_elements_by_xpath("//button[#class='btn btn-warning']")[0]
agree.click()
nonats = driver.find_element_by_link_text('OTC (Non-ATS) Download')
nonats.click()
driver.find_element_by_xpath("//img[#src='./assets/icon_download.png']").click()
driver.switch_to.window(driver.window_handles[0])
driver.find_element_by_xpath("(//div[#class='checkbox-inline'])[2]").click()
driver.find_element_by_xpath("(//div[#class='checkbox-inline'])[1]").click()
driver.implicitly_wait(5)
button = driver.find_element_by_xpath("//img[#src='./assets/icon_download.png']")
print(button.is_displayed())
button.click()
When I run my code in debugging mode line by line, everything works fine without any errors. Any help would be great.
Edit: This is my stack trace
Traceback (most recent call last):
File "C:\Users\derpe\Desktop\python projects personal\testing finra\untitled1.py", line 31, in <module>
button.click()
File "C:\Users\derpe\anaconda3\lib\site-packages\selenium\webdriver\remote\webelement.py", line 80, in click
self._execute(Command.CLICK_ELEMENT)
File "C:\Users\derpe\anaconda3\lib\site-packages\selenium\webdriver\remote\webelement.py", line 633, in _execute
return self._parent.execute(command, params)
File "C:\Users\derpe\anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\derpe\anaconda3\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
StaleElementReferenceException: stale element reference: element is not attached to the page document
(Session info: chrome=86.0.4240.75)
The checkbox clicks are triggering a page refresh while you're searching for the download link.
Call sleep to allow the refresh to finish.
driver.implicitly_wait(5)
import time # add this
time.sleep(1) # add this
button = driver.find_element_by_xpath("//img[#src='./assets/icon_download.png']")
print(button.is_displayed())
button.click()
I tried other selenium waits, but they did not work for me. Probably because the element search succeeds before the refresh starts but still clicks to late.
I am very new to coding and I am trying to make a form filler on Nike.com using the Selenium Chrome webdriver. However, a pop-up comes up about cookied and I am finding it hard to remove it so I can fill out the form.
This is what it looks like
and my code looks like this:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
#Initialise a chrome browser and return it
def initialisebrowser():
browser=webdriver.Chrome(r'''C:\Users\ben_s\Downloads\chromedriver''')
return browser
#Pass in the browser and url, and go to the url with the browser
def geturl(browser, url):
browser.get(url)
#Initialise the browser (and store the returned browser)
browser = initialisebrowser()
#Go to a url(nike.com) with the browser
geturl(browser,"https://www.nike.com/gb/en_gb/s/register")
button = browser.find_element_by_class_name("nsg-button.nsg-grad--nike-orange.yes-button.cookie-settings-button.js-yes-button")
button.click()
When I run this code, I get this error:
Traceback (most recent call last):
File "C:\Users\ben_s\Desktop\Nike Account Generator.py", line 19, in <module>
button = browser.find_element_by_class_name("nsg-button.nsg-grad--nike-orange.yes-button.cookie-settings-button.js-yes-button")
File "C:\Python\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 557, in find_element_by_class_name
return self.find_element(by=By.CLASS_NAME, value=name)
File "C:\Python\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 957, in find_element
'value': value})['value']
File "C:\Python\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 314, in execute
self.error_handler.check_response(response)
File "C:\Python\Python36\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"class name","selector":"nsg-button.nsg-grad--nike-orange.yes-button.cookie-settings-button.js-yes-button"}
(Session info: chrome=67.0.3396.87)
(Driver info: chromedriver=2.38.552522 (437e6fbedfa8762dec75e2c5b3ddb86763dc9dcb),platform=Windows NT 10.0.17134 x86_64)
Any ideas, pointers or solutions to the problem are much apprieciated
find_element_by_class_name recives one class as parameter
browser.find_element_by_class_name('yes-button')
The parameter you provided is combination of all the webelement classes, and is used by css_selector
browser.find_element_by_css_selector('.nsg-button.nsg-grad--nike-orange.yes-button.cookie-settings-button.js-yes-button')
Note that you need to add . before the first class as well, otherwise it is treated as tag name. For example in this case
browser.find_element_by_css_selector('button.nsg-button.nsg-grad--nike-orange.yes-button.cookie-settings-button.js-yes-button')
To make sure the button exists before clicking on it you can use explicit wait
button = WebDriverWait(driver, 10).until(expected_conditions.element_to_be_clickable((By.CLASS_NAME, 'yes-button')))
button.click()
use time.sleep to wait until the popup will load, and after that you can use cookie-settings-button-container class that is parrent of the btn this will work.
time.sleep(1)
button = browser.find_element_by_class_name("cookie-settings-button-container")
button.click()
from selenium import webdriver
from info import user_name, pass_word
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
import time
driver = webdriver.Firefox() #opening web browser
driver.get('https://tuportal.temple.edu/') #going to temple wbsite
username = driver.find_element_by_id("username") #finds space where to enteruser
password = driver.find_element_by_id("password") #same for password
username.send_keys(user_name()) #enters my username
password.send_keys(pass_word()) #enters my password
driver.find_element_by_name("_eventId_proceed").click()#click login
#waits until the element for student tools tab is found
element = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, "layout_8")))
element.click()
time.sleep(3)
driver.switch_to_frame("iFrame_AppTupChannelsStudentRegistration")#accessing iframe
#scrolls
element = driver.find_element_by_id('StudentRegistration_3')
driver.execute_script("return arguments[0].scrollIntoView();", element)
#end of scroll
element.click() #click look up classes
All of the code works up until the last line, "element.click()"
I am getting an error -->
Traceback (most recent call last):
File "C:\Users\Karl\Desktop\project\signin.py", line 31, in
element.click() #click look up classes
File "C:\Users\Karl\AppData\Local\Programs\Python\Python36\lib\site-packages\selenium\webdriver\remote\webelement.py", line 80, in click
self._execute(Command.CLICK_ELEMENT)
File "C:\Users\Karl\AppData\Local\Programs\Python\Python36\lib\site-packages\selenium\webdriver\remote\webelement.py", line 628, in _execute
return self._parent.execute(command, params)
File "C:\Users\Karl\AppData\Local\Programs\Python\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 312, in execute
self.error_handler.check_response(response)
File "C:\Users\Karl\AppData\Local\Programs\Python\Python36\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotInteractableException: Message: Element could not be scrolled into view
The confusing part is the 2 lines right before the line that causes the error are
element = driver.find_element_by_id('StudentRegistration_3')
driver.execute_script("return arguments[0].scrollIntoView();", element)
which do exactly what the error says hasn't happened, they scroll the the button I am trying to click on with the last line into view.
Can anyone identify a solution?
I am using python 3, selenium and Firefox are updated to current versions.
So I am learning how to use Selenium for web automation - I am trying to write a script that returns my American Express balance to my console. The first step is actually logging on successfully...
It appears that my action for clicking the login button raises an error of not finding the element, even though I can see it when I am on firebug.
This is my code:
from selenium import webdriver
driver = webdriver.Firefox()
baseurl = "https://www.americanexpress.com/canada/"
username = "myusername"
password = "mypassword"
xpaths = { 'usernameField' : "//input[#id='UserID']",
'passwordField' : "//input[#id='Password']",
'submitButton' : "//input[#id='loginButton']"
}
driver.get(baseurl)
driver.find_element_by_xpath(xpaths['usernameField']).clear()
driver.find_element_by_xpath(xpaths['usernameField']).send_keys(username)
driver.find_element_by_xpath(xpaths['passwordField']).clear()
driver.find_element_by_xpath(xpaths['passwordField']).send_keys(password)
driver.find_element_by_xpath(xpaths['submitButton']).click()
This is the console error message I get, where the browser has filled in my login details, but just has not clicked on the login button:
Traceback (most recent call last):
File "get_balance.py", line 29, in <module>
driver.find_element_by_xpath(xpaths['submitButton']).click()
File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 232, in find_element_by_xpath
return self.find_element(by=By.XPATH, value=xpath)
File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 664, in find_element
{'using': by, 'value': value})['value']
File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 175, in execute
self.error_handler.check_response(response)
File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 166, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: {"method":"xpath","selector":"//input[#id='loginButton']"}
Stacktrace:
at FirefoxDriver.prototype.findElementInternal_ (file:///var/folders/mx/dbnsd72j05zd4k2cls4cvmf00000gn/T/tmpKQDaey/extensions/fxdriver#googlecode.com/components/driver-component.js:10271)
at FirefoxDriver.prototype.findElement (file:///var/folders/mx/dbnsd72j05zd4k2cls4cvmf00000gn/T/tmpKQDaey/extensions/fxdriver#googlecode.com/components/driver-component.js:10280)
at DelayedCommand.prototype.executeInternal_/h (file:///var/folders/mx/dbnsd72j05zd4k2cls4cvmf00000gn/T/tmpKQDaey/extensions/fxdriver#googlecode.com/components/command-processor.js:12274)
at DelayedCommand.prototype.executeInternal_ (file:///var/folders/mx/dbnsd72j05zd4k2cls4cvmf00000gn/T/tmpKQDaey/extensions/fxdriver#googlecode.com/components/command-processor.js:12279)
at DelayedCommand.prototype.execute/< (file:///var/folders/mx/dbnsd72j05zd4k2cls4cvmf00000gn/T/tmpKQDaey/extensions/fxdriver#googlecode.com/components/command-processor.js:12221)
Any thoughts? Any advice/help is much appreciated, thanks!
This is an a element, not an input:
<a tabindex="0" href="#" id="loginButton" title="Login securely">
<span></span>
Log In
</a>
Change your xpath to: //a[#id="loginButton"].
Aside from that, for id attributes there is a find_element_by_id() method:
driver.find_element_by_id("loginButton").click()
Also, if you want to have element locators separated from the actual "action" code, you can configure it the following way (left a single xpath expression for a sake of an example):
from selenium.webdriver.common.by import By
locators = {
'usernameField': (By.ID, "UserID"),
'passwordField': (By.XPATH, "//input[#id='Password']"),
'submitButton': (By.ID, "loginButton")
}
Then, your "action" code would be using find_element():
username = driver.find_element(*locators['usernameField'])
username.clear()
username.send_keys(username)
password = driver.find_element(*locators['passwordField'])
password.clear()
password.send_keys(password)
login_button = driver.find_element(*locators['submitButton'])
login_button.click()