I am very new to coding and I am trying to make a form filler on Nike.com using the Selenium Chrome webdriver. However, a pop-up comes up about cookied and I am finding it hard to remove it so I can fill out the form.
This is what it looks like
and my code looks like this:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
#Initialise a chrome browser and return it
def initialisebrowser():
browser=webdriver.Chrome(r'''C:\Users\ben_s\Downloads\chromedriver''')
return browser
#Pass in the browser and url, and go to the url with the browser
def geturl(browser, url):
browser.get(url)
#Initialise the browser (and store the returned browser)
browser = initialisebrowser()
#Go to a url(nike.com) with the browser
geturl(browser,"https://www.nike.com/gb/en_gb/s/register")
button = browser.find_element_by_class_name("nsg-button.nsg-grad--nike-orange.yes-button.cookie-settings-button.js-yes-button")
button.click()
When I run this code, I get this error:
Traceback (most recent call last):
File "C:\Users\ben_s\Desktop\Nike Account Generator.py", line 19, in <module>
button = browser.find_element_by_class_name("nsg-button.nsg-grad--nike-orange.yes-button.cookie-settings-button.js-yes-button")
File "C:\Python\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 557, in find_element_by_class_name
return self.find_element(by=By.CLASS_NAME, value=name)
File "C:\Python\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 957, in find_element
'value': value})['value']
File "C:\Python\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 314, in execute
self.error_handler.check_response(response)
File "C:\Python\Python36\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"class name","selector":"nsg-button.nsg-grad--nike-orange.yes-button.cookie-settings-button.js-yes-button"}
(Session info: chrome=67.0.3396.87)
(Driver info: chromedriver=2.38.552522 (437e6fbedfa8762dec75e2c5b3ddb86763dc9dcb),platform=Windows NT 10.0.17134 x86_64)
Any ideas, pointers or solutions to the problem are much apprieciated
find_element_by_class_name recives one class as parameter
browser.find_element_by_class_name('yes-button')
The parameter you provided is combination of all the webelement classes, and is used by css_selector
browser.find_element_by_css_selector('.nsg-button.nsg-grad--nike-orange.yes-button.cookie-settings-button.js-yes-button')
Note that you need to add . before the first class as well, otherwise it is treated as tag name. For example in this case
browser.find_element_by_css_selector('button.nsg-button.nsg-grad--nike-orange.yes-button.cookie-settings-button.js-yes-button')
To make sure the button exists before clicking on it you can use explicit wait
button = WebDriverWait(driver, 10).until(expected_conditions.element_to_be_clickable((By.CLASS_NAME, 'yes-button')))
button.click()
use time.sleep to wait until the popup will load, and after that you can use cookie-settings-button-container class that is parrent of the btn this will work.
time.sleep(1)
button = browser.find_element_by_class_name("cookie-settings-button-container")
button.click()
Related
This is my current code, but whenever I run it I get an error on the last line
stale element reference: element is not attached to the page document
from selenium import webdriver
url = "https://otctransparency.finra.org/otctransparency/OtcDownload"
driver.get(url)
driver.maximize_window()
driver.implicitly_wait(5)
agree = driver.find_elements_by_xpath("//button[#class='btn btn-warning']")[0]
agree.click()
nonats = driver.find_element_by_link_text('OTC (Non-ATS) Download')
nonats.click()
driver.find_element_by_xpath("//img[#src='./assets/icon_download.png']").click()
driver.switch_to.window(driver.window_handles[0])
driver.find_element_by_xpath("(//div[#class='checkbox-inline'])[2]").click()
driver.find_element_by_xpath("(//div[#class='checkbox-inline'])[1]").click()
driver.implicitly_wait(5)
button = driver.find_element_by_xpath("//img[#src='./assets/icon_download.png']")
print(button.is_displayed())
button.click()
When I run my code in debugging mode line by line, everything works fine without any errors. Any help would be great.
Edit: This is my stack trace
Traceback (most recent call last):
File "C:\Users\derpe\Desktop\python projects personal\testing finra\untitled1.py", line 31, in <module>
button.click()
File "C:\Users\derpe\anaconda3\lib\site-packages\selenium\webdriver\remote\webelement.py", line 80, in click
self._execute(Command.CLICK_ELEMENT)
File "C:\Users\derpe\anaconda3\lib\site-packages\selenium\webdriver\remote\webelement.py", line 633, in _execute
return self._parent.execute(command, params)
File "C:\Users\derpe\anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\derpe\anaconda3\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
StaleElementReferenceException: stale element reference: element is not attached to the page document
(Session info: chrome=86.0.4240.75)
The checkbox clicks are triggering a page refresh while you're searching for the download link.
Call sleep to allow the refresh to finish.
driver.implicitly_wait(5)
import time # add this
time.sleep(1) # add this
button = driver.find_element_by_xpath("//img[#src='./assets/icon_download.png']")
print(button.is_displayed())
button.click()
I tried other selenium waits, but they did not work for me. Probably because the element search succeeds before the refresh starts but still clicks to late.
Hello I would like to ask if there is any way to refresh source code of site without refreshing page. The problem is when I load page http://107.170.101.241:8080/getTableColumn/ and put there some information - you can see in my code below and then click Analyse there is displayed new textarea. I want to get text from this textarea but I cant because source code is “old” and xpath cant find it. Last line of code is what I want to print to console. I tried time.sleep etc. and nothing helped.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import Select
import time
chromedriver = "/usr/local/bin/chromedriver"
driver = webdriver.Chrome(chromedriver)
driver.get("http://107.170.101.241:8080/getTableColumn/")
time.sleep(1)
datab = driver.find_element_by_xpath("//select[#name='dbVendor']")
database = Select(datab)
database.select_by_visible_text("Sybase")
datab2 = driver.find_element_by_xpath("//select[#name='options']")
database2 = Select(datab2)
database2.select_by_visible_text("Show By SQL Clause")
txt = driver.find_element_by_xpath("//textarea[#name='sql']")
txt.clear()
txt.send_keys("select trd.M_NB as 'Trade_number' from CRD_TRADE_REP trd")
txt1 = driver.find_element_by_xpath("//textarea[#name='metadata']")
txt1.clear()
txt1.send_keys("CRD_TRADE_REP, M_NB")
analyze = driver.find_element_by_xpath("//input[#type='submit']")
analyze.send_keys("")
analyze.send_keys(Keys.RETURN)
#cant find this textarea below
out = driver.find_element_by_xpath("//textarea[#name='outputText']")
Here is Traceback:
Traceback (most recent call last):
File "/Users/martinkubicka/Documents/fiverrgde.py", line 32, in <module>
out = driver.find_element_by_xpath("//textarea[#name='outputText']")
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 394, in find_element_by_xpath
return self.find_element(by=By.XPATH, value=xpath)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 976, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//textarea[#name='outputText']"}
(Session info: chrome=84.0.4147.135)
In devtools, If you scroll up from your element, you can see your output textarea is nested in:
<iframe name="result" id="result" style="height: 180px; width: 800px;" scrolling="no" frameborder="0">
These need extra handling in selenium.
Try this at the end of your script:
#Get the frame
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[#name='result']")))
#wait for your object to be ready - i use clickable as i like it
out = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH,"//textarea[#name='outputText']")))
print(out.text)
#do stuff your stuff to the "out" element here
#when ready to go back to the main page content (not the iframe)
driver.switch_to_default_content()
When i run your code with that addition i get the output:
Tables: tetSelect CRD_TRADE_REP(1,40) Columns: selectList
CRD_TRADE_REP.M_NB(1,12)
I'm trying to get a login script to select a user name input to enter in my user name. After this popup is done there will be another one asking for the password. I'm new to python and web interfaces so I'm having trouble identifying what element of the website I need to select to get this to work. Here is the code I have so far.
import selenium
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait as wait
DynamoForum = webdriver.Chrome()
DynamoForum.get("https://forum.dynamobim.com/")
login = DynamoForum.find_element_by_class_name("header-buttons").click()
#DynamoForum.switch_to_frame(DynamoForum.find_element_by_
#wait(DynamoForum,10).until(EC.frame_to_be_available_and_switch_to_it(
DynamoForum.find_element_by_xpath("//title[1]")))
wait(DynamoForum,10).until(EC.frame_to_be_available_and_switch_to_it(
DynamoForum.find_element_by_xpath(
"//iframe[#id='destination_publishing_iframe_autodesk_0']")))
DynamoForum.find_element_by_id("userName").send_heys("xxx")
The website is opening and the popup is starting but no text is being entered. Here is what my getting as a result:
Traceback (most recent call last):
File "C:/Users/cjr/PycharmProjects/DynamoForum/DynamoForum.py", line 17, in <module>
wait(DynamoForum, 10).until(EC.frame_to_be_available_and_switch_to_it(DynamoForum.find_element_by_xpath("//iframe[#id='destination_publishing_iframe_autodesk_0']")))
File "C:\Users\cjr\PycharmProjects\DynamoForum\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 394, in find_element_by_xpath
return self.find_element(by=By.XPATH, value=xpath)
File "C:\Users\cjr\PycharmProjects\DynamoForum\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 978, in find_element
'value': value})['value']
File "C:\Users\cjr\PycharmProjects\DynamoForum\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\cjr\PycharmProjects\DynamoForum\venv\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//iframe[#id='destination_publishing_iframe_autodesk_0']"}
(Session info: chrome=72.0.3626.119)
(Driver info: chromedriver=73.0.3683.20 (8e2b610813e167eee3619ac4ce6e42e3ec622017),platform=Windows NT 10.0.17134 x86_64)
Basically when you are clicking on login button, you are moving to another window and to access the element in new window you need to switch it from parent window to access this.Try the below code it should work.
from selenium import webdriver
DynamoForum = webdriver.Chrome()
DynamoForum.get("https://forum.dynamobim.com/")
Parent_window = DynamoForum.window_handles[0]
login = DynamoForum.find_element_by_class_name("header-buttons").click()
window_child= DynamoForum.window_handles[1]
DynamoForum.switch_to.window(window_child)
DynamoForum.find_element_by_id("userName").send_keys("xyz#gmail.com")
DynamoForum.find_element_by_id("verify_user_btn").click()
wait=WebDriverWait(DynamoForum,20)
wait.until(EC.visibility_of_element_located((By.ID,"password"))).send_keys("xxx")
DynamoForum.find_element_by_id("btnSubmit").click()
You need to switch to the iframe.
e.g.
iframe = driver.find_element_by_id('destination_publishing_iframe_autodesk_0')
driver.switch_to.frame(iframe)
driver.find_element_by_name('userName').send_keys('xxx')
See the switch_to function here : https://selenium-python.readthedocs.io/api.html?highlight=iframe
For reference:
python selenium cant find iframe xpath
https://seleniumwithjavapython.wordpress.com/selenium-with-python/intermediate-topics/handling-iframes-in-a-webpage/
I am taking a selenium course online and we are automating log ins. The lecture is outdated and the website we are using is updated (Quora).
The problem is, the input fields have randomly generated class names which makes the "find_element_by_id method useless.
I can not figure out how to log in for different sessions. How would I select these fields during different sessions ( that have different class names ). This is what my code looks like. Thank you in advance. I tried asking the course instructor but he hasnt responded all week.
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome(executable_path = r'/Users/mpbro17/Desktop/chromedriver')
driver.get('https://www.quora.com')
driver.find_element_by_id("__w2_fy0VXDC_email").clear()
driver.find_element_by_id("__w2_fy0VXDC_email").send_keys("programmingdude183#gmail.com")
driver.find_element_by_id("__w2_fy0VXDC_password").clear()
driver.find_element_by_id("__w2_fy0VXDC_password").send_keys("testpassword1")
driver.find_element_by_id("__w2_fy0VXDC_submit_button").click()
ERROR
Traceback (most recent call last):
File "/Users/mbpro/Desktop/projects/scraping_course/test_quora_login.py", line 13, in <module>
driver.find_element_by_name("email").clear()
File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", line 92, in clear
self._execute(Command.CLEAR_ELEMENT)
File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", line 494, in _execute
return self._parent.execute(command, params)
File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 236, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 192, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidElementStateException: Message: invalid element state: Element is not currently interactable and may not be manipulated
(Session info: chrome=53.0.2785.143)
(Driver info: chromedriver=2.24.417412 (ac882d3ce7c0d99292439bf3405780058fcca0a6),platform=Mac OS X 10.11.5 x86_64)
Try css selectors instead:
driver.find_element_by_css_selector('[id$="email"]')
driver.find_element_by_css_selector('[id$="password"]')
driver.find_element_by_css_selector('[id$="submit_button"]')
You could have used find_element_by_name method to find your elements (email and password) and cssSelector for Login button. I think, this won't make you disheartened.
driver.find_element_by_name("email").clear()
driver.find_element_by_name("email").send_keys("programmingdude183#gmail.com")
driver.find_element_by_name("password").clear()
driver.find_element_by_name("password").send_keys("testpassword1")
driver.find_element_by_css_selector("input[value='Login']").click()
You have to go with name as it standard.However there are two elements present if you go by name property.
So go by the below xpath
//div[#class='regular_login']//input[#name='email']
//div[#class='regular_login']//input[#name='password']
//div[#class='regular_login']//input[contains(#class,'submit')]
I'm attempting to create an automated login script to netflix website: https://www.netflix.com/it/
That's the code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Firefox()
while True:
driver.get("https://www.netflix.com/it/")
login = driver.find_element_by_css_selector(".authLinks.signupBasicHeader")
login.click()
element = driver.find_element_by_name("email")
element.send_keys("test1#email.com")
submit = driver.find_element_by_css_selector(".btn.login-button.btn-submit.btn-small")
submit.click()
element2 = driver.find_element_by_name("password")
element2.send_keys("test1")
submit.click()
But sometimes it works and sometimes it doesn't and it raises this exception:
Traceback (most recent call last):
File "netflix.py", line 35, in <module>
submit.click()
File "/home/user/.local/lib/python3.5/site-packages/selenium/webdriver/remote/webelement.py", line 72, in click
self._execute(Command.CLICK_ELEMENT)
File "/home/user/.local/lib/python3.5/site-packages/selenium/webdriver/remote/webelement.py", line 461, in _execute
return self._parent.execute(command, params)
File "/home/user/.local/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 236, in execute
self.error_handler.check_response(response)
File "/home/user/.local/lib/python3.5/site-packages/selenium/webdriver/remote/errorhandler.py", line 192, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotVisibleException: Message: Element is not currently visible and so may not be interacted with
Stacktrace:
at fxdriver.preconditions.visible (file:///tmp/tmp7otgskq8/extensions/fxdriver#googlecode.com/components/command-processor.js:10092)
at DelayedCommand.prototype.checkPreconditions_ (file:///tmp/tmp7otgskq8/extensions/fxdriver#googlecode.com/components/command-processor.js:12644)
at DelayedCommand.prototype.executeInternal_/h (file:///tmp/tmp7otgskq8/extensions/fxdriver#googlecode.com/components/command-processor.js:12661)
at DelayedCommand.prototype.executeInternal_ (file:///tmp/tmp7otgskq8/extensions/fxdriver#googlecode.com/components/command-processor.js:12666)
at DelayedCommand.prototype.execute/< (file:///tmp/tmp7otgskq8/extensions/fxdriver#googlecode.com/components/command-processor.js:12608)
The exception says that a part of the web page is invisible (even if that part IS in the page actually)... It's a sort of bug.
How can I bypass this?
No need to put the test code inside while loop, since login form should be submitted once. I guess when it works successfully, next iteration gives error since there is no form.
You are also clicking the submit button before and after password entry. It is better to click after form is fully filled, if possible and not testing empty form.
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("https://www.netflix.com/it/")
login = driver.find_element_by_css_selector(".authLinks.signupBasicHeader")
login.click()
element = driver.find_element_by_name("email")
element.send_keys("test1#email.com")
submit = driver.find_element_by_css_selector(".btn.login-button.btn-submit.btn-small")
submit.click()
element2 = driver.find_element_by_name("password")
element2.send_keys("test1")
submit = driver.find_element_by_css_selector(".btn.login-button.btn-submit.btn-small")
submit.click()