We Keep Coding

Locating a website element using selenium webdriver for automation task

I am working on a project using selenium webdriver which requires me to locate an element and click on it. The program starts by entering a website , clicking the searchbar , typing in a preentered string and clicking enter , up to this point everything is successful. The next thing I want it to do is find the first result of the search and click on it. This part I am having trouble with. I have successfully located all elements up to this point but i cant locate this one as an error pops up. Here is my code:
from selenium import webdriver
import time
trackname = input("Track Name: ")
driver = webdriver.Chrome('D:\WebDrivers\chromedriver.exe')
driver.get('https://music.apple.com/us/artist/search/166949667')
time.sleep(2)
searchbox = driver.find_element_by_tag_name('input')
searchbox.send_keys(trackname)
from keyboard import press
press('enter')
time.sleep(2)
result = driver.find_element_by_id('search-list-lockup__description')
result.click()
I have tried locating the element other ways but it wont work , I am guessing that the issue is that after searching I have to tell it to search on that page but I am not sure. Here is the error:
Traceback (most recent call last):
File "D:\Python Projekti\iTunesDataFiller\iTunesDataFiller.py", line 18, in <module>
result = driver.find_element_by_id('search-list-lockup__description')
File "D:\Python App\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 360, in find_element_by_id
return self.find_element(by=By.ID, value=id_)
File "D:\Python App\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 976, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "D:\Python App\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "D:\Python App\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="search-list-lockup__description"]"}
(Session info: chrome=89.0.4389.114)
Process finished with exit code 1
What do I do?

This could be due to the element you want to access not being available.
For example say you load the page, the element could not be visible to selenium. So basically you're trying to click on an invisible element.
I suggest using this template to make sure elements are loaded before accessing them.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import NoSuchElementException
waitshort = WebDriverWait(driver,.5)
wait = WebDriverWait(driver, 20)
waitLonger = WebDriverWait(driver, 100)
visible = EC.visibility_of_element_located
driver = webdriver.Chrome(executable_path='path')
driver.get('link')
element_you_want_to_access = wait.until(visible((By.XPATH,'xpath')))

Selenium timeoutexception with webdriver

First post on here and brand new to Python. I am trying to learn how to scrape data from a website. When you first load the website, a disclaimer window shows up and all I am trying to do is hit the accept button using the browser.find_element_by_id.
I am using the webdriverwait command to wait for the page to load prior to clicking the "Accept" button but I keep getting a Timeoutexception. Here is the code that I currently have:
from selenium import webdriver
#get the chrome webdriver path file
browser = webdriver.Chrome(executable_path=r"C:/Program Files (x86)/Google/Chrome/Application/chromedriver.exe")
browser.get('http://foreclosures.guilfordcountync.gov/')
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
#wait until element is loaded
wait = WebDriverWait(browser, 10)
wait.until(EC.presence_of_element_located((By.ID, "cmdAccept")))
element = browser.find_element_by_id("cmdAccept")
element.click()
Here is the error I keep getting:
Traceback (most recent call last):
File "C:/Users/Abbas/Desktop/Foreclosure_Scraping/Foreclosure_Scraping.py", line 33, in <module>
wait.until(EC.presence_of_element_located((By.ID, "cmdAccept")))
File "C:\Users\Abbas\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\support\wait.py", line 80, in until
raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:
I believe it has something to do with the calling out the ID of the button itself from the website but I honestly do not know. Any help is greatly appreciated.

Your attempts to locate the element are unsuccessful because of they are nested within an iframe. One must tell selenium to switch to the iframe that contains the desired element before attempting to click it or use it in any way. Try the following:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
#get the chrome webdriver path file
browser = webdriver.Chrome(executable_path=r"C:/Program Files (x86)/Google/Chrome/Application/chromedriver.exe")
browser.get('http://foreclosures.guilfordcountync.gov/')
browser.switch_to.frame(browser.find_element_by_name("ctl06"))
wait = WebDriverWait(browser, 10)
wait.until(EC.presence_of_element_located((By.ID, "cmdAccept")))
element = browser.find_element_by_id("cmdAccept")
element.click()

python and selenium send keys

Im trying to add input to a text box when i do try it doesn't find the element and it gives an error i don't know if im selecting the right element this is what i have so far
Traceback (most recent call last):
File "./fl_bot.py", line 22, in <module>
ui.WebDriverWait(browser, 10).until(EC.visibility_of_element_located((By.ID, "#billFirstName")))
File "/Library/Python/2.7/site-packages/selenium/webdriver/support/wait.py", line 80, in until
raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:
the code abouve is the error message im getting
from selenium.webdriver.support import ui
from selenium import webdriver
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
def get_page(model, sku):
url = "https://www.footlocker.com/product/model:"+str(model)+"/sku:"+ str(sku)+"/"
return url
browser = webdriver.Firefox()
page=browser.get(get_page(277097,"8448001"))
browser.find_element_by_xpath("//*[#id='pdp_size_select_mask']").click()
shoesize = ui.WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, 'a.grid_size:nth-child(8)')))
shoesize.click()
browser.find_element_by_xpath("//*[#id='pdp_addtocart_button']").click()
checkout = browser.get('https://www.footlocker.com/shoppingcart/default.cfm?sku=')
checkoutbutton = browser.find_element_by_css_selector('#cart_checkout_button').click()
ui.WebDriverWait(browser, 10).until(EC.visibility_of_element_located((By.ID, "#billFirstName")))
browser.find_element_by_id("#billFirstName").send_keys(Keys.RETURN)
every time it makes it to the end of the script it dosent type and it just stops
[1]: https://www.footlocker.com/checkout/?uri=checkout this is the page im trying to check out on

The FIRST NAME is a mandatory field to be filled up, so instead of send_keys(Keys.RETURN) try to send some text as follows along with the expected_condition as element_to_be_clickable :
ui.WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.XPATH, "//input[#id='billFirstName']")))
browser.find_element_by_xpath("//input[#id='billFirstName']").click()
browser.find_element_by_xpath("//input[#id='billFirstName']").clear()
browser.find_element_by_xpath("//input[#id='billFirstName']").send_keys("user_first_name")

Python Selenium Explicit WebDriverWait function only works with presence_of_element_located

I am trying to use Selenium WebDriverWait in Python to wait for items to load on a webpage , however, using any expected condition apart from presence_of_element_located seems to result in the error
selenium.common.exceptions.WebDriverException: Message: SyntaxError: missing ) in parenthetical
I thought it might be linked to the site I was trying against , however I get the same error on any site - see snippit below where I have replaced presence_of_element_located with visibility_of_element_located and am trying to confirm visibility of the search box on python.org.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
driver = webdriver.Firefox()
driver.get("http://www.python.org")
try:
element = WebDriverWait(driver, 5).until(EC.visibility_of_element_located((By.NAME,"q")))
element.send_keys("pycon")
element.send_keys(Keys.RETURN)
finally:
driver.quit()
The Full stack trace is as below , Any help would be appreciated !
Traceback (most recent call last):
File "C:\dev\test.py", line 51, in <module>
element = WebDriverWait(driver, 5).until(EC.visibility_of_element_located((By.NAME,"q")))
File "C:\Development\python\python35-32\lib\site-packages\selenium-3.0.0b3-py3.5.egg\selenium\webdriver\support\wait.py", line 71, in until
value = method(self._driver)
File "C:\Development\python\python35-32\lib\site-packages\selenium-3.0.0b3-py3.5.egg\selenium\webdriver\support\expected_conditions.py", line 78, in __call__
return _element_if_visible(_find_element(driver, self.locator))
File "C:\Development\python\python35-32\lib\site-packages\selenium-3.0.0b3-py3.5.egg\selenium\webdriver\support\expected_conditions.py", line 98, in _element_if_visible
return element if element.is_displayed() == visibility else False
File "C:\Development\python\python35-32\lib\site-packages\selenium-3.0.0b3-py3.5.egg\selenium\webdriver\remote\webelement.py", line 353, in is_displayed
self)
File "C:\Development\python\python35-32\lib\site-packages\selenium-3.0.0b3-py3.5.egg\selenium\webdriver\remote\webdriver.py", line 465, in execute_script
'args': converted_args})['value']
File "C:\Development\python\python35-32\lib\site-packages\selenium-3.0.0b3-py3.5.egg\selenium\webdriver\remote\webdriver.py", line 236, in execute
self.error_handler.check_response(response)
File "C:\Development\python\python35-32\lib\site-packages\selenium-3.0.0b3-py3.5.egg\selenium\webdriver\remote\errorhandler.py", line 192, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: SyntaxError: missing ) in parenthetical
Update - > After a few comments below i have done some testing on versions and browsers and this issue seems isolated to Python 3 and Firefox , the script works with Python 2.7 and works on both versions of python for Chrome webdriver .

These minor changes work for me.
visibility_of_element_located ===> presence_of_element_located
driver.quit() ===> driver.close()
See the following:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
driver = webdriver.Firefox()
driver.get("http://www.python.org")
try:
element = WebDriverWait(driver, 5).until(EC.presence_of_element_located((By.NAME,"q")))
element.send_keys("pycon")
element.send_keys(Keys.RETURN)
finally:
driver.close()

Copy pasted the same code and it works.
Dint have enough repo to post comments so had to put it in answer section.

Python Selenium - 'Unable to locate element' after made visible

I need your help. I'm trying to scrape some data from tripadvisor using Selenium in Python 2.7. However, I'm getting stuck at one point.
After browsing to the correct page, I'm trying to filter the hotels on certain prices. To do this, you do a mouse over or click on 'price' and then select the appropiate value like (€3 - € 13).
After clicking on price and then the value. I'm getting the error that the element is not visible or unable to locate, while it is clearly visible.
code
from urllib import urlopen
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
city = 'nha thrang'
url = 'http://www.tripadvisor.nl/Hotels'
driver = webdriver.Firefox()
# open browser
driver.get(url)
time.sleep(5)
# insert city & dates
driver.find_element_by_id('searchbox').send_keys(city)
driver.find_element_by_id('date_picker_in_188616').click()
driver.find_elements_by_class_name('day')[15].click()
driver.find_element_by_id('date_picker_out_188616').click()
driver.find_elements_by_class_name('day')[16].click()
time.sleep(5)
# click search
driver.find_element_by_id('SUBMIT_HOTELS').click()
# close popup
time.sleep(5)
try:
driver.switch_to.window(driver.window_handles[1])
driver.close()
driver.switch_to.window(driver.window_handles[0])
except:
''
# click on 'price'. Works!
driver.find_element_by_xpath('//div[starts-with(#class, "JFY_hotel_filter_icon enabled price sprite-price")]').click()
# click on particular price. doesn't work.
driver.find_element_by_xpath('//div[starts-with(#class, "jfy_tag_style jfy_filter_p_4 jfy_cloud")]').click()
Error
Traceback (most recent call last):
File "<pyshell#30>", line 1, in <module>
driver.find_element_by_xpath('//div[starts-with(#class, "jfy_tag_style jfy_filter_p_4 jfy_cloud")]').click()
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 230, in find_element_by_xpath
return self.find_element(by=By.XPATH, value=xpath)
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 662, in find_element
{'using': by, 'value': value})['value']
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 173, in execute
self.error_handler.check_response(response)
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 166, in check_response
raise exception_class(message, screen, stacktrace)
NoSuchElementException: Message: Unable to locate element: {"method":"xpath","selector":"//div[starts-with(#class, \"jfy_tag_style jfy_filter_p_4 jfy_cloud\")]"}
Stacktrace:
at FirefoxDriver.prototype.findElementInternal_ (file:///c:/users/j6057~1.kro/appdata/local/temp/tmpdgovsc/extensions/fxdriver#googlecode.com/components/driver-component.js:9641:26)
at FirefoxDriver.prototype.findElement (file:///c:/users/j6057~1.kro/appdata/local/temp/tmpdgovsc/extensions/fxdriver#googlecode.com/components/driver-component.js:9650:3)
at DelayedCommand.prototype.executeInternal_/h (file:///c:/users/j6057~1.kro/appdata/local/temp/tmpdgovsc/extensions/fxdriver#googlecode.com/components/command-processor.js:11635:16)
at DelayedCommand.prototype.executeInternal_ (file:///c:/users/j6057~1.kro/appdata/local/temp/tmpdgovsc/extensions/fxdriver#googlecode.com/components/command-processor.js:11640:7)
at DelayedCommand.prototype.execute/< (file:///c:/users/j6057~1.kro/appdata/local/temp/tmpdgovsc/extensions/fxdriver#googlecode.com/components/command-processor.js:11582:5)

You need to apply multiple changes to make it work:
use Chrome() driver to avoid opening multiple windows after clicking "Search"
don't use hardcoded time.sleep() intervals - use "Explicit Waits"
date picker element ids are changing, you need to use starts-with() to find them
use ActionChains() to hover the price element and wait for range to become visible
Working code (selecting "USD 25 - 50" range):
from selenium import webdriver
from selenium.webdriver import ActionChains
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
city = 'nha thrang'
url = 'http://www.tripadvisor.nl/Hotels'
driver = webdriver.Chrome()
driver.get(url)
# insert city & dates
searchbox = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, 'searchbox')))
searchbox.send_keys(city)
driver.find_element_by_xpath('//span[starts-with(#id, "date_picker_in_")]').click()
driver.find_elements_by_class_name('day')[15].click()
driver.find_element_by_xpath('//span[starts-with(#id, "date_picker_out_")]').click()
driver.find_elements_by_class_name('day')[16].click()
# click search
driver.find_element_by_id('SUBMIT_HOTELS').click()
# select price range
price = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, '//div[starts-with(#class, "JFY_hotel_filter_icon enabled price sprite-price")]')))
ActionChains(driver).move_to_element(price).perform()
price_range = WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH, '(//div[contains(#class, "jfy_filter_bar_price")]//div[#value="p 8"])[last()]')))
price_range.click()
Results into:

i got the same Traceback , try add this before find your elements:
driver.switch_to_window(driver.window_handles[1])#locate the first new page (handles)
anyway it works for me