We Keep Coding

unable to access HTML Select Option using selenium getting error. ElementNotInteractableException

I am trying to scrape data from this File-tuning dynamic website which is loading data through javascript (ajax) requests.
what is want to do is that It selects cars from type and then selects make, model, engine iteratively, and then I want to scrape data for each make, model, and engine.
Here is the code I write to select cars from type
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support.select import Select
PATH = "C:\SeleniumDrivers\geckodriver.exe"
driver = webdriver.Firefox(executable_path=PATH)
driver.get("https://file-tuning.com/chiptuning")
type_element_select = driver.find_element_by_id("type")
action = ActionChains(driver)
action.move_to_element(type_element_select)
action.click(type_element_select)
action.perform()
action.move_to_element(Select(type_element_select).select_by_value("cars"))
action.click(Select(type_element_select).select_by_value("cars"))
action.perform()
The error I get:
Traceback (most recent call last):
File "D:\Python\selenium\test.py", line 27, in <module>
action.move_to_element(Select(type_element_select).select_by_value("cars"))
File "C:\Users\Umair\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\support\select.py", line 82, in select_by_value
self._setSelected(opt)
File "C:\Users\Umair\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\support\select.py", line 212, in _setSelected
option.click()
File "C:\Users\Umair\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\remote\webelement.py", line 80, in click
self._execute(Command.CLICK_ELEMENT)
File "C:\Users\Umair\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\remote\webelement.py", line 633, in _execute
return self._parent.execute(command, params)
File "C:\Users\Umair\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\Umair\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotInteractableException: Message: Element <option> could not be scrolled into view
I also try this
types_element = driver.find_element_by_id("type")
types_object = Select(types_element)
types_object.select_by_visible_text("Cars")
but it also gives me the same exception.
how I can scrape through this site?
already have seen and tried other StackOverflow related question but didn't work out for me.

You need to use browser in full screen mode :
driver.maximize_window()
so that all web elements would be visible to Selenium.
Sample code :
PATH = "C:\SeleniumDrivers\geckodriver.exe"
driver = webdriver.Firefox(executable_path=PATH)
driver.maximize_window()
driver.implicitly_wait(30)
driver.get("https://file-tuning.com/chiptuning")
wait = WebDriverWait(driver, 10)
type_element_select = driver.find_element_by_id("type")
action = ActionChains(driver)
action.move_to_element(type_element_select)
action.click(type_element_select)
action.perform()
action.move_to_element(Select(type_element_select).select_by_value("cars"))
action.click(Select(type_element_select).select_by_value("cars"))
action.perform()

Program runs fine in debugging mode, but running normally gives errors

This is my current code, but whenever I run it I get an error on the last line
stale element reference: element is not attached to the page document
from selenium import webdriver
url = "https://otctransparency.finra.org/otctransparency/OtcDownload"
driver.get(url)
driver.maximize_window()
driver.implicitly_wait(5)
agree = driver.find_elements_by_xpath("//button[#class='btn btn-warning']")[0]
agree.click()
nonats = driver.find_element_by_link_text('OTC (Non-ATS) Download')
nonats.click()
driver.find_element_by_xpath("//img[#src='./assets/icon_download.png']").click()
driver.switch_to.window(driver.window_handles[0])
driver.find_element_by_xpath("(//div[#class='checkbox-inline'])[2]").click()
driver.find_element_by_xpath("(//div[#class='checkbox-inline'])[1]").click()
driver.implicitly_wait(5)
button = driver.find_element_by_xpath("//img[#src='./assets/icon_download.png']")
print(button.is_displayed())
button.click()
When I run my code in debugging mode line by line, everything works fine without any errors. Any help would be great.
Edit: This is my stack trace
Traceback (most recent call last):
File "C:\Users\derpe\Desktop\python projects personal\testing finra\untitled1.py", line 31, in <module>
button.click()
File "C:\Users\derpe\anaconda3\lib\site-packages\selenium\webdriver\remote\webelement.py", line 80, in click
self._execute(Command.CLICK_ELEMENT)
File "C:\Users\derpe\anaconda3\lib\site-packages\selenium\webdriver\remote\webelement.py", line 633, in _execute
return self._parent.execute(command, params)
File "C:\Users\derpe\anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\derpe\anaconda3\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
StaleElementReferenceException: stale element reference: element is not attached to the page document
(Session info: chrome=86.0.4240.75)

The checkbox clicks are triggering a page refresh while you're searching for the download link.
Call sleep to allow the refresh to finish.
driver.implicitly_wait(5)
import time # add this
time.sleep(1) # add this
button = driver.find_element_by_xpath("//img[#src='./assets/icon_download.png']")
print(button.is_displayed())
button.click()
I tried other selenium waits, but they did not work for me. Probably because the element search succeeds before the refresh starts but still clicks to late.

Unable to find element and input value using selenium

My objective is to take a value (in my case a shipment tracking #) and input it into a tracking field on a website using selenium. I am unable to input the value and get the following error message:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
chromedriver = "/Users/GUVA/Downloads/chromedriver"
driver = webdriver.Chrome(chromedriver)
driver.get("https://www.17track.net/en")
#data = df.values[1] # grabbing one tracking number from an excel file
# click "agree" to close window
python_button = driver.find_elements_by_xpath('//*[#id="modal-gdpr"]/div/div/div[3]/button')[0]
python_button.click()
# enter tracking number into text box
que=driver.find_element_by_id('//input[#id="jsTrackBox"]')
que.send_keys("data")
Traceback (most recent call last):
File "/Users/GUVA/PycharmProjects/Shipment_Tracking/Track Shipment.py", line 26, in <module>
que=driver.find_element_by_id('//input[#id="jsTrackBox"]')
File "/Users/GUVA/PycharmProjects/Shipment_Tracking/venv/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 360, in find_element_by_id
return self.find_element(by=By.ID, value=id_)
File "/Users/GUVA/PycharmProjects/Shipment_Tracking/venv/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 976, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "/Users/GUVA/PycharmProjects/Shipment_Tracking/venv/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/Users/GUVA/PycharmProjects/Shipment_Tracking/venv/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidSelectorException: Message: invalid selector: An invalid or illegal selector was specified
(Session info: chrome=83.0.4103.61)
Any suggestions please?

//input[#id="jsTrackBox"] is an xpath. So you need to fetch the element by using find_element_by_xpath method(where you are currently using find_element_by_id method).
Your code should be like:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
chromedriver = "/Users/GUVA/Downloads/chromedriver"
driver = webdriver.Chrome(chromedriver)
driver.get("https://www.17track.net/en")
#data = df.values[1] # grabbing one tracking number from an excel file
# click "agree" to close window
python_button = driver.find_elements_by_xpath('//*[#id="modal-gdpr"]/div/div/div[3]/button')[0]
python_button.click()
# enter tracking number into text box
que=driver.find_element_by_xpath("//div[#id='jsTrackBox']//div[#class='CodeMirror-scroll']")
que.click()
que.send_keys("data")

Python selenium unable to find whatsapp web textarea by xpath

I'm building a whatsapp automation script using python selenium. I'm unable to find the message text area to send message.
My code :
from selenium import webdriver
import pandas as pd
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
import time
import os
import mysql.connector
driver = webdriver.Chrome('/Users/evilslab/Downloads/chromedriver 3')
driver.get('https://web.whatsapp.com/send?phone=0097155960&text&source&data&app_absent')
time.sleep(10)
text = "Hello testing"
inp_xpath = '//*[#id="main"]/footer/div[1]/div[2]'
input_box = driver.find_element_by_xpath(inp_xpath)
time.sleep(2)
input_box.send_keys(text + Keys.ENTER)
time.sleep(2)
The terminal is :
Traceback (most recent call last):
File "/Users/evilslab/Documents/Websites/www.futurepoint.dev.cc/dobuyme/classy/whatsapp-selenium.py", line 18, in <module>
input_box = driver.find_element_by_xpath(inp_xpath)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 394, in find_element_by_xpath
return self.find_element(by=By.XPATH, value=xpath)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 976, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[#id="main"]/footer/div[1]/div[2]"}
(Session info: chrome=81.0.4044.138)

May be your xpath is wrong. When I inspect and get xpath I get //*[#id="main"]/footer/div[1]/div[2]/div/div[2]

i guess there is a problem with the xpath
find how to get specific xpath from here Is there a way to get the XPath in Google Chrome? , sometimes whatsapp change their class name

Error while scrolling to the bottom of a Web page using selenium

I'd like to scroll to the bottom of a page using Selenium with the Firefox driver.
Here is what I have so far:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
browser=webdriver.Firefox()
browser.get('http://nostarch.com')
htmlElem=browser.find_element_by_tag_name('html')
htmlElem.send_keys(Keys.END) #scrolls to bottom
Unfortunately, my code fails with the following exception:
Traceback (most recent call last):
File "<pyshell#26>", line 1, in <module>
htmlElem.send_keys(Keys.END) #scrolls to bottom
File "C:\Users\academy\AppData\Local\Programs\Python\Python35\lib\site-
packages\selenium\webdriver\remote\webelement.py", line 352, in send_keys
'value': keys_to_typing(value)})
File "C:\Users\academy\AppData\Local\Programs\Python\Python35\lib\site-
packages\selenium\webdriver\remote\webelement.py", line 501, in _execute
return self._parent.execute(command, params)
File "C:\Users\academy\AppData\Local\Programs\Python\Python35\lib\site-
packages\selenium\webdriver\remote\webdriver.py", line 308, in execute
self.error_handler.check_response(response)
File "C:\Users\academy\AppData\Local\Programs\Python\Python35\lib\site-
packages\selenium\webdriver\remote\errorhandler.py", line 194, in
check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotInteractableException: Message:
Element is not visible

Use WebDriverWait (explicit wait) till element to be visible i dont know too much about python so give just general syntax
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
browser=webdriver.Firefox()
browser.get('http://nostarch.com')
htmlElem=WebDriverWait(browser, 5).until(
EC.presence_of_element_located((By.find_element_by_tag_name, "html"))
htmlElem.send_keys(Keys.END)

To scroll to the bottom of a page using Selenium you can use :
Python
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
Java
((JavascriptExecutor)driver).executeScript("window.scrollTo(0, document.body.scrollHeight);");