We Keep Coding

selenium python element select from dropdown menu

Trying to select multiple elements from dropdown menu via selenium in python.
Website from URL. But Timeoutexception error is occurring.
I have tried Inspect menu from GoogleChrome. //label[#for="inputGenre"]/parent::div//select[#placeholder="Choose a Category"] gives exactly the select tag that I need. But unfortunately, with selenium I cannot locate any element within this tag. Any ideas why the error occur?
code is below;
slect_element = Select(WebDriverWait(driver, 10).until(EC.element_located_to_be_selected((By.XPATH, '//label[#for="inputGenre"]/parent::div//select[#placeholder="Choose a Category"]'))))
slect_element.select_by_index(1)
slect_element.select_by_value('23')
strangeness is it is possible to locate it and get its text values from code below;
drp_menu=driver.find_elements(By.XPATH,'//label[#for="inputGenre"]/parent::div//div[#class="dropdown-main"]/ul/li')
print(len(drp_menu))
ls_categories=[]
for i in drp_menu:
ls_categories.append(i.get_attribute('innerText'))
print is giving 15 elements, and get_attribute(innerText) gives text of each option element text.
Anyway, Thanks a lot #Prophet

That Select element is hidden and can't be used by Selenium as we use normal Select elements to select drop-down menu items.
Here we need to open the drop-down as we open any other elements by clicking them, select the desired options and click the Search button.
So, these lines are opening that drop down and selecting 2 options in the drop-down menu:
wait.until(EC.element_to_be_clickable((By.XPATH, "//div[#class='form-group'][contains(.,'Category')]//div[#class='dropdown-display-label']"))).click()
wait.until(EC.element_to_be_clickable((By.XPATH, "//div[#class='form-group'][contains(.,'Category')]//li[#data-value='23']"))).click()
wait.until(EC.element_to_be_clickable((By.XPATH, "//div[#class='form-group'][contains(.,'Category')]//li[#data-value='1']"))).click()
The result so far is:
And by finally clicking the Search button the result is:
The entire code is:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
options = Options()
options.add_argument("start-maximized")
webdriver_service = Service('C:\webdrivers\chromedriver.exe')
driver = webdriver.Chrome(service=webdriver_service, options=options)
wait = WebDriverWait(driver, 20)
url = "https://channelcrawler.com/"
driver.get(url)
wait.until(EC.element_to_be_clickable((By.XPATH, "//div[#class='form-group'][contains(.,'Category')]//div[#class='dropdown-display-label']"))).click()
wait.until(EC.element_to_be_clickable((By.XPATH, "//div[#class='form-group'][contains(.,'Category')]//li[#data-value='23']"))).click()
wait.until(EC.element_to_be_clickable((By.XPATH, "//div[#class='form-group'][contains(.,'Category')]//li[#data-value='1']"))).click()
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[type='submit']"))).click()

Changing country while scraping Aliexpress using python selenium

I'm working on a scraping project of Aliexpress, and I want to change the ship to country using selenium,for example change spain to Australia and click Save button and then scrap the page, I already found an answer it worked just I don't know how can I save it by clicking the button save using selenium, any help is highly appreciated. This is my code using for this task :
country_button = driver.find_element_by_class_name('ship-to')
country_button.click()
country_buttonn = driver.find_element_by_class_name('shipping-text')
country_buttonn.click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//li[#class='address-select-item ']//span[#class='shipping-text' and text()='Australia']"))).click()

Well, there are 2 pop-ups there you need to close first in order to access any other elements. Then You can select the desired shipment destination. I used WebDriverWait for all those commands to make the code stable. Also, I used scrolling to scroll the desired destination button before clicking on it and finally clicked the save button.
The code below works.
Just pay attention that after selecting a new destination pop-ups can appear again.
import time
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains
options = Options()
options.add_argument("--start-maximized")
s = Service('C:\webdrivers\chromedriver.exe')
driver = webdriver.Chrome(options=options, service=s)
url = 'https://www.aliexpress.com/'
wait = WebDriverWait(driver, 10)
actions = ActionChains(driver)
driver.get(url)
try:
wait.until(EC.element_to_be_clickable((By.XPATH, "//div[contains(#style,'display: block')]//img[contains(#src,'TB1')]"))).click()
except:
pass
try:
wait.until(EC.element_to_be_clickable((By.XPATH, "//img[#class='_24EHh']"))).click()
except:
pass
wait.until(EC.element_to_be_clickable((By.CLASS_NAME, "ship-to"))).click()
wait.until(EC.element_to_be_clickable((By.CLASS_NAME, "shipping-text"))).click()
ship_to_australia_element = driver.find_element(By.XPATH, "//li[#class='address-select-item ']//span[#class='shipping-text' and text()='Australia']")
actions.move_to_element(ship_to_australia_element).perform()
time.sleep(0.5)
ship_to_australia_element.click()
wait.until(EC.element_to_be_clickable((By.XPATH, "//button[#data-role='save']"))).click()
I mostly used XPath locators here. CSS Selectors could be used as well

Python Selenium click google "I agree" button

I am trying to scrape some google data but I first want to click the 'I agree' button that google pops up. This is the script I use to do that:
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
search_question = input("Ask a question: ")
driver = webdriver.Chrome("*Your Webdriver location*")
driver.wait = WebDriverWait(driver, 5)
driver.get("https://google.com")
time.sleep(1)
agree = driver.wait.until(EC.presence_of_element_located((By.XPATH, '//*[#id="introAgreeButton"]/span/span')))
agree.click()
# time.sleep(0.2)
search = driver.find_element_by_class_name("gLFyf")
search.send_keys(search_question)
search.send_keys(Keys.ENTER)
The problem is selenium doesn't seem to locate the button and therefore I get a timeout error. (I have tried also with find_element_by_xpath and still not working).

If you scroll up in the devtools inspector you'll notice that your element is within an iframe:
You need to switch to that frame first, click your button then switch back to the default content (the main page)
driver.get("https://google.com")
#active the iframe and click the agree button
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, "//iframe")))
agree = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="introAgreeButton"]/span/span')))
agree.click()
#back to the main page
driver.switch_to_default_content()
That works for me.
FYI - There's only 1 iframe on the page, that's why the xpath //iframe works. If there were multiple you'd need to identify it with higher accuracy.

If you already have been agreed,then agree button would not appear. That why it is not be able to find given XPath.
Try this:
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
search_question = input("Ask a question: ")
driver = webdriver.Chrome(".\chromedriver.exe")
driver.wait = WebDriverWait(driver, 5)
driver.get("https://google.com")
time.sleep(3)
# agree = driver.wait.until(EC.presence_of_element_located((By.XPATH, '//*[#id="introAgreeButton"]/span/span')))
# agree.click()
# time.sleep(0.2)
search = driver.find_element_by_class_name("gLFyf")
search.send_keys(search_question)
search.send_keys(Keys.ENTER)

Managed to get it working.
Seems like one of the few elements that are interactive in the popup when you first load the site is the language dropdown, which I found by class name.
Then the Agree button is detectable and you can find it by class too.
driver.get("https://www.google.com")
driver.maximize_window()
time.sleep(2)
#finds+clicks language dropdown, interaction unhides the rest of the popup html
driver.find_element_by_class_name('tHlp8d').click()
time.sleep(2)
#finds+clicks the agree button now that it has become visible
driver.find_element_by_id('L2AGLb').click()
you should now have the standard searchbar etc

i had some problems with clicking pop-ups and the problem turned out to be with that click().
im not sure that it might be the same issue with urs or not but try changing ur click to this :
agree = driver.find_element_by_xpath('//*[#id="introAgreeButton"]/span/span')
driver.execute_script("arguments[0].click();", agree)

The problem was that it didn't change frames here is the soulution code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome('C:\\Users\\gassp\\OneDrive\\Namizje\\Python.projects\\chromedriver.exe')
url = 'https://www.google.com/maps/'
driver.get(url)
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, '//*[#id="consent-bump"]/div/div[1]/iframe')))
agree = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="introAgreeButton"]/span/span')))
agree.click()
#back to the main page
driver.switch_to_default_content()
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="searchboxinput"]'))).send_keys('gostilne')
driver.submit()

selenium chromedriver different values of xpath between terminal and actual driver

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
url = 'https://www.msha.gov/mine-data-retrieval-system'
driver = webdriver.Chrome(executable_path='chromedriver')
driver.get(url)
#driver.find_element_by_xpath('//*[#id="mstr90"]/div[1]/div/div') error
#driver.find_elements_by_xpath('//input') gives 3 while in driver gives 10
I am unable to find element where the input "Search by Mine ID by typing here.." is, the document is fully loaded but it can't locate it. What I want to do is simply pass in an input "0100003" then submit

Iframe is present on your page. Before you interact with inputbox you need to switch on to iframee. Refer below code to resolve your issue.
wait = WebDriverWait(driver, 10)
driver.get("https://www.msha.gov/mine-data-retrieval-system")
driver.switch_to.frame("iframe1")
wait = WebDriverWait(driver, 10)
inputBox = wait.until(EC.element_to_be_clickable((By.XPATH, "//div[#class='mstrmojo-SimpleObjectInputBox-empty']"))).click()
inputBox1 = wait.until(EC.element_to_be_clickable((By.XPATH, "//div[#class='mstrmojo-SimpleObjectInputBox-container mstrmojo-scrollNode']//input")))
inputBox1.send_keys("0100003")
Updated Code to handle dropdown
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div#mstr100,mstrmojo-Popup.mstrmojo.SearchBoxSelector-suggest"))).click()
Note: please add below imports to your solution
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
Output:

The element you are trying to find is inside an iframe, so you will need to switch to that iframe first and then do your find element. Also, it's a best practice to use waits to give pages/elements time to load before a find element timeouts and throws an error.
iframe = WebDriverWait(driver, 15).until(EC.presence_of_element_located((By.CSS_SELECTOR, '#iframe1')))
driver.switch_to.frame(iframe)
mine_id = WebDriverWait(driver, 15).until(EC.presence_of_element_located((By.XPATH, '//*[#id="mstr90"]/div[1]/div/div')))
Then you need to click this element to make it interactable.
mine_id.click()
Once you click then you need to re-find the input box before sending keys.
mine_id_input = WebDriverWait(driver, 15).until(EC.presence_of_element_located((By.CSS_SELECTOR, '#mstr90 input')))
mine_id_input.send_keys('0100003')
To select the suggestion displayed:
suggestion = WebDriverWait(driver, 15).until(EC.presence_of_element_located((By.CSS_SELECTOR, '#mstr100')))
suggestion.click()
if you wanted to continue on interacting outside the iframe after this is done, you will want to switch back out of the iframe like this:
driver.switch_to.default_content()

Selenium Webdriver (Python) - Unable to locate element inside nested iframes

I am trying to scrape some data from an iframe located within a webpage. The URL of the webpage is https://www.nissanoflithiasprings.com/schedule-service. I am trying to access the button shown in the image below:
When I right-click on the button (located inside the iframe) to view the source code, I am able to see the HTML id and name (see screenshot below):
The "id" for the button is "new_customer_button". However, when I use selenium webdriver's driver.find_element_by_id("new_customer_button") to access the button, the code is not able to locate the button inside the iframe and throws the following error:
NoSuchElementException: no such element: Unable to locate element: {"method":"id","selector":"new_customer_button"}
Below is the code that I have tried so far:
from selenium import webdriver
chrome_path = r"C:\Users\gh455\Downloads\chromedriver_win32\chromedriver.exe"
driver = webdriver.Chrome(chrome_path)
driver.get("https://www.nissanoflithiasprings.com/schedule-service")
dest_iframe = driver.find_elements_by_tag_name('iframe')[0]
driver.switch_to.frame(dest_iframe)
driver.find_element_by_id("new_customer_button")
Not sure why this is happening. Any help will be appreciated. Thanks!

The element is inside multiple <iframe> tags, you need to switch to them one by one. You should also maximize the window and use explicit wait as it take some time to load
from selenium.webdriver.support import expected_conditions as ec
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
chrome_path = r"C:\Users\gh455\Downloads\chromedriver_win32\chromedriver.exe"
driver = webdriver.Chrome(chrome_path)
driver.maximize_window()
driver.get("https://www.nissanoflithiasprings.com/schedule-service")
wait = WebDriverWait(driver, 10)
# first frame - by css selector
wait.until(ec.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR, '[src^="https://consumer.xtime.com"]')))
# second frame - by ID
wait.until(ec.frame_to_be_available_and_switch_to_it('xt01'))
driver.find_element_by_id("new_customer_button")

To click() on the element with text as Make · Year · Model as the the desired element is within an nested <iframe>s so you have to:
Induce WebDriverWait for the desired parent frame to be available and switch to it.
Induce WebDriverWait for the desired child frame to be available and switch to it.
Induce WebDriverWait for the desired element_to_be_clickable().
You can use the following solution:
Code Block:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("start-maximized")
chrome_options.add_argument('disable-infobars')
driver = webdriver.Chrome(options=chrome_options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get("https://www.nissanoflithiasprings.com/schedule-service")
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe[src*='com/scheduling']")))
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe[src*='consumerportal']")))
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button.button.button--action.btn.btn-secondary#new_customer_button"))).click()
Browser Snapshot:
Here you can find a relevant discussion on Ways to deal with #document under iframe