We Keep Coding

driver.find_element(By.XPATH, "xpath") not working

I am trying to learn Selenium in Python and I have faced a problem which stopped me from processing.
As you might know, previous versions of Selenium have different syntax comparing the latest one, and I have tried everything to fill the form with my code but nothing happens. I am trying to find XPATH element from [https://demo.seleniumeasy.com/basic-first-form-demo.html] but whatever I do, I cannot type my message into the message field.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service as ChromeService
import time
options = webdriver.ChromeOptions()
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option("useAutomationExtension", False)
service = ChromeService(executable_path="C:/Users/SnappFood/.cache/selenium/chromedriver/win32/110.0.5481.77/chromedriver.exe")
driver = webdriver.Chrome()
driver.maximize_window()
driver.get("https://demo.seleniumeasy.com/basic-first-form-demo.html")
time.sleep(1000)
message_field = driver.find_element(By.XPATH,'//*[#id="user-message"]')
message_field.send_keys("Hello World")
show_message_button = driver.find_element(By.XPATH,'//*[#id="get-input"]/button')
show_message_button.click()
By this code, I expect to fill the message field in the form and click the "SHOW MESSAGE" button to print my typed text, but what happens is that my code only opens a new Chrome webpage with empty field.
I have to mention that I don't get any errors by PyCharm and the code runs with no errors.
I would really appreciate if you help me through this to understand what I am doing wrong.

You need to wait for the page loads correctly.
For this, the best approach is using WebDriverWait:
driver = webdriver.Chrome()
driver.maximize_window()
driver.get("https://demo.seleniumeasy.com/basic-first-form-demo.html")
# Wait for the page to load
wait = WebDriverWait(driver, 10)
wait.until(lambda driver: driver.find_element(By.XPATH, '//*[#id="user-message"]'))
message_field = driver.find_element(By.XPATH,'//*[#id="user-message"]')
message_field.send_keys("Hello World")
show_message_button = driver.find_element(By.XPATH,'//*[#id="get-input"]/button')
show_message_button.click()
Tested, it works fine.
Also you can use time.sleep() to wait the load if you know the loading times:
import time
driver = webdriver.Chrome()
driver.maximize_window()
driver.get("https://demo.seleniumeasy.com/basic-first-form-demo.html")
time.sleep(5)
message_field = driver.find_element(By.XPATH,'//*[#id="user-message"]')
message_field.send_keys("Hello World")
time.sleep(5)
show_message_button = driver.find_element(By.XPATH,'//*[#id="get-input"]/button')
show_message_button.click()
time.sleep(5)

with this xpath you have two elements //*[#id="user-message"] . you need to make it unique. Try below xpath this will work as expected.
message_field = driver.find_element(By.XPATH,'//input[#id="user-message"]')
browser snapshot:

Selenium element not interactable error on headless mode but works without headless

I'm trying to scrape the webpage ted.europa.eu using Python with Selenium to retrieve information from the tenders. The script is supposed to be executed once a day with the new publications. The problem I have is that navigating to the new tenders I need Selenium to apply a filter to get only the ones from the same day the script it's executed. I already have the script for this and works perfectly, the problem is that when I activate the headless mode I get the following error selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable: [object HTMLInputElement] has no size and location
This is the code I have that applies the filter I need:
import sys
import time
import re
from datetime import datetime
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains
from dotenv import load_dotenv
load_dotenv("../../../../.env")
sys.path.append("../src")
sys.path.append("../../../../utils")
from driver import *
from lted import LTED
from runnable import *
# start
print('start...')
counter = 0
start = datetime.now()
# get driver
driver = get_driver_from_url("https://ted.europa.eu/TED/browse/browseByMap.do%22)
actions = ActionChains(driver)
# change language to spanish
WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, "lgId")))
driver.find_element(By.ID, "lgId").click()
driver.find_element(By.XPATH, "//select[#id='lgId']/option[text()='español (es)']").click()
# click on "Busqueda avanzada"
WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, "goToSearch")))
driver.find_element(By.ID, "goToSearch").click()
# accept cookies and close tab
WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, "cookie-consent-banner")))
driver.find_element(By.XPATH, "//div[#id='cookie-consent-banner']/div[1]/div[1]/div[2]/a[1]").click()
driver.find_element(By.XPATH, "//div[#id='cookie-consent-banner']/div[1]/div[1]/div[2]/a[1]").click()
# click on specific date and set to today
WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, "publicationDateSpecific")))
element = driver.find_element(By.ID, "publicationDateSpecific")
actions.move_to_element(element).perform()
driver.find_element(By.ID, "publicationDateSpecific").click()
driver.find_element(By.CLASS_NAME, "ui-state-highlight").click()
# click on search
driver.find_element(By.ID, "search").click()
From the imports the only think I need to explain is that from the line from dirver import * comes the method get_driver_from_url() that is used later in the code. This method looks like this:
def get_driver_from_url(url):
options = webdriver.ChromeOptions()
options.add_argument("--no-sandbox")
options.add_argument("--disable-dev-shm-usage")
options.add_argument("--start-maximized")
options.add_argument("--headless")
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=options)
driver.get(url)
return driver
As I said this code works perfectly without the headless mode, but when activated I get the error.
At first got another error and searching on the Internet found out that it could be because the element is not on screen, so I added the argument "--start-maximized" to make sure the Chrome tab is as big as possible and added the ActionChains to use actions.move_to_element(element).perform(), but I get this error on this exact code line.
Also tried changing the line WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, "publicationDateSpecific"))) to WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.ID, "publicationDateSpecific"))) but it just didn't work.
Update: Also tried changing to EC.visibility_of_element_located as mentioned in this post but didn't work either
What am I doing wrong?

This is probably because of the window size.
Try adding this:
chrome_options = Options()
chrome_options.add_argument("--window-size=1920,1080")
chrome_options.add_argument("--start-maximized")
chrome_options.add_argument("--headless")

So, after a long time of try and error, I found that adding
element = driver.find_element(By.ID, "publicationDateSpecific")
driver.execute_script("window.scrollTo(0,"+str(element.location["y"])+")")
makes the script work both in headless mode and normal mode

Press a button from popup window with selenium

import time
from selenium import webdriver
from selenium.webdriver.common.by import By
chrome_options = webdriver.ChromeOptions()
prefs = {"profile.default_content_setting_values.notifications": 2}
chrome_options.add_experimental_option("prefs", prefs)
# Add experimental options to remove "Google Chrome is controlled by automated software" notification
chrome_options.add_experimental_option("useAutomationExtension", False)
chrome_options.add_experimental_option("excludeSwitches", ["enable-automation"])
driver = webdriver.Chrome(r'C:\Users\iwanh\Desktop\Drivers\chromedriver.exe', options=chrome_options)
driver.get("https://www.youtube.com/")
# We use driver.find_element with the help of the By import instead of find_element_by_name or id
accept_all = driver.find_element(By.ID, value="text")
time.sleep(7)
accept_all.submit()
So i have this code, and what i am trying to do is press the accept all button on the screenshot but I think the code doesnt recognize it with the id because it is some what of a popup window?
And here is the inspect element
Inspect Element
Screenshot

Right-click on the element and go to Copy > Copy (full) xpath.
Then use:
from selenium.webdriver.common.by import By
driver.find_elements(By.XPATH, 'xpath_value_here')
Edited to reflect the difference between 'copy xpath' and 'copy full xpath'; indeed, copy full xpath is the correct answer, as per the below comment.

How do I click on button in a new tab? Python Selenium [duplicate]

Since Firefox does not support Control + T anymore for the tab, I started using
driver.execute_script("window.open('URL', 'new_window')")
I am trying to display the title of the different tab I open and switch between them. For the example below, I expect the output to be facebook, google and back to facebook. Right now the output is facebook, facebook and facebook.
I tried the answer from here but it also did not work: Switch back to parent tab using selenium webdriver
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("http://www.facebook.com/")
print(driver.title)
driver.execute_script("window.open('http://google.com', 'new_window')")
print(driver.title)
driver.switch_to.window(driver.window_handles[0])
print(driver.title)
UPDATED:
I tried the follow code and it still did not work.
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("http://www.facebook.com/")
print(driver.title)
window_before = driver.window_handles[0]
driver.execute_script("window.open('http://google.com', 'new_window')")
window_after = driver.window_handles[1]
driver.switch_to.window(window_after)
print(driver.title)

A few words about Tab/Window switching/handling:
Always keep track of the Parent Window handle so you can traverse back later if required as per your usecase.
Always use WebDriverWait with expected_conditions as number_of_windows_to_be(num_windows) before switching between Tabs/Windows.
Always keep track of the Child Window handles so you can traverse whenever required.
Always use WebDriverWait with expected_conditions as title_contains("partial_page_title") before extracting the Page Title.
Here is your own code with some minor tweaks mentioned above:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Firefox(executable_path=r'C:\WebDrivers\geckodriver.exe')
driver.get("http://www.facebook.com/")
print("Initial Page Title is: %s" %driver.title)
windows_before = driver.current_window_handle
driver.execute_script("window.open('http://google.com')")
WebDriverWait(driver, 10).until(EC.number_of_windows_to_be(2))
windows_after = driver.window_handles
new_window = [x for x in windows_after if x != windows_before][0]
driver.switch_to.window(new_window)
WebDriverWait(driver, 20).until(EC.title_contains("G"))
print("Page Title after first window switching is: %s" %driver.title)
driver.close()
driver.switch_to.window(windows_before)
WebDriverWait(driver, 20).until(EC.title_contains("F"))
print("Page Title after second window switching is: %s" %driver.title)
driver.quit()
Console Output:
Initial Page Title is: Facebook – log in or sign up
Page Title after first window switching is: Google
Page Title after second window switching is: Facebook – log in or sign up

window.open will open the link in a new tab. Selenium Firefox driver doesn't have the ability to switch between tabs as there is only one window handle for both of them (unlike Chrome which has 2). If you give the window() command 'specs' parameter with width and height it will open a new window and you will be able to switch.
After opening the new window the driver is still focused on the first one, you need to switch to the new window first.
size = driver.get_window_size();
driver.execute_script("window.open('http://google.com', 'new_window', 'height=argument[0], width=argument[1]')", size['height'], size['width'])
driver.switch_to.window(driver.window_handles[1])
print(driver.title)

I've used driver.getWindowHandles(); to get all windows and driver.switchTo().window(handle); to switch to required one.

This command always takes you to the recently opened new tab in chrome. Tested on latest versions of Chrome and Pytest on mac.
selenium.switch_to.window(selenium.window_handles[-1])

Selenium Switch Tabs

Since Firefox does not support Control + T anymore for the tab, I started using
driver.execute_script("window.open('URL', 'new_window')")
I am trying to display the title of the different tab I open and switch between them. For the example below, I expect the output to be facebook, google and back to facebook. Right now the output is facebook, facebook and facebook.
I tried the answer from here but it also did not work: Switch back to parent tab using selenium webdriver
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("http://www.facebook.com/")
print(driver.title)
driver.execute_script("window.open('http://google.com', 'new_window')")
print(driver.title)
driver.switch_to.window(driver.window_handles[0])
print(driver.title)
UPDATED:
I tried the follow code and it still did not work.
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("http://www.facebook.com/")
print(driver.title)
window_before = driver.window_handles[0]
driver.execute_script("window.open('http://google.com', 'new_window')")
window_after = driver.window_handles[1]
driver.switch_to.window(window_after)
print(driver.title)

A few words about Tab/Window switching/handling:
Always keep track of the Parent Window handle so you can traverse back later if required as per your usecase.
Always use WebDriverWait with expected_conditions as number_of_windows_to_be(num_windows) before switching between Tabs/Windows.
Always keep track of the Child Window handles so you can traverse whenever required.
Always use WebDriverWait with expected_conditions as title_contains("partial_page_title") before extracting the Page Title.
Here is your own code with some minor tweaks mentioned above:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Firefox(executable_path=r'C:\WebDrivers\geckodriver.exe')
driver.get("http://www.facebook.com/")
print("Initial Page Title is: %s" %driver.title)
windows_before = driver.current_window_handle
driver.execute_script("window.open('http://google.com')")
WebDriverWait(driver, 10).until(EC.number_of_windows_to_be(2))
windows_after = driver.window_handles
new_window = [x for x in windows_after if x != windows_before][0]
driver.switch_to.window(new_window)
WebDriverWait(driver, 20).until(EC.title_contains("G"))
print("Page Title after first window switching is: %s" %driver.title)
driver.close()
driver.switch_to.window(windows_before)
WebDriverWait(driver, 20).until(EC.title_contains("F"))
print("Page Title after second window switching is: %s" %driver.title)
driver.quit()
Console Output:
Initial Page Title is: Facebook – log in or sign up
Page Title after first window switching is: Google
Page Title after second window switching is: Facebook – log in or sign up

window.open will open the link in a new tab. Selenium Firefox driver doesn't have the ability to switch between tabs as there is only one window handle for both of them (unlike Chrome which has 2). If you give the window() command 'specs' parameter with width and height it will open a new window and you will be able to switch.
After opening the new window the driver is still focused on the first one, you need to switch to the new window first.
size = driver.get_window_size();
driver.execute_script("window.open('http://google.com', 'new_window', 'height=argument[0], width=argument[1]')", size['height'], size['width'])
driver.switch_to.window(driver.window_handles[1])
print(driver.title)

I've used driver.getWindowHandles(); to get all windows and driver.switchTo().window(handle); to switch to required one.

This command always takes you to the recently opened new tab in chrome. Tested on latest versions of Chrome and Pytest on mac.
selenium.switch_to.window(selenium.window_handles[-1])