We Keep Coding

Unable to find input element with Selenium and Python

I am new in selenium and python.
I trying to find element using selenium but no matter what I tried (xpath, CSS) I always got the following message:
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element
Here the code I wrote :
import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
from time import sleep
from requests_html import HTML
options = Options()
# options.add_argument("--headless")
options.add_argument('disable-notifications')
options.add_argument('--disable-infobars')
options.add_argument('--disable-blink-features=AutomationControlled')
driver = webdriver.Chrome(executable_path=r'C:/Users/XXX/PycharmProjects/chromedriver.exe', options=options)
first_url = "https://www.micromania.fr/jeu-concours-summer-show.html"
driver.get(first_url)
# For the following, it works
refuse = driver.find_element_by_xpath('//*[#id="truste-consent-required"]').click()
time.sleep(2)
But when I tried to do a driver.find_element_by -CSS, Xpath, to find for the mail field to be able to fill it, I met the error message.
HTML of the mail field I tried to reach is there:
<input data-v-0f8f86ae="" type="email" placeholder="Email*" class="input">
UPDATE

You need to first click on the account icon, having class sidebar-login header-link-item icon-account no-decoration color-white. That click will do a XHR network call, which will bring some new elements into page, including email address field. Also, it's wise to wait for elements to load in page before searching them, instead of just searching for them. Also, it's wise to wrap the cookie button dismissal in a try/except block, just in case it's not popping up all the time.
Adapting the following snippet to your code will get you what you're after:
url='https://www.micromania.fr/jeu-concours-summer-show.html'
browser.get(url)
WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, ".sidebar-login.header-link-item.icon-account.no-decoration.color-white"))).click()
email_field = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#title='Adresse email']")))
email_field.click()
email_field.send_keys('my amazing email address')
print('done')
EDIT: for the email field I'm not able to see (restricted to French IPs only):
email_2_field = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[#class='home']/input[#type='email']")))
email_2_field.click()

Given the HTML:
<input data-v-0f8f86ae="" type="email" placeholder="Email*" class="input">
As the desired element is a dynamic element, to send a character sequence within the element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.input[type='email'][placeholder^='Email']"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#class='input' and starts-with(#placeholder, 'Email')][#type='email']"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Python + Selenium) How can I get this thing clicked?

Question.
I'm really new to Python and can't find a way to click this link.
My aim was to click the links one by one, and I got stuck from clicking the first link.
I searched several times and tried even more, but I can't even find what is the problem!
The links lead to a new window(Survey), and the following is the html structure.
<div id="bb_deployment6" class="stream_item active_stream_item" role="listitem" x-aria-selected="true" tabindex="0" style="padding-left: 20px;"><span class="stream_datestamp">1 hour</span><div class="stream_context">Survey [Today] Survey A: Click to submit survey </div><div class="stream_details"></div><div class="stream_context_bottom"></div></div>
<div id="bb_deployment5" class="stream_item" role="listitem" x-aria-selected="false" tabindex="-1" style="padding-left: 20px;"><span class="stream_datestamp">2 hour</span><div class="stream_context">Survey [Today] Survey B: Click to submit survey </div><div class="stream_details"></div><div class="stream_context_bottom"></div></div>
Here's what I've tried
First Shot
from selenium import webdriver
browser =webdriver.Chrome("C:\Pii\selenium\chromedriver.exe")
#Open the Site
browser.get("https://that site")
#Find & Click!!
browser.find_element_by_partial_link_text("Survey").click()
The first error code was
: Message: no such element: Unable to locate element: {"method":"partial link text","selector":"Survey"}
Second Shot: OK Maybe the loading time was too short?
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
browser =webdriver.Chrome("C:\Pii\selenium\chromedriver.exe")
browser.get("https://that site")
#Wait & Click
wait = WebDriverWait(browser, 10)
element = wait.until(EC.presence_of_element_located((By.PARTIAL_LINK_TEXT, "Survey")))
browser.find_element_by_partial_link_text("Survey").click()
and now it said
: selenium.common.exceptions.TimeoutException: Message:
Third Shot: Maybe the click part was the problem because of onclick?
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
browser=webdriver.Chrome("C:\Pii\selenium\chromedriver.exe")
browser.get("https://that site")
wait = WebDriverWait(browser, 10)
element = wait.until(EC.presence_of_element_located((By.PARTIAL_LINK_TEXT, "Survey")))
sample = browser.find_element_by_link_text("Survey")
browser.execute_script("arguments[0].click();",sample)
and it said
: selenium.common.exceptions.TimeoutException: Message:
The same message as above
Fourth Shot: Maybe I should use XPATH instead of text?
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
browser=webdriver.Chrome("C:\Pii\selenium\chromedriver.exe")
browser.get("https://that site")
wait = WebDriverWait(browser, 10)
element = wait.until(EC.presence_of_element_located((By.XPATH, '//*[#id="bb_deployment5"]/div[1]/a')))
sample = browser.find_element_by_xpath('//*[#id="bb_deployment5"]/div[1]/a')
browser.execute_script("arguments[0].click();",sample)
and the result was the same
I think I got something totally wrong, but I can't get what that is.
Any answer would be a great help. Thanks

Since the element is located in a different iframe, you should switch focus to that iframe and then search for the element. This is how you do it:
iframe = browser.find_element_by_class_name('cloud-iframe')
browser.switch_to.frame(iframe)
element = wait.until(EC.presence_of_element_located((By.XPATH, '//*[#id="bb_deployment5"]/div[1]/a')))
element.click()
This should work, but I am not 100% sure as I haven't yet seen the website myself.

Python Selenium element is not visible [duplicate]

I am trying to enter username and password in the following website:
https://www.thegreatcoursesplus.com/sign-in
driver = webdriver.Chrome()
driver.get('https://www.TheGreatCoursesPlus.com/sign-in')
driver.find_element_by_xpath('//h1[#class="sign-in-input"]').click()
This gave following exception:
selenium.common.exceptions.ElementNotVisibleException: Message: element not visible
Then I tried to use java script:
driver.execute_script("document.getElementsByClassName('sign-in-input')[0].click()")
cmd = "document.getElementsByClassName('label-focus')[0].value = 'abc#abc.com'"
driver.execute_script(cmd)
There are no errors but no text is sent to "Email Address" field.
Can someone please guide me on the correct way to enter email address, password and then click "Sign-in".

This error message...
selenium.common.exceptions.ElementNotVisibleException: Message: element not visible
...implies that the desired element was not visible within the HTML DOM while the WebDriver instance was trying to find it.
ElementNotVisibleException
ElementNotVisibleException is thrown when an element is present on the DOM Tree, but it is not visible, and so is not able to be interacted with.
Reason
One possitive take away from ElementNotVisibleException is the fact that the WebElement is present within the HTML and this exception is commonly encountered when trying to click() or read an attribute of an element that is hidden from view.
Solution
As ElementNotVisibleException ensures that the WebElement is present within the HTML so the solution ahead would be two folds as per the next steps as detailed below:
If you next step is to read any attribute of the desired element, then you need to induce WebDriverWait in-conjunction with expected_conditions clause set to visibility_of_element_located as follows:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
my_value = WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH, "element_xpath"))).get_attribute("innerHTML")
If you next step is to invoke click() on the desired element, then you need to induce WebDriverWait in-conjunction with expected_conditions clause set to element_to_be_clickable as follows:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "element_xpath"))).click()
This usecase
The xpath you constructed as //h1[#class="sign-in-input"] doesn't match any node. We need to create unique xpath to locate the elements representing Email Address, Password and Sign In button inducing WebDriverWait. The below code block will help you to achieve the same:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
options = Options()
options.add_argument("start-maximized")
options.add_argument("disable-infobars")
options.add_argument("--disable-extensions")
driver = webdriver.Chrome(chrome_options=options, executable_path="C:\\Utility\\BrowserDrivers\\chromedriver.exe")
driver.get('https://www.TheGreatCoursesPlus.com/sign-in')
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//div[#id='modal']//input[#name='email']"))).send_keys("abc#abc.com")
driver.find_element_by_xpath("//div[#id='modal']//input[#name='password']").send_keys("password")
driver.find_element_by_xpath("//div[#id='modal']//button[#class='color-site sign-in-button']").click()

for username use :
driver.find_element_by_xpath("//input(#type='email')").click()
driver.find_element_by_xpath("//input(#type='email')").send_keys( "username" )
for password use :
driver.find_element_by_xpath("//input(#type='password')").click()
driver.find_element_by_xpath("//input(#type='password')").send_keys( "password" )

There are two problems:
The first one is timing, it takes some time for the form to appear. You you can use explicit wait to solve it.
The second one is that the field id in <input> tag, not <h1> tag, and there are many fields that matches this xpath. I suggest you locate the form holding the fields and use it to locate each field
For the timing issue you can use explicit wait
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome()
driver.get('https://www.TheGreatCoursesPlus.com/sign-in')
form = WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH, '//div[#class="modal-body"]//form')))
form.find_element_by_name('email').send_keys(email)
form.find_element_by_name('password').send_keys(password)
form.find_element_by_name('sign-in-button').click()

Find element by only a part of its XPath using Selenium

I'm trying to find elements which have changing XPaths with only one part staying the same:
//*[#id="foo"]div[2]/div[1]/time
//*[#id="bar"]div/div[2]/div[1]/time
//*[#id="bat"]div[1]div[2]/div[1]/time
I already tried using driver.find_element_by_xpath("//*[contains(text(), '/div[2]/div[1]/time')]") but this doesn't seem to work.
Here is some example HTML:
<div class="entry-container">
<div class="entry-head">
<h3> some text </h3>
<time class="timestamp" datetime="2020-01-23 08:04:32 UTC">
Today at 18:34
</time>
</div>
</div>
I want to get the text from the time element.

The following XPath expression:
//div/div//time
will give you all time elements anywhere having two div elements as ancestor. This works for the example paths you provided.

Try this x path locator:
.//div[#class]/time

To handle dynamic element induce WebDriverWait() and visibility_of_element_located() and following xpath options.
XPath1 :
print(WebDriverWait(driver,10).until(EC.visibility_of_element_located((By.XPATH,"//h3[contains(.,'some text')]/following-sibling::time[1]"))).text)
XPath2 :
print(WebDriverWait(driver,10).until(EC.visibility_of_element_located((By.XPATH,"//h3[contains(.,'some text')]/following::time[1]"))).text)
XPath3 :
print(WebDriverWait(driver,10).until(EC.visibility_of_element_located((By.XPATH,"//div[#class='entry-head']//h3[contains(.,'some text')]/following-sibling::time[1]"))).text)
You need to add following libraries.
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By

try with below code using Xpath and CSS with webdriver wait so element will be loaded in DOM
Imports:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
Xpath -1
element = WebDriverWait(driver, 20).until(
EC.element_to_be_clickable((By.XPATH, "(//time[#class='timestamp'])[1]")))
OR
Xpath -2
element = WebDriverWait(driver, 20).until(
EC.element_to_be_clickable((By.XPATH, "(//time)[1]")))
By CSS
element = WebDriverWait(driver, 20).until(
EC.element_to_be_clickable((By.CSS_SELECTOR , "time.timestamp")))

How to click on the list of the <li> elements in an <ul> elements with selenium in python?

I tried to select 2002 in dropdown menu.
It doesn't work at any late.
I used xpath
driver.find_element_by_xpath("html/body/main/div/form/div[3]/div[1]/section/div[3]/fieldset/div[7]/dl[1]/dd/ul/li[1]/a").click()
but it doesn't work..I tried all the solutions I got...
How can I select this?

If you're able to open dropdown item but unable to click on item, you should try using Explicit Waits with WebDriverWait to wait until this element is visible and enable to click as below :-
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
element = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "ul#ulBirthYear a[data-value='2002']")))
element.click()
Or
element = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.LINK_TEXT, "2002")))
element.click()

First of all, try to avoid using absolute XPATH.
Use something like this:
'//ul[#id="uiBirthYear"]/li/a[#data-value="2002"]'
Also ensure, that the DOM is fully built, before you trying to get/click on this element.
Try to set an implicit wait
driver.implicitly_wait(10)
or an explicit wait (read more: http://selenium-python.readthedocs.io/waits.html)