We Keep Coding

Selenium python cannot find name or email address element

I am trying to automatically login the website using selenium with python. However, I got error as below.
Traceback (most recent call last):
File "C:\Users\KienThong\Automation\Learning\GmailDemo.py", line 15, in <module>
ID = WebDriverWait(driver, 10).until(
File "C:\Python385\lib\site-packages\selenium\webdriver\support\wait.py", line 89, in until
raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:
My code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
driver = webdriver.Chrome()
driver.get("https://www.englishforum.com/study/login/")
driver.maximize_window()
try:
ID = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.ID, "_xfUid-1-1643985254"))
)
ID.send_keys("dkthong2010#gmail.com")
finally:
driver.quit()

The reason why you are having an error is that the ID that you are using is dynamic and changes everytime the page is loaded. Therefore we need to use a static identifier which we can with name="login"
driver = webdriver.Chrome()
driver.get("https://www.englishforum.com/study/login/")
driver.maximize_window()
try:
ID = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.NAME, "login"))
)
ID.send_keys("dkthong2010#gmail.com")
finally:
driver.quit()

you dont need to tell the driver to wait till the element is present your can simply its xpath as it will executed only after page loads
so use:
ID = driver.find_element(By.XPATH, "/html/body/div[2]/div/div[4]/div/div[2]/div/div/div/div/form/div[1]/div/dl[1]/dd/input")
ID.send_keys("dkthong2010#gmail.com")
using xpath to locating the element is very easy and easy to locate.
just
1.inspect the element
2.right click on the element
3. copy full xpath

To send a character sequence to the Your name or email address field you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
driver.get('https://www.englishforum.com/study/login/')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[href$='dismiss-notice'] > span.button-text"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input[name='login']"))).send_keys("dkthong2010#gmail.com")
Using XPATH:
driver.get('https://www.englishforum.com/study/login/')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[contains(#href, 'dismiss-notice')]/span[#class='button-text']"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#name='login']"))).send_keys("dkthong2010#gmail.com")
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Browser Snapshot:

Automate login with selenium and python, how to correctly extract element for user and password

I am trying to automate the login of a bunch of users to the fitbit site to download their data using the python's module selenium. Unfortunately I do not fully understand by looking at the source html code of the page, how to select the right elements.
My code looking something like this:
driver = webdriver.Firefox()
driver.get("https://accounts.fitbit.com/login")
driver.find_element_by_id("email").send_keys('name.surname#gmail.com')
driver.find_element_by_id("pass").send_keys("myfakepass")
driver.find_element_by_id("loginbutton").click()
But I do not know how to extract the element since driver.find_element_by_id("email") returns the error:
>>> driver.find_element_by_id("email")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 360, in find_element_by_id
return self.find_element(by=By.ID, value=id_)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 976, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: [id="email"]
This makes totally sense to me since if I check the source page of https://accounts.fitbit.com/login I cannot detect any field called "email". But, probably due to my lack of experience, I cannot detect in the html source any of the elements that I need for the login.
Anyone could help me?
Thanks

Your locator seems wrong. Try with below xpath.
Induce WebDriverWait() and wait for element_to_be_clickable()
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//input[#type='email']"))).send_keys('name.surname#gmail.com')
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//input[#type='password']"))).send_keys("myfakepass")
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//button[text()='Login']"))).click()
You need to import below libraries.
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By

There is a code that works.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
driver = webdriver.Chrome('your path')
url = "https://accounts.fitbit.com/login"
driver.get(url)
time.sleep(2)
email = driver.find_element_by_id("ember660")
email.send_keys("Edward#elric.com")
password = driver.find_element_by_id("ember661")
password.send_keys("Password")
login_button = driver.find_element_by_id("ember701").click()
Your program didn't work because of the id which are wrong

I came up with a simple code that works.
It has been very useful to use the "Inspect" command integrated in chrome for finding the id or the xpath relative to each field. Also the package time was needed to avoid errors.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
driver = webdriver.Chrome('path to chromedriver')
url = "https://accounts.fitbit.com/login"
driver.get(url)
time.sleep(2)
email = driver.find_element_by_id("ember659")
email.send_keys("mail#gmail.com")
password = driver.find_element_by_id("ember660")
password.send_keys('actualpassword')
driver.find_element_by_xpath('//*[#id="loginForm"]/div[4]/div').click()

Need Help Trying to identify the username section and pw section of a page (python selenium)

I am essentially scraping chegg and I need help in trying to identify the css id= "emailForSignIn"
I am trying to find a way to detect where the text bar for the login section is for the chegg login page
Below is the function implemented:
def signin(): # Only use this function if you are using new instances of your browser each time
print('>>signing in!')
browser.get('https://www.chegg.com/auth?action=login&redirect=https%3A%2F%2Fwww.chegg.com%2F')
handle_captcha()
time.sleep(2)
email_elem = browser.find_element_by_id('emailForSignIn')
for character in 'alondra_calderon#ymail.com':
email_elem.send_keys(character)
time.sleep(0.1)
time.sleep(2)
password_elem = browser.find_element_by_id('passwordForSignIn')
for character in 'Blueyes97':
password_elem.send_keys(character)
time.sleep(0.1)
time.sleep(2)
browser.find_element_by_name('login').click()
try:
if WebDriverWait(browser, 5).until(EC.presence_of_element_located((By.XPATH, "/html/body/div[1]/div[3]/div[2]/div[2]/div/div[3]/div/oc-component/div/div/div/div[2]/div[1]/div[1]/div/form/div/div/div/div/div[3]/span"))):
print('redirecting back to login')
browser.get('https://www.chegg.com/auth?action=login')
handle_captcha()
signin()
handle_captcha()
except TimeoutException:
pass
if browser.find_element_by_tag_name('h1').text == 'Oops, we\'re sorry!':
return [0]
handle_captcha()
Below is the error:
PS C:\Users\Dami\Desktop\chegg_discord_bot-master> & 'C:\Program Files (x86)\Python38-32\python.exe' 'c:\Users\Dami\.vscode\extensions\ms-python.python-2020.8.101144\pythonFiles\lib\python\debugpy\launcher' '58627' '--' 'c:\Users\Dami\Desktop\cheggDiscordBot.py'
DevTools listening on ws://127.0.0.1:58638/devtools/browser/837850c8-796e-44be-950f-49f667f48f0a
>>signing in!
Traceback (most recent call last):
File "c:\Users\Dami\Desktop\cheggDiscordBot.py", line 288, in <module>
signin()
File "c:\Users\Dami\Desktop\cheggDiscordBot.py", line 257, in signin
email_elem = browser.find_element_by_id('emailForSignIn')
File "C:\Users\Dami\AppData\Roaming\Python\Python38\site-packages\selenium\webdriver\remote\webdriver.py",
line 360, in find_element_by_id
return self.find_element(by=By.ID, value=id_)
File "C:\Users\Dami\AppData\Roaming\Python\Python38\site-packages\selenium\webdriver\remote\webdriver.py",
line 976, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "C:\Users\Dami\AppData\Roaming\Python\Python38\site-packages\selenium\webdriver\remote\webdriver.py",
line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\Dami\AppData\Roaming\Python\Python38\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="emailForSignIn"]"}
(Session info: headless chrome=84.0.4147.125)

To send a character sequence to the Email and Password field you have to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
driver.get('https://www.chegg.com/auth?action=login&redirect=https%3A%2F%2Fwww.chegg.com%2F')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input#emailForSignIn"))).send_keys("alondra_calderon#ymail.com")
driver.find_element_by_css_selector("input#passwordForSignIn").send_keys("passwordForSignIn")
Using XPATH:
driver.get('https://www.chegg.com/auth?action=login&redirect=https%3A%2F%2Fwww.chegg.com%2F')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#id='emailForSignIn']"))).send_keys("alondra_calderon#ymail.com")
driver.find_element_by_xpath("//input[#id='passwordForSignIn']").send_keys("passwordForSignIn")
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Browser Snapshot:

Seems synchronization issue as below error indicates-
Unable to locate element: {"method":"css
selector","selector":"[id="emailForSignIn"]"}
Introduce Explicit wait in your script and change your code as given below:
WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.ID, "emailForSignIn"))).send_keys("alondra_calderon#ymail.com")
WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.ID, "passwordForSignIn"))).send_keys("Blueyes97")
driver.find_element_by_name('login').click()
Note: No need to loop through the email and password Selenium automatically enters the text in same way
Import following packages for that:
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait

Python3 - Selenium unable to find xpath provided

I am using Python 3 and Selenium to grab some image links from a website as below:
import sys
import os
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.proxy import Proxy, ProxyType
chrome_options = Options()
chrome_options.add_argument("--headless")
driver = webdriver.Chrome(chrome_options=chrome_options)
driver.get('https://www.sky.com/tv-guide/20200605/4101-1/107/Efe2-364')
link_xpath = '/html/body/main/div/div[2]/div[2]/div/div/div[2]/div/div[2]/div[1]/div/div/div[2]/div/img'
link_path = driver.find_element_by_xpath(link_xpath).text
print(link_path)
driver.quit()
When parsing this URL you can see the image in question in the middle of the page. When you right click in Google Chrome and inspect element, you can then right click the element itself within Chrome Dev Tools and get the xpath for this image.
All looks in order to me, however when running the above code I get the following error:
Traceback (most recent call last):
File "G:\folder\folder\testfilepy", line 16, in <module>
link_path = driver.find_element_by_xpath(link_xpath).text
File "G:\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 394, in find_element_by_xpath
return self.find_element(by=By.XPATH, value=xpath)
File "G:\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 978, in find_element
'value': value})['value']
File "G:\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "G:\Python36\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/main/div/div[2]/div[2]/div/div/div[2]/div/div[2]/div[1]/div/div/div[2]/div/img"}
(Session info: headless chrome=83.0.4103.61)
Can anyone tell me why Selenium is unable to find the xpath provided?

To extract the src attribute of the image you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
options = webdriver.ChromeOptions()
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
options.add_argument('--headless')
options.add_argument('--window-size=1920,1080')
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get('https://www.sky.com/tv-guide/20200605/4101-1/107/Efe2-364')
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.o-layout__item div.c-bezel.programme-content__image>img"))).get_attribute("src"))
Using XPATH:
options = webdriver.ChromeOptions()
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
options.add_argument('--headless')
options.add_argument('--window-size=1920,1080')
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get('https://www.sky.com/tv-guide/20200605/4101-1/107/Efe2-364')
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[#class='o-layout__item']//div[#class='c-bezel programme-content__image']/img"))).get_attribute("src"))
Console Output:
https://images.metadata.sky.com/pd-image/251eeec2-acb3-4733-891b-60f10f2cc28c/16-9/640
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Reference
You can find a couple of detailed discussion on NoSuchElementException in:
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element while trying to click Next button with selenium
selenium in python : NoSuchElementException: Message: no such element: Unable to locate element

You have the correct xpath, but don't use absolute paths, it's very vulnerable to change. Try this relative xpath : //div[#class="c-bezel programme-content__image"]//img.
And to achieve you mean, please use .get_attribute("src") not .text
driver.get('https://www.sky.com/tv-guide/20200605/4101-1/107/Efe2-364')
element = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, '//div[#class="c-bezel programme-content__image"]//img')))
print(element.get_attribute("src"))
driver.quit()
Or better way, use css selector. This should be faster:
element = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, '.c-bezel.programme-content__image > img')))
Reference : https://selenium-python.readthedocs.io/locating-elements.html

If you are working in headless mode, it usually is a good idea to add window size. Add this line to your options:
chrome_options.add_argument('window-size=1920x1080')

Your xpath seems to be correct. You wasn't able to locate because you forgot to handle the cookie. Try it by yourself. Put the driver on hold for few seconds and click agree to all cookies. And then you will see your element. There are multiple way to handle cookie. I was able to locate xpath by using my own xpath which is cleaner. I visit that element from nearest parent.
Hope this help.

Python Selenium - 'Unable to locate element' after made visible

I need your help. I'm trying to scrape some data from tripadvisor using Selenium in Python 2.7. However, I'm getting stuck at one point.
After browsing to the correct page, I'm trying to filter the hotels on certain prices. To do this, you do a mouse over or click on 'price' and then select the appropiate value like (€3 - € 13).
After clicking on price and then the value. I'm getting the error that the element is not visible or unable to locate, while it is clearly visible.
code
from urllib import urlopen
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
city = 'nha thrang'
url = 'http://www.tripadvisor.nl/Hotels'
driver = webdriver.Firefox()
# open browser
driver.get(url)
time.sleep(5)
# insert city & dates
driver.find_element_by_id('searchbox').send_keys(city)
driver.find_element_by_id('date_picker_in_188616').click()
driver.find_elements_by_class_name('day')[15].click()
driver.find_element_by_id('date_picker_out_188616').click()
driver.find_elements_by_class_name('day')[16].click()
time.sleep(5)
# click search
driver.find_element_by_id('SUBMIT_HOTELS').click()
# close popup
time.sleep(5)
try:
driver.switch_to.window(driver.window_handles[1])
driver.close()
driver.switch_to.window(driver.window_handles[0])
except:
''
# click on 'price'. Works!
driver.find_element_by_xpath('//div[starts-with(#class, "JFY_hotel_filter_icon enabled price sprite-price")]').click()
# click on particular price. doesn't work.
driver.find_element_by_xpath('//div[starts-with(#class, "jfy_tag_style jfy_filter_p_4 jfy_cloud")]').click()
Error
Traceback (most recent call last):
File "<pyshell#30>", line 1, in <module>
driver.find_element_by_xpath('//div[starts-with(#class, "jfy_tag_style jfy_filter_p_4 jfy_cloud")]').click()
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 230, in find_element_by_xpath
return self.find_element(by=By.XPATH, value=xpath)
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 662, in find_element
{'using': by, 'value': value})['value']
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 173, in execute
self.error_handler.check_response(response)
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 166, in check_response
raise exception_class(message, screen, stacktrace)
NoSuchElementException: Message: Unable to locate element: {"method":"xpath","selector":"//div[starts-with(#class, \"jfy_tag_style jfy_filter_p_4 jfy_cloud\")]"}
Stacktrace:
at FirefoxDriver.prototype.findElementInternal_ (file:///c:/users/j6057~1.kro/appdata/local/temp/tmpdgovsc/extensions/fxdriver#googlecode.com/components/driver-component.js:9641:26)
at FirefoxDriver.prototype.findElement (file:///c:/users/j6057~1.kro/appdata/local/temp/tmpdgovsc/extensions/fxdriver#googlecode.com/components/driver-component.js:9650:3)
at DelayedCommand.prototype.executeInternal_/h (file:///c:/users/j6057~1.kro/appdata/local/temp/tmpdgovsc/extensions/fxdriver#googlecode.com/components/command-processor.js:11635:16)
at DelayedCommand.prototype.executeInternal_ (file:///c:/users/j6057~1.kro/appdata/local/temp/tmpdgovsc/extensions/fxdriver#googlecode.com/components/command-processor.js:11640:7)
at DelayedCommand.prototype.execute/< (file:///c:/users/j6057~1.kro/appdata/local/temp/tmpdgovsc/extensions/fxdriver#googlecode.com/components/command-processor.js:11582:5)

You need to apply multiple changes to make it work:
use Chrome() driver to avoid opening multiple windows after clicking "Search"
don't use hardcoded time.sleep() intervals - use "Explicit Waits"
date picker element ids are changing, you need to use starts-with() to find them
use ActionChains() to hover the price element and wait for range to become visible
Working code (selecting "USD 25 - 50" range):
from selenium import webdriver
from selenium.webdriver import ActionChains
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
city = 'nha thrang'
url = 'http://www.tripadvisor.nl/Hotels'
driver = webdriver.Chrome()
driver.get(url)
# insert city & dates
searchbox = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, 'searchbox')))
searchbox.send_keys(city)
driver.find_element_by_xpath('//span[starts-with(#id, "date_picker_in_")]').click()
driver.find_elements_by_class_name('day')[15].click()
driver.find_element_by_xpath('//span[starts-with(#id, "date_picker_out_")]').click()
driver.find_elements_by_class_name('day')[16].click()
# click search
driver.find_element_by_id('SUBMIT_HOTELS').click()
# select price range
price = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, '//div[starts-with(#class, "JFY_hotel_filter_icon enabled price sprite-price")]')))
ActionChains(driver).move_to_element(price).perform()
price_range = WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH, '(//div[contains(#class, "jfy_filter_bar_price")]//div[#value="p 8"])[last()]')))
price_range.click()
Results into:

i got the same Traceback , try add this before find your elements:
driver.switch_to_window(driver.window_handles[1])#locate the first new page (handles)
anyway it works for me