We Keep Coding

Automate login with selenium and python, how to correctly extract element for user and password

I am trying to automate the login of a bunch of users to the fitbit site to download their data using the python's module selenium. Unfortunately I do not fully understand by looking at the source html code of the page, how to select the right elements.
My code looking something like this:
driver = webdriver.Firefox()
driver.get("https://accounts.fitbit.com/login")
driver.find_element_by_id("email").send_keys('name.surname#gmail.com')
driver.find_element_by_id("pass").send_keys("myfakepass")
driver.find_element_by_id("loginbutton").click()
But I do not know how to extract the element since driver.find_element_by_id("email") returns the error:
>>> driver.find_element_by_id("email")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 360, in find_element_by_id
return self.find_element(by=By.ID, value=id_)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 976, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: [id="email"]
This makes totally sense to me since if I check the source page of https://accounts.fitbit.com/login I cannot detect any field called "email". But, probably due to my lack of experience, I cannot detect in the html source any of the elements that I need for the login.
Anyone could help me?
Thanks

Your locator seems wrong. Try with below xpath.
Induce WebDriverWait() and wait for element_to_be_clickable()
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//input[#type='email']"))).send_keys('name.surname#gmail.com')
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//input[#type='password']"))).send_keys("myfakepass")
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//button[text()='Login']"))).click()
You need to import below libraries.
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By

There is a code that works.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
driver = webdriver.Chrome('your path')
url = "https://accounts.fitbit.com/login"
driver.get(url)
time.sleep(2)
email = driver.find_element_by_id("ember660")
email.send_keys("Edward#elric.com")
password = driver.find_element_by_id("ember661")
password.send_keys("Password")
login_button = driver.find_element_by_id("ember701").click()
Your program didn't work because of the id which are wrong

I came up with a simple code that works.
It has been very useful to use the "Inspect" command integrated in chrome for finding the id or the xpath relative to each field. Also the package time was needed to avoid errors.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
driver = webdriver.Chrome('path to chromedriver')
url = "https://accounts.fitbit.com/login"
driver.get(url)
time.sleep(2)
email = driver.find_element_by_id("ember659")
email.send_keys("mail#gmail.com")
password = driver.find_element_by_id("ember660")
password.send_keys('actualpassword')
driver.find_element_by_xpath('//*[#id="loginForm"]/div[4]/div').click()

Click on specific button with selenium

Python, Selenium, XPath.
I want to open this page https://www.tesla.com/en_gb/models/design#battery and click the performance button programmatically with python.
Here is what I want to click on:
image showing what I want to click on
My problem is properly describing the button. Maybe I don't understand xpath properly or there's a better method to point to the desired element.
Here's what I tried
from selenium import webdriver
browser = webdriver.Chrome('../Downloads/chromedriver.exe')
browser.get('https://www.tesla.com/en_gb/models/design#battery')
A = browser.find_element_by_xpath('/html/body/div/div/main/div/div/div[2]/div[5]/div/div[1]/div/div[2]/div[2]/div[1]')
A.click();
and I get this error
Traceback (most recent call last):
File "C:\Users\User\Desktop\666.py", line 4, in <module>
A = browser.find_element_by_xpath('/html/body/div/div/main/div/div/div[2]/div[5]/div/div[1]/div/div[2]/div[2]/div[1]')
File "C:\Users\User\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 394, in find_element_by_xpath
return self.find_element(by=By.XPATH, value=xpath)
File "C:\Users\User\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 978, in find_element
'value': value})['value']
File "C:\Users\User\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\User\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div/div/main/div/div/div[2]/div[5]/div/div[1]/div/div[2]/div[2]/div[1]"}
(Session info: chrome=80.0.3987.149)
full xpath {/html/body/div/div/main/div/div/div[2]/div[5]/div/div1/div/div[2]/div[2]/div1}
html of element i want to click on
<div role="button" tabindex="0" class="group--options_block m3-animate--all" aria-label="Performance"><div class="group--options_block_title"><span><p class="group--options_block--name text-loader--content" tabindex="-1">Performance</p></span><p class="group--options_block-container_price text-loader--content price-not-included">£95,800</p></div></div>
im copying and pasting the full xpath of the element i want. is that not the correct way to do this?
edit:
if it works the range should be 367 not 379

Use following xpath
//div[#class='group--options_block_title']/span/p
OR
//p[contains(text(),'Performance')]
OR CSS selector
div[aria-label='Performance']
Don't forget to introduce Implicit or Explicit wait to avoid synchronization issue in your scripts. reference

Please refer below code sometime sites are taking too long while loading so would be great if you use WebDriverWait in your solution. Also its not good practice to use Abs XPath.
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
import time
driver = webdriver.Chrome(executable_path=r"C:\New folder\chromedriver.exe")
driver.get('https://www.tesla.com/en_gb/models/design#battery')
wait = WebDriverWait(driver,30)
element = wait.until(EC.element_to_be_clickable((By.XPATH, "//p[contains(text(),'Performance')]")))
print element.text
element.click()
element1 = wait.until(EC.presence_of_element_located((By.XPATH, "//span[contains(text(),'367')]")))
print element1.text
Output:

selenium python issue - invalid element state

Getting error: "invalid element state" when using Chrome driver for selenium.
What I'm trying to do:
Pass in some data to http://www.dhl.de/onlinefrankierung
My first issue is that when I try to use the .click() method on the checkbox named "Nachnahme" nothing happens, no check is made.
When you make a check manually, the page refreshes and additional fields open up, which is what I'm trying to access.
The second issue, which throws the invalid element state, happens when trying to pass in data using the .send_keys() method.
Here is my code so far:
from selenium import webdriver
driver = webdriver.Chrome('C:\\Users\\Owner\\AppData\\Local\\Programs\\Python\\Python36-32\\Lib\\site-packages\\chromedriver.exe')
driver.get('http://www.dhl.de/onlinefrankierung')
product_element = driver.find_element(by='id', value='bpc_PAK02')
product_element.click()
services_element = driver.find_element(by='id', value='sc_NNAHME')
services_element.click()
address_element_name = driver.find_element(by='name', value='formModel.sender.name')
address_element_name.send_keys("JackBlack")
ERROR:
C:\Users\Owner\AppData\Local\Programs\Python\Python36-32\python.exe
"C:/Users/Owner/Desktop/UpWork/Marvin Sessner/script.py" Traceback
(most recent call last): File "C:/Users/Owner/Desktop/UpWork/Marvin
Sessner/script.py", line 23, in
address_element_name.send_keys("tester") File "C:\Users\Owner\AppData\Roaming\Python\Python36\site-packages\selenium\webdriver\remote\webelement.py",
line 352, in send_keys
'value': keys_to_typing(value)}) File "C:\Users\Owner\AppData\Roaming\Python\Python36\site-packages\selenium\webdriver\remote\webelement.py",
line 501, in _execute
return self._parent.execute(command, params) File "C:\Users\Owner\AppData\Roaming\Python\Python36\site-packages\selenium\webdriver\remote\webdriver.py",
line 308, in execute
self.error_handler.check_response(response) File "C:\Users\Owner\AppData\Roaming\Python\Python36\site-packages\selenium\webdriver\remote\errorhandler.py",
line 194, in check_response
raise exception_class(message, screen, stacktrace) selenium.common.exceptions.InvalidElementStateException: Message:
invalid element state (Session info: chrome=HIDDEN) (Driver info:
chromedriver=HIDDEN
(5b82f2d2aae0ca24b877009200ced9065a772e73),platform=Windows NT
10.0.15063 x86_64)

Putting just a small sleep between two actions solve the issue. Following code working perfectly fine.
Now before someone downvotes or put a comment about sleep. let me clarify, is this the best solution? No, it's not
But now you know why it was not working, Your action is generating some AJAX request and before it completes you are trying to do another action which is creating the issue.
The good solution would be to write the condition, which waits until that action is complete but meanwhile you have a working temporary solution.
import time
from selenium import webdriver
driver = webdriver.Chrome('h:\\bin\\chromedriver.exe')
driver.get('http://www.dhl.de/onlinefrankierung')
product_element = driver.find_element(by='id', value='bpc_PAK02')
product_element.click()
time.sleep(5)
services_element = driver.find_element(by='id', value='sc_NNAHME')
services_element.click()
time.sleep(5)
address_element_name = driver.find_element(by='name', value='formModel.sender.name')
address_element_name.send_keys("JackBlack")

If you use explicit waits, you can often avoid this error. In particular, if an element can be clicked (an input, button, select, etc.), you can wait for it to be clickable. Here is what will work in your case.
from selenium import webdriver
from selenium.webdriver.common import utils
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
def waitForElementClickable(timeout=5, method='id', locator=None):
try:
element = WebDriverWait(driver, timeout).until(
EC.element_to_be_clickable((method, locator))
)
print("Element", method + '=' + locator, "can be clicked.")
return element
except Exception:
print("Element", method + '=' + locator, "CANNOT be clicked.")
raise
options = Options()
options.add_argument('--disable-infobars')
driver = webdriver.Chrome(chrome_options=options)
driver.get('http://www.dhl.de/onlinefrankierung')
product_element = waitForElementClickable(method='id', locator='bpc_PAK02')
product_element.click()
services_element = waitForElementClickable(method='id', locator='sc_NNAHME')
services_element.click()
address_element_name = waitForElementClickable(method='name', locator='formModel.sender.name')
address_element_name.send_keys("JackBlack")

Python Selenium - 'Unable to locate element' after made visible

I need your help. I'm trying to scrape some data from tripadvisor using Selenium in Python 2.7. However, I'm getting stuck at one point.
After browsing to the correct page, I'm trying to filter the hotels on certain prices. To do this, you do a mouse over or click on 'price' and then select the appropiate value like (€3 - € 13).
After clicking on price and then the value. I'm getting the error that the element is not visible or unable to locate, while it is clearly visible.
code
from urllib import urlopen
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
city = 'nha thrang'
url = 'http://www.tripadvisor.nl/Hotels'
driver = webdriver.Firefox()
# open browser
driver.get(url)
time.sleep(5)
# insert city & dates
driver.find_element_by_id('searchbox').send_keys(city)
driver.find_element_by_id('date_picker_in_188616').click()
driver.find_elements_by_class_name('day')[15].click()
driver.find_element_by_id('date_picker_out_188616').click()
driver.find_elements_by_class_name('day')[16].click()
time.sleep(5)
# click search
driver.find_element_by_id('SUBMIT_HOTELS').click()
# close popup
time.sleep(5)
try:
driver.switch_to.window(driver.window_handles[1])
driver.close()
driver.switch_to.window(driver.window_handles[0])
except:
''
# click on 'price'. Works!
driver.find_element_by_xpath('//div[starts-with(#class, "JFY_hotel_filter_icon enabled price sprite-price")]').click()
# click on particular price. doesn't work.
driver.find_element_by_xpath('//div[starts-with(#class, "jfy_tag_style jfy_filter_p_4 jfy_cloud")]').click()
Error
Traceback (most recent call last):
File "<pyshell#30>", line 1, in <module>
driver.find_element_by_xpath('//div[starts-with(#class, "jfy_tag_style jfy_filter_p_4 jfy_cloud")]').click()
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 230, in find_element_by_xpath
return self.find_element(by=By.XPATH, value=xpath)
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 662, in find_element
{'using': by, 'value': value})['value']
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 173, in execute
self.error_handler.check_response(response)
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 166, in check_response
raise exception_class(message, screen, stacktrace)
NoSuchElementException: Message: Unable to locate element: {"method":"xpath","selector":"//div[starts-with(#class, \"jfy_tag_style jfy_filter_p_4 jfy_cloud\")]"}
Stacktrace:
at FirefoxDriver.prototype.findElementInternal_ (file:///c:/users/j6057~1.kro/appdata/local/temp/tmpdgovsc/extensions/fxdriver#googlecode.com/components/driver-component.js:9641:26)
at FirefoxDriver.prototype.findElement (file:///c:/users/j6057~1.kro/appdata/local/temp/tmpdgovsc/extensions/fxdriver#googlecode.com/components/driver-component.js:9650:3)
at DelayedCommand.prototype.executeInternal_/h (file:///c:/users/j6057~1.kro/appdata/local/temp/tmpdgovsc/extensions/fxdriver#googlecode.com/components/command-processor.js:11635:16)
at DelayedCommand.prototype.executeInternal_ (file:///c:/users/j6057~1.kro/appdata/local/temp/tmpdgovsc/extensions/fxdriver#googlecode.com/components/command-processor.js:11640:7)
at DelayedCommand.prototype.execute/< (file:///c:/users/j6057~1.kro/appdata/local/temp/tmpdgovsc/extensions/fxdriver#googlecode.com/components/command-processor.js:11582:5)

You need to apply multiple changes to make it work:
use Chrome() driver to avoid opening multiple windows after clicking "Search"
don't use hardcoded time.sleep() intervals - use "Explicit Waits"
date picker element ids are changing, you need to use starts-with() to find them
use ActionChains() to hover the price element and wait for range to become visible
Working code (selecting "USD 25 - 50" range):
from selenium import webdriver
from selenium.webdriver import ActionChains
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
city = 'nha thrang'
url = 'http://www.tripadvisor.nl/Hotels'
driver = webdriver.Chrome()
driver.get(url)
# insert city & dates
searchbox = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, 'searchbox')))
searchbox.send_keys(city)
driver.find_element_by_xpath('//span[starts-with(#id, "date_picker_in_")]').click()
driver.find_elements_by_class_name('day')[15].click()
driver.find_element_by_xpath('//span[starts-with(#id, "date_picker_out_")]').click()
driver.find_elements_by_class_name('day')[16].click()
# click search
driver.find_element_by_id('SUBMIT_HOTELS').click()
# select price range
price = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, '//div[starts-with(#class, "JFY_hotel_filter_icon enabled price sprite-price")]')))
ActionChains(driver).move_to_element(price).perform()
price_range = WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH, '(//div[contains(#class, "jfy_filter_bar_price")]//div[#value="p 8"])[last()]')))
price_range.click()
Results into:

i got the same Traceback , try add this before find your elements:
driver.switch_to_window(driver.window_handles[1])#locate the first new page (handles)
anyway it works for me

selecting element in python selenium

I'm trying to log onto a webpage with python selenium. I've found an element and it is enabled, but when I try to send_keys() to it I get an error. The main thing (I think) in the error output is
selenium.common.exceptions.ElementNotVisibleException: Message: Element is not currently visible and so may not be interacted with
My code is
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import contextlib
with contextlib.closing(webdriver.Firefox()) as driver:
driver.get('http://www.etoro.com/au')
elem = driver.find_element_by_class_name('inputUsername')
print 'enabled:', elem.is_enabled()
print 'selected:', elem.is_selected()
elem.send_keys('myusername')
And the output is
enabled: True
selected: False
Traceback (most recent call last):
File "3_trying_again.py", line 10, in <module>
elem.send_keys('ianafterglow')
File "/Users/ian/miniconda/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", line 303, in send_keys
self._execute(Command.SEND_KEYS_TO_ELEMENT, {'value': typing})
File "/Users/ian/miniconda/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", line 385, in _execute
return self._parent.execute(command, params)
File "/Users/ian/miniconda/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 173, in execute
self.error_handler.check_response(response)
File "/Users/ian/miniconda/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 166, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotVisibleException: Message: Element is not currently visible and so may not be interacted with
Stacktrace:
at fxdriver.preconditions.visible (file:///var/folders/5b/ym07nh6d74gcn_773ynwqkth0000gn/T/tmpN1MV8l/extensions/fxdriver#googlecode.com/components/command-processor.js:8959:12)
at DelayedCommand.prototype.checkPreconditions_ (file:///var/folders/5b/ym07nh6d74gcn_773ynwqkth0000gn/T/tmpN1MV8l/extensions/fxdriver#googlecode.com/components/command-processor.js:11618:15)
at DelayedCommand.prototype.executeInternal_/h (file:///var/folders/5b/ym07nh6d74gcn_773ynwqkth0000gn/T/tmpN1MV8l/extensions/fxdriver#googlecode.com/components/command-processor.js:11635:11)
at DelayedCommand.prototype.executeInternal_ (file:///var/folders/5b/ym07nh6d74gcn_773ynwqkth0000gn/T/tmpN1MV8l/extensions/fxdriver#googlecode.com/components/command-processor.js:11640:7)
at DelayedCommand.prototype.execute/< (file:///var/folders/5b/ym07nh6d74gcn_773ynwqkth0000gn/T/tmpN1MV8l/extensions/fxdriver#googlecode.com/components/command-processor.js:11582:5)
So, what do I need to do?

To make the username field to be visible, you need to move cursor to the login link:
....
driver.get('http://www.etoro.com/au')
action = webdriver.ActionChains(driver)
action.move_to_element(driver.find_element_by_xpath(
'.//a[#class="top-link"]/span[text()="Login"]'
))
action.perform()
# TODO Need to wait until the `inputUsername` field is visible
elem = driver.find_element_by_class_name('inputUsername')
...

You can use explicit waits:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
element = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.CLASSNAME, "inputUsername"))
)
...

I know that this problem is solved ,
I got stuck in similar problem and same error
I have fixed it by just make my script sleep for 2 seconds then resume it was just Connection speed problem
...
time.sleep(2)
...
don't forget to import time module
import time
wish that help anyone in future :D

I had similar issue, selenium was not able to focus and open the login modal. Instead it was focusing on the next element. This was the locator I was using:
elem = browser.find_element_by_xpath("//nav[2]/ul/li[3]/a").click()
I just changed [3] with [2] and it was able to locate the element and open the modal:
elem = browser.find_element_by_xpath("//nav[2]/ul/li[2]/a").click()