We Keep Coding

Locating a website element using selenium webdriver for automation task

I am working on a project using selenium webdriver which requires me to locate an element and click on it. The program starts by entering a website , clicking the searchbar , typing in a preentered string and clicking enter , up to this point everything is successful. The next thing I want it to do is find the first result of the search and click on it. This part I am having trouble with. I have successfully located all elements up to this point but i cant locate this one as an error pops up. Here is my code:
from selenium import webdriver
import time
trackname = input("Track Name: ")
driver = webdriver.Chrome('D:\WebDrivers\chromedriver.exe')
driver.get('https://music.apple.com/us/artist/search/166949667')
time.sleep(2)
searchbox = driver.find_element_by_tag_name('input')
searchbox.send_keys(trackname)
from keyboard import press
press('enter')
time.sleep(2)
result = driver.find_element_by_id('search-list-lockup__description')
result.click()
I have tried locating the element other ways but it wont work , I am guessing that the issue is that after searching I have to tell it to search on that page but I am not sure. Here is the error:
Traceback (most recent call last):
File "D:\Python Projekti\iTunesDataFiller\iTunesDataFiller.py", line 18, in <module>
result = driver.find_element_by_id('search-list-lockup__description')
File "D:\Python App\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 360, in find_element_by_id
return self.find_element(by=By.ID, value=id_)
File "D:\Python App\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 976, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "D:\Python App\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "D:\Python App\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="search-list-lockup__description"]"}
(Session info: chrome=89.0.4389.114)
Process finished with exit code 1
What do I do?

This could be due to the element you want to access not being available.
For example say you load the page, the element could not be visible to selenium. So basically you're trying to click on an invisible element.
I suggest using this template to make sure elements are loaded before accessing them.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import NoSuchElementException
waitshort = WebDriverWait(driver,.5)
wait = WebDriverWait(driver, 20)
waitLonger = WebDriverWait(driver, 100)
visible = EC.visibility_of_element_located
driver = webdriver.Chrome(executable_path='path')
driver.get('link')
element_you_want_to_access = wait.until(visible((By.XPATH,'xpath')))

Automate login with selenium and python, how to correctly extract element for user and password

I am trying to automate the login of a bunch of users to the fitbit site to download their data using the python's module selenium. Unfortunately I do not fully understand by looking at the source html code of the page, how to select the right elements.
My code looking something like this:
driver = webdriver.Firefox()
driver.get("https://accounts.fitbit.com/login")
driver.find_element_by_id("email").send_keys('name.surname#gmail.com')
driver.find_element_by_id("pass").send_keys("myfakepass")
driver.find_element_by_id("loginbutton").click()
But I do not know how to extract the element since driver.find_element_by_id("email") returns the error:
>>> driver.find_element_by_id("email")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 360, in find_element_by_id
return self.find_element(by=By.ID, value=id_)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 976, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: [id="email"]
This makes totally sense to me since if I check the source page of https://accounts.fitbit.com/login I cannot detect any field called "email". But, probably due to my lack of experience, I cannot detect in the html source any of the elements that I need for the login.
Anyone could help me?
Thanks

Your locator seems wrong. Try with below xpath.
Induce WebDriverWait() and wait for element_to_be_clickable()
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//input[#type='email']"))).send_keys('name.surname#gmail.com')
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//input[#type='password']"))).send_keys("myfakepass")
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//button[text()='Login']"))).click()
You need to import below libraries.
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By

There is a code that works.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
driver = webdriver.Chrome('your path')
url = "https://accounts.fitbit.com/login"
driver.get(url)
time.sleep(2)
email = driver.find_element_by_id("ember660")
email.send_keys("Edward#elric.com")
password = driver.find_element_by_id("ember661")
password.send_keys("Password")
login_button = driver.find_element_by_id("ember701").click()
Your program didn't work because of the id which are wrong

I came up with a simple code that works.
It has been very useful to use the "Inspect" command integrated in chrome for finding the id or the xpath relative to each field. Also the package time was needed to avoid errors.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
driver = webdriver.Chrome('path to chromedriver')
url = "https://accounts.fitbit.com/login"
driver.get(url)
time.sleep(2)
email = driver.find_element_by_id("ember659")
email.send_keys("mail#gmail.com")
password = driver.find_element_by_id("ember660")
password.send_keys('actualpassword')
driver.find_element_by_xpath('//*[#id="loginForm"]/div[4]/div').click()

Click on specific button with selenium

Python, Selenium, XPath.
I want to open this page https://www.tesla.com/en_gb/models/design#battery and click the performance button programmatically with python.
Here is what I want to click on:
image showing what I want to click on
My problem is properly describing the button. Maybe I don't understand xpath properly or there's a better method to point to the desired element.
Here's what I tried
from selenium import webdriver
browser = webdriver.Chrome('../Downloads/chromedriver.exe')
browser.get('https://www.tesla.com/en_gb/models/design#battery')
A = browser.find_element_by_xpath('/html/body/div/div/main/div/div/div[2]/div[5]/div/div[1]/div/div[2]/div[2]/div[1]')
A.click();
and I get this error
Traceback (most recent call last):
File "C:\Users\User\Desktop\666.py", line 4, in <module>
A = browser.find_element_by_xpath('/html/body/div/div/main/div/div/div[2]/div[5]/div/div[1]/div/div[2]/div[2]/div[1]')
File "C:\Users\User\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 394, in find_element_by_xpath
return self.find_element(by=By.XPATH, value=xpath)
File "C:\Users\User\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 978, in find_element
'value': value})['value']
File "C:\Users\User\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\User\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div/div/main/div/div/div[2]/div[5]/div/div[1]/div/div[2]/div[2]/div[1]"}
(Session info: chrome=80.0.3987.149)
full xpath {/html/body/div/div/main/div/div/div[2]/div[5]/div/div1/div/div[2]/div[2]/div1}
html of element i want to click on
<div role="button" tabindex="0" class="group--options_block m3-animate--all" aria-label="Performance"><div class="group--options_block_title"><span><p class="group--options_block--name text-loader--content" tabindex="-1">Performance</p></span><p class="group--options_block-container_price text-loader--content price-not-included">£95,800</p></div></div>
im copying and pasting the full xpath of the element i want. is that not the correct way to do this?
edit:
if it works the range should be 367 not 379

Use following xpath
//div[#class='group--options_block_title']/span/p
OR
//p[contains(text(),'Performance')]
OR CSS selector
div[aria-label='Performance']
Don't forget to introduce Implicit or Explicit wait to avoid synchronization issue in your scripts. reference

Please refer below code sometime sites are taking too long while loading so would be great if you use WebDriverWait in your solution. Also its not good practice to use Abs XPath.
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
import time
driver = webdriver.Chrome(executable_path=r"C:\New folder\chromedriver.exe")
driver.get('https://www.tesla.com/en_gb/models/design#battery')
wait = WebDriverWait(driver,30)
element = wait.until(EC.element_to_be_clickable((By.XPATH, "//p[contains(text(),'Performance')]")))
print element.text
element.click()
element1 = wait.until(EC.presence_of_element_located((By.XPATH, "//span[contains(text(),'367')]")))
print element1.text
Output:

selenium python issue - invalid element state

Getting error: "invalid element state" when using Chrome driver for selenium.
What I'm trying to do:
Pass in some data to http://www.dhl.de/onlinefrankierung
My first issue is that when I try to use the .click() method on the checkbox named "Nachnahme" nothing happens, no check is made.
When you make a check manually, the page refreshes and additional fields open up, which is what I'm trying to access.
The second issue, which throws the invalid element state, happens when trying to pass in data using the .send_keys() method.
Here is my code so far:
from selenium import webdriver
driver = webdriver.Chrome('C:\\Users\\Owner\\AppData\\Local\\Programs\\Python\\Python36-32\\Lib\\site-packages\\chromedriver.exe')
driver.get('http://www.dhl.de/onlinefrankierung')
product_element = driver.find_element(by='id', value='bpc_PAK02')
product_element.click()
services_element = driver.find_element(by='id', value='sc_NNAHME')
services_element.click()
address_element_name = driver.find_element(by='name', value='formModel.sender.name')
address_element_name.send_keys("JackBlack")
ERROR:
C:\Users\Owner\AppData\Local\Programs\Python\Python36-32\python.exe
"C:/Users/Owner/Desktop/UpWork/Marvin Sessner/script.py" Traceback
(most recent call last): File "C:/Users/Owner/Desktop/UpWork/Marvin
Sessner/script.py", line 23, in
address_element_name.send_keys("tester") File "C:\Users\Owner\AppData\Roaming\Python\Python36\site-packages\selenium\webdriver\remote\webelement.py",
line 352, in send_keys
'value': keys_to_typing(value)}) File "C:\Users\Owner\AppData\Roaming\Python\Python36\site-packages\selenium\webdriver\remote\webelement.py",
line 501, in _execute
return self._parent.execute(command, params) File "C:\Users\Owner\AppData\Roaming\Python\Python36\site-packages\selenium\webdriver\remote\webdriver.py",
line 308, in execute
self.error_handler.check_response(response) File "C:\Users\Owner\AppData\Roaming\Python\Python36\site-packages\selenium\webdriver\remote\errorhandler.py",
line 194, in check_response
raise exception_class(message, screen, stacktrace) selenium.common.exceptions.InvalidElementStateException: Message:
invalid element state (Session info: chrome=HIDDEN) (Driver info:
chromedriver=HIDDEN
(5b82f2d2aae0ca24b877009200ced9065a772e73),platform=Windows NT
10.0.15063 x86_64)

Putting just a small sleep between two actions solve the issue. Following code working perfectly fine.
Now before someone downvotes or put a comment about sleep. let me clarify, is this the best solution? No, it's not
But now you know why it was not working, Your action is generating some AJAX request and before it completes you are trying to do another action which is creating the issue.
The good solution would be to write the condition, which waits until that action is complete but meanwhile you have a working temporary solution.
import time
from selenium import webdriver
driver = webdriver.Chrome('h:\\bin\\chromedriver.exe')
driver.get('http://www.dhl.de/onlinefrankierung')
product_element = driver.find_element(by='id', value='bpc_PAK02')
product_element.click()
time.sleep(5)
services_element = driver.find_element(by='id', value='sc_NNAHME')
services_element.click()
time.sleep(5)
address_element_name = driver.find_element(by='name', value='formModel.sender.name')
address_element_name.send_keys("JackBlack")

If you use explicit waits, you can often avoid this error. In particular, if an element can be clicked (an input, button, select, etc.), you can wait for it to be clickable. Here is what will work in your case.
from selenium import webdriver
from selenium.webdriver.common import utils
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
def waitForElementClickable(timeout=5, method='id', locator=None):
try:
element = WebDriverWait(driver, timeout).until(
EC.element_to_be_clickable((method, locator))
)
print("Element", method + '=' + locator, "can be clicked.")
return element
except Exception:
print("Element", method + '=' + locator, "CANNOT be clicked.")
raise
options = Options()
options.add_argument('--disable-infobars')
driver = webdriver.Chrome(chrome_options=options)
driver.get('http://www.dhl.de/onlinefrankierung')
product_element = waitForElementClickable(method='id', locator='bpc_PAK02')
product_element.click()
services_element = waitForElementClickable(method='id', locator='sc_NNAHME')
services_element.click()
address_element_name = waitForElementClickable(method='name', locator='formModel.sender.name')
address_element_name.send_keys("JackBlack")

Selenium webdriver finding element in next link

open main(home) page and then go to login page using click() function , now i want to find element in this page how could I?
here is my code ...
import unittest,time,re
from selenium import webdriver
from selenium import selenium
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support import expected_conditions as EC
class PythonOrgSearch(unittest.TestCase):
def setUp(self):
#self.selenium = selenium("localhost", 4444, "*firefox","http://www.google.com/")
self.driver = webdriver.Firefox()
def test_search_in_python_org(self):
driver = self.driver
driver.get("https://bitbucket.org/")
elem = driver.find_element_by_id("user-options")
elem = elem.find_element_by_class_name("login-link")
elem.click()
print "check"
#elem = WebDriverWait(driver, 30).until(EC.elementToBeVisible(By.name("username")));
#elem.send_keys("my_username#bitbucket.org")
user_name_field = driver.find_element_by_id('id_username')
password_field = driver.find_element_by_id('id_password')
user_name_field.send_keys('your_user_name')
password_field.send_keys('your_password')
def tearDown(self):
pass
#self.driver.close()
if __name__ == "__main__":
unittest.main()
I got this error (file name python_org_search.py)
E
======================================================================
ERROR: test_search_in_python_org (__main__.PythonOrgSearch)
----------------------------------------------------------------------
Traceback (most recent call last):
File "python_org_search.py", line 25, in test_search_in_python_org
user_name_field = driver.find_element_by_id('id_username')
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 197, in find_element_by_id
return self.find_element(by=By.ID, value=id_)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 681, in find_element
{'using': by, 'value': value})['value']
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 164, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 164, in check_response
raise exception_class(message, screen, stacktrace)
NoSuchElementException: Message: u'Unable to locate element: {"method":"id","selector":"id_username"}' ; Stacktrace:
at FirefoxDriver.prototype.findElementInternal_ (file:///tmp/tmpKo1TXx/extensions/fxdriver#googlecode.com/components/driver_component.js:8860)
at FirefoxDriver.prototype.findElement (file:///tmp/tmpKo1TXx/extensions/fxdriver#googlecode.com/components/driver_component.js:8869)
at DelayedCommand.prototype.executeInternal_/h (file:///tmp/tmpKo1TXx/extensions/fxdriver#googlecode.com/components/command_processor.js:10831)
at DelayedCommand.prototype.executeInternal_ (file:///tmp/tmpKo1TXx/extensions/fxdriver#googlecode.com/components/command_processor.js:10836)
at DelayedCommand.prototype.execute/< (file:///tmp/tmpKo1TXx/extensions/fxdriver#googlecode.com/components/command_processor.js:10778)
----------------------------------------------------------------------
Ran 1 test in 30.994s
FAILED (errors=1)
I tried other solutions as well but still same error .

You better should try to retrieve unique elements like the username field and password field by it's id. The name might not be unique and misleading.
Try the following find_element_by_id('id_username') and find_element_by_id('id_password').
Sometimes you need to wait some time that your browser renders the page
and retrieves all contents - so it can be useful to wait some time,
e.g. three seconds before you look deeper for elements.
Code
import time
time.sleep(3)
user_name_field = driver.find_element_by_id('id_username')
password_field = driver.find_element_by_id('id_password')
user_name_field.send_keys('your_user_name')
password_field.send_keys('your_password')
password_field.send_keys(Keys.RETURN)
...
Moreover, I recommend you to use different variables names for the different fields. The variable elem could lead to difficult bugs.
Three seconds of waiting time like mentioned above can be unreliable - a more structured way of assuring some waiting time is it to tell it your driver through implicitly_wait:
driver = webdriver.Firefox()
driver.implicitly_wait(30) # maximally wait 30 seconds before raising an exception.
# ...
# go to the page etc.
There are two ways of waiting time a driver can do:
Wait explicitly: The driver waits this time before executing every next step.
Wait implicitly: If you look for an element the driver searches for this time before he raises an exception.
For more details see the documentation: http://www.seleniumhq.org/docs/04_webdriver_advanced.jsp#implicit-waits

The following code use elem of previous page, which does not exist.
elem = elem.find_element_by_name("username")
Instead, you should use driver:
elem = driver.find_element_by_name("username")

Try
elem = new WebDriverWait(Driver, 30).until(ExpectedConditions.visibilityOfElementLocated(By.name("username")));
elem.send_keys("my_username#bitbucket.org");

To fill any type of form...the best method is go through its name in any case where ever available.
driver=self.webdriver
driver.find_element_by_name("username").click()
driver.find_element_by_name("username").send_keys("XXXXX")
time.sleep(5)
driver.find_element_by_name("password").click()
driver.find_element_by_name("password").send_keys("XXXXX")