We Keep Coding

Why doesn't it find the element by the class?

I want to type something in the input field, but when I call it with the class it returns an error. The Website has enough time to load all Elements so that shouldn't be the problem.
My Code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
browser = webdriver.Firefox()
browser.get('https://www.tradingview.com/chart/')
print("a")
time.sleep(5)
elem = browser.find_element_by_id("header-toolbar-symbol-search") # Find the search box
print("b")
elem.click()
time.sleep(5)
crypto_search = browser.find_element_by_class_name("search-Hsmn_0WX upperCase-Hsmn_0WX input-3n5_2-hI")
print("c")
crypto_search.send_keys("VETUSD")
time.sleep(10)
browser.quit()
When I run the code it gives me this error:
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: .search-Hsmn_0WX upperCase-Hsmn_0WX input-3n5_2-hI
It gets to the lines where it prints the a and b but it stops at the line which calls the element with class.

1 Get rid of time.sleep() because your tests will become unreliable and slow. Use implicit/explicit waits
2 If you have multiple classes in one use css or xpath selectors.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.common.keys import Keys
browser = webdriver.Firefox()
browser.get('https://www.tradingview.com/chart/')
wait = WebDriverWait(browser, 10)
wait.until(EC.element_to_be_clickable((By.ID, "header-toolbar-symbol-search"))).click() # Find the search box
wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, ".search-Hsmn_0WX.upperCase-Hsmn_0WX.input-3n5_2-hI"))).send_keys("VETUSD")
# browser.close()
Here I wait till the first element is clickable and the second is visible, as an example of how explicit waits work in Selenium.
Note, how faster my version of test is.

Instead of
crypto_search = browser.find_element_by_class_name("search-Hsmn_0WX upperCase-Hsmn_0WX input-3n5_2-hI")
try the following:
crypto_search = browser.find_element_by_css_selector(".search-Hsmn_0WX.upperCase-Hsmn_0WX.input-3n5_2-hI")
I tried and it worked for me.
Also, because these class names are multiple and looking not too discriptive I would prefer the following selector:
crypto_search = browser.find_element_by_css_selector("[data-name='symbol-search-items-dialog'] input")

according to the documentation you need to use 'find_elements'.
This is because classes are used when there will be multiple of them on a page, so it doesn't make sense to select only one element with a class name, even if there is only one on the page.
If that element is the only one with that class on the page, try using
browser.find_elements_by_class_name("search-Hsmn_0WX upperCase-Hsmn_0WX input-3n5_2-hI")[0]

How to retrieve the Ping Download and Upload time from browser tests?

I am trying to automate speedtests with different browsers automatically, and the main part of the test is inside a loop. The problem is, sometimes, one element which has been selected before, and the script worked correctly, at the one of the next steps, exactly at the same loop and at the same page, but with different number, without any change in the xpath, selenium cannot select it again. So, I can not repeat my test as much as I want.
Most of the time I have this problem with Edge, and I think one reason can be, the xpath for elements which I found by help of Chrome or Firefox. ( I can not find the xpath in Edge first of all, I searched a lot about it).
I also put the different xpath that I use. Actually I want to get the numeric or string values of ping,download, upload location and server.
Please let me know, how can I solve this issue, I tried different sleep time and two different xpath. the script always gives me error when I am trying to select the element with class_name or css_selector.
firefox:
"/html/body/div[3]/div[2]/div/div/div/div[3]/div[1]/div[3]/div/div[3]/div/div[1]/div[2]/div[1]/div/div[2]/span"
chrome:
"//[#id='container']/div[2]/div/div/div/div[3]/div[1]/div[3]/div/div[3]/div/div[1]/div[2]/div[1]/div/div[2]/span"
chrome:
"//div[#class='result-item result-item-ping updated']/div[2]/span"
Other question is how can I wait for a page to load completely. this method WebDriverWait(driver,some seconds) does not work for me and i have to use time.sleep()
Error:
selenium.common.exceptions.NoSuchElementException: Message: No such element
element = driver.find_element_by_xpath("/html/body/div[3]/div[2]/div/div/div/div[3]/div[1]/div[3]/div/div[3]/div/div[1]/div[2]/div[1]/div/div[2]/span")

To automate the speedtests you can use the following solution:
Code Block:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Edge(executable_path=r'C:\WebDrivers\MicrosoftWebDriver.exe')
driver.get("https://www.speedtest.net/")
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a.js-start-test.test-mode-multi"))).click()
WebDriverWait(driver, 45).until(EC.url_contains("result"))
print("Ping :"+driver.find_element_by_css_selector("div[title='Reaction Time'] div.result-data.u-align-left>span").get_attribute("innerHTML"))
print("Download: "+driver.find_element_by_css_selector("div[title='Receiving Time'] div.result-data.u-align-left>span").get_attribute("innerHTML"))
print("Upload :"+driver.find_element_by_css_selector("div[title='Sending Time'] div.result-data.u-align-left>span").get_attribute("innerHTML"))
#driver.quit()
Console Output:
Ping :35
Download: 21.53
Upload :3.46
Browser Snapshot:

Use the following CSS locators to identify the values:
Download: *.result-data-large.number.result-data-value.download-speed*
Upload: *.result-data-large.number.result-data-value.upload-speed*
Ping: *.result-data-large.number.result-data-value.ping-speed*
Making use of getText(), you can retrieve their values. Wait for an element in the page to be visible to make sure the page is loaded successfully.

Try with:element = driver.find_element_by_xpath("/html/body/div[3]/div[2]/div/div/div/div[3]/div[1]/div[3]/div/div[3]/div/div[1]/div[2]/div[1]/div/div[2]/")
Maybe also you need to catch exception for: NoSuchElementException cases.

I've tested these CSS selectors and they work in both Chrome and Edge.
span.ping-speed # ping
span.download-speed # download
span.upload-speed # upload
div.server-current > div.result-label # server
If you want to know when the page is done loading, you can wait until the URL changes from https://www.speedtest.net to https://www.speedtest.net/results/<some number>. I would just use WebDriverWait and url_contains("results") , e.g.
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
WebDriverWait(driver, 10).until(EC.url_contains("results"))
There are some other approaches in this question.

WebDriverWait driverWait = new WebDriverWait(driver, 30000);
driver.get("https://www.speedtest.net/");
WebElement goLink = driver.findElement(By.cssSelector(".js-start-test.test-mode-multi"));
driverWait.until(ExpectedConditions.elementToBeClickable(goLink));
goLink.click();
By download = By.cssSelector(".result-data-large.number.result-data-value.download-speed");
By upload = By.cssSelector(".result-data-large.number.result-data-value.upload-speed");
By ping = By.cssSelector(".result-data-large.number.result-data-value.ping-speed");
driverWait.until(ExpectedConditions.urlMatches("https://www.speedtest.net/result/[0-9]"));
String downloadSpeed = driver.findElement(download).getText();
String uploadSpeed = driver.findElement(upload).getText();
String pingValue = driver.findElement(ping).getText();
System.out.println("Download: "+downloadSpeed + "\nUpload: "+ uploadSpeed + "\n Ping: "+pingValue);
Output
Download: 78.82
Upload: 45.93
Ping: 23

Unable to click on signs on a map

I've written a script in Python in association with selenium to click on each of the signs available in a map. However, when I execute my script, it throws timeout exception error upon reaching this line wait.until(EC.staleness_of(item)).
Before hitting that line, the script should have clicked once but It could not? How can I click on all the signs in that map cyclically?
This is the site link.
This is my code so far (perhaps, I'm trying with the wrong selectors):
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
link = "https://www.findapetwash.com/"
driver = webdriver.Chrome()
driver.get(link)
wait = WebDriverWait(driver, 15)
for item in wait.until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "#map .gm-style"))):
item.click()
wait.until(EC.staleness_of(item))
driver.quit()
Signs visible on that map like:
Post script: I know that this is their API https://www.findapetwash.com/api/locations/getAll/ using which I can get the JSON content but I would like to stick to the Selenium way. Thanks.

I know you wrote you don't want to use the API but using Selenium to get the locations from the map markers seems a bit overkill for this, instead, why not making a call to their Web service using requests and parse the returned json?
Here is a working script:
import requests
import json
api_url='https://www.findapetwash.com/api/locations/getAll/'
class Location:
def __init__(self, json):
self.id=json['id']
self.user_id=json['user_id']
self.name=json['name']
self.address=json['address']
self.zipcode=json['zipcode']
self.lat=json['lat']
self.lng=json['lng']
self.price_range=json['price_range']
self.photo='https://www.findapetwash.com' + json['photo']
def get_locations():
locations = []
response = requests.get(api_url)
if response.ok:
result_json = json.loads(response.text)
for location_json in result_json['locations']:
locations.append(Location(location_json))
return locations
else:
print('Error loading locations')
return False
if __name__ == '__main__':
locations = get_locations()
for l in locations:
print(l.name)
Selenium
If you still want to go the Selenium way, instead of waiting until all the elements are loaded, you could just halt the script for some seconds or even a minute to make sure everything is loaded, this should fix the timeout exception:
import time
driver.get(link)
# Wait 20 seconds
time.sleep(20)
For other possible workarounds, see the accepted answer here: Make Selenium wait 10 seconds

You can click one by one using Selenium if, for some reasons, you cannot use API. Also it is possible to extract information for each sign without clicking on them with Selenium.
Here code to click one by one:
signs = wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "li.marker.marker--list")))
for sign in signs:
driver.execute_script("arguments[0].click();", sign)
#do something
Try also without wait, probably will work.

StaleElementReferenceException in Python

I am trying to scrape data from the Sunshine List website (http://www.sunshinelist.ca/) using the Selenium package but I get the following error mentioned below. From several other related posts I understand that I need to use the WebDriverWait to explicitly ask the driver to wait/refresh but I am unable to identify where and how I should call the function.
Screenshot of Error
StaleElementReferenceException: Message: The element reference
of (tr class="even") stale: either the element is no longer attached to the DOM or the
page has been refreshed
import numpy as np
import pandas as pd
import requests
import time
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.common.exceptions import StaleElementReferenceException
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
ffx_bin = FirefoxBinary(r'C:\Users\BhagatM\AppData\Local\Mozilla Firefox\firefox.exe')
ffx_caps = DesiredCapabilities.FIREFOX
ffx_caps['marionette'] = True
driver = webdriver.Firefox(capabilities=ffx_caps,firefox_binary=ffx_bin)
driver.get("http://www.sunshinelist.ca/")
driver.maximize_window()
tablewotags1=[]
while True:
divs = driver.find_element_by_id('datatable-disclosures')
divs1=divs.find_elements_by_tag_name('tbody')
for d1 in divs1:
div2=d1.find_elements_by_tag_name('tr')
for d2 in div2:
tablewotags1.append(d2.text)
try:
driver.find_element_by_link_text('Next →').click()
except NoSuchElementException:
break
year1=tablewotags1[0::10]
name1=tablewotags1[3::10]
position1=tablewotags1[4::10]
employer1=tablewotags1[1::10]
df1=pd.DataFrame({'Year':year1,'Name':name1,'Position':position1,'Employer':employer1})
df1.to_csv('Sunshine List-1.csv', index=False)

If your problem is to click the "Next" button, you can do that with the xpath:
driver = webdriver.Firefox(executable_path=r'/pathTo/geckodriver')
driver.get("http://www.sunshinelist.ca/")
wait = WebDriverWait(driver, 20)
el=wait.until(EC.presence_of_element_located((By.XPATH,"//ul[#class='pagination']/li[#class='next']/a[#href='#' and text()='Next → ']")))
el.click()

For each click on the "Next" button -- you should find that button and click on it.
Or do something like this:
max_attemps = 10
while True:
next = self.driver.find_element_by_css_selector(".next>a")
if next is not None:
break
else:
time.sleep(0.5)
max_attemps -= 1
if max_attemps == 0:
self.fail("Cannot find element.")
And after this code does click action.
PS: Also try to add just time.sleep(x) after fiding element and then do click action.

Try this code below.
When the element is no longer attached to the DOM and the StaleElementReferenceException is invoked, search for the element again to reference the element.
Please do note I checked with Chrome:
try:
driver.find_element_by_css_selector('div[id="datatable-disclosures_wrapper"] li[class="next"]>a').click()
except StaleElementReferenceException:
driver.find_element_by_css_selector('div[id="datatable-disclosures_wrapper"] li[class="next"]>a').click()
except NoSuchElementException:
break

>>>Stale Exceptions can be handled using **StaleElementReferenceException** to continue to execute the for loop. When you try to get the element by any find_element method in a for loop.
from selenium.common import exceptions
and customize your code of for loop as:
for loop starts:
try:
driver.find_elements_by_id("data") //method to find element
//your code
except exceptions.StaleElementReferenceException:
pass

When you raise the StaleElementException that means that somthing changed in the site, but not in the list you have. So the trick is to refresh that list every time, inside the loop like this:
while True:
driver.implicitly_wait(4)
for d1 in driver.find_element_by_id('datatable-disclosures').find_element_by_tag_name('tbody').find_elements_by_tag_name('tr'):
tablewotags1.append(d1.text)
try:
driver.switch_to.default_content()
driver.find_element_by_xpath('//*[#id="datatable-disclosures_wrapper"]/div[2]/div[2]/div/ul/li[7]/a').click()
except NoSuchElementException:
print('Don\'t be so cryptic about error messages, they are good\n
...Script broke clicking next') #jk aside put some info there
break
Hope this help you, cheers.
Edit:
So I went to the said website, the layout is pretty straight forward, but the structure repeats itself like four times. So when you go about crawling the site like that something is bound to change.
So I’ve edited the code to only scrap one tbody tree. This tree comes from the first datatable-disclousure. And added some waits.

Selenium Python: Unable to select combo value

I am accessing this page and I have to pick one of Combo value that shows existing resume. I am getting error:
selenium.common.exceptions.ElementNotVisibleException: Message:
Element is not currently visible and so may not be interacted with
Code I am trying is:
from selenium import webdriver
from selenium.webdriver.support.select import Select
from time import sleep
from bs4 import BeautifulSoup
driver = webdriver.Firefox()
driver.get('https://www.glassdoor.com/job-listing/developer-one-north-interactive-JV_IC1128808_KO0,9_KE10,31.htm?jl=1584069572')
select_box = driver.find_element_by_id('ExistingResume')
select_box.click();
select_box = Select(select_box)
sleep(5)
select_box.select_by_value(RESUME_TEXT_VALUE)
Updated The required element is right there
Update #2: Checked that element is not visible in static html. Guess via JS being loaded.
Update #3: OK I made following changes which prints tag name that is select:
select_box = driver.find_element_by_id('ExistingResume')
print(select_box.tag_name)
Now mission is to select the value from that combo

You can use Keys.ARROW_DOWN to get the option and Keys.RETURN to select. See below:
>>> from selenium.webdriver.common.keys import Keys
>>> driver.find_element_by_id("ExistingResumeSelectBoxIt").click()
>>> d = driver.find_element_by_id("ExistingResumeSelectBoxIt")
>>> d.send_keys(Keys.ARROW_DOWN)
>>> d.send_keys(Keys.RETURN)
>>> driver.find_element_by_id("ExistingResumeSelectBoxIt").text
u'mesut gunes resume eng.pdf'
You must be logged in and have a resume too.

The error message is clean enough. The problems is that there is no element with id "ExistingResume" on the website.
If you open the console in your browser and type $("#ExistingResume"); jQuery cannot find that element on the website neither.
Make sure you didn't make a typo in the id or you have all the necessary permissions to access that element. (Maybe it's visible only for logged in users.)

First of all, you have to be logged in and have at least one existing uploaded resume.
You cannot directly control the select element with the resumes - it is invisible. Control the wrapping combo element that is visible.
Complete working code (including logging in):
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Firefox()
driver.get('https://www.glassdoor.com/job-listing/developer-one-north-interactive-JV_IC1128808_KO0,9_KE10,31.htm?jl=1584069572')
# log in
driver.find_element_by_css_selector("div.actions span.signin").click()
driver.find_element_by_css_selector("form.signInForm input.signin-email").send_keys("login")
driver.find_element_by_css_selector("form.signInForm input.signin-password").send_keys("password")
driver.find_element_by_css_selector("form.signInForm button.loginDlgSignInBtn").click()
# wait for the combo to appear
resume_name = "myResume.pdf"
# open up combo
combo = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, "ExistingResumeSelectBoxIt")))
combo.click()
# select resume
resume_item = combo.find_element_by_xpath("//li[#data-val = '%s']" % resume_name)
resume_item.click()