How to retrieve the Ping Download and Upload time from browser tests? - python

I am trying to automate speedtests with different browsers automatically, and the main part of the test is inside a loop. The problem is, sometimes, one element which has been selected before, and the script worked correctly, at the one of the next steps, exactly at the same loop and at the same page, but with different number, without any change in the xpath, selenium cannot select it again. So, I can not repeat my test as much as I want.
Most of the time I have this problem with Edge, and I think one reason can be, the xpath for elements which I found by help of Chrome or Firefox. ( I can not find the xpath in Edge first of all, I searched a lot about it).
I also put the different xpath that I use. Actually I want to get the numeric or string values of ping,download, upload location and server.
Please let me know, how can I solve this issue, I tried different sleep time and two different xpath. the script always gives me error when I am trying to select the element with class_name or css_selector.
firefox:
"/html/body/div[3]/div[2]/div/div/div/div[3]/div[1]/div[3]/div/div[3]/div/div[1]/div[2]/div[1]/div/div[2]/span"
chrome:
"//[#id='container']/div[2]/div/div/div/div[3]/div[1]/div[3]/div/div[3]/div/div[1]/div[2]/div[1]/div/div[2]/span"
chrome:
"//div[#class='result-item result-item-ping updated']/div[2]/span"
Other question is how can I wait for a page to load completely. this method WebDriverWait(driver,some seconds) does not work for me and i have to use time.sleep()
Error:
selenium.common.exceptions.NoSuchElementException: Message: No such element
element = driver.find_element_by_xpath("/html/body/div[3]/div[2]/div/div/div/div[3]/div[1]/div[3]/div/div[3]/div/div[1]/div[2]/div[1]/div/div[2]/span")

To automate the speedtests you can use the following solution:
Code Block:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Edge(executable_path=r'C:\WebDrivers\MicrosoftWebDriver.exe')
driver.get("https://www.speedtest.net/")
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a.js-start-test.test-mode-multi"))).click()
WebDriverWait(driver, 45).until(EC.url_contains("result"))
print("Ping :"+driver.find_element_by_css_selector("div[title='Reaction Time'] div.result-data.u-align-left>span").get_attribute("innerHTML"))
print("Download: "+driver.find_element_by_css_selector("div[title='Receiving Time'] div.result-data.u-align-left>span").get_attribute("innerHTML"))
print("Upload :"+driver.find_element_by_css_selector("div[title='Sending Time'] div.result-data.u-align-left>span").get_attribute("innerHTML"))
#driver.quit()
Console Output:
Ping :35
Download: 21.53
Upload :3.46
Browser Snapshot:

Use the following CSS locators to identify the values:
Download: *.result-data-large.number.result-data-value.download-speed*
Upload: *.result-data-large.number.result-data-value.upload-speed*
Ping: *.result-data-large.number.result-data-value.ping-speed*
Making use of getText(), you can retrieve their values. Wait for an element in the page to be visible to make sure the page is loaded successfully.

Try with:element = driver.find_element_by_xpath("/html/body/div[3]/div[2]/div/div/div/div[3]/div[1]/div[3]/div/div[3]/div/div[1]/div[2]/div[1]/div/div[2]/")
Maybe also you need to catch exception for: NoSuchElementException cases.

I've tested these CSS selectors and they work in both Chrome and Edge.
span.ping-speed # ping
span.download-speed # download
span.upload-speed # upload
div.server-current > div.result-label # server
If you want to know when the page is done loading, you can wait until the URL changes from https://www.speedtest.net to https://www.speedtest.net/results/<some number>. I would just use WebDriverWait and url_contains("results") , e.g.
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
WebDriverWait(driver, 10).until(EC.url_contains("results"))
There are some other approaches in this question.

WebDriverWait driverWait = new WebDriverWait(driver, 30000);
driver.get("https://www.speedtest.net/");
WebElement goLink = driver.findElement(By.cssSelector(".js-start-test.test-mode-multi"));
driverWait.until(ExpectedConditions.elementToBeClickable(goLink));
goLink.click();
By download = By.cssSelector(".result-data-large.number.result-data-value.download-speed");
By upload = By.cssSelector(".result-data-large.number.result-data-value.upload-speed");
By ping = By.cssSelector(".result-data-large.number.result-data-value.ping-speed");
driverWait.until(ExpectedConditions.urlMatches("https://www.speedtest.net/result/[0-9]"));
String downloadSpeed = driver.findElement(download).getText();
String uploadSpeed = driver.findElement(upload).getText();
String pingValue = driver.findElement(ping).getText();
System.out.println("Download: "+downloadSpeed + "\nUpload: "+ uploadSpeed + "\n Ping: "+pingValue);
Output
Download: 78.82
Upload: 45.93
Ping: 23

Related

Selenium does't open firefox

I'm trying use selenium and firefox, but it's just open the browser and set this error:
Error
With Chrome is ok.
This is the code:
from selenium import webdriver
driver = webdriver.Firefox(executable_path = "c:/my-apps/geckodriver.exe")
Thanks very much
When you try to find any element in an HTML page that does not exist, NoSuchElementException will be raised.
First Possible Solution:
You need to import
from selenium.common.exceptions import NoSuchElementException
Then you can use try except block
try:
your_element = driver.find_element_by_xpath(".//*[#id='loginForm:username']")
your_element.click()
except NoSuchElementException:
pass
Second Possible Solution:
Without importing anything, checking whether that element exist, if it does then it will be clicked
your_element = driver.find_elements_by_xpath(".//*[#id='loginForm:username']")
if len(your_element) > 0:
elem[0].click()
HEY You can use this code as given below
driver = webdriver.Firefox()
driver.implicitly_wait(5)
driver.maximize_window()
driver.get('URL')
I think you should put your geckodriver in the same folder where you are coding
and if this does not work you need to upgrade geckodriver
HEY According to your ERROR
when you open the website your 1 element is not correct
you need to change your 1 element of your website
wait=WebDriverWait(self.driver,60)
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR,"#loginForm:username"))).send_keys("a")
Generally Firefox is a tad slower than Chrome so your issue may be timing for find elements. What is recommended is waiting for the element to be clickable and then proceeding to send keys to it.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

Why am I getting the message that the search-bar is not intractable?

The code is supposed to type "fish" into the YouTube search bar using Selenium and a Chrome Browser.
I have tried the xpaths of mulitple divs that hold the tag and they didn't work either.(not sure if the error was the same though) The xpath in the code is for the <input> tag so it should be fine.
I also watched a tutorial and the xpath was exactly the same so that shouldn't be the problem since it worked for the YouTuber.
It also took me some time to figure out that the find_element_by_* are depreciated functions.
Could it be that the .send_keys has also been changed? I did try to find the selenium changes in 4.1.0 and it said nothing about it on a website that I found.
Should I maybe delete Selenium 4.1.0 and install an older version? For simplicity sake. Since there is probably a bigger number of tutorials for it.
from selenium import webdriver
from selenium.webdriver.common.by import By
setting = webdriver.ChromeOptions()
setting.add_argument("--incognito")
# I open the browser in incognito just so I don't clutter my search
# history with dumb stuff as I'm testing things out
# could it be a part of the problem?
driver = webdriver.Chrome(options = setting)
driver.get('http://youtube.com')
searchbox = driver.find_element(By.XPATH, '//*[#id="search"]')
searchbox.send_keys('fish')
Error Message:
selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable
wait=WebDriverWait(driver,60)
driver.get('http://youtube.com')
searchbox = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR,"input#search")))
searchbox.send_keys('fish')
In order to send_keys to that element wait for it to interactable and then send keys.
Imports:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
Outputs:

Python Selenium - send_keys not sending any information to element

I am trying out Selenium for the first time so I apologize if there is an obvious mistake or problem with my code.
from selenium import webdriver
driver = webdriver.Chrome()
driver.get('https://youtube.com')
searchBox = driver.find_element_by_id('search')
searchBox.send_keys('Programming')
searchButton = driver.find_element_by_id('search-icon-legacy')
searchButton.click()
So I tried this and it loads the page fine but, it does not input any characters into the searchBox (I quadruple checked that the id was correct - copied it directly from the inspector).
NOTE:
My internet is really REALLY slow and it takes YouTube approx. 20 seconds to fully load, so I thought that was an issue so I tried;
...
driver.get('https://youtube.com')
driver.implicitly_wait(30)
searchBox = driver.find_element_by_id('search')
...
But this did not work either.
I did use XPATH instead of finding it by element ID at the start and that did not work.
I checked and copied the XPATHs and IDs directly from the inspector and nothing so far has inputted anything into the textbox.
What could be the problem? (1)
and does the webdriver wait for the page to load/find the element before doing anything after it being initialized with the driver.get('websiteAddress')? (2)
NOTE: I double checked that I was selecting the right element as well.
To send keys to the input tag with id = search. We use webdriver waits to allow the element to be usable after driver.get so the page loads correctly.
driver.get('https://youtube.com')
WebDriverWait(driver, 30).until(EC.element_to_be_clickable((By.XPATH, "//input[#id='search']"))).send_keys("Programming")
Import
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
If you don't know the waiting time:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
delay = 40
WebDriverWait(driver, delay).until(EC.presence_of_element_located((By.XPATH, "//form[#role='form']//input[#id='username']")))
Then it just waits on the element, for as log as delay is, but will continue as soon as the element is found, that the best way to wait on slow connections.
To relate elements more easily you can use ChroPath, it is an extension for google chrome / edge that allows you to see the path of an element, through cssSelector, Abs XPath, Rel XPath and the tag name, so when your code is not working you can try these other ways. Particularly this extension helps me a lot.

Access nested elements in HTML using Python Selenium

I am trying to automate logging into a website (http://www.phptravels.net/) using Selenium - Python on Chrome. This is an open website used for automation tutorials.
I am trying to click an element to open a drop-down (My Account button at the top navbar) which will then give me the option to login and redirect me to the login page.
The HTML is nested with many div and ul/li tags.
I have tried various lines of code but haven't been able to make much progress.
driver.find_element_by_id('li_myaccount').click()
driver.find_element_by_link_text(' Login').click()
driver.find_element_by_xpath("//*[#id='li_myaccount']/ul/li[1]/a").click()
These are some of the examples that I tried out. All of them failed with the error "element not visible".
How do I find those elements? Even the xpath function is throwing this error.
I have not added any time wait in my code.
Any ideas how to proceed further?
Hope this code will help:
from selenium import webdriver
from selenium.webdriver.common.by import By
url="http://www.phptravels.net/"
d=webdriver.Chrome()
d.maximize_window()
d.get(url)
d.find_element(By.LINK_TEXT,'MY ACCOUNT').click()
d.find_element(By.LINK_TEXT,'Login').click()
d.find_element(By.NAME,"username").send_keys("Test")
d.find_element(By.NAME,"password").send_keys("Test")
d.find_element(By.XPATH,"//button[text()='Login']").click()
Use the best available locator on your html page, so that you need not to create xpath of css for simple operations
You may be having issues with the page not being loaded when you try and find the element of interest. You should use the WebDriverWait class to wait until a given element is present in the page.
Adapted from the docs:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
# Set up your driver here....
try:
element = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.ID, 'li_myaccount'))
)
element.click()
except:
#Handle any exceptions here
finally:
driver.quit()

Python with selenium webscraping unable to find elements

I am trying to write a webscraping in python that will activate the "onclick" functionality of certain buttons on a webpage because the tables with the data I want are converted to csv, which makes it much easier to access. But the problem is that I am unable to locate elements by xpath at all when using PhantomJs. How can I click the element and access the csv content that I want?
This is my code:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.common.by import By
from selenium.webdriver.common.proxy import *
url = "http://www.pro-football-reference.com/boxscores/201609180nwe.htm"
xpath = "//*[#id='all_player_offense']/div[1]/div/ul/li[1]/div/ul/li[3]/button"
path_to_phantomjs = 'browser/phantomjs'
browser = webdriver.PhantomJS(executable_path = path_to_phantomjs)
browser.get(url)
delay=3
element_present = EC.presence_of_element_located((By.ID, 'all_player_offense'))
WebDriverWait(browser, delay).until(element_present)
browser.find_element_by_xpath(xpath).click()
And I get this error:
selenium.common.exceptions.NoSuchElementException: Message: {"errorMessage":"Unable to find element with xpath '//*[#id='all_player_offense']/div[1]/div/ul/li[1]/div/ul/li[3]/button'","request":{"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"153","Content-Type":"application/json;charset=UTF-8","Host":"127.0.0.1:50989","User-Agent":"Python-urllib/2.7"},"httpVersion":"1.1","method":"POST","post":"{\"using\": \"xpath\", \"sessionId\": \"93ff24f0-9cbe-11e6-8711-bdfa3ff9cfb1\", \"value\": \"//*[#id='all_player_offense']/div[1]/div/ul/li[1]/div/ul/li[3]/button\"}","url":"/element","urlParsed":{"anchor":"","query":"","file":"element","directory":"/","path":"/element","relative":"/element","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/element","queryKey":{},"chunks":["element"]},"urlOriginal":"/session/93ff24f0-9cbe-11e6-8711-bdfa3ff9cfb1/element"}}
Screenshot: available via screen
IMPORTANT THING I FORGOT TO MENTION: As described in this this issue on GitHub, try putting set_window_size(width, height) or maximize_window()after setting the webdriver. You should also consider telling the webdriver to implicitly_wait(10) for the element to appear.
So there's a special maneuver you have to perform in order for the Selenium Webdriver to properly emulate what you're doing. In essence, to get the desired data, you'd have to:
A: Hover over the "Share & More" dropdown menu. Then
B: Click "Get table as CSV (Excel)".
For A, this involves having to place the emulated cursor on the element without clicking it. This idea of "mouse over" can be done with the move_to_element() function provided in the ActionChains class. So at the top you'd insert this:
from selenium.webdriver.common.action_chains import ActionChains
You want Selenium to find the specific element and move to it. You can achieve this with 2 lines of code:
dropdown = browser.find_element_by_xpath('//*[#id="all_player_offense"]/div[1]/div/ul/li[1]')
ActionChains(browser).move_to_element(dropdown).perform()
If you omit the above, you'll get an ElementNotVisibleException.
Now for B, you should be able to do browser.find_element_by_xpath(xpath).click().

Categories