We Keep Coding

Find Outdated ChromeDriver Version Without Opening Executable (Python)

Essentially, I'm trying to find the ChromeDriver version without actually opening it. The reason for this is that I want to auto update the driver. I've sorted the code out for updating it.
The issue is that when I intentionally download an unsupported version to check my code works, it says "This driver only supports Chrome Version 86 and you're on 88.001.xyz" etc.
I was wondering if there was any way of reading the 86 from the ChromeDriver executable so that I can recognise it's not equal to 88 (my chrome browser version)? By doing this, it'll trigger a procedure to go and download the correct chromedriver.
I've attached my code for checking the chromedriver version. I've tried headless but I'm sure it doesn't work.
def get_chrome_version():
global browser_version_number
options = Options()
options.headless = True
browser_version_driver = webdriver.Chrome("chromedriver_win32/chromedriver.exe", chrome_options=options)
# browser_version_driver = webdriver.Chrome("chromedriver_win32/chromedriver.exe")
# browser_version_driver.set_window_position(-10000,0)
browser_version_number = (browser_version_driver.capabilities['browserVersion'])
browser_version_number = browser_version_number.split(".")[0]
chromedriverversion = browser_version_driver.capabilities['chrome']['chromedriverVersion'].split('.')[0]
print(browser_version_number)
print(chromedriverversion)
if browser_version_number != chromedriverversion:
update_chrome_version()
browser_version_driver.quit()

you could have your python script make a shell call to execute the command ./chromedriver --version (using the relevant path), then parse the result to find your current installed version.

Alrighty, so I found the solution. It all lied within the exception that was outputted in the terminal. Basically my thinking was to save the exception as string to a variable. Once I could do that, I could split it up and get what was needed from it. This was really easy to do since the ChromeDriver devs had this exception message:
Message: session not created: This version of ChromeDriver only supports Chrome version 86
Current browser version is 88.0.4324.xyz with binary path C:\Program Files (x86)\Google\Chrome\Application\chrome.exe
So I needed to get the 86 to signify the ChromeDriver version and the 88 for the browser version. I've included the code below. If you have any questions, feel free to comment! I'm also aware that I may get crucified for the absolute spaghetti code I've written :). I just want to get something out and then clean up the tomato sauce off my shirt later!!
import logging
logger = logging.Logger('catch_all')
def get_chrome_version():
global browser_version_number
try:
browser_version_driver = webdriver.Chrome("chromedriver_win32/chromedriver.exe")
browser_version_number = (browser_version_driver.capabilities['browserVersion'])
browser_version_number = browser_version_number.split(".")[0]
chromedriverversion = browser_version_driver.capabilities['chrome']['chromedriverVersion'].split('.')[0]
print(browser_version_number)
print(chromedriverversion)
if browser_version_number != chromedriverversion:
update_chrome_version()
browser_version_driver.quit()
###############---------------REALLY IMPORTANT PART BELOW---------------###############
except Exception as e:
e = str(e) # Saves exception to a variable. Most importantly, converts to string to allow me to manipulate it.
print(e)
linesplit = e.split('This version of ChromeDriver only supports Chrome version ')[1]
checking_driverversion = linesplit.split('\n')[0]
print(checking_driverversion) # prints '86' which is my chromedriver right now
checking_browserversion = linesplit.split('\n')[1]
checking_browserversion = checking_browserversion.split('Current browser version is ')[1]
checking_browserversion = checking_browserversion.split('.')[0]
print(checking_browserversion) # prints '88' which is my browser version right now
browser_version_number = checking_browserversion
if checking_browserversion != checking_driverversion:
update_chrome_version()

browser_switcher_service.cc(238)] XXX Init() error with Python Selenium Script with Chrome for Web Scraping

I was using a small python script along with chrome drivers to download manga from Mangafox. It used to run fine until a few days ago when I updated the Chrome Browser. The following error is shown every time I try it now:
[14664:14280:0420/202509.245:ERROR:browser_switcher_service.cc(238)] XXX Init()
[14664:14280:0420/202509.678:ERROR:device_event_log_impl.cc(162)] [20:25:09.679] Bluetooth: bluetooth_adapter_winrt.cc:1186 Getting Radio failed. Chrome will be unable to change the power state by itself.
[14664:14280:0420/202509.695:ERROR:device_event_log_impl.cc(162)] [20:25:09.696] Bluetooth: bluetooth_adapter_winrt.cc:1264 OnPoweredRadioAdded(), Number of Powered Radios: 1
[14664:14280:0420/202509.696:ERROR:device_event_log_impl.cc(162)] [20:25:09.696] Bluetooth: bluetooth_adapter_winrt.cc:1283 OnPoweredRadiosEnumerated(), Number of Powered Radios: 1
I have used the selenium module with chrome drivers. I have tried updating my web drivers, tried making the code loop or sleep until the web-page loads completely.
My Code is as follows:
from selenium import webdriver
import os
import urllib.request
Main = 'https://ww3.mangafox.online/'
Name = str(input('Enter Name of Manga as on \'ww3.Mangafox.online\': '))
Name = Name.replace(' ', '-')
Name = Name.replace('\'', '-')
driver = webdriver.Chrome(r'C:\Users\freak\Assignments\SDP\Drivers\chromedriver.exe')
driver.get(Main + Name)
Tags = driver.find_elements_by_tag_name('a')
List = []
for Tag in Tags:
List.append(str(Tag.get_attribute('href')))
dir = os.path.join('C:\\','Users', 'freak', 'Assignments', 'SDP', 'Test', Name.replace('-', '_'))
print('Checking the existence of folder; %s' % dir)
if not os.path.exists(dir):
print('Folder not found. Attempting to create folder: %s' % dir)
os.mkdir(dir)
print('Folder successfully created')
Index = []
for i, element in enumerate(List):
if (str(Name) + '/chapter') in element:
Index.append(i)
Chapters = []
After this part is just a loop I used to download the images. But for some reason, an error is shown and the list made for Tags remains empty. The driver.find_elements_by_tag_name('a') fails completely.

These error messages...
[14664:14280:0420/202509.245:ERROR:browser_switcher_service.cc(238)] XXX Init()
[14664:14280:0420/202509.678:ERROR:device_event_log_impl.cc(162)] [20:25:09.679] Bluetooth: bluetooth_adapter_winrt.cc:1186 Getting Radio failed. Chrome will be unable to change the power state by itself.
[14664:14280:0420/202509.695:ERROR:device_event_log_impl.cc(162)] [20:25:09.696] Bluetooth: bluetooth_adapter_winrt.cc:1264 OnPoweredRadioAdded(), Number of Powered Radios: 1
[14664:14280:0420/202509.696:ERROR:device_event_log_impl.cc(162)] [20:25:09.696] Bluetooth: bluetooth_adapter_winrt.cc:1283 OnPoweredRadiosEnumerated(), Number of Powered Radios: 1
...implies that the on_init_ method failed in std::make_unique<base::ScopedClosureRunner>(std::move(on_init)).
Analysis
These errors are defined in bluetooth_adapter_winrt.cc as follows:
Getting Radio failed. Chrome will be unable to change the power state by itself:
void BluetoothAdapterWinrt::OnGetRadio(base::ScopedClosureRunner on_init,
ComPtr<IRadio> radio) {
DCHECK_CALLED_ON_VALID_THREAD(thread_checker_);
if (radio) {
radio_ = std::move(radio);
radio_was_powered_ = GetState(radio_.Get()) == RadioState_On;
radio_state_changed_token_ = AddTypedEventHandler(
radio_.Get(), &IRadio::add_StateChanged,
base::BindRepeating(&BluetoothAdapterWinrt::OnRadioStateChanged,
weak_ptr_factory_.GetWeakPtr()));
if (!radio_state_changed_token_)
BLUETOOTH_LOG(ERROR) << "Adding Radio State Changed Handler failed.";
return;
}
// This happens within WoW64, due to an issue with non-native APIs.
BLUETOOTH_LOG(ERROR)
<< "Getting Radio failed. Chrome will be unable to change the power "
"state by itself.";
Number of Powered Radios: 1:
void BluetoothAdapterWinrt::OnPoweredRadioAdded(IDeviceWatcher* watcher,
IDeviceInformation* info) {
if (++num_powered_radios_ == 1)
NotifyAdapterPoweredChanged(true);
BLUETOOTH_LOG(ERROR) << "OnPoweredRadioAdded(), Number of Powered Radios: "
<< num_powered_radios_;
}
void BluetoothAdapterWinrt::OnPoweredRadioRemoved(
IDeviceWatcher* watcher,
IDeviceInformationUpdate* update) {
if (--num_powered_radios_ == 0)
NotifyAdapterPoweredChanged(false);
BLUETOOTH_LOG(ERROR) << "OnPoweredRadioRemoved(), Number of Powered Radios: "
<< num_powered_radios_;
}
void BluetoothAdapterWinrt::OnPoweredRadiosEnumerated(IDeviceWatcher* watcher,
IInspectable* object) {
BLUETOOTH_LOG(ERROR)
<< "OnPoweredRadiosEnumerated(), Number of Powered Radios: "
<< num_powered_radios_;
// Destroy the ScopedClosureRunner, triggering the contained Closure to be
// run. Note this may destroy |this|.
DCHECK(on_init_);
on_init_.reset();
}
Deep Dive
These errors are the direct impact of the changes incorporated with google-chrome as per the details within the discussion in Chrome no longer accepts certificates that fallback to common name
Solution
Ensure that:
Selenium is upgraded to current levels Version 3.141.59.
ChromeDriver is updated to current ChromeDriver v84.0 level.
Chrome is updated to current Chrome Version 84.0 level. (as per ChromeDriver v84.0 release notes)
If your base Web Client version is too old, then uninstall it and install a recent GA and released version of Web Client.
Additional considerations
However it was observed that this error can be supressed by running Chrome as root user (administrator) on Linux, but that would be a deviation from the documentation in ChromeDriver - WebDriver for Chrome where it is mentioned:
A common cause for Chrome to crash during startup is running Chrome as root user (administrator) on Linux. While it is possible to work around this issue by passing '--no-sandbox' flag when creating your WebDriver session, i.e. the ChromeDriver session as such a configuration is unsupported and highly discouraged.
Ideally, you need to configure your environment to run Chrome as a regular user instead.
Suppressing the error
Finally, as per the documentation in Selenium Chrome Driver: Resolve Error Messages Regarding Registry Keys and Experimental Options these error logs can be supressed by adding the argument:
excludeSwitches: ['enable-logging']
So your effective code block will be:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_experimental_option("excludeSwitches", ["enable-logging"])
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get("https://www.google.com/")

Have you tried switching to the Firefox driver instead?
https://github.com/mozilla/geckodriver/releases
You'd just have to update your code:
driver = webdriver.Firefox(executable_path='/path/to/driver')

Python Selenium Firefox Custom Profiles Save

I just started learning selenium with python
from selenium import webdriver
MY_PROFILE = "D:\\FIREFOX_PROFILE"
FFP = webdriver.FirefoxProfile(MY_PROFILE)
print(FFP.profile_dir)
# OUTPUT: C:\Users\ABC\AppData\Local\Temp\****\***
# But it should be OUTPUT: D:\FIREFOX_PROFILE
DRIVER = webdriver.Firefox(firefox_profile = FFP)
print(FFP.profile_dir)
# OUTPUT: C:\Users\ABC\AppData\Local\Temp\****\***
# But it should be OUTPUT: D:\FIREFOX_PROFILE
I want to save my profile somewhere so that I can use it later on.
I also tried creating RUN -> firefox.exe -p and creating a new profile (I can't use the created profile). Nothing works.
I am using:
Selenium Version: 2.53.6
Python Version: 3.4.4
Firefox Version: Various(49.0.2, 45, 38 etc)
I searched in Google but I can't solve it. Is there any way to save the profile?

You need to take help of os module in python
import os
there you get functions (like .getcwd() ) to described in Files and Directories.
then use,
p = webdriver.FirefoxProfile()
p.set_preference('browser.download.folderList', 2 )
p.set_preference('browser.download.manager.showWhenStarting', false)
p.set_preference('browser.download.dir', os.getcwd())
p.set_preference('browser.helperApps.neverAsk.saveToDisk', 'text/csv/xls')
driver = webdriver.Firefox(p)
in short you can do so,
profile.set_preference("browser.helperApps.neverAsk.openFile","text/csv,application/x-msexcel,application/excel,application/x-excel,application/vnd.ms-excel,image/png,image/jpeg,text/html,text/plain,application/msword,application/xml")
possible duplicate of Setting selenium to use custom profile, but it keeps opening with default

How to pass a pytest function a browser to open to perform the test's actions in

I am trying to write test code for a web application in python using pytest and selenium.
My code looks like:
from selenium import webdriver
import pytest
import time
def test_OpenBE(browser):
browser.get('urlremoved.com')
def test_navigateToSubmit(browser):
browser.get('urlremoved.com')
time.sleep(10)
browser.find_element_by_id('button').click()
def test_Submission(browser):
browser.get('urlremoved.com')
browser.find_element_by_id('Name').send_keys("Name here")
browser.find_element_by_id("ID").send_keys("123456")
browser.find_element_by_id("email").send_keys("email#email.com")
I have found examples where passing the browser in this way worked. However, when I tried to run it using pytest, I recieved the error:
fixture 'browser' not found
available fixtures: cache, capfd, capsys, doctest_namespace, monkeypatch, pytestconfig, record_xml_property, recwarn, tmpdir, tmpdir_factory
use 'pytest --fixtures [testpath]' for help on them.
Does anyone know how to implement this so that the tests can run as intended?
EDIT: I downloaded chromedriver and set a variable
driver= webdriver.Chrome('chromedriver.exe')
This seems to have done something to make progress ubt now I am getting an error:
ValueError: Plugin already registered: pytest_webdriver=)>

You can initialize browser before calling it, for example.
browser = webdriver.Chrome()
Or you make a fixture yourself by defining the function ideally in a file called conftest.py. You can read more on the pytest website. Here's an example:
#pytest.yield_fixture(scope='session')
def browser():
driver = webdriver.Chrome()
yield driver
driver.quit()

pytest is looking for browser fixture. Probably you don't have this fixture declared. you can read about fixtures here -> https://docs.pytest.org/en/latest/fixture.html.
and you can use pytest-selenium plugin http://pytest-selenium.readthedocs.io/en/latest/index.html
however, you can try to implement something like this in the same file...
#pytest.fixture
def browser():
"pytest fixture for browser"
return pytest.config.getoption("-B")

Clarification for anyone who stumbles upon this when trying to figure our where to get a browser fixture for Playwright. The browser fixture is made available by pytest-playwright (https://pypi.org/project/pytest-playwright/). You probably don't need to go down a selenium/chromium rabbit hole to initialise it.

Tor Browser with RSelenium in Linux/Windows

Looking to use RSelenium and Tor using my Linux machine to return the Tor IP (w/Firefox as Tor Browser). This is doable with Python, but having trouble with it in R. Can anybody get this to work? Perhaps you can share your solution in either Windows / Linux.
# library(devtools)
# devtools::install_github("ropensci/RSelenium")
library(RSelenium)
RSelenium::checkForServer()
RSelenium::startServer()
binaryExtension <- paste0(Sys.getenv('HOME'),"/Desktop/tor-browser_en-US/Browser/firefox")
remDr <- remoteDriver(dir = binaryExtention)
remDr$open()
remDr$navigate("http://myexternalip.com/raw")
remDr$quit()
The error Error in callSuper(...) : object 'binaryExtention' not found is being returned.
For community reference, this Selenium code works in Windows using Python3:
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
from os.path import expanduser # Finds user's user name on Windows
# Substring inserted to overcome r requirement in FirefoxBinary
binary = FirefoxBinary(r"%s\\Desktop\\Tor Browser\\Browser\\firefox.exe" % (expanduser("~")))
profile = FirefoxProfile(r"%s\\Desktop\\Tor Browser\\Browser\\TorBrowser\\Data\\Browser\\profile.default" % (expanduser("~")))
driver = webdriver.Firefox(profile, binary)
driver.get('http://myexternalip.com/raw')
html = driver.page_source
soup = BeautifulSoup(html, "lxml") # lxml needed
# driver.close()
# line.strip('\n')
"Current Tor IP: " + soup.text.strip('\n')
# Based in part on
# http://stackoverflow.com/questions/13960326/how-can-i-parse-a-website-using-selenium-and-beautifulsoup-in-python
# http://stackoverflow.com/questions/34316878/python-selenium-binding-with-tor-browser
# http://stackoverflow.com/questions/3367288/insert-variable-values-into-a-string-in-python

Something like the following should work:
browserP <- paste0(Sys.getenv('HOME'),"/Desktop/tor-browser_en-US/Browser/firefox")
jArg <- paste0("-Dwebdriver.firefox.bin='", browserP, "'")
selServ <- RSelenium::startServer(javaargs = jArg)
UPDATE:
This worked for me on windows. Firstly run the beta version:
checkForServer(update = TRUE, beta = TRUE, rename = FALSE)
Next open a version of the tor browser manually.
library(RSelenium)
browserP <- "C:/Users/john/Desktop/Tor Browser/Browser/firefox.exe"
jArg <- paste0("-Dwebdriver.firefox.bin=\"", browserP, "\"")
pLoc <- "C:/Users/john/Desktop/Tor Browser/Browser/TorBrowser/Data/Browser/profile.meek-http-helper/"
jArg <- c(jArg, paste0("-Dwebdriver.firefox.profile=\"", pLoc, "\""))
selServ <- RSelenium::startServer(javaargs = jArg)
remDr <- remoteDriver(extraCapabilities = list(marionette = TRUE))
remDr$open()
remDr$navigate("https://check.torproject.org/")
> remDr$getTitle()
[[1]]
[1] "Congratulations. This browser is configured to use Tor."

This works in MacOS Sierra.
First you need to configure both the Firefox and Tor browser Manual Proxy.
Go to your Preferences>Advanced>Network>Settings
Set SOCKS Host: 127.0.0.1 Port:9150 Check -> on SOCKS v5 in the browser menu bar.
You will also need to have Tor Browser open whilst running the R script in Rstudio ....otherwise you will get a message in the firefox browser "The proxy server is refusing connections"
You will also need to copy the name of your firefox profile in the script profile-name
Open Finder and got to /Users/username/Library/Application Support/Firefox/Profiles/profile-name
My R test script
require(RSelenium)
fprof <- getFirefoxProfile("/Users/**username**/Library/Application\ Support/Firefox/Profiles/nfqudbv2.default-1484451212373",useBase=TRUE)
remDrv <- remoteDriver( browserName = "firefox"
, extraCapabilities = fprof)
remDrv$open()
remDrv$navigate("https://check.torproject.org/")
This will open an instance of the Firefox browser with the message
"Congratulations. This browser is configured to use Tor."

Caveat: I have not tested extensively, but it seems to work.
Relying on some ideas from #Ashley72 but avoiding manual setups and copying (as well as now defunct functions from Rselenium needed for the solution from #jdharrison) and some ideas from https://indranilgayen.wordpress.com/2016/10/24/make-rselenium-work-with-r/ adjust the following profile options (I usually adjust a number of other options, but they do not seem relevant for the question):
fprof <- makeFirefoxProfile(list(network.proxy.socks = "127.0.0.1", # for proxy settings specify the proxy host IP
network.proxy.socks_port = 9150L, # proxy port. Last character "L" for specifying integer is very important and if not specified it will not have any impact
network.proxy.type = 1L, # 1 for manual and 2 for automatic configuration script. here also "L" is important
network.proxy.socks_version=5L, #ditto
network.proxy.socks_remote_dns=TRUE))
Then you start the server as usual:
rD <- rsDriver(port = 4445L, browser = "firefox", version = "latest", geckover = "latest", iedrver = NULL, phantomver = "2.1.1",
verbose = TRUE, check = TRUE, extraCapabilities = fprof) # works for selenium server: 3.3.1 and geckover: 0.15.0; Firefox: 52
remDr <- rD[["client"]]
remDr <- rD$client
remDr$navigate("https://check.torproject.org/") # should confirm tor is setup
remDr$navigate("http://whatismyip.org/") # should confirm tor is setup
As you see, I have not made changes to the marionette option. I have no idea what the implications might be. Please comment.
EDIT: the Tor Browser has to be up and running, it seems. Otherwise, the browser opened by Rselenium gives an error "proxy server refusing connection."