I would like to use an existing installation of chrome (or firefox or brave browser) with selenium. Like that I could set prespecified settings / extensions (e.g. start nord-vpn when opening a new instance) that are active when the browser is opened with selenium.
I know there is selenium.webdriver.service with the "executeable-path" option, but it doesn't seem to work when you specify a specific chrome.exe, the usage seems to be for the chrome-driver only and then it still opens a "fresh" installation of chrome.
Starting selenium with extension-file I think is also not an option to use with the nord-vpn extension, as I have two-factor authentication active and login every single time would take too much time and effort, if possible at all.
Firefox profile
To use the existing installation of firefox you have to pass the profile path through set_preference() method using an instance of Option from selenium.webdriver.common.options as follows:
from selenium.webdriver import Firefox
from selenium import webdriver
from selenium.webdriver.firefox.service import Service
from selenium.webdriver.firefox.options import Options
profile_path = r'C:\Users\Admin\AppData\Roaming\Mozilla\Firefox\Profiles\s8543x41.default-release'
options=Options()
options.set_preference('profile', profile_path)
service = Service('C:\\BrowserDrivers\\geckodriver.exe')
driver = Firefox(service=service, options=options)
driver.get("https://www.google.com")
You can find a relevant detailed discussion in Error update preferences in Firefox profile: 'Options' object has no attribute 'update_preferences'
Chrome profile
Where as to use an existing installation of google-chrome you have to pass the user profile path through add_argument() using the user-data-dir key through an instance of Option from selenium.webdriver.common.options as follows:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
options = Options()
options.add_argument("user-data-dir=C:\\Users\\username\\AppData\\Local\\Google\\Chrome\\User Data\\Default")
s = Service('C:\\BrowserDrivers\\chromedriver.exe')
driver = webdriver.Chrome(service=s, options=options)
driver.get("https://www.google.com/")
You can find a relevant detailed discussion in How to open a Chrome Profile through Python
I'm using Selenium to access a site, but I constantly get Warning: Potential Security Risk Ahead. I searched all over the Internet for a solution to my problem, but the message to accept the certificate manually continues to appear.
I'm using:
Firefox: 85.0.1 (64-bit)
Geckodriver
Python language
I tested several solutions such as:
from selenium import webdriver
capabilities = webdriver.DesiredCapabilities().FIREFOX
capabilities['acceptSslCerts'] = True
#capabilities['acceptInsecureCerts'] = True
driver = webdriver.Firefox(capabilities=capabilities)
driver.get('xxxxxxxxx')
And,
from selenium import webdriver
profile = webdriver.FirefoxProfile()
profile.accept_untrusted_certs = True
driver = webdriver.Firefox(firefox_profile=profile)
driver.get('xxxxxxxxx')
I also tried solutions based on creating a new profile in Firefox.
The question is: How can I automate the acceptance of a website's certificate when I launch Firefox with Selenium (Python programming language)?
https://developer.mozilla.org/en-US/docs/Web/WebDriver/Capabilities/acceptInsecureCerts
from selenium import webdriver
capabilities = webdriver.DesiredCapabilities().FIREFOX
capabilities['acceptInsecureCerts'] = True
capabilities['marionette'] = True
driver = webdriver.Firefox(desired_capabilities=capabilities)
driver.get("https://self-signed.badssl.com/")
I solved my problem creating a new Firefox Profile and executing:
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
options = Options()
#Path to the Firefox Profile directory
options.profile = r'C:\Users\x\AppData\Roaming\Mozilla\Firefox\Profiles\x.x'
driver = webdriver.Firefox(options=options)
driver.get('https://xxxxx')
I'm trying to initiate a tor browsing session through Tor Browser 9.5 which uses the default Firefox v68.9.0esr using GeckoDriver and Selenium through Python on a windows-10 system. But I'm facing an error as:
Code Block:
from selenium import webdriver
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
import os
torexe = os.popen(r'C:\Users\username\Desktop\Tor Browser\Browser\TorBrowser\Tor\tor.exe')
profile = FirefoxProfile(r'C:\Users\username\Desktop\Tor Browser\Browser\TorBrowser\Data\Browser\profile.default')
profile.set_preference('network.proxy.type', 1)
profile.set_preference('network.proxy.socks', '127.0.0.1')
profile.set_preference('network.proxy.socks_port', 9050)
profile.set_preference("network.proxy.socks_remote_dns", False)
profile.update_preferences()
firefox_options = webdriver.FirefoxOptions()
firefox_options.binary_location = r'C:\Users\username\Desktop\Tor Browser\Browser\firefox.exe'
driver = webdriver.Firefox(firefox_profile= profile, options = firefox_options, executable_path=r'C:\WebDrivers\geckodriver.exe')
driver.get("https://www.tiktok.com/")
Where as the same code block works through Firefox and Firefox Nightly using the respective binaries.
Do I need any additional settings? Can someone help me out?
Firefox Snapshot:
Firefox Nightly Snapshot:
I managed to resolve this by updating to v9.5.1 and implementing the following changes:
Note that although the code is in C# the same changes to the Tor browser and how it is launched should be applied.
FirefoxProfile profile = new FirefoxProfile(profilePath);
profile.SetPreference("network.proxy.type", 1);
profile.SetPreference("network.proxy.socks", "127.0.0.1");
profile.SetPreference("network.proxy.socks_port", 9153);
profile.SetPreference("network.proxy.socks_remote_dns", false);
FirefoxDriverService firefoxDriverService = FirefoxDriverService.CreateDefaultService(geckoDriverDirectory);
firefoxDriverService.FirefoxBinaryPath = torPath;
firefoxDriverService.BrowserCommunicationPort = 2828;
var firefoxOptions = new FirefoxOptions
{
Profile = null,
LogLevel = FirefoxDriverLogLevel.Trace
};
firefoxOptions.AddArguments("-profile", profilePath);
FirefoxDriver driver = new FirefoxDriver(firefoxDriverService, firefoxOptions);
driver.Navigate().GoToUrl("https://www.google.com");
Important notes:
The following TOR configs need to be changed in about:config :
marionette.enabled: true
marionette.port: set to an unused port, and set this value to firefoxDriverService.BrowserCommunicationPort in your code. This was set to 2828 in my example.
note:
I am not sure whether this really is the definite answer (thus, I'd really appreciate feedback)
solution:
I've managed to send a get request to the check tor page (https://check.torproject.org/) and it displayed an unknown IP to me (additionally, IPs differ if you repeat the request after a time)
Essentially, I've set up the chrome driver to run TOR. Here's the code:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
tor_proxy = "127.0.0.1:9150"
chrome_options = Options()
chrome_options.add_argument("--test-type")
chrome_options.add_argument('--ignore-certificate-errors')
chrome_options.add_argument('--disable-extensions')
chrome_options.add_argument('disable-infobars')
chrome_options.add_argument("--incognito")
chrome_options.add_argument('--proxy-server=socks5://%s' % tor_proxy)
driver = webdriver.Chrome(options=chrome_options)
driver.get('https://check.torproject.org/')
Because the driver is not in headless mode you can inspect the resulting page yourself. It should read:
"Congratulations. This browser is configured to use Tor. [IP Info]. However, it does not appear to be Tor Browser. Click here to go to the download page"
Make sure that the chromedriver.exe file is linked on the path or provide the path to the file as an argument to the driver.Chrome() function.
Edit: make sure TOR browser is running in the background, thanks #Abhishek Rai for pointing that out
I am working on downloading HAR from Chrome for YouTube through Selenium Python Script.
Code Snippet:
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--proxy-server={0}".format(url))
chrome_options.add_argument("--enable-quic")
self.driver = webdriver.Chrome(chromedriver,chrome_options = chrome_options)
self.proxy.new_har(args['url'], options={'captureHeaders': True})
self.driver.get(args['url'])
result = json.dumps(self.proxy.har, ensure_ascii=False)
I want QUIC to be used whenever I download HAR but when I look at the packets through Wireshark Selenium driver is using TCP only. Is there a way to force Chrome Driver to use QUIC? Or Is there an alternate to BMP?
A similar thing has been asked for Firefox in this question How to capture all requests made by page in webdriver? Is there any alternative to Browsermob? and there was a solution with Selenium alone without need of any BMP. So is it possible for Chrome?
Workaround for this problem could be: start Chrome normally (with your default profile or create another profile) and enable quic manually. Then start chromedriver with your profile loaded.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=/home/user/.config/google-chrome")
driver = webdriver.Chrome(executable_path="/home/user/Downloads/chromedriver", chrome_options=options)
I am using Selenium to launch a browser. How can I deal with the webpages (URLs) that will ask the browser to accept a certificate or not?
In Firefox, I may have a website like that asks me to accept its certificate like this:
On the Internet Explorer browser, I may get something like this:
On Google Chrome:
I repeat my question: How can I automate the acceptance of a website's certificate when I launch a browser (Internet Explorer, Firefox and Google Chrome) with Selenium (Python programming language)?
For the Firefox, you need to set accept_untrusted_certs FirefoxProfile() option to True:
from selenium import webdriver
profile = webdriver.FirefoxProfile()
profile.accept_untrusted_certs = True
driver = webdriver.Firefox(firefox_profile=profile)
driver.get('https://cacert.org/')
driver.close()
For Chrome, you need to add --ignore-certificate-errors ChromeOptions() argument:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument('ignore-certificate-errors')
driver = webdriver.Chrome(chrome_options=options)
driver.get('https://cacert.org/')
driver.close()
For the Internet Explorer, you need to set acceptSslCerts desired capability:
from selenium import webdriver
capabilities = webdriver.DesiredCapabilities().INTERNETEXPLORER
capabilities['acceptSslCerts'] = True
driver = webdriver.Ie(capabilities=capabilities)
driver.get('https://cacert.org/')
driver.close()
Actually, according to the Desired Capabilities documentation, setting acceptSslCerts capability to True should work for all browsers since it is a generic read/write capability:
acceptSslCerts
boolean
Whether the session should accept all SSL certs
by default.
Working demo for Firefox:
>>> from selenium import webdriver
Setting acceptSslCerts to False:
>>> capabilities = webdriver.DesiredCapabilities().FIREFOX
>>> capabilities['acceptSslCerts'] = False
>>> driver = webdriver.Firefox(capabilities=capabilities)
>>> driver.get('https://cacert.org/')
>>> print(driver.title)
Untrusted Connection
>>> driver.close()
Setting acceptSslCerts to True:
>>> capabilities = webdriver.DesiredCapabilities().FIREFOX
>>> capabilities['acceptSslCerts'] = True
>>> driver = webdriver.Firefox(capabilities=capabilities)
>>> driver.get('https://cacert.org/')
>>> print(driver.title)
Welcome to CAcert.org
>>> driver.close()
For Firefox:
ProfilesIni profile = new ProfilesIni();
FirefoxProfile myprofile = profile.getProfile("default");
myprofile.setAcceptUntrustedCertificates(true);
myprofile.setAssumeUntrustedCertificateIssuer(true);
WebDriver driver = new FirefoxDriver(myprofile);
For Chrome we can use:
DesiredCapabilities capabilities = DesiredCapabilities.chrome();
capabilities.setCapability("chrome.switches", Arrays.asList("--ignore-certificate-errors"));
driver = new ChromeDriver(capabilities);
For Internet Explorer we can use:
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
Webdriver driver = new InternetExplorerDriver(capabilities);
For Firefox Python:
The Firefox Self-signed certificate bug has now been fixed:
accept ssl cert with marionette firefox webdrive python splinter
"acceptSslCerts" should be replaced by "acceptInsecureCerts"
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
caps = DesiredCapabilities.FIREFOX.copy()
caps['acceptInsecureCerts'] = True
ff_binary = FirefoxBinary("path to the Nightly binary")
driver = webdriver.Firefox(firefox_binary=ff_binary, capabilities=caps)
driver.get("https://expired.badssl.com")
For people coming to this question related to headless chrome via python selenium, you may find https://bugs.chromium.org/p/chromium/issues/detail?id=721739#c102 to be useful.
It looks like you can either do
chrome_options = Options()
chrome_options.add_argument('--allow-insecure-localhost')
or something along the lines of the following (may need to adapt for python):
ChromeOptions options = new ChromeOptions()
DesiredCapabilities caps = DesiredCapabilities.chrome()
caps.setCapability(ChromeOptions.CAPABILITY, options)
caps.setCapability("acceptInsecureCerts", true)
WebDriver driver = new ChromeDriver(caps)
And in C# (.net core) using Selenium.Webdriver and Selenium.Chrome.Webdriver like this:
ChromeOptions options = new ChromeOptions();
options.AddArgument("--ignore-certificate-errors");
using (var driver = new ChromeDriver(Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location),options))
{
...
}
ChromeOptions options = new ChromeOptions().addArguments("--proxy-server=http://" + proxy);
options.setAcceptInsecureCerts(true);
Javascript:
const capabilities = webdriver.Capabilities.phantomjs();
capabilities.set(webdriver.Capability.ACCEPT_SSL_CERTS, true);
capabilities.set(webdriver.Capability.SECURE_SSL, false);
capabilities.set('phantomjs.cli.args', ['--web-security=no', '--ssl-protocol=any', '--ignore-ssl-errors=yes']);
const driver = new webdriver.Builder().withCapabilities(webdriver.Capabilities.chrome(), capabilities).build();
In selenium python, you need to set desired_capabilities as:
desired_capabilities = {
"acceptInsecureCerts": True
}
I ran into the same issue with Selenium and Behat. If you want to pass the parameters via behat.yml, here is what it needs to look like:
default:
extensions:
Behat\MinkExtension:
base_url: https://my-app.com
default_session: selenium2
selenium2:
browser: firefox
capabilities:
extra_capabilities:
acceptInsecureCerts: true
Creating a profile and then a driver helps us get around the certificate issue in Firefox:
var profile = new FirefoxProfile();
profile.SetPreference("network.automatic-ntlm-auth.trusted-uris","DESIREDURL");
driver = new FirefoxDriver(profile);
For those who come to this issue using Firefox and the above solutions don't work, you may try the code below (my original answer is here).
from selenium import webdriver
profile = webdriver.FirefoxProfile()
profile.DEFAULT_PREFERENCES['frozen']['marionette.contentListener'] = True
profile.DEFAULT_PREFERENCES['frozen']['network.stricttransportsecurity.preloadlist'] = False
profile.DEFAULT_PREFERENCES['frozen']['security.cert_pinning.enforcement_level'] = 0
profile.set_preference('webdriver_assume_untrusted_issuer', False)
profile.set_preference("browser.download.folderList", 2)
profile.set_preference("browser.download.manager.showWhenStarting", False)
profile.set_preference("browser.download.dir", temp_folder)
profile.set_preference("browser.helperApps.neverAsk.saveToDisk",
"text/plain, image/png")
driver = webdriver.Firefox(firefox_profile=profile)
WebDriverManager.chromedriver().setup();
ChromeOptions options = new ChromeOptions();
options.addArguments("--ignore-certificate-errors");
driver = new ChromeDriver(options);
I have used it for Java with Chrome browser it is working nice
Delete all but the necessary certificate from your browser's certificate store and then configure the browser to automatically select the certificate when only one certificate is present.
Just an update regarding this issue.
Require Drivers:
Linux: Centos 7 64bit, Window 7 64bit
Firefox: 52.0.3
Selenium Webdriver: 3.4.0 (Windows), 3.8.1 (Linux Centos)
GeckoDriver: v0.16.0 (Windows), v0.17.0 (Linux Centos)
Code
System.setProperty("webdriver.gecko.driver", "/home/seleniumproject/geckodrivers/linux/v0.17/geckodriver");
ProfilesIni ini = new ProfilesIni();
// Change the profile name to your own. The profile name can
// be found under .mozilla folder ~/.mozilla/firefox/profile.
// See you profile.ini for the default profile name
FirefoxProfile profile = ini.getProfile("default");
DesiredCapabilities cap = new DesiredCapabilities();
cap.setAcceptInsecureCerts(true);
FirefoxBinary firefoxBinary = new FirefoxBinary();
GeckoDriverService service =new GeckoDriverService.Builder(firefoxBinary)
.usingDriverExecutable(new
File("/home/seleniumproject/geckodrivers/linux/v0.17/geckodriver"))
.usingAnyFreePort()
.usingAnyFreePort()
.build();
try {
service.start();
} catch (IOException e) {
e.printStackTrace();
}
FirefoxOptions options = new FirefoxOptions().setBinary(firefoxBinary).setProfile(profile).addCapabilities(cap);
driver = new FirefoxDriver(options);
driver.get("https://www.google.com");
System.out.println("Life Title -> " + driver.getTitle());
driver.close();
I was able to do this on .net c# with PhantomJSDriver with selenium web driver 3.1
[TestMethod]
public void headless()
{
var driverService = PhantomJSDriverService.CreateDefaultService(#"C:\Driver\phantomjs\");
driverService.SuppressInitialDiagnosticInformation = true;
driverService.AddArgument("--web-security=no");
driverService.AddArgument("--ignore-ssl-errors=yes");
driver = new PhantomJSDriver(driverService);
driver.Navigate().GoToUrl("XXXXXX.aspx");
Thread.Sleep(6000);
}
Whenever I run into this issue with newer browsers, I just use AppRobotic Personal edition to click specific screen coordinates, or tab through the buttons and click.
Basically it's just using its macro functionality, but won't work on headless setups though.
I had the exact same issue. However when I tried opening the website manually in the browser the certificate was correct, but in the details the name was "DONOTTRUST".
The difference of certificate was caused by Fiddler that was running in background and decrypting all HTTPS content before reencrypting it.
To fix my problem, just close Fiddler on machine. If you need to keep Fiddler opened, then you can uncheck Decrypt SSL in Fiddler Settings.
For .NET, what worked for me was the following...
var chromeOptions = new ChromeOptions { AcceptInsecureCertificates = true };
Pretty much, it tells the ChromeDriver options not to halt browser execution when an insecure certificate is detected, and to proceed as normal.
It looks like it still doesn't have a standard decision of this problem. In other words - you still can't say "Okay, do a certification, whatever if you are Internet Explorer, Mozilla or Google Chrome". But I found one post that shows how to work around the problem in Mozilla Firefox. If you are interested in it, you can check it here.