We Keep Coding

How to use a existing chrome installation with selenium webdriver?

I would like to use an existing installation of chrome (or firefox or brave browser) with selenium. Like that I could set prespecified settings / extensions (e.g. start nord-vpn when opening a new instance) that are active when the browser is opened with selenium.
I know there is selenium.webdriver.service with the "executeable-path" option, but it doesn't seem to work when you specify a specific chrome.exe, the usage seems to be for the chrome-driver only and then it still opens a "fresh" installation of chrome.
Starting selenium with extension-file I think is also not an option to use with the nord-vpn extension, as I have two-factor authentication active and login every single time would take too much time and effort, if possible at all.

Firefox profile
To use the existing installation of firefox you have to pass the profile path through set_preference() method using an instance of Option from selenium.webdriver.common.options as follows:
from selenium.webdriver import Firefox
from selenium import webdriver
from selenium.webdriver.firefox.service import Service
from selenium.webdriver.firefox.options import Options
profile_path = r'C:\Users\Admin\AppData\Roaming\Mozilla\Firefox\Profiles\s8543x41.default-release'
options=Options()
options.set_preference('profile', profile_path)
service = Service('C:\\BrowserDrivers\\geckodriver.exe')
driver = Firefox(service=service, options=options)
driver.get("https://www.google.com")
You can find a relevant detailed discussion in Error update preferences in Firefox profile: 'Options' object has no attribute 'update_preferences'
Chrome profile
Where as to use an existing installation of google-chrome you have to pass the user profile path through add_argument() using the user-data-dir key through an instance of Option from selenium.webdriver.common.options as follows:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
options = Options()
options.add_argument("user-data-dir=C:\\Users\\username\\AppData\\Local\\Google\\Chrome\\User Data\\Default")
s = Service('C:\\BrowserDrivers\\chromedriver.exe')
driver = webdriver.Chrome(service=s, options=options)
driver.get("https://www.google.com/")
You can find a relevant detailed discussion in How to open a Chrome Profile through Python

Is there a way to hide the browser while running selenium in Python?

I am working on a project with selenium to scrape the data, but I don't want the browser to open and pop up. I just wanted to hide the browser and also not to display it in the taskbar also...
Some also suggested to use phantomJS but I didn't get them. What to do now ...

If you're using Chrome you can just set the headless argument like so:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
driver_exe = 'chromedriver'
options = Options()
options.add_argument("--headless")
driver = webdriver.Chrome(driver_exe, options=options)

For chrome you could pass in the --headless parameter.
Alternatively you could let selenium work on a virtual display like this:
from selenium import webdriver
from xvfbwrapper import Xvfb
display = Xvfb()
display.start()
driver = webdriver.Chrome()
driver.get('http://www.stackoverflow.com')
print(driver.title)
driver.quit()
display.stop()
The latter has worked for me quite well.

To hide the browser while executing tests using Selenium's python you can use the minimize_window() method which eventually minimizes/pushes the Chrome Browsing Context effectively to the background using the following solution:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("--start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(chrome_options=options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
driver.get('https://www.google.co.in')
driver.minimize_window()
Alternative
As an alternative you can use the headless attribute to configure ChromeDriver to initiate google-chrome browser in Headless mode using Selenium and you can find a couple of relevant discussions in:
How to configure ChromeDriver to initiate Chrome browser in Headless mode through Selenium?

If you're using Firefox, try this:
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
driver_exe = 'path/to/firefoxdriver'
options = Options()
options.add_argument("--headless")
driver = webdriver.Firefox(driver_exe, options=options)
similar to what #Meshi answered in case of Chrome

if you want to hide chrome or selenium driver there is a library pyautogui
import pyautogui
window = [ x for x in pyautogui.getAllWindows()]
by this, you are getting all window title
now you need to find your window
for i in window:
if 'Google Chrome' in i.title:
i.hide()
or you can play with your driver title also

Loading ublock with custom filter in Chromium Selenium and Python3

I'm developing a scraping script to collect some data which is behind an authwall, I've got a custom filter in ublock which gets me past the authwall however when i load chromium with ublock using Selenium it doesn't have the filters. I'm using Linux if that helps.
I've tried getting it to pause before getting the information to allow me to check the filters in place and it is blank.
Here is a portion of the code
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_option_settings = Options()
chrome_option_settings.add_argument('--window-size=1920x1080')
extension_path = r'/home/user/.config/chromium/Default/Extensions/cjpalhdlnbpafiamejdnhcphjbkeiagm/1.20.0_0'
chrome_option_settings.add_argument('load-extension='+extension_path)
chrome_driver = "/usr/bin/chromedriver"
driver = webdriver.Chrome(chrome_options=chrome_option_settings, executable_path=chrome_driver)
driver.get(url)
I've also tried to load the Chrome profile using either however neither help.
chrome_options.add_argument("user-data-dir=/home/user/.config/chromium/Default")
or
chrome_options.add_argument("--profile-directory=/home/user/.config/chromium/Default")
Any help would be greatly appreciated

You could try using Options() and invoking add_extension with the path to ublock, hope this helps
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
executable_path = "path_to_webdriver"
extension_path = r'/home/user/.config/chromium/Default/Extensions/cjpalhdlnbpafiamejdnhcphjbkeiagm/1.20.0_0'
chrome_options.add_extension(extension_path)
driver = webdriver.Chrome(executable_path=executable_path, chrome_options=chrome_options)

Chrome opens with “Data;” with selenium chromedriver

Trying to open "Google" or any other page (website) from Chrome via selenium chrome driver in python.
The code is :
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from time import sleep
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support.ui import WebDriverWait
import time
driver = webdriver.Chrome()
driver.get('https://google.com')
However, this opens my chrome window with the specified link and "data;" tab.
Why that data; tab opens? How to fix it?
Using latest versions of Chrome and Chromedriver

You don't need as much module for this just remove all of those apart from:
from selenium import webdriver
And try again you will not get another tab with congaing data.

import time
time.sleep(1)
driver.switch_to.window(driver.window_handles[1])
driver.close()
driver.switch_to.window(driver.window_handles[0])
time.sleep(1)

I'm not sure if its the same problem, but some time ago I made an exe script to run in another PCs and in one of the PCs selenium just didn't worked with Chrome.
This is the question I posted, but the answers didn't help me, hope it works with you: Chromedriver do not open a new session, it opens a new tab in a existing session
If it doesn't work, I made a workaround to run with Firefox instead of Chrome to ensure it would work properly.

With selenium 4 (or newer), you can use the following code to launch a new browser, (without the Chrome is being controlled message), and then the browser navigates to Google in the same tab:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service as ChromeService
options = webdriver.ChromeOptions()
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option("useAutomationExtension", False)
service = ChromeService()
driver = webdriver.Chrome(service=service, options=options)
driver.get('https://google.com')

Python/Selenium incognito/private mode

I can not seem to find any documentation on how to make Selenium open the browser in incognito mode.
Do I have to setup a custom profile in the browser or?

First of all, since selenium by default starts up a browser with a clean, brand-new profile, you are actually already browsing privately. Referring to:
Python - Start firefox with Selenium in private mode
How might I simulate a private browsing experience in Watir? (Selenium)
But you can strictly enforce/turn on incognito/private mode anyway.
For chrome pass --incognito command-line argument:
--incognito Causes the browser to launch directly in incognito mode.
from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--incognito")
driver = webdriver.Chrome(chrome_options=chrome_options)
driver.get('https://google.com')
FYI, here is what it would open up:
For firefox, set browser.privatebrowsing.autostart to True:
from selenium import webdriver
firefox_profile = webdriver.FirefoxProfile()
firefox_profile.set_preference("browser.privatebrowsing.autostart", True)
driver = webdriver.Firefox(firefox_profile=firefox_profile)
FYI, this corresponds to the following checkbox in settings:

Note: chrome_options is now deprecated. We can use 'options' instead of chrome_options
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("--incognito")
driver = webdriver.Chrome(options=options)
driver.get('https://google.com')

I have initiated both Chrome and Firefox in incognito/Private mode using ChromeOptions and FirefoxOptions successfully using the code snippets in Java as below:
//For Firefox
FirefoxOptions options = new FirefoxOptions();
options.addArguments("-private");
DesiredCapabilities caps = new DesiredCapabilities();
caps.setCapability("moz:firefoxOptions",options);
//For Chrome
ChromeOptions options = new ChromeOptions();
options.addArguments("-incognito");
caps.setCapability(ChromeOptions.CAPABILITY, options);
WebDriver driver = new RemoteWebDriver(new URL(URL), caps);

There is a really simple way to make a window open in incognito mode:
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
# incognito window
chrome_options.add_argument("--incognito")
You can also use this library for maximizing the window and more, see the documentation: https://seleniumhq.github.io/selenium/docs/api/rb/Selenium/WebDriver/Chrome/Options.html

For firefox : (Python) ==>
from selenium import webdriver
firefox_options = webdriver.FirefoxOptions()
firefox_options.add_argument("--private")
browser = webdriver.Firefox(firefox_options=firefox_options)

//We need to add argument "--incogneto" in ChromeOptions object and pass this ChromeOptions instance to the web driver initialization.
ChromeOptions options = new ChromeOptions()
options.addArgument("start-maximized");
options.addArgument("--incognito");
ChromeDriver driver = new ChromeDriver(options);
driver.get("https://investopedia.com");

In Chrome Browser You Can Do This Using Python As Follows
As you can see when you uses chrome, you have the option of incognito mode in the options menu part of the chrome browser. So when you are using selenium, you can alter the things of options using
chrome_options = webdriver.ChromeOptions()
So, the code is:
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--incognito")
driver = webdriver.Chrome(executable_path="<path of chrome_driver.exe file>",options=chrome_options)
So the only thing you have to do is to give "webdriver.Chrome" this given value to its another parameter i.e. "options".

For python with opera
from selenium import webdriver
options = webdriver.opera.webdriver.Options()
options.add_argument("private")
driver = webdriver.Opera(executable_path="operadriver",options=options)

PowerShell
try{
# Import the Selenium DLLs
Add-Type -Path "$Seleniumlib\Selenium.WebDriverBackedSelenium.dll"
Add-Type -Path "$Seleniumlib\WebDriver.dll"
Add-Type -Path "$Seleniumlib\WebDriver.Support.dll"
}
catch [Exception]{
Write-Host ("Error: {0}" -f $_.Exception.Message)
exit 1
}
$options = New-Object OpenQA.Selenium.Chrome.ChromeOptions
$options.AddArgument("--incognito")
$driver = New-Object OpenQA.Selenium.Chrome.ChromeDriver($options)

Some options have been deprecated so for Firefox it worked out for me like this:
from selenium.webdriver.firefox.options import Options
from selenium import webdriver
firefox_options = Options()
firefox_options.add_argument("-private")
driver = webdriver.Firefox(options=firefox_options)