We Keep Coding

How do we can use both the headless mode option and user-data-dir option?

I want to use both of the two options for screenshot sites need login.
headless mode --headless
specify user data directory --user-data-dir (set fullpath on Windows)
But this seems not to work correctly because headless driver does not hold login status.
(in the below code, I used function construct_driver and the argument headless_flg=True)
I created the user-data (and profile dir) that is read from the headless driver by creating and opening the user-data firstly from the normal driver (no headless option, login the sites, and persisted.
I confirmed that the normal driver correctly read the same profile and already logined the site from second driver run.
(in the below code, I used function construct_driver and the argument headless_flg=False)
So, I think the cause of this problem is that headless driver do not read the user-data correctly.
Is there the way to specify the user-data on headless chrome driver?
This is my code (python):
from selenium import webdriver as wd
from selenium.webdriver.chrome import service as cs
BASE_DIR = "<fullpath of the user data dir to be located>"
DRIVER_VERSION = "101.0.4951.41"
def construct_driver(headless_flg=False, profile_no=1):
csi = cs.Service(f"{BASE_DIR}/{DRIVER_VERSION}/chromedriver")
options = wd.ChromeOptions()
userdata_dir = f"{BASE_DIR}/data/Chrome-{str(profile_no)}"
if(headless_flg):
options.add_argument('--headless')
options.add_argument(f"--user-data-dir={userdata_dir}")
# profile = "1"
# options.add_argument(f"--profile-directory=Default")
driver = wd.Chrome(service=csi, options=options)
driver.implicitly_wait(10)
return driver

I am not familiar with phyton, but managed your request in C#.
There are two complete different options and the start sequence.
Maybe it will help you a little bit or give you an idea.
// hide driver Console? true/false
ChromeDriverService service = ChromeDriverService.CreateDefaultService();
service.HideCommandPromptWindow = false;
// change Standard-Download-Folder
ChromeOptions options = new ChromeOptions();
var downloadDirectory = GlobalVars.RootPath + GlobalVars.strSymbol;
options.AddUserProfilePreference("download.default_directory", downloadDirectory);
options.AddUserProfilePreference("download.prompt_for_download", false);
options.AddUserProfilePreference("disable-popup-blocking", "true");
// start Selenium Driver:
webdriver = new ChromeDriver(service, options);

Python Selenium Chrome "user data directory is already in use"

I'm trying to make a script in python with selenium that will open a website and click a button.
Problem is that I have to close chrome before I can run the script otherwise I get this error:
Message: invalid argument: user data directory is already in use, please specify a unique value for --user-data-dir argument, or don't use --user-data-dir
Is there a way around this so I don't have to close chrome before running the script everytime?
My code:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import time
PATH = "C:\Program Files (x86)\chromedriver.exe"
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=C:/Users/username/AppData/Local/Google/Chrome/User Data")
driver = webdriver.Chrome(executable_path=r'C:\Program Files (x86)\chromedriver.exe', chrome_options=options)
driver.get("https://disboard.org/sv/dashboard/servers")
link = driver.find_element_by_link_text("bump")
link.click()

This error message...
invalid argument: user data directory is already in use, please specify a unique value for --user-data-dir argument, or don't use --user-data-dir
...implies that the user data directory Default is already in use so ChromeDriver was unable to access the directory and initiate/spawn a new Browsing Context i.e. Chrome Browser session.
Solution
In such cases, you can add/create a new Chrome Profile following the steps below and use it for the AUT (Application Under Test):
Open Google Chrome, and select the user icon in the top right and click on Add.
Enter a name for the new person, select an icon to help represent this new account and make it easier to find later. Then, select Add.
You will find the Chrome Profile shortcut created on the Desktop.
Additionally, you will find a new sub-directory Profile 1 being created beside Default
Now, you can use the Profile 1 sub-directory as follows:
options = Options()
options.add_argument("start-maximized")
options.add_argument("--profile-directory=Profile 1")
options.add_argument("--user-data-dir=C:/Users/user/AppData/Local/Google/Chrome/User Data")
driver = webdriver.Chrome(executable_path=r'C:\BrowserDrivers\chromedriver.exe', options=options)
driver.get("https://www.google.com/")
References
You can find a couple of relevant detailed discussions in:
How to use Chrome Profile in Selenium Webdriver Python 3
selenium.common.exceptions.InvalidArgumentException: Message: invalid argument: user data directory is already in use error with real Chrome Browser
Selenium: Point towards default Chrome session

If it is crucial for your use case that you can use both your Chrome profile and your automation with your profile at the same time, here's a possible solution:
Install a second (older or newer) version of Chrome as described in this thread
Enable synchronization on the desired profile in your current Chrome.
Log in with the desired profile from the second version of Chrome and enable synchronization there as well.
Use the chrome driver compatible with the second version of Chrome in your code.
There may be some problems with synchronization not loading everything that you need, but if it's something simple like saved passwords then this should work.

The reason for your issue is because you have added the following argument:
options.add_argument("user-data-dir=C:/Users/username/AppData/Local/Google/Chrome/User Data")
The argument is instructing selenium to use your local chrome profile. Hence, as you have a chrome session open, the automation cannot execute until you close down the browser session.
Remove the argument to execute an independent automation test profile.
See below for the remediated code provided from your question
from selenium import webdriver
PATH = "C:\Program Files (x86)\chromedriver.exe"
options = webdriver.ChromeOptions()
driver = webdriver.Chrome(executable_path=PATH, options=options)
driver.get("https://disboard.org/sv/dashboard/servers")
link = driver.find_element_by_link_text("bump")
link.click()

python selenium firefox - add_extension not working

Trying to add uBlock to a browser session but it's not working.
import selenium
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
from selenium.webdriver.firefox.options import Options as options
def establish_browser(type, hide):
browser = ''
if type == 'firefox':
ops = options()
ops.add_argument("--headless") if hide is True else ops.add_argument("--head")
profile = selenium.webdriver.FirefoxProfile()
profile.add_extension(extension='uBlock0#raymondhill.net.xpi')
browser = selenium.webdriver.Firefox(firefox_profile=profile, executable_path='geckodriver.exe', options=ops, firefox_binary=FirefoxBinary('C:/Program Files/Mozilla Firefox/firefox.exe'))
return browser
browser = establish_browser('firefox', False)
How should this be changed so uBlock works?
UPDATE
The chrome version appears to be working …
if type == 'chrome':
from selenium.webdriver.chrome.options import Options as options
ops = options()
ops.add_argument("--headless") if hide is True else ops.add_argument("--head")
ops.add_extension("ublock.crx")
browser = selenium.webdriver.Chrome(executable_path='chromedriver.exe', options=ops, desired_capabilities={'binary_location': 'C:/Program Files (x86)/Google/Chrome/Application/chrome.exe'})
is Firefox depreciated?

Since, for some reason, chrome's add_extension works but firefox's add_extension does not work (currently) … here is my workaround for adding extensions to firefox.
create a new firefox profile via right click windows start button > run > firefox.exe -P
Then add whatever extensions you want, ublock, adblock plus etc
call your profile folder with
profile = selenium.webdriver.FirefoxProfile("C:/test")
browser = selenium.webdriver.Firefox(firefox_profile=profile, options=ops)
Apparently profile.add_extension() is not a must have for this workaround
UPDATE! - added chrome profile
For symmetry purposes I have updated the chrome example code to use the chrome profile instead of calling the .crx directly.
install extensions onto chrome's default profile.
navigate to C:\Users\User\AppData\Local\Google\Chrome or where-ever chromes User Data folder is located. Call this folder directly (absolute path) or rename it and call the relative path. I have renamed it to chrome_profile:
ops = options()
ops.add_argument("--headless") if hide is True else ops.add_argument("--head")
ops.add_argument('user-data-dir=chrome_profile')
ops.add_argument('--profile-directory=Default')
ops.add_argument("--incognito")
browser = selenium.webdriver.Chrome(executable_path='chromedriver.exe', options=ops, desired_capabilities={'binary_location': 'C:/Program Files (x86)/Google/Chrome/Application/chrome.exe'})

To add to #Rhys' solution, an easier approach might be the following option from the official documentation which works as expected:
driver = webdriver.Firefox('path/to/executable')
driver.install_addon('~/path/to/addon.xpi')

How to use Chrome Profile in Selenium Webdriver Python 3

So whenever I try to use my Chrome settings (the settings I use in the default browser) by adding
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=C:\Users\... (my webdriver path)")
driver = webdriver.Chrome(executable_path="myPath", options=options)
it shows me the error code
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes n 16-17: truncated \UXXXXXXXX escape
in my bash. I don't know what that means and I'd be happy for any kind of help I can get. Thanks in advance!

The accepted answer is wrong. This is the official and correct way to do it:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = webdriver.ChromeOptions()
options.add_argument(r"--user-data-dir=C:\path\to\chrome\user\data") #e.g. C:\Users\You\AppData\Local\Google\Chrome\User Data
options.add_argument(r'--profile-directory=YourProfileDir') #e.g. Profile 3
driver = webdriver.Chrome(executable_path=r'C:\path\to\chromedriver.exe', chrome_options=options)
driver.get("https://www.google.co.in")
To find the profile folder on Windows right-click the desktop shortcut of the Chrome profile you want to use and go to properties -> shortcut and you will find it in the "target" text box.

To get the path, follow the steps below.
In the search bar type the following and press enter
This will then show all the metadata. There find the path to the profile

As per your question and your code trials if you want to open a Chrome Browsing Session here are the following options:
To use the default Chrome Profile:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=C:\\Users\\AtechM_03\\AppData\\Local\\Google\\Chrome\\User Data\\Default")
driver = webdriver.Chrome(executable_path=r'C:\path\to\chromedriver.exe', chrome_options=options)
driver.get("https://www.google.co.in")
Note: Your default chrome profile would contain a lot of bookmarks, extensions, theme, cookies etc. Selenium may fail to load it. So as per the best practices create a new chrome profile for your #Test and store/save/configure within the profile the required data.
To use the customized Chrome Profile:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("user-data-dir=C:\\Users\\AtechM_03\\AppData\\Local\\Google\\Chrome\\User Data\\Profile 2")
driver = webdriver.Chrome(executable_path=r'C:\path\to\chromedriver.exe', chrome_options=options)
driver.get("https://www.google.co.in")
Here you will find a detailed discussion on How to open a Chrome Profile through Python

This is how I managed to use EXISTING CHROME PROFILE in php selenium webdriver.
Profile 6 is NOT my default profile. I dont know how to run default profile. It is IMPORTANT not to add -- before chrome option arguments! All other variants of options didnt work!
<?php
//...
$chromeOptions = new ChromeOptions();
$chromeOptions->addArguments([
'user-data-dir=C:/Users/MyUser/AppData/Local/Google/Chrome/User Data',
'profile-directory=Profile 6'
]);
$host = 'http://localhost:4444/wd/hub'; // this is the default
$capabilities = DesiredCapabilities::chrome();
$capabilities->setCapability(ChromeOptions::CAPABILITY, $chromeOptions);
$driver = RemoteWebDriver::create($host, $capabilities, 100000, 100000);
To get name of your chrome profile, go to chrome://settings/manageProfile, click on profile icon, click "Show profile shortcut on my desktop". After that right click on desktop profile icon and go to properties, here you will see something like "C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" --profile-directory="Profile 6".
Also I recommend you to close all chrome instances before running this code. Also maybe you need to TURN OFF chrome settings > advanced > system > "Continue running background apps when Google Chrome is closed".

None of the given answers were working for me so I researched a bit and now the working code is for is this one. I copied the user dir folder from Profile Path from chrome://version/ and made another argument for the profile as shown below:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = webdriver.ChromeOptions()
options.add_argument('user-data-dir=C:\\Users\\gupta\\AppData\\Local\\Google\\Chrome\\User Data')
options.add_argument('profile-directory=Profile 1')
driver = webdriver.Chrome(executable_path=r'C:\Program Files (x86)\chromedriver.exe', options=options)
driver.get('https://google.com')

Are you sure you are meant to be putting in the webdriver path in the user-data-dir argument? That's usually where you put your chrome profile e.g. "C:\Users\yourusername\AppData\Local\Google\Chrome\User Data\Profile 1\". Also you will need to use either double backslashes or forward slashes in your directory path (both work). You can test if your path works by using the os library
e.g.
import os
os.list("C:\\Users\\yourusername\\AppData\\Local\\Google\\Chrome\\User Data\\Profile 1")
will give you the directory listing.
I might also add that occasionally if you manage to crash chrome while running webdriver with a nominated user profile, that it seems to record the crash in the profile and the next time you open chrome, you get the Chrome prompt to restore pages after it exited abnormally. For me personally this had been a bit of headache to deal with and I no longer use a user profile with chromedriver because of it. I could not find a way around it. Other people have reported it here, but none of their solutions seemed to work for me, or were not suitable for my test cases. https://superuser.com/questions/237608/how-to-hide-chrome-warning-after-crash
If you don't nominate a user profile it seems to create a new (blank) temporary one each time it runs

Make sure you've got the path to the profile right, and that you double escape backslashes in said path.
For example, typically the default profile on windows is located at:
"C:\\Users\\user\\AppData\\Local\\Google\\Chrome\\User Data\\Default"

I managed to launch my chrome profile using these arguments:
ChromeOptions options = new ChromeOptions();
options.addArguments("--user-data-dir=C:\\Users\\user\\AppData\\Local\\Google\\Chrome\\User Data");
options.addArguments("--profile-directory=Profile 2");
WebDriver driver = new ChromeDriver(options);
You can find out more about the web driver here

You simply have to replace the '\' to '/' in your paths and that'll resolve it.

Get profile name by navigating to chrome://version from your chrome browser (You'll see Profile Path, but you only want the profile name from it (e.g. Profile 1)
Close out all Chrome sessions using the profile you want to use. (or else you will get the following error: InvalidArgumentException)
Now make sure you have the code below (Make sure you replace UserFolder with the name of your userfolder.
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=C:\\Users\\EnterYourUserFolder\\AppData\\Local\\Google\\Chrome\\User Data") #leave out the profile
options.add_argument("profile-directory=Profile 1") #enter profile here
driver = webdriver.Chrome(executable_path="C:\\chromedriver.exe", chrome_options=options)

this worked for me 100% and it showed up my selected profile.

import os
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
# path of your chrome webdriver
dir_path = os.getcwd()
user_profile_path = os.environ[ 'USERPROFILE' ]
#if "frtkpr" which is ll be your custom profile does not exist it will be created.
option.add_argument( "user-data-dir=" + user_profile_path + "/AppData/Local/Google/Chrome/User Data/frtkpr" )
driver = webdriver.Chrome( dir_path + "/chromedriver.exe",chrome_options=option )
baseUrl = "https://www.facebook.com/"
driver.maximize_window()
driver.get( baseUrl )

how can I use selenium with my normal browser

Is it possible to connect selenium to the browser I use normally instead a driver? For normal browsing I am using chrome with several plugins - add block plus, flashblock and several more. I want to try to load a site using this specific configuration. How can I do that?
p.s - I dont want to connect only to an open browser like in this question :
How to connect to an already open browser?
I dont care if I spawn the process using a driver. I just want the full browser configuration - cookies,plugins,fonts etc.
Thanks

First, you need to download the ChromeDriver, then either put the path to the executeable to the PATH environment variable, or pass the path in the executable_path argument:
from selenium import webdriver
driver = webdriver.Chrome(executable_path='/path/to/executeable/chrome/driver')
In order to load extensions, you would need to set ChromeOptions:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = webdriver.ChromeOptions()
options.add_extension('Adblock-Plus_v1.4.1.crx')
driver = webdriver.Chrome(chrome_options=options)
You can also save the chrome user profile you have and load it to the ChromeDriver:
options = webdriver.ChromeOptions()
options.add_argument('--user-data-dir=/path/to/my/profile')
driver = webdriver.Chrome(chrome_options=options)
See also:
Running Selenium WebDriver using Python with extensions (.crx files)
ChromeDriver capabilities/options