We Keep Coding

How to get Selenium/GeckoDriverManager driver path in Python

Following this issue I encountered yesterday : selenium.common.exceptions.SessionNotCreatedException in Python Selenium using GeckoDriverManager, I would like to know if there was a way to delete at the end of a program execution the driver directory created by GeckoDriverManager.
Here's what I use to put in my codes :
import os, shutil
from selenium import webdriver
from selenium.webdriver.firefox.service import Service as FirefoxService
from webdriver_manager.firefox import GeckoDriverManager
service = FirefoxService(executable_path=GeckoDriverManager().install())
driver = webdriver.Firefox(service=service)
Every time this is executed, GeckoDriverManager creates a folder in my /tmp directory called rust_mozprofile<random chars and digits>. Is there any way or function to get the path to this folder, in order to delete it and its contents, for example using the code below?
def __cleanup(driverDirectoryPath):
if os.path.exists(driverDirectoryPath):
shutil.rmtree(driverDirectoryPath)

How to open a already logged-in chrome browser with undetected_chromedriver python

I want to open an instance of undetected_chromedriver with a pre-set Chrome profile (basically the same thing as this thread asks about but with undetected_chromedriver instead of selenium).
This code works for me, using selenium (the first bit is just cloning the Chrome profile directory into the project directory to avoid an error, based on this thread):
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
import undetected_chromedriver as uc
import os
import shutil
# copy Chrome 'Profile 2' folder into this python project folder (for some reason, this is the
# only way Selenium is able to open a chrome browser with a pre-set profile without Chrome getting
# upset about multiple uses of the data directory)
# first, delete the extra files that Chrome automatically adds to user data directories (needed
# if script has already been run)
dir = '{project directory}'
for files in os.listdir(dir):
path = os.path.join(dir, files)
if path not in [{files I want to keep in the project directory}]:
try:
shutil.rmtree(path)
except OSError:
os.remove(path)
# then copy chrome profile into project directory
src = 'C:/Users/{CHROME_USER_DIR}/AppData/Local/Google/Chrome/User Data/Profile 2'
dst = '{project directory}/User Data/Profile 2'
shutil.copytree(src, dst, symlinks=False, ignore=None, ignore_dangling_symlinks=False, dirs_exist_ok=False)
userdatadir = '{project directory}/User Data'
# initialize Chrome instance using Profile 2
options = webdriver.ChromeOptions()
options.add_argument(f"--user-data-dir={userdatadir}")
options.add_argument("profile-directory=Profile 2")
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=options)
driver.get('https://www.nyt.com/')
BUT, when I try to do the same thing using undetected_chromedriver:
uc_options = uc.ChromeOptions()
uc_options.add_argument(f"--user-data-dir={userdatadir}")
uc_options.add_argument("profile-directory=Profile 2")
driver = uc.Chrome(service=Service(ChromeDriverManager().install()), options=uc_options, use_subprocess=True)
Chrome errors out as so (see screenshot below), and the Chrome instance is not logged into the specified profile!
Thank you in advance for any help!

python selenium firefox - add_extension not working

Trying to add uBlock to a browser session but it's not working.
import selenium
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
from selenium.webdriver.firefox.options import Options as options
def establish_browser(type, hide):
browser = ''
if type == 'firefox':
ops = options()
ops.add_argument("--headless") if hide is True else ops.add_argument("--head")
profile = selenium.webdriver.FirefoxProfile()
profile.add_extension(extension='uBlock0#raymondhill.net.xpi')
browser = selenium.webdriver.Firefox(firefox_profile=profile, executable_path='geckodriver.exe', options=ops, firefox_binary=FirefoxBinary('C:/Program Files/Mozilla Firefox/firefox.exe'))
return browser
browser = establish_browser('firefox', False)
How should this be changed so uBlock works?
UPDATE
The chrome version appears to be working …
if type == 'chrome':
from selenium.webdriver.chrome.options import Options as options
ops = options()
ops.add_argument("--headless") if hide is True else ops.add_argument("--head")
ops.add_extension("ublock.crx")
browser = selenium.webdriver.Chrome(executable_path='chromedriver.exe', options=ops, desired_capabilities={'binary_location': 'C:/Program Files (x86)/Google/Chrome/Application/chrome.exe'})
is Firefox depreciated?

Since, for some reason, chrome's add_extension works but firefox's add_extension does not work (currently) … here is my workaround for adding extensions to firefox.
create a new firefox profile via right click windows start button > run > firefox.exe -P
Then add whatever extensions you want, ublock, adblock plus etc
call your profile folder with
profile = selenium.webdriver.FirefoxProfile("C:/test")
browser = selenium.webdriver.Firefox(firefox_profile=profile, options=ops)
Apparently profile.add_extension() is not a must have for this workaround
UPDATE! - added chrome profile
For symmetry purposes I have updated the chrome example code to use the chrome profile instead of calling the .crx directly.
install extensions onto chrome's default profile.
navigate to C:\Users\User\AppData\Local\Google\Chrome or where-ever chromes User Data folder is located. Call this folder directly (absolute path) or rename it and call the relative path. I have renamed it to chrome_profile:
ops = options()
ops.add_argument("--headless") if hide is True else ops.add_argument("--head")
ops.add_argument('user-data-dir=chrome_profile')
ops.add_argument('--profile-directory=Default')
ops.add_argument("--incognito")
browser = selenium.webdriver.Chrome(executable_path='chromedriver.exe', options=ops, desired_capabilities={'binary_location': 'C:/Program Files (x86)/Google/Chrome/Application/chrome.exe'})

To add to #Rhys' solution, an easier approach might be the following option from the official documentation which works as expected:
driver = webdriver.Firefox('path/to/executable')
driver.install_addon('~/path/to/addon.xpi')

Python/Webdriver: how do I add browser binaries to path when I do not have admin rights?

I need to make three exe files visible to Python by placing them in a location where Python can find them. Placing the files in the same folder with the .py file did not solve the problem
I have no admin rights on my laptop and I can not change the PATH (Windows 10 machine) neither can I place files in the folders where the PATH variable points to
what are my options to trick Python to find the files ?

In your answer above it looks like you've mixed up 2 solutions;
Solution 1:
chromedriver = "C:\\Utils\\WebDrivers\\chromedriver.exe"
driver = webdriver.Chrome(chromedriver)
browser.get('http://www.yahoo.com')
Solution 2:
chromedriver = "C:\\Utils\\WebDrivers\\chromedriver.exe"
os.environ["webdriver.chrome.driver"] = chromedriver
browser = webdriver.Chrome()
browser.get('http://www.yahoo.com')
In your solution you've called the constructor for the driver twice.
Solution 1 will create the driver based on the executable location you explicitly provided in the constructor.
Solution 2 will create the driver based on the environment variable for the executable.
In your answer what will happen is you'll create 2 instances of Chrome, both are valid and won't throw an error, but it'll just be messy. You should really only call the driver constructor once.

OK this did the trick
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import os
chromedriver = "C:\\Utils\\WebDrivers\\chromedriver.exe"
os.environ["webdriver.chrome.driver"] = chromedriver
driver = webdriver.Chrome(chromedriver)
browser = webdriver.Chrome()
browser.get('http://www.yahoo.com')

Using Extensions with Selenium (Python)

I am currently using Selenium to run instances of Chrome to test web pages. Each time my script runs, a clean instance of Chrome starts up (clean of extensions, bookmarks, browsing history, etc). I was wondering if it's possible to run my script with Chrome extensions. I've tried searching for a Python example, but nothing came up when I googled this.

You should use Chrome WebDriver options to set a list of extensions to load. Here's an example:
import os
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
executable_path = "path_to_webdriver"
os.environ["webdriver.chrome.driver"] = executable_path
chrome_options = Options()
chrome_options.add_extension('path_to_extension')
driver = webdriver.Chrome(executable_path=executable_path, chrome_options=chrome_options)
driver.get("http://stackoverflow.com")
driver.quit()
Hope that helps.

The leading answer didn't work for me because I didn't realize you had to point the webdriver options toward a .zip file.
I.e. chrome_options.add_extension('path_to_extension_dir') doesn't work.
You need: chrome_options.add_extension('path_to_extension_dir.zip')
After figuring that out and reading a couple posts on how to create the zip file via the command line and load it into selenium, the only way it worked for me was to zip my extension files within the same python script. This actually turned out to be a nice way for automatically updating any changes you might have made to your extension:
import os, zipfile
from selenium import webdriver
# Configure filepaths
chrome_exe = "path/to/chromedriver.exe"
ext_dir = 'extension'
ext_file = 'extension.zip'
# Create zipped extension
## Read in your extension files
file_names = os.listdir(ext_dir)
file_dict = {}
for fn in file_names:
with open(os.path.join(ext_dir, fn), 'r') as infile:
file_dict[fn] = infile.read()
## Save files to zipped archive
with zipfile.ZipFile(ext_file), 'w') as zf:
for fn, content in file_dict.iteritems():
zf.writestr(fn, content)
# Add extension
chrome_options = webdriver.ChromeOptions()
chrome_options.add_extension(ext_file)
# Start driver
driver = webdriver.Chrome(executable_path=chrome_exe, chrome_options=chrome_options)
driver.get("http://stackoverflow.com")
driver.quit()

This is because the selenium expects the packed extensions as the extension argument which will be with .crx extension.
I already had the extension in my chrome. Below are the steps I followed to pack the existing extension,
Click on your extension 'details'. In my Chrome version, it was on right top click (3 dots) -> 'More tools' -> 'Extensions'.
Have the developer mode enabled in your chrome
Click 'Pack extension' (As shown above) and pack it.
This will get stored in the extensions location. For me it was on /home/user/.config/google-chrome/Default/Extensions/fdjsidgdhskifcclfjowijfwidksdj/2.3.4_22.crx
That's it, you can configure the extension in your selenium as argument.
extension='/home/user/.config/google-chrome/Default/Extensions/fdjsidgdhskifcclfjowijfwidksdj/2.3.4_22.crx'
options = webdriver.ChromeOptions()
options.add_extension(extension)
NOTE: You can also find the 'fdjsidgdhskifcclfjowijfwidksdj' id in the extensions url as query param

If you wanna import any chrome extension in your selenium python scrip
Put your extension.crx.crx file in the same folder as your code or give the path
you can copy-paste this code and just change the file crx.crx name
import os
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
executable_path = "/webdrivers"
os.environ["webdriver.chrome.driver"] = executable_path
chrome_options = Options()
chrome_options.add_extension(' YOUR - EXTIONTION - NAME ')
driver = webdriver.Chrome(chrome_options=chrome_options)
driver.get("http://stackoverflow.com")
if this code is throwing an error maybe this will solve it

An alternative way is to use the unpacked folder method. I found the crx and zip method did not work at all on Selenium Grid. In this method, you need to copy the extension from the user data in a Chrome version where you have already manually installed it, the long ID made up of loads of letters like pehaalcefcjfccdpbckoablngfkfgfgj, to the user data directory of the Selenium-controlled Chrome (which you can choose at runtime using the first line of this code, and it will get populated automatically). It should be in the same equivalent directory (which is Extensions). The path must take you all the way to the directory where there is a manifest.json, hence in this example '1.1.0'
chrome_options.add_argument("user-data-dir=C:/Users/charl/OneDrive/python/userprofile/profilename"
unpacked_extension_path = 'C:/Users/charl/OneDrive/python/userprofile/profilename/Default/Extensions/pehaalcefcjfccdpbckoablngfkfgfgj/1.1_0'
chrome_options.add_argument('--load-extension={}'.format(unpacked_extension_path))
driver = webdriver.Chrome(options=chrome_options)

I also needed to add an extention to chrome while using selenium. What I did was first open the browser using selenium then add extention to the browser in the normal way like you would do in google chrome.

I've found the simpliest way ever.
So no ways were working for me, I tryed to make a crx file and more but nothing worked.
So I simply added the unpacked extension path like that:
options.add_argument('--load-extension={}'.format("HERE YOUR EXTENSION PATH"))
Replace the HERE YOUR EXTENSION PATH, by the path of your extension which is in one of your Chrome profiles.
To find it use this path on windows:
C:\Users[login_name]\AppData\Local\Google\Chrome\User Data\Default\Extensions
Then you have to choose the folder of the extension, it's by ID which you can find in the https://chrome.google.com/webstore/category/extensions, just type there your extension name and in the URL you'll get the ID.
Then there in this folder there will be the version of your extensions like: 10.0.3 choose it and it will be your path, so the path must end with the version.
Example:
options.add_argument('--load-extension={}'.format(r'C:\Users\nevo\AppData\Local\Google\Chrome\User Data\Default\Extensions\nkbihfbeogaeaoehlefnkodbefgpgknn\10.20.0_0'))
Note the "r" before the string to make it as raw, or I had to doble the backslashes.
And it works!