We Keep Coding

Browser started under selenium doesn't use addons

I'm using python+selenium, with profile that has addons.
On startup it shows them all momentarily, but then they are concealed. They exist, they are not disabled, but invisible and don't work. I can disable and enable it, then it appears on the taskbar and is functional.
When invoking firefox manually with the profile, it works.
Here's what printed to the log.
1596129664500 mozrunner::runner INFO Running command: "/usr/bin/firefox" "-marionette" "-foreground" "-no-remote" "-profile" "/tmp/rust_mozprofileO34n0s"
JavaScript error: resource:///modules/sessionstore/SessionStore.jsm, line 1325: uncaught exception: 2147746065
JavaScript error: resource://gre/modules/ExtensionContent.jsm, line 554: TypeError: Argument 1 of PrecompiledScript.executeInGlobal is not an object.
1596129672037 Marionette INFO Listening on port 41285
1596129672136 Marionette WARN TLS certificate errors will be ignored for this session
JavaScript error: undefined, line 14: Error: An unexpected error occurred
JavaScript error: moz-extension://45aaa1ae-14fe-4a8f-841d-6a9416fd5d09/lib/picture_in_picture_overrides.js, line 15: Error: Incorrect argument types for pictureInPictureParent.setOverrides.
1596129683512 Marionette INFO Stopped listening on port 41285
Can it be because of these errors?
The code itself is of no interest at all:
#!/usr/bin/env python
from selenium.webdriver import Firefox
from selenium.webdriver.firefox.options import Options
profile_path='./profile'
opts=Options()
opts.profile=profile_path
driver=Firefox(options=opts)
breakpoint()
Versions are probably more important:
Mozilla Firefox 68.9.0esr
geckodriver 0.26.0 (e9783a644016 2019-10-10
13:38 +0000)
I'm creating an empty directory, then running firefox --new-instance --profile ./profile, then installing addon manually into the profile. But here there is that profile

In your profile there is a file: prefs.js which contains a line user_pref("extensions.lastAppBuildId", "20200707180101"); this line might be at fault for the disabled addons. You could therefore test to decrease this number by 1 or remove the whole line (untested).
profile.set_preference("extensions.lastAppBuildId", "<apppID> -1 ")
Full example code:
from selenium.webdriver import FirefoxProfile
from selenium import webdriver
path = '%APPDATA%\Mozilla\Firefox\Profiles\azk4wioue.default' #path to your profile
profile = FirefoxProfile(path)
profile.set_preference("extensions.lastAppBuildId", "<apppID> -1 ")
driver = webdriver.Firefox(profile)
Example to use an existing firefox profile:
# go to the the following folder %APPDATA%\Mozilla\Firefox\Profiles\
# there the firefox profiles should be stored, the default one ending with .default
# now provide the profile to the driver like this:
profile = FirefoxProfile('%APPDATA%\Mozilla\Firefox\Profiles\azk4wioue.default')
driver = webdriver.Firefox(firefox_profile=profile)
Alternatively install the addon clean on each run through a temporary profile.
# go to https://addons.mozilla.org and search for the plugin you want, e.g.:https://addons.mozilla.org/en-US/firefox/addon/ublock-origin/
# rightclick on the button "add to firefox"
# download the target file to a folder of your choice
# then include the addon like this:
driver.install_addon('/Users/someuser/app/extension.xpi', temporary=True)
Alternatively 2, you can try setting the extension this way:
from selenium.webdriver import FirefoxProfile
from selenium import webdriver
profile = webdriver.FirefoxProfile()
profile.add_extension(extension='/Users/someuser/app/extension.xpi')
driver = webdriver.Firefox(profile)
If the addons are there after loading the profile but disabled, you can also try this:
def newTab(fx, url="about:blank"):
wnd = fx.execute(selenium.webdriver.common.action_chains.Command.NEW_WINDOW)
handle = wnd["value"]["handle"]
fx.switch_to.window(handle)
fx.get(url) # changes handle
return fx.current_window_handle
def retoggleAllTheAddons(fx):
initialHandlesCount = len(fx.window_handles)
addonsTabHandle = newTab(fx, "about:addons")
fx.execute_script("""
let hb = document.getElementById("html-view-browser");
let al = hb.contentWindow.window.document.getElementsByTagName("addon-list")[0];
let cards = al.getElementsByTagName("addon-card");
for(let card of cards){
card.addon.disable();
card.addon.enable();
}
""")
if len(fx.window_handles) != 1:
fx.switch_to.window(addonsTabHandle)
fx.close()

Firefox v67 Private Browsing with geckodriver does not always enable addons

The newest Firefox version 67 has private browsing add-ons disabled. These can be added by following this guide. https://support.mozilla.org/en-US/kb/extensions-private-browsing
The issue is that even using that guide the add-ons are not consistently enabled under the geckodriver.
This was functioning in Firefox v66 without issue.
I have run an instance with the default firefox profile (Profile0 from profile.ini) with two add-ons (coil and LastPass). I see the addons popup for about 2 seconds and then disappear.
I have manually enabled them inside the private browser by going to about: addons. It appears that if they are already enabled in the browser for private browser use and are not visible in private browsing changing permissions to disallow still enables them in private browsing, then allowing them keeps them enabled. Very strange.
I uninstalled an addon closed all browsers, started the script again and it worked the first time. Then on the second run, it was back to the 2 seconds enabled then disappear mode as mentioned above.
I've checked the temp profile created under temp when geckodriver copies the profile for consumption. The add-ons are included.
Manually opening up a private browser shows two add-ons so it appears specific to geckodriver + firefox v67. However, the private browser doesn't have me logged in on my addons.
def get_firefox_profile_dir(self):
from pathlib import Path
self.gecko_path = os.path.dirname(__file__)
if sys.platform in ['linux', 'linux2']:
import subprocess
self.ff_gecko = Path(self.gecko_path + '/geckodriver')
bits = 'uname -m'
ver_32_64 = subprocess.getstatusoutput(bits)
cmd = "ls -d /home/$USER/.mozilla/firefox/*.default/"
fp = subprocess.Popen([cmd], shell=True, stdout=subprocess.PIPE)
FF_PRF_DIR = fp.communicate()[0][0:-2]
FF_PRF_DIR_DEFAULT = str(FF_PRF_DIR, 'utf-8')
ff_ext_path = os.path.join(FF_PRF_DIR_DEFAULT, 'extensions')
self.ff_coil_loc = os.path.join(ff_ext_path, self.ff_coil_extId)
ff_coil_enabled = os.path.exists(self.ff_coil_loc)
if ff_coil_enabled:
if 'x86_64' in ver_32_64:
if not self.ff_gecko.is_file():
import wget
self.gecko_targz = 'geckodriver-v0.24.0-linux64.tar.gz'
wget.download(self.gecko_source_linux64, self.gecko_path)
self.file_unzip_tar(self.gecko_path + '/' + self.gecko_targz)
os.remove(self.gecko_path + '/' + self.gecko_targz)
if self.ff_gecko.is_file():
self.data_path = FF_PRF_DIR_DEFAULT
self.gecko = self.ff_gecko
return
if 'i368' in ver_32_64:
if not self.ff_gecko.is_file():
import wget
self.gecko_targz = 'geckodriver-v0.24.0-linux32.tar.gz'
wget.download(self.gecko_source_linux32, self.gecko_path)
self.file_unzip_tar(self.gecko_path + '/' + self.gecko_targz)
os.remove(self.gecko_path + '/' + self.gecko_targz)
if self.ff_gecko.is_file():
self.data_path = FF_PRF_DIR_DEFAULT
self.gecko = self.ff_gecko
return
elif sys.platform == 'win32' or 'nt':
from pathlib import Path
self.gecko = self.gecko_path + "\geckodriver.exe"
mozilla_profile = os.path.join(os.getenv('APPDATA'), r'Mozilla\Firefox')
mozilla_profile_ini = os.path.join(mozilla_profile, r'profiles.ini')
profile = configparser.ConfigParser()
profile.read(mozilla_profile_ini)
FF_PRF_DIR_DEFAULT = os.path.normpath(os.path.join(mozilla_profile, profile.get('Profile0', 'Path')))
ff_ext_path = os.path.join(FF_PRF_DIR_DEFAULT, 'extensions')
self.ff_coil_loc = os.path.join(ff_ext_path, self.ff_coil_extId)
ff_coil_enabled = os.path.exists(self.ff_coil_loc)
if ff_coil_enabled:
ff_gecko = Path(self.gecko)
if ff_gecko.is_file():
self.data_path = FF_PRF_DIR_DEFAULT
return
else:
import wget
wget.download(self.gecko_source_win64)
self.file_zunip('geckodriver-v0.24.0-win64.zip')
if ff_gecko.is_file():
os.remove('geckodriver-v0.24.0-win64.zip')
self.data_path = FF_PRF_DIR_DEFAULT
return
self.get_firefox_profile_dir()
self.driver = webdriver.Firefox(options=self.options, firefox_profile=self.data_path, executable_path=self.gecko)
self.driver.get(self.url) # OPEN URL
I expect the FF addons to be enabled as they have been manually permitted from the browser and that they will be logged in as they are in normal mode.

It appears that the core issue was related to the upgrade to v67 and some unknown items in the profile became broken. A profile reset fixed the issue.
https://support.mozilla.org/en-US/kb/refresh-firefox-reset-add-ons-and-settings
UPDATE: After futher testing I see that it shows short term stability and that extensions are hit and miss (more miss) when loading dynamically. In other words after profile reset addons work the first few times and then they do not load anymore with selenium

Python Selenium Exception AttributeError: "'Service' object has no attribute 'process'" in selenium.webdriver.ie.service.Service

I have a Selenium Python test suite. It starts to run but after a few mins the following error is thrown:
Exception AttributeError: "'Service' object has no attribute 'process'" in <bound method Service.__del__ of <selenium.webdriver.ie.service.Service object at 0x0000000002610DD8>> ignored
My test suite implementation is:
import unittest
from HTMLTestRunner2 import HTMLTestRunner
import os
import Regression_TestCase.RegressionProject_TestCase2
# get the directory path to output report file
#result_dir = os.getcwd()
result_dir = r"E:\test_runners\selenium_regression_test_5_1_1\ClearCore - Regression Test\TestReport"
# get all tests from SearchProductTest and HomePageTest class
search_tests = unittest.TestLoader().loadTestsFromTestCase(Regression_TestCase.RegressionProject_TestCase2.RegressionProject_TestCase2)
# create a test suite combining search_test
re_tests = unittest.TestSuite([search_tests])
# open the report file
outfile = open(result_dir + "\TestReport.html", "w")
# configure HTMLTestRunner options
runner = HTMLTestRunner.HTMLTestRunner(stream=outfile,
title='Test Report',
description='Smoke Tests')
# run the suite using HTMLTestRunner
runner.run(re_tests)
Can anyone help why this error is stopping my test suite from running? How do I solve this?

Provided you have installed selenium, and assuming that earlier in the console's traceback log you also got something like "'chromedriver' executable needs to be in PATH" in your script, you should be able to do:
from selenium import webdriver
driver = webdriver.Chrome("/path/to/chromedriver")
This should tell your script where to find chromedriver. On a Mac you can usually use: /usr/local/bin/chromedriver

Download chromium driver from https://sites.google.com/a/chromium.org/chromedriver/downloads
Unzip the file and then from your code, write something like:
from selenium import webdriver
driver = webdriver.Chrome("/path/to/chromedriver")
where /path/to/chromedriver is the location of your chromedriver.
This is the class declaration for Chrome Webdriver: selenium.webdriver.chrome.webdriver.WebDriver(executable_path='chromedriver', ...
taken from https://seleniumhq.github.io/selenium/docs/api/py/webdriver_chrome/selenium.webdriver.chrome.webdriver.html#module-selenium.webdriver.chrome.webdriver

Given what #CubeBot88 had already written, another way to get the chromedriver executable in PATH is to do as follow:
from os
from selenium import webdriver
os.environ['PATH'] += "/path/to/chromedriver"
driver = webdriver.Chrome()
The above way puts the path to chromedriver to environment variable PATH only in this program, allowing independent PATH in different situations.

This page comes up first on Google, but only suggests that you are declaring you path incorrectly.
If you are reading this, try making sure the chromedriver is an executable:
run this command in the path of the driver:
sudo chmod a+x chromedriver

Py program gives an error if I dont open it with interactive shell

I 've a porgram called searchMachine.py that if I run it straight from IDLE (F5..)it works just fine, but If I run it by just clicking on searchMachine.py it works untill somepoint and then I get an error. See below.
Thanks for any input you could give me.
By the way, any idea how I can make this chrome work in silent? for example minimized?
from selenium import webdriver
from selenium.webdriver.support.ui import Select
print('Please type the machine name.')
machinename = input()
print('')
print('Thank you.')
print('')
print('Results will show below.')
print('')
path_to_chromedriver = 'C:\python34\chromedriver\chromedriver.exe' # change path as needed
browser = webdriver.Chrome(executable_path = path_to_chromedriver)
url = 'http://app.corp'
browser.get(url)
browser.find_element_by_xpath("/html/body/div[4]/form/table[1]/tbody/tr/td[5]/input[1]").send_keys(machinename)
browser.find_element_by_css_selector('input[type=\"submit\"]').click() # working to press submit
xpath = '/html/body/div[4]/form/center[2]/table/thead'
for i in browser.find_elements_by_xpath(xpath):
print (i.text)
xpath = '/html/body/div[4]/form/center[2]/table/tbody/tr'
for i in browser.find_elements_by_xpath(xpath):
print (i.text)
browser.close()
import os
os.system("c:/code/close_chrome_driver.bat")
error is:

The error occurs because the unicode character ▾ or U+25BE BLACK DOWN-POINTING SMALL TRIANGLE cannot be written into Windows console window, as it does not have a match in the CP850 character set as used by your Windows console.
Now, Python croaks loudly when it cannot print the correct value. You could set the PYTHONIOENCODING environment variable to say cp850:ignore which would make the Python process ignore errors; or you can print to stderr instead of stdout, which would use the backslashescape error handling.

Closing all the opened chrome windows using selenium - python - osx

I have to download some files from a website, I'm using Python - Selenium - Chrome - Osx.
I have his code so far:
lnk = "www.foobar.com"
CHROMEDRIVER=webdriver.Chrome()
options = webdriver.ChromeOptions()
profile = {"plugins.plugins_list": [{"enabled":False, "name":"Chrome PDF Viewer"}],
"download.default_directory" : TEMP_DOWNLOAD}
options.add_experimental_option("prefs",profile)
driver = webdriver.Chrome(chrome_options = options)
driver.get(lnk)
while True:
if filter(os.path.isfile, glob.glob(TEMP_DOWNLOAD+'/*.crdownload')):
pass
else:
break
driver.quit()
This code starts the download of the file, waits the end of the download and then closes the webdriver.
Everything is working properly except that it opens 2 Chrome windows, one to open the link and the other to download the file, and the quit() method closes only the latter.
Is there a way to kill all the windows opened by Selenium (I'm trying to avoid firing a terminal command to kill the processes brute force)?
EDIT:
as Mukesh Takhtani said in comment in my code the problem is a pointless webdriver instance.

Use this. This I used for Firefox. You can use this for Chrome. Call kill_waste() in your python code and it would kill idle useless Chrome. Please note that this would work for OSX or FreeBSD. For Linux distros you would have to change the way you are going to use grep and cut
import commands
def kill_waste():
(_,firefox_processes) = commands.getstatusoutput("ps -ax | grep '/usr/local/bin/firefox -foreground' | cut -c1-24")
sleep(0.5)
firefox_processes = firefox_processes.splitlines()
for pid in firefox_processes:
values = pid.split()
time_value = values[3].split(':')
if ((values[2] == 'I' or values[2] == 'I+') and time_value[0] == '1') or time_value[0] == '2':
commands.getstatusoutput("kill -9 " + values[0])