We Keep Coding

Selenium not using Default Chrome Profile

Selenium will not load my default Chrome Profile and I cannot figure out why. I have tried both Profile 1 and a Default profile with the same error (below). I have confirmed with Task Manager that all Chrome windows are shut down before running this code. Any thoughts?
from selenium import webdriver
from bs4 import BeautifulSoup
import pandas as pd
import numpy as np
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.chrome.options import Options
import os
os.system("taskkill /f /im geckodriver.exe /T")
os.system("taskkill /f /im chromedriver.exe /T")
os.system("taskkill /f /im IEDriverServer.exe /T")
os.system("taskkill /f /im chrome.exe /T")
driver2 = r"C:\Users\xxx\.wdm\drivers\chromedriver\87.0.4280.20\win32\chromedriver.exe"
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=C:\\Users\xxx\\AppData\\Local\\Google\\Chrome\\User Data\\Profile 1")
driver = webdriver.Chrome(executable_path=driver2, chrome_options=options)
driver.get("https://www.google.co.in")
Traceback (most recent call last):
File "C:\Users\xxx\OneDrive\Python\pyReportRun.py", line 16, in <module>
driver = webdriver.Chrome(ChromeDriverManager().install(), options=options)
File "C:\Python38\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 76, in __init__
RemoteWebDriver.__init__(
File "C:\Python38\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 157, in __init__
self.start_session(capabilities, browser_profile)
File "C:\Python38\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "C:\Python38\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Python38\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Could not remove old devtools port file. Perhaps the given user-data-dir at C:\Users\xxx\AppData\Local\Google\Chrome\User Data\Profile 1 is still attached to a running Chrome or Chromium process

chrom_options.add_argument("user-data-dir=C:\\Users\robert.car\\AppData\\Local\\Google\\Chrome\\User Data")
chrom_options.add_argument("profile-directory=Profile 1")
user-data-dir considers profile as default , and you don't have to specify that . If its something else specify it through profile-directory argument
Step to create a profile:
open : chrome://version in address bar
copy the user dir folder completely to eg c:\tmp\newdir
open the copied user data (newdir) and search for folder called Default . This is the profile folder.
rename the Default folder as "Profile 1"
Now to use this :
chrom_options.add_argument("user-data-dir=c:\\tmp\\newdir")
chrom_options.add_argument("profile-directory=Profile 1")

This error message...
selenium.common.exceptions.WebDriverException: Message: unknown error: Could not remove old devtools port file. Perhaps the given user-data-dir at C:\Users\xxx\AppData\Local\Google\Chrome\User Data\Profile 1 is still attached to a running Chrome or Chromium process
...implies that the ChromeDriver was unable to initiate/spawn a new Browsing Context i.e. Chrome Browser session as the user-data-dir which you have passed an an argument is still attached to a running Chrome Browsing Context.
Details
Using Default Chrome Profile for Test Automation will be against all the best practices as the Default Chrome Profile may contain either/all of the following:
browser settings
Extensions
Bookmarks
Apps
Saved Passwords
Browsing History
etc
So the Default Chrome Profile may not be in compliance with you Test Specification and may occasionally raise exceptions while trying to load. Hence you should always use a customized Chrome Profile.
You can find a detailed discussion in How to open a Chrome Profile through --user-data-dir argument of Selenium
If your usecase still warrants to use the Default Chrome Profile you need to ensure that all the google-chrome, chromium or selenium-chromedriver are stopped/killed and you can follow the below mentioned details.
Here you can find a detailed discussion on Selenium : How to stop geckodriver process impacting PC memory, without calling driver.quit()?
Location of Default Chrome Profile
As per the documentation in How to Find Your Chrome Profile Folder on Windows, Mac, and Linux the location for Chrome’s default profile folder differs depending on your platform. The locations are:
Windows 7, 8.1, and 10: C:\Users\<username>\AppData\Local\Google\Chrome\User Data\Default
Mac OS X El Capitan: Users/<username>/Library/Application Support/Google/Chrome/Default
Linux: /home/<username>/.config/google-chrome/default
You need to replace <username> with the name of your user folder. The default profile folder is simply named Default (or default in Linux). However, if you’ve created additional profiles, their folder names are not as obvious. The name you assigned to the profile when you created it displays on a name button on the right side of the title bar on the Chrome window. Unfortunately, the name Chrome uses on the associated profile folder is a generic, numbered name like Profile 3.
If you need to know any of the Chrome Profile's folder name, you simply need to access chrome://version in the address bar and press Enter.
Snapshot:
The Profile Path shows the location of the current profile. For example, the location of my Default profile in my Windows 10 system is C:\Users\Soma Bhattacharjee\AppData\Local\Google\Chrome\User Data\Default. You can select the path and copy it and paste it into File Explorer in Windows, the Finder on OS X or into a file manager like Nautilus in Linux to access that folder.
Sample Code (Windows 10)
Finally, to access the Default Chrome Profile you can use the following Python based solution:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=C:\\Users\\username\\AppData\\Local\\Google\\Chrome\\User Data\\Default")
driver = webdriver.Chrome(executable_path=r'C:\WebDrivers\chromedriver.exe', chrome_options=options)
driver.get("https://www.google.co.in")
You can find a detailed discussion in How to use Chrome Profile in Selenium Webdriver Python 3

Scheduled trigger of chrome extension app

I'm trying to find a way to implement:
- this code using python
- schedule a timed trigger
- select a tab on the google chrome
I've copied and pasted the solution into my python just to test it but seem to be getting a URL error below, can someone help me understand why a variable is facing a syntax error?
Here is my code:
from selenium import webdriver
from selenium.webdriver import ChromeOptions
from Common_Methods.GenericMethods import *
import pyautogui #<== need this to click on extension
options = ChromeOptions()
#from stack overflow(https://stackoverflow.com/questions/53172127/click-on-elements-in-chrome-extension-with-selenium?rq=1):
#options.add_argument("--load-extension=" + r"C:\Users\supputuri\AppData\Local\Google\Chrome\User Data\Default\Extensions\fdpohaocaechififmbbbbbknoalclacl\5.1_0") #<== loading unpacked extension
options.add_argument("--load-extension=" + r"/Users/erikwayne/Library/Application Support/Google/Chrome/Profile 2/Extensions/fdpohaocaechififmbbbbbknoalclacl/6.5_0") #<== loading unpacked extension
driver = webdriver.Chrome(
executable_path=os.path.join(chrome_options=options)
url = "https://www.google.com/"
driver.get(url)
# get the extension box
extn = pyautogui.locateOnScreen(os.path.join(GenericMethods.get_full_path_to_folder('autogui_ref_snaps') + "/capture_full_screenshot.png"))
# click on extension
pyautogui.click(x=extn[0],y=extn[1],clicks=1,interval=0.0,button="left")
Here is my error message:
MBP:Testing chrome extension erikwayne$ python3 chromeClick_v1.py
File "chromeClick_v1.py", line 15
url = "https://www.google.com/"
^
SyntaxError: invalid syntax
Expanded debugging:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/pdb.py", line 1703, in main
pdb._runscript(mainpyfile)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/pdb.py", line 1572, in _runscript
self.run(statement)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/bdb.py", line 587, in run
exec(cmd, globals, locals)
File "<string>", line 1, in <module>
File "/Users/erikwayne/Downloads/Testing chrome extension/chromeClick_v1.py", line 11
url = "https://www.google.com/"
^
SyntaxError: invalid syntax
*Edit: Tried the modified code below from #RafalS, but had more error.
from selenium import webdriver
from selenium.webdriver import ChromeOptions
from Common_Methods.GenericMethods import *
import pyautogui #<== need this to click on extension
import os
options = ChromeOptions()
#from stack overflow(https://stackoverflow.com/questions/53172127/click-on-elements-in-chrome-extension-with-selenium?rq=1):
#options.add_argument("--load-extension=" + r"C:\Users\supputuri\AppData\Local\Google\Chrome\User Data\Default\Extensions\fdpohaocaechififmbbbbbknoalclacl\5.1_0") #<== loading unpacked extension
options.add_argument("--load-extension=" + r"/Users/erikwayne/Library/Application Support/Google/Chrome/Profile 2/Extensions/fdpohaocaechififmbbbbbknoalclacl/6.5_0") #<== loading unpacked extension
driver = webdriver.Chrome(executable_path=os.path.join(chrome_options=options))
url = "https://www.google.com/"
driver.get(url)
# get the extension box
extn = pyautogui.locateOnScreen(os.path.join(get_full_path_to_folder('downloads') + "/capture_full_screenshot.png"))
# click on extension
pyautogui.click(x=extn[0],y=extn[1],clicks=1,interval=0.0,button="left")

driver = webdriver.Chrome(
close the parentheses
driver = webdriver.Chrome()
You'll also need to import os and replace
GenericMethods.get_full_path_to_folder
with
get_full_path_to_folder
since you did a star import:
from Common_Methods.GenericMethods import *

I did it and sorted it out by installing the chrome webdriver to a more accessible folder path, ignoring ospaths or GenericMethods altogether.
As for the driver = webdriver.Chrome issue, the original code was missing a comma between arguments, so I just filled in in.
driver = webdriver.Chrome(executable_path=os.path.join(r'/Users/USERNAME/Downloads/Testing chrome extension/chromedriver/chromedriver'), options=options)
It's hacked together but it works.

Unable to open site url in browser

I have created simple basic automation script in Python using Selenium..
Getting unwanted exception.
File:-
import pandas as pd
from pandas import ExcelWriter
from selenium import webdriver
import selenium as sel
# Data = pd.read_excel(r"C:\Users\Admin\PycharmProjects\Web_Automation_Form_Filling\challenge.xlsx",sheet_name="Sheet1")
# browser = webdriver.Chrome(executable_path=r'C:\Program Files (x86)\Google\Chrome\Application\chrome.exe')
browser = webdriver.Chrome("C:\Program Files (x86)\Google\Chrome\Application\chrome.exe");
browser.sleep(1000);
browser.get("http://www.python.org")
Error log:-
C:\Users\Admin\PycharmProjects\Web_Automation_Form_Filling\venv\Scripts\python.exe C:/Users/Admin/PycharmProjects/Web_Automation_Form_Filling/venv/Web_Auto_Filling.py
Traceback (most recent call last):
File "C:/Users/Admin/PycharmProjects/Web_Automation_Form_Filling/venv/Web_Auto_Filling.py", line 10, in <module>
browser = webdriver.Chrome("C:\Program Files (x86)\Google\Chrome\Application\chrome.exe");
File "C:\Users\Admin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 73, in __init__
self.service.start()
File "C:\Users\Admin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\common\service.py", line 98, in start
self.assert_process_still_running()
File "C:\Users\Admin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\common\service.py", line 109, in assert_process_still_running
raise WebDriverException(
selenium.common.exceptions.WebDriverException: Message: Service C:\Program Files (x86)\Google\Chrome\Application\chrome.exe unexpectedly exited. Status code was: 0
Process finished with exit code 1
Any suggestion will be appreciated..
Thanks...

instead of chrome application try providing the chrome driver instead
more information on the site : https://sites.google.com/a/chromium.org/chromedriver/getting-started
Sample code :
import time
from selenium import webdriver
driver = webdriver.Chrome('/path/to/chromedriver') # Optional argument, if not specified will search path.
driver.get('http://www.google.com/');
time.sleep(5) # Let the user actually see something!
search_box = driver.find_element_by_name('q')
search_box.send_keys('ChromeDriver')
search_box.submit()
time.sleep(5) # Let the user actually see something!
driver.quit()
Download the ChromeDriver binary for your platform under the downloads section of this site
reference link to download : chrome driver

This code should work (better to use firefox for selenium):
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.common.keys import Keys
from bs4 import BeautifulSoup
# noinspection PyUnresolvedReferences
import wget
DesiredCapabilities.PHANTOMJS[
"phantomjs.page.settings.userAgent"
] = "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:16.0) Gecko/20121026 Firefox/16.0"
if browser == "firefox":
driver = webdriver.Firefox()
else:
driver = webdriver.PhantomJS(
CFG_phantomjs
) # r"D:/_devs/webserver/phantomjs-1.9.8/phantomjs.exe"
driver.get("https://tourwebsite")
username = driver.find_element_by_id("login_field")
password = driver.find_element_by_id("password")
username.clear()

The problem here in your codes is that you are passing chrome executable path rather than passing the path to chromedriver which is a different executable.
An appropriate version of chromedriver can be downloaded from here according to your Chrome version.
For more info, you can refer to the chromedriver documentation here.
And your final code should be something like:
from selenium import webdriver
path = 'C:/Users/Avinash/Downloads/chromedriver.exe'
driver = webdriver.Chrome(path)
driver.get('http://www.google.com/');
#..here what ever you want to do with page here
driver.quit()

How to use Brave web browser with python, selenium and chromedriver?

I switched from Google's Chrome to Brave web browser and am having a hard time getting it to work with Brave like it did with Chrome. Brave is based on chromium so I guessed it should not be that hard. I made sure that my Brave and Chromedriver are on the same version like this,
~/some/path $ chromedriver --version
ChromeDriver 76.0.3809.126 (d80a294506b4c9d18015e755cee48f953ddc3f2f-refs/branch-heads/3809#{#1024})
My chromedriver is also in /user/bin,
~/path $ cd /usr/bin/
/usr/bin $ ls | grep chromedriver
chromedriver
And to check the Brave version, I get: Version 0.68.132 Chromium: 76.0.3809.132 (Official Build) (64-bit)
Then I run this code,
from selenium import webdriver
driver = webdriver.Chrome(executable_path='/usr/bin/brave-browser')
driver.get("http://www.python.org")
driver.close()
This opens a Brave window but then instead of getting the page the driver is pointed to, an exception is thrown,
Traceback (most recent call last):
File "webscrap.py", line 3, in <module>
driver = webdriver.Chrome(executable_path='/usr/bin/brave-browser')
File "/home/username/.local/lib/python3.6/site-packages/selenium/webdriver/chrome/webdriver.py", line 73, in __init__
self.service.start()
File "/home/username/.local/lib/python3.6/site-packages/selenium/webdriver/common/service.py", line 98, in start
self.assert_process_still_running()
File "/home/username/.local/lib/python3.6/site-packages/selenium/webdriver/common/service.py", line 111, in assert_process_still_running
% (self.path, return_code)
selenium.common.exceptions.WebDriverException: Message: Service /usr/bin/brave-browser unexpectedly exited. Status code was: -11

I finally managed to make it work:
Try this python script (python3.7)
from selenium import webdriver
driver_path = "C:/Users/username/PycharmProjects/chromedriver.exe"
brave_path = "C:/Program Files (x86)/BraveSoftware/Brave-Browser/Application/brave.exe"
option = webdriver.ChromeOptions()
option.binary_location = brave_path
# option.add_argument("--incognito") OPTIONAL
# option.add_argument("--headless") OPTIONAL
# Create new Instance of Chrome
browser = webdriver.Chrome(executable_path=driver_path, chrome_options=option)
browser.get("https://www.google.es")
cheers.

The executable_path key is used to pass the absolute path of the WebDriver binary i.e. the chromedriver executable.
To initiate a Brave browser session additionally you have to pass the absolute location of the brave-browser binary through the binary_location argument of an instance of ChromeOptions.
So the effective code block will be:
from selenium import webdriver
chromedriver_path = '/usr/bin/chromedriver'
brave_path = '/usr/bin/brave-browser'
option = webdriver.ChromeOptions()
option.binary_location = brave_path
browser = webdriver.Chrome(executable_path=driver_path, options=option)
browser.get("https://www.google.es")
References
You can find a couple of relevant detailed discussions in:
DeprecationWarning: use options instead of chrome_options error using ChromeDriver and Chrome through Selenium on Windows 10 system
How to initiate Brave browser using Selenium and Python on Windows
DeprecationWarning: use options instead of chrome_options error using Brave Browser With Python Selenium and Chromedriver on Windows

This also works in windows 10 with Brave browser. I downloaded Chromedriver and put it in the folder with Brave.exe.
from selenium import webdriver
driver_path = "C:\\Users\\5150s\\AppData\\Local\\Programs\\Python\\Python38\\chromedriver.exe"
brave_path = "C:\\Program Files (x86)\\BraveSoftware\\Brave-Browser\\Application\\brave.exe"
option = webdriver.ChromeOptions()
option.binary_location = brave_path
browser = webdriver.Chrome(executable_path=driver_path, options=option)
browser.get("https://www.google.es")

The solutions above gave me some errors. This code removes the executable path and options errors.
Chromedriver is in the pycharm folder.
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
driver_path = "C:/Users/johnm/PycharmProjects/chromedriver.exe"
brave_path = "C:/Program Files/BraveSoftware/Brave-Browser/Application/brave.exe"
s=Service(driver_path)
option = webdriver.ChromeOptions()
option.binary_location = brave_path
browser = webdriver.Chrome(service=s, options=option)
browser.get("https://www.google.es")

How to start Firefox with with specific profile Selenium Python geckodriver

Here is my code:
profile = webdriver.FirefoxProfile('C:\\Users\\Administrator\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\kvycjolb.Prdel')
driver = webdriver.Firefox(profile)
Im not getting any error and firefox starts, but it just does not load with this profile: I have tried changing / to // etc.. but no luck.
This also does not work:
from selenium import webdriver
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
binary = FirefoxBinary("C:\\Program Files\\Mozilla Firefox\\firefox.exe")
profile = FirefoxProfile("C:\\Users\\Administrator\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\kvycjolb.Prdel")
driver = webdriver.Firefox(firefox_profile=profile, firefox_binary=binary, executable_path="C:\\aprog\\geckodriver.exe")
driver.get('https://google.com')
Im getting error:
C:\aprog>testff
Traceback (most recent call last):
File "C:\aprog\testff.py", line 7, in <module>
driver = webdriver.Firefox(firefox_profile=profile, firefox_binary=binary, e
xecutable_path="C:\\aprog\\geckodriver.exe")
File "C:\Python27\lib\site-packages\selenium\webdriver\firefox\webdriver.py",
line 152, in __init__
keep_alive=True)
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", l
ine 98, in __init__
self.start_session(desired_capabilities, browser_profile)
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", l
ine 188, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", l
ine 256, in execute
self.error_handler.check_response(response)
File "C:\Python27\lib\site-packages\selenium\webdriver\remote\errorhandler.py"
, line 194, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Unable to find a matchin
g set of capabilities

I think the official answer is found in documentation.
Presently that is:
# Custom profile folder to keep the minidump files
profile = tempfile.mkdtemp(".selenium")
print("*** Using profile: {}".format(profile))
# Use the above folder as custom profile
opts = Options()
opts.add_argument("-profile")
opts.add_argument(profile)
opts.binary = "/Applications/Firefox.app/Contents/MacOS/firefox"
driver = webdriver.Firefox(options=opts,
# hard-code the Marionette port so geckodriver can connect
service_args=["--marionette-port", "2828"])

To start Mozilla Firefox with a specific Firefox Profile through Selenium 3.4.3, geckodriver v0.18.0, Mozila Firefox 53.0 and Python 3.6, you need to create a separate Firefox Profile with the Firefox Profile Manager as per the documentation here.
I have created a Firefox Profile by the name debanjan. This profile got stored in this subdirectory:
"C:\Users\AtechM_03\AppData\Roaming\Mozilla\Firefox\Profiles"
The name of the profile (folder) is w8iy627a.debanjan. So while initiating the WebDriver instance we have to pass the absolute path of the Firefox Profile named as w8iy627a.debanjan as follows:
from selenium import webdriver
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
binary = FirefoxBinary("C:\\Program Files\\Mozilla Firefox\\firefox.exe")
profile = FirefoxProfile("C:\\Users\\AtechM_03\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\w8iy627a.debanjan")
driver = webdriver.Firefox(firefox_profile=profile, firefox_binary=binary, executable_path="C:\\Utility\\BrowserDrivers\\geckodriver.exe")
driver.get('https://google.com')
Let me know if this answers your question.

def setFirefoxDriver():
profilePath = r"PathHere"
driverPath = r"pathHere\driver.exe"
options = Options()
options.add_argument("-profile")
options.add_argument(profilePath)
dService = Service(driverPath)
d = webdriver.Firefox(service=dService, options=options)
return d
d = setFirefoxProfile()
d.get('https://www.amazon.com/)
to know profile paths search in your firefox about:support or about:profiles
You can test it loading your own profile and see if cookies are loading, ie: when I go to amazon.com amazon recognizes me.
Notice that you can't be using the same profile in 2 different instances, so if you wanna load your profile to test in selenium you shouldn't be using that firefox profile, but another one.

Always use double backslashes in the path (for Windows paths at least):
profile = webdriver.FirefoxProfile('C:\\Users\\Administrator\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\kvycjolb.Prree')
In your code, you use both backslashes and forward slashes.