I'm trying to open a Firefox browser with undetected_chromedriver.
But only getting a default Firefox browser instead of getting the url I provided.
What did I miss or do wrong?
Here is the code I made so far.
import undetected_chromedriver as uc
import time
if __name__ == '__main__':
driver_path = "C:/Users/jay/Desktop/py/geckodriver.exe"
Firefox_path = "C:/Program Files/Mozilla Firefox/firefox.exe"
option = uc.ChromeOptions()
option.binary_location = Firefox_path
driver = uc.Chrome(executable_path=driver_path, options=option)
driver.get('https://google.com')
time.sleep(10)
I'd appreciate it if you could help me with this.
undetected_chromedriver is ONLY for chromedriver.
It modifies values directly inside binary file chromedrive.exe and it doesn't know how to modify values inside file geckodriver.exe.
See also repo GitHub - undetected-chromedriver.
There is:
"Works ... on .... Chromium based browsers".
"Automatically downloads the driver binary and patches it."
It means it automatically uses chromedrive.exe and it can't even use geckodriver.exe. And chromedrive.exe doesn't know how to communicate with Firefox - so it can't open page.
You can use it only with browsers which use engine Chromium - like Brave and maybe Opera, Microsoft Edge (but I didn't test it).
So, this is the code I'm having troubles with:
def scrap():
options = webdriver.ChromeOptions();
options.add_argument('headless');
options.add_argument('--profile-directory=Profile 1')
options.add_argument("--user-data-dir=C:/Users/omarl/AppData/Local/Google/Chrome/User Data/")
options.add_argument("--remote-debugging-port=45447")
options.add_argument("--disable-gpu")
browser = webdriver.Chrome(executable_path=r"C:\Users\omarl\OneDrive\Escritorio\chromedriver.exe", options=options)
scrapURL = "https://es.wallapop.com/search?distance=30000&keywords=leggins&latitude=41.38804&longitude=2.17001&filters_source=quick_filters"
browser.get(scrapURL)
#...
And the error:
WebDriverException: unknown error: unable to discover open pages
I don't have any instances of chrome when I execute the script, and when I'm using it without the headless option it works fine. Any idea why this is happening? Please, note that I'm using the --remote-debuggin-port provided in similar questions.
I'm using ChromeDriver 86.0.4240.22
To invoke a Chrome Profile in Headless mode you can use the --user-data-dir argument only and you can safely remove --profile-directory argument as follows:
Code Block:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--window-size=1920,1080')
# options.add_argument('--profile-directory=Profile 1')
options.add_argument(r"--user-data-dir=C:\Users\Soma Bhattacharjee\AppData\Local\Google\Chrome\User Data\Default")
options.add_argument("--remote-debugging-port=9222")
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get('https://www.google.com/')
print("Chrome Headless launched")
Console Output:
DevTools listening on ws://127.0.0.1:9222/devtools/browser/93c67c41-e125-4d12-abc0-fcf0f07a62f4
Chrome Headless launched
References
You can find a couple of relevant detailed discussions in:
How to open a Chrome Profile through --user-data-dir argument of Selenium
Selenium: Point towards default Chrome session
Additional Considerations
Ensure that:
Selenium is upgraded to current released Version 3.141.0.
ChromeDriver is updated to current ChromeDriver v86.0 level.
Chrome is updated to current Chrome Version 86.0 level. (as per ChromeDriver v86.0 release notes).
Execute your #Test as non-root user.
Always invoke driver.quit() within tearDown(){} method to close & destroy the WebDriver and Web Client instances gracefully.
tl; dr
ChromeDriver remote debug port reservation race conditions
Have you tried using arg --no-sandbox?
A lot of people on Chrome Driver Error using Selenium: Unable to Discover Open Pages have had success with that argument.
I've successfully managed to load a Chrome Profile on MAC and I was trying to replicate the same on Linux but without success (Debian). I'm using Python, and the following works just fine on a MAC
from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("user-data-dir=/Users/username/Library/Application Support/Google/Chrome")
driver = webdriver.Chrome('./chromedriver', options=chrome_options)
The same code on Debian, just doesn't work...
from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--user-data-dir=/home/username/.config/google-chrome")
# I've tried also without the `--` but same outcome
# chrome_options.add_argument("user-data-dir=/home/username/.config/google-chrome")
driver = webdriver.Chrome('./chromedriver_linux', options=chrome_options)
I honestly now idea what's wrong. I'm using chromedriver 2.45 https://chromedriver.storage.googleapis.com/index.html?path=2.45/ and the issue is related to "Debian GNU/Linux 9 (stretch)" ...
In terms of launching Chrome, they both works. The difference is that on MAC it loads the profile, on Debian it doesn't.
Anyone has an idea why this is happening?
Right, so after many headaches, apparently this is something to do with the fact than I'm using CRD (Chrome Remote Desktop) to connect to the Linux instances!
In fact, you can check the profile location loading chrome://version. When connecting with CRD, this changes from the usual /home/user/.config/google-chrome to /home/user/.config/chrome-remote-desktop/chrome-profile/
All I needed to do is basically replace with the CRD directory to get all the profile information I wanted!
from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
#chrome_options.add_argument("--user-data-dir=/home/user/.config/google-chrome")
chrome_options.add_argument("--user-data-dir=/home/user/.config/chrome-remote-desktop/chrome-profile/")
driver = webdriver.Chrome('./chromedriver_linux', options=chrome_options)
Hopefully this will be helpful for others! :)
So whenever I try to use my Chrome settings (the settings I use in the default browser) by adding
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=C:\Users\... (my webdriver path)")
driver = webdriver.Chrome(executable_path="myPath", options=options)
it shows me the error code
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes n 16-17: truncated \UXXXXXXXX escape
in my bash. I don't know what that means and I'd be happy for any kind of help I can get. Thanks in advance!
The accepted answer is wrong. This is the official and correct way to do it:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = webdriver.ChromeOptions()
options.add_argument(r"--user-data-dir=C:\path\to\chrome\user\data") #e.g. C:\Users\You\AppData\Local\Google\Chrome\User Data
options.add_argument(r'--profile-directory=YourProfileDir') #e.g. Profile 3
driver = webdriver.Chrome(executable_path=r'C:\path\to\chromedriver.exe', chrome_options=options)
driver.get("https://www.google.co.in")
To find the profile folder on Windows right-click the desktop shortcut of the Chrome profile you want to use and go to properties -> shortcut and you will find it in the "target" text box.
To get the path, follow the steps below.
In the search bar type the following and press enter
This will then show all the metadata. There find the path to the profile
As per your question and your code trials if you want to open a Chrome Browsing Session here are the following options:
To use the default Chrome Profile:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=C:\\Users\\AtechM_03\\AppData\\Local\\Google\\Chrome\\User Data\\Default")
driver = webdriver.Chrome(executable_path=r'C:\path\to\chromedriver.exe', chrome_options=options)
driver.get("https://www.google.co.in")
Note: Your default chrome profile would contain a lot of bookmarks, extensions, theme, cookies etc. Selenium may fail to load it. So as per the best practices create a new chrome profile for your #Test and store/save/configure within the profile the required data.
To use the customized Chrome Profile:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("user-data-dir=C:\\Users\\AtechM_03\\AppData\\Local\\Google\\Chrome\\User Data\\Profile 2")
driver = webdriver.Chrome(executable_path=r'C:\path\to\chromedriver.exe', chrome_options=options)
driver.get("https://www.google.co.in")
Here you will find a detailed discussion on How to open a Chrome Profile through Python
This is how I managed to use EXISTING CHROME PROFILE in php selenium webdriver.
Profile 6 is NOT my default profile. I dont know how to run default profile. It is IMPORTANT not to add -- before chrome option arguments! All other variants of options didnt work!
<?php
//...
$chromeOptions = new ChromeOptions();
$chromeOptions->addArguments([
'user-data-dir=C:/Users/MyUser/AppData/Local/Google/Chrome/User Data',
'profile-directory=Profile 6'
]);
$host = 'http://localhost:4444/wd/hub'; // this is the default
$capabilities = DesiredCapabilities::chrome();
$capabilities->setCapability(ChromeOptions::CAPABILITY, $chromeOptions);
$driver = RemoteWebDriver::create($host, $capabilities, 100000, 100000);
To get name of your chrome profile, go to chrome://settings/manageProfile, click on profile icon, click "Show profile shortcut on my desktop". After that right click on desktop profile icon and go to properties, here you will see something like "C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" --profile-directory="Profile 6".
Also I recommend you to close all chrome instances before running this code. Also maybe you need to TURN OFF chrome settings > advanced > system > "Continue running background apps when Google Chrome is closed".
None of the given answers were working for me so I researched a bit and now the working code is for is this one. I copied the user dir folder from Profile Path from chrome://version/ and made another argument for the profile as shown below:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = webdriver.ChromeOptions()
options.add_argument('user-data-dir=C:\\Users\\gupta\\AppData\\Local\\Google\\Chrome\\User Data')
options.add_argument('profile-directory=Profile 1')
driver = webdriver.Chrome(executable_path=r'C:\Program Files (x86)\chromedriver.exe', options=options)
driver.get('https://google.com')
Are you sure you are meant to be putting in the webdriver path in the user-data-dir argument? That's usually where you put your chrome profile e.g. "C:\Users\yourusername\AppData\Local\Google\Chrome\User Data\Profile 1\". Also you will need to use either double backslashes or forward slashes in your directory path (both work). You can test if your path works by using the os library
e.g.
import os
os.list("C:\\Users\\yourusername\\AppData\\Local\\Google\\Chrome\\User Data\\Profile 1")
will give you the directory listing.
I might also add that occasionally if you manage to crash chrome while running webdriver with a nominated user profile, that it seems to record the crash in the profile and the next time you open chrome, you get the Chrome prompt to restore pages after it exited abnormally. For me personally this had been a bit of headache to deal with and I no longer use a user profile with chromedriver because of it. I could not find a way around it. Other people have reported it here, but none of their solutions seemed to work for me, or were not suitable for my test cases. https://superuser.com/questions/237608/how-to-hide-chrome-warning-after-crash
If you don't nominate a user profile it seems to create a new (blank) temporary one each time it runs
Make sure you've got the path to the profile right, and that you double escape backslashes in said path.
For example, typically the default profile on windows is located at:
"C:\\Users\\user\\AppData\\Local\\Google\\Chrome\\User Data\\Default"
I managed to launch my chrome profile using these arguments:
ChromeOptions options = new ChromeOptions();
options.addArguments("--user-data-dir=C:\\Users\\user\\AppData\\Local\\Google\\Chrome\\User Data");
options.addArguments("--profile-directory=Profile 2");
WebDriver driver = new ChromeDriver(options);
You can find out more about the web driver here
You simply have to replace the '\' to '/' in your paths and that'll resolve it.
Get profile name by navigating to chrome://version from your chrome browser (You'll see Profile Path, but you only want the profile name from it (e.g. Profile 1)
Close out all Chrome sessions using the profile you want to use. (or else you will get the following error: InvalidArgumentException)
Now make sure you have the code below (Make sure you replace UserFolder with the name of your userfolder.
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=C:\\Users\\EnterYourUserFolder\\AppData\\Local\\Google\\Chrome\\User Data") #leave out the profile
options.add_argument("profile-directory=Profile 1") #enter profile here
driver = webdriver.Chrome(executable_path="C:\\chromedriver.exe", chrome_options=options)
this worked for me 100% and it showed up my selected profile.
import os
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
# path of your chrome webdriver
dir_path = os.getcwd()
user_profile_path = os.environ[ 'USERPROFILE' ]
#if "frtkpr" which is ll be your custom profile does not exist it will be created.
option.add_argument( "user-data-dir=" + user_profile_path + "/AppData/Local/Google/Chrome/User Data/frtkpr" )
driver = webdriver.Chrome( dir_path + "/chromedriver.exe",chrome_options=option )
baseUrl = "https://www.facebook.com/"
driver.maximize_window()
driver.get( baseUrl )
I have following code:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument('--headless')
options.add_argument('--disable-gpu')
driver = webdriver.Chrome("chromedriver.exe", chrome_options=options)
driver.get("*some site*")
When i start debugging, it stucks at the driver.get("*some site*") line. It simply does not start anything.
Any suggestions please?
Everything is up to date. Using windows 7.
EDIT: using Python 3.6
EDIT 2: hope its helpful
try running with this options, for me (on my win machine) --no-sandbox helped.
--log-path and verbose obviously helps with debugging.
ch_options.add_argument('--headless')
ch_options.add_argument('--disable-gpu')
ch_options.add_argument('--no-sandbox')
ch_options.add_argument('--log-path=chromedriver.log')
ch_options.add_argument('--verbose')
and for user agent (in case some websites protest):
ch_options.add_argument(
'--user-agent="valid user agent :)"')
and starting it:
driver = webdriver.Chrome(chrome_options=ch_options)
my chromedriver is in PATH environmental variable so no need to pass executable in here.
btw. phantomjs driver is not updated any more so try to resolve this and switch to chromedriver. I.e. for me the phantomjs driver wont even let me change user-agent, just on python binding though.