I am trying to save a screenshot of a webpage, to do so I am trying to use Selenium. The problem is that once the webpage is opened, it stays blank with "data:" in the URL.
Here is my code:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(options=options,executable_path='aPath/chromedriver.exe',service_log_path='aPath/mylog.txt')
driver.get('http://myURL.html')
screenshot=driver.save_screenshot('aPath/my_screenshot.png')
driver.quit()
NB: I have checked that my chromedriver version is compatible with my chrome browser version.
You need to update the value of the Key executable_path with the absolute path of the chromedriver binary and service_args as follows:
driver = webdriver.Chrome(options=options,executable_path=r'C:\path\to\chromedriver.exe', service_args=["--log-path=C:\\path\\to\\mylog.log"])
You can find a couple of relevant discussions in:
Selenium doesn't open the specified URL and shows data:,
What are 'service_args' for the selenium-python chrome webdriver?
Thanks for your help guys, actually Guy was right, I had to specify the port:
options.add_argument('--remote-debugging-port=9222')
Now it works!
I found that adding the following option worked for me:
options.add_argument('--no-sandbox')
Details found in this answer:
selenium.common.exceptions.WebDriverException: Message: unknown error: unable to discover open pages using ChromeDriver through Selenium
Related
So, this is the code I'm having troubles with:
def scrap():
options = webdriver.ChromeOptions();
options.add_argument('headless');
options.add_argument('--profile-directory=Profile 1')
options.add_argument("--user-data-dir=C:/Users/omarl/AppData/Local/Google/Chrome/User Data/")
options.add_argument("--remote-debugging-port=45447")
options.add_argument("--disable-gpu")
browser = webdriver.Chrome(executable_path=r"C:\Users\omarl\OneDrive\Escritorio\chromedriver.exe", options=options)
scrapURL = "https://es.wallapop.com/search?distance=30000&keywords=leggins&latitude=41.38804&longitude=2.17001&filters_source=quick_filters"
browser.get(scrapURL)
#...
And the error:
WebDriverException: unknown error: unable to discover open pages
I don't have any instances of chrome when I execute the script, and when I'm using it without the headless option it works fine. Any idea why this is happening? Please, note that I'm using the --remote-debuggin-port provided in similar questions.
I'm using ChromeDriver 86.0.4240.22
To invoke a Chrome Profile in Headless mode you can use the --user-data-dir argument only and you can safely remove --profile-directory argument as follows:
Code Block:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--window-size=1920,1080')
# options.add_argument('--profile-directory=Profile 1')
options.add_argument(r"--user-data-dir=C:\Users\Soma Bhattacharjee\AppData\Local\Google\Chrome\User Data\Default")
options.add_argument("--remote-debugging-port=9222")
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get('https://www.google.com/')
print("Chrome Headless launched")
Console Output:
DevTools listening on ws://127.0.0.1:9222/devtools/browser/93c67c41-e125-4d12-abc0-fcf0f07a62f4
Chrome Headless launched
References
You can find a couple of relevant detailed discussions in:
How to open a Chrome Profile through --user-data-dir argument of Selenium
Selenium: Point towards default Chrome session
Additional Considerations
Ensure that:
Selenium is upgraded to current released Version 3.141.0.
ChromeDriver is updated to current ChromeDriver v86.0 level.
Chrome is updated to current Chrome Version 86.0 level. (as per ChromeDriver v86.0 release notes).
Execute your #Test as non-root user.
Always invoke driver.quit() within tearDown(){} method to close & destroy the WebDriver and Web Client instances gracefully.
tl; dr
ChromeDriver remote debug port reservation race conditions
Have you tried using arg --no-sandbox?
A lot of people on Chrome Driver Error using Selenium: Unable to Discover Open Pages have had success with that argument.
I'm trying to make a script in python with selenium that will open a website and click a button.
Problem is that I have to close chrome before I can run the script otherwise I get this error:
Message: invalid argument: user data directory is already in use, please specify a unique value for --user-data-dir argument, or don't use --user-data-dir
Is there a way around this so I don't have to close chrome before running the script everytime?
My code:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import time
PATH = "C:\Program Files (x86)\chromedriver.exe"
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=C:/Users/username/AppData/Local/Google/Chrome/User Data")
driver = webdriver.Chrome(executable_path=r'C:\Program Files (x86)\chromedriver.exe', chrome_options=options)
driver.get("https://disboard.org/sv/dashboard/servers")
link = driver.find_element_by_link_text("bump")
link.click()
This error message...
invalid argument: user data directory is already in use, please specify a unique value for --user-data-dir argument, or don't use --user-data-dir
...implies that the user data directory Default is already in use so ChromeDriver was unable to access the directory and initiate/spawn a new Browsing Context i.e. Chrome Browser session.
Solution
In such cases, you can add/create a new Chrome Profile following the steps below and use it for the AUT (Application Under Test):
Open Google Chrome, and select the user icon in the top right and click on Add.
Enter a name for the new person, select an icon to help represent this new account and make it easier to find later. Then, select Add.
You will find the Chrome Profile shortcut created on the Desktop.
Additionally, you will find a new sub-directory Profile 1 being created beside Default
Now, you can use the Profile 1 sub-directory as follows:
options = Options()
options.add_argument("start-maximized")
options.add_argument("--profile-directory=Profile 1")
options.add_argument("--user-data-dir=C:/Users/user/AppData/Local/Google/Chrome/User Data")
driver = webdriver.Chrome(executable_path=r'C:\BrowserDrivers\chromedriver.exe', options=options)
driver.get("https://www.google.com/")
References
You can find a couple of relevant detailed discussions in:
How to use Chrome Profile in Selenium Webdriver Python 3
selenium.common.exceptions.InvalidArgumentException: Message: invalid argument: user data directory is already in use error with real Chrome Browser
Selenium: Point towards default Chrome session
If it is crucial for your use case that you can use both your Chrome profile and your automation with your profile at the same time, here's a possible solution:
Install a second (older or newer) version of Chrome as described in this thread
Enable synchronization on the desired profile in your current Chrome.
Log in with the desired profile from the second version of Chrome and enable synchronization there as well.
Use the chrome driver compatible with the second version of Chrome in your code.
There may be some problems with synchronization not loading everything that you need, but if it's something simple like saved passwords then this should work.
The reason for your issue is because you have added the following argument:
options.add_argument("user-data-dir=C:/Users/username/AppData/Local/Google/Chrome/User Data")
The argument is instructing selenium to use your local chrome profile. Hence, as you have a chrome session open, the automation cannot execute until you close down the browser session.
Remove the argument to execute an independent automation test profile.
See below for the remediated code provided from your question
from selenium import webdriver
PATH = "C:\Program Files (x86)\chromedriver.exe"
options = webdriver.ChromeOptions()
driver = webdriver.Chrome(executable_path=PATH, options=options)
driver.get("https://disboard.org/sv/dashboard/servers")
link = driver.find_element_by_link_text("bump")
link.click()
While working with selenium webdriver, I want to set download location to a particular location and work with the headless browser. But I am unable to do both at once. Upon going headless, download location changes back.
Here is the piece of my code:
options = webdriver.ChromeOptions()
options.add_experimental_option("prefs",{
"download.default_directory":os.getcwd()+"\mydir",
"download.prompt_for_download":False,
"download.directory_upgrade": True
})
options.add_argument('--headless')
driver = webdriver.Chrome(chrome_options=options)
Unfortunately, chromedriver does not currently support headless downloads.
I am working on downloading HAR from Chrome for YouTube through Selenium Python Script.
Code Snippet:
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--proxy-server={0}".format(url))
chrome_options.add_argument("--enable-quic")
self.driver = webdriver.Chrome(chromedriver,chrome_options = chrome_options)
self.proxy.new_har(args['url'], options={'captureHeaders': True})
self.driver.get(args['url'])
result = json.dumps(self.proxy.har, ensure_ascii=False)
I want QUIC to be used whenever I download HAR but when I look at the packets through Wireshark Selenium driver is using TCP only. Is there a way to force Chrome Driver to use QUIC? Or Is there an alternate to BMP?
A similar thing has been asked for Firefox in this question How to capture all requests made by page in webdriver? Is there any alternative to Browsermob? and there was a solution with Selenium alone without need of any BMP. So is it possible for Chrome?
Workaround for this problem could be: start Chrome normally (with your default profile or create another profile) and enable quic manually. Then start chromedriver with your profile loaded.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=/home/user/.config/google-chrome")
driver = webdriver.Chrome(executable_path="/home/user/Downloads/chromedriver", chrome_options=options)
I am trying to submit information in a webpage, but selenium throws this error:
UnexpectedAlertPresentException: Alert Text: This page is asking you
to confirm that you want to leave - data you have entered may not be
saved. ,
>
It's not a leave notification; here is a pic of the notification -
.
If I click in never show this notification again, my action doesn't get saved; is there a way to save it or disable all notifications?
edit: I'm using firefox.
You can disable the browser notifications, using chrome options. Sample code below:
chrome_options = webdriver.ChromeOptions()
prefs = {"profile.default_content_setting_values.notifications" : 2}
chrome_options.add_experimental_option("prefs",prefs)
driver = webdriver.Chrome(chrome_options=chrome_options)
With the latest version of Firefox the above preferences didn't work.
Below is the solution which disable notifications using Firefox object
_browser_profile = webdriver.FirefoxProfile()
_browser_profile.set_preference("dom.webnotifications.enabled", False)
webdriver.Firefox(firefox_profile=_browser_profile)
Disable notifications when using Remote Object:
webdriver.Remote(desired_capabilities=_desired_caps, command_executor=_url, options=_custom_options, browser_profile=_browser_profile)
selenium==3.11.0
Usually with browser settings like this, any changes you make are going to get throws away the next time Selenium starts up a new browser instance.
Are you using a dedicated Firefox profile to run your selenium tests? If so, in that Firefox profile, set this setting to what you want and then close the browser. That should properly save it for its next use. You will need to tell Selenium to use this profile though, thats done by SetCapabilities when you start the driver session.
This will do it:
from selenium.webdriver.firefox.options import Options
options = Options()
options.set_preference("dom.webnotifications.enabled", False)
browser = webdriver.Firefox(firefox_options=options)
For Google Chrome and v3 of Selenium you may receive "DeprecationWarning: use options instead of chrome_options", so you will want to do the following:
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
options = webdriver.ChromeOptions()
options.add_argument('--disable-notifications')
driver = webdriver.Chrome(ChromeDriverManager().install(), options=options)
Note: I am using webdriver-manager, but this also works with specifying the executable_path.
This answer is an improvement on TH Todorov code snippet, based on what is working as of Chrome (Version 80.0.3987.163).
lk = os.path.join(os.getcwd(), "chromedriver",) --> in this line you provide the link to the chromedriver, which you can download from chromedrive link
import os
from selenium import webdriver
lk = os.path.join(os.getcwd(), "chromedriver",)
chrome_options = webdriver.ChromeOptions()
prefs = {"profile.default_content_setting_values.notifications" : 2}
chrome_options.add_experimental_option("prefs",prefs)
driver = webdriver.Chrome(lk, options=chrome_options)