I am trying to use selenium for scraping and when I try to start the WebDriver it automatically closes. I've tried everything and it still doesn't work.
def launchBrowser():
ch_options = webdriver.ChromeOptions()
ch_options.add_experimental_option("detach",True)
ch_options.add_experimental_option('excludeSwitches', ['enable-logging'])
ch_driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()),options=ch_options)
ch_driver.get(url)
launchBrowser()
I tried to implement some solutions I have seen but it doesn't work.
On my Mac, I use the below selenium code to search the Python keyword in headless Chrome.
from selenium import webdriver
wd = webdriver.Chrome(r"/opt/webdrivers/chromedriver")
wd.implicitly_wait(5)
wd.get("https://www.google.com")
element = wd.find_element_by_id('kw')
element.send_keys('Python\n')
element = wd.find_element_by_class_name('c-abstract')
print(element.text)
but, however, the Chrome will open up a window automatically.
my understanding of headless browser will now open a window. In spite of this, is it possible to restrain the GUI get up, let it run in silence?
I don't know about headless chrome but you can run chrome with headless mode by following options
options = webdriver.ChromeOptions()
options.add_argument('headless')
options.add_argument('window-size=1920x1080')
options.add_argument("disable-gpu")
wd = webdriver.Chrome(r"/opt/webdrivers/chromedriver")
I'm trying to use selenium on OSX to open a chrome window with existing cookies so that I can bypass login. When I don't add the argument 'chrome_options=option' to open chrome with my user settings, the driver.get function works fine and it opens a chrome window with no extensions loaded and browses to the url. But when I use the code as shown below, it simply opens chrome (with all extensions loaded) but does not browse to the URL. Am I missing something simple?
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.keys import Keys
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=/Users/me/Library/Application Support/Google/Chrome/")
driver = webdriver.Chrome("./assets/chromedriver",chrome_options=options)
driver.implicitly_wait(30)
driver.maximize_window()
driver.get("https://www.gmail.com/")
I am working on downloading HAR from Chrome for YouTube through Selenium Python Script.
Code Snippet:
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--proxy-server={0}".format(url))
chrome_options.add_argument("--enable-quic")
self.driver = webdriver.Chrome(chromedriver,chrome_options = chrome_options)
self.proxy.new_har(args['url'], options={'captureHeaders': True})
self.driver.get(args['url'])
result = json.dumps(self.proxy.har, ensure_ascii=False)
I want QUIC to be used whenever I download HAR but when I look at the packets through Wireshark Selenium driver is using TCP only. Is there a way to force Chrome Driver to use QUIC? Or Is there an alternate to BMP?
A similar thing has been asked for Firefox in this question How to capture all requests made by page in webdriver? Is there any alternative to Browsermob? and there was a solution with Selenium alone without need of any BMP. So is it possible for Chrome?
Workaround for this problem could be: start Chrome normally (with your default profile or create another profile) and enable quic manually. Then start chromedriver with your profile loaded.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=/home/user/.config/google-chrome")
driver = webdriver.Chrome(executable_path="/home/user/Downloads/chromedriver", chrome_options=options)
I am trying to submit information in a webpage, but selenium throws this error:
UnexpectedAlertPresentException: Alert Text: This page is asking you
to confirm that you want to leave - data you have entered may not be
saved. ,
>
It's not a leave notification; here is a pic of the notification -
.
If I click in never show this notification again, my action doesn't get saved; is there a way to save it or disable all notifications?
edit: I'm using firefox.
You can disable the browser notifications, using chrome options. Sample code below:
chrome_options = webdriver.ChromeOptions()
prefs = {"profile.default_content_setting_values.notifications" : 2}
chrome_options.add_experimental_option("prefs",prefs)
driver = webdriver.Chrome(chrome_options=chrome_options)
With the latest version of Firefox the above preferences didn't work.
Below is the solution which disable notifications using Firefox object
_browser_profile = webdriver.FirefoxProfile()
_browser_profile.set_preference("dom.webnotifications.enabled", False)
webdriver.Firefox(firefox_profile=_browser_profile)
Disable notifications when using Remote Object:
webdriver.Remote(desired_capabilities=_desired_caps, command_executor=_url, options=_custom_options, browser_profile=_browser_profile)
selenium==3.11.0
Usually with browser settings like this, any changes you make are going to get throws away the next time Selenium starts up a new browser instance.
Are you using a dedicated Firefox profile to run your selenium tests? If so, in that Firefox profile, set this setting to what you want and then close the browser. That should properly save it for its next use. You will need to tell Selenium to use this profile though, thats done by SetCapabilities when you start the driver session.
This will do it:
from selenium.webdriver.firefox.options import Options
options = Options()
options.set_preference("dom.webnotifications.enabled", False)
browser = webdriver.Firefox(firefox_options=options)
For Google Chrome and v3 of Selenium you may receive "DeprecationWarning: use options instead of chrome_options", so you will want to do the following:
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
options = webdriver.ChromeOptions()
options.add_argument('--disable-notifications')
driver = webdriver.Chrome(ChromeDriverManager().install(), options=options)
Note: I am using webdriver-manager, but this also works with specifying the executable_path.
This answer is an improvement on TH Todorov code snippet, based on what is working as of Chrome (Version 80.0.3987.163).
lk = os.path.join(os.getcwd(), "chromedriver",) --> in this line you provide the link to the chromedriver, which you can download from chromedrive link
import os
from selenium import webdriver
lk = os.path.join(os.getcwd(), "chromedriver",)
chrome_options = webdriver.ChromeOptions()
prefs = {"profile.default_content_setting_values.notifications" : 2}
chrome_options.add_experimental_option("prefs",prefs)
driver = webdriver.Chrome(lk, options=chrome_options)