We Keep Coding

Google chrome closes automatically after executing Python selenium code despite using the option class to keep it open

I have checked online and it was mentioned that Selenium closes the browser after running unless you use the option module or the driver.quit() or driver.close() functions but I used the option as shown in the code below but Chrome still closes after 2-3 seconds.
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_experimental_option("detach", True)
chrome_driver_path = r"C:\Development\chromedriver.exe"
serv = Service(chrome_driver_path)
driver = webdriver.Chrome(service=serv, options=chrome_options)
driver.get("https://www.google.com")

How do I stop Selenium from closing the driver during execution?

I am trying to learn Selenium to scrape some Javascript heavy websites. I can locate and extract information just fine. However, I find that for some sites I need to switch my user agent. I did it the following way to test it:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from fake_useragent import UserAgent
PATH ="C:/my/path/to/chromedriver.exe"
ua = UserAgent()
userAgent = ua.random
print(userAgent)
options = Options()
options.add_argument(f'user-agent={userAgent}')
driver = webdriver.Chrome(chrome_options=options, executable_path=PATH)
driver.get("https://www.whatismybrowser.com/detect/what-is-my-user-agent")
The code works and my user agent is switched, however there is one bug that occurs now which did not occur before. The webdriver/browser (Chrome driver) automatically closes after displaying the website for a second without me specifying the driver.quit() argument. When I do not switch my user agent it does not close unless I do and I want to study the page a bit before closing it. I have tried to wait using time.sleep() but this doesn't work.
How can I make the webdriver not close until specified?
Answers are greatly appreciated, preferably with a code example of how to implement the solution.

This should do you nicely:
options.add_experimental_option("detach", True)
in your code:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from fake_useragent import UserAgent
PATH ="C:/my/path/to/chromedriver.exe"
ua = UserAgent()
userAgent = ua.random
print(userAgent)
options = Options()
options.add_argument(f'user-agent={userAgent}')
options.add_experimental_option("detach", True)
driver = webdriver.Chrome(chrome_options=options, executable_path=PATH)
driver.get("https://www.whatismybrowser.com/detect/what-is-my-user-agent")

I am not sure if it is possible that the problem is related to the webdriver version you are using or not but when I tried to add time.sleep(n) to your code while using webdriver_manager library to download most recent version of ChromeWebDriver I had the chance to look at the website and the browser didn't close until the timer finished.
My code:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from fake_useragent import UserAgent
from webdriver_manager.chrome import ChromeDriverManager
import time
# PATH ="C:/my/path/to/chromedriver.exe"
ua = UserAgent()
userAgent = ua.random
print(userAgent)
options = Options()
options.add_argument(f'user-agent={userAgent}')
driver = webdriver.Chrome(ChromeDriverManager().install(), chrome_options=options)
driver.get("https://www.whatismybrowser.com/detect/what-is-my-user-agent")
time.sleep(100)

Is there a way to hide the browser while running selenium in Python?

I am working on a project with selenium to scrape the data, but I don't want the browser to open and pop up. I just wanted to hide the browser and also not to display it in the taskbar also...
Some also suggested to use phantomJS but I didn't get them. What to do now ...

If you're using Chrome you can just set the headless argument like so:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
driver_exe = 'chromedriver'
options = Options()
options.add_argument("--headless")
driver = webdriver.Chrome(driver_exe, options=options)

For chrome you could pass in the --headless parameter.
Alternatively you could let selenium work on a virtual display like this:
from selenium import webdriver
from xvfbwrapper import Xvfb
display = Xvfb()
display.start()
driver = webdriver.Chrome()
driver.get('http://www.stackoverflow.com')
print(driver.title)
driver.quit()
display.stop()
The latter has worked for me quite well.

To hide the browser while executing tests using Selenium's python you can use the minimize_window() method which eventually minimizes/pushes the Chrome Browsing Context effectively to the background using the following solution:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("--start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(chrome_options=options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
driver.get('https://www.google.co.in')
driver.minimize_window()
Alternative
As an alternative you can use the headless attribute to configure ChromeDriver to initiate google-chrome browser in Headless mode using Selenium and you can find a couple of relevant discussions in:
How to configure ChromeDriver to initiate Chrome browser in Headless mode through Selenium?

If you're using Firefox, try this:
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
driver_exe = 'path/to/firefoxdriver'
options = Options()
options.add_argument("--headless")
driver = webdriver.Firefox(driver_exe, options=options)
similar to what #Meshi answered in case of Chrome

if you want to hide chrome or selenium driver there is a library pyautogui
import pyautogui
window = [ x for x in pyautogui.getAllWindows()]
by this, you are getting all window title
now you need to find your window
for i in window:
if 'Google Chrome' in i.title:
i.hide()
or you can play with your driver title also

Using selenium: How to keep logged in after closing Driver in Python

I want to get my Whatsapp web (web.whatsapp.com) logged in, at the second time opening the Whatsapp web on chrome driver. Following is my code based on Python need your help.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_path = r"chromedriver.exe"
options = Options();
options.add_argument("user-data-
dir=C:/Users/Username/AppData/Local/Google/Chrome/User Data");
#options.add_argument("--start-maximized");
driver = webdriver.Chrome(chrome_path,chrome_options=options);
#driver = webdriver.Chrome();
driver.get('https://web.whatsapp.com/')

I tried on my Mac, below code and it worked perfectly fine, I don't need to login again
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("user-data-dir=/tmp/tarun")
driver = webdriver.Chrome(chrome_options=options)
driver.get('https://web.whatsapp.com/')
driver.quit()
For window you can try changing the path as below
options.add_argument("user-data-dir=C:\\Users\\Username\\AppData\\Local\\Google\\Chrome\\User Data")

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("user-data-dir=C:\\Users\\Username\\AppData\\Local\\Google\\Chrome\\User Data")
driver = webdriver.Chrome(chrome_options=options)
driver.get('https://web.whatsapp.com/')
driver.quit()
Here it is for Windows. Works perfect on Python 3.6

How can I get the page source, without showing the page opened, using selenium with chromedriver and python?

I'm using selenium with Chrome driver; How can I get the page source, without showing the page opened? What I should specify in webdriver.ChromeOptions()?
Here the code:
from selenium.common.exceptions import WebDriverException
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("???")
bowser = webdriver.Chrome(chrome_options=chrome_options)
browser = webdriver.Chrome()
try:
browser.get("www.google.com")
html_content = browser.page_source
#do stuff
browser.quit()
except WebDriverException:
print "Invalid URL"

You should not use ChromeDriver but some headless Webdriver like HtmlUnitDriver, explained here

If you are adamant to use selenium, then you can use any of the headless browsers such as htmlunit driver.
Else you can can just send a get request on the URL and get the response text.

Selenium / Chrome has a headless option, which allows you to load webpages from code:
from selenium.webdriver import Chrome
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("--headless")
browser = Chrome(options=chrome_options, executable_path='path_to_chromedriver')
browser.get('https://wwww.mywebsite.com')