We Keep Coding

Selenium with Python not able to fully open the website

I have tried the following code and tried to open the website as mentioned:
driver = webdriver.Chrome(r"..\chromedriver_win32\chromedriver.exe")
driver.get("https://example.com")
The website opens with the Chrome Browser but not with the Selenium using Python.
Please let me know what should I do to open the website completely.

You can run it with chrome options. I am able to launch your application with below code:
from time import sleep
from selenium import webdriver
PATH = "chromedriver path"
option = webdriver.ChromeOptions()
option.add_argument('--disable-blink-features=AutomationControlled')
option.add_argument("start-maximized")
option.add_experimental_option(
"excludeSwitches", ["enable-automation"])
option.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(PATH, options=option)
url = 'https://example.com'
driver.get(url)
driver.maximize_window()
sleep(20)
output:

Firefox browser does not open when using Selenium webdriver module

I am expecting that the below code will open Firefox browser window, but it doesn't, only prints my log sentence.
Can anyone tell me what I am doing wrong?
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
import time as tm
binary = r'C:\Users\asgar\AppData\Local\Mozilla Firefox\firefox.exe'
options = Options()
options.headless = True
options.binary = binary
cap = DesiredCapabilities().FIREFOX
cap["marionette"] = True #optional
driver = webdriver.Firefox(options=options, capabilities=cap, executable_path=r"C:\Users\asgar\PycharmProjects\firefoxselenium\geckodriver.exe")
driver.get("http://google.com/")
tm.sleep(10)
print ("Headless Firefox Initialized")
driver.quit()

Try this; just change the path to 'geckodriver.exe'.
from selenium import webdriver
profile = webdriver.FirefoxProfile()
profile.accept_untrusted_certs = True
wd = webdriver.Firefox(executable_path="C:/your_path/geckodriver.exe", firefox_profile=profile)
url = "https://www.google.com/"
wd.get(url)
Does that work?
For one thing, the 'options.headless = True' looks suspect. Also, there may be a setting under the 3 horizontal bars (Open Menu) in the upper right hand corner of the browser that controls the behavior of new browser windows opening, so look at one of the check boxes in 'Open Menu'.

Is there a way to hide the browser while running selenium in Python?

I am working on a project with selenium to scrape the data, but I don't want the browser to open and pop up. I just wanted to hide the browser and also not to display it in the taskbar also...
Some also suggested to use phantomJS but I didn't get them. What to do now ...

If you're using Chrome you can just set the headless argument like so:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
driver_exe = 'chromedriver'
options = Options()
options.add_argument("--headless")
driver = webdriver.Chrome(driver_exe, options=options)

For chrome you could pass in the --headless parameter.
Alternatively you could let selenium work on a virtual display like this:
from selenium import webdriver
from xvfbwrapper import Xvfb
display = Xvfb()
display.start()
driver = webdriver.Chrome()
driver.get('http://www.stackoverflow.com')
print(driver.title)
driver.quit()
display.stop()
The latter has worked for me quite well.

To hide the browser while executing tests using Selenium's python you can use the minimize_window() method which eventually minimizes/pushes the Chrome Browsing Context effectively to the background using the following solution:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("--start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(chrome_options=options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
driver.get('https://www.google.co.in')
driver.minimize_window()
Alternative
As an alternative you can use the headless attribute to configure ChromeDriver to initiate google-chrome browser in Headless mode using Selenium and you can find a couple of relevant discussions in:
How to configure ChromeDriver to initiate Chrome browser in Headless mode through Selenium?

If you're using Firefox, try this:
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
driver_exe = 'path/to/firefoxdriver'
options = Options()
options.add_argument("--headless")
driver = webdriver.Firefox(driver_exe, options=options)
similar to what #Meshi answered in case of Chrome

if you want to hide chrome or selenium driver there is a library pyautogui
import pyautogui
window = [ x for x in pyautogui.getAllWindows()]
by this, you are getting all window title
now you need to find your window
for i in window:
if 'Google Chrome' in i.title:
i.hide()
or you can play with your driver title also

How to bypass the message-"your connection is not private" on non-secure page using Selenium?

I'm trying to interact with the page "Your connection is not private".
The solution of using options.add_argument('--ignore-certificate-errors') is not helpful for two reasons:
I'm using an already open window.
Even if I was using a "selenium opened window" the script runs non stop, and the issue I'm trying to solve is when my browser disconnects from a splunk dashboard and I want it to automatically connect again(and it pops the private connection window).
How do I click on "Advanced" and then click on "Proceed to splunk_server (unsafe)?

For chrome:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument('--ignore-ssl-errors=yes')
options.add_argument('--ignore-certificate-errors')
driver = webdriver.Chrome(options=options)
If not work then this:
from selenium import webdriver
from selenium.webdriver import DesiredCapabilities
options = webdriver.ChromeOptions()
options.add_argument('--allow-insecure-localhost') # differ on driver version. can ignore.
caps = options.to_capabilities()
caps["acceptInsecureCerts"] = True
driver = webdriver.Chrome(desired_capabilities=caps)
For firefox:
from selenium import webdriver
profile = webdriver.FirefoxProfile()
profile.accept_untrusted_certs = True
driver = webdriver.Firefox(firefox_profile=profile)
driver.get('https://cacert.org/')
driver.close()
If not work then this:
capabilities = webdriver.DesiredCapabilities().FIREFOX
capabilities['acceptSslCerts'] = True
driver = webdriver.Firefox(capabilities=capabilities)
driver.get('https://cacert.org/')
driver.close()
Above all worked for me!

This is how i handle this problem:
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.remote.CapabilityType;
ChromeOptions capability = new ChromeOptions();
capability.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
capability.setCapability(CapabilityType.ACCEPT_INSECURE_CERTS,true);
WebDriver driver = new ChromeDriver(capability);

This chrome option is the silver bullet for me:
chromeOptions.addArguments("--allow-running-insecure-content");
If you need more, Open chrome & paste this URL:
chrome://flags/
One will find all the options and their impact on the chrome.

Either of below 2 solutions worked for me using Python Chrome Selenium Webdriver:
from selenium import webdriver
from selenium.webdriver import DesiredCapabilities
capabilities = DesiredCapabilities.CHROME.copy()
capabilities["acceptInsecureCerts"] = True
driver = webdriver.Chrome(desired_capabilities=capabilities)
And accepted solution:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument('--ignore-ssl-errors=yes')
options.add_argument('--ignore-certificate-errors')
driver = webdriver.Chrome(options=options)

is it possible to render a webpage directly to an image in python?

Now I am using selenium to save a web page to image.
from selenium import webdriver
browser = webdriver.Firefox()
browser.get("some url")
browser.save_screenshot(img)
browser.quit()
But there is a problem that each time it will popup a window.
Is there any way that can render a image directly to an image?

I found a workaround at How do I run Selenium in Xvfb?
It works fine in linux.
from pyvirtualdisplay import Display
display = Display(visible=0, size=(800, 600))
display.start()
browser = webdriver.Firefox()
browser.get("some url")
browser.save_screenshot(img)
browser.quit()
display.stop()

You can use phantomjs
Here you can find a good tutorial: http://www.realpython.com/blog/python/headless-selenium-testing-with-python-and-phantomjs/#.UtFpiXMuKAg

Selenium can now do this without any other libraries by using the headless option
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
options = Options()
options.headless = True
browser = webdriver.Firefox(options=options)
browser.set_window_size(1600,800)
browser.get('https://www.google.com')
browser.save_full_page_screenshot('./out.png')
browser.close()