We Keep Coding

My selenium script (in Python) for chrome runs but does nothing

vscode says it run but it does nothing. My pip is up to date and i installed selenium with the command prompt. i have a valid path (i think) for chromedriver. Here's my code:
from selenium import webdriver
chromedriver = "C:Users/P-Lou/Downloads/chromedriver.exe"
driver = webdriver.Chrome(chromedriver)
driver.get("http:google.com")
The result is this:
[Running] python -u "w:\Python\Seletest\app.py"
[Done] exited with code=0 in 0.067 seconds

I tried this code by only changing the browser to firefox and removing the chromedriver = part and it worked fine, you might be running the browser in headless mode, thats why nothing pops up. You can change that using The Chromedriver.options(set_headless="false")
(not sure if that is the right code, you can check yourself here)

Presuming the ChromeDriver binary location being C:\Users\P-Lou\Downloads on your windows system you can use the following solution:
from selenium import webdriver
driver = webdriver.Chrome(executable_path=r'C:\Users\P-Lou\Downloads\chromedriver.exe')
driver.get("http:google.com")

Selenium and Firefox issues [duplicate]

There are about 100 posts about the same issue but none of them seem to work for me, hence asking again. I'm trying to launch a Firefox browser using Python and Selenium and I get the following error:
WebDriverException: Message: The browser appears to have exited before we could connect. If you specified a log_file in the FirefoxBinary constructor, check it for details.
I tried each and every answer on the web but nothing seems to work.
This is my code:
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
caps = DesiredCapabilities.FIREFOX
caps["marionette"] = False
binary = FirefoxBinary('d:\\Desktop\\IEDriver\\geckodriver.exe')
options = Options()
options.set_headless(headless=True)
driver = webdriver.Firefox(firefox_binary=binary, firefox_options=options, executable_path=r'd:\\Desktop\\IEDriver\\geckodriver.exe')
driver.get("http://google.com/")
print ("Headless Firefox Initialized")
driver.quit()
If I set caps["marionette"] = True then the error I get is
SessionNotCreatedException: Message: Unable to find a matching set of capabilities
Versions of software I'm running:
Firefox: 62.0 (64 bit)
Selenium: 3.14.0
Gecko: 0.21.0
Python: 3
OS: Windows 8.1 64 bit
Any help would be highly appreciated.
EDIT: I've uninstalled and re-installed Firefox but didn't work. Also tried installing Firefox 61.0.2, still no luck.

This error message...
WebDriverException: Message: The browser appears to have exited before we could connect.
If you specified a log_file in the FirefoxBinary constructor, check it for details.
...implies that the GeckoDriver was unable to initiate/spawn a new WebBrowser i.e. Firefox Browser session.
You need to take care of a couple of things as follows:
To set the FirefoxBinary you need to use the FirefoxOptions() and instead of passing the absolute path of geckodriver binary, you have to pass the absolute path of the desired firefox binary.
As you are using GeckoDriver v0.21.0 you have to mandatorily use marionette so either keep it unchanged (by default true) or set marionette to true.
Your own code with incorporating the minor changes will be:
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
binary = r'C:\Program Files\Mozilla Firefox\firefox.exe'
options = Options()
options.set_headless(headless=True)
options.binary = binary
cap = DesiredCapabilities().FIREFOX
cap["marionette"] = True #optional
driver = webdriver.Firefox(firefox_options=options, capabilities=cap, executable_path="C:\\Utility\\BrowserDrivers\\geckodriver.exe")
driver.get("http://google.com/")
print ("Headless Firefox Initialized")
driver.quit()
Console Output:
Headless Firefox Initialized
Here you can find a detailed discussion on Unable to find a matching set of capabilities with selenium 3.4.3, firefox 54.0 and gecko driver 0.17

Make sure (especially on Windows (Win 10)) that your browser and controller (python/C/java/perl/etc) is all either x64 or win32, Microsoft will not thunk between them anymore.
So, if your trying to control a 64 bit browser (what will be downloaded by default from firefox) from a x32 bit python, it will exit before you can connect.. go and install a win32 version of firefox for the magic to happen

After trying almost all of the answers on different forums, a simple self trial resolved the problem and i.e. you need to have python, firefox browser and geckodriver in either 62 bit or 32 bit. Mismatch in this caused the problem in my case.
After ensuring that you are using the same bit version for all the 3 components, just use following lines to run firefox:
ffPath = "C:\\Drivers\\geckodriver.exe"
os.environ["webdriver.firefox.driver"] = ffPath
driver = webdriver.Firefox(executable_path=ffPath)
driver.get(url)

The issue for me was the mis-match version between python and gekodriver . once all the involved party were 64 bit it worked like a charm

This error has troubled me a lot.
Install this:
pip install -U selenium

binary = r'C:\Program Files\Mozilla Firefox\firefox.exe'
options = Options()
options.binary = binary
browser = webdriver.Firefox(firefox_options=options, executable_path=r"C:\Drivers\geckodriver.exe")

Running Selenium with Headless Chrome Webdriver

So I'm trying some stuff out with selenium and I really want it to be quick.
So my thought is that running it with headless chrome would make my script faster.
First is that assumption correct, or does it not matter if i run my script with a headless driver?
Anyways I still want to get it to work to run headless, but I somehow can't, I tried different things and most suggested that it would work as said here in the October update
How to configure ChromeDriver to initiate Chrome browser in Headless mode through Selenium?
But when I try that, I get weird console output and it still doesn't seem to work.
Any tipps appreciated.

To run chrome-headless just add --headless via chrome_options.add_argument, i.e.:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
#chrome_options.add_argument("--disable-extensions")
#chrome_options.add_argument("--disable-gpu")
#chrome_options.add_argument("--no-sandbox") # linux only
chrome_options.add_argument("--headless")
# chrome_options.headless = True # also works
driver = webdriver.Chrome(options=chrome_options)
start_url = "https://duckgo.com"
driver.get(start_url)
print(driver.page_source.encode("utf-8"))
# b'<!DOCTYPE html><html xmlns="http://www....
driver.quit()
So my thought is that running it with headless chrome would make my
script faster.
Try using chrome options like --disable-extensions or --disable-gpu and benchmark it, but I wouldn't count with much improvement.
References: headless-chrome

Install & run containerized Chrome:
docker pull selenium/standalone-chrome
docker run --rm -d -p 4444:4444 --shm-size=2g selenium/standalone-chrome
Connect using webdriver.Remote:
driver = webdriver.Remote('http://localhost:4444/wd/hub', webdriver.DesiredCapabilities.CHROME)
driver.set_window_size(1280, 1024)
driver.get('https://www.google.com')

from time import sleep
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("--headless")
driver = webdriver.Chrome(executable_path="./chromedriver", options=chrome_options)
url = "https://stackoverflow.com/questions/53657215/running-selenium-with-headless-chrome-webdriver"
driver.get(url)
sleep(5)
h1 = driver.find_element_by_xpath("//h1[#itemprop='name']").text
print(h1)
Then I run script on our local machine
➜ python script.py
Running Selenium with Headless Chrome Webdriver
It is working and it is with headless Chrome.

If you are using Linux environment, may be you have to add --no-sandbox as well and also specific window size settings. The --no-sandbox flag is no needed on Windows if you set user container properly.
Use --disable-gpu only on Windows. Other platforms no longer require it. The --disable-gpu flag is a temporary work around for a few bugs.
//Headless chrome browser and configure
WebDriverManager.chromedriver().setup();
ChromeOptions chromeOptions = new ChromeOptions();
chromeOptions.addArguments("--no-sandbox");
chromeOptions.addArguments("--headless");
chromeOptions.addArguments("disable-gpu");
// chromeOptions.addArguments("window-size=1400,2100"); // Linux should be activate
driver = new ChromeDriver(chromeOptions);

Once you have selenium and web driver installed. Below worked for me with headless Chrome on linux cluster :
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("--headless")
options.add_argument("--disable-extensions")
options.add_argument("--disable-dev-shm-usage")
options.add_argument("--no-sandbox")
options.add_experimental_option("prefs",{"download.default_directory":"/databricks/driver"})
driver = webdriver.Chrome(chrome_options=options)

Todo (tested on headless server Debian Linux 9.4):
Do this:
# install chrome
curl -sS -o - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list
apt-get -y update
apt-get -y install google-chrome-stable
# install chrome driver
wget https://chromedriver.storage.googleapis.com/77.0.3865.40/chromedriver_linux64.zip
unzip chromedriver_linux64.zip
mv chromedriver /usr/bin/chromedriver
chown root:root /usr/bin/chromedriver
chmod +x /usr/bin/chromedriver
Install selenium:
pip install selenium
and run this Python code:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("no-sandbox")
options.add_argument("headless")
options.add_argument("start-maximized")
options.add_argument("window-size=1900,1080");
driver = webdriver.Chrome(chrome_options=options, executable_path="/usr/bin/chromedriver")
driver.get("https://www.example.com")
html = driver.page_source
print(html)

As stated by the accepted answer:
options.add_argument("--headless")
These tips might help to speed things up especially for headless:
There are quite a few things you can do in headless that you cant do in non headless
Since you will be using Chrome Headless, I've found adding this reduces the CPU usage by about 20% for me (I found this to be a CPU and memory hog when looking at htop)
--disable-crash-reporter
This will only disable when you are running in headless This might speed things up for you!!!
My settings are currently as follows and I reduce the CPU (but only a marginal time saving) by about 20%:
options.add_argument("--no-sandbox");
options.add_argument("--disable-dev-shm-usage");
options.add_argument("--disable-renderer-backgrounding");
options.add_argument("--disable-background-timer-throttling");
options.add_argument("--disable-backgrounding-occluded-windows");
options.add_argument("--disable-client-side-phishing-detection");
options.add_argument("--disable-crash-reporter");
options.add_argument("--disable-oopr-debug-crash-dump");
options.add_argument("--no-crash-upload");
options.add_argument("--disable-gpu");
options.add_argument("--disable-extensions");
options.add_argument("--disable-low-res-tiling");
options.add_argument("--log-level=3");
options.add_argument("--silent");
I found this to be a pretty good list (full list I think) of command line switches with explanations: https://peter.sh/experiments/chromium-command-line-switches/
Some additional things you can turn off are also mentioned here: https://github.com/GoogleChrome/chrome-launcher/blob/main/docs/chrome-flags-for-tools.md
I hope this helps someone

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("--headless")
driver = webdriver.Chrome(executable_path=r"C:\Program
Files\Google\Chrome\Application\chromedriver.exe", options=chrome_options)
This is ok for me.

Selenium webdriver seems not working properly with Firefox 49.0. Am I missing something?

I have gone through previous questions but did not find anyone else running into my issue.
This simple code hangs
from selenium import webdriver
d = webdriver.Firefox();
The browser gets launched, but it just hangs there. After sometime, it crashes and I get the error
selenium.common.exceptions.WebDriverException: Message: Can't load the profile.
Profile Dir: /tmp/tmpn_MQnf If you specified a log_file in the
FirefoxBinary constructor, check it for details.
I have Firefox49 on Ubuntu 14.04 LTS. I had selenium 2.53.6 and reading a previous post, I upgraded to selenium 3.0.0.b3. I also downloaded geckdriver and put it in /usr/bin
It looks like I was still running older version of selenium. But when I uninstalled that and installed selenium 3.0.0.b3, I see the following error -
selenium.‌​common.exceptions.We‌​bDriverException:
Message: Service geckodriver unexpectedly exited. Status code was: 2
What am I missing?

Try renaming the downloaded Gecko Driver to Wires and Set the Capabilities as mentioned Below.
System.setProperty("webdriver.gecko.driver", "//home//.....//BrowserDrivers//wires");
DesiredCapabilities capabilities = DesiredCapabilities.firefox();
capabilities.setCapability("marionette", true);
Driver = new MarionetteDriver(capabilities);
I tried with the Above code on FireFox 49 on Linux Ubuntu 14.04 LTS... Code Works Fine For Me in Java..
Also try to downgrade the selenium WebDriver from Beta to 2.53 as Beta version is unstable..

Headless Selenium + Xvfb + Chrome on OSX 10.11

Okay, so first I learned that Xvfb wasn't included with my OS X version, so I installed it from http://www.xquartz.org/.
and that seemed to have worked:
which xvfb
/opt/X11/bin/xvfb
But when I try using it with either pyvirtualdisplay and xvfbwrapper, following advice I found on this question How do I run Selenium in Xvfb? My script runs without errors but just opens in a Chrome browser window:
from selenium import webdriver
from pyvirtualdisplay import Display
display = Display(visible=0, size=(800, 600))
display.start()
browser = webdriver.Chrome()
browser.get('google.com')
Am I doing something wrong here?

I believe that Chrome is built for Quartz ui framework, so it ignores the X11 windowing engine. You will need to install an X11 version of a browser and then execute that.

For me, this code works fine on OSX 10.13. You don't need pyvirtualdisplay, because you can run chrome in headless mode. Just download chromedriver that fits to your chrome version and put it to usr/local/bin
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument('--headless')
options.add_argument('--mute-audio')
options.add_argument('--lang=de')
options.add_argument('--window-size=800,600')
options.add_argument('--disable-notifications')
options.add_argument('--enable-popup-blocking')
browser = webdriver.Chrome(chrome_options=options, executable_path='/usr/local/bin/chromedriver')
browser.get('some url')