i use selenium for instagram appicatin.
i wonder there is any way to run selenium without browser opening in windows?
there is some solution in linux like install Xvfb but i don't see any solution for windows!
Do you mean that --headless?
You can do some operations without opening Chrome if you have chrome-driver.
Here is how to set headless:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chromeoption = Options()
chromeoption.add_argument('--headless')
browser = webdriver.Chrome(options=chromeoption)
# Then it is your work.
browser.get(url="xxx")
Related
So I'm trying some stuff out with selenium and I really want it to be quick.
So my thought is that running it with headless chrome would make my script faster.
First is that assumption correct, or does it not matter if i run my script with a headless driver?
Anyways I still want to get it to work to run headless, but I somehow can't, I tried different things and most suggested that it would work as said here in the October update
How to configure ChromeDriver to initiate Chrome browser in Headless mode through Selenium?
But when I try that, I get weird console output and it still doesn't seem to work.
Any tipps appreciated.
To run chrome-headless just add --headless via chrome_options.add_argument, i.e.:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
#chrome_options.add_argument("--disable-extensions")
#chrome_options.add_argument("--disable-gpu")
#chrome_options.add_argument("--no-sandbox") # linux only
chrome_options.add_argument("--headless")
# chrome_options.headless = True # also works
driver = webdriver.Chrome(options=chrome_options)
start_url = "https://duckgo.com"
driver.get(start_url)
print(driver.page_source.encode("utf-8"))
# b'<!DOCTYPE html><html xmlns="http://www....
driver.quit()
So my thought is that running it with headless chrome would make my
script faster.
Try using chrome options like --disable-extensions or --disable-gpu and benchmark it, but I wouldn't count with much improvement.
References: headless-chrome
Install & run containerized Chrome:
docker pull selenium/standalone-chrome
docker run --rm -d -p 4444:4444 --shm-size=2g selenium/standalone-chrome
Connect using webdriver.Remote:
driver = webdriver.Remote('http://localhost:4444/wd/hub', webdriver.DesiredCapabilities.CHROME)
driver.set_window_size(1280, 1024)
driver.get('https://www.google.com')
from time import sleep
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("--headless")
driver = webdriver.Chrome(executable_path="./chromedriver", options=chrome_options)
url = "https://stackoverflow.com/questions/53657215/running-selenium-with-headless-chrome-webdriver"
driver.get(url)
sleep(5)
h1 = driver.find_element_by_xpath("//h1[#itemprop='name']").text
print(h1)
Then I run script on our local machine
➜ python script.py
Running Selenium with Headless Chrome Webdriver
It is working and it is with headless Chrome.
If you are using Linux environment, may be you have to add --no-sandbox as well and also specific window size settings. The --no-sandbox flag is no needed on Windows if you set user container properly.
Use --disable-gpu only on Windows. Other platforms no longer require it. The --disable-gpu flag is a temporary work around for a few bugs.
//Headless chrome browser and configure
WebDriverManager.chromedriver().setup();
ChromeOptions chromeOptions = new ChromeOptions();
chromeOptions.addArguments("--no-sandbox");
chromeOptions.addArguments("--headless");
chromeOptions.addArguments("disable-gpu");
// chromeOptions.addArguments("window-size=1400,2100"); // Linux should be activate
driver = new ChromeDriver(chromeOptions);
Once you have selenium and web driver installed. Below worked for me with headless Chrome on linux cluster :
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("--headless")
options.add_argument("--disable-extensions")
options.add_argument("--disable-dev-shm-usage")
options.add_argument("--no-sandbox")
options.add_experimental_option("prefs",{"download.default_directory":"/databricks/driver"})
driver = webdriver.Chrome(chrome_options=options)
Todo (tested on headless server Debian Linux 9.4):
Do this:
# install chrome
curl -sS -o - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list
apt-get -y update
apt-get -y install google-chrome-stable
# install chrome driver
wget https://chromedriver.storage.googleapis.com/77.0.3865.40/chromedriver_linux64.zip
unzip chromedriver_linux64.zip
mv chromedriver /usr/bin/chromedriver
chown root:root /usr/bin/chromedriver
chmod +x /usr/bin/chromedriver
Install selenium:
pip install selenium
and run this Python code:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("no-sandbox")
options.add_argument("headless")
options.add_argument("start-maximized")
options.add_argument("window-size=1900,1080");
driver = webdriver.Chrome(chrome_options=options, executable_path="/usr/bin/chromedriver")
driver.get("https://www.example.com")
html = driver.page_source
print(html)
As stated by the accepted answer:
options.add_argument("--headless")
These tips might help to speed things up especially for headless:
There are quite a few things you can do in headless that you cant do in non headless
Since you will be using Chrome Headless, I've found adding this reduces the CPU usage by about 20% for me (I found this to be a CPU and memory hog when looking at htop)
--disable-crash-reporter
This will only disable when you are running in headless This might speed things up for you!!!
My settings are currently as follows and I reduce the CPU (but only a marginal time saving) by about 20%:
options.add_argument("--no-sandbox");
options.add_argument("--disable-dev-shm-usage");
options.add_argument("--disable-renderer-backgrounding");
options.add_argument("--disable-background-timer-throttling");
options.add_argument("--disable-backgrounding-occluded-windows");
options.add_argument("--disable-client-side-phishing-detection");
options.add_argument("--disable-crash-reporter");
options.add_argument("--disable-oopr-debug-crash-dump");
options.add_argument("--no-crash-upload");
options.add_argument("--disable-gpu");
options.add_argument("--disable-extensions");
options.add_argument("--disable-low-res-tiling");
options.add_argument("--log-level=3");
options.add_argument("--silent");
I found this to be a pretty good list (full list I think) of command line switches with explanations: https://peter.sh/experiments/chromium-command-line-switches/
Some additional things you can turn off are also mentioned here: https://github.com/GoogleChrome/chrome-launcher/blob/main/docs/chrome-flags-for-tools.md
I hope this helps someone
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("--headless")
driver = webdriver.Chrome(executable_path=r"C:\Program
Files\Google\Chrome\Application\chromedriver.exe", options=chrome_options)
This is ok for me.
I followed setting up chrome requirements for selenium.webdriver.Chrome, and I used the following code Running webdriver chrome with Selenium:
import os
from selenium import webdriver
from pyvirtualdisplay import Display
display = Display(visible=0, size=(800, 600))
display.start()
driver = webdriver.Chrome()
driver.get("http://www.google.com")
print driver.page_source.encode('utf-8')
I was very surprised to see that the code works, but no browser pops up. I would think this is almost impossible to develop, as I can't see what I'm doing right/wrong. How do I use selenium to actually see a functional Chrome driver (python)? Thank you
ANSWER:
set visible to 1, install emulator:
sudo apt-get install xvfb xserver-xephyr
display = Display(visible=1, size=(800, 600))
In java we add these two lines for executing code in chrome
System.setProperty("webdriver.chrome.driver", "/path/to/chromedriver");
WebDriver driver = new ChromeDriver();
not sure if there something similar for python as well!
Okay, so first I learned that Xvfb wasn't included with my OS X version, so I installed it from http://www.xquartz.org/.
and that seemed to have worked:
which xvfb
/opt/X11/bin/xvfb
But when I try using it with either pyvirtualdisplay and xvfbwrapper, following advice I found on this question How do I run Selenium in Xvfb? My script runs without errors but just opens in a Chrome browser window:
from selenium import webdriver
from pyvirtualdisplay import Display
display = Display(visible=0, size=(800, 600))
display.start()
browser = webdriver.Chrome()
browser.get('google.com')
Am I doing something wrong here?
I believe that Chrome is built for Quartz ui framework, so it ignores the X11 windowing engine. You will need to install an X11 version of a browser and then execute that.
For me, this code works fine on OSX 10.13. You don't need pyvirtualdisplay, because you can run chrome in headless mode. Just download chromedriver that fits to your chrome version and put it to usr/local/bin
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument('--headless')
options.add_argument('--mute-audio')
options.add_argument('--lang=de')
options.add_argument('--window-size=800,600')
options.add_argument('--disable-notifications')
options.add_argument('--enable-popup-blocking')
browser = webdriver.Chrome(chrome_options=options, executable_path='/usr/local/bin/chromedriver')
browser.get('some url')
I am trying to test a webpage with Selenium. My code is below.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Firefox()
driver.get("http://www.python.org")
assert "Python" in driver.title
elem = driver.find_element_by_name("q")
elem.send_keys("selenium")
elem.send_keys(Keys.RETURN)
driver.close()
When I run the above code, a Firefox browser window pops up as expected. However, the size of the window's fields are very small (see picture below). So small that the left most tab runs into the window's zoom button. This sizing issue does not occur when I use the Chrome WebDriver. Has anyone had this sizing issue? Any has anyone found out how to solve this issue?
Here is my mini-stack:
selenium 2.42.1 (installed via pip)
python 2.7.6
firefox 29.0.1
Mac OSX 10.9.3
This is the issue with the latest selenium package version:
Selenium launches FF with tiny font on OSX Macbook
As a workaround, downgrade selenium to 2.40.0 version:
pip install selenium==2.40.0
I've used Firefox with Webdriver on this machine before, but suddenly when I use any code that utilizes Firefox as the driver, Firefox will open and just sit there blankly. Absolutely nothing loads, my program hangs until I close the window at which point it raises the "browser not open" error.
Tried reinstalling. Chrome/IE Drivers work but I don't really want to use them.
Included a pic of exactly what happens when I use any code, for instance
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Firefox()
url = "http://google.com"
driver.get(url)
Even this gets me nothing.
upgrade to selenium version 2.29.0
pip install -U selenium
there was a bug fix in the previous version