How do you determine why Firefox is crashing when run from inside some Django unittests via Selenium?
My testcase is:
from django.contrib.staticfiles.testing import StaticLiveServerTestCase
from pyvirtualdisplay import Display
from selenium import webdriver
class Tests(StaticLiveServerTestCase):
def setUp(self):
super(Tests, self).setUp()
self.vdisplay = Display(visible=0, size=(1920, 1080), backend='xvfb')
self.vdisplay.start()
profile = webdriver.FirefoxProfile()
log_path = '/tmp/tests.log'
self.driver = webdriver.Firefox(profile, log_path=log_path)
def test_abc(self):
blah
When I run this on a headless server with:
python manage.py test functional_tests --nomigrations --failfast
it almost immediately errors with:
ERROR: test_abc (myproject.functional_tests.tests.Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/myproject/src/buildbot/worker3/myproject_runtests/build/src/myproject/functional_tests/tests.py", line 15, in setUp
self.driver = webdriver.Firefox(profile, log_path=log_path)
File "/usr/local/myproject/src/buildbot/worker3/myproject_runtests/build/.env/local/lib/python2.7/site-packages/selenium/webdriver/firefox/webdriver.py", line 158, in __init__
keep_alive=True)
File "/usr/local/myproject/src/buildbot/worker3/myproject_runtests/build/.env/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 154, in __init__
self.start_session(desired_capabilities, browser_profile)
File "/usr/local/myproject/src/buildbot/worker3/myproject_runtests/build/.env/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 243, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/usr/local/myproject/src/buildbot/worker3/myproject_runtests/build/.env/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 312, in execute
self.error_handler.check_response(response)
File "/usr/local/myproject/src/buildbot/worker3/myproject_runtests/build/.env/local/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 237, in check_response
raise exception_class(message, screen, stacktrace)
WebDriverException: Message: Process unexpectedly closed with status: 1
So webdriver is having trouble initializing a Firefox instance. However, the log file only shows:
1528498122788 geckodriver INFO geckodriver 0.19.0
1528498122799 geckodriver INFO Listening on 127.0.0.1:39255
1528498123950 mozrunner::runner INFO Running command: "/usr/bin/firefox" "-marionette" "-profile" "/tmp/rust_mozprofile.LrTWF7H6fk2y"
My versions are:
Ubuntu Version is 16.04
Selenium Version is 3.8.1
Geckodriver Version is 0.19.0
Firefox Version is 58.0.1
I tried upgrading to the most recent Selenium 3.12, Geckodriver 0.20.1, and Firefox 60, but I get the exact same error.
I also tried running all system package upgrades, and rebooting, but that had no effect.
How do I fix this, or at least get a better error message about why Firefox is crashing?
Edit: This is not the same as this question, which returns a very different error message. It's solution does not resolve my error.
First, maybe you should enable trace log by:
capabilities = DesiredCapabilities.FIREFOX.copy()
capabilities["moz:firefoxOptions"] = {
"log": {
"level": "trace",
},
}
driver = webdriver.Firefox(capabilities=capabilities)
Or
opts = Options()
opts.log.level = "trace"
driver = webdriver.Firefox(firefox_options=opts)
Second, as far as I think, this issue is usually casued by Graphic environment requirement. It seems if you are not running a firefox instance in a session with valid DISPLAY, it will exit with code 1.
So maybe your question can be resolved by:
options = Options()
profile = webdriver.FirefoxProfile()
options.add_argument("--headless")
driver = webdriver.Firefox(profile, firefox_options=options)
Finally, I must say I'm actually not sure about your problem as there is just limited information.
Related
I originally wanted to get a headless browser running to dynamically parse a web page. I've been having a series of problems with this, even after adding in waits to allow website to load. After commenting out the headless option, I used an xming server as a display for the web page to see what it actually looks like after the browser is created.
Here's a screenshot of what I see:
a new tab in chrome, and the browser seems to never get to the .get().
I also get a popup saying "Chrome Automation Extension has crashed..."
I've been searching a lot, some suggest not running as admin/root, so I didn't use sudo before running it. That didn't change message.
Below is my code and the terminal output:
options = webdriver.ChromeOptions()
# options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
options.add_argument('--ignore-certificate-errors')
options.add_argument('--lang=en_US')
options.add_argument('--user-data-dir')
options.add_argument("--disable-gpu")
# options.add_argument("--remote-debugging-port=9222")
browser = webdriver.Chrome(executable_path='/usr/bin/chromedriver', options=options)
browser.get("http://www.google.com")
Ubuntu terminal:
Traceback (most recent call last):
File "practice3.py", line 40, in
browser = webdriver.Chrome(executable_path='/usr/bin/chromedriver', options=options)
File "etc/anaconda3/lib/python3.8/site-packages/selenium/webdriver/chrome/webdriver.py", line 76, in init
RemoteWebDriver.init(
File "etc/anaconda3/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in init
self.start_session(capabilities, browser_profile)
File "etc/anaconda3/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "etc/anaconda3/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "etc/anaconda3/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: DevToolsActivePort file doesn't exist
(Driver info: chromedriver=2.41.578700 (2f1ed5f9343c13f73144538f15c00b370eda6706),platform=Linux 4.4.0-18362-Microsoft x86_64)
Also:
Ubuntu 20.04 LTS
Selenium version 3.141.0
Chrome version 84.0.4147.105
~$ /usr/bin/chromedriver
Starting ChromeDriver 2.41.578700 (2f1ed5f9343c13f73144538f15c00b370eda6706) on port 9515
Only local connections are allowed.
I've seen lots of chrome crashes when there was version mismatch between chromedriver and chrome. Since you're using Chrome version 84.0.4147.105, you should be using nothing but ChromeDriver 84.0.4147.30
Second thing, are you launching the script from an IDE ? if positive that is also another common root cause. Finally I'd stop using XMimg since it adds another variable to the equation. Instead, troubleshoot taking screenshots directly from selenium code regardless you are headless or not.
I have gotten this to work in Jessie/Stretch with previous Firefox versions but can't seem to get it to work with Raspberry Pi 3B or 4 running Raspbian Buster and the latest Selenium, 3.141.0 in Python.
Since iceweasel is no longer available I install firefox-esr via apt and get the latest ARM geckodriver, which is currently 0.23.0, from https://github.com/mozilla/geckodriver/releases then run the code below:
from selenium import webdriver
from pyvirtualdisplay import Display
from selenium.webdriver.firefox.options import Options
options = Options()
options.headless = True
capabilities = webdriver.DesiredCapabilities().FIREFOX
capabilities["marionette"] = False
driver = webdriver.Firefox(options=options, executable_path='/usr/bin/geckodriver', capabilities=capabilities)
driver.get('https://www.google.com')
print(driver.page_source)
driver.close()
display.stop()
This returns the error:
Message: Can't load the profile. Possible firefox version mismatch. You must use GeckoDriver instead for Firefox 48+. Profile Dir: /tmp/tmpxMUUcF If you specified a log_file in the FirefoxBinary constructor, check it for details.
Traceback (most recent call last):
File "SeleniumDebugExample.py", line 50, in <module>
driver = webdriver.Firefox(options=options, executable_path='/srv/main/geckodriver', capabilities=capabilities)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/webdriver.py", line 191, in __init__
self.binary, timeout)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/extension_connection.py", line 52, in __init__
self.binary.launch_browser(self.profile, timeout=timeout)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/firefox_binary.py", line 73, in launch_browser
self._wait_until_connectable(timeout=timeout)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/firefox_binary.py", line 114, in _wait_until_connectable
% (self.profile.path))
WebDriverException: Message: Can't load the profile. Possible firefox version mismatch. You must use GeckoDriver instead for Firefox 48+. Profile Dir: /tmp/tmpxMUUcF If you specified a log_file in the FirefoxBinary constructor, check it for details.
I am specifying geckodriver though so I am not sure what the issue is...
If anyone has gotten headless selenium to work on Raspbian Buster and can share some could it would be greatly appreciated!
After a bunch of testing it seems the issue was with adding the marrionette=False capability... Back in Jessie with iceweasel I had needed to add that for it to work but it seems with firefox-esr that is no longer necessary... I was able to get this working with just the below:
from selenium import webdriver
from pyvirtualdisplay import Display
driver = webdriver.Firefox(executable_path='/usr/bin/geckodriver')
driver.get('https://www.google.com')
print(driver.page_source)
driver.close()
display.stop()
so today I went to run a program I made in python 3, but it kept crashing when it tried to open chromium. I haven't updated python or the chromedriver.exe so it was strange why it didn't work.
Current version of chrome driver I have installed: 74.0.3729.6
I have tried the following things but it didn't work:
Updating to the latest version of the chrome driver (v77.0.3865.10)
Downgrading to v73 of the chrome driver
Nothing has worked. Could someone please assist me.
I made this test program with just it launching the chrome browser:
main.py
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("--window-size=1920,1080")
driver = webdriver.Chrome(options=options)
driver.create_options()
driver.maximize_window()
crash message:
DevTools listening on ws://127.0.0.1:50904/devtools/browser/c1dc7138-e0cb-4ce4-a
561-56588f5ffd26
Traceback (most recent call last):
File "main.py", line 5, in <module>
driver = webdriver.Chrome(options=options)
File "C:\Users\myUser\AppData\Local\Programs\Python\Python37-32\lib\site-packages
\selenium\webdriver\chrome\webdriver.py", line 81, in __init__
desired_capabilities=desired_capabilities)
File "C:\Users\myUser\AppData\Local\Programs\Python\Python37-32\lib\site-packages
\selenium\webdriver\remote\webdriver.py", line 157, in __init__
self.start_session(capabilities, browser_profile)
File "C:\Users\myUser\AppData\Local\Programs\Python\Python37-32\lib\site-packages
\selenium\webdriver\remote\webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "C:\Users\myUser\AppData\Local\Programs\Python\Python37-32\lib\site-packages
\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\myUser\AppData\Local\Programs\Python\Python37-32\lib\site-packages
\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: session not crea
ted: This version of ChromeDriver only supports Chrome version 74
(Driver info: chromedriver=74.0.3729.6 (255758eccf3d244491b8a1317aa76e1ce10d57
e9-refs/branch-heads/3729#{#29}),platform=Windows NT 6.3.9600 x86_64)
The version of both Chrome (or Chromium) and Chromedriver must coincide.
As you can see in the error, your current Chromedriver only works with Chromium version 74.
Check the version of you Chromium and install the corresponding version of Chromedriver. To do this you can run the following command in a terminal.
apt-cache policy chromium
I'm trying to run a scraper using Selenium driver
My py file is (just start)
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import Select
from selenium.common.exceptions import NoSuchElementException
# Set the url we want
<and so on>
However I get the following error:
Traceback (most recent call last):
File "main.py", line 37, in <module>
driver = webdriver.Chrome(chrome_options=options)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/chrome/webdriver.py", line 81, in __init__
desired_capabilities=desired_capabilities)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
self.start_session(capabilities, browser_profile)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_respons
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally
(unknown error: DevToolsActivePort file doesn't exist)
(The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming has crashed.)
(Driver info: chromedriver=2.46.628388 (4a34a70827ac54148e092aafb70504c4ea7ae926),platform=Linux 4.4.0-140-generic
Searching for this error, it seems as if its incompatibility in ChromeDriver and Google-Chrome and Selenium however my Selenium version is 3.141.0, my chrome version is 72.0.3626.119 and my ChromeDriver is 2.46.
So they are all latest and they all seem compatible.
Any ideas what wrong?
I've ran into this error on Ubuntu 18.04 and it could be one of several things that's causing Chrome to crash. Try setting these flags:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("--no-sandbox")
options.add_argument("--headless")
driver = webdriver.Chrome("/usr/local/bin/chromedriver", chrome_options=options)
The --headless flag ensures that Chrome starts in headless mode as I assume you don't have a desktop running on your Ubuntu. the --no-sandbox flag turns off the sandboxing in Linux that sometimes causes problems at the start.
Having a bit of a rough time trying to get a webdriver spawning under Linux.
I'm on CentOS 7.3 I've installed Chrome, Chromium and a host of other python libs. I'm running:
chrome_options = webdriver.ChromeOptions()
thechromedriver = webdriver.Chrome(executable_path='/home/skyscraper/chromedriver', chrome_options=chrome_options)
The alternative with a different path runs fine under Windows so the problem is definitely in my port to Linux.
I've ran:
chmod a+x chromedriver
And it runs under root so it should have access to the chromedriver.
If I spawn it through a threadpool with a timeout it hits that timeout every time. Running it normally it eventually throws an error:
Traceback (most recent call last):
File "Skyscrapercomp.py", line 492, in <module>
thechromedriver = webdriver.Chrome(executable_path='/home/skyscraper/chromedriver', chrome_options=chrome_options)
File "/usr/lib/python2.7/site-packages/selenium/webdriver/chrome/webdriver.py", line 69, in __init__
desired_capabilities=desired_capabilities)
File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 92, in __init__
self.start_session(desired_capabilities, browser_profile)
File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 179, in start_session
response = self.execute(Command.NEW_SESSION, capabilities)
File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 236, in execute
self.error_handler.check_response(response)
File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 192, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally
(Driver info: chromedriver=2.27.440175 (9bc1d90b8bfa4dd181fbbf769a5eb5e575574320),platform=Linux 3.10.0-514.2.2.el7.x86_64 x86_64)
My takeaway here was "Chrome failed to start: exited abnormally" which doesn't give me a lot to go on.
I've used pip to install the latest Selenium and Yum to install Chromium and Chrome. Feeling a bit dead in the water not sure what my next step is so I'd really appreciate a prod in the right direction.
/Updates:
I checked in /usr/bin google-chrome is there which is the default location it wants.
Also tried putting the chromedriver into /usr/bin and not giving it a location:
TheBrowser = webdriver.Chrome()
TheBrowser.get("http://www.google.com")
print TheBrowser.page_source.encode('utf-8')
But this gives the same error.
Try to use logging to find the problem:
chrome_options = webdriver.ChromeOptions()
service_log_path = 'chromedriver.log'
service_args = ['--verbose']
thechromedriver = webdriver.Chrome(executable_path='/home/skyscraper/chromedriver', chrome_options=chrome_options, service_args=service_args, service_log_path=service_log_path)
In my case there was an error with display (no X11 on my Ubuntu server). This helped me:
from pyvirtualdisplay import Display
display = Display(visible=0, size=(800, 600))
display.start() # don't forget to .stop() it at the end of program