We Keep Coding

Selenium - Python script works from CLI, but errors when launching with cron

Ill start with the fact that I am a total beginner at Python scripts and programming in general. I want to automate a Raspberry Pi to boot up, open Chromium in full screen, go to a web page, and log in all automatically. Basically so non technical people can just turn it on and bang, there is it no keyboard or anything needed, just a display. I've been working on this for about a week and have learned a ton, but I have hit a wall and cant get past it.
Running on a Ras Pi 4+, with Raspbian 10
Installed Selenium and Chromedriver
I have the Selenium script done in Python:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
chrome_options.add_argument('--start-fullscreen')
chrome_options.add_argument('user-data-dir=/home/pi/Documents/website_login/Chromium_user_data')
chrome_options.add_argument('disable-infobars')
chrome_options.add_argument("--disable-extensions")
chrome_options.add_argument("--disable-gpu")
driver = webdriver.Chrome('/usr/lib/chromium-browser/chromedriver', options=chrome_options)
driver.get("https://www.mywebsite.com")
driver.implicitly_wait(10)
#driver.find_element_by_id('rcc-confirm-button').click() #comment out because after run once the cookies banner gets saves in user settings
driver.find_element_by_link_text('Log in').click()
delay = 5
driver.find_element_by_id('ddlsubsciribers').send_keys('agency')
driver.find_element_by_id('memberfname').send_keys('user')
driver.find_element_by_id('memberpwd').send_keys('password')
driver.find_element_by_id('login').click()
It works fine when I run it from the terminal. Opens chromium, goes to the page, does the Selenium magic, logs in and is exactly what I want.
I then try to get it to launch on startup with crontab -e on the Pi using this command:
#reboot sleep 20; /usr/bin/python3 /home/pi/Documents/website_login/iar_login.py > /home/pi/Documents/website_login/iar_errorlog.err >2&1
Nothing happens when I reboot and I get the following error messages in the log file:
Traceback (most recent call last):
File "/home/pi/Documents/website_login/iar_login.py", line 16, in <module>
driver = webdriver.Chrome('/usr/lib/chromium-browser/chromedriver', options=chrome_options)
File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/chrome/webdriver.py", line 81, in __init__
desired_capabilities=desired_capabilities)
File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
self.start_session(capabilities, browser_profile)
File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally.
(unknown error: DevToolsActivePort file doesn't exist)
(The process started from chrome location /usr/bin/chromium-browser is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
I have searched and read everything I can on the errors. I think it may be a permissions error of some type, but I am out of ideas to try.

Fixed. adding 'export DISPLAY=:0' in the crontab file fixed it. Selenium was trying to execute but it couldn't find a display.

My python scripts using selenium don't work anymore. Chrome driver version problem

My scripts don't work anymore and I can't figure it out.
It is a chrome version problem apparently... But I don't know how to switch to another version (not the latest?) Does exist another way?
My terminal indicates :
Traceback (most recent call last):
File "/Users/.../Documents/SCRIPTS/PYTHON/Scripts/# -- coding: utf-8 --.py", line 21, in
driver = webdriver.Chrome()
File "/opt/anaconda3/lib/python3.7/site-packages/selenium/webdriver/chrome/webdriver.py", line 81, in init
desired_capabilities=desired_capabilities)
File "/opt/anaconda3/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in init
self.start_session(capabilities, browser_profile)
File "/opt/anaconda3/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/opt/anaconda3/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/opt/anaconda3/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: session not created: Chrome version must be between 71 and 75
(Driver info: chromedriver=2.46.628411 (3324f4c8be9ff2f70a05a30ebc72ffb013e1a71e),platform=Mac OS X 10.14.5 x86_64)
Any idea?

If you're running your test with ChromeDriver & Chrome browser, from the log given, i can see the ChromeDriver was not compatible with your current Chrome Browser. Because the ChromeDriver version 2.46 aims to support Chrome browser from v71-v73. (You can refer to the ChromeDriver change log here http://chromedriver.chromium.org/downloads ). Since v75, there's a major and notable change is ChromeDriver now runs in W3C standard compliant mode by default.
The easiest way is to download the newer ChromeDriver which is compatible with your current Chrome Browser rather than sticking with the old , non supported ChromeDriver version.

This possibly happens, as your Chrome Browser or Chromium may be updated to newer versions automatically. But you still run your selenium scripts using the old version of the chromedriver.
Check the current version of your Google chrome or Chromium, then download the chromedriver for that specific version.
Then your scripts may work fine!

Can't run selenium Chromedriver (was working fine before)

so today I went to run a program I made in python 3, but it kept crashing when it tried to open chromium. I haven't updated python or the chromedriver.exe so it was strange why it didn't work.
Current version of chrome driver I have installed: 74.0.3729.6
I have tried the following things but it didn't work:
Updating to the latest version of the chrome driver (v77.0.3865.10)
Downgrading to v73 of the chrome driver
Nothing has worked. Could someone please assist me.
I made this test program with just it launching the chrome browser:
main.py
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("--window-size=1920,1080")
driver = webdriver.Chrome(options=options)
driver.create_options()
driver.maximize_window()
crash message:
DevTools listening on ws://127.0.0.1:50904/devtools/browser/c1dc7138-e0cb-4ce4-a
561-56588f5ffd26
Traceback (most recent call last):
File "main.py", line 5, in <module>
driver = webdriver.Chrome(options=options)
File "C:\Users\myUser\AppData\Local\Programs\Python\Python37-32\lib\site-packages
\selenium\webdriver\chrome\webdriver.py", line 81, in __init__
desired_capabilities=desired_capabilities)
File "C:\Users\myUser\AppData\Local\Programs\Python\Python37-32\lib\site-packages
\selenium\webdriver\remote\webdriver.py", line 157, in __init__
self.start_session(capabilities, browser_profile)
File "C:\Users\myUser\AppData\Local\Programs\Python\Python37-32\lib\site-packages
\selenium\webdriver\remote\webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "C:\Users\myUser\AppData\Local\Programs\Python\Python37-32\lib\site-packages
\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\myUser\AppData\Local\Programs\Python\Python37-32\lib\site-packages
\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: session not crea
ted: This version of ChromeDriver only supports Chrome version 74
(Driver info: chromedriver=74.0.3729.6 (255758eccf3d244491b8a1317aa76e1ce10d57
e9-refs/branch-heads/3729#{#29}),platform=Windows NT 6.3.9600 x86_64)

The version of both Chrome (or Chromium) and Chromedriver must coincide.
As you can see in the error, your current Chromedriver only works with Chromium version 74.
Check the version of you Chromium and install the corresponding version of Chromedriver. To do this you can run the following command in a terminal.
apt-cache policy chromium

Selenium doesn't open the specified URL and shows data:,

I am trying to open the URL using selenium in chrome. I have chromedriver available with me.
following is the code I want to execute.
from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--disable-infobars")
driver = webdriver.Chrome(executable_path="./chromedriver", chrome_options=chrome_options)
driver.get("https://google.com")
The browser is opened successfully but it doesn't open the specified URL. The URL in the browser is data:,.
Any help will be greatly appreciated. Please!
Please see the attached image.
Note: Selenium version : 3.14.0
I get the following error on closing the chrome tab.
File "test.py", line 6, in <module>
driver = webdriver.Chrome(executable_path="./chromedriver", chrome_options=chrome_options)
File "/home/speedious/anaconda3/lib/python3.5/site-packages/selenium/webdriver/chrome/webdriver.py", line 75, in __init__
desired_capabilities=desired_capabilities)
File "/home/speedious/anaconda3/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 156, in __init__
self.start_session(capabilities, browser_profile)
File "/home/speedious/anaconda3/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 251, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/home/speedious/anaconda3/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 320, in execute
self.error_handler.check_response(response)
File "/home/speedious/anaconda3/lib/python3.5/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited normally
(chrome not reachable)
(The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
(Driver info: chromedriver=2.42.591071 (0b695ff80972cc1a65a5cd643186d2ae582cd4ac),platform=Linux 4.10.0-37-generic x86_64)

This error message...
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited normally
(chrome not reachable)
(The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
...implies that the ChromeDriver instance was unable to start the Chrome Browser process.
Your main issue is the google-chrome is no longer present at the expected default location of /usr/bin/
As per ChromeDriver - Requirements the server expects you to have Chrome installed in the default location for each system:
1 For Linux systems, the ChromeDriver expects /usr/bin/google-chrome to be a symlink to the actual Chrome binary. You can also override the Chrome binary location as follows:
A Windows OS based example:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("start-maximized")
options.binary_location("C:\\path\\to\\chrome.exe")
driver = webdriver.Chrome(executable_path=r'C:\path\to\chromedriver.exe', chrome_options=options)
driver.get('http://google.com/')
Additional Considerations
Upgrade ChromeDriver to current ChromeDriver v2.42 level.
Keep Chrome version between Chrome v68-70 levels. (as per ChromeDriver v2.42 release notes)
Clean your Project Workspace through your IDE and Rebuild your project with required dependencies only.
If your base Web Client version is too old, then uninstall it through Revo Uninstaller and install a recent GA and released version of Web Client.
Execute your #Test.

I found another reason for this behavior, which I don't see listed here. I believe Selenium libraries require Chrome Developer Tools to not be disabled. I was running into this issue and getting this regkey set got me past it.
Key - HKLM\Software\Policies\Google\Chrome
DWORD - DeveloperToolsDisabled
Value - 0

Selenium Chrome Driver Under Linux

Having a bit of a rough time trying to get a webdriver spawning under Linux.
I'm on CentOS 7.3 I've installed Chrome, Chromium and a host of other python libs. I'm running:
chrome_options = webdriver.ChromeOptions()
thechromedriver = webdriver.Chrome(executable_path='/home/skyscraper/chromedriver', chrome_options=chrome_options)
The alternative with a different path runs fine under Windows so the problem is definitely in my port to Linux.
I've ran:
chmod a+x chromedriver
And it runs under root so it should have access to the chromedriver.
If I spawn it through a threadpool with a timeout it hits that timeout every time. Running it normally it eventually throws an error:
Traceback (most recent call last):
File "Skyscrapercomp.py", line 492, in <module>
thechromedriver = webdriver.Chrome(executable_path='/home/skyscraper/chromedriver', chrome_options=chrome_options)
File "/usr/lib/python2.7/site-packages/selenium/webdriver/chrome/webdriver.py", line 69, in __init__
desired_capabilities=desired_capabilities)
File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 92, in __init__
self.start_session(desired_capabilities, browser_profile)
File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 179, in start_session
response = self.execute(Command.NEW_SESSION, capabilities)
File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 236, in execute
self.error_handler.check_response(response)
File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 192, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally
(Driver info: chromedriver=2.27.440175 (9bc1d90b8bfa4dd181fbbf769a5eb5e575574320),platform=Linux 3.10.0-514.2.2.el7.x86_64 x86_64)
My takeaway here was "Chrome failed to start: exited abnormally" which doesn't give me a lot to go on.
I've used pip to install the latest Selenium and Yum to install Chromium and Chrome. Feeling a bit dead in the water not sure what my next step is so I'd really appreciate a prod in the right direction.
/Updates:
I checked in /usr/bin google-chrome is there which is the default location it wants.
Also tried putting the chromedriver into /usr/bin and not giving it a location:
TheBrowser = webdriver.Chrome()
TheBrowser.get("http://www.google.com")
print TheBrowser.page_source.encode('utf-8')
But this gives the same error.

Try to use logging to find the problem:
chrome_options = webdriver.ChromeOptions()
service_log_path = 'chromedriver.log'
service_args = ['--verbose']
thechromedriver = webdriver.Chrome(executable_path='/home/skyscraper/chromedriver', chrome_options=chrome_options, service_args=service_args, service_log_path=service_log_path)
In my case there was an error with display (no X11 on my Ubuntu server). This helped me:
from pyvirtualdisplay import Display
display = Display(visible=0, size=(800, 600))
display.start() # don't forget to .stop() it at the end of program