We Keep Coding

Firefox fails to open url when using webdriver python API

I am new to python and trying to build a selenium code to open a website via firefox.
I am using a debian stretch machine for the tests. The versions of the tools are given below :
geckodriver 0.24.0 ( 2019-01-28) , Python 2.7.13 , Mozilla Firefox 52.7.3 , selenium (3.141.0)
I see that the firefox window opens( even though in headless). But it doesnt proceed further with opening the website. The firefox instance waits for sometime after opening and then stops, the script gives error like
WebHandle.open_application("Firefox" , "http://www.google.com")
File "/home/yyyyy/yyyyy/yyyyy/yyyyy/script.py", line 49, in open_application
driver = webdriver.Firefox(firefox_options = options, executable_path='/usr/local/bin/geckodriver')
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/webdriver.py", line 174, in init
keep_alive=True)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 157, in init
self.start_session(capabilities, browser_profile)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: connection refused
Code :
from selenium import webdriver
from selenium.webdriver import Firefox
from selenium.webdriver.firefox.options import Options as FirefoxOptions
options = FirefoxOptions()
options.add_argument("--headless")
print (" * Opening firefox session")
driver = webdriver.Firefox(firefox_options = options, executable_path='/usr/local/bin/geckodriver')
driver.get("https://www.google.com")
driver.maximize_window()
print(driver.title)
While looking into the geckodriver.log , I see the following errors
1624193615192 mozrunner::runner INFO Running command: "/usr/bin/firefox" "-marionette" "--headless" "-foreground" "-no-remote" "-profile" "/tmp/rust_mozprofile.Wwi0327B9dAL"
1624193616996 Marionette INFO Listening on port 2828
[Child 28357] ###!!! ABORT: Aborting on channel error.: file /build/firefox-esr-52.7.3esr/ipc/glue/MessageChannel.cpp, line 2152
[Child 28357] ###!!! ABORT: Aborting on channel error.: file /build/firefox-esr-52.7.3esr/ipc/glue/MessageChannel.cpp, line 2152
###!!! [Child][MessageChannel] Error: (msgtype=0x3E0003,name=PCompositable::Msg_Destroy) Channel error: cannot send/recv
###!!! [Child][MessageChannel] Error: (msgtype=0x3E0003,name=PCompositable::Msg_Destroy) Channel error: cannot send/recv
Thanks already for any help in fixing the issue

Try placing the geckodriver.exe file in a separate folder & drive and updating the path in executable_path.
eg:
driver = webdriver.Firefox(firefox_options = options, executable_path=r"D:/Python/drivers/geckodriver.exe")

selenium.common.exceptions.WebDriverException:unknown error:cannot connect to chrome at 127.0.0.1:9222 from unknownerror:unable to discover open pages

I managed to connect selenium driver to an already opened chrome session with the following method in python
def iniciar_google_previa():
os.system('google-chrome --no-sandbox --remote-debugging-port=9222 --user-data-dir="/home/natanael/.config/google-chrome/Default" &')
time.sleep(2)
options1 = Options()
options1.add_experimental_option("debuggerAddress", "127.0.0.1:9222")
chrome_driver = "/usr/local/bin/chromedriver"
driver = webdriver.Chrome(executable_path=ChromeDriverManager().install(), options=options1)
return driver
And it worked really good, then I can open any site with driver.get("site_url") to start scraping.
I wanted to try the same with docker containers. Installed all the requierements for the dockerfile, built it and then tried to run it but I get the following error
File "final1.py", line 104, in <module>
iniciar_ciclo()
File "final1.py", line 21, in iniciar_ciclo
driver = iniciar_google_previa()
File "final1.py", line 67, in iniciar_google_previa
driver = webdriver.Chrome(executable_path=ChromeDriverManager().install(), options=options1)
File "/opt/app-root/lib/python3.6/site-packages/selenium/webdriver/chrome/webdriver.py", line 81, in __init__
desired_capabilities=desired_capabilities)
File "/opt/app-root/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
self.start_session(capabilities, browser_profile)
File "/opt/app-root/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/opt/app-root/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/opt/app-root/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot connect to chrome at 127.0.0.1:9222
from unknown error: unable to discover open pages
I thought it was because it was trying to run a GUI so I added
--headless
google-chrome --no-sandbox --headless --remote-debugging-port=9222 --user-data-dir="/home/natanael/.config/google-chrome/Default"
but got the same error.
already tried installing
xvfb
and then
xvfb-run google-chrome --no-sandbox --remote-debugging-port=9222 --user-data-dir="/home/natanael/.config/google-chrome/Default"
which gave the output
DevTools listening on ws://127.0.0.1:9222/devtools/browser/52bbc92b-e096-4897-8661-233ee573edaf
apparentley it executes the chrome session but is still unable to connect to it and I don't know what else to try

Error on Connect Selenium Driver to an Existing Chrome Browser Instance [duplicate]

This question already has answers here:
How can I reconnect to the browser opened by webdriver with selenium?
(4 answers)
Closed 2 years ago.
I was following this tutorial (link below), but an error is happening that I don't know how I can solve it.
https://medium.com/#harith.sankalpa/connect-selenium-driver-to-an-existing-chrome-browser-instance-41435b67affd
I am trying to use the browser that is open to perform a search, because it is already logged into the account that I need.
I'm using chrome --remote-debugging-port=1024 to open chrome, after I execute this code below.
I found some solutions, but none solved my problem, either because the solution was in Java and I didn't understand or I didn't know how to rewrite in python.
Code:
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.chrome.options import Options as ChromeOptions
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
if __name__ == '__main__':
options = ChromeOptions()
options.add_argument('start-maximized')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
options.add_argument('--remote-debugging-port=1024')
options.add_argument('--disable-setuid-sandbox')
options.add_experimental_option("debuggerAddress", "localhost:1024")
options.binary_location = "C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe"
d = DesiredCapabilities.CHROME
d["loggingPrefs"] = {"browser": "ALL"}
driver = webdriver.Chrome(
executable_path=ChromeDriverManager().install(),
options=options,
desired_capabilities=d
)
driver.get("https://google.com.br")
Error:
Traceback (most recent call last):
File "C:/Users/danit/Desktop/project/main.py", line 22, in <module>
desired_capabilities=d
File "C:\Users\danit\Desktop\project\venv\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 81, in __init__
desired_capabilities=desired_capabilities)
File "C:\Users\danit\Desktop\project\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 157, in __init__
self.start_session(capabilities, browser_profile)
File "C:\Users\danit\Desktop\project\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "C:\Users\danit\Desktop\project\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\danit\Desktop\project\venv\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot connect to chrome at localhost:1024
from chrome not reachable
Thanks a lot.

The error says chrome is not reachable. Most probably that instance has been deleted
I am showing a simple method to save session data( all cookies will be saved ) and then load selenium to load from that instance.
Look at the following example
from selenium.webdriver.chrome.options import Options
from selenium import webdriver
session = "mySession"
chrome_driver_path = '/home/aahnik/Downloads/apps/chromedriver'
whatsapp_web_url = "https://web.whatsapp.com/"
chrome_options = Options()
chrome_options.add_argument(f'--user-data-dir={session}')
driver = webdriver.Chrome(
options=chrome_options, executable_path=chrome_driver_path)
driver.get(whatsapp_web_url)
Now execute this code. WhatsApp web will open. Login by scanning the QR Code.
Now close the window, and then terminate the program.
Now will see a folder named mySession in your current user directory.
Execute this code again.
This time you will find that you are already logged into WhatsApp.
Hope this helped.

selenium.common.exceptions.InvalidSessionIdException using GeckoDriver Selenium Firefox in headless mode through Python

I am tryin to use python an selenium to automate some cases on firefox headless mode. I think, I did every settings that I need but still webdrvier is not initialized.
My problem is getting the exception approximately 30-60 sec after executing my code and exception message is not clear actually message section is empty. As you can see on the geckodriver.log firefox running in headless mode.
I use Firefox 56.0.1, python 3.6, selenium 3.141.0. Here is my code and logs;
Code:
import os
import time
import selenium
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
options = Options()
options.set_headless(True)
dir = "C:\\Python36\\Lib\\site-packages\\selenium\\webdriver\\firefox"
ff_driver_path = dir + "\\geckodriver.exe"
firefox_capabilities = DesiredCapabilities().FIREFOX
firefox_capabilities['marionette'] = True
profile = webdriver.FirefoxProfile()
binary = FirefoxBinary('C:\\Program Files (x86)\\Mozilla Firefox\\firefox.exe')
driver = webdriver.Firefox(desired_capabilities=firefox_capabilities, firefox_profile=profile, firefox_binary=binary, firefox_options=options, executable_path=ff_driver_path)
driver.get("https://ipaddress/")
Logs:
File "<string>", line 20, in <module>
File "C:\Python36\lib\site-packages\selenium\webdriver\firefox\webdriver.py", line 174, in __init__
keep_alive=True)
File "C:\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 157, in __init__
self.start_session(capabilities, browser_profile)
File "C:\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "C:\Python36\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Python36\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidSessionIdException: Message:
geckodriver logs:
1544600615819 mozrunner::runner INFO Running command: "C:\\Program Files (x86)\\Mozilla Firefox\\firefox.exe" "-marionette" "-headless" "-foreground" "-no-remote" "-profile" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\2\\rust_mozprofile.SDw8B2IEG5cE"
*** You are running in headless mode.
1544600616120 addons.xpi WARN Error parsing extensions state: [Exception... "Component returned failure code: 0x80520012 (NS_ERROR_FILE_NOT_FOUND) [amIAddonManagerStartup.readStartupData]" nsresult: "0x80520012 (NS_ERROR_FILE_NOT_FOUND)" location: "JS frame :: resource://gre/modules/addons/XPIProvider.jsm :: loadExtensionState :: line 1554" data: no] Stack trace: loadExtensionState()#resource://gre/modules/addons/XPIProvider.jsm:1554 < getInstallState()#resource://gre/modules/addons/XPIProvider.jsm:1589 < checkForChanges()#resource://gre/modules/addons/XPIProvider.jsm:3109 < startup()#resource://gre/modules/addons/XPIProvider.jsm:2188 < callProvider()#resource://gre/modules/AddonManager.jsm:269 < _startProvider()#resource://gre/modules/AddonManager.jsm:739 < startup()#resource://gre/modules/AddonManager.jsm:906 < startup()#resource://gre/modules/AddonManager.jsm:3090 < observe()#jar:file:///C:/Program%20Files%20(x86)/Mozilla%20Firefox/omni.ja!/components/addonManager.js:65
1544600616661 Marionette INFO Enabled via --marionette
[Child 2656] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 346
[Child 2656] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 346
1544600618075 Marionette INFO Listening on port 64093
JavaScript error: jar:file:///C:/Program%20Files%20(x86)/Mozilla%20Firefox/omni.ja!/components/captivedetect.js, line 352: TypeError: this._runningRequest is null
JavaScript error: jar:file:///C:/Program%20Files%20(x86)/Mozilla%20Firefox/omni.ja!/components/captivedetect.js, line 352: TypeError: this._runningRequest is null
JavaScript error: jar:file:///C:/Program%20Files%20(x86)/Mozilla%20Firefox/omni.ja!/components/captivedetect.js, line 352: TypeError: this._runningRequest is null
JavaScript error: jar:file:///C:/Program%20Files%20(x86)/Mozilla%20Firefox/omni.ja!/components/captivedetect.js, line 352: TypeError: this._runningRequest is null
JavaScript error: jar:file:///C:/Program%20Files%20(x86)/Mozilla%20Firefox/omni.ja!/components/captivedetect.js, line 352: TypeError: this._runningRequest is null
JavaScript error: jar:file:///C:/Program%20Files%20(x86)/Mozilla%20Firefox/omni.ja!/components/captivedetect.js, line 352: TypeError: this._runningRequest is null
Any ideas on what I am doing wrong? Thank you!

This error message...
selenium.common.exceptions.InvalidSessionIdException: Message:
...implies that the GeckoDriver was unable to initiate/spawn a new WebBrowsing Session i.e. Firefox Browser session.
As per the log messages geckodriver version number is no longer reflected on startup, so you must be using:
GeckoDriver version => 0.22.0 (2018-09-15)
But you are using:
Selenium 3.141.0
Firefox 56.0.1
So, your main issue is the incompatibility between the version of the GeckoDriver, Selenium and Firefox binaries you are using.
Solution
If you are using GeckoDriver-Selenium-Firefox combo, you need to follow the following compatibility chart for the Supported platforms:

This message also occured when switching virtual desktops in MS Windows. You asked:
Where do I find more informations about those combos?
You could use a tool that automatically downloads the driver for you.
from webdriver_manager.chrome import ChromeDriverManager
from seleniumrequests import Chrome
driver = Chrome(ChromeDriverManager().install())

Selenium Python selenium.common.exceptions.WebDriverException: Message: connection refused using geckodriver and firefox

I'm having issues running my automation test scripts. When I run my script, a browser will appear but it will not type the URL and waits for 10 seconds until it throws an exception. Is there any solutions I can use so then I can get my automation test scripts to work?
Geckodriver.log:
1523997052492 geckodriver INFO geckodriver 0.20.1
1523997052531 geckodriver INFO Listening on 127.0.0.1:37807
1523997052592 mozrunner::runner INFO Running command: "/usr/bin/firefox/firefox" "-marionette" "--headless" "-profile" "/tmp/rust_mozprofile.PU1cngaAJ5Tg"
1523997054831 Marionette INFO Listening on port 2828
Stack Traces:
Error
Traceback (most recent call last):
File
"/home/kavin/PycharmProjects/untitled/Testing/purchaseAmazonItems.py", line 13, in setUp
self.driver = webdriver.Firefox(firefox_binary=binary, firefox_options=opts)
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/firefox/webdriver.py", line 162, in __init__
keep_alive=True)
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 154, in __init__
self.start_session(desired_capabilities, browser_profile)
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 243, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 312, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: connection refused
Code:
def setUp(self):
binary = FirefoxBinary('/usr/bin/firefox/firefox')
opts = FirefoxOptions()
opts.add_argument("--headless")
self.driver = webdriver.Firefox(firefox_binary=binary, firefox_options=opts)
driver = self.driver
driver.get('https://www.amazon.com/')
Specs:
Ubuntu 16.04 geckodriver 0.20.1 firefox
59.0.2+build1-0ubuntu0.16.04.3 Python 3.6 Pycharm 2016.3 Selenium 3.11.0

In absence of the error stack trace configuration issues are pretty hard to debug. Having said that I don't see any major issues in your code block. You may require to perform some additional steps as follows :
Pass the Key executable_path along with the Value referring to the absolute path of the GeckoDriver as follows :
def setUp(self):
binary = FirefoxBinary('/usr/bin/firefox/firefox')
opts = FirefoxOptions()
opts.add_argument("--headless")
self.driver = webdriver.Firefox(firefox_binary=binary, firefox_options=opts, executable_path='/path/to/geckodriver')
driver = self.driver
driver.get('https://www.amazon.com/')
Clean your Project Workspace through your IDE and Rebuild your project with required dependencies only.
Use CCleaner tool to wipe off all the OS chores before and after the execution of your Test Suite.
If your base Web Client version is too old, then uninstall it through Revo Uninstaller and install a recent GA and released version of Web Client.
Take a System Reboot.
Execute your #Test.
Always invoke driver.quit() within tearDown(){} method to close & destroy the WebDriver and Web Client instances gracefully.
Alternative
As an alternative you can also try to use the set_headless(headless=boolean_value) as follows :
def setUp(self):
binary = FirefoxBinary('/usr/bin/firefox/firefox')
opts = FirefoxOptions()
opts.set_headless(headless=True)
self.driver = webdriver.Firefox(firefox_binary=binary, firefox_options=opts, executable_path='/path/to/geckodriver')
driver = self.driver
driver.get('https://www.amazon.com/')
Here you can find a detailed discussion on How to make firefox headless programatically in Selenium with python?

These two commands both start a webdriver on the same port. The second one causes the error because the port is already in use:
self.driver = webdriver.Firefox(firefox_binary=binary)
browser = webdriver.Firefox(firefox_options=opts)
To correct this, set the options before initializing the driver (in the first command).
self.driver = webdriver.Firefox(firefox_binary=binary, firefox_options=opts)