We Keep Coding

selenium.common.exceptions.WebDriverException:unknown error:cannot connect to chrome at 127.0.0.1:9222 from unknownerror:unable to discover open pages

I managed to connect selenium driver to an already opened chrome session with the following method in python
def iniciar_google_previa():
os.system('google-chrome --no-sandbox --remote-debugging-port=9222 --user-data-dir="/home/natanael/.config/google-chrome/Default" &')
time.sleep(2)
options1 = Options()
options1.add_experimental_option("debuggerAddress", "127.0.0.1:9222")
chrome_driver = "/usr/local/bin/chromedriver"
driver = webdriver.Chrome(executable_path=ChromeDriverManager().install(), options=options1)
return driver
And it worked really good, then I can open any site with driver.get("site_url") to start scraping.
I wanted to try the same with docker containers. Installed all the requierements for the dockerfile, built it and then tried to run it but I get the following error
File "final1.py", line 104, in <module>
iniciar_ciclo()
File "final1.py", line 21, in iniciar_ciclo
driver = iniciar_google_previa()
File "final1.py", line 67, in iniciar_google_previa
driver = webdriver.Chrome(executable_path=ChromeDriverManager().install(), options=options1)
File "/opt/app-root/lib/python3.6/site-packages/selenium/webdriver/chrome/webdriver.py", line 81, in __init__
desired_capabilities=desired_capabilities)
File "/opt/app-root/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
self.start_session(capabilities, browser_profile)
File "/opt/app-root/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/opt/app-root/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/opt/app-root/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot connect to chrome at 127.0.0.1:9222
from unknown error: unable to discover open pages
I thought it was because it was trying to run a GUI so I added
--headless
google-chrome --no-sandbox --headless --remote-debugging-port=9222 --user-data-dir="/home/natanael/.config/google-chrome/Default"
but got the same error.
already tried installing
xvfb
and then
xvfb-run google-chrome --no-sandbox --remote-debugging-port=9222 --user-data-dir="/home/natanael/.config/google-chrome/Default"
which gave the output
DevTools listening on ws://127.0.0.1:9222/devtools/browser/52bbc92b-e096-4897-8661-233ee573edaf
apparentley it executes the chrome session but is still unable to connect to it and I don't know what else to try

Issue with browser = Browser('firefox') part of my code

I am running updated versions of my all modules within my python. I am using conda-forge and spyder 3.7.
I am just attempting to open a webpage using some basic code and I get stuck there.
I have searcher just about everywhere on the site, but I am not sure what I should do since I am quite new to this.
I have updated the modules I am using and continued after every update but it was to no avail.
Here is my code:
from splinter import Browser
browser = Browser('firefox')
browser.visit('https://google.com')
Previously written as:
from splinter import Browser
browser = Browser()
browser.visit('https://google.com')
Same issue with both.
Here is the error:
Traceback (most recent call last):
File "<ipython-input-12-71643798329a>", line 2, in <module>
browser = Browser('firefox')
File "build/bdist.macosx-10.5-x86_64/egg/splinter/browser.py", line 53, in Browser
return driver(*args, **kwargs)
File "build/bdist.macosx-10.5-x86_64/egg/splinter/driver/webdriver/firefox.py", line 33, in __init__
self.driver = Firefox(firefox_profile)
File "C:\Users\lkerzabi\AppData\Local\Continuum\anaconda3\lib\site-packages\selenium\webdriver\firefox\webdriver.py", line 174, in __init__
keep_alive=True)
File "C:\Users\lkerzabi\AppData\Local\Continuum\anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 157, in __init__
self.start_session(capabilities, browser_profile)
File "C:\Users\lkerzabi\AppData\Local\Continuum\anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "C:\Users\lkerzabi\AppData\Local\Continuum\anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\lkerzabi\AppData\Local\Continuum\anaconda3\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
SessionNotCreatedException: Unable to find a matching set of capabilities
I expect to be able to open a webpage, but it just gives me this error.

Have you checked this issue from Github Selenium 3.4.0 Unable to find matching capabilities ?
Particularly those two answers:
First option
Second option
Seems like you are experiencing the same issue

Selenium [Python Binding, Chrome Driver]: Disable Timeouts Completely

We have a Selenium process that fetches data from our data provider for further processing. The process tries to get all historical data, and since we do not have functionality to reliably filter out data that has been edited recently, we must get full data dumps from time to time to ensure that no records are dropped.
The Selenium process is written as a Python script. It goes to a specific website, logs in, presses a button that triggers report generation, and then downloads the report. The issue is that for full dump reports, the report generation has started to take longer than the default Selenium timeouts.
We use the Google Chrome driver, but would be willing to change the browser/driver if only other drivers have the ability to fully disable timeouts.
Below is a simplified, stripped-down version of the code with just the bare bones needed to appropriately describe this issue. The places where the report times out are marked out. It's waiting for a response from the website after clicking a button. How to disable that, and why what we currently do (can be seen below) is not enough?
from selenium import webdriver
def selectPopUp(driver, main_window_handle, maxWait=100):
"""
Selects a pop-up window. If no pop-up window appears, raises an
Exception.
:param maxWait: how long to wait (in seconds) before raising the error?
"""
popup_window_handle = None
i = 0
while (not popup_window_handle) and (i < maxWait):
for handle in driver.window_handles:
if handle != main_window_handle:
popup_window_handle = handle
break
# Wait a litle and re-iterate.
time.sleep(1)
i += 1
if popup_window_handle != None:
driver.switch_to.window(popup_window_handle)
return popup_window_handle
else:
raise Exception("No pop-up window appeared although the program was expecting one.")
def selectOption(dropdown, option, possibilities):
"""
Validates and selects an option from a drop-down.
:param dropdown: the Selenium reference to the dropdown selection.
:param option: the name of the option we'll select
:param possibilities: the possibilities we allow to choose from --
will raise an Exception
if `option` is not in `possibilities`.
:returns: Selenium selection.
"""
# Select the default Excel option
if option in possibilities:
return dropdown.find_element_by_xpath("//option[#value='" + option + "']")\
.click()
else:
raise Exception("Invalid choice! Use one of the following: \n\n " + \
"\n ".join(possibilities))
chromeOptions = webdriver.ChromeOptions()
prefs = {"download.default_directory" : DOWNLOAD_DIRECTORY}
chromeOptions.add_experimental_option("prefs", prefs)
driver = webdriver.Chrome(executable_path=settings.chromeDriver,
chrome_options=chromeOptions)
# Set an infite timeout time so that the slow Crystal reports
# would stand a chance of not timing out too.
driver.set_page_load_timeout(9999999999)
driver.set_script_timeout(9999999999)
# We go to the relevant website - that part is edited out.
# Once in the website, we select the current window handle
# to be able to come back to it after navigating through pop-ups.
main_window_handle = driver.current_window_handle
reportDropDown = driver.find_element_by_id(REPORT_ELEMENT_ID)
selectedReport = reportDropDown.find_element_by_xpath("//option[#value='" + SELCTED_REPORT_TITLE + "']")
selectedReport.click()
########################################################################
# >>>>>>>> The first place where the process keeps timing out. <<<<<<< #
########################################################################
# Click on the export button to open export pop-up.
exportButton = driver.find_element_by_name(r'crytlViewer$ctl02$ctl00')
exportButton.click()
# Now a pop-up with a download button appears. Select it.
popUpHandle = selectPopUp(driver, main_window_handle)
# Now, select a particular download format.
formatDropDown = driver.find_element_by_id("exportFormatList")
selectedFormat = selectOption(formatDropDown, reportFormat, REPORT_FORMAT_LIST)
# Download the report.
driver.find_element_by_id("submitexport").click()
#########################################################################
# >>>>>>>> The second place where the process keeps timing out. <<<<<<< #
#########################################################################
An representative example of an error message we get would be the following.
Traceback (most recent call last):
File "/home/ubuntu/main/oodle/core/utils.py", line 296, in repeatUntilSuccess
return func()
File "/home/ubuntu/main/oodle/salesforce/data_feed.py", line 277, in <lambda>
cleanUp=True),
File "/home/ubuntu/main/oodle/salesforce/data_feed.py", line 322, in pushReports
'liveEnvironment': self.liveEnvironment}]\
File "/home/ubuntu/main/oodle/core/reporting.py", line 1160, in __getitem__
self.handlers[name].PreparePandas(**paramDict)
File "/home/ubuntu/main/oodle/reports/vienna_salesforce_data_feed/Salesforce_LoanObject_Parsed.py", line 38, in PreparePandas
loan = self.manager.handlers[crystalReport].PreparePandas()
File "/home/ubuntu/main/oodle/core/reporting.py", line 1231, in PreparePandas
return self.TransformRaw(self.GetRaw(fileFrom))
File "/home/ubuntu/main/oodle/core/reporting.py", line 1387, in GetRaw
self.PrepareExcel(fileFrom)
File "/home/ubuntu/main/oodle/core/reporting.py", line 1367, in PrepareExcel
fileTo=fileTo)
File "/home/ubuntu/main/oodle/vienna/crystal.py", line 293, in downloadReport
self.downloadReport_no_error_handling(reportTitle, reportFormat, fileTo)
File "/home/ubuntu/main/oodle/vienna/crystal.py", line 247, in downloadReport_no_error_handling
self.driver.find_element_by_id("submitexport").click()
File "/home/ubuntu/.local/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", line 77, in click
self._execute(Command.CLICK_ELEMENT)
File "/home/ubuntu/.local/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", line 493, in _execute
return self._parent.execute(command, params)
File "/home/ubuntu/.local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 256, in execute
self.error_handler.check_response(response)
File "/home/ubuntu/.local/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
raise exception_class(message, screen, stacktrace)
TimeoutException: Message: timeout
(Session info: chrome=60.0.3112.78)
(Driver info: chromedriver=2.29.461571 (8a88bbe0775e2a23afda0ceaf2ef7ee74e822cc5),platform=Linux 4.4.0-1052-aws x86_64)
After edits suggested by Alexey Dolgopolov, we get the following error message, which is slightly different from the previous one:
Traceback (most recent call last):
File "/home/ubuntu/main/oodle/core/utils.py", line 296, in repeatUntilSuccess
return func()
File "/home/ubuntu/main/oodle/salesforce/data_feed.py", line 277, in <lambda>
cleanUp=True),
File "/home/ubuntu/main/oodle/salesforce/data_feed.py", line 322, in pushReports
'liveEnvironment': self.liveEnvironment}]\
File "/home/ubuntu/main/oodle/core/reporting.py", line 1160, in __getitem__
self.handlers[name].PreparePandas(**paramDict)
File "/home/ubuntu/main/oodle/reports/vienna_salesforce_data_feed/Salesforce_LoanObject_Parsed.py", line 38, in PreparePandas
loan = self.manager.handlers[crystalReport].PreparePandas()
File "/home/ubuntu/main/oodle/core/reporting.py", line 1231, in PreparePandas
return self.TransformRaw(self.GetRaw(fileFrom))
File "/home/ubuntu/main/oodle/core/reporting.py", line 1387, in GetRaw
self.PrepareExcel(fileFrom)
File "/home/ubuntu/main/oodle/core/reporting.py", line 1367, in PrepareExcel
fileTo=fileTo)
File "/home/ubuntu/main/oodle/vienna/crystal.py", line 318, in downloadReport
try:
File "/home/ubuntu/main/oodle/vienna/crystal.py", line 254, in downloadReport_no_error_handling
exportButton.click()
File "/home/ubuntu/.local/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", line 77, in click
self._execute(Command.CLICK_ELEMENT)
File "/home/ubuntu/.local/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", line 494, in _execute
return self._parent.execute(command, params)
File "/home/ubuntu/.local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 236, in execute
self.error_handler.check_response(response)
File "/home/ubuntu/.local/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 192, in check_response
raise exception_class(message, screen, stacktrace)
TimeoutException: Message: timeout
(Session info: chrome=60.0.3112.78)
(Driver info: chromedriver=2.29.461571 (8a88bbe0775e2a23afda0ceaf2ef7ee74e822cc5),platform=Linux 4.10.0-26-generic x86_64)

When you find_element_by_<whatever> an implicitly_wait timeout plays. The default value is 0.
I guess, you don't need to disable timeouts. Try using Explicit Wait. Read about it http://selenium-python.readthedocs.io/waits.html and https://seleniumhq.github.io/selenium/docs/api/py/webdriver_support/selenium.webdriver.support.wait.html#module-selenium.webdriver.support.wait.
And use something like this:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
...
selectedReport.click()
exportButton = WebDriverWait(driver, 9999, 5).until(
EC.visibility_of_element_located(
(By.NAME, r'crytlViewer$ctl02$ctl00')
)

Selenium/PhantomJS: Can not send_keys to input (TypeError - undefined is not a function)

My simple code code:
from selenium import webdriver
from time import sleep
browser = webdriver.PhantomJS()
# Maximized the window to make sure the element will be displayed
browser.maximize_window()
browser.get('https://banhang.shopee.vn/account/signin')
# Delay a few seconds to allow all javascript to be loaded properly
sleep(5)
tag = browser.find_element_by_xpath('/html/body/div[2]/div/div[2]/div/div[4]/div/div/div/div/div[2]/form/div[1]/div[3]/input')
tag.send_keys('1234567890')
The exception being raised:
File "/miniconda2/envs/flecom/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", line 347, in send_keys
self._execute(Command.SEND_KEYS_TO_ELEMENT, {'value': keys_to_typing(value)})
File "/miniconda2/envs/flecom/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", line 494, in _execute
return self._parent.execute(command, params)
File "/miniconda2/envs/flecom/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 236, in execute
self.error_handler.check_response(response)
File "/miniconda2/envs/flecom/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 163, in check_response
raise exception_class(value)
WebDriverException: Message: TypeError - undefined is not a function (evaluating '_getTagName(currWindow).toLowerCase()')
If I change webdriver.PhantomJS() to webdriver.Chrome(), it worked without any exceptions. What did I do wrong?
Some other info:
PhantomJS v2.1
Selenium 3.0.2
Python 2.7.13
Ubuntu 16.04 64bit
PS: I also tried to click on the element before send_keys, but it did not make any differences.

Trying reverting to PhantomJS 1.9.8
I had this same error using PhantomJS 2.1.1 and reverting solved it for me.

Python click Js link with selenium

Iam tested webscraping the page:http://www.guiadosquadrinhos.com/todas-capas-disponiveis
I need navigate in pagination geting info pages favorite.
I need click link javascript next page:
javascript:__doPostBack('ctl00$MainContent$lstProfileView$dataPagerNumeric2$ctl02$ctl00')
from selenium import webdriver
import time
driver = webdriver.PhantomJS(executable_path='C:\Python27\Tools\phantomjs\phantomjs.exe')
driver.get("http://www.guiadosquadrinhos.com/todas-capas-disponiveis")
#print(driver.find_elements_by_class_name("numero_capinha")[0].text)
#driver.find_elements_by_class_name("next_last")[0].click()
#time.sleep(5)
print(driver.find_elements_by_class_name("numero_capinha")[0].text)
driver.find_elements_by_class_name("next_last")[0].click()
print(driver.find_elements_by_class_name("numero_capinha")[0].text)
My code return:
sobreontem - Independente
Traceback (most recent call last): File "teste_selenium.py", line 10, in
driver.find_elements_by_class_name("next_last")[0].click() File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webelement.py",
line 74, in click
self._execute(Command.CLICK_ELEMENT) File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webelement.py",
line 453, in _execute
return self._parent.execute(command, params) File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py",
line 201, in execute
self.error_handler.check_response(response) File "C:\Python27\lib\site-packages\selenium\webdriver\remote\errorhandler.py",
line 181, in check_response
raise exception_class(message, screen, stacktrace) selenium.common.exceptions.ElementNotVisibleException: Message:
{"errorMessage":"Element is not currently visible and may not be
manipulated","request":{"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"81","Content-Type":"application/json;charset=UTF-8","Host":"127.0.0.1:63160","User-Agent":"Python-urllib/2.7"},"httpVersion":"1.1","method":"POST","post":"{\"sessionId\":
\"5d3cfdc0-5d3b-11e5-b784-67706273a0bb\", \"id\":
\":wdc:1442494581220\"}","url":"/click","urlParsed":{"anchor":"","query":"","file":"click","directory":"/","path":"/click","relative":"/click","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/click","queryKey":{},"chunks":["click"]},"urlOriginal":"/session/5d3cfdc0-5d3b-11e5-b784-67706273a0bb/element/%3Awdc%3A1442494581220/click"}}
Screenshot: available via screen
Whats is problem ?

Since, you are doing the following -
driver.find_elements_by_class_name("next_last")[0].click()
If you look in the source, there are multiple elements with this class name and the first one is disabled, because that is for the previous button and you are on first page.