The following script(s):
from selenium import webdriver
wd = webdriver.Firefox()
or
from selenium import webdriver
wd = webdriver.Chrome()
will very nearly always fail on my Windows 7 machine, producing the following stack trace(s):
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\username\AppData\Local\Programs\Python\Python36-32\lib\site-packages\selenium\webdriver\firefox\webdriver.py", line 152, in __init__
keep_alive=True)
File "C:\Users\username\AppData\Local\Programs\Python\Python36-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 98, in __init__
self.start_session(desired_capabilities, browser_profile)
File "C:\Users\username\AppData\Local\Programs\Python\Python36-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 188, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "C:\Users\username\AppData\Local\Programs\Python\Python36-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 254, in execute
response = self.command_executor.execute(driver_command, params)
File "C:\Users\username\AppData\Local\Programs\Python\Python36-32\lib\site-packages\selenium\webdriver\remote\remote_connection.py", line 464, in execute
return self._request(command_info[0], url, body=data)
File "C:\Users\username\AppData\Local\Programs\Python\Python36-32\lib\site-packages\selenium\webdriver\remote\remote_connection.py", line 488, in _request
resp = self._conn.getresponse()
File "C:\Users\username\AppData\Local\Programs\Python\Python36-32\lib\http\client.py", line 1334, in getresponse
response.begin()
File "C:\Users\username\AppData\Local\Programs\Python\Python36-32\lib\http\client.py", line 300, in begin
version, status, reason = self._read_status()
File "C:\Users\username\AppData\Local\Programs\Python\Python36-32\lib\http\client.py", line 260, in _read_status
line_tmp = self.fp.readline(_MAXLINE + 1)
File "C:\Users\username\AppData\Local\Programs\Python\Python36-32\lib\socket.py", line 586, in readinto
return self._sock.recv_into(b)
socket.timeout: timed out
And the same for Chrome (except ...\webdriver\chrome\webdriver.py, obviously). This behavior is very consistent, although on rare occasions reattempting to open a webdriver within the same Python session will allow Selenium to successfully open the browser, and (so far as I've been able to tell) operate correctly from there on out.
This behavior is not observed on my Linux machine, which is on the same network; any attempt to open a webdriver works correctly.
I'm stumped, all socket timeout issues I've found through Google have been related to attempting to access a webpage, not just creating a new webdriver object. I can provide any additional information required to solve this.
Here is the Answer to your Question -
While you work with Selenium 3.4.3, chromedriver v2.30, geckodriver v0.17.0, Google Chrome 59.0 and Mozilla Firefox 53.0 through Python 3.6.1 you can consider the following options:
Chrome :
To initialize chromedriver you can consider mentioning the absolute path of the chromedriver through executable_path argument and additionally specify/use the chrome_options argument to configure the required browser properties as follows:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("start-maximized")
options.add_argument("disable-infobars")
options.add_argument("--disable-extensions")
driver = webdriver.Chrome(chrome_options=options, executable_path="C:\\Utility\\BrowserDrivers\\chromedriver.exe")
driver.get("https://www.google.co.in")
Firefox :
To initialize geckodriver you can consider mentioning the absolute path of the geckodriver through executable_path argument and additionally specify/use the firefox_binary argument to configure the required browser as follows:
from selenium import webdriver
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
binary = FirefoxBinary('C:\\Program Files\\Mozilla Firefox\\firefox.exe')
caps = DesiredCapabilities().FIREFOX
driver = webdriver.Firefox(firefox_binary=binary, executable_path="C:\\Utility\\BrowserDrivers\\geckodriver.exe")
driver.get('https://stackoverflow.com')
Let me know if this Answers your Question.
Related
This is not a duplicate. I want to achieve this on a headless linux server. No GUI, no browser profile file. I need to set profile preferences in code.
My goal: download an image file generated by a .php + .js script using python selenium firefox on full headless mode.
My issue:
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
from selenium.webdriver.common.by import By
profile = FirefoxProfile()
profile.set_preference("browser.download.folderList", 2)
profile.set_preference("browser.download.manager.showWhenStarting", False)
profile.set_preference("browser.download.dir", "/home/ubuntu")
profile.set_preference("browser.helperApps.neverAsk.saveToDisk", "image/png")
options = Options()
options.headless = True
options.profile = profile
driver = webdriver.Firefox(options=options)
driver.get("https://old.ferramentas.eucreio.org/citacao/")
print(driver.title)
driver.close()
returns:
get_assets.py:6: DeprecationWarning: firefox_profile has been deprecated, please use an Options object
profile = FirefoxProfile()
get_assets.py:14: DeprecationWarning: Setting a profile has been deprecated. Please use the set_preference and install_addons methods
options.profile = profile
Traceback (most recent call last):
File "get_assets.py", line 16, in <module>
driver = webdriver.Firefox(options=options)
File "/home/ubuntu/.local/lib/python3.8/site-packages/selenium/webdriver/firefox/webdriver.py", line 179, in __init__
RemoteWebDriver.__init__(
File "/home/ubuntu/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 268, in __init__
self.start_session(capabilities, browser_profile)
File "/home/ubuntu/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 359, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/home/ubuntu/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 424, in execute
self.error_handler.check_response(response)
File "/home/ubuntu/.local/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: Connection refused (os error 111)
If I sneak a options.set_preference() (instead of using the FirefoxProfile instance) in there it doesn't work either. Also trying to set firefox_profile as webdriver.Firefox() parameter has no different outcome.. just says everything is deprecated.
I'm having such a headache with this.. just feel something is so off here and I can't say what.
I've also exhausted the entire internet looking for a code that shed some light on this and.. just nothing.
The internet page is set to download the image as soon as it loads. And all I want is a headless download 😁
Someone with some more experience could help me please? 🙏
Code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome(executable_path = r"C:\Users\Dell\Desktop\chromedriver.exe")
driver.get("http:/https://stackoverflow.com/")
Traceback (most recent call last):
File "C:\Users\Dell\Desktop\Intro.py", line 3, in
driver = webdriver.Chrome(executable_path = r"C:\Users\Dell\Desktop\chromedriver.exe")
File "C:\Users\Dell\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 76, in init
RemoteWebDriver.init(
File "C:\Users\Dell\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 157, in init
self.start_session(capabilities, browser_profile)
File "C:\Users\Dell\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "C:\Users\Dell\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\Dell\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: failed to write prefs file
This can cause if your C drive has not enough space. So use any tools like Disk Cleaner or CCleaner to free up some space.
Here is my version of the working code.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome(executable_path="C:\Windows\chromedriver.exe")
driver.get("https://stackoverflow.com/")
Try adding chromedriver to C:\Windows.
Check both chrome and chromedriver having the same version.
This question already has answers here:
How can I reconnect to the browser opened by webdriver with selenium?
(4 answers)
Closed 2 years ago.
I was following this tutorial (link below), but an error is happening that I don't know how I can solve it.
https://medium.com/#harith.sankalpa/connect-selenium-driver-to-an-existing-chrome-browser-instance-41435b67affd
I am trying to use the browser that is open to perform a search, because it is already logged into the account that I need.
I'm using chrome --remote-debugging-port=1024 to open chrome, after I execute this code below.
I found some solutions, but none solved my problem, either because the solution was in Java and I didn't understand or I didn't know how to rewrite in python.
Code:
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.chrome.options import Options as ChromeOptions
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
if __name__ == '__main__':
options = ChromeOptions()
options.add_argument('start-maximized')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
options.add_argument('--remote-debugging-port=1024')
options.add_argument('--disable-setuid-sandbox')
options.add_experimental_option("debuggerAddress", "localhost:1024")
options.binary_location = "C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe"
d = DesiredCapabilities.CHROME
d["loggingPrefs"] = {"browser": "ALL"}
driver = webdriver.Chrome(
executable_path=ChromeDriverManager().install(),
options=options,
desired_capabilities=d
)
driver.get("https://google.com.br")
Error:
Traceback (most recent call last):
File "C:/Users/danit/Desktop/project/main.py", line 22, in <module>
desired_capabilities=d
File "C:\Users\danit\Desktop\project\venv\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 81, in __init__
desired_capabilities=desired_capabilities)
File "C:\Users\danit\Desktop\project\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 157, in __init__
self.start_session(capabilities, browser_profile)
File "C:\Users\danit\Desktop\project\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "C:\Users\danit\Desktop\project\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\danit\Desktop\project\venv\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot connect to chrome at localhost:1024
from chrome not reachable
Thanks a lot.
The error says chrome is not reachable. Most probably that instance has been deleted
I am showing a simple method to save session data( all cookies will be saved ) and then load selenium to load from that instance.
Look at the following example
from selenium.webdriver.chrome.options import Options
from selenium import webdriver
session = "mySession"
chrome_driver_path = '/home/aahnik/Downloads/apps/chromedriver'
whatsapp_web_url = "https://web.whatsapp.com/"
chrome_options = Options()
chrome_options.add_argument(f'--user-data-dir={session}')
driver = webdriver.Chrome(
options=chrome_options, executable_path=chrome_driver_path)
driver.get(whatsapp_web_url)
Now execute this code. WhatsApp web will open. Login by scanning the QR Code.
Now close the window, and then terminate the program.
Now will see a folder named mySession in your current user directory.
Execute this code again.
This time you will find that you are already logged into WhatsApp.
Hope this helped.
This question already has answers here:
How to open a Chrome Profile through Python
(2 answers)
How to open a Chrome Profile through --user-data-dir argument of Selenium
(3 answers)
How to use existing login token for telegram web using selenium webdriver
(2 answers)
Closed 2 years ago.
I'm trying use Selenium to open chrome using a specific profile. I located the profile location in chrome://version and used the following code:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument(
"user-data-dir=../../Library/Application Support/Google/Chrome/Profile 2/")
driver = webdriver.Chrome(
executable_path='./chromedriver', options=options)
driver.get("https://google.com")
But instead, it throws this error:
Traceback (most recent call last):
File "select_files.py", line 10, in <module>
driver = webdriver.Chrome(
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/chrome/webdriver.py", line 76, in __init__
RemoteWebDriver.__init__(
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
self.start_session(capabilities, browser_profile)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot parse internal JSON template: Line: 1, column: 1, Unexpected token.
I tried using without the /profile 2/ but it still comes up with this error
I am using Python 3, Selenium 1.25.9, Chrome Version 85.0.4183.83, Chrome Driver MacOS Catalina
I try to run chromedriver via selenium in headless mode.
IMPORTANT
The code runs perfectly fine if I eliminate the following code lines (but is not headless):
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
This is the error I get when I try to implement the headless argument:
Traceback (most recent call last):
File "camel.py", line 83, in <module>
executable_path=executable_path)
File "/home/.local/lib/python3.6/site-packages/selenium/webdriver/chrome/webdriver.py", line 81, in __init__
desired_capabilities=desired_capabilities)
File "/home/.local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
self.start_session(capabilities, browser_profile)
File "/home/.local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/home/.local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/home/.local/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: failed to wait for extension background page to load: chrome-extension://jkompbllimaoekaogchhkmkdogpkhojg/_generated_background_page.html
from unknown error: page could not be found: chrome-extension://jkompbllimaoekaogchhkmkdogpkhojg/_generated_background_page.html
This are the lines 81, 82 and 83
chrome_options.add_extension(extension_path)
driver = webdriver.Chrome(options=chrome_options,
executable_path=executable_path)
This is the code (the crhomedriver execution parts):
from selenium import webdriver
from selenium.webdriver import Chrome
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.expected_conditions import presence_of_element_located
import os
BASE_DIR = os.path.dirname(os.path.abspath(__file__))
chrome_options = Options()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
log_path = os.path.join(BASE_DIR, 'cronJobChromeDriver.log')
executable_path = os.path.join(BASE_DIR, 'chromedriver_linux64/chromedriver')
extension_path = os.path.join(
BASE_DIR, 'chromedriver_linux64/extension_2_8_9_0.crx')
print('executable_path', executable_path)
The bottom line is, No, google-chrome-headless doesn't supports extensions.
In the one of his comment, alexclarke#chromium.org mentioned:
I realize a lot of folks would like to use extensions with headless but unfortunately that's a large project which we have /no plans to do/. The problem is Headless Chromium is a content embedder which means it doesn't have access to anything from other content embedders such as chrome and unfortunately extensions are a chrome feature.
In another comment he further added, if you're using Selenium through DevTools you can build a proxy. Next you can filter URLs and modify headers via Network.setRequestInterception and Network.continueInterceptedRequest.
Reference
You can find a relevant detailed discussion in:
Is it possible to run Google Chrome in headless mode with extensions?
this is now possible by modifying the following flag:
chrome_options.add_argument('--headless=chrome')
I tested it successfully.
I found it here: https://bugs.chromium.org/p/chromium/issues/detail?id=706008#c5
Chrome does not support headless, but apparently Firefox does.
Some relevant discussions:
https://sqa.stackexchange.com/questions/32611/selenium-chromedriver-headless-chrome-failed-to-wait-for-extension-backgro
Is it possible to run Google Chrome in headless mode with extensions?