I need to open chromium or chrome with selenium in Ubuntu 22 deployed in Digital Ocean VPS droplet. The issue is it doesnt open a chrome tab, the chromium and chromedriver are same version.This works fine in my localhost, with ubuntu 22.However, I was not able to run it when deployed in vps
options = webdriver.ChromeOptions()
options.add_extension(pluginfile)
driver = webdriver.Chrome('chromedriver', options=options)
I tried several things including the suggestions here Selenium: WebDriverException:Chrome failed to start: crashed as google-chrome is no longer running so ChromeDriver is assuming that Chrome has crashed
The error is
(unknown error: DevToolsActivePort file doesn't exist)
(The process started from chrome location /snap/bin/chromium is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
As I already mentioned, this will work perfectly fine in localhost with same OS and chromedriver and chromium version.
Any idea how to make it work in a VPS?
After several attempt, I finally found the reason
VPS doesn't allow selenium to open the browser.It only works in headless mode, which doesn't fit my requirements since I have plugins and headless mode will not run with plugins
Related
I have confirmed that my code works on Windows and was working on Arch a few months back (deleted my Arch VM, so can't test right now). My original code selects data and downloads a file from a site that requires login credentials, so cant paste code referencing those links. Code worked fine on other OSes, but I cant get it to work in my Ubuntu VM that I just created. I tested on the python.org website to download the latest python .tgz file and having the same results, works on Windows, but not on the VM. Once the link is clicked, it begins to download the file. After the download begins, Chrome closes within a second or so and kills the download leaving it partially downloaded and unusable. Very small files seem to complete downloading before Chrome exits, but not larger ones. No errors are shown and the code completes successfully in VS Code.
I also tested having the Chrome Driver open the website and I would manually click the link. Chrome would shutdown and partially download the file the same as having my code perform the actions.
Has anyone else experienced this before? Is it an issue with Ubuntu?
Here is my current software/packages installed -
selenium 4.1.0,
webdriver_manager 3.5.2,
VS Code 1.64.2,
Python 3.10.2,
Ubuntu 20.04.3 LTS on VMWare
Here is the code
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.chrome.service import Service
py_website = 'https://www.python.org/downloads/release/python-3102/'
options = webdriver.ChromeOptions()
options.add_experimental_option("detach", True)
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), chrome_options=options)
driver.get(py_website)
driver.find_element_by_xpath("//a[#href='https://www.python.org/ftp/python/3.10.2/Python-3.10.2.tgz']").click()
Edit 2/16/2022 12:50pm CST
Tested on a new Mint VM and had to add the following code for Chrome to launch. Seems to be working on Mint, but Ubuntu still having the same issue with the added arguments.
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
With out this code, I received the following error on Mint.
WebDriverException: Message: unknown error: Chrome failed to start: crashed.
(unknown error: DevToolsActivePort file doesn't exist)
(The process started from chrome location /opt/google/chrome/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Edit 2/16/2022 21:30pm CST
Tried using PyCharm and it is working flawlessly. Must be something with my settings in VS Code.
I want to use Selenium to load a PDF page on an Amazon EC2 instance without using headless mode, since I want to load PDFs. Due to issues described here (https://github.com/puppeteer/puppeteer/issues/1872), headless Chrome won't load PDFs, but I am having general trouble getting Selenium Python to run on Chrome without adding the "headless" option (which looks something like this):
options = Options()
options.headless = True # I want to change this to False
driver = webdriver.Chrome(options=options)
If headless is set to False, it simply crashes and says something along the lines of
(unknown error: DevToolsActivePort file doesn't exist)
(The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Here is a guide that sets up Selenium on Amazon EC2, but it uses headless as well: https://praneeth-kandula.medium.com/running-chromedriver-and-selenium-in-python-on-an-aws-ec2-instance-2fb4ad633bb5
Any tips/advice?
I am trying to open the website in selenium python but it is showing blank page. but when i open that website in normal google chrome it is working .
here is the code i am writing to open the website.
from selenium import webdriver
browser = webdriver.Chrome()
browser.get("https://shop.coles.com.au/a/wentworth-point/home")
Error I am Getting in chrome console Failed to load resource: the server responded with a status of 429 () and Failed to load resource: net::ERR_UNKNOWN_URL_SCHEME –
You must specify the chromedriver path. Something like this:
https://seleniumbyexamples.github.io/navget
Verify that version of chrome driver is same of chrome browser installed on your machine
and make sure that path of chrome driver is set in your PATH variable.
http://chromedriver.chromium.org/downloads
The error code 429 that you are getting means too many requests are sent in a given amount time. Just try to clean your build history and run it again. Regarding the version of Chrome, if that was to be the issue the browser must have not opened in the first place and you would have got other exception like SessionNotCreatedException: Message: session not created: This version of ChromeDriver only supports Chrome version 83
Just to be sure you mat check for the version of chrome you are using and the driver downloaded in the system, if it is not appropriate download the latest chrome driver from web and provide in the executable path to chrome or else add it in environment variables of your system.
So I have made a program on one computer using selenium and that worked, Now using it in another computer I get this error:
selenium.common.exceptions.WebDriverException: Message: Can not connect to the Service chromedriver
Now this same issue was mention in:
Selenium python: Can not connect to the Service %s" % self.path
Selenium python: Can not connect to the Service %s" % self.path
Selenium and Python3 ChromeDriver raises Message: Can not connect to the Service chromedriver
however the solutions mentioned didnt work.
I am using chrome version 79 and have installed chromedriver 79, I tested writing chromedriver in command line and that works which means path is configured right, I have made sure 127.0.0.1 localhost is also in etc/hosts
Below is my code which works on my computer (so i doubt its an issue with the code):
chrome_options = Options()
chrome_options.add_argument("--headless")
with webdriver.Chrome(chrome_options=chrome_options) as driver:
driver.set_window_size(800, 460) # takes two arguments, width and height of the browser and it has to be called before using get()
driver.execute_script("document.body.style.zoom='150%'")
driver.get("file:\\"+url) # takes one argument, which is the url of the website you want to open
driver.find_element_by_tag_name('body').screenshot(output) # avoids scrollbar
In the last question I also tried this modification:
chrome_options = Options()
chrome_options.add_argument("--headless")
with webdriver.Chrome("C:\\chromedriver.exe",chrome_options=chrome_options) as driver:
driver.set_window_size(800, 460) # takes two arguments, width and height of the browser and it has to be called before using get()
driver.execute_script("document.body.style.zoom='150%'")
driver.get("file:\\"+url) # takes one argument, which is the url of the website you want to open
driver.find_element_by_tag_name('body').screenshot(output) # avoids scrollbar
which would instead give me this almost identical error message:
selenium.common.exceptions.WebDriverException: Message: Can not connect to the Service C:\chromedriver.exe
I am unsure where the issue could lie.
I am using windows by the way.
EDIT: Things I tried and didn't work:
1- running everything as both admin and normal
2- re-installing chrome
3- using the beta-chroma 80 and webdriver 80
4- using normal chrome 79 and webdriver 79
5- Having both the script and the driver in the same directory (while using a
correct path)
6- Having an external path and have it setup as needed
7- Using it in the PATH folder.
8- Adding "127.0.0.1 localhost" to etc/hosts
9- Running service test
I have ensured in every test that everything was in it's correct placement, I have ran a reboot before every new test, and they always give me the same error, which also happens to occur if I had an incorrect path as well, but once I ran the code for a service and it gave me a different error as I had my webdriver in C:/ which required admin privilages, however re-running the test again with the correct privilages gave back the same error
Update the issue isn't exclusive to the chrome driver. Even following setup instructions for either the Firefox or edge drivers end up on the same issues. It makes me suspect that the connection is facing some issue. I have tried running the test codes provided by Mozilla for the setup and it didn't work.
Unsure if that does help much or at all.
This error message...
selenium.common.exceptions.WebDriverException: Message: Can not connect to the Service chromedriver
...implies that the ChromeDriver was unable to initiate/spawn a new Browsing Context i.e. Chrome Browser session.
You need to take care of a couple of things:
Ensure that you have downloaded the exact format of the ChromeDriver binary from the download location pertaining to your underlying OS among:
chromedriver_linux64.zip: For Linux OS
chromedriver_mac64.zip: For Mac OSX
chromedriver_win32.zip: For Windows OS
Ensure that /etc/hosts file contains the following entry:
127.0.0.1 localhost
Ensure that ChromeDriver binary have executable permission for the non-root user.
Ensure that you have passed the proper absolute path of ChromeDriver binary through the argument executable_path as follows:
with webdriver.Chrome(executable_path=r'C:\path\to\chromedriver.exe', chrome_options=chrome_options) as driver:
So your effective code block will be:
options = Options()
options.add_argument("--headless")
options.add_argument('--no-sandbox') # Bypass OS security model
options.add_argument('--disable-gpu') # applicable to windows os only
options.add_argument("--disable-dev-shm-usage") # overcome limited resource problems
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
with webdriver.Chrome(executable_path=r'C:\path\to\chromedriver.exe', options=options) as driver:
driver.set_window_size(800, 460) # takes two arguments, width and height of the browser and it has to be called before
driver.execute_script("document.body.style.zoom='150%'")
driver.get("file:\\"+url) # takes one argument, which is the url of the website you want to open
driver.find_element_by_tag_name('body').screenshot(output) # avoids scrollbar
Mandatory Considerations
Finally, to avoid incompatibility between the version of the binaries you are using ensure that:
Selenium is upgraded to current levels Version 3.141.59.
ChromeDriver is updated to current ChromeDriver v79.0.3945.36 level.
Chrome is updated to current Chrome Version 79.0 level. (as per ChromeDriver v79.0 release notes)
Clean your Project Workspace through your IDE and Rebuild your project with required dependencies only.
If your base Web Client version is too old, then uninstall it and install a recent GA and released version of Web Client.
Take a System Reboot.
Execute your #Test as non-root user.
References
You can find a couple of reference discussions in:
Python Selenium “Can not connect to the Service %s” % self.path in linux server
selenium.common.exceptions.WebDriverException: Message: Can not connect to the Service chromedriver.exe while opening chrome browser
How to configure ChromeDriver to initiate Chrome browser in Headless mode through Selenium?
I am experiencing a very strange behaviour when testing Chrome via selenium webdriver.
Instead of navigating to pages like one would expect the 'get' command leads only to the download of tiny files (no type or.apsx files) from the target site.
Importantly - this behavior only occurs when I pass chrome_options as an argument
to the Chrome driver.
The same testing scripts work flawless with firefox driver.
Code:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options # tried with and without
proxy = '127.0.0.1:9951' # connects to a proxy application
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--proxy-server=%s' % proxy)
driver = webdriver.Chrome(chrome_options=chrome_options)
driver.get('whatismyip.com')
Leads to the automatic download of a file called download (no file extension, Size 2 Byte).
While calling other sites results in the download of small aspx files.
This all happens while the browser page remains blank and no interaction with
elements happen = the site is not loaded at all.
No error message, except element not found is thrown.
This is really strange.
Additional info:
I run Debian Wheezy 32 bit and use Python 2.7.
Any suggestions how to solve this issue?
I tried your code and captured the traffic on localhost using an SOCKS v5 proxy through SSH. It is definitely sending data through the proxy but no data is coming back. I have confirmed the proxy was working using Firefox.
I'm running Google Chrome on Ubuntu 14.04 LTS 64-bit. My Chrome browser gives me the following message when I try to configure a proxy in its settings menu:
When running Google Chrome under a supported desktop environment, the
system proxy settings will be used. However, either your system is not
supported or there was a problem launching your system configuration.
But you can still configure via the command line. Please see man
google-chrome-stable for more information on flags and environment
variables.
Unfortunately I don't have a man page for google-chrome-stable.
I also discovered that according to the selenium documentation Chrome is using the system wide proxy settings and according to their documentation it is unknown how to set the proxy in Chrome programmatically: http://docs.seleniumhq.org/docs/04_webdriver_advanced.jsp#using-a-proxy