I have a web scraping project with python using selenium. I need to distribute it other people who don't have python. So, I wanted to pack my code into an exe file.
The code is completed, no problem working on python. Then packed into an exe with pyinstaller, no problem working by clicking the exe. (I'm adding latest geckodriver.exe into the folder of exe file)
Unfortunately, as soon as I trigger the exe from task scheduler, I'm getting below error.
I guess the problem is about Geckodriver paths, but couldn't figure it out.
url = "https://www.myurl.com"
options = Options()
options.headless = False
driver = webdriver.Firefox(options=options) #Line34 is here
driver.get(url)
Related
I did not get a specific chrome option to work fine, when started chrome per python selenium webdriver script.
The goal is to start chrome in a python selenium script with a fake webcam, which plays a video file.
The problematic options are "use-file-for-fake-video-capture" combined with "use-fake-device-for-media-stream":
options.add_argument("--use-fake-device-for-media-stream")
options.add_argument("--use-file-for-fake-video-capture=C:\Testfiles\videofiles\bus_cif.y4m")
Both Options are working totally fine used in a batch-file with chrome.exe. For example, the following batch is working fine and used fake webcam videofile:
"C:\Program Files\Google\Chrome\Application\chrome.exe" --use-fake-device-for-media-stream --use-file-for-fake-video-capture=C:\Testfiles\videofiles\bus_cif.y4
Also, the rest of my python chrome webdriver script options are working fine, so the code itself should be correct. For example, the following options are working fine:
options.add_argument("--allow-file-access-from-files")
options.add_argument("--use-fake-ui-for-media-stream")
options.add_argument("--disable-infobars")
Chrome 97.x (with newest stable chromium webdriver files)
Is there something special about the option "use-file-for-fake-video-capture" when used in a python script with options? Or perhaps the path have to be in another format?
Thanks a lot!
Figured it out. Spent hours yesterday, but today shortly after posting to stackoverflow, I found the solution myself:
Problem is the path I used:
Windows path in Python
When using the following, everything works fine and fake video is used even when chrome is started by python script:
options = Options()
options.add_argument("--use-fake-ui-for-media-stream")
options.add_argument("--use-fake-device-for-media-stream")
options.add_argument(r'--use-file-for-fake-video-capture=C:\Testfiles\videofiles\bus_cif.y4m')
self.driver = webdriver.Chrome(options=options)
Important is the r' when using windows path with backslashes. I think the other solution would be using slash instead of backslash.
I wrote a small automation script using Python and Selenium Web Driver and I made it into an executable and the executable worked fine on my computer but when I tried it on another computer I got that chromedriver isn't in path (note: I included chromedriver using the add binary command and both computers are running the same operating system: Mac OS)
I am not quite sure why this happened, my suspicion is the following:
In my .py code I used the following two lines to initiate the chromedriver
path = "/Users/sergio/chromedriver"
driver = webdriver.Chrome(path)
However, the other computer doesn't even have this path so might be causing it even has this path. Now one might suggest simply using driver = webdriver.Chrome() but here comes the bigger problem: I tried having my chromedriver in a system path like /usr/local/bin and when I did that driver = webdriver.Chrome() worked but compiling the program using pyinstaller didn't work, I ran:
pyinstaller --onefile --noconsole --add-binary 'usr/local/bin/chromedriver:.' finalregtool.py
but I got a lot of errors including Unable to find "/Users/sergio/usr/local/bin/chromedriver" when adding binary and data files
This led to me thinking that pyinstaller automatically adds /Users/sergio to the given path which makes my path wrong so I had to use something inside /Users/sergio , I tried putting my chromedriver in Applications but compilation resulted in not finding the path again, for some reasons compilation worked when the path was /Users/sergio/chromedriver so I went with this path but the code didn't work anymore (chromedriver not found in path) so I had to specify the path manually.
Now I am kind of stuck in a deadlock not knowing what to do. Any help would be appreciated. I hope my explanation was clear enough.
My only end goal is compiling my selenium script and running it on other computers without having them install anything extra, I don't care how this should be done so if anyone has a solution outside pyinstaller I don't mind.
Update: added a bounty
There're couple of good solutions for that issue:
1.Install webdriver-manager package:
pip install webdriver-manager
what I prefer mostly and use it like:
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
driver = webdriver.Chrome(ChromeDriverManager().install())
2.Launch Remote Selenium Hub called Selenoid and lauch browser remotely:
driver = webdriver.Remote(command_executor='ip_of_your_selenoid_instance:4444/wd/hub',desired_capabilities=DesiredCapabilities.CHROME)
3.You can launch Chrome docker container and also going to by remote url, what will be always localhost:
driver = webdriver.Remote(command_executor='http://127.0.0.1:4444/wd/hub',desired_capabilities=DesiredCapabilities.CHROME)
You need to launch a standalone chrome browser:
docker run -d -p 4444:4444 selenium/standalone-chrome
and then in your python script launch browser using Remote webdriver:
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
driver = webdriver.Remote("http://127.0.0.1:4444/wd/hub", DesiredCapabilities.CHROME)
docker-compose:
# docker-compose.yml
selenium:
image: selenium/standalone-firefox
ports:
- 4444:4444
pyinstaller unpacks everything into a temporary directory during execution of exe, when using the --onefile mode. Refer to this answer and the related post to get an idea about how to get the correct relative path. You might have to use
path = resource_path("chromedriver")
Also, you can directly add the chromedriver to PATH environment variable in the runtime so you wouldn't have to specify the path explicitly and you can simply call webdriver.Chrome()
os.environ["PATH"] += os.pathsep + resource_path("chromedriver")
pyinstaller conciders usr/local/bin/chromedriver:. as a relative path, thus adds /Users/sergio in an attempt to make it absolute, try using a different location. This location doesn't matter anyway, once this file is found pyinstaller will add this to the root directory of the compiled exe, and unpack it when the exe is executed.
The modern way to deal with the chromedriver binary is to use WebDriverManager.
First you need to install the manager:
pip install webdriver-manager
Then you use it (for Chrome):
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
driver = webdriver.Chrome(ChromeDriverManager().install())
In my .NET Core project I use not the most modern approach I have described, but the latest stable NuGet from here https://www.nuget.org/packages/Selenium.WebDriver.ChromeDriver/.
Basically the nuget contains the chromedriver and copies it every time to your assembly folder (\bin\Debug\netcoreapp3.1 in my case). This is the line I have in my project:
"<PackageReference Include="Selenium.WebDriver.ChromeDriver" Version="*" />"
The star means the latest (highest number) stable package will be used as explained here. Of course you can install the latest version manually, and then on each 2 major Chrome versions to update the NuGet package reference so that is matches your Chrome version. (87 chromedriver works with 87 and 88 Chrome, but not with 89).
Then the driver is instantiated with this lines:
var driver = new ChromeDriver(Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
References:
Error message: "'chromedriver' executable needs to be available in the path"
https://www.automatetheplanet.com/webdriver-net50/
path = "/Users/sergio/chromedriver"
driver = webdriver.Chrome(path)
Change the above code to the following format
driver = webdriver.Chrome(executable_path='C:/Users/sergio/chromedriver.exe')
I have automated a task using selenium in python and now I am compiling exe from that script.
From what I know I wrote this piece of code in my script.
PATH = "/home/blackhawk/tools/webdriver/chrome/chromedriver"
driver = webdriver.Chrome(PATH)
Here the script is using chromdriver from my machine. Currently, my application is explicitly asking for the path of chromedriver.
Is there any way I can add this file inside my application.
What changes I should do to my PATH varible as well to make it work.
Here is the repository with test.py
I installed the selenium module with pip installer. Then I tried to make the code to open firefox, then open a new tab to go to google.
Code:
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("http://www.google.com")
The expected result should have opened firefox and then go to google.com.
But what actually happened was the program didn't produce any error, it just didn't open firefox, and the screen wasn't frozen either.
Download a matching version of Geckodriver and unpack the geckodriver.exe to the location where current user can execute programs from (normally it's any place inside your home folder)
Amend your code to include the location of the aforementioned geckodriver like:
driver = webdriver.Firefox(executable_path="/path/to/the/geckodriver/binary")
If this doesn't help - provide the path to the Firefox executable as well:
driver = webdriver.Firefox(executable_path="/path/to/the/geckodriver/binary", firefox_binary="/path/to/firefox/binary")
Instead of steps 2 and 3 you can add both firefox and geckodriver to your OS PATH
References:
Selenium with Python - Getting Started
Selenium With Python
Selenium using Python - Geckodriver executable needs to be in PATH
We have an Ubuntu server which we use for running Selenium tests with Chrome and Firefox (I installed ChromeDriver) and I also want to run the tests locally on my Windows 10 computer. I want to keep the Python code the same for both computers. But I didn't find out how to install the ChromeDriver on Windows 10? I didn't find it on the documentation [1, 2].
Here is the code that runs the test in Chrome:
import unittest
from selenium import webdriver
class BaseSeleniumTestCase(unittest.TestCase):
...
...
...
...
def start_selenium_webdriver(self, chrome_options=None):
...
self.driver = webdriver.Chrome(chrome_options=chrome_options)
...
I also found How to run Selenium WebDriver test cases in Chrome? but it seems to be not in Python (no programming language is tagged, what is it?)
Update #1: I found some Python code in https://sites.google.com/a/chromium.org/chromedriver/getting-started, but where do I put the file in Windows 10 if I want to keep the same Python code for both computers?
Update #2: I downloaded and put chromedriver.exe in C:\Windows and it works, but I didn't see it documented anywhere.
As Uri stated in the question, under Update #2, downloading the latest release of chromedriver and placing it in C:\Windows corrects the issue.
I had the same issue with Chrome hanging when the browser window opens (alongside a command prompt window).
The latest drivers can be found at:
https://sites.google.com/chromium.org/driver
The version in the chromedriver_win32.zip file is working on my 64-bit system.
Download the chromedriver.exe and save it to a desired location
Specify the executable_path to its saved path
The sample code is below:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument('headless')
driver = webdriver.Chrome(executable_path="path/to/chromedriver.exe", chrome_options=options)
driver.get("example.html")
# do something here...
driver.close()
As Uri stated in Update #2 of the question, if we put the chromedriver.exe under C:/Windows, then there is no need to specify executable_path since Python will search under C:/Windows.
Let me brief out the requirements first.
You need to download the chrome web driver zip from here. https://chromedriver.storage.googleapis.com/index.html?path=2.33/
Extract the file and store it in a desired location.
Create a new project in Eclipse and include the following code in your class.
System.setProperty("webdriver.chrome.driver", "C:\\temp\\chromedriver.exe");
WebDriver driver = new ChromeDriver();
Explanation : System.setProperty(key,value):
Key is default and same for all the systems, value is the location of your chromedriver extract file.