Adding Chrome Webdriver with python script while converting it to exe - python

I have automated a task using selenium in python and now I am compiling exe from that script.
From what I know I wrote this piece of code in my script.
PATH = "/home/blackhawk/tools/webdriver/chrome/chromedriver"
driver = webdriver.Chrome(PATH)
Here the script is using chromdriver from my machine. Currently, my application is explicitly asking for the path of chromedriver.
Is there any way I can add this file inside my application.
What changes I should do to my PATH varible as well to make it work.
Here is the repository with test.py

Related

Geckodriver problem while triggering .exe from Task Scheduler

I have a web scraping project with python using selenium. I need to distribute it other people who don't have python. So, I wanted to pack my code into an exe file.
The code is completed, no problem working on python. Then packed into an exe with pyinstaller, no problem working by clicking the exe. (I'm adding latest geckodriver.exe into the folder of exe file)
Unfortunately, as soon as I trigger the exe from task scheduler, I'm getting below error.
I guess the problem is about Geckodriver paths, but couldn't figure it out.
url = "https://www.myurl.com"
options = Options()
options.headless = False
driver = webdriver.Firefox(options=options) #Line34 is here
driver.get(url)

Getting chrome driver not in path when moving my selenium executable to another computer

I wrote a small automation script using Python and Selenium Web Driver and I made it into an executable and the executable worked fine on my computer but when I tried it on another computer I got that chromedriver isn't in path (note: I included chromedriver using the add binary command and both computers are running the same operating system: Mac OS)
I am not quite sure why this happened, my suspicion is the following:
In my .py code I used the following two lines to initiate the chromedriver
path = "/Users/sergio/chromedriver"
driver = webdriver.Chrome(path)
However, the other computer doesn't even have this path so might be causing it even has this path. Now one might suggest simply using driver = webdriver.Chrome() but here comes the bigger problem: I tried having my chromedriver in a system path like /usr/local/bin and when I did that driver = webdriver.Chrome() worked but compiling the program using pyinstaller didn't work, I ran:
pyinstaller --onefile --noconsole --add-binary 'usr/local/bin/chromedriver:.' finalregtool.py
but I got a lot of errors including Unable to find "/Users/sergio/usr/local/bin/chromedriver" when adding binary and data files
This led to me thinking that pyinstaller automatically adds /Users/sergio to the given path which makes my path wrong so I had to use something inside /Users/sergio , I tried putting my chromedriver in Applications but compilation resulted in not finding the path again, for some reasons compilation worked when the path was /Users/sergio/chromedriver so I went with this path but the code didn't work anymore (chromedriver not found in path) so I had to specify the path manually.
Now I am kind of stuck in a deadlock not knowing what to do. Any help would be appreciated. I hope my explanation was clear enough.
My only end goal is compiling my selenium script and running it on other computers without having them install anything extra, I don't care how this should be done so if anyone has a solution outside pyinstaller I don't mind.
Update: added a bounty
There're couple of good solutions for that issue:
1.Install webdriver-manager package:
pip install webdriver-manager
what I prefer mostly and use it like:
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
driver = webdriver.Chrome(ChromeDriverManager().install())
2.Launch Remote Selenium Hub called Selenoid and lauch browser remotely:
driver = webdriver.Remote(command_executor='ip_of_your_selenoid_instance:4444/wd/hub',desired_capabilities=DesiredCapabilities.CHROME)
3.You can launch Chrome docker container and also going to by remote url, what will be always localhost:
driver = webdriver.Remote(command_executor='http://127.0.0.1:4444/wd/hub',desired_capabilities=DesiredCapabilities.CHROME)
You need to launch a standalone chrome browser:
docker run -d -p 4444:4444 selenium/standalone-chrome
and then in your python script launch browser using Remote webdriver:
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
driver = webdriver.Remote("http://127.0.0.1:4444/wd/hub", DesiredCapabilities.CHROME)
docker-compose:
# docker-compose.yml
selenium:
image: selenium/standalone-firefox
ports:
- 4444:4444
pyinstaller unpacks everything into a temporary directory during execution of exe, when using the --onefile mode. Refer to this answer and the related post to get an idea about how to get the correct relative path. You might have to use
path = resource_path("chromedriver")
Also, you can directly add the chromedriver to PATH environment variable in the runtime so you wouldn't have to specify the path explicitly and you can simply call webdriver.Chrome()
os.environ["PATH"] += os.pathsep + resource_path("chromedriver")
pyinstaller conciders usr/local/bin/chromedriver:. as a relative path, thus adds /Users/sergio in an attempt to make it absolute, try using a different location. This location doesn't matter anyway, once this file is found pyinstaller will add this to the root directory of the compiled exe, and unpack it when the exe is executed.
The modern way to deal with the chromedriver binary is to use WebDriverManager.
First you need to install the manager:
pip install webdriver-manager
Then you use it (for Chrome):
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
driver = webdriver.Chrome(ChromeDriverManager().install())
In my .NET Core project I use not the most modern approach I have described, but the latest stable NuGet from here https://www.nuget.org/packages/Selenium.WebDriver.ChromeDriver/.
Basically the nuget contains the chromedriver and copies it every time to your assembly folder (\bin\Debug\netcoreapp3.1 in my case). This is the line I have in my project:
"<PackageReference Include="Selenium.WebDriver.ChromeDriver" Version="*" />"
The star means the latest (highest number) stable package will be used as explained here. Of course you can install the latest version manually, and then on each 2 major Chrome versions to update the NuGet package reference so that is matches your Chrome version. (87 chromedriver works with 87 and 88 Chrome, but not with 89).
Then the driver is instantiated with this lines:
var driver = new ChromeDriver(Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
References:
Error message: "'chromedriver' executable needs to be available in the path"
https://www.automatetheplanet.com/webdriver-net50/
path = "/Users/sergio/chromedriver"
driver = webdriver.Chrome(path)
Change the above code to the following format
driver = webdriver.Chrome(executable_path='C:/Users/sergio/chromedriver.exe')

'chromedriver' executable needs to be in PATH.' after adding chromedriver to my PATH

I have spent hours trying to get this working to no avail. I am trying to run this test script:
from selenium import webdriver
driver = webdriver.Chrome()
driver.get("https://www.nytimes.com")
headlines = driver.find_elements_by_class_name("story-heading")
for headline in headlines:
print(headline.text.strip())
I have installed selenium and downloaded chromedriver as well as adding it to my PATH using sudo nano /etc/paths and adding entering the path /Users/Myname/Desktop/chromedriver. I closed out the terminal and checked it using echo $PATH and that path showed in my PATH so I'm sure it's there and /Users/Myname/Desktop/chromedriver is the location of my chromedriver exe file. However when I run the .py script above it returns "'chromedriver' executable needs to be in PATH." and I don't know what else to try.
Any help would be greatly appreciated. Thanks!
You have to add the actual path to the *.exe or chromedriver.exe in the function call, there is no need to add it to your PC's PATH variable.
NOTE: Just to clarify, for example
driver = webdriver.Chrome("/Users/Myname/Desktop/chromedriver.exe") # In Actual Add The Path To chromedriver.exe or whatever *.exe in here
Only you can know where you exe is located so just add the exe path in the function rest it will be done for you.
Happy Coding!

'chromedriver' executable needs to be in PATH in Python Django

I am trying to execute a Python script in Django. My view file is:
def generate_traffic(request):
#other code
out = run([sys.executable, 'C://Users//Maisum Abbas//learning_users//trafficapp//traffic.py', "--domain", url, "--thread", requests, "--max-clicks", maximum, "--min-clicks", minimum])
If I run the script alone without running it in Django, it works fine. Now, the problem that I am facing is that the script is not able to detect chromedriver if I run it from my function generate_traffic in view.py. I don't know what seems to be the problem since the chromedriver is in the same directory as of my other files.
myFolder
-view.py
-traffic.py (the script I am executing)
-chromedriver.exe
-otherfiles
Kindly guide me of what I am doing wrong or what seems to be the problem here.
chromedriver.exe should be added to the PATH environment variable. Apparently, you are using Windows so here are the steps to add it in Windows 10 PATH:
Search for View Advanced System Settings in the start menu and open it.
Click on the Environment Variables button.
In the newly opened window, choose Path and click on Edit.
In the new window, click on New to add a new file/dir into PATH.
For your case it should be something like this:
Place your chromedriver.exe file in your Python Directory -> Scripts Directory, it will resolve the issue. Then just write it like this in your code:
driver = webdriver.Chrome(options=options)
or
driver = webdriver.Chrome()
depending on your use.

Can Python install files into PATH?

In order to install Selenium, step 3 on (this site indicates needing to install the chromedriver file in your PATH. I am on a work computer that does not have access to the system PATH directly. I have tried listing in the local PATH (I'm on Windows 7) variable chain like so: C:\Users\mknerr\AppData\Local\Programs\Python\Python36-32\Scripts\;C:\Users\mknerr\AppData\Local\Programs\Python\Python36-32\;C:\Users\mknerr\AppData\Local\atom\bin;C:\Users\mknerr\Programs\ChromeDriver\
(The .exe is in the ChromeDriver folder)
When I run the script with webDriver.Chrome(), I still get a WebDriverException that chromedriver needs to be in my PATH. If anyone has an idea why this isn't working from my local PATH, I'd love to hear them.
However, my real question is when I distribute this script to the rest of my team, they will likely have the same issue since my script will be calling chromedriver, which none of them will have installed, much less in their PATH. Can Python directly install a program or dependency in the PATH so they don't have to go directly accessing environment variables? I can guarantee nobody is going to feel comfortable doing that.
You can place chromedriver.exe in the same folder as the executable. Just run the program with the driver right next to it.
In our internal automation framework, we actually just distribute the Chromedriver executable as part of the framework, in the same folder as the framework's entry point.
Then, whenever we need a browser session, we do something similar to this:
import os
from selenium import webdriver
chromedriver_location = os.path.join(
os.path.dirname(os.path.abspath(__file__)), 'chromedriver.exe')
context.browser = webdriver.Chrome(executable_path=chromedriver_location)
chromedriver.exe is the default Windows name of the executable, of course; change to whatever you need.
This avoids any user setup other than installing the framework itself. No messing with PATH or any other local files.

Categories