So I made a program that start a headless browser ( Chrome ) with Selenium in Python.
I have a CyberGhost VPN.
When I try to start my script with the VPN active, Selenium open a chrome windows, with "data" in the url but doesn't proceed further.
Otherwise, it works when I start the script with inactive VPN and turn the VPN on when selenium is running normally.
I don't understart what can be the problem ...
Thanks :)
VPN is handled at the OS level, before Selenium gets anywhere near. You can try to setup a config with CyberGhostVPN and run an exec command to start your connection. After that, then start Webdriver and it should be running over VPN. See here on Calling an external command from Python
It may also be due to a firewall issue, so you can also try to deactivate the included firewall in CyberGhostVPN.
Related
I am automating a process using Selenium Firefox and Python. I am running Selenium on a Google Virtual Instance and accessing using RDP. All went fine when I ran it through RDP, but the problem is when I disconnect from RDP, Selenium doesn't work.
The part of selenium that doesn't work is :
WebDriverWait(driver, 15).until(ec.element_to_be_clickable((By. CSS_SELECTOR,'div.van-key__wrapper:nth-child(9) div:nth-child(1)'))).click()
I have tested everything all over the internet like gpedit regedit everything and also not tell to put bat file or command to stay connected to local user like chromedriver (selenium).
I have been using https://hub.docker.com/r/selenium/standalone-chrome on my Synology NAS to use Selenium Webdriver to perform automated requests.
I don't remember the command I ran but I started the container and run driver = webdriver.Remote("http://127.0.0.1:4444/wd/hub") in Python to connect to the selenium chrome image.
However I have a use case that requires me to use undetected-chromedriver. How do I install something like https://hub.docker.com/r/bruvv/undetected_chromedriver and connect to it from my NAS' python terminal?
Beware, everyone can publish on docker hub and so there are numerous undetected-chromedriver's. So what you are trying to install is someone else's (failed) attempt.
official: https://hub.docker.com/r/ultrafunk/undetected-chromedriver
as per #nnhthuan 's comment, some more detail.
undetected-chromedriver will start the Chrome binary, but will do it from python instead of letting the chromedriver binary run Chrome. As undetected-chromedriver does not officially support headless mode, you'll need a way to run "windowed" chrome on docker. To make this happen, you could use Xvfb to emulate a X-server desktop. If you forget this step, you won't be able to connect to chrome as chrome closes itself down (no screens found) even before undetected-chromedriver is able to connect, and so it crashes.
To ensure xvfb keeps running, you could use for example something like this in your entrypoint:
#!/bin/bash
export DISPLAY=:1
function keepUpScreen() {
echo "running keepUpScreen()"
while true; do
sleep .25
if [ -z $(pidof Xvfb) ]; then
Xvfb $DISPLAY -screen $DISPLAY 1280x1024x16 &
fi;
done;
}
keepUpScreen &
echo "running: ${#}"
exec "$#"
once your image is running stable, you could set your chromedriver debug_host to your internal ip address instead of 127.0.0.1, and debug_port to a static value. This would enable connections from remote hosts.
Don't forget to forward them in docker.
I have an application that runs on IE only and I am in the process of automating some stuff on it. The challenge am having is that, The application doesn't launch when I don't have an Active logged on session on the server from where am testing from (am connecting via RDP). Its very inconveniencing that you cannot run tests via IE in the "dead of the night" without a logged on user session. I have done lots of research and I don't seem to get a conclusive answer whether there is any workaround for this.
To Simulate this:
My environment python 3.9 , IE11 , Windows server 2019
script to launch Google
from selenium import webdriver
from time import sleep
url='https://www.google.com/'
driver = webdriver.Ie(r'C:\IEDriverServer\IEDriverServer.exe')
driver.get(url)
sleep(1)
driver.quit()
create a Batch file to launch the python script
"C:\TEST\venv\Scripts\python.exe" "C:\TEST\test.py"
EXIT
schedule the batch script to run using windows scheduler
Close the logged on session. in my case the scripts are running from windows server 2019.
Results
Note that, there is no issue when using chrome hence suspect its a limitation on the IE itself.
I'm making a program with selenium (python). It was working, and out of nowhere, the webdriver no longer works. I'm developing on a windows environment (and it works fine), but once I upload the code to the production server (Ubuntu), I try to open the web driver and it only displays data;, the driver hangs, then closes. No code after that continues.
Example:
print("Starting web driver")
driver = webdriver.Chrome(driver_path, options=opt)
print("Opening URL") # This code doesn't run
driver.get(config.url) # This code doesn't run
Things I've tried:
Running it on Windows (it works properly)
Updating the webdriver
Running it outside of the venv
Running the driver in a new, isolated environment
Wrapping the entire code within try-except (no errors output)
Running with the argument --headless
Edit: I'm running python3.7, chrome webdriver V 75.0.3770.90
Edit2: the driver_path var is the relative path to the chromedriver file. The opt are my list of chrome options:
opt = Options()
opt.add_argument('--no-sandbox')
opt.add_argument('--disable-dev-shm-usage')
profile = {"plugins.always_open_pdf_externally": True,
"download.default_directory": download_directory,
"download.prompt_for_download": False,
"download.directory_upgrade": True}
opt.add_experimental_option("prefs", profile)
I'm also using gunicorn as my webserver, but running it with the default (flask) webserver, I still encounter the issue. I'm also executing functions I've written for selenium through a flask-based web application (its a web app for work) The ubuntu machine that's running the script has a desktop environment installed.
After 15-25 seconds, the windows closes with no output in the terminal. After ~90 seconds, I get a message in the terminal saying:
Message: session not created
from disconnected: unable to connect to renderer
(Session info: chrome=75.0.3770.90)
I've also noticed that the chrome driver takes more time to open than usual.
Edit3: I've literally deleted then entire virtual machine and reinstalled it from scratch, and I'm still running into the same issue, I've reverted to an older version and it still doesn't run, which makes no logical sense. My only thought is that there is some configuration error or that something is interfering with it.
Edit4: I was able to get the log from the webdriver by adding the argument --verbose
Heres the log:
[1562179109.454][INFO]: resolved localhost to ["::1","127.0.0.1"]
[1562179111.454][WARNING]: Timed out connecting to Chrome, retrying...
[1562179111.454][INFO]: resolved localhost to ["::1","127.0.0.1"]
[1562179115.454][WARNING]: Timed out connecting to Chrome, retrying...
[1562179115.455][INFO]: resolved localhost to ["::1","127.0.0.1"]
[1562179123.454][WARNING]: Timed out connecting to Chrome, retrying...
[1562179123.455][INFO]: resolved localhost to ["::1","127.0.0.1"]
[1562179139.455][WARNING]: Timed out connecting to Chrome, giving up.
[1562179139.506][INFO]: [42e538ee02eb06b9ac776969dddf01d1] RESPONSE InitSession ERROR session not created
from disconnected: unable to connect to renderer
(Session info: chrome=75.0.3770.90)
[1562179139.506][DEBUG]: Log type 'driver' lost 9 entries on destruction
[1562179139.506][DEBUG]: Log type 'browser' lost 0 entries on destruction
I'm not too familiar with Linux, but from what I've seen in the past, I have a feeling its something to do with /etc/hosts (idk?)
Edit5:
I've noticed this started happening after I installed windscribe (vpn) which makes me think that windscribe is interfering with the connection somehow.
I turns out, windscribe (vpn) interferes with the connection to the chrome webdriver, I believe it has something to do with its built-in firewall, after uninstalling it, calling sudo apt autoremove -y and a reboot, it works properly!
Edit: I re-installed the VPN (windscribe) and de-activated the included firewall and it worked properly after that.
I'm trying to run a Selenium Script (Using PHP) using a Webserver.
I'm working on Kali and to simulate the Webserver I use Xampp.
I tried to run the selenium script on Xampp by the following steps:
-Download the Php Webdriver Bindings, put them in the folder 'htdpcs' of xampp and edit the 'example.php' file following the settings of my own device.
-Download and execute the Selenium Server Standalone, on port :4444.
In the end, I download the geckodriver and I execute the file, but I got the this error:
OSError: [Errno 98] Address already in use
How to fix it in order to run the php-selenium script?
The Selenium Server will fork a geckodriver as soon as it needs one to start a new browser session. You should not start a geckodriver yourself. If you want to use its Webdriver API yourself you can start it with the --webdriver-port argument.