I have been using https://hub.docker.com/r/selenium/standalone-chrome on my Synology NAS to use Selenium Webdriver to perform automated requests.
I don't remember the command I ran but I started the container and run driver = webdriver.Remote("http://127.0.0.1:4444/wd/hub") in Python to connect to the selenium chrome image.
However I have a use case that requires me to use undetected-chromedriver. How do I install something like https://hub.docker.com/r/bruvv/undetected_chromedriver and connect to it from my NAS' python terminal?
Beware, everyone can publish on docker hub and so there are numerous undetected-chromedriver's. So what you are trying to install is someone else's (failed) attempt.
official: https://hub.docker.com/r/ultrafunk/undetected-chromedriver
as per #nnhthuan 's comment, some more detail.
undetected-chromedriver will start the Chrome binary, but will do it from python instead of letting the chromedriver binary run Chrome. As undetected-chromedriver does not officially support headless mode, you'll need a way to run "windowed" chrome on docker. To make this happen, you could use Xvfb to emulate a X-server desktop. If you forget this step, you won't be able to connect to chrome as chrome closes itself down (no screens found) even before undetected-chromedriver is able to connect, and so it crashes.
To ensure xvfb keeps running, you could use for example something like this in your entrypoint:
#!/bin/bash
export DISPLAY=:1
function keepUpScreen() {
echo "running keepUpScreen()"
while true; do
sleep .25
if [ -z $(pidof Xvfb) ]; then
Xvfb $DISPLAY -screen $DISPLAY 1280x1024x16 &
fi;
done;
}
keepUpScreen &
echo "running: ${#}"
exec "$#"
once your image is running stable, you could set your chromedriver debug_host to your internal ip address instead of 127.0.0.1, and debug_port to a static value. This would enable connections from remote hosts.
Don't forget to forward them in docker.
Related
So I made a program that start a headless browser ( Chrome ) with Selenium in Python.
I have a CyberGhost VPN.
When I try to start my script with the VPN active, Selenium open a chrome windows, with "data" in the url but doesn't proceed further.
Otherwise, it works when I start the script with inactive VPN and turn the VPN on when selenium is running normally.
I don't understart what can be the problem ...
Thanks :)
VPN is handled at the OS level, before Selenium gets anywhere near. You can try to setup a config with CyberGhostVPN and run an exec command to start your connection. After that, then start Webdriver and it should be running over VPN. See here on Calling an external command from Python
It may also be due to a firewall issue, so you can also try to deactivate the included firewall in CyberGhostVPN.
I'm very new to docker. I want to build my python application within a docker container. As I build the application I want to be testing / running it in Pycharm and in the container I build.
How do I connect Pycharm pro to a specific container or image (either python or Anaconda)?
When I create a project, click pure python and then add remote, then clicking docker I get the following result
I'm running on Mac OS X El Capitan (10.11.6) with Docker version 1.12.1 and Pycharm Pro 2016.2.3
Docker-for-mac only supports connections over the /var/run/docker.sock socket that is listening on your OSX host.
If you try to add this to pycharm, you'll get the following message:
"Cannot connect: java.lang.ExceptionInInitializerError, caused by: java.lang.IllegalStateException: Only supported on Linux"
So PyCharm really only wants to connect to a docker daemon over a TCP socket, and has support for the recommended TLS protection of that socket. The Certificates folder defaults to the certificate folder for the default docker-machine machine, "default".
It is possible to implement a workaround to expose Docker for Mac via a TCP server if you have socat installed on your OSX machine.
On my system, I have it installed via homebrew:
brew install socat
Now that's installed, I can run socat with the following parameters:
socat TCP-LISTEN:2376,reuseaddr,fork,bind=127.0.0.1 UNIX-CLIENT:/var/run/docker.sock
WARNING: this will make it possible for any process running as any user on your whole mac to access your docker-for-mac. The unix socket is protected by user permissions, while 127.0.0.1 is not.
This socat command tells it to listen on 127.0.0.1:2376 and pass connections on to /var/run/docker.sock. The reuseaddr and fork options allow this one command to service multiple connections instead of just the very first one.
I can test that socat is working by running the following command:
docker -H tcp://127.0.0.1:2376 ps
If you get a successful docker ps response back, then you know that the socat process is doing its job.
Now, in the PyCharm window, I can put the same tcp://127.0.0.1:2376 in place. I should get a "Connection successful" message back:
This workaround will require that socat command to be running any time you want to use docker from PyCharm.
If you wanted to do the same thing, but with TLS, you could set up certificates and make them available for both pycharm and socat, and use socat's OPENSSL-LISTEN instead of the TCP-LISTEN feature. I won't go into the details on that for this answer though.
I'm running a cron job which runs a Python script that uses Selenium. Selenium requires a display, so I've installed Xvfb, started the display, and firefox:
sudo Xvfb :10 -ac
export DISPLAY=:10
firefox
This works when I run these commands in the console, but I want to be able to do it with cron. How can I do this? If I run the virtual display as the main user, will they have access to the virtual display when it's needed by Python script/Selenium?
I would suggest you to use selenium Ghost Driver(it's running PhantomJS).
I perform headless web session tests with selenium (python, ubuntu server 15, firefox), which can last for hours. I do make use of pyvirtualdisplay + xvfb.
My python scripts begin like this:
from pyvirtualdisplay import Display
virtualdisplay = True
if virtualdisplay:
display = Display(visible=0, size=(1920, 1240))
display.start()
How is it possible to peep what's goin' on without actually get screenshots, e.g. vnc session?
I tried several solutions, but they didn't work because perhaps they are outdated or too general.
using x11vnc can do the trick. Just add this line to bash script you are using to launch tests:
x11vnc -q -bg -display $DISPLAY
After that you can connect to your virtual display on default port 5900 (or any other of your choice). Keys -q and -bg forces x11vnc to be quiet and run in background respectively.
Of course, you should set up port forwarding for SSH connection:
ssh -L 5900:localhost:5900 yourhost
So here's my setup:
Using a flask server with uwsgi, and through a controller action, calling a python script that uses splinter (which uses selenium) to automate the gui. The web server doesn't have a display, so I'm using xvfb.
Sshing into the machine and running xvfb and exporting display=:99, and then running the python script works great. But running it through a controller action does not work - I get the following error:
WebDriverException: Message: The browser appears to have exited before we could connect.
(this is the same error that is returned when xvfb isn't running)
ps aux shows that xvfb is running as the same user as the web server (I've isolated everything, and have a separate controller action that executes:
p = subprocess.Popen("Xvfb :99 &", stdout=fstdout,stderr=fstderr, shell=True))
and DISPLAY is set to :99 on both root and the web server user.
I could install vncserver and try that, but I suspect I will end up with the same problem. I've also tried to avoid calling xvfb directly and using PyVirtualDisplay instead, but same problem.
edit: it errors on this line (if using splinter):
browser = Browser()
or, if selenium:
with pyvirtualdisplay.Display(visible=True):
binary = FirefoxBinary()
driver = webdriver.Firefox(None, binary)
(it errors on the last line there)
Any ideas?