Background
I have built a chrome extension to run tasks automatically with python and selenium on my localhost.
I would like to use my extension on my smartphone (with a different network). For this, I just need to use a specific browser and its running well
The Problem
In order for my extension to work on a different device, I need a server to receive a request passing all the information to start the job.
The API is done, but I don't know how to proceed with the server part.
What I've Tried
I tried to host on Heroku, it's working and I can receive requests, but web-driver isn't working. To being headless and the server is located in Europe, the website is blocking my access to the content.
Also, I tried to use a proxy, but I need authentication, but that doesn't work with selenium.
Further Explanation
Basically, I need to enable my chrome extension to do a request directly to my personal computer or use a server with a graphical user interface to set-up the proxy manually, but I don't have any idea how to proceed with this or if that is even the best option.
Any thoughts about a good work around?
Related
I am trying to create a python application while using eel to create a user interface in html. My operating system is Ubuntu Linux and I'm using Firefox to display the web interface.
The problem I'm having is every time I run the python code, Firefox opens a blank page saying "Unable to connect" followed by "Firefox can't establish a connection to the server at localhost:8000". However, if I click the "Try Again" button once, twice, or three times, my interface is displayed.
Once open, I can navigate to different pages but I also noticed that once I navigate to a different page, some of my javascript stops working (specifically a window.close() function). I don't know if this is related but I thought I would mention it just in case.
Any advice on the matter would be greatly appreciated.
Thank you.
I changed my browser from firefox to chromium and now my interface loads on startup the first time. I know some documentation says it can be used with firefox, and it can, but it seems to be kind of buggy and works better with other browsers.
However, I'm still having trouble with my javascript not running but that will be another question.
when I try to use python + Selenium to run my code, the error message shows in the terminal during the program is running, it has no effect of my program, but sort of annoying, does anyone have any solution to tell me why the error shows, and how to disable the error message.
my network is located in China, and our network policy disables the access to Google, is the may the cause?
[21792:15920:1230/144009.402:ERROR:gcm_channel_status_request.cc(145)] GCM channel request failed.
anyway, thanks in advance.
Note: This is not a definitive answer; it is very rough conclusions drawn from a quick investigation.
GCN is Google Cloud Messaging. It appears that the Selenium Chrome extension is using GCN. See for example https://pushwizard.com/chrome-gcm-messaging.
I see these messages when my Python Selenium Chrome application is sleeping and I hibernate the machine. It may occur at other times.
I surmise that the Selenium Python backend is using GCN and uses a ping or keep-alive type of message to find out if the Chrome browser is still there. I further surmise that since my Python application takes much less memory than the Chrome browser, my app wakes up first, pings the browser, and reports that there was no response.
Since it's not causing a problem this is enough for me.
Google Cloud Messaging is deprecated - you should use Firebase Cloud Messaging instead.
You may need to update your selenium webdriver for chrome.
Since your access to google ( and I think google services too ) is disabled this also could be the issue. To test this, you should go and implement the selenium application with firefox (or any other non chromium based browser) ; which shouldn't use google services inherently. ( but i am not 100% sure about that )
I am currently using python to write some appium test. Because I am behind a corporate firewall my traffic needs to go via a proxy.
I have set my http_proxy and https_proxy variables, but it seems like this is not being picked up by python during execution.
I tried the exact same test using javascript and node and the proxy get picked up and everything works so I am sure the problem is python not following the proxy settings.
How can I make sure python is using correct proxy settings?
I am using python 2.7 on macos mojave
Thanks!
So I figured out that appium currently does not support options to provide a proxy when making a remote connection. As temporary solution I modified the remote_connection module of selenium that appium inherits forcing it to use a proxy url for the connection.
My python knowledge is not that good but I think it shoudnt take much effort for someone to make a module that wraps/override the appium webdriver remote connection to include a proxy option.
So, I have multiple selenium chrome drivers doing automated tasks with python on multiple linux servers. 99% of the tasks is automated until it runs into situations that require human involvement.
I am currently using VNC to connect to my linux server and control the chrome but I am not enjoying at all.
I am simply trying to find a better solution.
Thanks!
EDIT:
To be more specific, since the chrome drives sometimes need to be controlled by other people and I actually don't want third-party to connect to my server with VNC.
I have actually tried to re-direct the whole browsing session to local PC when human control is needed (like what proxy servers do) but I didn't succeed.
I am now developing a webpage crawler, unfortunately the website generates the results by ajax. Following some coders suggestion, I tried to use selenium, a test automation tool for python.
As the example given in the documentation:
driver = webdriver.Firefox()
This code executes to open the Firefox browser. And then do something just like filling the form, submitting and so on.
Frankly speaking, this example works well on my PC(ubuntu 12.10), but my project will finally transfer to a CentOS server.
What I am considering is whether the code(need to open a browser gui) can be successfully ran on the CentOS server over ssh because no KDE such as gnome provided on that machine.....
And if without browser gui, the code cannot work well, then is there any other solutions?
Any reply would be admired~
You can probably use the HtmlUnit driver if you enable javascript. The only way to be sure though is to test it out. Another option would be to try and run with an X framebuffer.