I've no idea on how to do this and all the documentation that I could find by google did not help. A while back I was introduced to selenium through this tutorial and now that I'm more comfortable with it, I want my selenium "bot" to run on a webserver 24/7, receiving orders from me through facebook messenger (something I already did with it running on my local machine).
I tried to find answers online and was overwhelmed by the amount of information, finding nothing that is clear to understand. All the pages I've been through require me to learn about a large array of things and have been very specific about their tools. And some times I try to follow along something just to receive an error I don't understand nor is it explained on said something how to fix it.
I also asked this question on Reddit only to be downvoted without answer. I've no idea how to run selenium + chrome on a server.
Take me for the stupidest person on earth, How can I do this in the most clear steps? I'd prefer to use chrome with selenium, through python or php.
You can try it by making your chromedriver run headlessly. I was introduced to it by this tutorial. a headless browser means a web browser without a graphical user interface. Headless browsers provide automated control of a web page in an environment similar to your local browser and you can get screenshots too.
If headless browser is giving you an error which can't be resolved(like screen sharing error), then you can try aws or Google Cloud like platforms
Related
Ive been working alot with browser automation and python lately, and I've been using selenium and chromedriver but I have found a few limitations. For example, it's very easy for websites to tell that you are using selenium aswell as each chrome instance taking up alot of computer memory while running. I was wondering if there are any alternative python libraries that can also control a browser window in the same ways that selenium does?
Thanks
There is Pylenium that I'm aware of. Its based on top of selenium but exposes some cypress styled DSLs. You can check out the documentation over here :
https://elsnoman.gitbook.io/pylenium/
I'm trying to write a python executable that runs in the background and records the amount of time I spend on facebook. This is one of my first projects in software development. The problem is, I haven't found a way to access the chrome browser and look at what URL's are open.
I've used selenium webdriver, but it seems to only be able to open up new tabs.
I've found some kind of google extension called google tabs api, but i'm not entirely sure if it even is a library I can import into python or if it's even meant for python.
For all my projects written in python where I use selenium to scrape websites I can only run the script from my own machine and if I were to send the script to a client if say he needed it to run on a daily basis, it most probably wouldn't work.
Is there a way to use selenium webdriver in a way for the script to be portable and able to run from any platform so that I could send it to my clients and be confident that it would work. I couldn't find anything definite on the internet that would help me.
If this is not possible with selenium is it possible with some other python module? So far for pages that use javascript I used selenium for scraping. Should I switch to something else for portability? Please advise me. I would really appreciate if someone could point me in the right direction.
I would download a version of your browser driver (e.g. chromedriver for Chrome) for all available platforms and put all of them in the script folder.
I would then zip it and share it with the customer.
It would also be quite easy to build a script that automatically checks local Operating System and dowloads the needed driver from internet (using Python wget or similar) but I do not see a serious advantage in using this approach.
As a final thought it is also possible using Selenium with remote WebDriver but that would complicate things and leave you with a server to mantain and update.
I am a basic python programmer.
I would like to change the settings of google chrome, especially the proxy and LAN settings automatically with a program.
I tried looking at the webbrowser library, but I don't think its sufficient.
I looked at the selenium library, but I do not know where to start looking for my requirements in the library, could someone provide a resource or a method on how to automate my tasks? I looked at another questions which describes a process using c#. But thats not my problem. I want the required documentation for my exact purpose.
http://seleniumhq.github.io/selenium/docs/api/py/api.html
which of these must I use?
What you need to use is the Selenium WebDriver API, with Python bindings (lots of other languages are also supported, and many frameworks are available).
Here's a good Python example, plus there's a huge number of helpful questions on this site.
Here's a good description of the ChromeDriver (the WebDriver library that automates a Chrome browser) complete with installation instructions and example Python code.
I personally recommend BrowserMob as a proxy server for Selenium. Although it's Java-based, once it's running you can control it via a REST API - I assume you want to be able to manipulate requests/responses, that kind of thing?
In any case, there are lots of proxy servers available, and the configuration to use one within your Python code is pretty standard. See this example (see also the links).
I have process that uses iMacros in Firefox to open some websites and click on some buttons and do some stuff (not any weird stuff, internal work pages). The problem is that I basically can't use my computer while that happens.
I want to automate this via python and found this:
Integrating iMacros scripts into python
However the answer to that question and the links mention that I need the business or enterprise version of it.
Is there a way to just do something like:
Open firefox (I know how)
Use (as a plugin) iMacros to run a iim script in x location
Thanks!!
You can have 100% control over Firefox with Python, as both are open source. The trick is to figure out details. Here are some starting points
Python can script Firefox with Selenium WebDriver
With some tricks, you can dive deeper into Firefox what basic Selenium interaction offers, like opening a web pages. This would include giving direct commands to plugins. Here is an example of settings Firefox profile in a mode that normal security restrictions do not apply.
You need to study Firefox architecture how you can trigger iMacros plugin commands from Selenium. This is the tricky part as this is very marginal use case and there might not be much information available. Expect spending few days of learning Firefox internals.
My guess is that you can disable Firefox security, and then use Selenium WebDriver to run a JavaScript snippet which gives direct commands to iMacros component.