Selenium webdriver to make it lightweight (memory wise) without headless - python

I'm wondering if there's any option for selenium webdriver (chrome) to be run statically such that I can't interact with the browser and am only looking for the display. Similarly, is there any settings in "options" for chromedriver to make browsers very lightweight?
Lastly, would it be possible to live-stream the automated browser and display it somewhere else?

For a static non-interactive display have you considered a screenshot? That can open in a browser window or in any photo editor..
To decrease memory there are lots of options.
Have a look at this link for chrome command line options. If you search for memory there are 41 hits - should give you things you can try. Potentially if you disable some of the features that could decrease chrome's foot print.
The last bit you inquire about remotely viewing chrome in another location. When you launch Chrome add this option:
--remote-debugging-port=9222
The in another browser open up http://<IP_for_RemoteChrome>:9222/
See here for more info.

Related

How can I make Selenium Chromedriver anti-detectable settings work on tabs opened by webdriver?

In Selenium Chromedriver added proxy, fake user agent, disabled Webrtc by extension, spoofed Webgl with selenium-stealth, spoofed time zone and language as follows:
driver.execute_cdp_cmd('Emulation.setTimezoneOverride', tz_params)
options.add_experimental_option('prefs', {'intl.accept_languages': language})
so that on https://pixelscan.net/ my browser is not detected as a bot. But it works only on one(first) tab opened. On the website I work with the authorization window pops up, which does not save all the settings that are necessary for the browser to remain undetectable (of the above remains only the proxy and useragent)
How to make these parameters work on pop-up windows opened by a website?
I would also be glad if there are any alternative libraries that can help me achieve the desired result
I found a way out - an extension that changes my location and language depending on what country my ip address is. Unfortunately, such an extension for WebGL I have not found, only those that change drivers WebGL to random values (I need the values specified by me)
selenium-stealth settings is effective only on the first tab and the settings doesn't propogates to the subsequent tabs or window handles is a known issue for quite some time now.
This issue is being actively tracked through the discussion Settings doesn't apply on all tabs (but only on the first one)

One session per tab with Selenium Web-driver?

I would like to create a new session for each tab I open and then control the sessions individually using Selenium python. Is this possible?
#reynoldsnp: Firefox has an official addon that does this, but I'm not sure if you can get selenium to interact with the addon. addons.mozilla.org/en-GB/firefox/addon/multi-account-containers If you figure out a way to do it, I would love to know how.
(I can't comment yet due to my reputation score, therefore quoted comment).
I don't know how to actually interact with the extension but if you have a known set of sites you would like to open:
Try this:
Make a firefox profile for your use with selenium. Multiple profiles
Windows 8/8.1/10:
Press Win + R on your keyboard.
Type firefox --new-instance --ProfileManager
Open Firefox in that profile by selecting the new profile in the setup wizard. Install the extension in that profile.
Set up the containers you would like in the extension, to, by default, open up with a specific site.
Ensure that the checkbox is ticked.
Start selenium with that profile like this:
from selenium import webdriver
profile = webdriver.FirefoxProfile('path/to/your/profile') # on windows found here: %APPDATA%/Mozilla/Firefox/Profiles
driver = webdriver.Firefox(firefox_profile=profile)
Navigate between tabs, effectively containers, using selenuium.
First, no, you cannot. While tabs runs as a process, they are attached to the session ID which initially open the browser. This is how the protocol works https://www.w3.org/TR/webdriver/#new-session
They have, however, a unique ID which you can use to identify them by and switch between them.
driver.window_handles
will give you the list of open tabs. Each tab is fully isolated. You can now choose between
driver.switch_to_window("any open tab taken from windows handles list")
driver.do_something
driver.switch_to_window("any other tab from windows handles list")
driver.do_something_else_on_other_tab
# or (this option can let you run in parallel)
driver a = ChromeDriver()
driver b = ChromeDriver()
a.do_something
b.do_something
As suggested (and I personally do myself) open new session for each tab you want, that way you can parallel them and run much faster, all in all.
I am not sure the performance difference is that significant between multiple browsers or multiple tabs... they should use almost the same resources.

Open Application (such as zoom.us) with Selenium Webdriver

I want to be able to use pure selenium webdriver to open a zoom link in Chrome and then redirect me to the zoom.us application.
When I execute this:
from selenium import webdriver
def main():
driver = webdriver.Chrome()
driver.get("https://zoom.us/j/000-000-000")
main()
I receive a pop-up saying
https://zoom.us wants to open this application.
and I must press a button titled open zoom.us to open the app.
Is there a way to press this pop-up button through selenium. Or, is there some other way to open zoom from chromedriver?
NOTE: I only want to use selenium. I have been able to implement pyautogui to click on the button but that is not what I am looking for.
Solution for Java:
driver.switchTo().alert().accept();
Solution for Python:
driver.switch_to.alert.accept()
There are a lot of duplicated questions regarding this issue. Here is one of them, and it is quite sure that selenium is not capable of achieving such job since it only interacts with the chrome page. I previously encountered this issue as well and here is my solution to it. It might look really unprofessional, but fortunately it works.
The logic of my solution is to change the setting of chrome in order to skip the popup and directly open the application you want. However, the Chrome team has removed this feature in the latter version for some reasons, and we need to get it back manually. Then, we know that everytime when selenium starts to do the thing it opens a new Chrome page with NO customized settings just like the incognito page. Therefore we need to do something to let selenium opened a Chrome page with your customized setting, so that we can make sure that the popup, which we changed manually to skip, can be skipped successfully.
Type the following code in your terminal.
defaults write com.google.Chrome ExternalProtocolDialogShowAlwaysOpenCheckbox -bool true
This enables you to change the setting of skipping popups, which is the feature Chrome team removed.
Restart Chrome,and open the zoom (or whatever application) page to let the popup display. If you do the 1st step correctly you will be able to see there is a checkbox shown next to the "Open Zoom.us" saying if you check it chrome will open this application without asking, that is, to skip the popup for this application.
Now we need to let selenium open the Chrome with our customized setting. To do this, type "chrome://version" in the search tab of your ordinary Chrome (Not automated page opened by selenium). Go to "Profile Path", and copy this path without the last word "default". For example:
/Users/MYNAME/Library/Application Support/Google/Chrome/Default
This is my profile path, but I only copy everything except the last word Default, so this is what I need to copy.
/Users/MYNAME/Library/Application Support/Google/Chrome/
This is for Mac users, but for Windows only the path is different(starts with C:// or something), steps are same.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
option = Options()
option.add_argument('THE PATH YOU JUST COPIED')
driver = webdriver.Chrome(executable_path='YOUR PATH TO CHROMEDRIVER', options=option)
driver.get("google.com") #Or anything else
We use "options" to let selenium open a page with our customized profile. Now you will see selenium opens a Chrome page with all your account profile, settings, and it just appears like your ordinary chrome page.
Run your code. But before that, remember to quit ALL CHROME sessions manually. For Mac, make sure that there is no dot under Chrome icon indicating that Chrome is not running for any circumstances. THIS STEP IS CRITICAL otherwise selenium will open a chrome page and it just stops there.
Here are all the steps. Again, this solution is vert informal and I personally don't think it is a "solution" to this problem. I will try to figure out a better way of achieving this in the future. But I still posted this as an alternative simply because I guess it might be helpful to some extent for somebody just like me. Hope it works for you, and good luck.

Where can I find a list of all available ChromeOptions with selenium?

Where can I find a list of all available Chrome Options with selenium?
While crawling with selenium in a python environment, I had to use flash in the chrome webdriver.
so I tried to find a setting that would allow Flash and I found these answers.
https://sqa.stackexchange.com/questions/30312/enable-flash-player-on-chrome-62-while-running-selenium-test
Selenium.Chrome where can I find a list of all available ChromeOption arguments? C#
but I really want to find official document
about these arguments :
profile.default_content_setting_values.plugins
profile.content_settings.plugin_whitelist.adobe-flash-player
I think that exist because some people said like above two answers.
but I can't found it.
Here are the links of the most widely used ChromeDriver properties and ChromeOptions:
ChromeDriver - WebDriver for Chrome: Capabilities & ChromeOptions
Peter Beverloo: List of Chromium Command Line Switches
Chromium Project: Chrome Preferences
Chromium Project: Chrome Switches
open chrome://version in chrome address bar
open the folder of Profile Path:, there will be a file perference
copy and open the perference file, it's a JSON file, you can use JSON formater or JSON viewer online.
important, please open the copied one to avoid break your chrome if you did changes.
I think the configuration related to flash under profile.content_settings
open chrome://settings/content/flash in chrome address bar, do waht you wanted settings for flash, your changes will be updated into perference file,
Find out the difference between the updated and copied perference file.
The difference should be you wanted chrome perference.
set perference when build ChromeOptions, like profile.content_settings.xxx=xxx
Alternative, you can pre-config the flash settings(chrome://settings/content/flash) in current user profile and use user profile to build ChromeOptions.
Some reference:
Manage Flash in Chrome
Default user preferences
Preferences vs. Policies
Policy List
Policy List - DefaultPluginsSetting
Policy more close to using registry/system level setting to effect all users in one machine; Preference work under user profile only effect one user, user can use it to customize upon Policy settings. But Policy has higher priority than Preference.

Can't turn off images in Selenium / Firefox

I am looking to disable images in Firefox when using Selenium. It should be a simple update of the preferences in firefox, which is documented on the instructions on Disable images in Selenium Python
However when i run, images display, and when i enter about:config, the value for permissions.default.image is still 1, rather than 2 which i have tried setting it to.
My code (written in Python) is:
from selenium import webdriver
firefox_profile = webdriver.FirefoxProfile()
firefox_profile.set_preference("permissions.default.image", 2)
driver = webdriver.Firefox(firefox_profile)
driver.get(web_address)
For reference, this code works perfect with another change to preference e.g. turning off csv files with the line firefox_profile.set_preference("permissions.default.stylesheet",2). The only difference i can tell between the csv setting and the image one, is that the line permissions.default.image already exists in about:config (i.e. without me setting it), however the line permissions.default.stylesheet does not. ... it seems that i can add new lines in with the value i want, but not change an existing one (or it is beein over-ridden by Selenium after i enter my value).
From what I understand, this problem is related to the following Firefox issues:
Remove "Load images automatically" checkbox from Prefs
Get rid of options that kill our product
That means that permissions.default.image is frozen, cannot be changed and does nothing.
Alternatives:
use Image Block extension
switch to Chrome (Disable images in Selenium ChromeDriver)
I had this problem. the solution is bellow in 3 steps.
1- creating new profile for Firefox. in Windows completely close Firefox. press (Window+R) , write firefox.exe -p then press enter and create a new profile.
2-open Firefox with the created profile then open about:config in navigation bar and find permissions.default.image and make it's number 2.
3-change your code like bellow
ProfilesIni profile = new ProfilesIni();
FirefoxProfile myprofile = profile.getProfile("your_profile_name");
WebDriver driver = new FirefoxDriver(myprofile);
I've created a code sample for disabling images.
#DISABLE IMAGES ON FIREFOX
def disable_images(driver):
driver.get("about:config")
driver.find_element("id","warningButton").click()
searchArea=driver.find_element("id","about-config-search")
searchArea.send_keys("permissions.default.image")
editButton=driver.find_element("xpath","/html/body/table/tr[1]/td[2]/button")
editButton.click()
editArea=driver.find_element("xpath","/html/body/table/tr[1]/td[1]/form/input")
editArea.send_keys("2")
saveButton=driver.find_element("xpath","/html/body/table/tr[1]/td[2]/button")
saveButton.click()

Categories