ChromeDriver/Selenium take a black screenshot when in a separate tab - python

I'm currently writing a visual test that require adblock to be installed to stop some ads from ruining the test. However when I run the Chromedriver with the adblock extension installed it brings up the successfully installed adblock tab. This tab screws up my screenshots because the driver is working in a previous tab that opens. Is there any way to stop the adblock tab from opening or prevent the black screenshot from happening? Here is the initialization for the driver.
def setUp(self):
chrome_options = Options()
chrome_options.add_extension('AdBlock_v2.6.4.crx')
self.driver = webdriver.Chrome('chromedriver', port=0, chrome_options=chrome_options)
self.driver.implicitly_wait(15)

handles = self.driver.window_handles
self.driver.switch_to_window(handles[-1])
seems to be the best way to handle it. It simply tells the driver to switch to the tab which is considered a window. I feel silly for trying to find a more complex solution

Related

Is it possible to use Selenium in already opened Chrome?

This code currently used to manipulate Chrome..
option = Options()
option.add_argument("--incognito")
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=option)
But this code makes to open a new chrome window.
What I want is to manipulate the chrome that's already open, is it possible?
Somebody say, Using Chrome in debug mode is answer, but I don't think that's a good choice.
I want to open chrome without any control.

start Selenium chrome driver minimized (or equivalent)

I'm making a web scraping program, but to keep from being blocking by anti-scraping software I gotta keep the driver out of headless mode. sometimes I also need to restart the driver to clear the cookies, so whenever it opens back up I minimize it immediately, but it still gets in the way of whatever Im doing for about a second, and this program runs for hours so its incredibly annoying
Im thinking theres probably something I can add like driver.add_option("start in minimized") or driver.add_option("start in off screen") (equivalent to driver.set_window_position(-2000,0) but before the driver opens
does anyone know of an options setting I can add for this?
heres what my current code looks like, though this is more of a feature question than a bug fixing problem
import undetected_chromedriver.v2 as uc
def start_uc():
'''opens a UC chrome driver with prefered settings'''
options = uc.ChromeOptions()
# just some options passing in to skip annoying popups
options.add_argument('--no-first-run --no-service-autorun --password-store=basic')
return uc.Chrome(options=options)
driver = start_uc()
driver.minimize_window()
for item in list:
#scrape item info
if blocked from page:
driver.close()
driver = start_uc()
driver.minimize_window()
Unfortunately there is no such option / capability.
You can set driver to start maximized with
options.add_argument("start-maximized")
But there is no such option as options.add_argument("start-minimized").
As you mentioned, you can minimize the driver screen immediately after it's creating with driver.set_window_position(-2000,0) but, again, this will be applied only after the driver is opened.

How to use one selenium webdriver for multiple browsing tabs in one window and access them individually?

Based on what I've read in stackoverflow, it seems that I need to send keys to open tab and then open a particular website. However, it's quite complicated to do so and I ended up using multiple webdrivers because it's way simpler.
Here's my dummy code:
from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--incognito")
driver1 = webdriver.Chrome(executable_path=dirPath + "/chromedriver" + str(1),
options = chrome_options
driver2 = webdriver.Chrome(executable_path=dirPath + "/chromedriver" + str(2),
options = chrome_options
driver1.get("website1")
driver2.get("website2")
This then can be parallelized using multiprocessing. However it seems that this consumes a lot of RAM especially if I parallelize 10 webdrivers, each for one window. I was thinking of opening 10 tabs in 1 browser and then access each tab separately but got no solution. The goal is to open multiple tabs in one browser and use something like driver.get() in that tab without using keys.
Additional info: this setup crashed with seg fault 11 on pycharm but works fine in terminal
There's no built-in method for Selenium to open a new tab. However you can pass a keyboard shortcut like Ctrl + T to open a new tab
driver.find_element_by_css_name('body').send_keys(Keys.CONTROL + "T")
driver.switch_to.window(driver.window_handles[-1]) # This will switch the Selenium control to the current tab
But even this probably won't work in Chrome. I recommend you use some GUI automation tool like pyautogui to do it:
pyautogui.hotkey('ctrl', 't') # This will open a new tab
driver.switch_to.window(driver.window_handles[-1])
You can try both.

Can't open more than max number of tabs with Selenium Webdriver?

I am trying to open a list of different URLs, opening one per tab, but when the number exceeds 20 ~ 21, stop opening tabs.
I've tried to separate the list into groups of 20, and creating new instances of the webdriver, and that works fine, but I would like to know if it's a way to enable more number of tabs using the same instance?
from selenium import webdriver
import time
driver = webdriver.Firefox()
driver.get('https://stackoverflow.com/')
for i in range(30):
driver.execute_script("window.open('');")
print(len(driver.window_handles))
time.sleep(3)
driver.quit()
I was trying, to open 30 tabs at once but only opens 21.
I'm using python 3.5.0, Firefox 68.0.2 & geckodriver 0.24.0
If you look at the stackoverflow tab, you should see a yellow bar saying the rest has been blocked by the pop-up blocker. (This happens because execute_script runs the script in the context of the web page.)
To override, set dom.popup_maximum preference to a larger value:
opts = webdriver.FirefoxOptions()
opts.set_preference("dom.popup_maximum", 50)
driver = webdriver.Firefox(options=opts)
Please don't make use of "window.open()" to get new tabs or windows opened. Instead use the new WebDriver New Window API, which all the latest versions of the official Selenium bindings have been already integrated. Note, it's not part of all the drivers yet, but for recent Firefox releases it works.
Given that you are using the Python bindings the following can be used:
driver.switch_to.new_window('tab')
By using this approach there shouldn't be a limitation for opening a lot of tabs.

Using a headless Chrome instance as an OBS source

Would it be possible to use an instance of headless Chrome that is created by Selenium in Python3
Here is the code that i'm using to launch headless chrome.
options = Options()
options.set_headless(True)
driver_chrome = webdriver.Chrome(options=options)
driver_chrome.get("http://google.com/")
as a source in the broadcasting software OBS? I know that OBS is able to detect regular instances of non-headless chrome by using Window Capture, but I'm not sure how to apply this to an instance of headless Chrome.
If there is no immediate solution, would one be able to point me in the right direction in what steps to take that would allow this to happen?

Categories