I have 40 Python unit tests and each of them open a Selenium driver as they are separate files and cannot share the same driver.
from selenium import webdriver
webdriver.Firefox()
The above commands will take the focus to the new opened window. For example, if I am on my editor and typing something, in the middle of my work, suddenly a selenium browser is opening and Linux switch to that window. I am not sure if Windows or Mac have a similar problem or not.
This means that every time I run a unit, I cannot use my computer as it keeps switching away from the application that I am currently using.
How can I tell Selenium not to switch to the opened window?
Here is an example of running Selenium/Firefox on linux, in headless mode. You can see various imports as well - gonna leave them there. Browser will start in headless mode, will go to ddg page and print out the page source, then quit.
from selenium.common.exceptions import NoSuchElementException, TimeoutException
from selenium import webdriver
from selenium.webdriver.firefox.service import Service
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.firefox.options import Options as Firefox_Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support.ui import Select
from selenium.webdriver.support import expected_conditions as EC
firefox_options = Firefox_Options()
firefox_options.add_argument("--width=1280")
firefox_options.add_argument("--height=720")
firefox_options.headless = True
driverService = Service('chromedriver/geckodriver') ## path where geckodriver is
browser = webdriver.Firefox(service=driverService, options=firefox_options)
wait = WebDriverWait(browser, 20)
browser.get('https://duckduckgo.com')
print(browser.page_source)
browser.quit()
Browser can also take screenshots while in headless mode, if any unit test requires it.
A reasonably well structured documentation for Selenium can be found at https://www.selenium.dev/documentation/ (with some gaps, of course, but generally decent).
You can run your Selenium drivers in headless mode. This will prevent moving focus on opened by Selenium browsers and will not disturb you working on your PC machine.
Related
I am grinding through day 2 of me learning Python 3.7 with Selenium.
I am accessing a web page using WebDriver. I have been making progress, but am stymied now. Though I can easily disable the Firefox password manager popup window on my normal Browser (Options/Privacy and Security/Location/Settings), my script's remotely-run (think that is by definition) browser does not recognize that configuration, and the Firefox popup shows up.
The script can ignore the popup and navigate the target site until the very last page that I need to access. At that point, the HTML for that page is inaccessible, until I manually click on the Firefox popup, dismissing it. As soon as I do that, the HTML code for that web page lights up in Firefox Web Developer Inspector.
Now, that HTML code may be inaccessible for other reasons (like I said, day 2 of the learning curve), but is there some library or commands within Webdriver that allow me to automate the dismissal of that FireFox popup. It is not part of the HTML of any page, so I am at a loss.
Edit: I should mention also, the bulk of that last page's content is blank until I manually dismiss the FireFox popup.
I have added the following code, but still am getting the same popup:
from selenium import webdriver
#Using Firefox to access the Web
options = webdriver.FirefoxOptions()
options.set_preference("dom.webnotifications.enabled", False)
driver = webdriver.Firefox(options=options)
driver.maximize_window()
Second Edit: This is the current code section defining the profile, and I am still getting the pop up password manager.
import datetime
import time
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
#Using Firefox to access the Web
profile = webdriver.FirefoxProfile()
#profile.set_preference("dom.push.enabled", False)
profile.set_preference("dom.webnotifications.enabled", False)
profile.update_preferences()
driver = webdriver.Firefox(firefox_profile=profile)
driver.maximize_window()
about:config
preference security.insecure_field_warning.contextual.enabled to false
My code is running ok on some PCs, but there's one that it do not open a new private session, in this particular case, it loads the current chrome profile, in the same window that already is opened.
I tried looking on some stackoverflow questions to see if that happened to anybody else, but without success. Tried looking on this command line list "https://peter.sh/experiments/chromium-command-line-switches/" to see if something can help but I have no clue on how to make it work.
Even when there's no session opened, it loads the default user profile and nothing happens.
Selenium, chrome and chromedriver are up to date and with matching versions.
from selenium import webdriver
from selenium.webdriver.support.ui import Select
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait as wait
from selenium.common.exceptions import NoSuchElementException as NSEE
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--disable-gpu')
chrome_options.add_argument('--disable-extensions')
chrome_options.add_argument('--start-maximized')
browser = webdriver.Chrome(options=chrome_options)
You can explicitly pass the incognito flag to chrome before you instantiate the webdriver.
chrome_options.add_argument('-incognito')
I suspect this is only happening on machines that are already signed in on chrome.
I am doing a simple experiment with Amazon and Webdriver. However, using Webdriver Headless cannot find elements and errors out, but non-headless works.
Any suggestions how to get it working headless?
There is a comment right above the --headless flag.
from selenium import webdriver
import sys
import os
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
def get_inventory(url):
chrome_options = Options()
# Making it headless breaks. Commenting
# this line, making it non-headless works.
chrome_options.add_argument("--headless")
chrome_options.add_experimental_option(
"prefs", {'profile.managed_default_content_settings.javascript': 2})
chrome_options.binary_location = '/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary'
driver = webdriver.Chrome(executable_path=os.path.abspath(
'chromedriver'), chrome_options=chrome_options)
driver.set_window_size(1200, 1000)
try:
driver.get(url)
add_to_cart_button_xp = '//*[#id="add-to-cart-button"]'
add_to_cart_button = driver.find_element_by_xpath(add_to_cart_button_xp)
add_to_cart_button.click()
driver.get('https://www.amazon.com/gp/cart/view.html/ref=lh_cart')
qty_field_xp = '//div/input[starts-with(#name, "quantity.") and #type="text"]'
qty_field = driver.find_element_by_xpath(qty_field_xp)
qty_field.clear()
qty_field.send_keys("2")
update_link_xp = f'//input[#value="Update" and #type="submit"]'
update_link = driver.find_element_by_xpath(update_link_xp)
update_link.click()
url = 'https://www.amazon.com/Pexio-Professional-Stainless-Food-Safe-Dishwasher/dp/B07BGBSY9F'
get_inventory(url)
I think you just had some selector issues. I checked the elements and updated the quantity setting; everything else should be pretty much the same, aside from the binary locations.
from selenium import webdriver
import sys
import os
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
def get_inventory(url):
chrome_options = Options()
chrome_options.add_argument("--headless")
driver = webdriver.Chrome(
executable_path='/usr/bin/chromedriver',
chrome_options=chrome_options,
)
chrome_options.add_experimental_option(
"prefs",
{'profile.managed_default_content_settings.javascript': 2},
)
chrome_options.binary_location = '/usr/bin'
driver.set_window_size(1200, 1000)
try:
driver.get(url)
driver.save_screenshot("/tmp/x1.png")
driver.find_element_by_xpath('//*[#id="add-to-cart-button"]').click()
driver.get('https://www.amazon.com/gp/cart/view.html/ref=lh_cart')
driver.find_element_by_xpath("//span[#data-action='a-dropdown-button']").click()
driver.find_element_by_xpath("//*[#class='a-dropdown-link'][text()[contains(., '2')]]").click()
driver.find_element_by_class_name("nav-logo-base").click()
driver.save_screenshot("/tmp/confirm.png")
driver.close()
except Exception as e:
print(e)
url = 'https://www.amazon.com/Pexio-Professional-Stainless-Food-Safe-Dishwasher/dp/B07BGBSY9F'
get_inventory(url)
I've run this with and without --headless and it's working fine for me. I navigated to the homepage at the end so you can confirm the quantity change worked (hence the screenshot).
What is the behavior you see?
When I enabled headless, scripts started failing because running headless slows execution down.
I currently run chrome with these options:
'--no-sandbox', '--headless', '--window-size=1920,1080', '--proxy-server="direct://"', '--proxy-bypass-list=*'
The last two options supposedly help with the slowness, but I didn't see any difference.
Hope this helps.
I verified your claim on my Mac (using /Applications/Google Chrome.app/Contents/MacOS/Google Chrome).
My guess is that, since you are moving from an item page to the cart page of Amazon, the cookies are lost, so that the cart page won't show any item, and therefore won't contain any text input with a name starting with “quantity”, which is what the exception is about.
Googling for headless chrome cookies yields this page, which in turn points to this page, the content of which could also be about your problem. Be it this, or be it a particularly smart behavior of the Amazon website, the fact remains: the cookie that stores the cart (or a key thereof, but the result is the same) is not read by the cart page when in headless mode.
Trying to open "Google" or any other page (website) from Chrome via selenium chrome driver in python.
The code is :
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from time import sleep
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support.ui import WebDriverWait
import time
driver = webdriver.Chrome()
driver.get('https://google.com')
However, this opens my chrome window with the specified link and "data;" tab.
Why that data; tab opens? How to fix it?
Using latest versions of Chrome and Chromedriver
You don't need as much module for this just remove all of those apart from:
from selenium import webdriver
And try again you will not get another tab with congaing data.
import time
time.sleep(1)
driver.switch_to.window(driver.window_handles[1])
driver.close()
driver.switch_to.window(driver.window_handles[0])
time.sleep(1)
I'm not sure if its the same problem, but some time ago I made an exe script to run in another PCs and in one of the PCs selenium just didn't worked with Chrome.
This is the question I posted, but the answers didn't help me, hope it works with you: Chromedriver do not open a new session, it opens a new tab in a existing session
If it doesn't work, I made a workaround to run with Firefox instead of Chrome to ensure it would work properly.
With selenium 4 (or newer), you can use the following code to launch a new browser, (without the Chrome is being controlled message), and then the browser navigates to Google in the same tab:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service as ChromeService
options = webdriver.ChromeOptions()
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option("useAutomationExtension", False)
service = ChromeService()
driver = webdriver.Chrome(service=service, options=options)
driver.get('https://google.com')
I wrote a simple scraper sometime ago which opens up chrome browser and scrapes some data from a website. However now everytime I run that script it does not open the url I provide but instead redirects to GDPR consent website. I removed --incognito mode from options but it is still the same. The chrome opens then the script crashes because it is atuomatically redirected to that GDPR consent webpage.
How can I go around this issue?
Here is the code to reproduce the error.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
option = webdriver.ChromeOptions()
#option.add_argument("--incognito")
browser = webdriver.Chrome(executable_path='chromedriverpath', chrome_options=option)
rval=[]
browser.get("https://finance.yahoo.com/quote/AAPL/key-statistics?p=AAPL")
timeout = 10
WebDriverWait(browser, timeout)
values_element = browser.find_elements_by_xpath("//td[#class='Fz(s) Fw(500) Ta(end)']")
print(browser)
values = [x.text for x in values_element]
rval.append(values[8])
for title, value in zip(stockname, rval):
print(title + ': ' + value)
evdict=dict(zip(stockname, rval))
So to to bypass such popup window which block selenium from scraping the data I needed to add:
browser.find_element_by_xpath("//input[#type='submit' and #value='OK']").click()
Which would click on the proper button and close the window for me. Then Selenium works without further problems.