I'm trying to make a list of headless Chrome webdrivers the following way:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--disable-gpu')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
options.add_argument('--log-level=3')
browsers = {}
def add_browser(browser_id):
browsers[browser_id] = webdriver.Chrome(executable_path=chromedriver, options=options)
browsers[browser_id].get(URL)
However, I would like to close browsers that have been active and idle for too long. How may I implement that?
If it helps - this is being used for a Flask app.
from datetime import datetime
def add_browser(browser_id):
browsers[browser_id] = {
browser: webdriver.Chrome(executable_path=chromedriver, options=options),
last_active: datetime.now()
}
browsers[browser_id][browser].get(URL)
// do some stuff... scrape links? navigate through pages? input text?
Then when writing your script in the //do some stuff section, you can now do one of two things:
// "check in" to confirm the session is still active
browsers[browser_id][last_active] = datetime.now()
// during a loop that you are worried about getting stuck in due to the browser being idle
idle_time = datetime.now() - browsers[browser_id][last_active]
if idle_time.seconds > maximum_idle_time:
browsers[browser_id][browser].quit()
Related
I want to keep Firefox Login status when using Selenium WebDriver (geckodriver). I used below python code:
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
import time
try:
fp = webdriver.FirefoxProfile('C:/Users/admin/AppData/Roaming/Mozilla/Firefox/Profiles/mow8ff69.selenium')
options = Options()
options.headless = False
browser = webdriver.Firefox(fp, options=options)
browser.get('https://stackoverflow.com/users/login');
time.sleep(2)
txt_username = browser.find_element_by_id('email')
txt_username.clear()
txt_username.send_keys('***#gmail.com') #replace with your own username
time.sleep(1)
txt_password = browser.find_element_by_id('password')
txt_password.clear()
txt_password.send_keys('*****') #replace with your own password
time.sleep(1)
txt_password.submit()
time.sleep(10)
except Exception as ex:
print('Error: {0}'.format(ex))
finally:
browser.quit()
As shown above, I have been using a User Profile created by "Firefox -p". However, after running above code, when I reopen Firefox using that profile and then go to stackoverflow.com, I'm still not logged in.
So it looks like that Selinum WebDriver actually runs firefox in a "sandbox" which cannot write to the User Profile at all? I know that ChromeDriver has a "--no-sandbox" option but it doesn't seem to be available in geckodriver? Is there any workaround?
I cannot switch to ChromeDriver because there is another bug with it: the "--headless" option actually conflicts with "--user-data-dir" option whereas I really need both.
So I want to store a Whatsapp Web session in order to not having to scan Whatsapp Web's QR-Code every time. I did it with the following code:
options = webdriver.ChromeOptions()
options.add_argument("--user-data-dir=C:/Users/Pascal/AppData/Local/Google/Chrome/User Data")
browser = webdriver.Chrome(executable_path="C:/Users/Pascal/Desktop/chromedriver.exe", options = options)
browser.get("https://web.whatsapp.com/")
The above code worked perfectly (Chromebrowser) but the following code which is almost the same does not work:
options = webdriver.FirefoxOptions()
options.add_argument("--user-data-dir=C:/Users/Pascal/AppData/Roaming/Mozilla/Firefox/Profiles/iddwgmst.default-release")
browser = webdriver.Firefox(executable_path="C:/Users/Pascal/Desktop/geckodriver.exe", options = options)
browser.get("https://web.whatsapp.com/")
Why does it not work with firefox? The QR-Code comes up every time but I have loaded the firefox profile to the browser/driver so it seems like firefox does not store whatsapp webs data... But then, if I go into whatsapp web in my normal firefox browser, it stores the data again and I don't have to rescan... I am confused by this problem.
I really want it with firefox to be working cause chromedriver does not support emojis :/
Any Ideas?
I solved it.
For Firefox it works with:
profile = webdriver.firefox.firefox_profile.FirefoxProfile("C:/Users/Pascal/AppData/Roaming/Mozilla/Firefox/Profiles/iddwgmst.default-release")
self.browser = webdriver.Firefox(executable_path = os.path.dirname(os.path.realpath(__file__)) + "\\geckodriver.exe" , firefox_profile = profile)
self.browser.get("https://web.whatsapp.com/")
But for chrome it works with:
options = webdriver.ChromeOptions()
options.add_argument("--user-data-dir=C:/Users/Pascal/AppData/Local/Google/Chrome/User Data")
self.browser = webdriver.Chrome(executable_path = os.path.dirname(os.path.realpath(__file__)) + "\\chromedriver.exe", options = options)
self.browser.get("https://web.whatsapp.com/")
Here, geckodriver and chromedriver are in the same folder as the main.py
the issue am dealing with is trying to get selenium to run in the background while getting data like webpage elements xpath and ids and being able to use it while remaining activity in the background and not keep poping up browser tab in front of other running programs
You should try running your browser in headless mode. Here is a code snippet of the function that gives the instance of chrome in headless mode.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
def browser_open(headless=False):
options = Options()
options = webdriver.ChromeOptions()
options.add_argument("disable-gpu")
options.add_argument("no-default-browser-check")
options.add_argument("no-first-run")
options.add_argument("no-sandbox")
options.add_argument("window-size=1300x744")
if headless == True:
options.add_argument("headless")
chrome_browser = webdriver.Chrome(executable_path=os.path.join(os.getcwd(), "chromedriver"), chrome_options=options)
chrome_browser.maximize_window()
return chrome_browser
This is my code so far:
username_input = "username"
password_input = "password"
url='myurl'
browser = webdriver.Chrome(r'chromedriver.exe')
browser.get(url)
browser.maximize_window()
username = browser.find_element_by_id("j_username")
password = browser.find_element_by_id("j_password")
username.send_keys(str(username_input))
password.send_keys(str(password_input))
browser.find_element_by_xpath('//*[#id="inner-box"]/form/label[3]/input').click()
time.sleep(2)
Once I have logged in everything is in French but I need it in English.. how do I do this?
I have tried several things such as Chrome Options but didn't understand it/wasn't working.
Any help will be appreciated.
add prefs below to auto translate french to english
options = Options()
prefs = {
"translate_whitelists": {"fr":"en"},
"translate":{"enabled":"true"}
}
options.add_experimental_option("prefs", prefs)
browser = webdriver.Chrome(chrome_options=options)
you can remove r'chromedriver.exe' if the location is in same folder with your script.
The correct solution is:
from selenium import webdriver
chrome_path = "D:\chromedriver_win32\chromedriver"
custom_options = webdriver.ChromeOptions()
prefs = {
"translate_whitelists": {"ru":"en"},
"translate":{"enabled":"true"}
}
custom_options.add_experimental_option("prefs", prefs)
driver=webdriver.Chrome(chrome_path, options=custom_options)
I suppose you have to set up Chrome options like:
chrome_options = Options()
chrome_options.add_argument("--lang=en")
The change of webpage language is determined by the browser settings. I tried practically almost all the strategies discussed and mentioned in the forum, but none of them worked for me. I was able to successfully achieve it by following the instructions outlined below.
Create a new Chrome profile (i.e. Profile 2). Then move the new profile directory in the "Documents" directory of "Users"
Now open Google Chrome (from new profile), "run as administrator" mode > open www.google.com > at the bottom of the page, click on "setting" > now click on "search setting" > select the "region setting" > select "United Kingdom" for opening the webpage only in English language.
Now follow the following java code snippet.
System.setProperty("webdriver.chrome.driver", "C:\\Testing Work Space\\chromedriver.exe");
// Chrome actual new profile path is "C:\Users\shah\Documents\Profile 2\"
// but you have to keep the chromeProfilePath till "\Documents\" as follows
String chromeProfilePath = "C:\\Users\\shah\\Documents\\";
ChromeOptions chroOption = new ChromeOptions();
chroOption.addArguments("user-data-dir=" + chromeProfilePath);
// Here you specify the new Chrome profile folder (Profile 2)
chroOption.addArguments("profile-directory=Profile 2");
WebDriver driver = new ChromeDriver(chroOption);
driver.get("https://facebook.com");
All other answers not working for me. And I found bruteforce solution:
import pyautogui
from selenium import webdriver
from selenium.webdriver import ActionChains
from selenium.webdriver.chrome.options import Options
url = 'https://ifconfig.me/'
options = Options()
options.add_argument('--lang=fr') # set your language here
browser = webdriver.Chrome(options=options)
browser.get(url)
actionChains = ActionChains(browser)
actionChains.context_click().perform()
# here maybe problem. Debug it:
for i in range(3):
pyautogui.sleep(1)
pyautogui.press('up')
pyautogui.press('enter')
As of September 2022, It seems like setting prefs doesn't work for chrome version 105. In order to solve this, you can either downgrade your chrome to version 95 or use selenium standalone docker. For the docker approach, You should pull and run standalone docker using:
docker pull selenium/standalone-chrome:95.0-chromedriver-95.0-20211102
docker run -d -p 4444:4444 --shm-size="2g" selenium/standalone-chrome:95.0-chromedriver-95.0-20211102
Then in your code use remoteDriver and apply prefs like this:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument('--lang=en')
prefs = {
"translate_whitelists": {"es": "en"},
"translate": {"enabled": "true"}
}
options.add_experimental_option("prefs", prefs)
driver = webdriver.Remote(command_executor='http://localhost:4444/wd/hub', options=options)
driver.get("https://www.amazon.es/")
I am trying to submit information in a webpage, but selenium throws this error:
UnexpectedAlertPresentException: Alert Text: This page is asking you
to confirm that you want to leave - data you have entered may not be
saved. ,
>
It's not a leave notification; here is a pic of the notification -
.
If I click in never show this notification again, my action doesn't get saved; is there a way to save it or disable all notifications?
edit: I'm using firefox.
You can disable the browser notifications, using chrome options. Sample code below:
chrome_options = webdriver.ChromeOptions()
prefs = {"profile.default_content_setting_values.notifications" : 2}
chrome_options.add_experimental_option("prefs",prefs)
driver = webdriver.Chrome(chrome_options=chrome_options)
With the latest version of Firefox the above preferences didn't work.
Below is the solution which disable notifications using Firefox object
_browser_profile = webdriver.FirefoxProfile()
_browser_profile.set_preference("dom.webnotifications.enabled", False)
webdriver.Firefox(firefox_profile=_browser_profile)
Disable notifications when using Remote Object:
webdriver.Remote(desired_capabilities=_desired_caps, command_executor=_url, options=_custom_options, browser_profile=_browser_profile)
selenium==3.11.0
Usually with browser settings like this, any changes you make are going to get throws away the next time Selenium starts up a new browser instance.
Are you using a dedicated Firefox profile to run your selenium tests? If so, in that Firefox profile, set this setting to what you want and then close the browser. That should properly save it for its next use. You will need to tell Selenium to use this profile though, thats done by SetCapabilities when you start the driver session.
This will do it:
from selenium.webdriver.firefox.options import Options
options = Options()
options.set_preference("dom.webnotifications.enabled", False)
browser = webdriver.Firefox(firefox_options=options)
For Google Chrome and v3 of Selenium you may receive "DeprecationWarning: use options instead of chrome_options", so you will want to do the following:
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
options = webdriver.ChromeOptions()
options.add_argument('--disable-notifications')
driver = webdriver.Chrome(ChromeDriverManager().install(), options=options)
Note: I am using webdriver-manager, but this also works with specifying the executable_path.
This answer is an improvement on TH Todorov code snippet, based on what is working as of Chrome (Version 80.0.3987.163).
lk = os.path.join(os.getcwd(), "chromedriver",) --> in this line you provide the link to the chromedriver, which you can download from chromedrive link
import os
from selenium import webdriver
lk = os.path.join(os.getcwd(), "chromedriver",)
chrome_options = webdriver.ChromeOptions()
prefs = {"profile.default_content_setting_values.notifications" : 2}
chrome_options.add_experimental_option("prefs",prefs)
driver = webdriver.Chrome(lk, options=chrome_options)