We Keep Coding

Is it possible to maintain login session in selenium-python?

I use Selenium below method.
open chrome by using chromedriver selenium
manually login
get information of webpage
However, after doing this, Selenium seems to get the html code when not logged in.
Is there a solution?

Try this code:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.chrome.options import Options
options = Options()
# path of the chrome's profile parent directory - change this path as per your system
options.add_argument(r"user-data-dir=C:\\Users\\User\\AppData\\Local\\Google\\Chrome\\User Data")
# name of the directory - change this directory name as per your system
options.add_argument("--profile-directory=Default")
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=options)
You can get the chrome profile directory by passing this command - 'chrome://version/' in chrome browser.
Add the code for login after the above code block, then if you execute the code for the second time onwards you can see the account is already logged in.
Before running the code close all the chrome browser windows and execute.

Instead of storing and maintaining the login session another easy approach would be to use pickle library to store the cookies post login.
As an example to store the cookies from Instagram after logging in and then to reuse them you can use the following solution:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
import pickle
# first login
driver.get('http://www.instagram.org')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input[name='username']"))).send_keys("_SelmanFarukYılmaz_")
driver.find_element(By.CSS_SELECTOR, "input[name='password']").send_keys("Selman_Faruk_Yılmaz")
driver.find_element(By.CSS_SELECTOR, "button[type='submit'] div").click()
pickle.dump( driver.get_cookies() , open("cookies.pkl","wb"))
driver.quit()
# future logins
driver = webdriver.Chrome(service=s, options=options)
driver.get('http://www.instagram.org')
cookies = pickle.load(open("cookies.pkl", "rb"))
for cookie in cookies:
driver.add_cookie(cookie)
driver.get('http://www.instagram.org')

Selenium login via microsoft

I am trying to automate some data scraping from an app. Whole scraping thing works just fine but the problem is that log in is through microsoft and it is also using MS authentication so as far as my knowledge goes, it cant be fully automated.
The thing is that I have to log in only if I'm accessing this app through selenium, and while accessing it manually there's no need to log in (app just opens).
I'm wondering if there is a way to make selenium not ask me to log in every time?
(I'm using chromium driver)

Set user profile, so you can login as that user.
Example:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = webdriver.ChromeOptions()
options.add_argument(r"--user-data-
dir=C:\path\to\chrome\user\data") #e.g. C:\Users\You\AppData\Local\Google\Chrome\User Data
options.add_argument(r'--profile-directory=YourProfileDir') #e.g. Profile 3
driver = webdriver.Chrome(executable_path=r'C:\path\to\chromedriver.exe', chrome_options=options)
driver.get("https://www.google.co.in")

While accessing the website manually as you are logged in by default you can avoid logging in everytime by storing the cookies during first time logging in using pickle and reuse during your next login attempts as follows:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
import pickle
driver.execute("get", {'url': 'https://example.com/'})
# login using valid credentials
pickle.dump( driver.get_cookies() , open("cookies.pkl","wb"))
driver.quit()
driver = webdriver.Chrome(service=s, options=options)
driver.execute("get", {'url': 'https://example.com/'})
cookies = pickle.load(open("cookies.pkl", "rb"))
for cookie in cookies:
driver.add_cookie(cookie)
driver.execute("get", {'url': 'https://example.com/'})

Python Selenium Use User Profile Chrome

I will do a project by using selenium. But there is a problem for me. I have to use chrome with my settings. My websites login, my history...
But when I use the selenium, It creates a new chrome browser that with a default settings. No Websites login here and others.
My code:
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
driver =webdriver.Chrome(ChromeDriverManager().install())
How can i use my current chrome settings or how can I change webdriver by location.
For example(maybe):
driver =webdriver.Chrome(location="C\\Users\\Desktop\\chrome.exe)

I fix my problem with this way:
from selenium import webdriver
import time
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=C:\\Users\\Fatih\\AppData\\Local\\Google\\Chrome\\User Data")
options.add_argument('--profile-directory=Default')
driver = webdriver.Chrome(executable_path="chromedriver.exe", options=options)
my "chromedriver.exe" is in same path with my python files.

Chromedriver do not open a new session, it opens a new tab in a existing session

My code is running ok on some PCs, but there's one that it do not open a new private session, in this particular case, it loads the current chrome profile, in the same window that already is opened.
I tried looking on some stackoverflow questions to see if that happened to anybody else, but without success. Tried looking on this command line list "https://peter.sh/experiments/chromium-command-line-switches/" to see if something can help but I have no clue on how to make it work.
Even when there's no session opened, it loads the default user profile and nothing happens.
Selenium, chrome and chromedriver are up to date and with matching versions.
from selenium import webdriver
from selenium.webdriver.support.ui import Select
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait as wait
from selenium.common.exceptions import NoSuchElementException as NSEE
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--disable-gpu')
chrome_options.add_argument('--disable-extensions')
chrome_options.add_argument('--start-maximized')
browser = webdriver.Chrome(options=chrome_options)

You can explicitly pass the incognito flag to chrome before you instantiate the webdriver.
chrome_options.add_argument('-incognito')
I suspect this is only happening on machines that are already signed in on chrome.

Headless webdriver returns error but non-headless works

I am doing a simple experiment with Amazon and Webdriver. However, using Webdriver Headless cannot find elements and errors out, but non-headless works.
Any suggestions how to get it working headless?
There is a comment right above the --headless flag.
from selenium import webdriver
import sys
import os
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
def get_inventory(url):
chrome_options = Options()
# Making it headless breaks. Commenting
# this line, making it non-headless works.
chrome_options.add_argument("--headless")
chrome_options.add_experimental_option(
"prefs", {'profile.managed_default_content_settings.javascript': 2})
chrome_options.binary_location = '/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary'
driver = webdriver.Chrome(executable_path=os.path.abspath(
'chromedriver'), chrome_options=chrome_options)
driver.set_window_size(1200, 1000)
try:
driver.get(url)
add_to_cart_button_xp = '//*[#id="add-to-cart-button"]'
add_to_cart_button = driver.find_element_by_xpath(add_to_cart_button_xp)
add_to_cart_button.click()
driver.get('https://www.amazon.com/gp/cart/view.html/ref=lh_cart')
qty_field_xp = '//div/input[starts-with(#name, "quantity.") and #type="text"]'
qty_field = driver.find_element_by_xpath(qty_field_xp)
qty_field.clear()
qty_field.send_keys("2")
update_link_xp = f'//input[#value="Update" and #type="submit"]'
update_link = driver.find_element_by_xpath(update_link_xp)
update_link.click()
url = 'https://www.amazon.com/Pexio-Professional-Stainless-Food-Safe-Dishwasher/dp/B07BGBSY9F'
get_inventory(url)

I think you just had some selector issues. I checked the elements and updated the quantity setting; everything else should be pretty much the same, aside from the binary locations.
from selenium import webdriver
import sys
import os
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
def get_inventory(url):
chrome_options = Options()
chrome_options.add_argument("--headless")
driver = webdriver.Chrome(
executable_path='/usr/bin/chromedriver',
chrome_options=chrome_options,
)
chrome_options.add_experimental_option(
"prefs",
{'profile.managed_default_content_settings.javascript': 2},
)
chrome_options.binary_location = '/usr/bin'
driver.set_window_size(1200, 1000)
try:
driver.get(url)
driver.save_screenshot("/tmp/x1.png")
driver.find_element_by_xpath('//*[#id="add-to-cart-button"]').click()
driver.get('https://www.amazon.com/gp/cart/view.html/ref=lh_cart')
driver.find_element_by_xpath("//span[#data-action='a-dropdown-button']").click()
driver.find_element_by_xpath("//*[#class='a-dropdown-link'][text()[contains(., '2')]]").click()
driver.find_element_by_class_name("nav-logo-base").click()
driver.save_screenshot("/tmp/confirm.png")
driver.close()
except Exception as e:
print(e)
url = 'https://www.amazon.com/Pexio-Professional-Stainless-Food-Safe-Dishwasher/dp/B07BGBSY9F'
get_inventory(url)
I've run this with and without --headless and it's working fine for me. I navigated to the homepage at the end so you can confirm the quantity change worked (hence the screenshot).

What is the behavior you see?
When I enabled headless, scripts started failing because running headless slows execution down.
I currently run chrome with these options:
'--no-sandbox', '--headless', '--window-size=1920,1080', '--proxy-server="direct://"', '--proxy-bypass-list=*'
The last two options supposedly help with the slowness, but I didn't see any difference.
Hope this helps.

I verified your claim on my Mac (using /Applications/Google Chrome.app/Contents/MacOS/Google Chrome).
My guess is that, since you are moving from an item page to the cart page of Amazon, the cookies are lost, so that the cart page won't show any item, and therefore won't contain any text input with a name starting with “quantity”, which is what the exception is about.
Googling for headless chrome cookies yields this page, which in turn points to this page, the content of which could also be about your problem. Be it this, or be it a particularly smart behavior of the Amazon website, the fact remains: the cookie that stores the cart (or a key thereof, but the result is the same) is not read by the cart page when in headless mode.