We Keep Coding

Selenium profile import -chrome [duplicate]

I am attempting to load a chrome browser with selenium using my existing account and settings from my profile.
I can get this working using ChromeOptions to set the userdatadir and profile directory. This loads the browser with my profile like i want, but the browser then hangs for 60 seconds and times out without advancing through any more of the automation.
If I don't use the user data dir and profile settings, it works fine but doesn't use my profile.
The reading I've done points to not being able to have more than one browser open at a time with the same profile so I made sure nothing was open while I ran the program. It still hangs for 60 seconds even without another browser open.
m_Options = new ChromeOptions();
m_Options.AddArgument("--user-data-dir=C:/Users/Me/AppData/Local/Google/Chrome/User Data");
m_Options.AddArgument("--profile-directory=Default");
m_Options.AddArgument("--disable-extensions");
m_Driver = new ChromeDriver(#"pathtoexe", m_Options);
m_Driver.Navigate().GoToUrl("somesite");
It always hangs on the GoToUrl. I'm not sure what else to try.

As per your code trials you were trying to load the Default Chrome Profile which will be against all the best practices as the Default Chrome Profile may contain either of the following:
Extensions
Bookmarks
Browsing History
etc
So the Default Chrome Profile may not be in compliance with you Test Specification and may raise exception while loading. Hence you should always use a customized Chrome Profile as below.
To create and open a new Chrome Profile you need to follow the following steps :
Open Chrome browser, click on the Side Menu and click on Settings on which the url chrome://settings/ opens up.
In People section, click on Manage other people on which a popup comes up.
Click on ADD PERSON, provide the person name, select an icon, keep the item Create a desktop shortcut for this user checked and click on ADD button.
Your new profile gets created.
Snapshot of a new profile SeLeNiUm
Now a desktop icon will be created as SeLeNiUm - Chrome
From the properties of the desktop icon SeLeNiUm - Chrome get the name of the profile directory. e.g. --profile-directory="Profile 2"
Get the absolute path of the profile-directory in your system as follows :
C:\\Users\\Thranor\\AppData\\Local\\Google\\Chrome\\User Data\\Profile 2
Now pass the value of profile-directory through an instance of ChromeOptions with AddArgument method along with key user-data-dir as follows :
m_Options = new ChromeOptions();
m_Options.AddArgument("--user-data-dir=C:/Users/Me/AppData/Local/Google/Chrome/User Data/Profile 2");
m_Options.AddArgument("--disable-extensions");
m_Driver = new ChromeDriver(#"pathtoexe", m_Options);
m_Driver.Navigate().GoToUrl("somesite");
Execute your Test
Observe Chrome gets initialized with the Chrome Profile as SeLeNiUm

If you want to run Chrome using your default profile (cause you need a extension), you need to run your script using another browser, like Microsoft Edge or Microsoft IE and your code will lunch a Chrome instance.
My Code in PHP:
namespace Facebook\WebDriver;
use Facebook\WebDriver\Remote\DesiredCapabilities;
use Facebook\WebDriver\Remote\RemoteWebDriver;
use Facebook\WebDriver\Chrome\ChromeOptions;
require_once('vendor/autoload.php');
$host = 'http://localhost:4444/';
$options = new ChromeOptions();
$options->addArguments(array(
'--user-data-dir=C:\Users\paulo\AppData\Local\Google\Chrome\User Data',
'--profile-directory=Default',
'--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36'
));
$caps = DesiredCapabilities::chrome();
$caps->setCapability(ChromeOptions::CAPABILITY, $options);
$caps->setPlatform("Windows");
$driver = RemoteWebDriver::create($host, $caps);
$driver ->manage()->window()->maximize();
$driver->get('https://www.google.com/');
// your code goes here.
$driver->quit();

i guys, in my enviroment with chrome 63 and selenum for control, i have find same problem (60 second on wait for open webpage).
To fix i have find a way by setting a default webpage in chrome ./[user-data-dir]/[Profile]/Preferences file, this is a json data need to insert in "Preferences" file for obtain result
...
"session":{
"restore_on_startup":4,
"startup_urls":[
"http://localhost/test1"
]
}
...
For set "Preferences" from selenium i have use this sample code
ChromeOptions chromeOptions = new ChromeOptions();
//set my user data dir
chromeOptions.addArguments("--user-data-dir=/usr/chromeDataDir/");
//start create data structure to for insert json in "Preferences" file
Map<String, Object> prefs = new HashMap<String, Object>();
prefs.put("session.restore_on_startup", 4);
List<String> urlList = new ArrayList<String>();
urlList.add("http://localhost/test1");
prefs.put("session.startup_urls", urlList);
//set in chromeOptions data structure
chromeOptions.setExperimentalOption("prefs", prefs);
//start chrome
ChromeDriver chromeDriver = new ChromeDriver(chromeOptions);
//this get command for open web page, response instant
chromeDriver.get("http://localhost/test2")
i have find information here https://chromedriver.chromium.org/capabilities

Selenium: Save and Load LocalStorage to/from File

I'm currently writing a script in python to tell me how many unread messages I have in WhatsApp.
To get the count of unread messages selenium opens web.whatsapp.com however I have to authenticate every time. I found out that WhatsApp saves the data to authenticate in the LocalStorage so I'm trying to figure out how I can save the contents from LocalStorage to a file and then later read from it and set all the keys.
I tried:
localStorage = driver.execute_script('return window.localStorage;')
print(localStorage)
but when I do that my terminal running the script just crashes.

Create a new user profile on your browser, activate it and login to web.whatsapp.com using the newly created profile, close the browser. Run the python script and initiate the webdriver using the new profile and you should still be logged in, i.e.:
The example below is for Firefox and web.whatsapp.com, but the general concept can be used on other browsers and websites.
1 - Type about:profiles on the browser url box an press enter
2 - Click Create a New Profile
3 - Choose a name and folder for the new profile (take note of the profile location), in this case : d:\ff_profiles\selenium_user
4 - Activate the new browser profile
5 - Login to any website that you want to skip the login process on selenium, in this case, web.whatsapp.com
6 - Once you've logged in successfully (after scanning the QR code) close the browser
7 - Using the profile on your script
from selenium import webdriver
fp = webdriver.FirefoxProfile('d:\\ff_profiles\\selenium_user')
driver = webdriver.Firefox(firefox_profile=fp)
driver.get("https://web.whatsapp.com")
# you should still be logged in.

How to use existing login token for telegram web using selenium webdriver

I'm trying to read telegram messages from https://web.telegram.org with selenium.
When i open https://web.telegram.org in firefox i'm already logged in, but when opening the same page from selenium webdriver(firefox) i get the login page.
I saw that telegram web is not using cookies for the auth but rather saves values in local storage. I can access the local storage with selenium and have keys there such as: "dc2_auth_key", "dc2_server_salt", "dc4_auth_key", ... but I'm not sure what to do with them in order to login(and if i do need to do something with them then why? its the same browser why wont it work the same when opening without selenium?)
To reproduce:
open firefox and login to https://web.telegram.org then run this code:
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("https://web.telegram.org")
# my code is here but is irrelevant since im at the login page.
driver.close()

When you open https://web.telegram.org manually using Firefox, the Default Firefox Profile is used. As you login and browse through the website, the websites stores Authentication Cookies within your system. As the cookies gets stored within the local storage of the Default Firefox Profile even on reopening the browsers you are automatically authenticated.
But when GeckoDriver initiates a new web browsing session for your tests everytime a temporary new mozprofile is created while launching Firefox which is evident from the following log:
mozrunner::runner INFO Running command: "C:\\Program Files\\Mozilla Firefox\\firefox.exe" "-marionette" "-profile" "C:\\Users\\ATECHM~1\\AppData\\Local\\Temp\\rust_mozprofile.fDJt0BIqNu0n"
You can find a detailed discussion in Is it Firefox or Geckodriver, which creates “rust_mozprofile” directory
Once the Test Execution completes and quit() is invoked the temporary mozprofile is deleted in the following process:
webdriver::server DEBUG -> DELETE /session/f84dbafc-4166-4a08-afd3-79b98bad1470
geckodriver::marionette TRACE -> 37:[0,3,"quit",{"flags":["eForceQuit"]}]
Marionette TRACE 0 -> [0,3,"quit",{"flags":["eForceQuit"]}]
Marionette DEBUG New connections will no longer be accepted
Marionette TRACE 0 <- [1,3,null,{"cause":"shutdown"}]
geckodriver::marionette TRACE <- [1,3,null,{"cause":"shutdown"}]
webdriver::server DEBUG Deleting session
geckodriver::marionette DEBUG Stopping browser process
So, when you open the same page using Selenium, GeckoDriver and Firefox the cookies which were stored within the local storage of the Default Firefox Profile aren't accessible and hence you are redirected to the Login Page.
To store and use the cookies within the local storage to get authenticated automatically you need to create and use a Custom Firefox Profile.
Here you can find a relevant discussion on webdriver.FirefoxProfile(): Is it possible to use a profile without making a copy of it?

You can auth using your current data from local storage.
Example:
driver.get(TELEGRAM_WEB_URL);
LocalStorage localStorage = ((ChromeDriver) DRIVER).getLocalStorage();
localStorage.clear();
localStorage.setItem("dc2_auth_key","<YOUR AUTH KEY>");
localStorage.setItem("user_auth","<YOUR USER INFO>");
driver.get(TELEGRAM_WEB_URL);

Can we change firefox profile settings via Selenium in flight? [duplicate]

ProfilesIni profile = new ProfilesIni();
FirefoxProfile ffprofile = profile.getProfile("default");//using firefox default profile
ffprofile.setPreference("permissions.default.image", 2); // this make ff to block web page images
WebDriver ff = new FirefoxDriver(ffprofile); // executing firefox with specified profile
ff.navigate().to("www.google.com"); // loading web page
//codes for changing image blocking ???????????
How can I change the image blocking after loading some web pages?

It is possible to modify preferences in flight via dev toolbar CLI but it may introduce higher overhead than loading images.
Here is Python example:
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains, Keys
ff = webdriver.Firefox()
ff.get('http//<URL>')
ac = ActionChains(ff)
# SHIFT+F2 opens dev toolbar
ac.key_down(Keys.SHIFT).send_keys(Keys.F2).key_up(Keys.SHIFT).perform()
# command to disable images
ac.send_keys('pref set permissions.default.image 2').perform()
ac.send_keys(Keys.ENTER).perform()
# command to disable flash
ac.send_keys('pref set plugin.state.flash 0').perform()
ac.send_keys(Keys.ENTER).perform()
# disable dev toolbar
ac.key_down(Keys.SHIFT).send_keys(Keys.F2).key_up(Keys.SHIFT).perform()
ac.key_down(Keys.SHIFT).send_keys(Keys.F2).key_up(Keys.SHIFT).perform()
# reload the page to confirm there are no images or flash
ff.refresh()

Since the dev toolbar CLI does not work for me (since the key combination does not open the CLI) here is how I managed to change profile settings of a running firefox instance:
IWebDriver driver = //your instance
driver.Navigate().GoToUrl("about:config");
driver.FindElement(By.Id("warningButton")).Click();
IJavaScriptExecutor js = (IJavaScriptExecutor)driver;
js.ExecuteScript("window.Services.prefs.setIntPref('permissions.default.image', " + 2 + ")");
It's C# but the conversion in Java should not be too hard.
The idea is that the about:config tab declares a lot of Javascript object allowing to change the profile settings, so we just have to go on that page and to execute some JS code.

Run the firefox from command line firefox.exe -p
After that create a new profile, set the neccessery settings and always call this profile.
FirefoxProfile ffprofile = profile.getProfile("profileName");

Make selenium grab all cookies

I was told to do a cookie audit of our front facing sites, now we own alot of domains, so Im really not going to manually dig through each one extracting the cookies. I decided to go with selenium. This works up till the point where I want to grab third party cookies.
Currently (python) I can do
driver.get_cookies()
For all the cookies that are set from my domain, but this doesn't give me any Google, Twitter, Vimeo, or other 3rd party cookies
I have tried modifying the cookie permissions in the firefox driver, but it doesn't help. Anyone know how I can get hold of tehm

Your question has been answered on StackOverflow here
Step 1: You need to download and install "Get All Cookies in XML" extension for Firefox from here (don't forget to restart Firefox after installing the extension).
Step2: Execute this python code to have Selenium's FirefoxWebDriver save all cookies to an xml file and then read this file:
from xml.dom import minidom
from selenium import webdriver
import os
import time
def determine_default_profile_dir():
"""
Returns path of Firefox's default profile directory
#return: directory_path
"""
appdata_location = os.getenv('APPDATA')
profiles_path = appdata_location + "/Mozilla/Firefox/Profiles/"
dirs_files_list = os.listdir(profiles_path)
default_profile_dir = ""
for item_name in dirs_files_list:
if item_name.endswith(".default"):
default_profile_dir = profiles_path + item_name
if not default_profile_dir:
assert ("did not find Firefox default profile directory")
return default_profile_dir
#load firefox with the default profile, so that the "Get All Cookies in XML" addon is enabled
default_firefox_profile = webdriver.FirefoxProfile(determine_default_profile_dir())
driver = webdriver.Firefox(default_firefox_profile)
#trigger Firefox to save value of all cookies into an xml file in Firefox profile directory
driver.get("chrome://getallcookies/content/getAllCookies.xul")
#wait for a bit to give Firefox time to write all the cookies to the file
time.sleep(40)
#cookies file will not be saved into directory with default profile, but into a temp directory.
current_profile_dir = driver.profile.profile_dir
cookie_file_path = current_profile_dir+"/cookie.xml"
print "Reading cookie data from cookie file: "+cookie_file_path
#load cookies file and do what you need with it
cookie_file = open(cookie_file_path,'r')
xmldoc = minidom.parse(cookie_file)
cookie_file.close()
driver.close()
#process all cookies in xmldoc object

Selenium can only get the cookies of the current domain:
getCookies
java.util.Set getCookies()
Get all the cookies for the current domain. This is the equivalent of
calling "document.cookie" and parsing the result
Anyway, I heard somebody used a Firefox plugin that was able to save all the cookies in XML. As far as I know, it is your best option.

Yes, I don't believe Selenium allows you to interact with cookies other than your current domain.
If you know the domains in question, then you could navigate to that domain but I assume this is unlikely.
It would be a massive security risk if you could access cookies cross site

You can get any cookie out of the browsers sqlite database file in the profile folder.
I added a more complete answer here:
Selenium 2 get all cookies on domain