We Keep Coding

how to handle website which req. Domain Validated SSL certificate to login using selenium

I want to know how can I select Domain Validated SSL certificate to login into a website. While I run selenium code, I'm unable to accept the only displayed certificate. What is the best way to handle such cases.

Try to use acceptInsecureCerts capability:
options = webdriver.ChromeOptions()
options.set_capability("acceptInsecureCerts",True)
driver = webdriver.Chrome(options=options)

Adding proxy support to Selenium multi session opening script in Python

I recently made a script that allow me to create multiple Browser sessions targeting one URL. I would like to add proxy support to it in order to not get banned when running it. I tried to use the Proxy lib from selenium but it just get ignored.
My Question : How can I add proxy support into this script while using Selenium in python ? (each session will get a random proxy)
Here is my code

You could use the stem library which allows you to use Tor in python. Read the docs here to see how to use it.
The two basic parts missing from your code are the following:
from selenium.webdriver.common.proxy import Proxy, ProxyType
chrome_options.add_argument('--proxy-server=#yourproxyhere#'
Tor!
Here you can see how I set up my stem + selenium project:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.proxy import Proxy, ProxyType
from time import sleep
from stem import Signal
from stem.control import Controller
#this gives you a new identity
with Controller.from_port(port = 9051) as controller:
controller.authenticate()
controller.signal(Signal.NEWNYM)
#set the proxy in selenium to 127.0.0.1:9150 and have your Tor Browser open!
link = 'https://some-link.com' #target url
prox= 'socks5://127.0.0.1:9150' #Here you connect to your localhost which connects to a Tor network
#some chrome_options
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--proxy-server=%s' % prox)
chrome_options.add_argument("--window-size=400,600")
#the following also deactivates location tracking!
prefs = {"profile.default_content_setting_values.geolocation" :2}
chrome_options.add_experimental_option("prefs",prefs)
driver = webdriver.Chrome("path_to_chromedriver", chrome_options=chrome_options)
driver.get(link)

Here is an updated version of my code including the new proxy support elements, I want it to use proxies form a txt file via a proxies = read_from_txt("proxies.txt") and randomly rotate between them using random lib :
Thank you for quick replies, really appreciate it.

Find out the proxy server used by WebDriver

I run selenium WebDriver with a proxy.
chrome_options.add_argument('--proxy-server="'94.242.58.108:1448'"
driver = Chrome(chrome_options=chrome_options)
Driver know which proxy server it is using now.
Could you tell me how to find out the used proxy server?
I need this information in another function.

See the java doc http://seleniumhq.github.io/selenium/docs/api/java/org/openqa/selenium/Proxy.html
we have getHttpProxy() to retrieve the proxy.
Proxy p=new Proxy(); // used at the start to set proxy and pass to chrome options and driver
System.out.println(p.getHttpProxy()); // to get proxy when required.

Setting up proxy with selenium / python

I am using selenium with python.
I need to configure a proxy.
It is working for HTTP but not for HTTPS.
The code I am using is:
# configure firefox
profile = webdriver.FirefoxProfile()
profile.set_preference("network.proxy.type", 1)
profile.set_preference("network.proxy.http", '11.111.11.11')
profile.set_preference("network.proxy.http_port", int('80'))
profile.update_preferences()
# launch
driver = webdriver.Firefox(firefox_profile=profile)
driver.get('https://www.iplocation.net/find-ip-address')
Also. Is there a way for me to completely block any outgoing traffic from my IP and restrict it ONLY to the proxy IP so that I don't accidently mess up the test/stats by accidently switching from proxy to direct connection?
Any tips would help!
Thanks :)

Check out browsermob proxy for setting up a proxies for use with selenium
from browsermobproxy import Server
server = Server("path/to/browsermob-proxy")
server.start()
proxy = server.create_proxy()
from selenium import webdriver
profile = webdriver.FirefoxProfile()
profile.set_proxy(proxy.selenium_proxy())
driver = webdriver.Firefox(firefox_profile=profile)
proxy.new_har("google")
driver.get("http://www.google.co.uk")
proxy.har # returns a HAR JSON blob
server.stop()
driver.quit()
You can use a remote proxy server with the RemoteServer class.
Is there a way for me to completely block any outgoing traffic from my IP and restrict it ONLY to the proxy IP
Yes, just look up how to setup proxies for whatever operating system you're using. Just use caution because some operating systems will ignore proxy rules based on certain conditions, for example, if using a VPN connection.

Webdriver phantomjs no longer following link on click

I use a simple webdriver phantomjs script to update some adverts on preloved.co.uk. This script worked great until recently, but then started failing with the "Click submitted but load failed" error after the login link was clicked. In accordance with this I updated my version of phantomjs to latest stable, 1.9.7 following the guide here. However, now the login click does not seem to register either, and the page does not reload.
The first step is simply getting to login form page.
from selenium import webdriver
br = webdriver.PhantomJS(service_log_path='/path/to/logfile.log')
url = "http://www.preloved.co.uk"
br.get(url)
# Go to login page
login_button = br.find_element_by_xpath('//div[#id="header-status-login"]/a')
login_button.click()
Normally (and if you replace the browser line with br = webdriver.Firefox() for example), this results in reloading to login page, and the script proceeds from there, but now it appears the click does not load the new page at all and br.current_url is still 'http://www.preloved.co.uk/'
Why doesn't this load work?
Even if I extract the href and do an explicit GET it doesn't seem to follow and reload:
newurl=login_button.get_attribute('href')
br.get(newurl)
br.current_url is still 'http://www.preloved.co.uk/'.

The login page is secured through https. Recently the POODLE vulnerability forced websites to move away from SSLv3 for https, but since PhantomJS uses SSLv3 per default the login page doesn't load. See also this answer.
This can be fixed by passing --ssl-protocol=tlsv1 or --ssl-protocol=any to PhantomJS or upgrading PhantomJS to at least version 1.9.8. It seems that the service_args argument could be used for that in the python bindings for Selenium.
It looks like in the current official implementation the service_args cannot be passed from WebDriver to the Service in PhantomJS. You can sub-class it.
from selenium import webdriver
from selenium.webdriver.phantomjs.service import Service
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.remote.webdriver import WebDriver as RemoteWebDriver
class PhantomJSService(webdriver.PhantomJS):
def __init__(self, executable_path="phantomjs", port=0,
desired_capabilities=DesiredCapabilities.PHANTOMJS,
service_args=None, service_log_path=None):
self.service = Service(executable_path,
port=port, service_args=service_args,
log_path=service_log_path)
self.service.start()
try:
RemoteWebDriver.__init__(self,
command_executor=self.service.service_url,
desired_capabilities=desired_capabilities)
except:
self.quit()
raise
It seems that this webdriver fork contains the necessary arguments to set those options.