I am having a strange issue. I have used the below code for a while to launch a anonymous wedriver instance, however somehow my settings of Chrome have refreshed and now the traffic is not going through the proxy. I tried making changes to the code and the chrome options but I can't figure out what I am doing wrong.
Also, it has somehow stopped adding the uBlock extension.
def anon_drive():
os.startfile(r"C:\Users\xxx\Downloads\tor-win32-0.2.9.9\Tor\tor.exe")
time.sleep(10)
chrome_path = r'C:\Users\xxx\Desktop\chromedriver.exe'
chop = webdriver.ChromeOptions()
chop.add_extension(r'C:\Users\Sid\Desktop\Ublock\uBlock-Origin_v1.11.0.crx')
driver = webdriver.Chrome(chrome_path, chrome_options=chop)
driver = webdriver.Chrome(chrome_path)
return driver
I have the Chrome options set to "Use a proxy server for your LAN is checked" I have tried setting the Advanced options to 127.0.0.1:9050 and 9051.
Still getting
Socks version 67 not recognised.(Tor is not an http proxy.)
I am not sure what I am doing wrong. There are some answers which I found but I am unable to find one which would perfectly.
#JamesKPolk gave me the answer.
Steps:
Chrome>Settings>Advanced Settings>Open Proxy Settings> LAN Settings>Use a Proxy Server for LAN> Advanced> Socks: 127.0.0.1 Port 9050
Press OK and everything should work.
Note: Leave everything except Socks empty.
Related
I am trying to use Chrome along with python webdriver + selenium, but it seems not working when I set the proxy settings? Here is my code:
from selenium import webdriver
PROXY = 'http://42.115.88.220:53281'
chromeOptions = webdriver.ChromeOptions()
chromeOptions.add_argument('--proxy-server=%s' % PROXY)
chromeOptions.add_argument("ignore-certificate-errors")
wbe = webdriver.Chrome(options=chromeOptions)
wbe.get("http://icanhazip.com")
When I run the above codes, the browser gives me: "This site can’t be reached" error:
This site can’t be reached
The connection was reset.
Try:
Checking the connection
Checking the proxy and the firewall
Running Windows Network Diagnostics
ERR_CONNECTION_RESET
Some Efforts: I tried requests with my proxy server, and it works. So it shouldn't be the problem of my proxy server.
import requests
proxies = {"http": "http://42.115.88.220:53281"}
r = requests.get("http://icanhazip.com", proxies = proxies)
print (r.status_code)
This gives me a response code of 200 and good response.
Goal: My final goal is to build a web-crawler with headless chrome with PROXY, so now I am testing a non-headless one first. But it seems there have been something wrong with this PROXY issue.
I would be really appreciated if anyone could help me out with this problem!!!
Try this.
For me it seems that you have used wrong type of headless mode. For chrome selenium browsers it's important to set --headless argument correct.
from selenium import webdriver
PROXY = 'http://ip:port'
chromeOptions = webdriver.ChromeOptions()
chromeOptions.add_argument('--proxy-server=%s' % PROXY)
chromeOptions.add_argument("ignore-certificate-errors")
# Headless mode for chrome browser
chromeOptions.add_argument('--headless=chrome')
wbe = webdriver.Chrome('your_driver_path_or_service', options=chromeOptions)
wbe.get("http://icanhazip.com")
print(wbe.title)
print(wbe.current_url)
print(wbe.page_source)
# Output:
# http://icanhazip.com/
# <html><head><meta name="color-scheme" content="light dark"></head><body><pre
# style="word-wrap: break-word; white-space: pre-wrap;">your ip
# </pre></body></html>
I run selenium WebDriver with a proxy.
chrome_options.add_argument('--proxy-server="'94.242.58.108:1448'"
driver = Chrome(chrome_options=chrome_options)
Driver know which proxy server it is using now.
Could you tell me how to find out the used proxy server?
I need this information in another function.
See the java doc http://seleniumhq.github.io/selenium/docs/api/java/org/openqa/selenium/Proxy.html
we have getHttpProxy() to retrieve the proxy.
Proxy p=new Proxy(); // used at the start to set proxy and pass to chrome options and driver
System.out.println(p.getHttpProxy()); // to get proxy when required.
I used to set up proxy on chrome like in a code below, but when i updated to selenium 3.8.1 proxy stops working, i dont get any errors it just doesn't use proxy server and i dont know why. My chromedriver is also up to date.
options = webdriver.ChromeOptions()
options.add_argument('--proxy-server=192.99.55.120:3128')
driver = webdriver.Chrome(executable_path='C:\chromedriver_win32\chromedriver.exe', chrome_options=options)
driver.get("http://google.com/")
Would like to receive any advice, maybe alternative way to set up proxy for chromedriver.
If someone still interested, this is how i have finally solved the problem
from selenium.webdriver import Proxy
settings = {
"httpProxy": "192.99.55.120:3128",
"sslProxy": "192.99.55.120:3128"
}
proxy = Proxy(settings)
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
cap = DesiredCapabilities.CHROME.copy()
cap['platform'] = "WINDOWS"
cap['version'] = "10"
proxy.add_to_capabilities(cap)
from selenium.webdriver.chrome.webdriver import WebDriver as ChromeDriver
driver = ChromeDriver(desired_capabilities=cap, executable_path='C:\chromedriver_win32\chromedriver.exe')
try
options.add_argument('--proxy-server="http=192.99.55.120:3128;https=192.99.55.120:3128"')
also try running your chrome binary directly with these params to see whether it works or not
chrome.exe --proxy-server="http=192.99.55.120:3128"
If the navigator asks for the credentials username and password for the proxy and you need to handle this : (only if the alert come up)
driver.get("http://username:password#google.com/")
I am using selenium with python.
I need to configure a proxy.
It is working for HTTP but not for HTTPS.
The code I am using is:
# configure firefox
profile = webdriver.FirefoxProfile()
profile.set_preference("network.proxy.type", 1)
profile.set_preference("network.proxy.http", '11.111.11.11')
profile.set_preference("network.proxy.http_port", int('80'))
profile.update_preferences()
# launch
driver = webdriver.Firefox(firefox_profile=profile)
driver.get('https://www.iplocation.net/find-ip-address')
Also. Is there a way for me to completely block any outgoing traffic from my IP and restrict it ONLY to the proxy IP so that I don't accidently mess up the test/stats by accidently switching from proxy to direct connection?
Any tips would help!
Thanks :)
Check out browsermob proxy for setting up a proxies for use with selenium
from browsermobproxy import Server
server = Server("path/to/browsermob-proxy")
server.start()
proxy = server.create_proxy()
from selenium import webdriver
profile = webdriver.FirefoxProfile()
profile.set_proxy(proxy.selenium_proxy())
driver = webdriver.Firefox(firefox_profile=profile)
proxy.new_har("google")
driver.get("http://www.google.co.uk")
proxy.har # returns a HAR JSON blob
server.stop()
driver.quit()
You can use a remote proxy server with the RemoteServer class.
Is there a way for me to completely block any outgoing traffic from my IP and restrict it ONLY to the proxy IP
Yes, just look up how to setup proxies for whatever operating system you're using. Just use caution because some operating systems will ignore proxy rules based on certain conditions, for example, if using a VPN connection.
Whenever I open Firefox using Selenium in python using the command
browser = webdriver.Firefox()
The default proxy configuration is set to "use system proxy setting". I have not configured any proxy in the system. Still whenever the browser opens, it says "The proxy server is refusing connection".
How do I open the browser so that the default proxy setting is set to "no proxy" ?
Please help. Thanks in advance.
I will post from my memory
import os
from selenium import webdriver
profile = webdriver.FirefoxProfile()
profile.set_preference('network.proxy.Kind','Direct')
webdriver.Firefox(profile)
To use the default profile you have to specify it
profile = webdriver.FirefoxProfile(path_to_profile_in_your_pc)
webdriver.Firefox(profile)