We Keep Coding

using proxy with selenium webdriver firefox in python not working

I would like to make a program that modifies my ip when I go to consult a site with selenium, I use webdriver firefox, unfortunately the site that I use for my tests returns my ip and not the ip that I indicated in the options, could you tell me the error please.
The program launches and the firefox page opens (I don't use headless for the test), but it's my ip that is returned and not the one from the specified proxy.
here is my program
from selenium import webdriver
options = webdriver.FirefoxOptions()
proxy = f'{"137.74.65.101"}:{"80"}'
options.add_argument(f'--proxy-server={proxy}')
driver = webdriver.Firefox(options=options)
driver.get('https://httpbin.org/ip')

contentEncodingError when opening website in Firefox using selenium and python3

I'm trying to perform a fairly simple action using Selenium, namely opening google images in Firefox browser.
I also use a proxy server running on the localhost.
from selenium.webdriver import Firefox, FirefoxOptions
from selenium.webdriver.firefox.service import Service
from selenium.webdriver.common.proxy import Proxy, ProxyType
options = FirefoxOptions()
service = Service()
options.add_argument("--headless")
options.accept_insecure_certs = True
proxy = Proxy({
'httpProxy': proxy_addr,
'sslProxy': proxy_addr,
'proxyType': ProxyType.MANUAL
})
options.proxy = proxy
b = Firefox(service=service, options=options)
b.execute("get", {'url': 'http://images.google.com'})
But unfortunately, I'm getting an error like this:
selenium.common.exceptions.WebDriverException: Message: Reached error
page:
about:neterror?e=contentEncodingError&u=https%3A//images.google.com/%3Fgws_rd%3Dssl&c=UTF-8&d=The%20page%20you%20are%20trying%20to%20view%20cannot%20be%20shown%20because%20it%20uses%20an%20invalid%20or%20unsupported%20form%20of%20compression.
I would be very grateful for any thoughts and advice what exactly might be the problem and at least approximately what should be paid attention to.
I'm using:
debian
firefox-esr
selenium == 4.2.0
geckodriver-v0.31.0

This error message...
selenium.common.exceptions.WebDriverException: Message: Reached error page: about:neterror?e=contentEncodingError&u=https%3A//images.google.com/%3Fgws_rd%3Dssl&c=UTF-8&d=The%20page%20you%20are%20trying%20to%20view%20cannot%20be%20shown%20because%20it%20uses%20an%20invalid%20or%20unsupported%20form%20of%20compression.
...implies that there are some configuration settings mismatch while GeckoDriver initiates/spawns a new Browsing Context i.e. firefox session and is often observed as:
Solution
As per the mozilla support docs you need to try out the following steps:
Try to reset the network.http.accept-encoding prefs on the about:config page in case they show as user set (bold). You can open the about:config page via the location/address bar. You can accept the warning and click "I'll be careful" to continue.
If you are having Avast Antivirus or Malwarebytes installed, you may need to disable those in the test machine before executing the tests.

How to set proxy in Firefox Webdriver Permanently

Since I cannot set a proxy from Geckodriver directly, I will change it manually by using
from selenium import webdriver
myProxy = "xxxxxxxxx:yyyy"
ip, port = myProxy.split(":")
profile = webdriver.FirefoxProfile()
profile.set_preference('network.proxy_type',5)
profile.set_preference('network.proxy.http','xxxxxxxxx')
profile.set_preference('network.proxy.http_port',yyyy)
profile.update_preferences()
driver=webdriver.Firefox(firefox_profile=profile)
driver.get('http://www.google.co.th')
time.sleep(3)
driver.close()
I am trying to execute, the firefox has changes into proxy mode
However, when I execute script via Robot Framework, which it also execute via Geckodriver, it doesn't change into proxy.
I need to know the way to permanently change proxy in Geckodriver

With SeleniumLibrary's Open Browser keyword you could set any preferences like you would do in your Python script.
For example the following test would open a browser every time with these options:
Proxy HTTP port set to 777.
Proxy type set to "5".
Proxy HTTP "xxxxxxxxx".
*** Settings ***
Library SeleniumLibrary
*** Test Cases ***
Test
Open Browser http://example.com Firefox ff_profile_dir=set_preference("network.proxy_type", "5");set_preference("network.proxy.http", "xxxxxxxxx");set_preference("network.proxy.http_port", 777) # Defining profile using FirefoxProfile mehtods.
Sleep 1 min reason=Verify proxy settings manually in the opened browser.
Values checked on the Firefox's about:config page during sleep:

InvalidArgumentException: Message: invalid argument: user data directory is already in use error while initiating Chrome with ChromeDriver Selenium

I am trying to open a web page using the selenium python library with my default user, it is critical that the script uses the default user but if my chrome browser is already open the script crashes and gives me this error:
selenium.common.exceptions.InvalidArgumentException: Message: invalid argument: user data directory is already in use, please specify a unique value for --user-data-dir argument, or don't use --user-data-dir
I have tried all the solutions given here :
Selenium chromedriver won't launch URL if another chrome instance is open
Selenium won't open a new URL in a new tab (Python & Chrome)
and read that there was a bug in older chromedriver versions but it was fixed in chrome 74 (which im using ) :
https://github.com/SeleniumHQ/docker-selenium/issues/741
from selenium import webdriver
import time
from getpass import getuser
def run():
# Chrome driver path
chromedriver = r'C:\Users\user1\Downloads\chromedriver_win32\chromedriver_new.exe'
# Get chrome webdriver options and set open the browser as headless
chrome_options = webdriver.ChromeOptions()
#chrome_options.add_argument("--headless")
# Fix for selenium Issue 2907
#chrome_options.add_argument('--log-level=3')
#chrome_options.add_experimental_option('excludeSwitches', ['enable-logging'])
# Load current user default profile
current_user = getuser()
chrome_options.add_argument(
r"--user-data-dir=C:\Users\{}\AppData\Local\Google\Chrome\User Data".format(current_user))
# didable "Chrome is being controled by an automated test software"
chrome_options.add_argument('disable-infobars')
# get Chrome to stay open
chrome_options.add_experimental_option("detach", True)
# open browser with options and driver
driver = webdriver.Chrome(options=chrome_options, executable_path=chromedriver)
driver.get(r'https://www.youtube.com/watch?v=dQw4w9WgXcQ')
if __name__ == '__main__':
run()
if i run it without a chrome browser open its fine if not it crashes

This error message...
selenium.common.exceptions.InvalidArgumentException: Message: invalid argument: user data directory is already in use, please specify a unique value for --user-data-dir argument, or don't use --user-data-dir
...implies that the ChromeDriver was unable to initiate/spawn a new Browsing Context i.e. Chrome Browser session using the specified user data directory as it was already in use.
I was able to reproduce the error in my local windows-10box as follows:
Code Block:
from selenium import webdriver
import getpass
options = webdriver.ChromeOptions()
options.add_argument("start-maximized")
options.add_argument(r"--user-data-dir=C:\Users\{}\AppData\Local\Google\Chrome\User Data".format(getpass.getuser()))
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get("https://www.google.com/")
Complete relevant traceback:
[18516:23156:0204/032227.883:ERROR:cache_util_win.cc(21)] Unable to move the cache: Access is denied. (0x5)
[18516:23156:0204/032227.898:ERROR:cache_util.cc(141)] Unable to move cache folder C:\Users\Soma Bhattacharjee\AppData\Local\Google\Chrome\User Data\ShaderCache\GPUCache to C:\Users\Soma Bhattacharjee\AppData\Local\Google\Chrome\User Data\ShaderCache\old_GPUCache_000
[18516:23156:0204/032227.898:ERROR:disk_cache.cc(178)] Unable to create cache
[18516:23156:0204/032227.898:ERROR:shader_disk_cache.cc(605)] Shader Cache Creation failed: -2
Opening in existing browser session.
Traceback (most recent call last):
.
selenium.common.exceptions.InvalidArgumentException: Message: invalid argument: user data directory is already in use, please specify a unique value for --user-data-dir argument, or don't use --user-data-dir
Analysis
The error stack trace clearly complains of Access is denied as the program was unable to move the cache folder ..\ShaderCache\GPUCache to ..\ShaderCache\old_GPUCache_000. hence the creation of cache failed and subsequently creation of Shader Cache Creation failed. Though these issues raises the InvalidArgumentException but forcefully able to open a new window within the existing Chrome Browser Session.
Snapshot of existing Chrome Browser Session:
Snapshot of new window within the existing Chrome Browser Session:
Conclusion
Though the error is thrown still the new Chrome window gets initiated but remains attached with the already opened Chrome session but the new window can't be controlled by the WebDriver instance.
Solution
You need to take care of a couple of things:
If you are using the Default Chrome Profile to access webpages for your other work on the same Test Machine, you shouldn't set user-data-dir as the User Data as it remains locked by the other Chrome process you have initiated manually.
In the above scenario you need to create and use another Chrome Profile and you can find a detailed discussion in How to open a Chrome Profile through Python
If you are executing your tests in a isolated test system, you can set user-data-dir as ..\User Data\Default to access the Default Chrome Profile.
In the above scenario you need to create and use another Chrome Profile and you can find a detailed discussion in How to use Chrome Profile in Selenium Webdriver Python 3
However as per best practices you must always create a new Chrome Profile to execute your tests as the Default Chrome Profile may contain Extensions, Bookmarks, Browsing History, etc, and may not load properly.
You can find a detailed discussion in How to open a Chrome Profile through --user-data-dir argument of Selenium

I also wanted to run Selenium using my default Chrome profile but I came across the same issue. I solved it by copying my UserData folder to another location then I used the new location. Here is my complete code:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=C:\\Users\\myusername\\Desktop\\User Data")
options.add_argument("--profile-directory=Profile 1");
browser = webdriver.Chrome(options=options)
browser.get('https://www.google.com')
Remove the following line from the code if you would like to use your default Chrome profile not a specific profile you created for Selenium.
options.add_argument("--profile-directory=Profile 1");

How to open URL through default Chrome profile using Python Selenium Webdriver

I am on Mac OS X using selenium with python 3.6.3. Im using this code, but browser Google chrome closes immediately after being launched with selenium I start this code, Google chrome opens new windows with Default profile, but chrome wont open the url google.com.
Whats problem with code? Thanks for the help!
FILE_NAME_PROFILE = '/Users/User/Library/Application Support/Google/Chrome'
options = webdriver.ChromeOptions()
options.add_argument('--user-data-dir='+FILE_NAME_PROFILE)
driver = webdriver.Chrome('assets/chromedriver', chrome_options=options)
driver.get("https://google.com")

I am using two arguments and work well in development
"user-data-dir=C:\Users\NameUser\AppData\Local\Google\Chrome\User Data"
"profile-directory=Default"
If you want use another profile (not default) you have to create it and only you have to change the second argument. All profiles are stored in 'User Data' folder
"profile-directory=Profile 1"