We Keep Coding

How to use Firefox profiles in Selenium Python (windows)

I'm unable lo load my Firefox profile on the Selenium Webdriver.
i have tried several ways commented here on SO, but non did the trick. Either i loose connection to the driver or the profile does not get loaded.
this is what i have so far:
from selenium import webdriver
options = Options()
options.headless = False
fp = webdriver.FirefoxProfile(r'C:\\Users\\xxXX\\AppData\\Local\\Mozilla\\Firefox\\Profiles\\y73h6ogb.default-release')
driver = webdriver.Firefox(fp, options=options, executable_path='geckodriver/geckodriver.exe')
driver.get(URL)

According to the documentation you can load a profile like this:
from selenium.webdriver import Firefox, FirefoxOptions, FirefoxProfile
options = FirefoxOptions()
options.headless = True
profile = FirefoxProfile("PATH_TO_PROFILE\\1234asdf.some_profile")
driver = Firefox(
firefox_profile=profile,
executable_path="PATH_TO_DRIVER\\geckodriver.exe",
options=options,
)
driver.get("https://python.org")
driver.close()
Or by adding the profile to the options:
...
options = FirefoxOptions()
options.headless = True
options.profile = FirefoxProfile("PATH_TO_PROFILE\\1234asdf.some_profile")
driver = Firefox(
executable_path="PATH_TO_DRIVER\\geckodriver.exe",
options=options,
)
...

By using webdriver for firefox, You will be able to do that!

How do I set a proxy for chrome's webdriver in python

I have already tried:
'''
from selenium import webdriver
PROXY = "23.23.23.23:3128" # IP:PORT or HOST:PORT
options = webdriver.ChromeOptions()
options.add_argument('--proxy-server=%s' % PROXY)
chrome = webdriver.Chrome('./chromedriver',options=options) #<----- I used './chromedriver' to set a PATH
chrome.get("http://whatismyipaddress.com")
'''
The above code is taken from how do i set proxy for chrome in python webdriver.
I am getting an error from google chrome saying ERR_NO_SUPPORTED_PROXIES, even when I am using public ones.

you can try this:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
PROXY = "23.23.23.23:3128" # IP:PORT or HOST:PORT
capabilities = dict(DesiredCapabilities.CHROME)
capabilities['proxy'] = {'proxyType': 'MANUAL','httpProxy': '23.23.23.23:3128','ftpProxy': '23.23.23.23:3128','sslProxy': '23.23.23.23:3128','noProxy': '','class': "org.openqa.selenium.Proxy",'autodetect': False}
chrome_options = Options()
chrome_options.add_argument("--disable-notifications")
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--verbose')
chrome_options.add_argument('--disable-gpu')
chrome_options.add_argument('--disable-software-rasterizer')
chrome_options.add_argument('--headless')
chrome = webdriver.Chrome('./chromedriver',options=chrome_options) #<----- I used './chromedriver' to set a PATH
chrome.get("http://whatismyipaddress.com")

How to change the download location using python and selenium webdriver

I've written the code to download files for each month in a range of years for every precinct and place. However, since I can't change the name of the files through selenium, I was hoping to download each place's files into a separate folder. Here's my code
options = webdriver.ChromeOptions()
options.add_argument('download.default_directory=/Users/name/Downloads/' + p)
driver = webdriver.Chrome(chrome_options=options, executable_path="/Users/name/Downloads/chromedriver")
driver.get("https://jpwebsite.harriscountytx.gov/PublicExtracts/search.jsp")
where p is the id of a particular precinct and place. Unfortunately the files are downloaded to /Users/name/Downloads. I've added chromedriver to PATH and just used
driver = webdriver.Chrome(chrome_options=options)
but that gives me this:
[Errno 2] No such file or directory.
What am I doing wrong? Thanks!

You can use timestamp to create new directory. Also use preference dictionary for chrome options with prompt_for_download and directory_upgrade params. try below example:
from selenium import webdriver
import time
timestr = time.strftime("%Y%m%d-%H%M%S")
options = webdriver.ChromeOptions()
prefs = {
"download.default_directory": r"C:\Users\XXXX\downdir\stamp"+timestr,
"download.prompt_for_download": False,
"download.directory_upgrade": True
}
options.add_experimental_option('prefs', prefs)
driver = webdriver.Chrome(chrome_options=options)
driver.get("https://jpwebsite.harriscountytx.gov/PublicExtracts/search.jsp")

Try this, it will work smoothly
import webdriver
chrome_options = webdriver.ChromeOptions()
prefs = {'download.default_directory' : 'path for your folder that you want'}
chrome_options.add_experimental_option('prefs', prefs)
driver = webdriver.Chrome(chrome_options=chrome_options)

Try this code it works for me, just create a profile for chrome and define the download location for the tests
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("download.default_directory=D:/Sele_Downloads")
driver = webdriver.Chrome(chrome_options=options)
driver.get("https://jpwebsite.harriscountytx.gov/PublicExtracts/search.jsp")

Chrome driver not working with proxy

I am trying to use Chrome Driver with proxy first time but it does not seem to work. Below is my code:
from selenium import webdriver
proxy = '1.1.1.1'
proxy_url = 'https://usr:pwd#' + proxy.strip().rstrip('\n')
print(proxy_url)
service = webdriver.chrome.service.Service(CHROME_PATH)
service.start()
options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--proxy-server=%s' % proxy_url)
options = options.to_capabilities()
driver = webdriver.Remote(service.service_url, options)
driver.set_window_size(1400, 1000)
driver.get("http://google.com")
print(driver.page_source)
No matter what site I use it returns:
<html xmlns="http://www.w3.org/1999/xhtml"><head></head><body></body></html>
Sorry I can't share original proxy as they don't belong to me. Can anyone help me out?

Selenium raises BadStatusLine error

Hi I'm using selenium in my web app on heroku. I used it to login to instagram but it raises a BadStatusLine error whenever selenium is sending keys to the user field/password field.
My code:
display = Display(visible=0, size=(800, 600))
display.start()
options = Options()
options.binary_location = "/app/.apt/usr/bin/google-chrome-stable"
driver = webdriver.Chrome(chrome_options = options)
driver.get("http://www.instagram.com/")
#click login button to show login form
driver.implicitly_wait(20) # seconds
login_btn = driver.find_element_by_link_text("Log in")
login_btn.click()
#enter username and password
driver.implicitly_wait(20) # seconds
driver.find_element_by_name("username").send_keys(login)
driver.find_element_by_name("password").send_keys(password)
Error:

For anyone who need, you will need
-Two buildpacks
-Set stack to cedar-14
-Have chromedriver in your files. Point to it and set chromedriver options
from selenium.webdriver.chrome.options import Options
dir_path = os.path.dirname(os.path.realpath(__file__))
chromedriver_path = os.path.join(dir_path, "chromedriver")
options = Options()
options.binary_location = "/app/.apt/usr/bin/google-chrome-stable"
driver = webdriver.Chrome(executable_path = chromedriver_path, chrome_options = options)