We Keep Coding

Python User/Pass autofill selenium

I was following along with a tutorial to create the below program to
auto log into a github account. When I run the program it takes me to the sign in
page but does not not autofill the fields and submit. The tutorial was using "find_element_by_id" method. However, it was not recognized. After some googling I came up with the "By" class and it seems to work well earlier in the program when I used "By.LINK_TEXT" but fails when using it for .ID?
or maybe it is an issue with the "send_keys" method?
Any assistance would be greatly appreciated.
from selenium.webdriver.common.keys import Keys
import re
from selenium.webdriver.common.by import By
options = webdriver.ChromeOptions()
options.add_experimental_option("detach", True)
browser = webdriver.Chrome(
options=options, executable_path="C:\\Windows\\chromedriver.exe")
browser.get("https://github.com")
signin_link = browser.find_element(By.LINK_TEXT, "Sign in")
signin_link.click()
username_box = browser.find_element(By.ID, "login_field")
username_box.send_keys("username")
password_box = browser.find_element(By.ID, "password")
password_box.send_keys("password")
password_box.submit()
`
I tried the various constants for the By class (name, id, link_text, etc.)
None of them seems to make any difference. I imagine the problem is somewhere else in the program.

Add WebDriveWait so the page can load before trying to pass the username and password
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import re
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
options = webdriver.ChromeOptions()
options.add_experimental_option("detach", True)
browser = webdriver.Chrome(
options=options, executable_path="C:\\Windows\\chromedriver.exe")
browser.get("https://github.com")
signin_link = browser.find_element(By.LINK_TEXT, "Sign in")
signin_link.click()
# wait until the page loads
WebDriverWait(browser, 10).until(EC.presence_of_element_located((By.ID, 'login_field')))
username_box = browser.find_element(By.ID, "login_field")
username_box.send_keys("username")
password_box = browser.find_element(By.ID, "password")
password_box.send_keys("password")
password_box.submit()

python selenium checking for element in chrome doesn't work as expected

This works just fine in Firefox. When I use Chrome, once the page fully loads it doesn't print "element loaded", and it doesn't go to timeout either. It just waits forever.
I've tried using visibility_of_element_located instead of presence_of_element_located but it makes no difference. I've tried all_elements too. Any advice?
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.common.keys import Keys
browser = webdriver.Chrome()
browser.get("http://url.com")
timeout = 10
email = ("email#gmail.com")
try:
email_form_wait = WebDriverWait(browser, timeout).until(EC.presence_of_element_located((By.XPATH, '//*[#id="username"]')))
print ("element loaded")
email_form = browser.find_element(By.XPATH, '//*[#id="username"]')
email_form.send_keys(email, Keys.ENTER)
except TimeoutException:
print ("Loading took too much time!")
Update
For debugging reasons, I've been trying this
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
import time
browser = webdriver.Chrome()
browser.get("http://url.com")
time.sleep(5)
print("end wait")
This works as expected on Chrome, but when I add
email_form = browser.find_element(By.XPATH, '//*[#id="username"]')
print("element found")
at the end of the code, it doesn't even print "end wait", it's stuck waiting forever and then it prints "end wait" and returns an error only after I force close the browser.

Site Isolation is a security feature in Chrome that offers additional protection against some types of security bugs. It uses Chrome's sandbox to make it harder for untrustworthy websites to access or steal information from your accounts on other websites.
Try opening chrome driver with the following options supplied.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service
options = webdriver.ChromeOptions()
options.add_argument("--disable-site-isolation-trials")
chrome_path = r"C:\Users\hpoddar\Desktop\Tools\chromedriver_win32\chromedriver.exe"
s = Service(chrome_path)
driver = webdriver.Chrome(service=s, chrome_options=options)
driver.get(url)
username = driver.find_element(By.XPATH, '//input[#id="username"]')
username.send_keys('email#gmail.com')
submit = driver.find_element(By.CSS_SELECTOR, '.ca56ae105.c03eb9739.c026ac3bb.ca0d6234f._button-login-id')
submit.click()
password = driver.find_element(By.XPATH, '//input[#id="password"]')
password.send_keys('password')
continueButton = driver.find_element(By.CSS_SELECTOR, '.ca56ae105.c03eb9739.c026ac3bb.ca0d6234f._button-login-password')
continueButton.click()

Why isn't the webpage loding while scraping linkedin?

In this code I am trying to scrape a Linkedin profile using Selenium
but the driver is not able to load the page I guess IP has been
blocked and I am new to the concept of proxy rotating or any concept
that is used in such cases. It would be a great help if you could help
me understand how this is done.
from selenium import webdriver
import time
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome(executable_path=r'C:\Users\chromedriver.exe')
def linkedin_login():
global driver
options = webdriver.ChromeOptions()
options.add_argument("start-maximized")
options.add_experimental_option('excludeswitches', ['enable-automation'])
options.add_experimental_option("detach", True)
try:
driver.get('https://www.linkedin.com/login')
username = 'username'
password = 'password'
WebDriverWait(driver, 30).until(EC.element_to_be_clickable((By.ID, 'username'))).send_keys(username)
driver.find_element_by_id('password').send_keys(password)
driver.find_element_by_class_name('btn__primary--large from__button--floating').click()
time.sleep(8)
except ImportError:
print('Closing')
def search_profiles():
search_profile = input('What profile do you want to search?')
search_profile = search_profile.split()
search = search_profile[0] + "%20" + search_profile[1]

The "Sign In" button's class name is incorrect, specifically is missing a dot(.)
Instead of:
driver.find_element_by_class_name('btn__primary--large from__button--floating').click()
Use:
driver.find_element_by_class_name('btn__primary--large.from__button--floating').click()
That will click the button.
Also, if you run the code you shared, you are calling the webdriver but not calling your function.
I tested the below code and worked fine (remember to update your path and LinkedIn credentials):
from selenium import webdriver
import time
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome(executable_path='/home/armentaahumada/Downloads/chromedriver')
def linkedin_login():
global driver
options = webdriver.ChromeOptions()
options.add_argument("start-maximized")
options.add_experimental_option('excludeswitches', ['enable-automation'])
options.add_experimental_option("detach", True)
try:
driver.get('https://www.linkedin.com/login')
username = 'username'
password = 'password'
WebDriverWait(driver, 30).until(EC.element_to_be_clickable((By.ID, 'username'))).send_keys(username)
driver.find_element_by_id('password').send_keys(password)
driver.find_element_by_class_name('btn__primary--large.from__button--floating').click()
time.sleep(8)
except ImportError:
print('Closing')
def search_profiles():
search_profile = input('What profile do you want to search?')
search_profile = search_profile.split()
search = search_profile[0] + "%20" + search_profile[1]
linkedin_login()

I can't locate the element but after inspect element manually the code works

I get this error when I run the code :
Message: no such element: Unable to locate element: {"method":"xpath","selector":"//div[#data-test-id="birth-date__day"]/select"
If I inspect Element manually in browser the code works. I was thinking it's because of some frame but can't find it. Also I tried to first click and then try to use "Select"but still not works.
This is the code :
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.by import By
import requests
import time
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import Select
options = webdriver.ChromeOptions()
options.add_experimental_option("useAutomationExtension", False)
options.add_experimental_option("excludeSwitches",["enable-automation"])
driver_path = 'C:\Program Files (x86)\chromedriver.exe'
driver = webdriver.Chrome(executable_path=driver_path, options=options)
url = "https://account.mail.ru/signup?from=main&rf=auth.mail.ru"
Fname = "John"
Lname = "Micheals"
driver.get(url)
time.sleep(4)
day = driver.find_element_by_xpath('//div[#data-test-id="birth-date__day"]/select')
Select(day).select_by_value("11")
Please if someone can help.

Why does my Selenium URL_to_be statement not work?

Selenium does not seem to register that I manually go to the publish0x.com page.
Does anyone know a solution?
My goal is to manually do the captcha at the login page and afterwards, when I log in and land on the main page I want the script to resume.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
import time
import datetime
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
import sys
from selenium.webdriver.support.ui import WebDriverWait
def waitForLoad(inputXPath):
Wait = WebDriverWait(driver, 10)
Wait.until(EC.presence_of_element_located((By.XPATH, inputXPath)))
email = '
password = '
options = Options()
options.add_experimental_option("detach", True)
options.add_argument("--window-size=1920,1080")
## options.add_argument("user-data-dir=/Users/vadim/Library/Application Support/BraveSoftware/Brave-Browser")
options.binary_location = '/Applications/Brave Browser.app/Contents/MacOS/Brave Browser'
driver_path = '/usr/local/bin/chromedriver'
driver = webdriver.Chrome(options=options, executable_path=driver_path)
driver.get('https://www.publish0x.com/login')
waitForLoad('//*[#id="email"]')
E_Mail_vak = driver.find_element_by_xpath('//*[#id="email"]')
E_Mail_vak.send_keys(email)
Pass_vak = driver.find_element_by_xpath('//*[#id="password"]')
Pass_vak.send_keys(password)
frame = driver.find_element_by_xpath('//iframe[contains(#src, "recaptcha")]')
driver.switch_to.frame(frame)
Captcha = driver.find_element_by_xpath("//*[#id='recaptcha-anchor']")
Captcha.click()
wait = WebDriverWait(driver, 500)
wait.until(EC.url_to_be("publish0x.com"))
driver.get('https://www.publish0x.com/newposts')
post = driver.find_element_by_css_selector('#main > div.infinite-scroll > div:nth-child(1) > div.content')
title = post.find_element_by_css_selector('h2 > a').text
author = post.find_element_by_css_selector('p.text-secondary > small:nth-child(4) > a').text
title.click()
slider = driver.find_element_by_xpath('//*[#id="tipslider"]')

There are two ways I can think of, one by adding an Input statement like this:
options.add_argument('--disable-gpu')#For properly seeing the outputs
input("Please do the captcha and press any key...)
In this way, the user would complete the data and then press any key for the script to continue.
The other way is by adding a try and except statement.
try:
driver.find_element_by_id("some-element")
except NoSuchElementException:
#do something like
print("Captcha Failed or Incomplete...")
In this, replace the element id "some-element" with any element that is present and only present after the user logs in, for e.g elements like Profile_nav or Settings are only present when someone logs in. So if the element doesn't exist then it would mean that the user didn't complete the captcha.