I am trying to log on to Target's website using Selenium in Python with the Chrome WebDriver..
When I am prompted to log in, I use the following code:
self.browser.find_element_by_name("password").send_keys(pw)
self.browser.find_element_by_id("login").submit()
After the field is submitted, I am presented with this error in the DOM:
DOM Error Alert
..and this in the console:
401 Error
Note:
I have tried logging in with Selenium on Instagram, and it works.. So I know it has something to do with the structure of Target's website. Has anyone run into this issue before?
Thanks!
So I originally tried solving this issue using Chrome, but could not figure out why the page would not precede after entering the login data. I thought maybe the page was protected by some bot software, but could not find any proof.
I decided to try Safari on my MAC, and actually had success. See the below code:
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as ec
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.keys import Keys
driver = webdriver.Safari(executable_path='/usr/bin/safaridriver')
driver.get('https://www.target.com/')
action = ActionChains(driver)
driver.find_element(By.XPATH, '//*[#id="account"]').click()
WebDriverWait(driver, 30).until(ec.presence_of_element_located((By.XPATH, '//*[#id="accountNav-signIn"]')))
action.send_keys(Keys.ENTER)
action.perform()
WebDriverWait(driver, 10).until(ec.presence_of_element_located((By.XPATH, '//h2[#class="sc-hMqMXs sc-esjQYD eXTUDl"]')))
driver.find_element(By.ID, 'username').click()
driver.find_element(By.ID, 'username').send_keys('foo')
time.sleep(5)
driver.find_element(By.ID, 'password').click()
driver.find_element(By.ID, 'password').send_keys('bar')
time.sleep(5)
driver.find_element(By.XPATH, "//button[#id=\'login\']").send_keys(Keys.ENTER)
time.sleep(10)
driver.quit()
You will notice some time.sleeps which I am using to slow the program down (you can take these out).
I also tried on FireFox and Edge, but had the same problems as Chrome.
Conclusion, it seems there could be some sort of bot protection which is blocking you from using Chrome (also Edge and FireFox). Given these webdrivers are being detected as automated. Safari (I believe) does not get detected as such.
I would also suggest reading through the below post, it may offer more insight.
Website navigates to no-access page using ChromeDriver and Chrome through Selenium probably Bot Protected
Related
Recently the login for this site changed and no longer recognizes my Python bot. Specifically, the issue appears to be occurring on the login page where it is unable to select the username input textbox. The id for it is 'loginId' and the correct XPATH appears to be "//*[#name='loginId']"
The line I am attempting to use (that used to work) is:
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//*[#name='loginId']"))).send_keys(username)
The error message I am receiving clearly states it is not finding the element and is timing out:
File "C:\Users\Matt\Python3.9\lib\site-packages\selenium\webdriver\support\wait.py", line 80, in until
raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:
I have tried all of the suggested names/paths/ids found in both Katalon Recorder and Selenium IDE. This does not appear to be an iFrame. Not sure what is going on.
Any thoughts or input would be helpful here. Link is provided up above if you can check it out that would be helpful. Thank you in advance!
Supplemental code [EDIT 07/05/2022]:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.chrome.options import Options
options = webdriver.ChromeOptions()
driver = webdriver.Chrome('C:\\Users\\Matt\\Documents\\Splitt\\Chromedriver\\chromedriver.exe', options=options)
driver.get("https://www.stellarmls.com/")
driver.maximize_window()
WebDriverWait(driver, 6).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="login-form-row"]/form/div/div[3]/div/a'))).click()
driver.find_element(By.ID, "loginId").send_keys('username')
I didn't use your code and instead plugged in your site to a Selenium script I had and it worked without issue:
driver.find_element(By.NAME, "loginId").send_keys(username)
It essentially does the same thing without the explicit wait. The fact that it was missing didn't seem to be a problem.
The code is supposed to type "fish" into the YouTube search bar using Selenium and a Chrome Browser.
I have tried the xpaths of mulitple divs that hold the tag and they didn't work either.(not sure if the error was the same though) The xpath in the code is for the <input> tag so it should be fine.
I also watched a tutorial and the xpath was exactly the same so that shouldn't be the problem since it worked for the YouTuber.
It also took me some time to figure out that the find_element_by_* are depreciated functions.
Could it be that the .send_keys has also been changed? I did try to find the selenium changes in 4.1.0 and it said nothing about it on a website that I found.
Should I maybe delete Selenium 4.1.0 and install an older version? For simplicity sake. Since there is probably a bigger number of tutorials for it.
from selenium import webdriver
from selenium.webdriver.common.by import By
setting = webdriver.ChromeOptions()
setting.add_argument("--incognito")
# I open the browser in incognito just so I don't clutter my search
# history with dumb stuff as I'm testing things out
# could it be a part of the problem?
driver = webdriver.Chrome(options = setting)
driver.get('http://youtube.com')
searchbox = driver.find_element(By.XPATH, '//*[#id="search"]')
searchbox.send_keys('fish')
Error Message:
selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable
wait=WebDriverWait(driver,60)
driver.get('http://youtube.com')
searchbox = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR,"input#search")))
searchbox.send_keys('fish')
In order to send_keys to that element wait for it to interactable and then send keys.
Imports:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
Outputs:
I am grinding through day 2 of me learning Python 3.7 with Selenium.
I am accessing a web page using WebDriver. I have been making progress, but am stymied now. Though I can easily disable the Firefox password manager popup window on my normal Browser (Options/Privacy and Security/Location/Settings), my script's remotely-run (think that is by definition) browser does not recognize that configuration, and the Firefox popup shows up.
The script can ignore the popup and navigate the target site until the very last page that I need to access. At that point, the HTML for that page is inaccessible, until I manually click on the Firefox popup, dismissing it. As soon as I do that, the HTML code for that web page lights up in Firefox Web Developer Inspector.
Now, that HTML code may be inaccessible for other reasons (like I said, day 2 of the learning curve), but is there some library or commands within Webdriver that allow me to automate the dismissal of that FireFox popup. It is not part of the HTML of any page, so I am at a loss.
Edit: I should mention also, the bulk of that last page's content is blank until I manually dismiss the FireFox popup.
I have added the following code, but still am getting the same popup:
from selenium import webdriver
#Using Firefox to access the Web
options = webdriver.FirefoxOptions()
options.set_preference("dom.webnotifications.enabled", False)
driver = webdriver.Firefox(options=options)
driver.maximize_window()
Second Edit: This is the current code section defining the profile, and I am still getting the pop up password manager.
import datetime
import time
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
#Using Firefox to access the Web
profile = webdriver.FirefoxProfile()
#profile.set_preference("dom.push.enabled", False)
profile.set_preference("dom.webnotifications.enabled", False)
profile.update_preferences()
driver = webdriver.Firefox(firefox_profile=profile)
driver.maximize_window()
about:config
preference security.insecure_field_warning.contextual.enabled to false
I am using Selenium to log into a HikVision IP camera web client and I am experiencing an issue. The automated browser seems to successfully log in, as it loads the next page, but when I return page_source(), it shows that the webdriver is still stuck on the log in page. I have tried implementing wait/until and implicit waits for the web page to load but it does not seem to be a loading issue, as no matter how long I wait, I get the same problem.
Here is a code snippet showing how I log in:
user = driver.find_element_by_id("username")
password = driver.find_element_by_id("password")
# log in
user.clear()
user.send_keys("admin")
password.clear()
password.send_keys("Snipr123")
driver.find_element_by_css_selector(".btn.btn-primary.login-btn").click()
The .clear() was just to get rid of the preloaded text that was giving me issues.
After the login click() the automated browser successfully loads the web client, but the webdriver doesn't, since page_source() still returns the log in page.
Any ideas on what could be going wrong would be greatly appreciated.
Once you login using the line of code:
driver.find_element_by_css_selector(".btn.btn-primary.login-btn").click()
Yneed to induce WebDriverWait for the visibility_of_element_located() of any of the visible element within the DOM Tree and then extract the page source as follows:
driver.find_element_by_css_selector(".btn.btn-primary.login-btn").click()
WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CLASS_NAME, "table")))
print(driver.page_source)
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
since I updated my firefox to 49.02 and selenium to 3.0.1, my previous code to sign in to my bing account does not work.
there is a Sign in link in www.bing.com, I could successfully click this link by calling:
driver.get("http://www.bing.com")
driver.implicitly_wait(20)
driver.find_element_by_link_text('Sign in').click()
however, after the upgrade, I receive a strange error message which contains no message at all:
selenium.common.exceptions.ElementNotVisibleException: Message:
if I only call driver.find_element_by_link_text('Sign in'), I will receive no error message. This seems like selenium could successfully locate this link, but somehow it can not click this button.
I have also tried to locate Sign in by it class name or by clicking the icon instead, but all such efforts are useless.
I do not know if the error is caused by Microsoft to block automated logging in or the error in my code. Helps appreciated!
calling driver.find_element_by_xpath('//a[span = "Sign in"]').click() as suggested by alecxe still does not resolve the issue.
Wait for the link to be clickable:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
wait = WebDriverWait(driver, 10)
sign_in = wait.until(EC.element_to_be_clickable((By.LINK_TEXT, "Sign in")))
sign_in.click()