This question already has an answer here:
Selenium "selenium.common.exceptions.NoSuchElementException" when using Chrome
(1 answer)
Closed 2 years ago.
I'm trying to scrape the promotion information of each product from a website by clicking on the product and go to its detailed page. When the spider clicks on the product, the web will ask it to log in, and I tried the following code:
def __init__(self):
self.driver = webdriver.Chrome(executable_path = '/usr/bin/chromedriver')
...
def start_scraping(self, response):
self.driver.get(response.url)
self.driver.find_element_by_id('fm-login-id').send_keys('iamgooglepenn')
self.driver.find_element_by_id('fm-login-password').send_keys('HelloWorld1_')
self.driver.find_element_by_class_name('fm-button fm-submit password-login').click()
...
However, there is NoSuchElementException when I run it.
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="fm-login-id"]"}
'spider_exceptions/NoSuchElementException': 14,
The HTML of the login page is as follows:
<div class='input-plain-wrap input-wrap-loginid'>
<input id='fm-login-id' class='fm-text' name='fm-login-id'...>
event
</div>
So, I'm pretty sure the id should be 'fm-login-id'. The reason I could think of that might cause this issue is that this login page is a popup.
Basically, it pops up in the middle of the main page. Looking at the HTML of the site, I can see that the login type seems to be a new HTML window
<!DOCTYPE html>
<html>event
....
<\html>
I'm not sure if this is the issue, and if so, how to fix it? Also, is there other reasons that might've caused the issue?
The popup will have an ID. You might have to add f'#{popup_id}' to the end of response.url. Like this URL: https://stackoverflow.com/questions/62906380/nosuchelementexception-when-using-selenium-python/62906409#62906409. It contains #62906409 because 62906409 is the ID of an element in the page.
The login page inside a frame, you need switch it first:
#switch it first
self.driver.switch_to.frame(driver.find_element_by_id('J_loginIframe'))
self.driver.find_element_by_id('fm-login-id').send_keys('iamgooglepenn')
self.driver.find_element_by_id('fm-login-password').send_keys('HelloWorld1_')
And for login button you can't use .find_element_by_class_name, this method just for single class name. This element having multiple class name, so use .find_element_by_css_selector like bellow:
#submit button
self.driver.find_element_by_css_selector('.fm-button.fm-submit.password-login').click()
The login content seems to be nested in an iFrame element (if you trace it all the way to the top, you should find an iFrame with id="sufei-dialog-content"), which means you need to switch to that iFrame for that nested html before selecting your desired element, otherwise it will not work.
First you will need to use driver.switch_to.frame("sufei-dialog-content"), and then select your element with driver.find_element_by_name() or whatever you had.
A similar issue can be found here: Selenium and iframe in html
Just a simple mistake:
<div class='input-plain-wrap input-wrap-loginid'>
<input id='fm-login-id class='fm-text' name='fm-login-id'...>
event
</div>
is actually supposed to be:
<div class='input-plain-wrap input-wrap-loginid'>
<input id='fm-login-id' class='fm-text' name='fm-login-id'...>
event
</div>
You forgot a single-quote.
Have you tried driver.find_element_by_name('fm-login-id')?
You should try finding the elements by their XPaths. You just have to inspect the element, right-click on it and copy its XPath. The XPath of the first <input ... is //*[#id="fm-login-id"].
Related
I'm creating an Instagram bot but cannot figure out how to navigate to the next post.
Here is what I tried
#Attempt 1
next_button = driver.find_element_by_class_name('wpO6b ')
next_button.click()
#Attempt 2
_next = driver.find_element_by_class_name('coreSpriteRightPaginationArrow').click()
Neither of two worked and I get a NoSuchElementException or ElementClickInterceptedException . What corrections do I need to make here?
This is the button I'm trying to click(to get to the next post)
I have checked your class name coreSpriteRightPaginationArrow and i couldn't find any element with that exact class name. But I saw the class name partially. So it might help if you try with XPath contains as shown below.
//div[contains(#class,'coreSpriteRight')]
another xpath using class wpO6b. there are 10 elements with same class name so filtered using #aria-label='Next'
//button[#class='wpO6b ']//*[#aria-label='Next']
Try these and let me know if it works.
I have tried below code and it's clicking next button for 10 times
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
if __name__ == '__main__':
driver = webdriver.Chrome('/Users/yosuvaarulanthu/node_modules/chromedriver/lib/chromedriver/chromedriver') # Optional argument, if not specified will search path.
driver.maximize_window()
driver.implicitly_wait(15)
driver.get("https://www.instagram.com/instagram/");
time.sleep(2)
driver.find_element(By.XPATH,"//button[text()='Accept All']").click();
time.sleep(2)
#driver.find_element(By.XPATH,"//button[text()='Log in']").click();
driver.find_element(By.NAME,"username").send_keys('username')
driver.find_element(By.NAME,"password").send_keys('password')
driver.find_element(By.XPATH,"//div[text()='Log In']").click();
driver.find_element(By.XPATH,"//button[text()='Not now']").click();
driver.find_element(By.XPATH,"//button[text()='Not Now']").click();
#it open Instagram page and clicks 1st post and then it will click next post button for the specified range
driver.get("https://www.instagram.com/instagram/");
driver.find_element(By.XPATH,"//div[#class='v1Nh3 kIKUG _bz0w']").click();
for page in range(1,10):
driver.find_element(By.XPATH,"//button[#class='wpO6b ']//*[#aria-label='Next']" ).click();
time.sleep(2)
driver.quit()
As you can see, the next post right arrow button element locator is changing between the first post to other posts next page button.
In case of the first post you should use this locator:
//div[contains(#class,'coreSpriteRight')]
While for all the other posts you should use this locator
//a[contains(#class,'coreSpriteRight')]
The second element //a[contains(#class,'coreSpriteRight')] will also present on the first post page as well, however this element is not clickable there, it is enabled and can be clicked on non-first pages only.
As you can see on the picture below, the wp06b button is inside a lot of divs, in that case you might need to give Selenium that same path of divs to be able to access the button or give it a XPath.
It's not the most optimized but should work fine.
driver.find_element(By.XPATH("(.//*[normalize-space(text()) and normalize-space(.)='© 2022 Instagram from Meta'])[1]/following::*[name()='svg'][2]")).click()
Note that the XPath leads to a svg, so basically we are clicking on the svg element itself, not in the button.
I have developed an application using Angular, Node.js/Express.js and MySQL. The application has a login page that is displayed when any user visits the link. After logging in, the user is taken to the home page with a lot of other pages that are displayed in the navbar. I am trying to perform an automation test on the application and I am getting this error when I try to make my automated test click on one of the links in the navbar.
TypeError: 'WebElement' object is not subscriptable
After trying different things the error still does not get resolved. I ended up getting an error of Message: element not interactable
I have found the element using the console in Chrome and it is also clickable when I perform the click action in the console $x("//a")[1].click() or $$("[class = 'navbar-nav ml-auto'] li a")[1].click() but when I run the script in selenium then it throws the error. Can anybody tell me how can I make the navbar item clickable in selenium?
Here's my code:
from selenium import webdriver
browser = webdriver.Chrome()
browser.get('http://name-of-the-local-server:3000')
browser.implicitly_wait(3);
email_element = browser.find_element_by_css_selector("input[name = 'usermail']")
password_element = browser.find_element_by_css_selector("input[name = 'passcode']")
submit_btn_element = browser.find_element_by_css_selector("button[name = 'loginButton']")
email_element.send_keys("someone#example.com")
password_element.send_keys("pass123")
submit_btn_element.click()
#executes perfectly until here
navbar_element = browser.find_elements_by_css_selector("[class = 'navbar-nav ml-auto'] li a")
navbar_element[1].click()
EDIT: I have added a bit of HTML that I want to click as by default it loads the home page
<a _ngcontent-imt-c24="" routerlinkactive="active" routerlink="/admin"
class="nav-link" href="/admin"><i _ngcontent-imt-c24="" class="bi bi-people"
style="font-size: 1.3rem;"></i> Admin</a>
<i _ngcontent-sun-c24="" class="bi bi-people" style="font-size: 1.3rem;"></i>
Any help would be appreciated. Thanks!
Instead of this :
navbar_element = browser.find_element_by_xpath("//a")
use this
navbar_element = browser.find_elements_by_xpath("//a")
find_element will return a single web element where as find_elements will return list of web elements
You are using navbar_element[0].click() if stored in list, it will click on first web element.
or iterate over a list like this :
for link in navbar_element:
ActionChains(driver).move_to_element(link).click().perform()
I am trying to create an automated login system for my local library website (http://mcls.ent.sirsi.net/client/en_US/mclweb) using Python and Selenium. However, my script is unable to find the username box.
The HTML from the website for the username box looks like this
<input maxlength="30" class="user_name_input" id="j_username" name="j_username" type="text">
and this is the code I used to find it
username = browser.find_element_by_id('j_username')
username.send_keys(u)
however, I am getting the following error:
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"id","selector":"j_username"}
Am I supposed to use a different function? Or did I use find_element_by_id() wrong? Thanks in advance.
The login form lies on a frame. You have to switch to the frame.
browser.get("http://mcls.ent.sirsi.net/client/en_US/mclweb")
browser.find_element_by_class_name('loginLink').click()
time.sleep(5)
browser.switch_to.frame(1) //login iframe is the second frame in the page
time.sleep(5)
browser.find_element_by_id('j_username').send_keys(u)
Alternative way, in case you want to do without switching to frame:
browser.get("http://mcls.ent.sirsi.net/client/en_US/mclweb/search/patronlogin")
browser.find_element_by_id('j_username').send_keys(u)
U can also login to that site without using iframe
browser.get("http://mcls.ent.sirsi.net/client/en_US/login")
browser.find_element_by_id('j_username').send_keys(u)
I have a list of URLs that I need to iterate over. The process I am working on is that selenium opens each of the URLs in the list, clicks a button to open the form, and pass some strings into the form.
I have gotten to the point of clicking the button to open the form. I can not pass in any strings into the form however using any of the elements. I get error 'Unable to locate element'
This is my code so far, easily_apply is list of the URLs:
for i in easily_apply:
driver.get(i)
test = driver.find_element_by_class_name('button')
test.click()
test.find_element_by_name("applicant.name")
test.send_keys("John Smith")
This is the HTML in question:
<input type="text" aria-describedby="label-input-applicant.name-error" aria-labelledby="label-input-applicant.name" id="input-applicant.name" name="applicant.name" class="icl-TextInput-control icl-TextInput-control--sm">
Thank you in advance.
edit:
Code with xpath, not working, getting error 'Unable to locate elements':
for i in easily_apply:
driver.get(i)
test = driver.find_element_by_class_name('indeed-apply-button')
test.click()
test.find_element_by_xpath('//input[#type="text"]')
test.send_keys("John Smith")
edit2:
code with wait in it, still getting same error 'Unable to locate elements':
for i in easily_apply:
driver.get(i)
test = driver.find_element_by_class_name('indeed-apply-button')
test.click()
wait = ui.WebDriverWait(driver,60)
test.find_element_by_name('applicant.name')
test.send_keys("John Smith")
I looked at the html, assuming you are using the code from your previous post, obtaining all the easily apply list.
The element you are looking for is inside nested iframe. you need to switch to that iframe and then look for the element
Replace the time.sleep with Webdriverwait
driver.find_element_by_css_selector('a[class="indeed-apply-button"]').click()
driver.switch_to.frame(driver.find_element_by_css_selector('iframe[name*=indeed-ia]'))
import time
time.sleep(5)
driver.switch_to.frame(driver.find_element_by_tag_name('iframe'))
driver.find_element_by_id('input-applicant.name').send_keys('Applicant Name')
Try using xpath.
In Selenium automation, if the elements are not found by the general locators like id, class, name, etc. then XPath is used to find an element on the web page .
Syntax is:
Xpath=//tagname[#attribute='value']
Hope this helps.
driver.find_element_by_css_selector("#jl_1c27f21fec51d296 > a").click()
time.sleep(10)
driver.find_element_by_css_selector('a[class="indeed-apply-button"]').click()
I am trying to locate a tag on a page, but so far finding it by name/id/xpath have not worked. This is what I've tried to locate it by xpath:
db = driver.find_element_by_xpath("/html/body/form")
and also
driver.find_element_by_xpath("//form[#id='formControl']")
Both of these return the error:
selenium.common.exceptions.NoSuchElementException: Message: u'Unable to locate element:
It's as if it can not find the form on the page. Any ideas of how to fix?
Here's the beginning of the html on the page:
<html>
<body id="bodyControl" class="PersonalizationPage ContentPage Layout_PageUsesRegions" onunload="closeTimeoutWarningPopupWindow();return true;" onload="return __MasterOnLoad();" onresize="return __MasterOnResize();">
<form name="formControl" method="post" action="default.wl?appflag=65.14&bhcp=1&MT=208&rs=LAWS2.0&strRecreate=no&sv=Split&transfertoken=10281204301250d0dbca913e4aa6887c7d425f4ede9c&vr=2.0" id="formControl" target="_top">
Try this.
driver.find_element_by_id("formControl")
And two things:
Make sure that the form isn't inside of a frame on the page. If it is, you will have to switch to the frame (it is functionally a different webpage displayed in a box.)
Make sure that you've got the right browser window selected, if you're manipulating multiple windows, you're going to have to switch to the one you want to search through.