I have a list of URLs that I need to iterate over. The process I am working on is that selenium opens each of the URLs in the list, clicks a button to open the form, and pass some strings into the form.
I have gotten to the point of clicking the button to open the form. I can not pass in any strings into the form however using any of the elements. I get error 'Unable to locate element'
This is my code so far, easily_apply is list of the URLs:
for i in easily_apply:
driver.get(i)
test = driver.find_element_by_class_name('button')
test.click()
test.find_element_by_name("applicant.name")
test.send_keys("John Smith")
This is the HTML in question:
<input type="text" aria-describedby="label-input-applicant.name-error" aria-labelledby="label-input-applicant.name" id="input-applicant.name" name="applicant.name" class="icl-TextInput-control icl-TextInput-control--sm">
Thank you in advance.
edit:
Code with xpath, not working, getting error 'Unable to locate elements':
for i in easily_apply:
driver.get(i)
test = driver.find_element_by_class_name('indeed-apply-button')
test.click()
test.find_element_by_xpath('//input[#type="text"]')
test.send_keys("John Smith")
edit2:
code with wait in it, still getting same error 'Unable to locate elements':
for i in easily_apply:
driver.get(i)
test = driver.find_element_by_class_name('indeed-apply-button')
test.click()
wait = ui.WebDriverWait(driver,60)
test.find_element_by_name('applicant.name')
test.send_keys("John Smith")
I looked at the html, assuming you are using the code from your previous post, obtaining all the easily apply list.
The element you are looking for is inside nested iframe. you need to switch to that iframe and then look for the element
Replace the time.sleep with Webdriverwait
driver.find_element_by_css_selector('a[class="indeed-apply-button"]').click()
driver.switch_to.frame(driver.find_element_by_css_selector('iframe[name*=indeed-ia]'))
import time
time.sleep(5)
driver.switch_to.frame(driver.find_element_by_tag_name('iframe'))
driver.find_element_by_id('input-applicant.name').send_keys('Applicant Name')
Try using xpath.
In Selenium automation, if the elements are not found by the general locators like id, class, name, etc. then XPath is used to find an element on the web page .
Syntax is:
Xpath=//tagname[#attribute='value']
Hope this helps.
driver.find_element_by_css_selector("#jl_1c27f21fec51d296 > a").click()
time.sleep(10)
driver.find_element_by_css_selector('a[class="indeed-apply-button"]').click()
Related
I'm creating an Instagram bot but cannot figure out how to navigate to the next post.
Here is what I tried
#Attempt 1
next_button = driver.find_element_by_class_name('wpO6b ')
next_button.click()
#Attempt 2
_next = driver.find_element_by_class_name('coreSpriteRightPaginationArrow').click()
Neither of two worked and I get a NoSuchElementException or ElementClickInterceptedException . What corrections do I need to make here?
This is the button I'm trying to click(to get to the next post)
I have checked your class name coreSpriteRightPaginationArrow and i couldn't find any element with that exact class name. But I saw the class name partially. So it might help if you try with XPath contains as shown below.
//div[contains(#class,'coreSpriteRight')]
another xpath using class wpO6b. there are 10 elements with same class name so filtered using #aria-label='Next'
//button[#class='wpO6b ']//*[#aria-label='Next']
Try these and let me know if it works.
I have tried below code and it's clicking next button for 10 times
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
if __name__ == '__main__':
driver = webdriver.Chrome('/Users/yosuvaarulanthu/node_modules/chromedriver/lib/chromedriver/chromedriver') # Optional argument, if not specified will search path.
driver.maximize_window()
driver.implicitly_wait(15)
driver.get("https://www.instagram.com/instagram/");
time.sleep(2)
driver.find_element(By.XPATH,"//button[text()='Accept All']").click();
time.sleep(2)
#driver.find_element(By.XPATH,"//button[text()='Log in']").click();
driver.find_element(By.NAME,"username").send_keys('username')
driver.find_element(By.NAME,"password").send_keys('password')
driver.find_element(By.XPATH,"//div[text()='Log In']").click();
driver.find_element(By.XPATH,"//button[text()='Not now']").click();
driver.find_element(By.XPATH,"//button[text()='Not Now']").click();
#it open Instagram page and clicks 1st post and then it will click next post button for the specified range
driver.get("https://www.instagram.com/instagram/");
driver.find_element(By.XPATH,"//div[#class='v1Nh3 kIKUG _bz0w']").click();
for page in range(1,10):
driver.find_element(By.XPATH,"//button[#class='wpO6b ']//*[#aria-label='Next']" ).click();
time.sleep(2)
driver.quit()
As you can see, the next post right arrow button element locator is changing between the first post to other posts next page button.
In case of the first post you should use this locator:
//div[contains(#class,'coreSpriteRight')]
While for all the other posts you should use this locator
//a[contains(#class,'coreSpriteRight')]
The second element //a[contains(#class,'coreSpriteRight')] will also present on the first post page as well, however this element is not clickable there, it is enabled and can be clicked on non-first pages only.
As you can see on the picture below, the wp06b button is inside a lot of divs, in that case you might need to give Selenium that same path of divs to be able to access the button or give it a XPath.
It's not the most optimized but should work fine.
driver.find_element(By.XPATH("(.//*[normalize-space(text()) and normalize-space(.)='© 2022 Instagram from Meta'])[1]/following::*[name()='svg'][2]")).click()
Note that the XPath leads to a svg, so basically we are clicking on the svg element itself, not in the button.
I am trying to scrape the yahoo finance webpage (comment section). I want to click on a button whose class is seen in the picture below:
I want to select the button with the following code, but I am getting an InvalidSelectorException. I do not understand why.
Note that in my code I have replaced the space with . because that's what I usually do, but I have also tried without replace the spaces and in both cases it is not working.
link = 'https://finance.yahoo.com/quote/AMD/community?p=AMD'
path = r"""chromedriver.exe"""
driver = webdriver.Chrome(executable_path=path)
driver.get(link)
driver.find_element_by_class_name('sort-filter-button.O(n):h.O(n):a.Fw(b).M(0).P(0).Ff(i).C(#000).Fz(16px)')
You can check the below
#This page is taking more time to load
sleep(15)
element = driver.find_element_by_xpath("//button[#aria-label='Sort Reactions']")
element.click()
update
element = driver.find_element_by_xpath("//button[contains(#class,'sort-filter-button')]")
element.click()
This question already has an answer here:
Selenium "selenium.common.exceptions.NoSuchElementException" when using Chrome
(1 answer)
Closed 2 years ago.
I'm trying to scrape the promotion information of each product from a website by clicking on the product and go to its detailed page. When the spider clicks on the product, the web will ask it to log in, and I tried the following code:
def __init__(self):
self.driver = webdriver.Chrome(executable_path = '/usr/bin/chromedriver')
...
def start_scraping(self, response):
self.driver.get(response.url)
self.driver.find_element_by_id('fm-login-id').send_keys('iamgooglepenn')
self.driver.find_element_by_id('fm-login-password').send_keys('HelloWorld1_')
self.driver.find_element_by_class_name('fm-button fm-submit password-login').click()
...
However, there is NoSuchElementException when I run it.
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="fm-login-id"]"}
'spider_exceptions/NoSuchElementException': 14,
The HTML of the login page is as follows:
<div class='input-plain-wrap input-wrap-loginid'>
<input id='fm-login-id' class='fm-text' name='fm-login-id'...>
event
</div>
So, I'm pretty sure the id should be 'fm-login-id'. The reason I could think of that might cause this issue is that this login page is a popup.
Basically, it pops up in the middle of the main page. Looking at the HTML of the site, I can see that the login type seems to be a new HTML window
<!DOCTYPE html>
<html>event
....
<\html>
I'm not sure if this is the issue, and if so, how to fix it? Also, is there other reasons that might've caused the issue?
The popup will have an ID. You might have to add f'#{popup_id}' to the end of response.url. Like this URL: https://stackoverflow.com/questions/62906380/nosuchelementexception-when-using-selenium-python/62906409#62906409. It contains #62906409 because 62906409 is the ID of an element in the page.
The login page inside a frame, you need switch it first:
#switch it first
self.driver.switch_to.frame(driver.find_element_by_id('J_loginIframe'))
self.driver.find_element_by_id('fm-login-id').send_keys('iamgooglepenn')
self.driver.find_element_by_id('fm-login-password').send_keys('HelloWorld1_')
And for login button you can't use .find_element_by_class_name, this method just for single class name. This element having multiple class name, so use .find_element_by_css_selector like bellow:
#submit button
self.driver.find_element_by_css_selector('.fm-button.fm-submit.password-login').click()
The login content seems to be nested in an iFrame element (if you trace it all the way to the top, you should find an iFrame with id="sufei-dialog-content"), which means you need to switch to that iFrame for that nested html before selecting your desired element, otherwise it will not work.
First you will need to use driver.switch_to.frame("sufei-dialog-content"), and then select your element with driver.find_element_by_name() or whatever you had.
A similar issue can be found here: Selenium and iframe in html
Just a simple mistake:
<div class='input-plain-wrap input-wrap-loginid'>
<input id='fm-login-id class='fm-text' name='fm-login-id'...>
event
</div>
is actually supposed to be:
<div class='input-plain-wrap input-wrap-loginid'>
<input id='fm-login-id' class='fm-text' name='fm-login-id'...>
event
</div>
You forgot a single-quote.
Have you tried driver.find_element_by_name('fm-login-id')?
You should try finding the elements by their XPaths. You just have to inspect the element, right-click on it and copy its XPath. The XPath of the first <input ... is //*[#id="fm-login-id"].
So i tried to practice on my selenium skills using the website https://instagram.com
I Can Find an object using selenium & click, can not send keys to it.
So basically , i tried to automate comments on Instagram , iv'e found the "Add Comment" , successfully clicked on it , but when i tried to send keys i got an error.
CODE SECTION:
comment_picture = driver.find_elements_by_tag_name('textarea')
for l in comment_picture:
try:
print l.get_attribute("class")
l.click()
time.sleep(1)
l.send_keys('test')
ERROR SECTION:
Message: stale element reference: element is not attached to the page document
The expected result should be that i can comment on each photo on Instagram.
I don't want an answer. i really want to learn selenium. if someone know that i'm doing wrong , it would be great if i will get an hint and not a full answer.
stale element is because the element has been modified when you click, you have to re-find the element like this
comment_picture = driver.find_elements_by_tag_name('textarea')
index = 1 # xpath index start from 1
for txt in comment_picture:
try:
txt.click()
time.sleep(1)
# re-search the textarea
txt = driver.find_element_by_xpath('(//textarea)[%s]' % index)
txt.send_keys('test')
index = index + 1
element references returned by the .find_element_* functions are from the intial page load. When you click(), you are navigating away from the intial page, making all of the element references stale. You will need to call find_elements again before you send keys to the new elements.
You have to make sure you are signed in and able to comment.
also executable_path='your/path'
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from time import sleep
driver = webdriver.Chrome(executable_path="/home/chromedriver")
driver.get('https://www.instagram.com/p/Bs0y4Myg3Hk/')
comment=driver.find_element_by_xpath('//*[#id="react-root"]/section/main/div/div/article/div[2]/section[3]/div/form/textarea')
comment.send_keys('hello')
comment.send_keys(Keys.RETURN)
sleep(10)
driver.close()
My code so far is:
from selenium import webdriver
driver = webdriver.Chrome()
driver.get('http://moodle.tau.ac.il/')
driver.find_element_by_xpath("id('page-content')//form[#id='login']// \
input[#type='submit']").click()
Now I'm trying to fill up the login form and I succeeded to find the division
that follows id= content, easy to see in the image:
The following code line I used:
elem = driver.find_element_by_xpath("id('content'))
but it doesn't recognize anything in it and I cant get any further, what should I do to locate the input element?
It doesn't recognize anything because it is in an iframe. Therefore, you first have to switch to the iframe and then search the login form.
Switch to the iframe:
frame = driver.find_element_by_id('credentials')
driver.switch_to.frame(frame)
Or:
driver.switch_to.frame('credentials')