I'm trying to scrape this site that has pagination. The problem I'm facing is having selenium locate the next button.
What I've tried:
next_button = driver.find_element_by_css_selector(
'ul[class="css-12ke8jn e65zztl0"] button[aria-label="Next"]').click()
and
page_amount = driver.find_element_by_css_selector(
'/html/body/div[1]/div[2]/div/div/div/div[2]/div[1]/main/div[3]/div/div[2]/div[2]/nav/ul/button').click()
None of these work and I'm kinda stuck. The reason I'm using aria-label for the first one is because when the next button is selected the previous button changes to the same class as the next button. Note: The button is inside a ul.
It might not work finding the element because it's not visible in UI - it is loaded but not visible, the easiest way is to move to that element and click on it.
next_button = driver.find_element_by_css_selector('[aria-label=\'Next\']')
actions = ActionChains(driver)
actions.move_to_element(next_button).perform()
next_button.click()
next_button = driver.find_element_by_xpath('//button[#class="css-1lkjxdl eanm77i0"]').click()
You was using xpath variable and finding it by css. for css selector you have to use the class (.css-1lkjxdl) and use the above code it will work and accept the answer. Thanks!!
aria-label is an attribute, not an element.
Your xpath should be fixed as follow:
button[#aria-label="Next"]
To find this button anywhere in the page, you can try:
//button[#aria-label="Next"]
Then, you can try:
button = driver.find_element_by_xpath('//button[#aria-label="Next"]')
Related
I'm creating an Instagram bot but cannot figure out how to navigate to the next post.
Here is what I tried
#Attempt 1
next_button = driver.find_element_by_class_name('wpO6b ')
next_button.click()
#Attempt 2
_next = driver.find_element_by_class_name('coreSpriteRightPaginationArrow').click()
Neither of two worked and I get a NoSuchElementException or ElementClickInterceptedException . What corrections do I need to make here?
This is the button I'm trying to click(to get to the next post)
I have checked your class name coreSpriteRightPaginationArrow and i couldn't find any element with that exact class name. But I saw the class name partially. So it might help if you try with XPath contains as shown below.
//div[contains(#class,'coreSpriteRight')]
another xpath using class wpO6b. there are 10 elements with same class name so filtered using #aria-label='Next'
//button[#class='wpO6b ']//*[#aria-label='Next']
Try these and let me know if it works.
I have tried below code and it's clicking next button for 10 times
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
if __name__ == '__main__':
driver = webdriver.Chrome('/Users/yosuvaarulanthu/node_modules/chromedriver/lib/chromedriver/chromedriver') # Optional argument, if not specified will search path.
driver.maximize_window()
driver.implicitly_wait(15)
driver.get("https://www.instagram.com/instagram/");
time.sleep(2)
driver.find_element(By.XPATH,"//button[text()='Accept All']").click();
time.sleep(2)
#driver.find_element(By.XPATH,"//button[text()='Log in']").click();
driver.find_element(By.NAME,"username").send_keys('username')
driver.find_element(By.NAME,"password").send_keys('password')
driver.find_element(By.XPATH,"//div[text()='Log In']").click();
driver.find_element(By.XPATH,"//button[text()='Not now']").click();
driver.find_element(By.XPATH,"//button[text()='Not Now']").click();
#it open Instagram page and clicks 1st post and then it will click next post button for the specified range
driver.get("https://www.instagram.com/instagram/");
driver.find_element(By.XPATH,"//div[#class='v1Nh3 kIKUG _bz0w']").click();
for page in range(1,10):
driver.find_element(By.XPATH,"//button[#class='wpO6b ']//*[#aria-label='Next']" ).click();
time.sleep(2)
driver.quit()
As you can see, the next post right arrow button element locator is changing between the first post to other posts next page button.
In case of the first post you should use this locator:
//div[contains(#class,'coreSpriteRight')]
While for all the other posts you should use this locator
//a[contains(#class,'coreSpriteRight')]
The second element //a[contains(#class,'coreSpriteRight')] will also present on the first post page as well, however this element is not clickable there, it is enabled and can be clicked on non-first pages only.
As you can see on the picture below, the wp06b button is inside a lot of divs, in that case you might need to give Selenium that same path of divs to be able to access the button or give it a XPath.
It's not the most optimized but should work fine.
driver.find_element(By.XPATH("(.//*[normalize-space(text()) and normalize-space(.)='© 2022 Instagram from Meta'])[1]/following::*[name()='svg'][2]")).click()
Note that the XPath leads to a svg, so basically we are clicking on the svg element itself, not in the button.
url = http://ptvtelecom.com/
If you follow the url and click the button which says 'combrobar' which should be visible in the middle of the screen, it takes you to a form that needs to be filled out. I was wondering how to fill out the form using selenium.
So ive already tried finding the element by id and by name but it does not work. Any help on how to find the element of the first text box for instance would be greatly aprreaciated.
option = webdriver.ChromeOptions()
option.add_argument(" — incognito")
browser =
webdriver.Chrome(executable_path='/Users/grsanchez/downloads/chromedriverM',
options=option)
browser.get('http://ptvtelecom.com/')
browser.find_element_by_xpath('//*
[#id="cobertura"]/div/div[2]/div/div/p/a').click()
Here is where it goes wrong
name = browser.find_element_by_id('nombre')
name.send_keys('user1')
read the comments in the code to understand why your code isn't working.
basically, you're trying to select something that exists inside an iframe.
option = webdriver.ChromeOptions()
option.add_argument("--incognito")
browser = webdriver.Chrome(executable_path='/Users/grsanchez/downloads/chromedriverM',
options=option)
browser.get('http://ptvtelecom.com/')
## finding the button that shows the form
btn = browser.find_element_by_css_selector('#cobertura .boton-cobertura')
## using js to click it, to avoid getting issues in case the button wasn't visible
driver.execute_script("arguments[0].click();", btn)
## the element you want to select is actually inside an iframe, so we need to switch to it, if we want to select anything
driver.switch_to.frame(driver.find_element_by_css_selector('#popmake-1432 iframe'));
## selecting the name input and sending a string
name = driver.find_element_by_css_selector('#nombre')
name.send_keys('user1')
PS to return to the main frame, you can do this :
driver.switch_to.default_content()
you need to switch to iframe something like driver.switchTo().frame("a077aa5e");
then use your locators inside the iframe
I'm newbie in Selenium. I start to learn Selenium via book. And I struggle with unclear behavior of Selenium. For educational purposes I use this site:
http://magento-demo.lexiconn.com/ - I'm trying to find search button by its class name, (which is: class='button search button') or by it xpath
search_button = self.driver.find_element_by_xpath('/html/body/div/div[2]/header/div/div[4]/form/div[1]/button')
or
search_button = self.driver.find_element_by_class_name('button')
but each time selenium unable to find it. Please help me to understand reason of such behavior. Thank you
I used Selenium IDE and it shows me XPATH: //button[#type='submit']
when I tried to find element by xpath,I have got the same error and it is strange. Please advise.
My code is:
import unittest
from selenium import webdriver
class HomePageTest(unittest.TestCase):
#classmethod
def setUpClass(cls):
#create new Firefox session
cls.driver = webdriver.Firefox()
cls.driver.implicitly_wait(30)
cls.driver.maximize_window()
#navvigate to application home page
cls.driver.get('http://magento-demo.lexiconn.com/')
def test_search__text_field_max_length(self):
#get the search text box
search_field=self.driver.find_element_by_id("search")
#check maxlenght attribute st to 128
self.assertEqual("128",search_field.get_attribute("maxlength"))
def test_search_button_enabled(self):
# get Search button
search_button = self.driver.find_element_by_class_name('button')
# check Search button is enabled
self.assertTrue(search_button.is_enabled())
#classmethod
def tearDown(self):
#close the browser window
self.driver.quit()
if __name__=='__main__':
unittest.main(verbosity=2)
Try this :
search_button = self.driver.find_element_by_xpath('//button[#class="button search-button"]')
Try downloading the selenium IDE plugin, install and start recording. Click on the button you want and view how its target is recorded in the IDE. Programmatically, selenium will accept the same xpaths and other selectors as the IDE. After it's been recorded in the IDE, there is a pull down on the target field that lets you see all the different ways you can select that element, ie xpath vs. by class etc.
http://www.seleniumhq.org/projects/ide/
you might try:
css=button.button.search-button
//button[#type='submit']
//form[#id='search_mini_form']/div/button
I think the issue is that your locator isn't specific enough. There is more than one button on the page and more than one element with class=button on the page. This CSS selector is working for me.
self.driver.find_element_by_css_selector("button[title='Search']")
Try this way using xpath locator
Explanation: Use title attribute of <button> tag.
self.driver.find_element_by_xpath("//button[#title='Search']")
OR
Explanation: Use title and type attribute of <button> tag.
self.driver.find_element_by_xpath("//button[#title='Search'][#type='submit']")
Having serious issues here. Someone please help.
I am trying to login to a website. - This Works!
Redirect to the page I want after Login - This Works!
Now once in, I have to hover over the settings icon so the dropdown shows, and then click on the "Settings" options that has NO ID or CLASS or HREF.
Now there is a couple of reasons I cant do this. Number 1 is if I try to click on the link after the hover, it tells me that it's hidden and I cant interact with it. Also the menu options in the form are populated and appended once you hover, through ajax I think. They are not on the initial page load.
wait = WebDriverWait(driver, 10)
box = wait.until(EC.visibility_of_element_located((By.ID, "yucs-help_button")))
menuButton = driver.find_element_by_id("yucs-help_button")
ActionChains(driver).move_to_element(menuButton).perform()
After the above code is deployed I print the driver.page_source and can see (below) that the new menu options are there, if you don't hover, the below code will not be on the page.
Now the <a> i'm trying to click is the <span>Settings</span> option, and for the life of me, it will not work. Either can't find it, not clickable, can't interact with it, etc, etc, etc. xpath, css_selector, partial_name, nothing finds this thing. Also whats weird is once you click on it, from a browser, it appends an ID to <span> So weird. Any ideas?
<a data-ylk="rspns:nav;t3:tl-lst;t5:custitm;slk:custitm;elm:itm;elmt:custitm;itc:0;cpos:2" class="C(#000)! Td(u):h " data-mad="options" data-customevt="true" href="#" data-rapid_p="18"><span>Settings</span></a>
To perform mouse over event on element you should try to use .execute_script() using following java script :-
wait = WebDriverWait(driver, 10)
box = wait.until(EC.visibility_of_element_located((By.ID, "yucs-help_button")))
menuButton = driver.find_element_by_id("yucs-help_button")
driver.execute_script("var clickEvent = document.createEvent('MouseEvents');clickEvent.initEvent('mouseover', true, true); arguments[0].dispatchEvent(clickEvent);", menuButton)
Now after successfully mouse over you should try to click on Settings link as below :-
driver.find_element_by_xpath("//span[contains(text(), 'Settings')]/parent::a[#data-mad = 'options']").click()
Hope it will help you..:)
currently i'm writing a web scraper that will work in infinite loop. It gets a page, searches for some buttons and clicks one of them. But sometimes it doesn't! I save a screenshot in case of some fail and it showed me that page didnt changed after button clicked.
driver.find_element_by_xpath('//input[#name = "btn"]').click()
time.sleep(3)
I have bypassed this with a loop checking does we see that element still.
while driver.find_elements_by_xpath('//input[#name = "Submit"]') != []:
driver.find_element_by_xpath('//input[#name = "Submit"]').click()
But hope to find a root cause of this. What it could be?
I also faced a similar problem with my application. Clicking the element through action class worked for me.
Here is the sample code in Java:
WebElement webElement = driver.findElement(By.id("Your ID Here"));
Actions builder = new Actions(driver);
builder.moveToElement(webElement).click(webElement);
builder.perform();
If clicking with action class does not work, you can also try clicking element by Javascript.
WebElement webElement = driver.findElement(By.id("Your ID here"));
JavascriptExecutor executor = (JavascriptExecutor) driver;
executor.executeScript("arguments[0].click();", webElement);