I have been wondering how I could see when I am at the last page on an amazon listing. I have tried to get at the last page number on the bottom of the screen with nothing working, so I tried a different approach. To see if the 'Next' button can still be clicked. Here is what i have so far, any ideas on why it wont go to the next page?
from selenium.webdriver.chrome.webdriver import WebDriver
from selenium import webdriver
import time
def next():
giveawayPage = 1
currentPageURL = 'https://www.amazon.com/ga/giveaways?pageId=' + str(giveawayPage)
while True:
try:
nextButton = driver.find_element_by_xpath('//*[#id="giveawayListingPagination"]/ul/li[7]')
except:
nextPageStatus = 'false'
print('false')
else:
nextpageStatus = 'true'
giveawayPage = giveawayPage + 1
driver.get(currentPageURL)
if (nextPageStatus == 'false'):
break
if __name__ == '__main__':
driver = webdriver.Chrome('./chromedriver')
driver.get('https://www.amazon.com/ga/giveaways?pageId=1')
next()
The reason this doesn't work is because if you go to the last page of an Amazon Giveaway, the element that you're selecting is still there, it's just not clickable. On most pages, the element looks like:
<li class="a-last">...</li>
On the last page, it looks instead like:
<li class="a-disabled a-last">...</li>
So rather than checking if the element exists, it might be better to check if the element has the class 'a-disabled'.
the li for Next button is always exist, there are several ways to check if it last page:
check if the li has multiple class or not only a-last but also has class a-disabled
//li[#class="a-last"]
check if the <a> element exist in the li
//*[#id="giveawayListingPagination"]/ul/li[7]/a
Related
I'm creating an Instagram bot but cannot figure out how to navigate to the next post.
Here is what I tried
#Attempt 1
next_button = driver.find_element_by_class_name('wpO6b ')
next_button.click()
#Attempt 2
_next = driver.find_element_by_class_name('coreSpriteRightPaginationArrow').click()
Neither of two worked and I get a NoSuchElementException or ElementClickInterceptedException . What corrections do I need to make here?
This is the button I'm trying to click(to get to the next post)
I have checked your class name coreSpriteRightPaginationArrow and i couldn't find any element with that exact class name. But I saw the class name partially. So it might help if you try with XPath contains as shown below.
//div[contains(#class,'coreSpriteRight')]
another xpath using class wpO6b. there are 10 elements with same class name so filtered using #aria-label='Next'
//button[#class='wpO6b ']//*[#aria-label='Next']
Try these and let me know if it works.
I have tried below code and it's clicking next button for 10 times
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
if __name__ == '__main__':
driver = webdriver.Chrome('/Users/yosuvaarulanthu/node_modules/chromedriver/lib/chromedriver/chromedriver') # Optional argument, if not specified will search path.
driver.maximize_window()
driver.implicitly_wait(15)
driver.get("https://www.instagram.com/instagram/");
time.sleep(2)
driver.find_element(By.XPATH,"//button[text()='Accept All']").click();
time.sleep(2)
#driver.find_element(By.XPATH,"//button[text()='Log in']").click();
driver.find_element(By.NAME,"username").send_keys('username')
driver.find_element(By.NAME,"password").send_keys('password')
driver.find_element(By.XPATH,"//div[text()='Log In']").click();
driver.find_element(By.XPATH,"//button[text()='Not now']").click();
driver.find_element(By.XPATH,"//button[text()='Not Now']").click();
#it open Instagram page and clicks 1st post and then it will click next post button for the specified range
driver.get("https://www.instagram.com/instagram/");
driver.find_element(By.XPATH,"//div[#class='v1Nh3 kIKUG _bz0w']").click();
for page in range(1,10):
driver.find_element(By.XPATH,"//button[#class='wpO6b ']//*[#aria-label='Next']" ).click();
time.sleep(2)
driver.quit()
As you can see, the next post right arrow button element locator is changing between the first post to other posts next page button.
In case of the first post you should use this locator:
//div[contains(#class,'coreSpriteRight')]
While for all the other posts you should use this locator
//a[contains(#class,'coreSpriteRight')]
The second element //a[contains(#class,'coreSpriteRight')] will also present on the first post page as well, however this element is not clickable there, it is enabled and can be clicked on non-first pages only.
As you can see on the picture below, the wp06b button is inside a lot of divs, in that case you might need to give Selenium that same path of divs to be able to access the button or give it a XPath.
It's not the most optimized but should work fine.
driver.find_element(By.XPATH("(.//*[normalize-space(text()) and normalize-space(.)='© 2022 Instagram from Meta'])[1]/following::*[name()='svg'][2]")).click()
Note that the XPath leads to a svg, so basically we are clicking on the svg element itself, not in the button.
Im using selenium to check if FB pages exist. When i enter the page title in the search bar it works fine but after the second loop the name of the page gets attached to the preview search and i cant find a way to clear the previous search.
For example it looks for
xyz for the first time
then it looks for
xyzabc when i just want to look for abc this time.
How can i clear the search bar so i can just enter the input without the previous input?
Here is my code
for page_target in df.page_name.values:
time.sleep(3)
inputElement = driver.find_element_by_name("q")
inputElement.send_keys(page_target)
inputElement.submit()
time.sleep(5)
html = driver.page_source
soup = BeautifulSoup(html, 'html.parser').get_text()
title = soup.find(page_target)
#if page exists add 1 to the dic otherwise -1
if title > 0:
dic_holder[page_target] = 1
else:
dic_holder[page_target] = -1
driver.find_element_by_name("q").clear()
time.sleep(3)
You can use
WebElement.clear();//to clear the previous search item
WebElement.sendkeys(abc);//to insert the new search
Also I guess you have a sticky search in your application hence I recommend you to use this method everytime you insert something in the searchbox
Few ways to do it:
Use element.clear(). I see that you already tried in your code, not sure how it didn't work but I guess it is not text box or input element?
Use javascript: driver.execute_script('document.getElementsByName("q")[0].value=""');
Emulate Ctrl+A?
from selenium.webdriver.common.keys import Keys
elem.send_keys(Keys.CONTROL, 'a')
elem.send_keys("page 1")
I want to open 'text' botton but I can't open it.
How can I do?
from selenium import webdriver
import time
driver = webdriver.Chrome(path)
driver.get("https://www.pdfescape.com/account/unregistered/")
time.sleep(0.5)
elem = driver.find_element_by_id("dStartNew").click()
time.sleep(0.5)
elem = driver.find_element_by_id("PdfNew_input_pc").click()
driver.find_element_by_css_selector("[value=\"2\"]").click()
#time.sleep(0.5)
elem = driver.find_element_by_id("PdfNew_input_ps").click()
driver.find_element_by_css_selector("[value=\"a4\"]").click()
#time.sleep(0.5)
driver.find_element_by_xpath('//*[#id="PdfNew_form"]/input[1]').click()
time.sleep(0.5)
driver.find_elements_by_class_name('Add-Text').click()
Problem code : driver.find_elements_by_class_name('Add-Text').click()
The element you wanted to click has no class name - it has the title "Add Text", also you are using find_elements instead of find_element which changes the result type.
So you could select it with:
driver.find_element_by_css_selector('a[title=\"Add Text\"]').click()
Also please let me add another improvement:
driver.find_element_by_xpath('//*[#id="PdfNew_form"]/input[1]').click()
could be much more stable (for the case of page changes) written as:
driver.find_element_by_css_selector('input[type=\"submit\"]').click()
driver.find_elements_by_class_name('Add-Text'),click()
Above code should return a list of elements and you can not click on list of elements
This element is not present in the DOM
To click we use '.' instead of ','
I am trying to keep clicking to next page on this website, each time appending the table data to a csv file and then when I reach the last page, append the table data and break the while loop
Unfortunately, for some reason it keeps staying on the last page, and I have tried several different methods to catch the error
while True:
try :
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.LINK_TEXT, 'Next'))).click()
except :
print("No more pages left")
break
driver.quit()
I also tried this one:
try:
link = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,'.pagination-next a')))
driver.execute_script('arguments[0].scrollIntoView();', link)
link.click()
except:
keep_going = False
I've tried putting print statements in and it just keeps staying on the last page.
Here is the HTML for the first page/last page for the next button, I'm not sure if I could do something utilizing this:
HTML for first page:
<li role="menuitem" ng-if="::directionLinks" ng-class="{disabled: noNext()||ngDisabled}" class="pagination-next" style="">Next</li>
Next
</li>
HTML for last page:
<li role="menuitem" ng-if="::directionLinks" ng-class="{disabled: noNext()||ngDisabled}" class="pagination-next disabled" style="">Next</li>
Next
</li>
You can solve the problem as below,
Next Button Will be enabled until the Last Page and it will be disabled in the Last Page.
So, you can create two list to find the enabled button element and disabled button element. At any point of time,either enabled element list or disabled element list size will be one.So, If the element is disabled, then you can break the while loop else click on the next button.
I am not familiar with python syntax.So,You can convert the below java code and then use it.It will work for sure.
Code:
boolean hasNextPage=true;
while(hasNextPage){
List<WebElement> enabled_next_page_btn=driver.findElements(By.xpath("//li[#class='pagination-next']/a"));
List<WebElement> disabled_next_page_btn=driver.findElements(By.xpath("//li[#class='pagination-next disabled']/a"));
//If the Next button is enabled/available, then enabled_next_page_btn size will be one.
// So,you can perform the click action and then do the action
if(enabled_next_page_btn.size()>0){
enabled_next_page_btn.get(0).click();
hasNextPage=true;
}else if(disabled_next_page_btn.size()>0){
System.out.println("No more Pages Available");
break;
}
}
The next_page_btn.index(0).click() wasn't working, but checking the len of next_page_btn worked to find if it was the last page, so I was able to do this.
while True:
next_page_btn = driver.find_elements_by_xpath("//li[#class = 'pagination-next']/a")
if len(next_page_btn) < 1:
print("No more pages left")
break
else:
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.LINK_TEXT, 'Next'))).click()
Thanks so much for the help!
How about using a do/while loop and just check for the class "disabled" to be included in the attributes of the next button to exit out? (Excuse the syntax. I just threw this together and haven't tried it)
string classAttribute
try :
do
{
IWebElement element = driver.findElement(By.LINK_TEXT("Next"))
classAttribute = element.GetAttribute("class")
element.click()
}
while(!classAttribute.contains("disabled"))
except :
pass
driver.quit()
xPath to the button is:
//li[#class = 'pagination-next']/a
so every time you need to load next page you can click on this element:
next_page_btn = driver.find_elements_by_xpath("//li[#class = 'pagination-next']/a")
next_page_btn.index(0).click()
Note: you should add a logic:
while True:
next_page_btn = driver.find_elements_by_xpath("//li[#class = 'pagination-next']/a")
if len(next_page_btn) < 1:
print("No more pages left")
break
else:
# do stuff
I have to click on each search result one by one from this url:
Search Guidelines
I first extract the total number of results from the displayed text so that I can set the upper limit for iteration
upperlimit=driver.find_element_by_id("total_results")
number = int(upperlimit.text.split(' ')[0])
The loop is then defiend as
for i in range(1,number):
However, after going through the first 10 results on the first page, list index goes out of range (probably because there are no more links to click). I need to click on "Next" to get the next 10 results, and so on till I'm done with all search results. How can I go around doing that?
Any help would be appreciated!
The problem is that the value of element with id total_results changes after the page is loaded, at first it contains 117, then changes to 44.
Instead, here is a more robust approach. It processes page by page until there is no more pages left:
from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException
driver = webdriver.Firefox()
url = 'http://www.nice.org.uk/Search.do?searchText=bevacizumab&newsearch=true#/search/?searchText=bevacizumab&mode=&staticTitle=false&SEARCHTYPE_all2=true&SEARCHTYPE_all1=&SEARCHTYPE=GUIDANCE&TOPICLVL0_all2=true&TOPICLVL0_all1=&HIDEFILTER=TOPICLVL1&HIDEFILTER=TOPICLVL2&TREATMENTS_all2=true&TREATMENTS_all1=&GUIDANCETYPE_all2=true&GUIDANCETYPE_all1=&STATUS_all2=true&STATUS_all1=&HIDEFILTER=EGAPREFERENCE&HIDEFILTER=TOPICLVL3&DATEFILTER_ALL=ALL&DATEFILTER_PREV=ALL&custom_date_from=&custom_date_to=11-06-2014&PAGINATIONURL=%2FSearch.do%3FsearchText%40%40bevacizumab%26newsearch%40%40true%26page%40%40&SORTORDER=BESTMATCH'
driver.get(url)
page_number = 1
while True:
try:
link = driver.find_element_by_link_text(str(page_number))
except NoSuchElementException:
break
link.click()
print driver.current_url
page_number += 1
Basically, the idea here is to get the next page link, until there is no such ( NoSuchElementException would be thrown). Note that it would work for any number of pages and results.
It prints:
http://www.nice.org.uk/Search.do?searchText=bevacizumab&newsearch=true&page=1
http://www.nice.org.uk/Search.do?searchText=bevacizumab&newsearch=true&page=2#showfilter
http://www.nice.org.uk/Search.do?searchText=bevacizumab&newsearch=true&page=3#showfilter
http://www.nice.org.uk/Search.do?searchText=bevacizumab&newsearch=true&page=4#showfilter
http://www.nice.org.uk/Search.do?searchText=bevacizumab&newsearch=true&page=5#showfilter
There is not even the need to programatically press on the Next button, if you see carrefully, the url just needs a new parameter when browsing other result pages:
url = "http://www.nice.org.uk/Search.do?searchText=bevacizumab&newsearch=true&page={}#showfilter"
for i in range(1,5):
driver.get(url.format(i))
upperlimit=driver.find_element_by_id("total_results")
number = int(upperlimit.text.split(' ')[0])
if you still want to programatically press on the next button you could use:
driver.find_element_by_class_name('next').click()
But I haven't tested that.