I am working on selenium in python, I want to scrape all pages, but I am in trouble:
Here is the element I want to click:
I am using the folloing code:
link=driver.find_element_by_link_text ('2')
link.click()
But it give click on another element
Deos there exist another way to get next page?
First of all, sees like your element what you're trying to click overlapped by another one, so you need to wait for its becoming being clickable or other one disappear:
el = WebDriverWait(driver, 15).until(EC.element_to_be_clickable((By.XPATH,//div[#id="pagination_wrapper"]//li[#value="1"])))
or
WebDriverWait(driver, LONG_TIMEOUT
).until_not(EC.presence_of_element_located((By.XPATH,"//div[#class='close_cookie_alert']")))
Here's like you can find all of yours elements:
link1 = driver.find_element_by_xpath('//div[#id="pagination_wrapper"]//li[#value="1"]')
link2 = driver.find_element_by_xpath('//div[#id="pagination_wrapper"]//li[#class="2"]')
link3 = driver.find_element_by_xpath('//div[#id="pagination_wrapper"]//li[contains(text(),"text of the third element")]')
if usual click doesn't work, try to use click via javascript, like that:
driver.execute_script("arguments[0].click();", link1)
or, just move to the next page with:
driver.get('new_page')
Related
I've been trying to access elements of a website using Selenium but keep getting this message :
selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable
The problem I'm having seems to be that the website uses the same classes' name between two tabs, and just changes the visibility of one to display the other. For instance, when the first tab is displayed the first div's visibility is visible and the second is hidden. If the second tab is displayed, then the second div is visible, and the first hidden.
For the first tab, I have no problem accessing the desired elements. However, if the second tab is displayed, I get the above error message.
Is there a way to switch between to div like one would do between two iframe ?
This is what my code looks like for the moment :
driver.get(url) #get url
driver.find_element(By.XPATH, "//input[#id='ctl00_cphMaster_ConsultationCitoyenCriteresRecherche_CritereRechercheTabContainer_RechercheSimpleTabPanel_RechercheSimpleControl_MotsClesTextBox']").send_keys(string) #enter searched term
driver.find_element(By.XPATH, "//input[#id='ctl00_cphMaster_ConsultationCitoyenCriteresRecherche_CritereRechercheTabContainer_RechercheSimpleTabPanel_RechercheSimpleControl_PorteeRadioButtonList_1']").click()
driver.find_element(By.XPATH, "//input[#class='Bouton BoutonRechercher']").click() #click the search button
WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.XPATH,"//div[#class='Header']")))
driver.find_element(By.XPATH, '//*[#id="__tab_ctl00_cphMaster_ConsultationCitoyenResultatsRecherche_ResultatsRechercheTabContainer_RechercheLETabPanel"]').click() #switch tab
WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.XPATH,"//div[#class='Header']")))
while True:
listings = driver.find_elements_by_class_name("InscriptionSummaryValue")
c = 1
for i in listings:
page = int(driver.find_element(By.XPATH, "//span[#class='CurrentPage']").text) #can't find the element because it seems to be located in the first div
I'm facing an issue locating the element on screen when there are no unique identifiers like ID, text etc. As it opens URL, I need to scroll down and click on button - 'Get Started' to proceed!...
Below is my code:
global driver
driver = webdriver.Chrome(ChromeDriverManager().install())
driver.maximize_window()
driver.get("https://My URL")
driver.implicitly_wait(10)
screen = driver.find_element(By.XPATH, '//div[#class="swiper-wrapper"]')
screen.click() (- This step doesnt through any error, i tried using this to scroll down the screen)
element = driver.find_element(By.XPATH, '//span[contains(text(),"Get Started")]')
driver.execute_script("arguments[0].scrollIntoView(true);", element )
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//span[contains(text(),"Get Started")]'))).click()
or
element.click()
Please help me in determining how to locate the element.
enter image description here
In this case you are trying to find span which is inside #shadow-root, Selenium won't be able to click elements inside #shadow-root in easy way. Look here: How to click button inside #shadow-root (closed) using Selenium and Python
But in your case you probably don't need/want to click this specific span, becouse you have custom element ion-button, which has some click event listener attached to it.
You can check XPATH value in browser devtools. If you go to Elements tab, you can right click desired element and choose option Copy>Copy Xpath. Also make sure your element is not inside iframe tag, it could cause problem as well.
Im trying to perform select and click action from the search box result dropdown for testing purpose. Though i dont get ant error but i'm stuck and not able to do so, search results came then disappeared immediately. Please any one help me out. Im using Python script to automate webdriver. Here is the screenshot below for reference.
. I have tried webdriverwait for same action but it gives Timeout exception error. If there is any child actions from CSS to perform let me know. Here is what i tried
search = driver.find_element_by_id('searchInput')
search.send_keys("flowers")
dropdown = WebDriverWait(driver, 4).until(
EC.presence_of_element_located((By.XPATH, "//li[text()='flowers']")))
Apart from this snippet, i want to rather just perform enter key operation, where i get query result for 'flower' on this ecomm. website.
Here is the website URL- https://paytmmall.com
Once you type flower in the input field, there are multiple options appearing based on the input provided. They are in li tags and under b tag.
Code :
driver = webdriver.Chrome(driver_path)
driver.maximize_window()
#driver.implicitly_wait(30)
wait = WebDriverWait(driver, 30)
driver.get("https://paytmmall.com/")
search = wait.until(EC.visibility_of_element_located((By.ID, "searchInput")))
search.send_keys("flowers")
time.sleep(3)
wait.until(EC.visibility_of_element_located((By.XPATH, "(//li)[4]/descendant::b[contains(text(),'flowers')]"))).click()
time.sleep is just for visibility purpose. you can remove that as well.
Also this xpath (//li)[4]/descendant::b[contains(text(),'flowers')] is based on xpath indexing , since I think you wanna select the 4th option which is flower itself. In case you wanna select a different option, you would have to write the different xpath.
In case you are looking to just select the searched item, it's better to pass enter key once you type flower in the input field.
You can use the below code for that :
search = wait.until(EC.visibility_of_element_located((By.ID, "searchInput")))
search.send_keys("flowers")
time.sleep(3)
search.send_keys(Keys.RETURN)
The suggested options are not containing the text directly in li elements, they are inside child elements inside li elements.
Try this instead:
search = driver.find_element_by_id('searchInput')
search.send_keys("flowers")
dropdown = WebDriverWait(driver, 4).until(
EC.visibility_of_element_located((By.XPATH, "//li//*[text()='flowers']")))
my problem is that the first on is clicking and the second one doesnt and i dont undersatand why
WebDriverWait(rootdiv, 10).until(EC.element_to_be_clickable((By.XPATH, "//li[#onclick[contains(.,JTC)]]")))
rootdiv.find_element_by_xpath("//li[#onclick[contains(.,'JTC')]]").click()
WebDriverWait(depdiv, 10).until(EC.element_to_be_clickable((By.XPATH, "//li[#onclick[contains(.,AJU)]]")))
depdiv.find_element_by_xpath("//li[#onclick[contains(.,'AJU')]]").click()
depdiv and root div are children to look under for the certain li cause the root changes from the first to the second..
i've checked that the div is visible and in the first one, its clicking, the second time it cant find the object and im getting a time error
part of the code im trying to fed from..
<div class = "divCombo4">
<ul><li>....</li><ul>
<ul><li>...</li><ul>
</div>
and my depdiv is rooting to the find_element_by_id("divCombo4")
one of those ul li contains
onclick="selecionou('AJU', this,'.txtBusca4', 'false', 'destino', 'Estouem2', 'AJU');"
First of all, you need a dot to make the expressions context-specific. Also, the wait.until() returns a WebElement instance and you can use it instead of issuing a "find element" command again:
WebDriverWait(rootdiv, 10).until(EC.element_to_be_clickable((By.XPATH, ".//li[#onclick[contains(.,JTC)]]"))).click()
WebDriverWait(depdiv, 10).until(EC.element_to_be_clickable((By.XPATH, ".//li[#onclick[contains(.,AJU)]]"))).click()
I'm having problem selecting the iframe and accessing the different elements inside it. The iframe name is dynamically generated (e.g. frame11424758092173 or frame0005809321 or frame32138092173). The problem is that Selenium can't find the iframe no matter what i do....
switching to most recent frame doesn't work:
iframe = driver.find_elements_by_tag_name('iframe')[0]
driver.switch_to_frame(iframe)
Waiting for frame gets a timeout exception:
try:
iframe = WebDriverWait(driver, 5).until(EC.frame_to_be_available_and_switch_to_it(By.TAG_NAME('iframe')))
except:
logger.error(traceback.format_exc())
The following lines of code also times out:
try:
iframe = WebDriverWait(driver, 5).until(EC.presence_of_element_located((By.TAG_NAME, u"iframe")))
driver.switch_to_frame(iframe)
except:
logger.error(traceback.format_exc())
I have also tried iterating through the frames but it can't find it. The returned list is empty
iframes = driver.find_elements_by_tag_name('iframe')
#iframes is empty
really need some help...
Have you tried locating the iframe by its XPath and using the contains method?:
iframe = driver.find_element_by_xpath('//iframe[contains(#name, "frame")]')
driver.switch_to_frame(iframe)
Now you can access elements within the iframe.
To exit the iframe use:
driver.switch_to_default_content()
The contains method lets you get an element by a partial attribute value. Pretty useful for dynamically generated IDs, names, etc. You can search by other attributes as well using XPath. For example, say your iframe element has the attribute value = "3". You could use:
iframe = driver.find_element_by_xpath('//iframe[contains(#name, "frame")][#value = "3"]')
driver.switch_to_frame(iframe)
This approach can be used with any number of attributes as well.
You could also try getting the element by its selector. Keep in mind that this limits what you can do with it:
driver.execute_script('document.querySelector("INSERT SELECTOR HERE").doSomething();')
To get the Selector and/or XPath you're going to want to inpect the element using your browser (Chrome in my case). Right click on the element. Click Inspect. Then right click on the HTML element and click Copy > Copy Xpath or Copy > Copy Selector.
If that doesn't work for me, my last resort is to go the url of the iframe.To get that, you need to right-click on the area of the webpage where the iframe exists and click View Frame Source. It'll then lead you to a new page. The url of that page will be shown in the top of the browser after view-source:. You can then simply navigate to that url:
driver.get('insert url of iframe here')
And now you have access to the elements within the iframe. I do not recommend this approach if you are manipulating elements within the iframe and then exiting the iframe. Your changes will get lost. This will only work if you are scraping info off of that iframe, NOT if you are manipulating the elements within. Finding the iframe element and switching into it is usually better and safer.