So, I was absolutely baffled as to how to do this in Selenium, and couldn't find the answer anywhere, so I'm sharing my experience.
I was trying to select an iframe and having no luck (or not repeatably anyway). The HTML is:
<iframe id="upload_file_frame" width="100%" height="465px" frameborder="0" framemargin="0" name="upload_file_frame" src="/blah/import/">
<html>
<body>
<div class="import_devices">
<div class="import_type">
<a class="secondary_button" href="/blah/blah/?source=blah">
<div class="import_choice_image">
<img alt="blah" src="/public/images/blah/import/blah.png">
</div>
<div class="import_choice_text">Blah Blah</div>
</a>
</div>
</div>
</body>
</html>
The Python code (using the selenium library) was trying to find this iframe using this:
#timed(650)
def test_pedometer(self):
sel = self.selenium
...
time.sleep(10)
for i in range(5):
try:
if sel.select_frame("css=#upload_file_frame"): break
except: pass
time.sleep(10)
else: self.fail("Cannot find upload_file_frame, the iframe for the device upload image buttons")
Repeated fails with every combination of Selenium commands I could find.
The occasional success would not be reproducible, so perhaps it was some sort of race condition or something? Never did find the right way to get it in selenium proper.
This worked for me with Python (v. 2.7), webdriver & Selenium when testing with iframes and trying to insert data within an iframe:
self.driver = webdriver.Firefox()
## Give time for iframe to load ##
time.sleep(3)
## You have to switch to the iframe like so: ##
driver.switch_to.frame(driver.find_element_by_tag_name("iframe"))
## Insert text via xpath ##
elem = driver.find_element_by_xpath("/html/body/p")
elem.send_keys("Lorem Ipsum")
## Switch back to the "default content" (that is, out of the iframes) ##
driver.switch_to.default_content()
If iframe is dynamic node, it's also possible to wait for iframe appearence explicitly and then switch to it using ExpectedConditions:
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait as wait
driver = webdriver.Chrome()
driver.get(URL)
wait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it("iframe_name_or_id"))
If iframe doesn't have #id or #name it can be found as common WebElement using driver.find_element_by_xpath(), driver.find_element_by_tag_name(), etc..:
wait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it(driver.find_element_by_xpath("//iframe[#class='iframe_class']")))
To switch back from iframe:
driver.switch_to.default_content()
What finally worked for me was:
sel.run_script("$('#upload_file_frame').contents().find('img[alt=\"Humana\"]').click();")
Basically, don't use selenium to find the link in the iframe and click on it; use jQuery. Selenium has the capability to run an arbitrary piece of javascript apparently (this is python-selenium, I am guessing the original selenium command is runScript or something), and once I can use jQuery I can do something like this: Selecting a form which is in an iframe using jQuery
You don't need to use JavascriptExecutor. All you needed to do was switch into the frame and then switch back out, like so:
// do stuff on main window
driver.switch_to.frame(frame_reference)
// then do stuff in the frame
driver.switch_to.default_content()
// then do stuff on main window again
As long as you are careful with this, you will never have a problem. The only time I always use a JavascriptExecutor is to get window focus since I think using Javascript is more reliable in that case.
To shift Selenium's focus within the <iframe> you can use either of the following Locator Strategies:
Using ID:
driver.switch_to.frame("upload_file_frame")
Using CSS_SELECTOR:
driver.switch_to.frame(driver.find_element(By.CSS_SELECTOR, "iframe#upload_file_frame"))
Using XPATH:
driver.switch_to.frame(driver.find_element(By.XPATH, "//iframe[#id='upload_file_frame']"))
Ideally, you have to induce WebDriverWait for the desired frame to be available and switch to it and you can use either of the following Locator Strategies:
Using ID:
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.ID,"upload_file_frame")))
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe#upload_file_frame")))
Using XPATH:
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[#id='upload_file_frame']")))
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Reference
You can find a couple of relevant discussions in:
Ways to deal with #document under iframe
Switch to an iframe through Selenium and python
Selenium's selectFrame command accepts all the standard locators like css=, but it also has a an extra set of locators that work specifically with FRAME and IFRAME elements.
As the doc says:
selectFrame ( locator ) Selects a frame within the current window.
(You may invoke this command multiple times to select nested frames.)
To select the parent frame, use "relative=parent" as a locator; to
select the top frame, use "relative=top". You can also select a frame
by its 0-based index number; select the first frame with "index=0", or
the third frame with "index=2".
You may also use a DOM expression to identify the frame you want
directly, like this: dom=frames["main"].frames["subframe"]
Arguments: locator - an element locator identifying a frame or iframe
In general, you'll have better luck using the specialized locators, especially if you establish the right context first (e.g., select_frame("relative=top"); select_frame("id=upload_file_frame");).
you can use this simple code to look for the iframe using xpath
sample use
set_iframe("/html/body/div[2]/table/")
def set_iframe(xpath):
print('set_iframe')
driver.switch_to.default_content()
seq = driver.find_elements_by_tag_name('iframe')
for x in range(0, len(seq)):
driver.switch_to.default_content()
print ("iframe-"+str(x))
try:
driver.switch_to.frame(int(x))
driver.find_element_by_xpath(xpath)
return str(x)
except:
continue
Related
I'm trying to write some text to an element which looks like this:
The HTML code looks like this:
What I'am trying to do is to figure out, how it would be possible to send text in the form via Selenium.
I've tried alot of things but unfortinatly nothing works. Currently this is the code I have at the moment. I choose to use #class as an indicator of the frame as this value doesn´t change everytime the website reloads.
iframeDescription = driver.find_element_by_xpath("//iframe[#class='ifrm']")
driver.switch_to.frame(iframeDescription)
time.sleep(2)
print(iframeDescription)
formInput = driver.find_element(By.CSS_SELECTOR, "html")
formInput.send_keys("a random Text I wish would appear inside this Box")
I would appreciate some advice.
Edit: thanks for the quick answers guys, I don´t have input tho- or am I missing something? I´ve wrote "test" into the form on the website to try and locate it.
Pic3
The element is within an <iframe> so you have to:
Induce WebDriverWait for the desired frame to be available and switch to it.
Induce WebDriverWait for the desired element to be clickable.
Note: You can't send a character sequence to the body tag, instead you have to locate the relevant <input> tag to send the text.
You can use either of the following Locator Strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe[title='Standard']")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "body input"))).send_keys("Sawa")
Using XPATH:
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[#title='Standard']")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//body//input"))).send_keys("Sawa")
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Reference
You can find a couple of relevant discussions in:
Switch to an iframe through Selenium and python
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element while trying to click Next button with selenium
selenium in python : NoSuchElementException: Message: no such element: Unable to locate element
I'm building a scraper with selenium, and I got it up and running except for one data field I'm missing. I need to first push a button, then get the value. But I haven't been able to push the button, because I can't seem to build the correct xpath expression to get it with selenium,
The url is this one: https://www.ikea.com/mx/es/p/kallax-estante-blanco-80275887/
And I need to click the button "Medidas" (scroll halfway through the page, right between "Detalles del producto" and "Productos similares") to open a side panel and get the info I need.
But so far, I haven't been able to set the correct XPATH expression for the button.
I tried with
driver.find_element_by_xpath(
"//button[#class='range-revamp-chunky-header']"
).click()
But this way it clicked the first button ("Detalles del producto")
Also tried with something like
driver.find_element_by_xpath(
"//button[#class='range-revamp-chunky-header' AND #text='Medidas']"
).click()
But I haven't been able to make it work, I just got this error:
Message: no such element: Unable to locate element: {"method":"xpath","selector":"//button[#class='range-revamp-chunky-header' and #text='Medidas']"}
By the way, the list of buttons is dynamic... sometimes, there can be 1, 2, 3 or 4 buttons with the same class, only difference I can see is the text of the button, so getting always the 2nd button with that class (like in the URL provided) won't always work.
To click on the element with text as Medidas you can use the following Locator Strategy:
Using xpath:
driver.find_element(By.XPATH, "//span[text()='Medidas']").click()
Ideally you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategy:
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//*[contains(., 'login or register')]"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
I am trying to scrape information on a website where the information is not immediately present. When you click a certain button, the page begins to load new content on the bottom of the page, and after it's done loading, red text shows up as "Assists (At Least)". I am able to find the first button "Go to Prop builder", which doesn't immediately show up on the page, but after the script clicks the button, it times out when trying to find the "Assists (At Least)" text, in spite of the script sleeping and being present on the screen.
from selenium import webdriver
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
import time
from bs4 import BeautifulSoup
driver = webdriver.Chrome()
driver.get('https://www.bovada.lv/sports/basketball/nba')
# this part succeeds
element = WebDriverWait(driver, 10).until(
EC.presence_of_element_located(
(By.XPATH, "//span[text()='Go to Prop builder']")
)
)
element.click()
time.sleep(5)
# this part fails
element2 = WebDriverWait(driver, 6).until(
EC.visibility_of_element_located(
(By.XPATH, "//*[text()='Assists (At Least)']")
)
)
time.sleep(2)
innerHTML = driver.execute_script('return document.body.innerHTML')
driver.quit()
soup = BeautifulSoup(innerHTML, 'html.parser')
The problem is the Assist element is under a frame. You need to switch to the frame like this:
frame = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CLASS_NAME,"player-props-frame")))
driver.switch_to.frame(frame)
Increase the timeout to confirm the timeout provided is correct, You can also confirm using debug mode. If still issue persist, please check "Assists (At Least)" element do not fall under any frame.
You can also share the DOM and proper error message if issue not resolved.
I have a couple of suggestions you could try,
Make sure that the content loaded at the bottom of the is not in a frame. If it is, you need to switch to the particular frame
Check the XPath is correct, try the XPath is matching from the Developer Console
Inspect the element from the browser, once the Developer console is open, press CTRL +F and then try your XPath. if it's not highlighting check frames
Check if there is are any iframes in the page, search for iframe in the view page source, and if you find any for that field which you are looking for, then switch to that frame first.
driver.switch_to.frame("name of the iframe")
Try adding a re-try logic with timeout, and a refresh button if any on the page
st = time.time()
while st+180>time.time():
try:
element2 = WebDriverWait(driver, 6).until(
EC.visibility_of_element_located(
(By.XPATH, "//*[text()='Assists (At Least)']")
)
)
except:
pass
The content you want is in an iFrame. You can access it by switching to it first, like this:
iframe=driver.find_element_by_css_selector('iframe[class="player-props-frame"]')
driver.switch_to.frame(iframe)
Round brackets are the issue here (at least in some cases...). If possible, use .contains selector:
//*[contains(text(),'Assists ') and contains(text(),'At Least')]
How to know if an element is in an iframe or not?
I would like to learn how we can see if an element that we are searching is in an iframe or not. I could not find a submit button in a website until I saw a stackoverflow about it. They said that the submit button was in an iframe. When I look in the source code I can't see any iframe.
<button type="submit" class="btn btn-submit" name="submitBtn">Créer un compte</button>
If you want to have a look at the website: https://mail.protonmail.com/create/new?language=en
Normally you can use Ctrl+U to see source HTML in browser and then you can use Ctrl+F to search any text ie. iframe. But Ctrl+U gives original HTML from server without elements added by JavaScript. But Mostly iframe is in original HTML and this method works.
But if page uses JavaScript to add even iframe then you may need DevTools in Firefox/Chrome to see HTML with all added elements.
You can also use Selenium to check if there are iframe - ie.
driver.find_elements_by_tag_name('iframe')
but if page uses JavaScript to add iframe then you may have to wait for new elements.
You can use primitive but easy to remember
time.sleep(seconds)
or Selenium has methods to check if elements is on page - see doc Waits.
Because it may raise error when it can't find element so it need try/except and code is longer.
wait = WebDriverWait(driver, 5)
try:
wait.until(EC.presence_of_element_located((By.TAG_NAME, "iframe")))
except Exception as ex:
#print(ex)
print('len(all_iframes): 0')
BTW: It could use selenium.common.exceptions.TimeoutException instead of Exception but it is harder to remember so I skip it here.
This code use time.sleep() to wait.
BTW: to switch to next iframe you have to first go back to main frame. Without switching it may search next iframe inside current iframe.
import selenium.webdriver
import time
url = 'https://mail.protonmail.com/create/new?language=en'
driver = selenium.webdriver.Firefox()
driver.get(url)
time.sleep(5) # JavaScript may need some time to add all elements
all_iframes = driver.find_elements_by_tag_name('iframe')
print('len(all_iframes):', len(all_iframes))
for number, frame in enumerate(all_iframes):
print('--- frame:', number, '---')
# switch to iframe
driver.switch_to.frame(frame)
# search buttons in iframe
all_buttons = driver.find_elements_by_tag_name('button')
print('len(all_buttons):', len(all_buttons))
# go back to main frame
driver.switch_to.default_content()
Result:
len(all_iframes): 2
--- frame: 0 ---
len(all_buttons): 0
--- frame: 1 ---
len(all_buttons): 1
EDIT:
Version with WebDriverWait but it needs to wait also for buttons because it doesn't wait full 5 seconds but at once when it find iframes it try to search buttons but JavaScript may need also time to add buttons in iframe
import selenium.webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
#import time
url = 'https://mail.protonmail.com/create/new?language=en'
driver = selenium.webdriver.Firefox()
driver.get(url)
#time.sleep(5) # JavaScript may need time to add all elements
wait = WebDriverWait(driver, 5)
try:
wait.until(EC.presence_of_element_located((By.TAG_NAME, "iframe")))
except Exception as ex:
#print(ex)
print('len(all_iframes): 0')
all_iframes = driver.find_elements_by_tag_name('iframe')
print('len(all_iframes):', len(all_iframes))
for number, frame in enumerate(all_iframes):
print('--- frame:', number, '---')
# switch to iframe
driver.switch_to.frame(frame)
try:
wait.until(EC.presence_of_element_located((By.TAG_NAME, "button")))
all_buttons = driver.find_elements_by_tag_name('button')
print('len(all_buttons):', len(all_buttons))
except Exception as ex:
#print(ex)
print('len(all_buttons): 0')
# go back to main frame
driver.switch_to.default_content()
Right click on the button, if it's inside a frame it will show you frame options , Refer to this article-- https://www.guru99.com/handling-iframes-selenium.html
I am trying to automate logging into a website (http://www.phptravels.net/) using Selenium - Python on Chrome. This is an open website used for automation tutorials.
I am trying to click an element to open a drop-down (My Account button at the top navbar) which will then give me the option to login and redirect me to the login page.
The HTML is nested with many div and ul/li tags.
I have tried various lines of code but haven't been able to make much progress.
driver.find_element_by_id('li_myaccount').click()
driver.find_element_by_link_text(' Login').click()
driver.find_element_by_xpath("//*[#id='li_myaccount']/ul/li[1]/a").click()
These are some of the examples that I tried out. All of them failed with the error "element not visible".
How do I find those elements? Even the xpath function is throwing this error.
I have not added any time wait in my code.
Any ideas how to proceed further?
Hope this code will help:
from selenium import webdriver
from selenium.webdriver.common.by import By
url="http://www.phptravels.net/"
d=webdriver.Chrome()
d.maximize_window()
d.get(url)
d.find_element(By.LINK_TEXT,'MY ACCOUNT').click()
d.find_element(By.LINK_TEXT,'Login').click()
d.find_element(By.NAME,"username").send_keys("Test")
d.find_element(By.NAME,"password").send_keys("Test")
d.find_element(By.XPATH,"//button[text()='Login']").click()
Use the best available locator on your html page, so that you need not to create xpath of css for simple operations
You may be having issues with the page not being loaded when you try and find the element of interest. You should use the WebDriverWait class to wait until a given element is present in the page.
Adapted from the docs:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
# Set up your driver here....
try:
element = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.ID, 'li_myaccount'))
)
element.click()
except:
#Handle any exceptions here
finally:
driver.quit()