hi team,
I am trying to access a dynamic drop down with div as tag but I an able find it but not interact with it
as it changes its style type as shown below.
<div style ="display :none;"></div>"
to
<div style ="display :block;"></div>"
I am unable to click on this, please have a look into the screenshot for detail.
Info you have to click on the element to access this dynamic dropdown,
div is not clickable object in HTML. If it has assigned some JavaScript code to display when you click it then you may need also JavaScript to click it
driver.execute_script("arguments[0].click()", item)
and the same way you can change style
driver.execute_script("arguments[0].style.display = 'block';", item)
In this minimal working example I remove all img on this page.
from selenium import webdriver
url = 'https://stackoverflow.com/questions/65931008/unable-to-interact-with-this-dynamic-drop-down-using-python-selenium'
driver = webdriver.Firefox()
driver.get(url)
all_items = driver.find_elements_by_xpath('//img')
for item in all_items:
print(item.text)
#driver.execute_script("arguments[0].click()", item)
driver.execute_script("arguments[0].style.display = 'none';", item)
Solution -> actually the element we are looking here is masked element which means actual Id of this element is diff so by chance I am able to find it( for this you have go through HTML code row by row and find it, quiet a static way to do but that's how I did it) and place it in the code.
please do comment if you know more efficient way to work around masked elements.
Regards,
Anubhav
Related
I have a script going using Selenium and on a new window pop up I am trying to retrieve this element:
<div class="custom-control p-2">somenumber</div>
The full path to this class is
html/body.modal-open/div#modalUserPRMLead.modal.fade.show/div.modal-dialog/div.modal-content/div.modal-body/div.row.justify-content-center.p-3/div.col-md-12/div.custom-control.p-2
I have tried variants of these to grab just the text of 'somenumber'
driver.find_element(By.CSS_SELECTOR,'row justify-content-center p-3').get_attribute('innerHTML')
driver.find_element(By.XPATH("html/body.modal-open/div.col-md-12./div.custom-control.p-2")).text
driver.find_element(By.XPATH("custom-control p-2")).text
driver.find_element(By.XPATH("div.row.justify-content-center.p-3")).text
driver.find_element(By.XPATH,custom-control p-2).text
driver.find_element(By.XPATH,div.custom-control.p-2).text
The errors I get is that it cannot locate the tag from current session. It's a new window that pops up as a result of previous script, I did driver.current_url just to make sure it was capturing the link to the new window in current session and it was.
I'm not understanding how to index down to the div class and just grab the text. Any help would be greatly appreciated. The script prior to window pop for form input was much easier because I could tag everything using By.NAME or By.ID
Your CSS selector is wrong. XPaths don't really looks like XPaths...
Try
driver.find_element(By.XPATH, '//div[#class="custom-control p-2"]').text
or
driver.find_element(By.CLASS_NAME, 'custom-control').text
In case element is dynamic you might need to apply wait
You can also check basic syntax of XPath and CSS-selector
Here is the inspect result for the button that says +5 per day
>span class="text user-links entry-method-title ng-scope ng-binding" ng-include="::'views/entry-text/'+entry_method.entry_type+'.html'">
Click For a Daily Bonus Entry"
</span>
<div class="entry-method bonus template" data-remove-popovers="" id="em6129519" ng-class="{expanded: entryState.expanded == entry_method, template: entry_method.template, 'completed-entry-method': !canEnter(entry_method) && isEntered(entry_method)}" ng-repeat="entry_method in ::entry_methods">
here is the HTML given information when I inspect the link/button, I have tried to use XPath, CSS, link text, and class name and it keeps giving me an error saying it cannot identify the element. Does anyone have a suggestion for how to identify this, it is on gleam.io for a giveaway I'm trying to automate this so i don't have to log in and press this everyday. This is my first ever web interfacing project with python.
Here is my most recent try
driver.maximize_window()
time.sleep(10)
driver.execute_script("window.scrollTo(0, 1440)")
time.sleep(10)
button2 = driver.find_element_by_class_name("text user-links entry-method-title ng-scope ng-binding")
button2.click()
Similar to a previous issue, Selenium find_element_by_class_name and find_element_by_css_selector not working, you can't have spaces in your class name when using driver.find_element_by_class_name. Instead, find the element via css_selector and replace each space with a dot.
driver.find_element_by_css_selector("span.text.user-links.entry-method-title.ng-scope.ng-binding")
That'll fix what you have above, but keep in mind there are other ways to make selenium actions more reliable (eg. WebDriverWait, etc). And there may be a cleaner selector to use than the one above.
I believe the element you want to access is contained within an "iframe", thus you must first switch to iframe before you can access it using selectors.
driver.switch_to.frame(x.find_element_by_xpath("PUT IFRAME XPATH HERE"))
This is HTML code of that page
From there I want to access the 2nd element by using class name "maxbutton-1" as it has 3 same buttons and I can't use xpath or any constant selector so want to use the indexing with class and can't find anything to do that in python particular.
Also tried the method used in java to do same thing but it didn't worked.
Link of that same page
just trying to automate the movie downloading process for any movie.
Thank you.
To click on first, second or third button, try to change number of element:
el1 = driver.find_element_by_xpath("(//a[#class='maxbutton-1 maxbutton maxbutton-download-links'])[1]")
el2 = driver.find_element_by_xpath("(//a[#class='maxbutton-1 maxbutton maxbutton-download-links'])[2]")
el3 = driver.find_element_by_xpath("(//a[#class='maxbutton-1 maxbutton maxbutton-download-links'])[3]")
then you can extract element href/link attribute like that:
link = el.get_attribute('href')
or click it like that:
el.click()
I am attempting to scrape Kickstarter based on the project names alone. Using the project name and the base URL I can get to the search page. In order to scrape the project page, I need to use Selenium to click on the URL. However, I cannot point Selenium to the correct element to click on. I would also like this to be dynamic so I do not need to put the project name each time.
<div class="type-18 clamp-5 navy-500 mb3">
<a href="https://www.kickstarter.com/projects/1980119549/knife-block-
designed-by-if-and-red-dot-winner-jle?
ref=discovery&term=Knife%20block%20-
%20Designed%20by%20IF%20and%20Red%20dot%20winner%20JLE%20Design"
class="soft-black hover-text-underline">Knife block -
Designed by IF and
Red dot winner JLE Design
</a>
</div>`
driver = webdriver.Chrome(chrome_path)
url = 'https://www.kickstarter.com/discover/advanced?ref=nav_search&term=Knife
block - Designed by IF and Red dot winner JLE Design'
driver.get(url)
elem = driver.find_elements_by_link_text('Knife block - Designed by IF and Red
dot winner JLE Design')
elem.click()
How can I get the elem to point to the correct link?
In regards to your attempt, your code had a typo: using find_elements.... returns a list of elements so the method .click() would not work. You mean to use find_element.
To dynamically click links, use an XPath instead. The resulting code would be:
elem = driver.find_element_by_xpath('//div[contains(#class, "type-18")]/a')
elem.click()
This would grab the first match. You could do find_elements and iterate over the elements but this would be a bad approach because since you're clicking the links, each time that renders the previous page stale. If there's more than one, you could use the same XPath but indexed:
first_elem = driver.find_element_by_xpath('(//div[contains(#class, "type-18")]/a)[1]')
first_elem.click()
# ...
second_elem = driver.find_element_by_xpath('(//div[contains(#class, "type-18")]/a)[2]')
second_elem.click()
# And so forth...
I'm having problem selecting the iframe and accessing the different elements inside it. The iframe name is dynamically generated (e.g. frame11424758092173 or frame0005809321 or frame32138092173). The problem is that Selenium can't find the iframe no matter what i do....
switching to most recent frame doesn't work:
iframe = driver.find_elements_by_tag_name('iframe')[0]
driver.switch_to_frame(iframe)
Waiting for frame gets a timeout exception:
try:
iframe = WebDriverWait(driver, 5).until(EC.frame_to_be_available_and_switch_to_it(By.TAG_NAME('iframe')))
except:
logger.error(traceback.format_exc())
The following lines of code also times out:
try:
iframe = WebDriverWait(driver, 5).until(EC.presence_of_element_located((By.TAG_NAME, u"iframe")))
driver.switch_to_frame(iframe)
except:
logger.error(traceback.format_exc())
I have also tried iterating through the frames but it can't find it. The returned list is empty
iframes = driver.find_elements_by_tag_name('iframe')
#iframes is empty
really need some help...
Have you tried locating the iframe by its XPath and using the contains method?:
iframe = driver.find_element_by_xpath('//iframe[contains(#name, "frame")]')
driver.switch_to_frame(iframe)
Now you can access elements within the iframe.
To exit the iframe use:
driver.switch_to_default_content()
The contains method lets you get an element by a partial attribute value. Pretty useful for dynamically generated IDs, names, etc. You can search by other attributes as well using XPath. For example, say your iframe element has the attribute value = "3". You could use:
iframe = driver.find_element_by_xpath('//iframe[contains(#name, "frame")][#value = "3"]')
driver.switch_to_frame(iframe)
This approach can be used with any number of attributes as well.
You could also try getting the element by its selector. Keep in mind that this limits what you can do with it:
driver.execute_script('document.querySelector("INSERT SELECTOR HERE").doSomething();')
To get the Selector and/or XPath you're going to want to inpect the element using your browser (Chrome in my case). Right click on the element. Click Inspect. Then right click on the HTML element and click Copy > Copy Xpath or Copy > Copy Selector.
If that doesn't work for me, my last resort is to go the url of the iframe.To get that, you need to right-click on the area of the webpage where the iframe exists and click View Frame Source. It'll then lead you to a new page. The url of that page will be shown in the top of the browser after view-source:. You can then simply navigate to that url:
driver.get('insert url of iframe here')
And now you have access to the elements within the iframe. I do not recommend this approach if you are manipulating elements within the iframe and then exiting the iframe. Your changes will get lost. This will only work if you are scraping info off of that iframe, NOT if you are manipulating the elements within. Finding the iframe element and switching into it is usually better and safer.