Trying to click on the button on "https://euw.op.gg/summoner/userName=JengaSneaky". But I can't find the element that trigger it.
The pic shows one of the buttons. I want Selenium to click on it so I can scrape the data that pops up. I've tried to find the element but it says I can't use it with click().
The problem with these kind of buttons is that they are generated each time you load the html ( or the page ) based on the data contained in a database that changes everyday.
That's why they don't come with a unique id or something to distinguish them from others.
The thing you could do here is to find them by the CSS SELECTOR or by XPATH.
But you will always have to check if they changed everyday.
What you could also try to do is locate them if they contain a specific text.
Related
I have a problem getting text from a table that can be shown by a button click, the data for that table are present even before the button be clicked, but when I try to get it, it returns nothing, if I try to push the button first, it successfully gets the text, is there a solution for this ? By the way, I am using Xpath to get the element.
table = f'(//div[child::table[child::tbody[child::tr[child::td[text()="Student Name:"]]]]])[{self.row}]'
self.studentName = self.dataBrowser.find_element(By.XPATH, table+'/table[1]/tbody/tr[1]/td[2]').text
Probably you are missing a delay here.
I mean probably you are extracting the text from web element before it is fully loaded, containing it text content.
In several cases waiting for the visibility of the element you are extracting the text from will resolve your problem.
To give more precise answer we need to see all your code, including the page you are working on to make some debugging and testing.
I'm new to using Selenium, and I am having trouble figuring out how to click through all iterations of a specific element. To clarify, I can't even get it to click through one as it's a dropdown but is defined as an element.
I am trying to scrape fanduel; when clicking on a specific game you are presented with a bunch of main title bets and in order to get the information I need to click the dropdowns to get to that information. There is also another drop down that states, "See More" which is a similar problem, but assuming this gets fixed I'm assuming I will be able to figure that out.
So far, I have tried to use:
find_element_by_class_name()
find_element_by_css_selector()
I have also used them in the sense of elements, and tried to loop through and click on each index of the list, but that did not work.
If there are any ideas, they would be much appreciated.
FYI: I am using beautiful soup to scrape the website for the information, I figured Selenium would be helpful making the information that isn't currently accessible, accessible.
This image shows the dropdowns that I am trying to access, in this case the dropdown 'Win Margin'. The HTML code is shown to the left of it.
This also shows that there are multiple dropdowns, varying in amount based off the game.
You can also try using action chains from selenium
menu = driver.find_element_by_css_selector(".nav")
hidden_submenu = driver.find_element_by_css_selector(".nav # submenu1")
ActionChains(driver).move_to_element(menu).click(hidden_submenu).perform()
Source: here
am new to coding and have been trying to select a dropdown box to select underlying options. Have been able to interact with other drop-down on this page, though this particular one seems different as I've been unable to access/interact with it
I've included a snapshot of the webpage's drop-down box, source codes for the specific drop-down box, and codes that I've already tried and had errors with.
Would appreciate any help/pointers. Am a complete newb to coding (python or HTML!), but know a little about Excel VBA.
website: https://www.firstsentierinvestors.com/sg/en/retail/performance/price-and-performance.html
Tried to paste a picture though it seems I'm too new to do so directly. I'm trying to interact with the "Select Fund" drop-down box on the page. Have tried the following codes:
browser.find_element_by_id('strategy').send_keys(fund_strat) #works fine
browser.find_element_by_xpath('//*[#id="share-class"]').click()
#this line directly above does not work, including variants below:
browser.find_element_by_xpath('THE BELOW XPATHS').click()
//*[#id="price-performance-form"]/div[3]
//*[#id="price-performance-form"]/div[3]/div
//*[#id="price-performance-form"]/div[3]/div/a
//*[#id="price-performance-form"]/div[3]/div/a/span
//*[#id="share-class"]
//*[#id="share-class"]/option[1]
Also tried using CSS selector
browser.find_element_by_css_selector('a.option-selected').click()
browser.find_element_by_css_selector('a.option-selected').send_keys('NAME OF FUNDS ETC')
Image describing problem
If the below line works fine for you then,
browser.find_element_by_xpath('//*[#id="share-class"]').click()
#this line directly above does not work, including variants below:
You need to wait for the element which is visible on the DOM.
i.e.,css = <div class="custom-select custom-select-open">
These days most of the web apps are using AJAX techniques.
So you need to add a wait duration (till the element gets visible on the page or get loaded on the DOM).
So, Try using Selenium Waits(put below code):
from selenium.webdriver.support import expected_conditions as EC
wait = WebDriverWait(driver, 10)
element = wait.until(EC.element_to_be_clickable((By.XX, 'element')))
XX = CSS, XPATH, ID, etc.
Now, If you want to write on the selected text box then just try to send keys on input element.
browser.find_element_by_css_selector('.editable-select input')
for much better selector use parent elements to locate the exact element.
CSS = 'div[class$="custom-select custom-select-open"] .editable-select input'
If you want to select dropdown options and you know the Values. Try using below CSS.
browser.find_element_by_css_selector('select option[value$="Your Value"]')
or
If you don't know the values then please attach whole DOM feed after typing into the text box. I'll try to help you out with the Dropdown selection of item.
Else, there is few more steps are there that you need to workout. Get all elements using inside text.
Note: You can approach the text search of an element using XPATH. Example : xpath = "//*[contains(text(),'your text')]"
I am attempting to scrape the Census website for ACS data. I have scripted the whole processes using Selenium except the very last click. I am using Python. I need to click a download button that is in a window that pops when the data is zipped and ready, but I can't seem to identify this button. It also seems that the button might change names based on when it was last run, for example, yui-gen2, yui-gen3, etc so I am thinking I might need to account for this someone. Although I normally only see yui-gen2.
Also, the tag seems to be in a "span" which might be adding to my difficulty honing in on the button I need to click.
Please help if you can shed any light on this for me.
code snippet:
#Refine search results to get tables
driver.find_element_by_id("prodautocomplete").send_keys("S0101")
time.sleep(2)
driver.find_element_by_id("prodsubmit").click()
driver.implicitly_wait(100)
time.sleep(2)
driver.find_element_by_id("check_all_btn_above").click()
driver.implicitly_wait(100)
time.sleep(2)
driver.find_element_by_id("dnld_btn_above").click()
driver.implicitly_wait(100)
driver.find_element_by_id("yui-gen0-button").click()
time.sleep(10)
driver.implicitly_wait(100)
driver.find_element_by_id("yui-gen2-button").click()
enter image description here
enter image description here
Instead of using the element id, which as you pointed out varies, you can use XPath as Nogoseke mentioned or CSS Selector. Be careful to not make the XPath/selector too specific or reliant on changing values, in this case the element id. Rather than using the id in XPath, try expressing the XPath in terms of the DOM structure (tags):
//*/div/div/div/span/span/span/button[contains(text(),'Download')]
TIL you can validate your XPath by using the search function, rather than by running it in Selenium. I right-clicked the webpage, "inspect element", ctrl+f, and typed in the above XPath to validate that it is the Download button.
For posterity, if the above XPath is too specific, i.e. it is reliant on too many levels of the DOM structure, you can do something shorter, like
//*button[contains(text(),'Download')]
although, this may not be specific enough and may require an additional field, since there may be multiple buttons on the page with the 'Download' text.
Given the HTML you provided, you should be able to use
driver.find_element_by_id("yui-gen2-button")
I know you said you tried it but you didn't say if it works at all or what error message you are getting. If that never works, you likely have an IFRAME that you need to switch to.
If it works sometimes but not consistently due to changing ID, you can use something like
driver.find_element_by_xpath("//button[.='Download']")
On the code inspection view on Chrome you can right click on the item you want to find and copy the xpath. You can they find your element by xpath on Selenium.
I want to get the transcript from the youtube video below (I know that there is a way to get the cc captions if available, but often they are not).
I use phantomjs to do this in python.
url = 'https://www.youtube.com/watch?v=wOn8xawC-HQ'
phantom_driver = webdriver.PhantomJS(executable_path="./phantomjs-2.1.1- macosx/bin/phantomjs")
phantom_driver.get(url)
The transcript only appears after clicking on the "More ..." button which I can access with:
phantom_driver.find_element_by_id('action-panel-overflow-button').click()
... this creates a div with the compound class "yt-uix-menu-content yt-ui-menu-content yt-uix-kbd-nav yt-uix-menu-content-hidden" which contains a menu with a <ul> consisting of 4 elements in it.
I need to click on one of these four elements to open the transcript box (specifically, I need to click the button with the compound class "yt-ui-menu-item has-icon yt-uix-menu-close-on-select action-panel-trigger action-panel-trigger-transcript").
However, running
phantom_driver.find_element_by_class_name('action-panel-trigger-transcript').click()
does not work since the element seems still hidden (Element is not currently visible and may not be manipulated).
I believe what I need is clicking on the "More..." button and then directly clicking on the opened menu.
I've looked into this post on select menues and this one on using the html after the click. Neither did quite solve my problem.
What am I missing in my script? Do I need to "refresh" the retrieved html after the click?
"action-panel-overflow-button" is button id not name.
you can click this element by it's xpath
this works fine:
phantom_driver.find_element_by_xpath('//*[#id="action-panel-overflow-button"]').click()
phantom_driver.find_element_by_xpath('//*[#class="yt-ui-menu-item has-icon yt-uix-menu-close-on-select action-panel-trigger action-panel-trigger-transcript"]').click()