I want to click this button in a website using selenium but I am not sure how to select it.
<a data-gfm-analytics-element="btn_showmore_browse" data-area="norma_category" class="button hollow expanded-mobile js-load-more-results" href="#" style="display: inline-block;">Show More</a>
I used this code to do it but it returns NoSuchElementException: Message: no such element
btn = driver.find_element(By.CLASS_NAME,'button hollow expanded-mobile js-load-more-results')
button hollow expanded-mobile js-load-more-results are actually 4 class names while By.CLASS_NAME receives single class name value. You can use CSS Selector or Xpath with multiple class names
btn = driver.find_element(By.CSS_SELECTOR,'a.button.hollow.expanded-mobile.js-load-more-results')
Also btn_showmore_browse seems to be unique value, so possibly this will work too:
btn = driver.find_element(By.CSS_SELECTOR,'a[data-gfm-analytics-element="btn_showmore_browse"]')
There are more possible options
Related
I'm fairly new to the web scraping world but I really need to do some web scraping on the Thesaurus website for a project I'm working on. I have successfully created a program using beautifulsoup4 that asks the user for a word, then returns the most likely synonyms based on Thesaurus. However, I would like to not only have those synonyms but also the synonyms of every sense of the word (which is depicted on Thesaurus by a list of buttons above the synonyms). I noticed that when clicking a button, the name of the classes also change, so I did a little digging and decided to go with Selenium instead of beautifulsoup.
I have now a code that writes a word on the search bar and clicks it, however, I'm unable to get the synonyms or the said buttons, simply because the find_element finds nothing, and being new to this, I'm afraid I'm using the wrong syntax.
This is my code at the moment (it looks for synonyms of "good"):
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
import time
PATH = "C:\Program Files (x86)\chromedriver_win32\chromedriver.exe"
driver = webdriver.Chrome(PATH)
driver.get("https://thesaurus.com")
search = driver.find_element_by_id("searchbar_input")
search.send_keys('good')
search.send_keys(Keys.RETURN)
try:
headword = WebDriverWait(driver,10).until(
EC.presence_of_element_located((By.ID, "headword"))
)
print(headword.text)
#buttons = headword.find_element_by_class_name("css-bjn8wh e1br8a1p0")
#print(buttons.text)
meanings = WebDriverWait(driver,10).until(
EC.presence_of_element_located((By.ID, "meanings"))
)
print(meanings.text)
#words = meanings.find_elements_by_class_name("css-1kg1yv8 eh475bn0")
#print(words.text)
except:
print('failed')
driver.quit()
For the first part, I want to access the buttons. The headword is simply the element that contains all the buttons I want to press. This is the headword element according to the inspect tool:
<div id="headword" class="css-bjn8wh e1br8a1p0">
<div class="css-vw3jp5 e1ibdjtj4">
*unecessary stuff*
<div class="css-bjn8wh e1br8a1p0">
<div class="postab-container css-cthfds ew5makj3">
<ul class="css-gap396 ew5makj2">
<li data-test-pos-tab="true" class="active-postab css-kgfkmr ew5makj4">
<a class="css-sc11zf ew5makj1">
<em class="css-1v93s5a ew5makj0">adj.</em>
<strong>pleasant, fine</strong>
</a>
</li>
<li data-test-pos-tab="true" class=" css-1ha4k0a ew5makj4">
*similar stuff*
<li data-test-pos-tab="true" class=" css-1ha4k0a ew5makj4">
...
where each one these <li data-test-pos-tab="true" class=" css-1ha4k0a ew5makj4"> is a button I want to click. So far I have tried a bunch of things like the one showed in the code, and also things like:
buttons = headword.find_elements_by_class_name("css-1ha4k0a ew5makj4")
buttons = headword.find_elements_by_css_selector("css-1ha4k0a ew5makj4")
buttons = headword.find_elements_by_class_name("postab-container css-cthfds ew5makj3")
buttons = headword.find_elements_by_css_selector("postab-container css-cthfds ew5makj3")
but in any case Selenium can find these elements.
For the second part I want the synonyms. Here is the meaning element:
<div id="meanings" class="css-16lv1yi e1qo4u831">
<div class="css-1f3egm3 efhksxz0">
*unecessary stuff*
<div data-testid="word-grid-container" class="css-ixatld e1cc71bi0">
<ul class="css-1ngwve3 e1ccqdb60">
<li>
<a font-weight="inherit" href="/browse/acceptable" data-linkid="nn1ov4" class="css-1kg1yv8 eh475bn0">
</a>
</li>
<li>
<a font-weight="inherit" href="/browse/bad" data-linkid="nn1ov4" class="css-1kg1yv8 eh475bn0">
...
where each of these elements is a synonym I want to get. Similarly to the previous case I tried several things such as:
synGrid = meanings.find_element_by_class_name("css-ixatld e1cc71bi0")
synGrid = meanings.find_element_by_css_selector("css-ixatld e1cc71bi0")
words = meanings.find_elements_by_class_name("css-1kg1yv8 eh475bn0")
words = meanings.find_elements_by_css_selector("css-1kg1yv8 eh475bn0")
And again Selenium cannot find these elements...
I would really appreciate some help in order to achieve this, even if it is just a push in the right direction instead of giving a full solution.
Hope I wrote all the needed information, if not, please let me know.
If you use css selector then you have to use dot for class
css_selector(".css-ixatld.e1cc71bi0")
and hash for id
css_selector("#headword")
like you would use in files .css
In css selector you can use also other methods avaliable in CSS.
See css selectors on w3schools.com
Selenium converts class_name to css selector but class_name() expects single name and Selenium has problems when there are two or more names. When it converts class_name to css_selector then it adds dot only before first name but it needs dot also before second and other names. So you have to manually add second dot
class_name("css-ixatld.e1cc71bi0")
See if this works:
meanings = driver.find_elements_by_xpath(".//div[#id='meanings']/div[#data-testid='word-grid-container']/ul/li")
for e in meanings:
e.find_element_by_tag_name("a").click()
//Add a implicit wait if you need
driver.back()
I used the following line to click the Availability Grid button, but it failed to locate the element.
Class Sarsa-button-content is used everywhere so, I added text together to make it unique. However, it couldn't find it. What am I missing?
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//a[#class='sarsa-button-content']/span[text()='Availability Grid']"))).click()
<div class="sticky-top-wrapper" style="top: 80px;">
<div class="site-filter-container" id="site-filter-container">
<a data-component="Button" class="sarsa-button view-by-availability-grid--button-tracker sarsa-button-primary sarsa-button-sm" href="/site/123456/availability">
<span class="sarsa-button-inner-wrapper">
<span class="sarsa-button-content">Availability Grid</span>
your xpath is wrong, try this:
//span[contains(#class, 'sarsa-button-content') and text() = 'Availability Grid']
I'm trying to get Selenium to click on View All Companies button, but i'm not sure what am I doing wrong. It returns no element found
html code
<div class="screener-toggles">
<div class="buttons">
<span class="button selected" data-name="advanced-screener">Search by Screener<span data-name="advanced-screener" class="arrow selected"></span></span>
<span class="button" data-name="alpha-factors">Search by Alpha Factors<span data-name="alpha-factors" class="arrow"></span></span>
<span class="button" data-name="all-companies">View All Companies<span data-name="all-companies" class="arrow"></span></span>
</div>
</div>
python code I wrote
element1 = driver.find_elements_by_class_name('View All Companies')
element1.click()
# I have tried all-companies instead of View All Companies as well. But still doesn't work
Should I not be using find_elements_by_class_name?
Any advice on what I am doing wrong is greatly appreciated!
try xpath: "//span[contains(text(),'View All Companies')]"
View All Companies is text, not the class. Try looking by text with css_selector or xpath
element1 = find_element_by_css_selector('span:contains("View All Companies")')
element1 = find_element_by_xpath('//span[contains(text(), "View All Companies")]')
Or by the data-name attribute which contains all-companies
element1 = find_element_by_css_selector('span[data-name*="all-companies"]')
Yes, you should not use the find_elements_by_class_name instead of use find_element_by_class_name.
find_elements_by_class_name is used when your expecting your locator to return more than 1 element. for a specific element use only find_element_by_class_name.
Another thing is I am not able to see any class name as View All Companies in your HTML code. Please look into your HTML and select classname or other locator carefully
Hope it will help you
I am using Selenium in python to click a button in a dialog. I try to click"OK", but it keeps getting errors
the buttons show "display:block" in CSS, cause "element is not visible" error
find_element_by_xpath, but the xpath of the element keeps changing
the class names are the same, how to choose the "OK" button?
Here are the code
<div<class="ui-dialog-buttonpane ui-widget-content ui-helper-clearfix">
<div<class="ui-dialog-buttonset">
<button type="button" class="large ui-button ui-widget ui-state-default ui-corner-all ui-button-text-only" role="button" aria-disabled="false">
<span class="ui-button-text">Cancel</span>
</button>
<button type="button" class="orange large ui-button ui-widget ui-state-default ui-corner-all ui-button-text-only" role="button" aria-disabled="false">
<span class="ui-button-text">OK</span>
</button>
</div>
</div>
Thank you:)
If the element is invisible in the DOM like css { display: None}, {opacity: 0}... etc, , selenium will not be able to SEE it even given that you try to wait or time.sleep, instead you should use execute_script to run a JavaScript to trigger the desired event, something like this:
driver.execute_script('document.querySelector("span.ui-button-text").click();')
You should wait for the element to be ready before trying to click on it. Use waitForVisible or similar to achieve that.
For example, something like this:
element = WebDriverWait(driver, 10).until(
lambda driver : driver.find_element_by_xpath(element_xpath)
)
If the class remains the same, then you should select that class with element_xpath. The last thing you need to determine is what other attribute designates the button as ready. Then you can wait for the specific argument like:
def find(driver):
e = driver.find_element_by_xpath(element_xpath)
if (e.get_attribute(some_attribute)==some_value):
return False
return e
element = WebDriverWait(driver, 10).until(find)
Two different things can happen here.
• The selector is not explicitly returning the intended element
• Element load issue.
If both cases are true use explicit wait with a correct selector. In terms of selector I like using text bases search in such scenario. Notice I am using xpatth contains to make sure it eliminates any leading or tailing white spaces.
element = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH , "//span[contains(text(),'OK')]")))
API doc here
I am testing a website that has buttons that are not discernible by element or id. However, I am able to identify it by xpath or css class. How do you click on a button using one of those two attributes. I'm able to assert the button is there using get_element_by_xpath; however, I seem unable to use the click_element, click_link, or click_button using id_or_elem. What statement would you use to click on a button like this?
Edit:
The test code looks like this: (webpage name removed on purpose)
class TestMyTest(cases.SSTTestCase):
def test_mytestcase_home_page(self):
go_to('http://www.mywebpage.com')
assert_title_contains('Page Name')
assert_element(tag='a', text='Log in')
click_element(id_or_elem='page_nav')
assert_title_contains('Login')
assert_element(id='tbLUser')
write_textfield('tbLUser','username')
assert_element(id='tbLPass')
write_textfield('tbLPass','badpassword')
assert_element(css_class='btn-login1')
get_element_by_xpath('//*[#id="login_box"]/div/a/img')
The code I'm trying to test from:
<div class="btn-login1">
<a href="javascript:void(0)" onclick=javascript:return Validate(document.theLoginForm,'/mypage/mywebpage.asp');" onmouseout="MM_swapImgRestore();" onmouseover="MM_swapImage('lg5_login','','/assets/page4/lg5_login_f2.png',1);">
<img name="lg5_login" src="http://www.mywebpage.com/login/lg5_login.png" border="0" alt="Log in Now">
</a>
</div>
If this answer is helpful to someone else. We we able to solve the issue with the following solutions.
click_element(get_element_by_xpath('your xpath'), wait=True)
click_link(get_element_by_xpath('your xpath'), wait=True)
click_button(get_element_by_xpath('your xpath'), wait=True)
Parameters: id_or_elem The identifier of the element, or its element object.
All of the click actions should support id or element