for chrome, I install ChroPath to find elements on the page.
I want to find XPath for like elements on Instagram Page, but seems that not work :
//span[contains(#class,'glyphsSpriteHeart__outline__24__grey_9 u-__7')]
also, I try it :
/html[1]/body[1]/div[3]/div[1]/div[2]/div[1]/article[1]/div[2]/section[1]/span[1]/button[1]/span[1]
when selenium click :
elenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"div._2dDPU.vCf6V div.zZYga div.PdwC2._6oveC article.M9sTE.L_LMM.JyscU div.eo2As section.ltpMr.Slqrh span.fr66n button.coreSpriteHeartOpen.oF4XW.dCJp8 > span.glyphsSpriteHeart__outline__24__grey_9.u-__7"}
how can I find XPath? any good extension or something?
how can I find XPath? any good extension or something?
You cannot "find" the Xpath of an element. There are many, many XPath's that will find any element. Some will be stable, others will be unstable. The decision on which Xpath to use is based upon your understanding and experience of Selenium, and you understanding of how the Application Under Test is written and behaves.
If you are looking for a tool to experiment with different XPaths, then Chrome's built-in Developer Tools Console allows you to test both Xpath & CSS Selectors;
In your specific scenario about finding elements by class name, then CSS Selector is a much better choice than XPath as CSS selectors will treat multiple classes as an array where as XPath sees "class" as a literal string, hence why you needed to use "contains".
This might help:
https://selectorgadget.com/
This as well, to understand what you are manipulating:
https://www.w3schools.com/xml/xpath_syntax.asp
As for your example where you go down the tree using index numbers (ie: /html[1]/body[1]), A slight change in the site will make your script to fail. Find a way to build something more robust! Also have a look at CSS selectors if you object's appearance is known in advance.
To get all like buttons on instagram use css selector below:
span[aria-label="Like"]
You can get some helpful details here: https://www.w3schools.com/cssref/css_selectors.asp
Related
I've been researching this for two days now. There seems to be no simple way of doing this. I can find an element on a page by downloading the html with Selenium and passing it to BeautifulSoup, followed by a search via classes and strings. I want to click on this element after finding it, so I want to pass its Xpath to Selenium. I have no minimal working example, only pseudo code for what I'm hoping to do.
Why is there no function/library that lets me search through the html of a webpage, find an element, and then request it's Xpath? I can do this manually by inspecting the webpage and clicking 'copy Xpath'. I can't find any solutions to this on stackoverflow, so please don't tell me I haven't looked hard enough.
Pseudo-Code:
*parser is BeautifulSoup HTML object*
for box in parser.find_all('span', class_="icon-type-2"): # find all elements with particular icon
xpath = box.get_xpath()
I'm willing to change my code entirely, as long as I can locate a particular element, and extract it's Xpath. So any other ideas on entirely different libraries are welcome.
I'm searching for a tag in class.I tried many methods but I couldn't get the value.
see source code
The data I need is inside the "data-description".
How can i get the "data-description" ?
I Tried some method but didn't work
driver.find_element_by_name("data-description")
driver.find_element_by_css_selector("data-description")
I Solved this method:
icerisi = browser.find_elements_by_class_name('integratedService ')
for mycode in icerisi:
hizmetler.append(mycode.get_attribute("data-description"))
Thanks for your help.
I think css selector would work best here. "data-description" isn't an element, it's an attribute of an element. The css selector for an element with a given attribute would be:
[attribute]
Or, to be more specific, you could use:
[attribute="attribute value"]
Here's a good tip:
Most web browsers have a way of copying an elements Selector or XPATH. For example, in Safari if you view the source code then right-click on an element it will give you the option to copy it. Then select XPATH or Selector and in your code use driver.find_element_by_xpath() or driver.find_element_by_css_selector(). I am certain Google Chrome and Firefox have similar options.
This method is not always failsafe, as the XPATH can be very specific, meaning that slight changes to the website will cause your script to crash, but it is a quick and easy solution, and is especially useful if you don't plan on reusing your code months or years later.
So, I have an XPath (I've verified this works and has 1 unique value via Google Chrome Tools.
I've tried various methods to try and get this xpath, initally using right click > copy xpath in chrome gave me:
//*[#id="hdr_f0f7cdb71b9a3f44782b87386e4bcb3e"]/th[2]/span/a
However, this ID changes on every reload.
So, i eventually got it down to:
//th[#name="name"]/span/a/text()
element = driver.find_element_by_xpath("//th[#name='name']/span/a/text()")
print(element)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//th[#name='name']/span/a/text()"}
check this:
len(driver.find_elements_by_xpath('//*[contains(#id, "hdr_")]'))
if you won't get a lot of elements you're done with this:
driver.find_elements_by_xpath('//*[contains(#id, "hdr_")]')
You should not be using /text() for a WebElement. Use "//th[#name='name']/span/a" as the Xpath and print the text using element.text (Not sure about the exact method for Python, but in Java it is element.getText() )
I will suggest using absolute XPath rather than relative XPath it might resolve it if the id is changing with every load. please see below how absolute XPath is different from relative using the google search bar.
Relative XPath - //*[#id="tsf"]/div[2]/div/div[1]/div/div[1]/input
absolute XPath - /html/body/div/div[3]/form/div[2]/div/div[1]/div/div[1]/input
I understand as you said you cannot share a link but people here can help if you can share inspect element snapshot showing DOM. so that if there is an issue in XPath it can be rectified. Thanks :-)
I'm trying to learn how to click around using selenium. I have tried some different websites like reddit, google etc without success.
driver.get('https://www.dropbox.com/login')
driver.find_element_by_xpath('//a[#href="' + 'https://www.dropbox.com/forgot?email_from_login=' + '"]').click()
and
continue_link = driver.find_element_by_partial_link_text('Sign in')
They both exist but neither works. What am I doing wrong?
You're being a little specific in your xpath, so there're many more ways you might bugger something up. Using the xpath you can simply do:
driver.find_element_by_xpath("//*[#class='forgot-password-link']").click()
I make no assumptions, but just in case you aren't already, there's a very handy tool in Chrome's inspect element which lets you click an element and jumps to its node in the inspector.
Can you try the following:
from selenium.webdriver.common.by import By
driver.findElement(By.cssSelector(".login-button.signin-button.button-primary")).click()
You can use a shorter single class selector to Sign In button
driver.find_element_by_css_selector(".login-button").click()
Sign in with Google
driver.find_element_by_css_selector(".sign-in-text").click()
For forgot password
driver.find_element_by_css_selector(".forgot-password-link").click()
Single class css selectors will be the fastest method (faster than xpath and compound class)
I am scraping individual listing pages from justproperty.com (individual listing from the original question no longer active).
I want to get the value of the Ref
this is my xpath:
>>> sel.xpath('normalize-space(.//div[#class="info_div"]/table/tbody/tr/td[norma
lize-space(text())="Ref:"]/following-sibling::td[1]/text())').extract()[0]
This has no results in scrapy, despite working in my browser.
The following works perfectly in lxml.html (with modern Scrapy uses):
sel.xpath('.//div[#class="info_div"]//td[text()="Ref:"]/following-sibling::td[1]/text()')
Note that I'm using // to get between the div and the td, not laying out the explicit path. I'd have to take a closer look at the document to grok why, but the path given in that area was incorrect.
Don't create XPath expression by looking at Firebug or Chrome Dev Tools, they're changing the markup. Remove the /tbody axis step and you'll receive exactly what you're look for.
normalize-space(.//div[#class="info_div"]/table/tr/td[
normalize-space(text())="Ref:"
]/following-sibling::td[1]/text())
Read Why does my XPath query (scraping HTML tables) only work in Firebug, but not the application I'm developing? for more details.
Another XPath that gets the same thing: (.//td[#class='titles']/../td[2])[1]
I tried your XPath using XPath Checker and it works fine.