I'd like to scrape a website and it has elements something like:
<mn>567</mn>
<mo>+</mo>
<mn>213</mn>
How do I extract the text from these elements using Selenium? [element].text does not seem to work.
Related
While using selenium to automate reverse address search I am unable to retrieve the information from a card in the DOM. When I copy the specific XPATH of the text not to my surprise its a text object instead of an element. Any solutions? image
So, I know there's a bunch of ways to obtain an element on a webpage by looking up its tag in selenium, but I was wondering if it was possible to do the reverse. Can I obtain the immediate enclosing tag of some text that I look up on a page using selenium?
Hi I am attempting to scrape multiple pages using selenium in python. I am interested in extracting all elements that fall within a span class element, basically what I would like to do is get the span class elements then extract the link within it. For each page it is possible to achieve this by using the xpath, however the xpath changes for each object and for each page. here is an example of what the web elements look like:
essentially I would like to extract the elements this is consistent in all the pages that I will be scraping. SO my idea is to get these elements then to get the href elements for these. I have tried to get all the elements on the page using this code
driver.find_elements_by_xpath("//span[#class='Text__StyledText-jknly0-0 cCEhaW']")
However this has not worked and it returns nothing. I also do not want to use the inner class because it varies by page as well so the only real element to use if I want to automate the scraping without getting too messy is that element I mention. Any way to extract the links for this span class elements on the page?
try this xpath
//span[contains(#class,'Text__StyledText')]//a[contains(#class,'Anchor__StyledAnchor')]
To actually grab that element we use the following
driver.find_elements_by_css_selector("span.Text__StyledText-jknly0-0.cCEhaW")
Currently i use Python with selenium for scraping purpose. There are many ways in selenium to scrape data. And I used to use css selectors.
But then I realised that,
Only tagNames are those things which always are on websites.
For example,
Not every website uses classes or Id's like, take an example of Wikipedia. They use normally just tags in it.
like <h1>, <a> without having any classes or id in it.
There comes the limitation for scraping USING tagNames, as they scrape every element under their tags.
For example : if I want to scrape table contents which are under <p> tag, then it scrapes the table contents as well as all the descriptions which are not needed.
My question is: is it possible to scrape the required elements under the tags which do not copy every other elements under their tags?
Like if I want to scrape content from, say Amazon then it will select only product names under h1 tags, not scraping all the headings under the h1 tag which are not product names.
If you find any other method/locator to use, even except the tagName also then also you can tell me. But the condition is that it must be present on every website/ most of the websites
Any help would be appreciated 😊...
I am trying to get a csv file but selenium is not able to find the element
i am able to fill the form. website url is "https://www.nseindia.com/products/content/equities/equities/eq_security.htm"
to download the csv i have to click on element with text = Download file in csv format but it's not finding it.
print(browser.find_element_by_xpath('//*
#id="historicalData"]/div1/span[2]/a').click())
i have tried using css selector and link text, tag name but getting error not able to locate element.
link is highlighted here
So what you are going to need here is
browser.find_element_by_xpath('//a[contains(text(), 'Download file in csv format')]').click()
An absolute xpath, with a lot of elements in it, is very sensitive to change.