I have this in my code:
link_tag = "//div[#class= 'yuRUbf']//a/#href"
With this code I get this error
The result of the xpath expression "//div[#class= 'yuRUbf']//a/#href" is: [object Attr]. It should be an element.
I don't know any other way to scrape the URL from that div class. How can I fix this?
This code works fie on the scraper chrome extension but not on python.
You have to change your XPath to select an element node (as the error message suggests) - and not an attribute node - and, after that, get its attribute. So use
link_tag = "//div[#class= 'yuRUbf']//a"
links = driver.find_elements_by_xpath(link_tag)
and then extract the attribute with
links[0].get_attribute("href")
to get the #href attribute of the first matching element.
This solution might work.
div = driver.find_element(By.CSS_SELECTOR,"div[class='yuRUbf']")
url = div.get_attribute("href")
Related
Friends, I'm doing a web scraping. I'm looking for an element that has the XPath changed every time it's searched. For that I'm looking for a way to get the correct xpath.
For this I will need to get the Xpath of an element that I can already locate, it is not the element that I need to locate. But with his Xpath I can find the Xpath of the desired element. so i'm searching:
element = self.chrome.find_element_by_xpath("//div[text()='Apelação']")
With that I need to get your XPath which is:
//*[#id="consultarProcessoForm:dtProcessos_data"]/tr[2]/td[4]/div
How can I do this?
I tried using this; but it returned nothing
print(element.get_attribute("id"))
This is the code of the page:
enter image description here
I am trying to get the href with selenium and python.
This is my page:
Some class information are changing depending on which elements. So I am trying basically to get all href for <a id="job____ .....
links.append(job.find_element_by_xpath('//a[#aria-live="polite"]//span').get_attribute(name="href"))
I tried couple of things but can't figure out how. How can i get all my href from the screenshot above?
Try this, but take care your xpath
"//a[#aria-live="polite"]//span"
will get a span, and i dont see any span with href on your html. Maybe this xpath solve it
//a[./span[#aria-live="polite"]]
links.append(job.find_element_by_xpath('//a[./span[#aria-live="polite"]]').get_attribute("href"))
But it wont get all urls, this with find_elements (return a list), extend your url list with list comprehension
links.extend([x.get_attribute("href") for x in job.find_elements_by_xpath('//a[./span[#aria-live="polite"]]')])
edit 1, other xpath solution
links.extend(["website_base_url"+x.get_attribute("href") for x in job.find_elements_by_xpath('//a[contains(#id, "job_")]')])
list_of_elements_with_href = wd.find_elements_by_xpath("//a[contains(#href,'')]")
for el_with_href in list_of_elements_with_href :
links.append(el.with_href.get_attribute("href"))
or if you need more specify:
list_of_elements_with_href = wd.find_elements_by_xpath("//a[contains(#href,'') and contains(#id,'job_')]")
Based on your description and attached image, I think you have got the wrong xpath. Try the following code.
find_links = driver.find_elements_by_xpath("//a[starts-with(#id,'job_')]")
links = []
for link in find_links:
links.append(link.get_attribute("href"))
Please note elements in find_elements_by_xpath instead of element.
I am unable to test this solution as you have not provided the website.
Here is the code of the web
The xpath of search-results-list container grid is
//[#id="product_type_products_list"]/div/div[2]/div
and the xpath of result is
//*[#id="product_type_products_list"]/div/div[2]/div/div[1]
I have try using :
elems = driver.find_elements_by_xpath('//*[#id="product_type_products_list"]/div/div[2]/div')
url = driver.find_element_by_link_text(elems[0].text).get_attribute("href")
print(url)
this give the link to the beginning of the web.
Thank you for your consideration.
The code you've provided doesn't look like a valid HTML to me, however you can try the following XPath expression:
//div[#class='result']/descendant::a
More information:
XPath Tutorial
XPath Axes
XPath Functions and Operators
Try Narrowing it down to the <'A> Tag by appending the xpath like so:
elems = driver.find_elements_by_xpath('.//*[#id="product_type_products_list"]/div/div[2]/div/div[1]/a')
Then just retrieve the href attribute like you did earlier but using the same element:
url = elems[0].get_attribute("href")
I am using the following code using Python 3.6 and selenium:
element = driver.find_element_by_class_name("first_result_price")
print(element)
on the website it is like this
`website: span class="first_result_price">712
however if I print element I get a completely different number?
Any suggestions?
many thanks!!
"element" is a type of object called WebElement that Selenium adds. If you want to find the text inside that element, you have to say
element.text
Which should return what you're looking for, '712', albeit in string form.
I'm trying to get the XPATH for Code Generator field form (Facebook) in order to fill it (of course before I need to put a code with "numbers").
In Chrome console when I get the XPATH I get:
//*[#id="approvals_code"]
And then in my test I put:
elem = driver.find_element_by_xpath("//*[#id='approvals_code']")
if elem: elem.send_keys("numbers")
elem.send_keys(Keys.ENTER)
With those I get:
StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
What means wrong field name. Does anyone know how to properly get a XPATH?
This error usually comes if element is not present in the DOM.
Or may be element is in iframe.