I know the very basics of using following-siblings but here I have a situation where it looks a bit more complicated.
I want to find the element with text Total 6.5 where the header is Total games. How can I do it with following-siblings and contains text?
<div class="group">
<div class="header_1">
<div class="section_1">
<div class="expander"></div>
<div class="star"></div>
<div class="text_3">Total games</div>
</div>
</div>
<div class="body_1">
<div class="horizontal">
<div class="grid">
<div class="row_common">
<div class="cell_wrap">
<div class="cell_align_wrap">
<div class="common_text">Total 6.5</div>
</div>
</div>
</div>
<div class="row_common">
...
</div>
</div>
</div>
</div>
</div>
This one should locate required element
//div[#class="header_1" and contains(., "Total games")]/following-sibling::div[#class="body_1"]//div[#class="common_text"]
you can also simplify it as
//div[#class="header_1" and contains(., "Total games")]/following::div[#class="common_text"]
Related
Im struggling with scraping a few pages ... it happens when the structure of the page implies a lot of nested divs...
Here is the code page:
<div>
<section class="ui-accordion-header ui-state-default ui-corner-all ui-accordion-icons" role="tab" id="ui-id-1" aria-controls="ui-id-2" aria-selected="false" aria-expanded="false" tabindex="0"><span class="ui-accordion-header-icon ui-icon ui-icon-triangle-1-e"></span>
<div class="detail-avocat">
<div class="nom-avocat">Me <span class="avocat_name">NAME </span></div>
<div class="type-avocat">Avocat postulant au Tribunal Judiciaire</div>
</div>
<div class="more-info">Plus d'informations</div>
</section>
<div class="ui-accordion-content ui-helper-reset ui-widget-content ui-corner-bottom" style="display: none;" id="ui-id-2" aria-labelledby="ui-id-1" role="tabpanel" aria-hidden="true">
<div class="details">
<div class="detail-avocat-row ">
<div class="detail-avocat-content overflow-h">
<span>Structure :</span>
<div>
<p>Cabinet individuel NAME</p>
</div>
</div>
</div>
<div class="detail-avocat-row ">
<div class="detail-avocat-content overflow-h">
<span>Adresse :</span>
<div>
<p>21 rue Belle Isle 57000 VILLE</p>
</div>
</div>
</div>
<div class="detail-avocat-row ">
<div class="detail-avocat-content overflow-h">
<span>Mail :</span>
<div>
<p>cabinet#mail.fr</p>
</div>
</div>
</div>
<div class="detail-avocat-row">
<div class="detail-avocat-content overflow-h">
<span>Tél :</span>
<div>
<p>Telnum</p>
</div>
</div>
</div>
<div class="detail-avocat-row">
<div class="detail-avocat-content overflow-h">
<span>Fax :</span>
<div>
<p> </p>
</div>
</div>
</div>
<div class="contact-avocat"> Contacter </div>
</div>
</div>
</div>
And here is my python code:
divtel = self.driver.find_elements(by=By.XPATH,
value=f'//div[#class="detail-avocat-content overflow-h"]/div/p')#div[#class="detail-avocat-content overflow-h"]')
for p in divtel:
print(p.text)
It doesnt print anything...with other similar pages it prints the text but in this case it doesnt altough there is text in the nested span and div/p . Do you know why?
How can i resolve my problem please?
thank you
The method .text works only when the webelement containing the text is visible in the webpage. If otherwise the webelement is hidden, you have to use .get_attribute('innerText') or .get_attribute('textContent') or .get_attribute('innerHTML') (see here for difference between them). So for example change
print(p.text)
to
print(p.get_attribute('innerText'))
The HTML is located below, If the span value is less than 20%, then I want to remove the span child up until the <div class="action"> parent only.
So for example:
<div class="item">
<div class="info">
<div class="action">
<div class="content">
<span class="content-name"> 5% </span>
</div>
</div>
</div>
</div>
From the above HTML, these code should only be removed:
<div class="action">
<div class="content">
<span class="content-name"> 5% </span>
</div>
</div>
So what should left is:
<div class="item">
<div class="info">
</div>
</div>
This is my current python code:
items = WebDriverWait(driver, 10).until(EC.visibility_of_all_elements_located((By.XPATH, "//span[#class='content-name']")))
for item in items:
percentage_text = re.findall("\d+", item.text)[0]
if int(percentage_text) <= 20:
driver.execute_script("arguments[0].remove();", item)
But it only removes the span class and not its parent.
Here is the full HTML, I think it needs javascript to remove elements but I am very new on javascript I researched for more than 2 hours and I still can't find solutions. Thank you very much.
<div class="item">
<div class="info">
<div class="action">
<div class="content">
<span class="content-name"> 5% </span>
</div>
</div>
</div>
</div>
<div class="item">
<div class="info">
<div class="action">
<div class="content">
<span class="content-name"> 95% </span>
</div>
</div>
</div>
</div>
<div class="item">
<div class="info">
<div class="action">
<div class="content">
<span class="content-name"> 32% </span>
</div>
</div>
</div>
</div>
<div class="item">
<div class="info">
<div class="action">
<div class="content">
<span class="content-name"> 15% </span>
</div>
</div>
</div>
</div>
get to the parent of the parent:
driver.execute_script("arguments[0].parentElement.parentElement.remove();", item)
i am looking for a solution to get the second parent (div) of a known element and then get a child element with sub-childrens in selenium with python.
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title></title>
</head>
<body>
<div class="p1">
<div class="p2">...</div>
<div class="p2">
<div class="p3">
<span class="target">b_Kunden</span>
</div>
</div>
<div class="p2">...</div>
<div class="p2">...</div>
<div class="p2">
<div class="p3">
<div class="p4">
<div class="p5">
<div class="p6">
<button type="button" class="b1">button i want to click</button>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="p1">
<div class="p2">...</div>
<div class="p2">
<div class="p3">
<span class="target">different</span>
</div>
</div>
<div class="p2">...</div>
<div class="p2">...</div>
<div class="p2">
<div class="p3">
<div class="p4">
<div class="p5">
<div class="p6">
<button>some button</button>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="p1">
<div class="p2">...</div>
<div class="p2">
<div class="p3">
<span class="target">different</span>
</div>
</div>
<div class="p2">...</div>
<div class="p2">...</div>
<div class="p2">
<div class="p3">
<div class="p4">
<div class="p5">
<div class="p6">
<button>some button</button>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>
I am getting the element I am looking for with xpath and checking for a specific text ("b_Kunden")(working fine):
temp = browser.find_elements_by_xpath("//*[contains(text(), 'b_Kunden')]")[0]
** I cannot access it trough simple className etc. This is just
dummy HTML in environment I need to go up to p1 class from where span text is "b_Kunden" and then down to child element p6 to click the button inside it **
The reason for this is I need to press exactly the button of the section where the span text is b_Kunden. Because the amount of the sections is variable I cannot count and access them trough example: [1] operator for classes. I need to find the term "b_Kunden" and press the relate button in the p1 section to it.
I would be glad if someone could help me out on how to solve this issue.
Best regards,
Liam
This might help you (assuming that the 2nd parent you referred to is going backwards in heirarchy (higher up the DOM):
browser.find_elements_by_xpath("//*[contains(text(), 'b_Kunden')][1]//parent::div//parent::div")
UPDATE (code per the comment and html block provided by #Liam)
//*[text()='b_Kunden']//ancestor::div//button
Here is the HTML with DOM highlighted to the button. I hope this is what you are looking for.
HTML DOM highlighted snapshot
Requirement:
/html/body/div[3]/div[4]/div/div[7]/div/div/div/div/p/b - Contains word "TITLE"
/html/body/div[3]/div[4]/div/div[8]/div/div/div/div/p - Contains "This is my description"
Actual HTML:
<div class="secadvheading section">
<div class="section-custom">
<div class="container-fluid">
<div class="row">
<div class="col-md-12">
<p class="mt-15"><b>TITLE</b></p>
</div>
</div>
</div>
</div>
</div>
<div class="paragraphText parbase section">
<div class="section-custom ">
<div class="container-fluid">
<div class="row">
<div class="col-md-12">
<p>This is my desciption</p>
</div>
</div>
</div>
</div>
Question:
How to get text content paragraph text after "TITLE" div?
Tried
driver.find_element_by_xpath("//*[contains(text(),'TITLE')]/following-sibling::p")
didn't worked. I may have multiple "TITLE in same page" how can i gracefully look for TITILE div (multiple elements) and get the description for the same?
You need to go out of TITLE's node first--go to ancestor node the use following-sibling. Try this:
//b[text()='TITLE']/ancestor::div[#class='secadvheading section']/following-sibling::div[#class='paragraphText parbase section']//p
I have problems to press this type of buttons with Selenium since the name by which I look for "5dbnhpbwuny6rmr65h86" and the button are in different div in Python.
Complete HTML code: https://codeshare.io/a39b3g.
Image HTML.
Example HTML code:
<div class="o_kanban_view o_kanban_dashboard o_pos_kanban o_cannot_create o_kanban_ungrouped" style="display: flex;"><div class="o_kanban_record">
<div class="o_kanban_card_header">
<div class="o_kanban_card_header_title">
<div class="o_primary">5dbnhpbwuny6rmr65h86</div>
<div class="o_secondary">Unused</div>
</div>
<div class="o_kanban_manage_button_section">
<a class="o_kanban_manage_toggle_button" href="#">Más <i class="fa fa-caret-down"></i></a>
</div>
</div>
<div class="container o_kanban_card_content o_visible">
<div class="row">
<div class="col-xs-6 o_kanban_primary_left">
<button class="btn btn-default oe_kanban_action oe_kanban_action_button" data-name="open_session_cb" data-type="object" type="button">New Session
</button>
</div>
<div class="col-xs-6 o_kanban_primary_right">
</div>
</div>
</div><div class="container o_kanban_card_manage_pane o_invisible">
<div class="row">
<div class="col-xs-6 o_kanban_card_manage_section o_kanban_manage_view">
<div class="o_kanban_card_manage_title">
<span>Ver</span>
</div>
<div>
<a data-name="341" data-type="action" href="#" class=" oe_kanban_action oe_kanban_action_a">Sesiones</a>
</div>
<div>
<a data-name="342" data-type="action" href="#" class=" oe_kanban_action oe_kanban_action_a">Pedidos de ventas</a>
</div>
</div>
<div class="col-xs-6 o_kanban_card_manage_section o_kanban_manage_new">
<div class="o_kanban_card_manage_title">
<span>Informes</span>
</div>
<div>
<a data-name="343" data-type="action" href="#" class=" oe_kanban_action oe_kanban_action_a">Pedidos</a>
</div>
</div>
</div>
<div class="o_kanban_card_manage_settings row">
<div class="col-xs-12 text-right">
<a data-type="edit" href="#" class=" oe_kanban_action oe_kanban_action_a">Configuración</a>
</div>
</div>
</div>
</div><div class="o_kanban_record o_kanban_ghost"></div><div class="o_kanban_record o_kanban_ghost"></div><div class="o_kanban_record o_kanban_ghost"></div><div class="o_kanban_record o_kanban_ghost"></div><div class="o_kanban_record o_kanban_ghost"></div><div class="o_kanban_record o_kanban_ghost"></div></div>
I came up with something like that, but I do not have the right solution:
for div in driver.find_elements_by_xpath("//div[#class='o_kanban_record']"):
if div.find_elements_by_xpath("//div[contains(text() , '5dbnhpbwuny6rmr65h86')]") != []:
div.find_elements_by_xpath("//button[contains(text() , 'New Session')]").click()
Thanks!
To click() on the New Session button for any of the Strings e.g.
iuijg6bzr2xs9gsueq2i or 5dbnhpbwuny6rmr65h86, you can take help of a function and pass any String to get the relevant New Session button clicked.
The final solution, which detects the state of the button is:
driver.find_elements_by_xpath("//div[#class='o_primary' and contains(text(), '%s')]/parent::div[*]/parent::div[*]/parent::div[*]/descendant::button[#data-name='open_session_cb']" % (shop))[0].click()
OR
driver.find_elements_by_xpath("//div[#class='o_primary' and contains(text(), '%s')]/parent::div[*]/parent::div[*]/parent::div[*]/descendant::button[#data-name='open_ui']" % (shop))[0].click()
Get all divs and buttons:
divs = driver.find_elements_by_css_selector(".o_primary")
buttons = driver.find_elements_by_css_selector(".btn.btn-default.oe_kanban_action.oe_kanban_action_button")
Go through the list of div elements and find needed one and do click action on the appropriate button:
for div, button in zip(divs, buttons):
if div.text == "5dbnhpbwuny6rmr65h86":
button.click()
Have you heard about Splinter? It is an abstraction layer on top of Selenium and it allows you to find elements by text: https://splinter.readthedocs.io/en/latest/finding.html
driver.find_by_text('5dbnhpbwuny6rmr65h86')
find_by_text returns a list of elements,so it should be
element = driver.find_by_text('5dbnhpbwuny6rmr65h86').first
element.find_by_xpath("//button[contains(text() , 'New Session')]").first.click()
Note: Untested