I am having trouble accessing elements:
<fieldset>
<legend>
Legend1
</legend>
<table width=100%" cellspacing="3" bgcolor="white">
<tbody>
<tr>...</tr>
<tr>...</tr>
</tbody>
</table>
<fieldset>
<legend>
Legend2
</legend>
<table width="100%" cellspacing="3" bgcolor="white" align="center">
<tbody>
<tr>
<td></td>
<td class="reportLabel" nowrap="">Label1</td>
<td class="reportField>Field1</td>
<td></td>
</tr>
</tbody>
</table>
<fieldset>
...
I can access everything in the first table (before entering a sub-fieldset). However, I can't access anything from the fieldset on. The error I get is:
Message: Unable to find element with xpath == ...
Is there something special I have to do when there are new fieldsets? Similar to having to switch frames?
The command I'm using is:
ret = self.driver.find_element_by_xpath("//fieldset/legend[text()='Legend2']/following::table/tbody/tr/td[#class='reportlabel'][text()='Label1']")
The reason I'm including the legend and following it with 'following' is that there are a lot of different sections and legends within a previous one, and I'd like to ensure that the field is indeed in the proper section.
I have also tried simpler things, though, such as:
ret = self.driver.find_element_by_xpath("//fieldset/table/tbody/tr/td[#class='reportLabel][text()='Label1']")
I am using:
IE11 (same issue on Firefox, though)
Selenium 2.44.0
Python 2.7
Windows 7
32 bit IEDriverServer.exe
Does anyone know why I can't access these elements?
Your second XPATH looks correct unless the fact that you are missing a ' after reportLabel. Corrected:
//fieldset/table/tbody/tr/td[#class='reportLabel'][text()='Label1']
Working xpath as per OP's comment
//legend[contains(.,'Legend2')]//..//td[contains(text(),'Label1')]
Related
I need to select the radio button below with value="QMBT-0029104.xlsx;SYPFJPC2MQHC-5-688478#QHBTW"
HTML:
<form name="ViewQueryForm" method="post" action="/equery/getAttachments.do">
<div class="txt_align_left innerdvcont" id="tabmenu1" style="display:">
<div class="clear"></div>
<div class="txt_align_left innerdvcont" id="tabmenu011" name="tabmenu011">
<table width="100%" border="0" cellspacing="0" cellpadding="0">
<thead>
<tr>
<th width="10%" style="text-align: left"></th>
<th width="60%" style="text-align: left">Attachment </th>
<th width="30%" style="text-align: left">Date </th>
</tr>
</thead>
<tbody>
<tr>
<td align="center" valign="top">
<input type="radio" name="getAttachmentValue" id="getAttachmentValue" value="Dispute_1466718.xlsx;SYPFJPC2MQHC-5-687433#QHBTW"> </td>
<td style="padding:5px 4px">Dispute_1466718.xlsx</td>
<td style="padding:5px 4px">2021-02-16T10:34:08.617</td>
</tr>
</tbody><tbody>
<tr>
<td align="center" valign="top">
<input type="radio" name="getAttachmentValue" id="getAttachmentValue" value="QMBT-0029104.xlsx;SYPFJPC2MQHC-5-688478#QHBTW"> </td>
<td style="padding:5px 4px">QMBT-0029104.xlsx</td>
<td style="padding:5px 4px">2021-03-27T08:08:46.09</td>
</tr>
</tbody>
</table>
</div>
So far I have been able to click on it using the code below:
radiobutton2 = driver.find_element_by_xpath("//input[#value='QMBT-0029104.xlsx;SYPFJPC2MQHC-5-688478#QHBTW']");
radiobutton2.click()
However, the value changes every time which means that it's not something that I can use when running the code. Is there any way to select the second radio button by default for example.
Alternatively, I will know the QMBT-00000 reference, so is there a way to select the radio button by searching for that text?
I have tried:
radiobutton2 = driver.find_element_by_xpath('//*[contains(text(), "QMBT-0029104") and #id="getAttachmentValue"]');
radiobutton2.click()
However, that gives me an error:
Unable to locate element
Great first question.
If the radio button you need to select will always be the second option, you can select it by the index (below is in C#, but should be similar for Python):
// get all elements where id = "getAttachmentValue"
var radioButtons = driver.FindElements(By.Id("getAttachmentValue"));
// click second element
radioButtons[1].Click();
Edit (Python):
from selenium.webdriver.common.by import By
radioButtons = driver.find_elements(By.ID, 'getAttachmentValue')
radioButtons[1].click()
I added your HTML to a simple HTML file and was able to select the second radio button using the above example (using C#):
You can select the second radio button by xpath using array index notation. Instead of targeting the value attribute, use the name attribute instead. The value of the name attribute seems to be consistent:
xpath = "(//input[#type = 'radio' and #name = 'getAttachmentValue'])[2]"
radio_button = driver.find_element_by_xpath(xpath)
radio_button.click()
You probably should not use the id attribute in your locator. The Id attribute value must be unique for the entire web page, but the Id attribute value is repeated for each radio button. This is invalid HTML. It is best to avoid using attributes in locators where the values of those attributes violate the HTML spec. You never fully know how the browser is going to treat it.
For your use case, the name attribute would work best. It is perfectly valid HTML to have repeated name attribute values, and the value of the name attribute appears to be stable for each page view.
If you have only two same ids, the answer is simple. Try this.
driver.find_element_by_css_selector("#getAttachmentValue:nth-of-type(2)")
If there are more - the solution may be more complicated.
How may IDs are there?
Hi we are running this code and want to loop over rows. I've got something completely wrong how Pythn handles the xpath selectors. It works in my Chrome xpath browsers, just not in python.
we capture a data table in table this works
then we grab all underlying rows in TR
My question is: how can I grab the tbody/tr's and color properly and most logically? I have tried // and ./ and / ...
For color_rows = table.xpath('/tbody/tr') I would expect to be able to use /tbody/tr directly because the data is directly under the table. Somehow I have to use // to get it to work, why?
For color = color_row.xpath('/td[1]/b/text()').get().strip() I would expect to be able to use /td[1]/b/text() directly because the data is directly under the tr. Somehow I have to use // to get it to work, why?
table = response.xpath('//div[#class="content"]//table[contains(#class,"table")]')
color_rows = table.xpath('/tbody/tr')
for color_row in color_rows:
color = color_row.xpath('/td[1]/b/text()').get().strip()
Our html data looks like this
<table class="table">
<thead>
<tr>
<th id="ctl00_cphCEShop_colColore" class="text-left" colspan="2">Colore</th>
<th>S</th>
<th>M</th>
<th>L</th>
</tr>
</thead>
<tbody>
<tr>
<td id="x">
<b>White</b>
<input type="hidden" name="data" value="3230/201">
</td>
<td id="avail">
Avail:
</td>
<td id="1">
<div>
<input name="cell" type="text" class="form-control">
<div class="text-center">179</div>
</div>
</td>
<td id="2">
<div>
<input name="cell" type="text" class="form-control">
<div class="text-center">360</div>
</div>
</td>
etc etc
When you want to locate node somewhere in DOM you need to use // (exception is root node - you can use one slash only '/html'):
response.xpath('//tbody/tr')
When you want to start search from the node you've already found you should use either ./ for child or .// for descendant:
table.xpath('./tbody/tr')
Note the dot in the beginning of XPath expression that points on context node
P.S. I also would recommend not to use tbody in your XPath as it might not be present in page source code but be added by Browser while rendering DOM !! Always inspect the real HTML and the dom there.
I used Python with flask web framework.
I have a web page with 3 rows (or it can be less or more) with server names which can be updated by users. You can see page layout in the attachment.
page layout
I am using a for loop inside html code, to get lines to fill according to hostnames and host number. The code is:
...
<table class="table table-hover">
<tbody>
{% for h in todos_host %}
<tr class="thead-dark">
<th scope="row" width="30%">Check link: {{ h.hostname }}</th>
<td width="70%"><input class="form-control" name="checklink" type="text" placeholder="ex. http://xxxx"></td>
</tr>
{% endfor %}
</tbody>
</table>
...
End user will put different data in fields marked with "ex. http://xxxx" placeholder.
In my main script i am trying to get the value of rows with request.form.get('checklink') but this one gets only first value (for gerenbach server). How can I use this method with loop in order to get each row's value ?
I am using selenium in python. I have come across this table webelement. I need to check if a string is present in the webelement and return a corresponding string in case its present.
<table width="700px" class="tableListGrid">
<thead>
<tr class="tableInfoTrBox">
<th>Date</th>
<th>Task Code</th>
<!-- th>Phone Number</th -->
<th>Fota Job</th>
<th colspan="2" class="thLineEnd">Task Description</th>
</tr>
</thead>
<tbody>
<tr class="tableTr_r">
<td>2018-04-06 05:48:29</td>
<td>FU</td>
<!-- td></td -->
<td>
57220180406-JSA69596727
</td>
<td style="text-align:left;">
updated from [A730FXXU1ARAB/A730FOJM1ARAB/A730FXXU1ARAB] to [A730FXXU2ARC9/A730FOJM2ARC1/A730FXXU2ARC9]
</td>
<td>
<table class="btnTypeE">
<tr>
<td>
View
</td>
</tr>
</table>
</td>
</tr>
</tbody>
</table>
I need to search for "A730FXXU2ARC9/A730FOJM2ARC1/A730FXXU2ARC9" in this element and return "57220180406-JSA69596727" which is present in same row at a different place in the web page. Is it possible to do in selenium ?
EDIT: Cleaned the code to only contain useful data.
It can be achieved by finding the element using the following Xpath:
//td[contains(., 'A730FXXU2ARC9/A730FOJM2ARC1/A730FXXU2ARC9')]/preceding-sibling::td[1]/a
Xpath can be read as
find td which contains "A730FXXU2ARC9/A730FOJM2ARC1/A730FXXU2ARC9". Then find the td
preceding the found td and move to the a tag
After this you can get text using selenium
driver.find_element(By.XPATH, '//td[contains(., 'A730FXXU2ARC9/A730FOJM2ARC1/A730FXXU2ARC9')]/preceding-sibling::td[1]/a').text
To look out for a text e.g.A730FXXU2ARC9/A730FOJM2ARC1/A730FXXU2ARC9 and find out an associated text e.g. 57220180406-JSA69596727, you can write a function as follows :
def test_me(myString):
myText = driver.find_element_by_xpath("//table[#class='tableListGrid']//tbody/tr[#class='tableTr_r']//td[.='" + myString + "']//preceding::td[1]/a").get_attribute("innerHTML")
Now, from your main()/#Test you can call the function with the desired text as follows :
test_me("A730FXXU2ARC9/A730FOJM2ARC1/A730FXXU2ARC9")
i try get href from this table
<div class="squad-container">
<table class="table squad sortable" id="page_team_1_block_team_squad_8-table">
<thead>
<tr class="group-head">
<th colspan="4">Goalkeepers </th>
</tr>
</thead>
<tbody>
<tr>
<td style="width:50px;">Reda Sayed</td>
<td style="vertical-align: top;">
<div><a href="/474798/" >Reda Sayed</a></div>
<div style="padding-left: 27px;">25 years old</div>
</td>
</tr>
</tbody>
i use
response.xpath('//table[#class="table squad sortable"]//tr//td//a/#href').extract_first()
and didnt work with i need know what is the problem in code and what is different if i use double // or single slash
I don't think there is any problem with your xpath from we human's perspective. However, the xpath or css can be different from your spider's perspective, i.e. your spider may 'see' page differently.
Try using 'scrapy shell' to test your xpath or css and see if any data can be extracted. Here is the link to the doc in case you need: https://doc.scrapy.org/en/latest/topics/shell.html
To sum up: modify the xpath you wrote, 'cause your spider won't find any data with that xpath, and scrapy shell can help you.:)