Selenium cant find class to iterate and scrape text - python

as you can see in the first picture, my objective is to click and open every "Ver detalles" and to get all the text within it (which is shown in the third picture.
Here is the HTML for this first screen:
And here is the screen that opens once you click "ver detalles"
And its HTML
So far this I have made up some lines of code but I know it is useless because I am looking by XPATH and not by Class (and this will only return data for one), but whenever I look by class and try to iterate it doesn't find the class.
Please let me know if I made myself clear. Thanks beforehand
EDIT:
Thanks #cruisepandey it helped me open "ver detalles". Now I'm stucked trying to get the text out of it and click de X to close it and move on to the next "ver detalles".
This is the code I have so far, i have tried looking up by class, tag, etc but can't seem to find a way :(.
def order_data():
list_of_ver_detalles = driver.find_elements(By.XPATH, "//span[contains(text(),'Ver detalle')]/..")
sleep(3)
for ver in list_of_ver_detalles:
ver.click()
print(driver.find_element_by_class_name("jss672").text)
sleep(2)
driver.find_element(By.XPATH, "/html[1]/body[1]/div[6]/div[3]/div[1]/div[1]/div[1]/img[1]").click
Here is the text I am trying to print
And here is the X I am trying to click

I want to clarify few of your doubts when you say it is useless because I am looking by XPATH and not by Class - no it is not. for finding more than one element with any locator (assuming that locator in DOM can represent multiple entity) all you have to do is to use find_elements instead of find_element.
store all of Ver detalle like this :
list_of_ver_links = driver.find_elements(By.XPATH, "//span[contains(text(),'Ver detalle')]/..")
for ver in list_of_ver_links:
ver.click()
#Now write the code to fetch order details here

Related

XPATH syntax for element in {} with Selenium for Python

I am currently trying to click each button on a webpage with Selenium in Python, the class and text is always the same for each button but each button has different ids. The ids, however, are within "data-paramaters" in {} and I can't figure out how to get the correct syntax for the xpath.
Here is a snippet of the website for one of the buttons:
<span class="contains-icon-details gc-btn gc-btn--s" data-isneededpromise="false" data-parameters="{"partner":"gs", "realId": "8da1d6a9-44d1-4556-bc12-92699749a30a", "tnId": "102086182829", "type": "details"}">More Details</span>
It seems the realId and the tnId are unique, so I would need to find the buttons with either one of those.
This works:
driver.find_element_by_xpath("//span[#class='contains-icon-details gc-btn gc-btn--s']").click()
but of course only for the first button as the class is always the same.
I tried something like this:
driver.find_element_by_xpath("//*[contains(#tnId, '102086182829')]").click()
but I get
Unable to locate element: //*[contains(#tnId, '102086182829')]
so definitely not the correct syntax.
I tried to find a solution online, but with no luck so far. Can anybody point me into the right direction? Thanks in advance.
In case realId value or tnId value is unique your XPath can be
driver.find_element_by_xpath("//*[contains(#data-parameters, '8da1d6a9-44d1-4556-bc12-92699749a30a')]").click()
or
driver.find_element_by_xpath("//*[contains(#data-parameters, '102086182829)]").click()
you should filter by the "data-parameters" attribute.
Try
driver.find_element_by_xpath("//span[contains(#data-parameters, '102086182829')]").click()
This is a dirty implementation of what you need. It would be better to extract the data-parameters field, deserialize JSON and check for the needed field;
spans = driver.find_element_by_xpath("//span[#class='contains-icon-details gc-btn gc-btn--s']")
for span in spans:
data_parameters = span.get_attribute("data-parameters")
try:
data_parameters = json.loads(data_parameters)
except:
continue
if 'tnId' in data_parameters and data_parameters['tnId'] == "102086182829":
span.click()
break

How to find an Element by index in selenium Webdriver for python

This is HTML code of that page
From there I want to access the 2nd element by using class name "maxbutton-1" as it has 3 same buttons and I can't use xpath or any constant selector so want to use the indexing with class and can't find anything to do that in python particular.
Also tried the method used in java to do same thing but it didn't worked.
Link of that same page
just trying to automate the movie downloading process for any movie.
Thank you.
To click on first, second or third button, try to change number of element:
el1 = driver.find_element_by_xpath("(//a[#class='maxbutton-1 maxbutton maxbutton-download-links'])[1]")
el2 = driver.find_element_by_xpath("(//a[#class='maxbutton-1 maxbutton maxbutton-download-links'])[2]")
el3 = driver.find_element_by_xpath("(//a[#class='maxbutton-1 maxbutton maxbutton-download-links'])[3]")
then you can extract element href/link attribute like that:
link = el.get_attribute('href')
or click it like that:
el.click()

Selenium Webdriver failing to click. Unsure why

I have the following code;
if united_states_hidden is not None:
print("Country removed successfully")
time.sleep(10)
print("type(united_states_hidden) = ")
print(type(united_states_hidden))
print("united_states_hidden.text = " + united_states_hidden.text)
print("united_states_hidden.id = " + united_states_hidden.id)
print(united_states_hidden.is_displayed())
print(united_states_hidden.is_enabled())
united_states_hidden.click()
The outputs to the console are as follows:
Country removed successfully
type(united_states_hidden) =
<class 'selenium.webdriver.remote.webelement.WebElement'>
united_states_hidden.text = United States
united_states_hidden.id = ccea7858-6a0b-4aa8-afd5-72f75636fa44
True
True
As far as I am aware this should work as it is a clickable web element, however, no click is delivered to the element. Any help would be appreciated as I can't seem to find anything anywhere else. The element I am attempting to click is within a selector box.
Seems like a valid WebElement given you can print all of the info. like you did in your example.
It's possible the element located is not the element that is meant to be clicked, so perhaps the click is succeeding, but not really clicking anything.
You could try using a Javascript click and see if that helps:
driver.execute_script("arguments[0].click();", united_states_hidden)
If this does not work for you, we may need to see the HTML on the page and the locator strategy you are using to find united_states_hidden so that we can proceed.

How to scrapr active data generated by js on a map

I'm new python user and I want to scrape data from this website: https://www.telerad.be/Html5Viewer/index.html?viewer=telerad_fr
My problem is that the data are dynamically generated. I read few possibilities to fix but none is satisfying. With selenium I need a name or Xpath to click on button but here there is nothing.
import requests
from lxml import html
page = requests.get('https://www.telerad.be/Html5Viewer/index.html?viewer=telerad_fr')
tree = html.fromstring(page.content)
cities = tree.xpath('//*[#id="map-container"]/div[6]/div[2]/div/div[2]/div/div/div[1]/div/p[1]/text()[2]')
print('Cities: ', cities)
There actually IS an xpath to click on the buttons:
//*[#id='0_layer']/*[#fill]
Here, try this (selenium):
dotList = driver.find_elements_by_xpath("//*[#id='0_layer']/*[#fill]")
for dot in dotList:
dot.click()
cities = driver.find_element_by_xpath("//div[#data-region-name='NavigationMapRegion']//p[1]")
print("Cities: ", cities.text)
closeBtn = driver.find_element_by_xpath("//*[#class='panel-header-button right close-16']")
closeBtn.click(); #the modal can intercept clicks on some dots, thats why we close it here after extracting the info we need.
this code clicks (or at least tries to, if no StaleElementExceptions occur) all the orange dots on the map, and print the "Cities" content (based on your Xpath).
If anyone finds an error in the code, please edit this answer, i wrote this on notepad++.

Selenium Find Sub-Child href Element

I'm trying to click the following link using selenium.
<div id="RECORD_2" class="search-results-item">
<a hasautosubmit="true" oncontextmenu="javascript:return IsAllowedRightClick(this);" class="smallV110" href="#;cacheurlFromRightClick=no"></a>
</div>
Which record to click is not known before the code is executed. Record_2 has multiple children, and the one included is the one I want to click. The link is edited for the sake of privacy. I tried to do something like that where name is the record variable, however it doesn't work.
driver.find_element_by_css_selector("css=div#"RECORD_%s" % (name).smallV110")
I'm a complete newbie to selenium so I couldn't figure out a way to sort this out. I would appreciate any help. Thanks!
Note that this is not Selenium IDE and you don't need the css= at the beginning of a selector.
There are multiple ways to locate the link element, e.g.:
driver.find_element_by_css_selector(".search-results-item a.smallV110")
driver.find_element_by_css_selector("[id^=RECORD] a.smallV110") # id starts with "RECORD"
If you know the id value beforehand:
id_i_know = 2
driver.find_element_by_css_selector("[id=RECORD_%d] a.smallV110" % id_i_know)
You don't have to have that smallV110 class attribute check - I've added it to increase chances of not matching other a elements inside the div (not sure what they are, you have not posted the entire HTML).

Categories