I want to retrieve with Selenium a full url, using part of it.
<div class="productcarousel__mainitem slick-slide slick-current slick-active" data-slick-index="0" aria-hidden="false" style="width: 280px; position: relative; left: 0px; top: 0px; z-index: 999; opacity: 1;" tabindex="-1" role="option" aria-describedby="slick-slide00">
<span class="productcarousel__fullscreen"></span>
<img src="https://cloudxyd.com/image/.../gtfrgrfrtg.jpg">
</div>
I tried this command without any success:
links = driver.find_elements_by_xpath("//*[contains(text(), 'url(//cloudxyd.com/image/')]")
After getting the xpath, you'll have to use get_attribute('src') on links to get the url
links = driver.find_elements_by_xpath("//*[contains(text(), 'url(//cloudxyd.com/image/')]")
url=links.get_attribute('src')
You may also use css selector.
link = driver.find_element_by_css_selector(".productcarousel__mainitem.slick-slide.slick-current.slick-active img").get_attribute("src")
use the below xpath :
//div[contains(#class, 'productcarousel__mainitem')]/img
and get the URL like this :
img_url = driver.find_element(By.XPATH, "//div[contains(#class, 'productcarousel__mainitem')]/img
").get_attribute('src')
print(img_url)
or if the above mentioned xpath highlights more than one entry in DOM, you could probably do something like this :
for img_url in driver.find_elements(By.XPATH, "//div[contains(#class, 'productcarousel__mainitem')]/img")
print(img_url.get_attribute('src'))
Related
I'm currently a student trying to learn how to make a bot. I've been successful by myself traversing through a website until I need to send keys to an iframe. I tried the following to get a NoSuchElementException.
#imports from the beginning
from selenium import webdriver as wd
from selenium.common.exceptions import NoSuchElementException
import chromedriver_binary
iframe = wd.find_element_by_xpath('//*[#id="card-number-container"]/iframe')
wd.switch_to.frame(iframe)
num = wd.find_element_by_xpath('//*[#id="card-number-container"]')
type(num)
num.send_keys("867-5309")
I tried switching to the iframe to interact with it. The error starts when defining num, but I have no idea what else to put to get there. Here's some of the HTML from the website.
<div id="card-number-container" class="_3lT_PetJ2w_rU8njR_SoIK _1ysVkT9CIcv98OnRX5h1Rj flex-microform">
<iframe hspace="0" vspace="0" frameborder="0" scrolling="no" allowtransparency="true" marginwidth="0" marginheight="0" src="https://flex.cybersource.com/cybersource/assets/microform/0.11.4/iframe.html?keyId=03eSdEAPS81iBTM0NCSfE2l3z43E8yw1#{"microformId":"cca792b9-2c83-422c-875b-bdb03b080eee","fieldId":"914ffbab-2597-40e3-9cd0-b5098cea829f","jwt":"eyJraWQiOiJ3ZiIsImFsZyI6IlJTMjU2In0.eyJmbHgiOnsicGF0aCI6Ii9mbGV4L3YyL3Rva2VucyIsImRhdGEiOiI5SGorT1h6Z3lWc2xIc0FxVjE3bEJSQUFFQjJZU1VaQzJjY0dLK0xFWndOVEtMRXhkTGVLS3NSZFRVQjZEODdGWEFTcHM2a3RPTWIwbUNiWTBDZUlOYVE5eTVJSEd4TVgzb2Ziby9keGZCZGVvaFhpRHl1V2tlQXRpZlFneXpXR21KaDUiLCJvcmlnaW4iOiJodHRwczovL2ZsZXguY3liZXJzb3VyY2UuY29tIiwiandrIjp7Imt0eSI6IlJTQSIsImUiOiJBUUFCIiwidXNlIjoiZW5jIiwibiI6ImlHbEJXdmlVVmlzTHpvZ0d1S0VNSDdjNVNtQVQxenYybXh0bExNU1k2a05XaUNQWW1RYW5BS2xRemp5MjN1NzlROG5SZk56aW50a3JWR1hhYk8ycndOQTRaYTBTdXp0ODVhU2steDdNSU9PSDVrQkRpeEJGWmc0dXQ2RDhZRHhHUW5uR25BZEZGZHF5Umx4VnhiZU53WUFxOHpxZFJ1UG9VckY0ZDYxZGFwejJ6ZFU1Z2QyZWFLZXEwQTdCTE9CdjhTbThLU1R6bDY3VndKNmJvX0VLM3FsUldBd3NiLXRSMFIzV0xZTFBDb2lVcThMeUNkVWpjSG84ckQtaGw1VzZ3ekkxR0x3S21RRVN3d0ZLdmZDSmwzTWs1TFQ0NU1WUmtHX1ViNlFBMkZwaDNKOXhGNkY2clNVM2Y0WnVFNlpEQnlIeXZGekVpdlJYMURjWVFxblpuUSIsImtpZCI6IjAzZVNkRUFQUzgxaUJUTTBOQ1NmRTJsM3o0M0U4eXcxIn19LCJjdHgiOlt7ImRhdGEiOnsidGFyZ2V0T3JpZ2lucyI6WyJodHRwczovL3d3dy5wb2tlbW9uY2VudGVyLmNvbSIsImh0dHBzOi8vdGVzdC5wb2tlbW9uY2VudGVyLmNvbSIsImh0dHA6Ly9sb2NhbGhvc3Q6MzAwMCJdLCJtZk9yaWdpbiI6Imh0dHBzOi8vZmxleC5jeWJlcnNvdXJjZS5jb20ifSwidHlwZSI6Im1mLTAuMTEuMCJ9XSwiaXNzIjoiRmxleCBBUEkiLCJleHAiOjE2MzQyNDg5OTIsImlhdCI6MTYzNDI0ODA5MiwianRpIjoiYWF6VWc3QThadXJXZG8yZyJ9.BsUePIvHdd5gkjdKMQ2iaBxyiakGWK9fe0bZNdaCheKfThTqG2HvsipskZ6ec42mira64uEFoCR43qGT7NIoGnGy-CmZ5pYWeuM8yzULD6pSPYhN42afQN08Mht9KjeKSL7W-NlTqT_a3KPadYsm2PIAACDj3hRhVWf0A0Avkd88EN0Yuch47KiVBVJb4QtVONcIVum0rIvr5msnj2zaJyC6WvHhiUu8UutOHqeoqHT3pxdcQNoW6xai6yQjK38yUAVcfKhZpcSnlC-be5FYK38eYSjlAp0XgFSAi6_nllsAs8oj9OoB-A57HpBRyNS0QFVLSt3bmoQky2HMr8gGGg","microformConfig":{"styles":{"input":{"font-size":"12px"},":disabled":{"cursor":"not-allowed"},"valid":{"color":"#43a047"},"invalid":{"color":"#d32f2f"}}},"config":{"placeholder":"Card Number..."},"fieldType":"number"}" style="overflow: hidden; position: relative; border: none; width: 100%; height: 100%;" cd_frame_id_="e0aaadb108c0aacce0bb0c878467c1c7"></iframe>
</div>
I'm trying to display text in what seems to be a container in web using selenium webdriver for python.
Here is the inspect element -
<div style="overflow-x: hidden;">
<div class="view-container" style="flex-direction: row;
transition: all 0s ease 0s; transform: translate(0%, 0px); direction: ltr;
display: flex; will-change: transform;">
<div aria-hidden="false" data-swipeable="true" style="width: 100%; flex-
shrink: 0; overflow: auto;">
<div>
<div class="qs-text">What is the answer to this question?</div>
I want it to display "What is the answer to this question"
I'm trying to use the below, but it doesn't return anything. -
element = driver.find_element_by_class_name("qs-text").text
print(element)
I tried find_element_by_css_selector("qs-text").text but that didn't help either.
Can you please tell me what I'm doing wrong
How about this:
element = driver.find_element_by_xpath("//*[contains(#class, 'qs-text') and contains(text(), 'answer')]")
print(element)
I am trying to switch to frame but Selenium can't locate the element.
Here is HTML code which I get with Selenium before locating:
<div class="jqmPopOverlay" style="height: 100%; width: 100%; position: fixed; left: 0px; top: 0px; z-index: 2999; opacity: 0.7;"></div><div class="login login-main jqmID1" id="loginForm" style="z-index: 3000; top: 0px; left: 319.5px; display: block;">
<div class="titlebar">
<a class="close" href="#" tabindex="7">✕</a>
</div>
<div class="tooltipstered" id="loginFrame"><iframe frameborder="0" id="easyXDM_default127_provider" name="easyXDM_default127_provider" scrolling="no" style="height: 509.667px;" width="100%"></iframe></div>
</div>
<script async="" defer="" type="text/javascript">undefined</script><script async="" src="//www.googleadservices.com/pagead/conversion_async.js" type="text/javascript"></script><script async="" src="https://www.google-analytics.com/plugins/ua/ec.js" type="text/javascript"></script><script async="" src="https://www.google-analytics.com/analytics.js" type="text/javascript"></script><script async="" src="//www.googletagmanager.com/gtm.js?id=GTM-4DBM"></script><script type="text/javascript">
Here is Python code:
browser = webdriver.Chrome('chromedriver.exe')
browser.get(url)
frame = browser.find_element_by_id("easyXDM_default6255_provider")
browser.switch_to_frame(frame)
Here is the error:
NoSuchElementException: Message: no such element: Unable to locate element: {"method":"id","selector":"easyXDM_default6255_provider"}
(Session info: chrome=67.0.3396.99)
(Driver info: chromedriver=2.40.565498 (ea082db3280dd6843ebfb08a625e3eb905c4f5ab),platform=Windows NT 10.0.17134 x86_64)
As you can see there is the element in HTML, but it can't be located.
What am I doing wrong?
Try this:
browser = webdriver.Chrome('chromedriver.exe')
browser.get(url)
WebDriverWait(browser, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, "//*[#id = 'loginFrame']/iframe")))
Note: you have to add some imports:
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.wait import WebDriverWait
Firstly, your frame probably has generic id and name. That's why I have proposed another selector. Secondly I have added WebDriverWait, which waits at least 10 seconds until frame will be available and only then switches to it.
PS: when you are done with content inside iframe, don't forget to switch to default content like this:
browser.switch_to.default_content()
otherwise you will be not able to interact with the whole DOM.
I have a web element with a tooltip that shows the following message:
● Client Book Revenue $20,966,618
The HTML code for that tooltip is below. I am able to hover over the web element using Selenium Webdriver which makes the tooltip visible, but I can't figure out how to get the text from it. Could somebody please help?
<div class="highcharts-tooltip" style="position: absolute; left: 755px; top: 0px; display: block; opacity: 1; pointer-events: none; visibility: visible;">
<span style="position: absolute; font-family: "Roboto",sans-serif; font-size: 12px; white-space: nowrap; color: rgb(51, 51, 51); margin-left: 0px; margin-top: 0px; left: 0px; top: 0px;">
<div class="client-rate-bench-chart">
<table class="table rdo-table-tooltip">
<tbody>
<tr>
<td>
<span style="color:rgba(45,108,162,1)">●</span>
Client Book Revenue
</td>
<td> $20,966,618 </td>
</tr>
</tbody>
</table>
</div>
</span>
</div>
You can grab the table and then grab the first instance of <tr>
from bs4 import BeautifulSoup
from selenium import webdriver
driver = webdriver.Firefox()
driver.get(URL)
html = driver.page_source # this is how you get the HTML
soup = BeautifulSoup(html)
table = soup.find('table', class_='rdo-table-tooltip')
tooltip = table.find('tr')
text = tooltip.text
text will have a lot of extra whitespace because of how the HTML is formatted, but you can strip that out - just split on all whitespace and then re-join the elements like this
final_text = ' '.join(text.split())
print final_text
# ● Client Book Revenue $20,966,618
For multiple <tr>s you can use .find_all('tr') and then use a list comprehension to get a list of the contents of the rows. It would look something like this
soup = BeautifulSoup(html)
table = soup.find('table', class_='rdo-table-tooltip')
tooltips = table.find_all('tr')
text = [' '.join(tooltip.text.split()) for tooltip in tooltips]
Then text will be a list of strings containing the text from each <tr>
As an alternative you could use re.findall to return all the instances of text between tags. This will involve some cleaning up afterwards but I have found it pretty handy in general when working with Selenium.
import re
tooltips = re.findall('<tr>(.*?)<tr>', html.replace('\n', ''))
for tooltip in tooltips:
print tooltip
<div class="j-C j-C-yj" style="max-height: none; -moz-user-select: none; visibility: visible; left: 218px; top: 105px; display: none;" role="menu" aria-haspopup="true">
<div id=":1r" class="j-qn" style="-moz-user-select: none;" role="separator"></div>
<div id=":1u" class="j-qn" style="-moz-user-select: none;" role="separator"></div>
</div>
so far i am creating xpath to select element with id=":1r" is
"(//div[contains(#class,'j-C') and contains(#class ,'j-C-yj')]/div)[1])"
i had also tried
"(//div[contains(#class,'j-C') and contains(#class ,'j-C-yj')]/div)[1])"
but none is working please help !
P.S : I cannot find element with id as id of the page is created dynamically
Just use xpath('.//div[contains(#class, "j-C") and contains(#class, "j-C-yj")]') as another answer already showed you.
Another Updates:
As OP kept changing the constraints of the question, here is the full solution does exactly what OP eants.
Sample: I'm using lxml to parse your string and do the xpath
from lxml import etree
s = '''<div class="j-C j-C-yj" style="max-height: none; -moz-user-select: none; visibility: visible; left: 218px; top: 105px; display: none;" role="menu" aria-haspopup="true">
...: <div id=":1r" class="j-qn" style="-moz-user-select: none;" role="separator"></div>
...: <div id=":1u" class="j-qn" style="-moz-user-select: none;" role="separator"></div>
...: </div>'''
# I need to wrap your string with <root> element otherwise first div will become the root
tree = etree.fromstring('<root>'+s+'</root>')
# xpath always returns a list, so just loop through the list and the first element is what you want
for node in tree.xpath('.//div[contains(#class, "j-C") and contains(#class, "j-C-yj")]'):
print etree.tostring(node[0])
<div id=":1r" class="j-qn" style="-moz-user-select: none;" role="separator"/>
Seems like there is a problem in your xpath towards the end. Try this piece of code:
driver.find_element_by_xpath("//div[contains(#class,'j-C') and contains(#class ,'j-C-yj')]/div[1]")