Im trying to expand the arrow as below from https://www.xpi.com.br/investimentos/fundos-de-investimento/lista/#/
I'm usingo the code below:
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait as wait
from selenium.webdriver.support import expected_conditions as EC
from time import sleep
url = 'https://www.xpi.com.br/investimentos/fundos-de-investimento/lista/#/'
driver = webdriver.Chrome(options=options)
driver.get(url)
sleep(1)
expandir = driver.find_elements_by_class_name("sly-row")[-4]
expandir.click()
sleep(4)
expandir_fundo = wait(driver, 5).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div[class='arrow-details']")))
expandir_fundo.click()
And i'm getting the error: TimeoutException: Message
I tried to use the code below too:
expandir_fundo = driver.find_element_by_xpath('//*[#id="investment-funds"]/div/div/div[2]/article/article/section[2]/div[1]/div[10]')
expandir_fundo.click()
and got the error: ElementClickInterceptedException: Message: element click intercepted: Element ... is not clickable at point (1479, 8).
find below part of HTML:
<div class="funds-table-row sc-jAaTju kdqiDh sc-ckVGcZ drRPta">
<div class="fund-name sc-jKJlTe hZlCDP" title="Bahia AM Maraú Advisory FIC de FIM" style="cursor:pointer;"><div>Bahia AM Maraú Advisory FIC de F...</div><p class="sc-brqgnPfbcFSC">Multimercado</p></div>
<div class="morningstar sc-jKJlTe hZlCDP">-</div>
<div class="minimal-initial-investment sc-jKJlTe hZlCDP">20.000</div>
<div class="administration-rate sc-jKJlTe hZlCDP">1,90</div>
<div class="redemption-quotation sc-jKJlTe hZlCDP"><div>D+30<p class="sc-brqgnP fbcFSC" style="font-size: 12px; color: rgb(24, 25, 26);">(Dias Corridos)</p></div></div>
<div class="redemption-settlement sc-jKJlTe hZlCDP"><div>D+1<p class="sc-brqgnP fbcFSC" style="font-size: 12px; color: rgb(24, 25, 26);">(Dias Úteis)</p></div></div>
<div class="risk sc-jKJlTe hZlCDP"><span class="badge-suitability color-neutral-dark-pure sc-jWBwVP hvQuvX" style="background-color: rgb(215, 123, 10);">8<span><strong>Perfil Médio</strong><br>A nova pontuação de risco leva em consideração critérios de risco, mercado e liquidez. Para saber mais, clique aqui.</span></span></div>
<div class="profitability sc-jKJlTe hZlCDP"><div class="sc-kEYyzF lnwNVR"></div><div class="sc-kkGfuU jBBLoV"><div class="sc-jKJlTe hZlCDP">0,92</div><div class="sc-jKJlTe hZlCDP">0,48</div><div class="sc-jKJlTe hZlCDP">5,03</div></div></div><div class="invest-button sc-jKJlTe hZlCDP"><button class="xp__button xp__button--small" data-wa="fundos-de-investimento; listagem - investir; investir Bahia AM Maraú Advisory FIC de FIM">Investir</button></div>
<div class="arrow-details sc-jKJlTe hZlCDP"><i type="arrow" data-wa="" class="eab2eu-0 eUjuAo"></i></div></div>
The HTML "arrow" is:
<div class="arrow-details sc-jKJlTe hZlCDP">
<i type="arrow" data-wa="" class="eab2eu-0 eUjuAo">
::before
</i>
</div>
You can check the below lines of code
option = Options()
#Disable the notification popUp
option.add_argument("--disable-notifications")
driver = webdriver.Chrome(r"ChromeDriverPath",chrome_options=option)
driver.get("https://www.xpi.com.br/investimentos/fundos-de-investimento/lista/#/")
#Clicked on the cookie, which is under the shadow-DOM So used execute_script(), Also used sleep() before clicking on cookies because at some point it throws the JS Error Shadow-DOM null.
sleep(5)
cookie_btn = driver.execute_script('return document.querySelector("#cookies-policy-container").shadowRoot.querySelector("soma-context > cookies-policy-disclaimer > div > soma-card > div > div:nth-child(2) > soma-button").shadowRoot.querySelector("button")')
cookie_btn.click()
#There can be a multiple way to scroll, Below is one of them
driver.execute_script("window.scrollBy(0,700)")
#There are multiple rows which have expand button so used the index of the XPath if you want to click on multiple try to use loop
expandir_fundo = WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.XPATH, "((//*[#class='eab2eu-0 eUjuAo'])[1])")))
expandir_fundo.click()
import
from time import sleep
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.chrome.options import Options
While #YaDav MaNish answer seems to be working, I would rather use customized query Selector to click on accept cookies button, not generated by browser.
cookie_btn = driver.execute_script('return document.querySelector("#cookies-policy-container").shadowRoot.querySelector("soma-context soma-card soma-button").shadowRoot.querySelector("button")')
cookie_btn.click()
should work.
Your replies and the cookies code, gave me an ideia to usa the execute_script, to solve my problem.
Find below My code
This first page I only open the URL, remove the cookies and expand all information.
# Setting options
options = Options()
options.add_argument('--window-size=1900,1000')
options.add_argument("--disable-notifications")
# Opening Website
url = 'https://www.xpi.com.br/investimentos/fundos-de-investimento/lista/#/'
driver = webdriver.Chrome(r"chromedriver", options=options)
driver.get(url)
# Removing Cookies
cookie_btn = driver.execute_script('return document.querySelector("#cookies-policy-container").'\
'shadowRoot.querySelector("soma-context > cookies-policy-disclaimer > div > soma-card > div > '\
'div:nth-child(2) > soma-button").shadowRoot.querySelector("button")')
cookie_btn.click()
# Expanding the whole list
expandir = driver.find_elements_by_class_name("sly-row")[-4]
expandir.click()
sleep(4)
# Creating content
page_content = driver.page_source
site = BeautifulSoup(page_content, 'html.parser')
This second part I used driver.execute_sript to open the arrow as I went getting information.
fundos = site.find_all('div',class_='funds-table-row')
for cont, fundo in enumerate(fundos):
nome = fundo.find('div', class_='fund-name')['title']
driver.execute_script(f"return document.getElementsByClassName('arrow-details'){[cont + 1]}.click()")
page_detalhe = driver.page_source
site2 = BeautifulSoup(page_detalhe, 'html.parser')
details= site2.find('section', class_='has-documents').find_all('div')
tax_id = details[2].get_text()
Thanks for everyone for help.
Related
Hi I am working on the script to automate downloads of videos from this side https://pixabay.com/videos/
I can find a class with href(href is an attribute with URL) but after that Selenium gives me a bug with any error only a result of print(xy.get_atribute("href)) is None:
my code:
from selenium import webdriver
from selenium.webdriver.firefox.service import Service
from webdriver_manager.firefox import GeckoDriverManager
from selenium.webdriver.common.by import By
from mutagen.mp3 import MP3
import requests
import time
tag = "city "
while True:
s = Service(GeckoDriverManager().install())
driver = webdriver.Firefox(service=s)
driver.minimize_window()
tester = tag.split()
print(tester)
print(len(tester))
if len(tester) == 2:
tag = tester[0] + "%20" + tester[1]
print(tag)
print("2 " + tag)
driver.get("https://pixabay.com/cs/videos/search/" + tag )
images = driver.find_elements(By.CLASS_NAME, 'item' )
print(images)
n = 0
lenght = 0
for image in images:
image = image.get_attribute("href")
print(image)
break
HTML on the side
<html lang="cs" prefix="og: http://ogp.me/ns#">
<head>.</head> <body class="" data-new-gr-C-5-check-loaded="14.1050." data-gr-ext-installed>
<noscript>_</noscript> <div id="wrapper"> > <div id="header">...</div> <div id="content" class="clearfix">
::before <div id="search-term" style="display:none">city</div> <div class="media_list">
<div style="border-bottom:1px solid #f0f1f4"> </div> <div style="background:#e8eaec" class="external-media">.</div> <div style="background: #f6f5fa"> <div style="max-width: 1824px;padding: 10px 3px 20px;margin: auto">
<h1 style="font-size: 13px;color:#bbb;margin:0 19px; position:relative;top:2px">70 videa zdarma z city</h1> <div class="related-keywords">...</div> <div class="row-masonry video video-search-results"> flex <div class="row-masonry-cell" style="flex-basis: 355.55555555555554px; flex-grow: 1.7777777777777777; flex-shrink: 1.7777777777777777; max-width: 622.2222222222222px"> <div class="row-masonry-cell-outer" style="padding-top: 56.25%"> <div class="row-masonry-cell-inner"> <div itemscope itemtype="schema.org/videoobject" class="item" data-w="1920" data-h="1980">
<meta itemprop="license" content="https://creativecommons.org/licenses/publicdomain/"> <meta itemprop="contentUrl" content="//player.vimeo.com/external/142621375.mobile.mp4?s=e9a3c9616798b6f3de74d579ea8314acc75fad72&profile_id=116"> <meta itemprop="thumbnailUrl" content="https://i.vimeocdn.com/video/539965294-5d28c2680682aa5173e86fa74acb94671783ba7e2dc2892682e897c8a158af75-d_640x360.jpg"> <meta itemprop="name" content="New York City, Manhattan, Lidé"> <meta itemprop="description" content="New York City, Manhattan, Lidé, Auta, Rozcestí, Amerika"> <meta itemprop="duration" content="TM145"> <meta itemprop="uploadDate" content="2022-02-16"> <a href="/cs/videos/new-york-city-manhattan-lidC3%A9-auta-1944/"> == $ <div class="media" data-mp4="//player.vimeo.com/external/142621375.mobile.mp4?s=e9a3c9616798b6f3de74d579ea8314acc75fad72&profile_id=116">
<img class="video-preview" src="https://i.vimeocdn.com/video/539965294-5d28c26...d 640x360.jpg" alt="New York City, Manhattan, Lidé, Auta">
<i></i> </div> </a> ►<em class="info-corner">-</em>
<em class="info-line">-</em> flex </div> </div> </div>
To extract the values of the href attributes you have to induce WebDriverWait for visibility_of_all_elements_located() and you can use either of the following locator strategies:
Using CSS_SELECTOR:
driver.get("https://pixabay.com/videos/search/madona/")
print([my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div.item>a")))])
Using XPATH:
driver.get("https://pixabay.com/videos/search/madona/")
print([my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[#class='item']/a")))])
Console Output:
['https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fthe-blessed-virgin-mary-in-front-of-the-roman-catholic-diocese-public-place-in-gm1297534089-390641862%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=96d4db58e5ed4dfa33719b6789ec2e54c6a9e93c', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fshining-star-landscape-above-the-nativity-scene-in-bethlehem-in-the-middle-of-the-gm1284414826-381542953%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=63c5e9d5424c6ed9234b7558a47c2f9bd34b0b2f', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fvirgin-mary-statue-and-stained-glass-window-cathedral-la-major-marseille-france-gm523679382-92792807%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=eba8300b6dc8d879d5c0c20ad4159667075b13a8', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fmary-and-joseph-kissing-and-touching-baby-jesus-gm1331078248-414315796%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=0d8025f74d6831ec1728a487c6c348fbb0489c0e', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fthe-blessed-virgin-mary-in-front-of-the-roman-catholic-diocese-public-place-in-gm1297536160-390641882%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=1a8d9dc7db4a76fe4fe8b15d050e950a4ef6ceee', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fmary-and-joseph-speaking-and-taking-care-of-baby-jesus-gm1331074139-414312596%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=71968733d6752db862f8f614ec81965c72c045c7', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fvirgin-mary-over-the-village-of-maaloula-in-syria-view-of-the-virgin-mary-in-the-gm1225424859-360681300%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=77ee3cc8968570cce77488fd36324f9a21aa8cdb', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fstatue-of-st-mary-in-the-church-gm1192697685-338971692%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=0bc49cf096c333282f735ce562f59c9eb17309f4', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fnotre-dame-de-paris-exterior-beautiful-statue-of-virgin-and-child-architecture-gm980707058-266394749%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=69ab20290bfcaef4b644d7f888618d569cd0c994', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fmary-and-joseph-with-baby-jesus-in-barn-gm1331079736-414317051%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=dbae2b824a22623d12eaf1127f0fd33f4de2cc41', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fchristmas-nativity-gm113747428-13537033%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=496f4633dd6cd6394f1a06959e4c6dd089d3aeff', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fsculpture-of-the-image-of-nossa-senhora-aparecida-the-patroness-of-brazil-gm1348138435-425434800%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=c745fbc9d7b69eeaa2186c9843f8d566590b5e61', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fchurch-icon-close-up-gm824010680-134889129%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=89a9e0bee0d8d49c4dad52407e63e5ab37a98b55', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fangled-detail-of-icon-image-of-virgin-mary-in-st-nicholas-orthodox-cathedral-in-nice-gm860958856-143133885%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=0fc269e8cf9d4fc2c0a6384537349b33fa98f963', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fvitaleta-chapel-aerial-view-in-the-wonderful-valley-of-orcia-tuscany-la-toscana-drone-gm1309396875-399121012%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=356539420e275a4aae53df6dfc2ec1e22a469f5a', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fburning-candles-in-the-cathedral-of-chartres-gm1358325815-431987333%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=3e6b1cd3f8922c80cb2ea4aacbc5f0ca13b48024', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Flourdes-france-sanctuary-of-our-lady-of-lourdes-a-famous-pilgrimage-place-gm1352878773-428151138%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=e29f6ee34464b11218e1bba7f72d321db571968a', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fsanctuary-of-our-lady-of-lourdes-gm1348213795-425484844%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=00fc1155414568f9c01d4d8e07b3e7d6396e9c76', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Frosaries-on-the-bridge-in-lourdes-gm1348213738-425484842%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=ffc4132d8125f9da46baff336297150fee0617e2', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fa-statue-of-the-virgin-mary-in-lourdes-gm1348213561-425484841%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=198c52a01d8f0d8e2c2079ea5238cfc1f3da8d00']
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Well, first of all, you haven't actually provided what the error is, which would be helpful.
Additionally, your for loop is overwriting the image var which is just bad practice, but shouldn't break anything.
Finally, it looks like you've written an infinite loop with a non-conditional break at the end and no continue. Is this supposed to be a loop?
Unable to click the "Continue to payment button" on shopify site. I have seen several similar post but most of them are for js and do not mention the spinner part of the error.
driver.find_element_by_xpath ('//*[#id="continue_button"]/svg')
<div class="content-box__row">
<div class="radio-wrapper" data-shipping-method="shopify-Standard%20Shipping-15.00">
<div class="radio__input">
<input class="input-radio" data-checkout-total-shipping="$15.00" data-checkout-total-shipping-cents="1500" data-checkout-shipping-rate="$15.00" data-checkout-original-shipping-rate="$15.00" data-checkout-total-price="$94.00" data-checkout-total-price-cents="9400" data-checkout-payment-due="$94.00" data-checkout-payment-due-cents="9400" data-checkout-payment-subform="required" data-checkout-subtotal-price="$79.00" data-checkout-subtotal-price-cents="7900" data-checkout-total-taxes="$0.00" data-checkout-total-taxes-cents="0" data-checkout-multiple-shipping-rates-group="false" data-backup="shopify-Standard%20Shipping-15.00" type="radio" value="shopify-Standard%20Shipping-15.00" name="checkout[shipping_rate][id]" id="checkout_shipping_rate_id_shopify-standard20shipping-15_00" />
</div>
<label class="radio__label" for="checkout_shipping_rate_id_shopify-standard20shipping-15_00">
<span class="radio__label__primary" data-shipping-method-label-title="Standard Shipping">
Standard Shipping
</span>
<span class="radio__label__accessory">
<span class="content-box__emphasis">
$15.00
</span>
</span>
</label> </div> <!-- /radio-wrapper-->
</div>
</div>
</div>
</div>
</div>
<div class="step__footer" data-step-footer>
<button name="button" type="submit" id="continue_button" class="step__footer__continue-btn btn" aria-busy="false"><span class="btn__content" data-continue-button-content="true">Continue to payment</span><svg class="icon-svg icon-svg--size-18 btn__spinner icon-svg--spinner-button" aria-hidden="true" focusable="false"> <use xlink:href="#spinner-button" /> </svg></button>
<a class="step__footer__previous-link" href="/18292275/checkouts/38df275516a513f1c08f6c470ef014d0?step=contact_information"><svg focusable="false" aria-hidden="true" class="icon-svg icon-svg--color-accent icon-svg--size-10 previous-link__icon" role="img" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 10 10"><path d="M8 1L7 0 3 4 2 5l1 1 4 4 1-1-4-4"/></svg><span class="step__footer__previous-link-content">Return to information</span></a>
</div>
Try this xpath :
//span[text()='Continue to payment']/..
In code :
Without explicit waits :
Code :
driver.find_element_by_xpath("//span[text()='Continue to payment']/..").click()
With Explicit waits :
Code :
wait = WebDriverWait(driver, 10)
wait.until(EC.element_to_be_clickable((By.XPATH, "//span[text()='Continue to payment']/.."))).click()
Imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Update 1 :
from selenium.webdriver.common.action_chains import ActionChains
ActionChains(driver).move_to_element(driver.find_element_by_xpath("//span[text()='Continue to payment']/..")).click().perform()
Update 2 :
driver = webdriver.Chrome(driver_path)
driver.maximize_window()
driver.implicitly_wait(50)
driver.get('https://seelbachs.com/products/sagamore-spirit-cask-strength-rye-whiskey')
wait = WebDriverWait(driver, 50)
frame_xpath = '/html/body/div[5]/div/div/div/div/iframe'
wait = WebDriverWait(driver, 10)
# wait until iframe appears and select iframe
wait.until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, frame_xpath)))
# select button
xpath = '//*[#id="enter"]'
time.sleep(2)
element= driver.find_element_by_xpath(xpath)
ActionChains(driver).move_to_element(element).click(element).perform()
# go back to main page
driver.get('https://seelbachs.com/products/sagamore-spirit-cask-strength-rye-whiskey')
# add to cart
atc= driver.find_element_by_xpath('//button[#class="btn product-form__cart-submit product-form__cart-submit--small"]')
atc.click()
# check out
co= driver.find_element_by_xpath ('//*[#id="shopify-section-cart-template"]/div/form/footer/div/div[2]/input[2]')
co.click()
# enter email
driver.find_element_by_xpath('//*[#id="checkout_email"]').send_keys('no#yahoo.com')
time.sleep(1)
# enter first name
driver.find_element_by_xpath('//*[#id="checkout_shipping_address_first_name"]').send_keys('John')
time.sleep(1)
# enter last name
driver.find_element_by_xpath('//*[#id="checkout_shipping_address_last_name"]').send_keys('Smith')
time.sleep(1)
# enter address
driver.find_element_by_xpath ('//*[#id="checkout_shipping_address_address1"]').send_keys('111 South Street')
# enter city
driver.find_element_by_xpath ('//*[#id="checkout_shipping_address_city"]').send_keys('Cocoa')
# enter zip
driver.find_element_by_xpath ('//*[#id="checkout_shipping_address_zip"]').send_keys('263153')
# enter phone
driver.find_element_by_xpath ('//*[#id="checkout_shipping_address_phone"]').send_keys('5555555'+ u'\ue007')
select = Select(wait.until(EC.visibility_of_element_located((By.ID, "checkout_shipping_address_province"))))
select.select_by_value('UK')
wait.until(EC.element_to_be_clickable((By.ID, "continue_button"))).click()
ctp = driver.find_element_by_id('continue_button')
ctp.click()
Solved this issue. I am now able to click the "Continue to Payment" button.
This is the code
`<div id="test-container"><div id="text-container"><div id="text-highlight" style="width: 48px; left: 30px; opacity: 1;"></div> <div id="test-text" class="fast-fade-in" style="min-height: 77px;"><span class="test-word">we </span><span class="test-word">like </span><span class="test-word">form </span><span class="test-word">world </span><span class="test-word">plan </span><span class="test-word">at </span><span class="test-word">look </span><span class="test-word">good </span><span class="test-word">number </span><span class="test-word">few </span><span class="test-word">more </span><span class="test-word">write </span></div></div> <div id="test-bar"><input type="text" id="test-input" spellcheck="false" autocomplete="off"> <div class="bar-items"><div id="wpm-display-container" class="bar-item"><span style="">0 <small> WPM</small></span></div> <div id="timer-display-container" class="bar-item"><span style="">0:10</span></div> <button id="reset-button" aria-label="Reset Test" class="bar-item button-highlight"><svg xmlns="http://www.w3.org/2000/svg" id="refresh-icon" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="feather feather-refresh-ccw"><polyline points="1 4 1 10 7 10"></polyline> <polyline points="23 20 23 14 17 14"></polyline> <path d="M20.49 9A9 9 0 0 0 5.64 5.64L1 10m22 4l-4.64 4.36A9 9 0 0 1 3.51 15"></path></svg></button></div></div></div>`
input element or the feild is this
I want to access it using selenium and want to use sendKeys() function but the problem is that even find_element_by_id() is not working
here is my code, it might look strange because I am using it in repl.it
`from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
list=[]
chrome_options = Options()
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
driver = webdriver.Chrome(options=chrome_options)
driver.get("https://typetest.io/")
element = driver.find_element_by_id("test-input")
element.sendKeys("Hello")`
try with explicit wait :
chrome_options = Options()
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
driver = webdriver.Chrome(options=chrome_options)
driver.maximize_window()
driver.get("https://typetest.io/")
wait = WebDriverWait(driver, 10)
wait.until(EC.element_to_be_clickable((By.ID, "test-input"))).send_keys("Hello")
Imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
You are missing a wait before finding the element to make the page loaded.
This should work better:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
list=[]
chrome_options = Options()
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
driver = webdriver.Chrome(options=chrome_options)
wait = WebDriverWait(driver, 20)
driver.get("https://typetest.io/")
element = wait.until(EC.visibility_of_element_located((By.ID, "test-input")))
element.sendKeys("Hello")
I am trying to get the text from each bar in the following plot.
Here is what I tried:
driver = webdriver.Chrome('d:/chromedriver.exe')
driver.get('https://dph.georgia.gov/covid-19-daily-status-report')
frame = driver.find_element_by_css_selector('#covid19dashdph > iframe')
driver.switch_to.frame(frame)
element = driver.find_element_by_xpath('//*[#id="root"]/div/div[3]/div[4]/div/div[4]/div/div')
print(element.text) # return ''
# action = ActionChains(driver)
# action.move_by_offset(1, 1)
My question is:
how to get the text value because I saw the text in the source page
How to move mouse cursor one bar by the other to get the next daily case number.
I just clicked on the svg tag and printed it's value which was in a tag on the site.
driver.get('https://dph.georgia.gov/covid-19-daily-status-report')
frame=WebDriverWait(driver, 30).until(EC.presence_of_element_located((By.CSS_SELECTOR, '#covid19dashdph > iframe')))
driver.switch_to.frame(frame)
svg=WebDriverWait(driver, 30).until(EC.presence_of_element_located((By.CSS_SELECTOR, " div.MuiBox-root.jss326 > div > svg")))
svg.click()
element=WebDriverWait(driver, 30).until(EC.presence_of_element_located((By.CSS_SELECTOR, "div.MuiBox-root.jss326 > div > div")))
print(element.text)
Import
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
Outputs
07Jun20
Confirmed Cases 524
7-day Moving Average 720.7
The html tag consists of:
<div class="c3-tooltip-container" style="position: absolute; pointer-events: none; display: none; top: 529.5px; left: 74.5px;">
<table class="c3-tooltip">
<tbody>
<tr><th colspan="2">07Jun20</th></tr>
<tr class="c3-tooltip-name--Confirmed-Cases">
<td class="name"><span style="background-color:#33a3ff"></span>Confirmed Cases</td>
<td class="value">524</td></tr>
<tr class="c3-tooltip-name--\37 -day-Moving-Average">
<td class="name"><span style="background-color:#ffcc32"></span>7-day Moving Average</td>
<td class="value">720.7</td>
</tr></tbody></table></div>
Using libraries Selenium/Splinter and trying to get the URL from each element to download pdf statements from wellsfargo. When scraping a table it provides links of pdf - looking to click on each link and then somehow download them to a location on the computer.
import selenium
from splinter import Browser
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.by import By
driver = webdriver.Chrome('actual_path')
driver.get('https://www.wellsfargo.com/')
driver.delete_all_cookies
mainurl = "https://www.wellsfargo.com/"
# login function - working
username = driver.find_element_by_id("userid")
username.send_keys("actual_username")
passy = driver.find_element_by_id("password")
passy.send_keys("actual_password")
submitbutton = driver.find_element_by_xpath("""//*[#id="frmSignon"]/div[5]""")
driver.find_element_by_xpath('/html/body/div[3]/section/div[1]/div[3]/div[1]/div/div[1]/a[1]').click()
driver.implicitly_wait(sleeptime)
driver.find_element_by_link_text('View Statements').click()
################## NEED HELP -TO SAVE PDF ELEMENTS AND DOWNLOAD #############
elem = driver.find_elements_by_class_name("document-title")
counttotal = 0
for pdf in elem:
counttotal = counttotal + 1
elem[counttotal].click()
driver.back()
when trying to print for i in elem: print(i) - it prints the elements but not the url link, is there any way to get the link from this element?
# Sample Doc To Click & Download
<div class="documents"><div data-message-container="stmtdiscMessages"><!------------ Error messages -----------------><!----------- Account messages ---------------></div><h3>Statements</h3><p>Deposit account statements are available online for up to 7 years.</p><div class="document large"><div class="document-details account-introtext"> <a role="link" tabindex="0" data-pdf="true" data-url="https://connect.secure.wellsfargo.com/edocs/documents/retrieve/34278aaf-8f37-43de-7d8e-e368124d5f62?_x=gTHPa3PEVAvnSu-uI5vThRyJCGUu-2f4" class="document-title" style="touch-action: auto;">Statement 08/31/19 (21K, PDF)</a></div></div><div class="document large">
#document number 2
<div class="document-details account-introtext"> <a role="link" tabindex="0" data-pdf="true" data-url="https://connect.secure.wellsfargo.com/edocs/documents/retrieve/9efe2b61-8233-8s65-2738-677ef63291f7?_x=h8i20NifIc9dRVCvj9I8pkic0S80i" class="document-title" style="touch-action: auto;">Statement 07/31/19 (21K, PDF)</a></div></div><div class="document large">
#document number 3, etc.
<div class="document-details account-introtext"> <a role="link" tabindex="0" data-pdf="true" data-url="https://connect.secure.wellsfargo.com/edocs/documents/retrieve/7eece2e7-e27e-4445-8s4d-fa5899c5c96b?_x=037X7K-IdhVOVevUISRnQT74qL793tIW" class="document-title" style="touch-action: auto;">Statement 06/30/19 (24K, PDF)</a></div></div><div class="document large">
You can retrieve any attribute from an element with the get_attribute function:
elements = driver.find_elements_by_class_name("document-title")
pdf_urls = []
for element in elements:
pdf_urls.append(element.get_attribute('data-url'))
Or if you are used to list comprehensions, here's a more pythonic way:
elements = driver.find_elements_by_class_name("document-title")
pdf_urls = [element.get_attribute('data-url') for element in elements]