We Keep Coding

Select a button with Selenium

I have a webpage I am trying to fill in. In the middle of the page there is a button i need to click, whose info in the inspection are as follow:
<label class="btn btn-default col-md-6 ng-binding active btn-success" ng-class="{'btn-success active': paziente.consenso_informato == 1 }" style="">
<input type="radio" ng-model="paziente.consenso_informato" name="consenso_informato" ng-required="true" ng-value="1" class="ng-not-empty ng-dirty ng-valid-parse ng-valid ng-valid-required ng-touched" value="1" required="required" style=""> Si
</label>
So I tried this code but there is no way I can click it:
consent_xpath = "//label[#ng-class='{\"btn-success active\": paziente.consenso_informato == 1 }']/input[#value='1']"
# Wait for the element to be visible
wait = WebDriverWait(wd, 1)
element = wait.until(EC.visibility_of_element_located((By.XPATH, consent_xpath)))
# Scroll down to the element and click on it
wd.execute_script("arguments[0].scrollIntoView();", element)
element.click()
Also tried this:
try:
element = wd.find_element_by_xpath("//label[#ng-class='{'btn-success active': paziente.consenso_informato == 1 }']")
if element.is_displayed():
element.click()
print('found')
break
except:
wd.execute_script("window.scrollBy(0, 100);")

try this:
from selenium import webdriver
driver = webdriver.Chrome()
driver.get("your url")
button = driver.find_element( By.CLASS_NAME, 'ng-not-empty.ng-dirty.ng-valid-parse.ng-valid.ng-valid-required.ng-touched')
button.click()

This solved my problem:
# Click yes on consenso informato
overlay_locator = (By.CSS_SELECTOR, ".loading-layer")
target_locator = (By.XPATH, "/html/body/div[2]/div/form/div[2]/div[2]/div[2]/div/div/label[1]")
# Wait for the overlay to disappear
wait = WebDriverWait(wd, 10)
wait.until(EC.invisibility_of_element_located(overlay_locator))
# Wait for the target element to be clickable
target = wait.until(EC.element_to_be_clickable(target_locator))
target.click()

Selenium python : my current_url doesnt update after click

Im scraping a website where I need to retrieve values from the url when i click on a button providing different form values.
I have a problem: when i click the button and retrieve the current_url, the provided values in the forms doesnt reflect in the url which should be updated (it's a search button). There is no new tab created.
My code to retrieve the url value is :
driver = webdriver.Firefox()
driver.get(url)
arrlist = []
idlist = []
service=value
for i in key_list:
form = driver.find_elements(by=By.XPATH, value='//input[#id="geo_nav"]')
form[0].send_keys(i)
form2=driver.find_elements(by=By.XPATH, value='//input[#id="sev_nav"]')
form2[0].send_keys(service)
button=driver.find_elements(by=By.XPATH, value='//button[#data-role="filter-apply"]')
button[0].click()
time.sleep(5)
url=driver.current_url
print(dept)
print(i)
id=re.findall(r"(?<=\[population\]=)(\d{9})",url)[0]
arrlist.append(i)
idlist.append(id)
the button html code is :
<button class="filter-apply cta-navigate relative hide-mobile flex withNumber" data-role="filter-apply">
<p class="hide-mobile m-r-4">Appliquer</p>
<div class="svg relative">
<span class="filters-apply-length">2</span>
<svg height="18" viewBox="0 0 16 18" width="16" xmlns="http://www.w3.org/2000/svg"><path d="m10.877 17.457 2.026 1.533v-4.553c0-.166.042-.329.12-.475l4.3-7.962h-10.68l4.122 7.978c.074.142.112.3.112.459zm3.026 4.543c-.213 0-.426-.068-.603-.203l-4.026-3.045c-.25-.189-.397-.484-.397-.797v-3.274l-4.765-9.222c-.161-.31-.148-.681.034-.979.181-.298.505-.48.854-.48h14c.352 0 .678.185.859.488.18.302.188.677.021.987l-4.977 9.215v6.31c0 .379-.214.726-.554.895-.141.07-.294.105-.446.105z" fill="#0579c7" fill-rule="evenodd" transform="translate(-4 -4)"></path></svg> </div>
</button>
I've tried to use
driver.switch_to.window(driver.window_handles[-1]);
following this post : Python Selenium Chromedriver - Can't Get current_url of new opened tab after click()
But I dont have tab or new windows issues.
I tried to click autocompletion lists in the 2 forms in inputand one of the form produces a modification of the url but not the other (the one of which effects on the url i need to monitor).
The form code that works is :
<form data-component="sev_nav_input" data-no-results="Sans résultats" data-default-pho="Services" data-selected-name="Achat compulsif" data-selected-id="5928" class="filter-input filter-services relative">
<input type="text" placeholder="Services" autocomplete="off" name="sev_nav" id="sev_nav" data-role="js_filter" data-id="5928" class="autocomplete-with-result">
<span id="clear-sev-input" class="clear-sev-input" style="display: none;">
<img src="data:image/svg+xml;base64,PHN2ZyBoZWlnaHQ9IjI0IiB2aWV3Qm94PSIwIDAgMjQgMjQiIHdpZHRoPSIyNCIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj48ZyBmaWxsPSJub25lIiBmaWxsLXJ1bGU9ImV2ZW5vZGQiPjxwYXRoIGQ9Im0wIDBoMjR2MjRoLTI0eiIvPjxwYXRoIGQ9Im0xMS43NSAyMC41YzIuNDIyIDAgNC40ODYtLjg1MyA2LjE5MS0yLjU1OSAxLjcwNi0xLjcwNSAyLjU1OS0zLjc3IDIuNTU5LTYuMTkxIDAtMi40MjItLjg1My00LjQ4Ni0yLjU1OS02LjE5MS0xLjcwNS0xLjcwNi0zLjc3LTIuNTU5LTYuMTkxLTIuNTU5LTIuNDIyIDAtNC40OTIuODYtNi4yMSAyLjU3OC0xLjY5NSAxLjY5My0yLjU0IDMuNzUtMi41NCA2LjE3MnMuODUzIDQuNDg2IDIuNTU5IDYuMTkxYzEuNzA1IDEuNzA2IDMuNzcgMi41NTkgNi4xOTEgMi41NTl6bTMuMTY0LTQuNDE0Yy0uMDUyIDAtLjExNy0uMDQtLjE5NS0uMTE3bC0yLjk2OS0yLjkzLTIuOTMgMi45NjljLS4wNTIuMDUyLS4xMy4wNzgtLjIzNC4wNzhzLS4xODItLjAyNi0uMjM0LS4wNzhsLS44Mi0uODZjLS4wNTMtLjA1Mi0uMDc5LS4xMy0uMDc5LS4yMzRzLjAyNi0uMTcuMDc4LS4xOTVsMi45NjktMi45NjktMi45NjktMi45M2MtLjE1Ni0uMTU2LS4xNTYtLjMxMiAwLS40NjhsLjgyLS44MmMuMDc5LS4wNzkuMTU3LS4xMTguMjM1LS4xMTguMDUyIDAgLjExNy4wNC4xOTUuMTE3bDIuOTY5IDIuODkgMi45NjktMi44OWMuMDc4LS4wNzguMTQzLS4xMTcuMTk1LS4xMTcuMDc4IDAgLjE1Ni4wNC4yMzQuMTE3bC44Ni44MmMuMTU2LjE1Ny4xNTYuMzEzIDAgLjQ3bC0yLjk2OSAyLjkyOSAyLjkzIDIuOTNjLjA3OC4wNzguMTE3LjE1Ni4xMTcuMjM0IDAgLjEwNC0uMDQuMTgyLS4xMTcuMjM0bC0uODIuODJjLS4wNzkuMDc5LS4xNTcuMTE4LS4yMzUuMTE4eiIgZmlsbD0iIzE0OWM5NyIvPjwvZz48L3N2Zz4K"></span>
<span class="gradient"></span>
<span class="gradient" style="display: none;"></span>
<div class="spinner" style="display: none;"></div>
<div id="services-list" class="services-list" style="display: none;"><ul data-role="autocomplete-list" class="autocomplete-list"> </ul></div></form>
The form code that doesnt work is :
<form data-component="geo_nav_input" data-selected-name="" data-selected-id="" data-selected-neighborhood-id="0" data-selected-type="" data-no-results="Sans résultats" data-pho="Localité" data-default-pho="Localité" class="filter-input relative">
<div class="hide">Chercher des professionnels en/à...</div>
<span class="icon-x toggle_geo_nav hide"></span>
<label for="geo_nav" class="hidden-label">Localité</label>
<input type="text" placeholder="Localité" autocomplete="off" name="geo_nav" id="geo_nav" data-role="js_filter" data-id="" data-neighborhoodid="0" data-type="" class="autocomplete-with-result"> <span id="clear-geo-input" class="clear-geo-input" style="display: none;">
<img src="data:image/svg+xml;base64,PHN2ZyBoZWlnaHQ9IjI0IiB2aWV3Qm94PSIwIDAgMjQgMjQiIHdpZHRoPSIyNCIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj48ZyBmaWxsPSJub25lIiBmaWxsLXJ1bGU9ImV2ZW5vZGQiPjxwYXRoIGQ9Im0wIDBoMjR2MjRoLTI0eiIvPjxwYXRoIGQ9Im0xMS43NSAyMC41YzIuNDIyIDAgNC40ODYtLjg1MyA2LjE5MS0yLjU1OSAxLjcwNi0xLjcwNSAyLjU1OS0zLjc3IDIuNTU5LTYuMTkxIDAtMi40MjItLjg1My00LjQ4Ni0yLjU1OS02LjE5MS0xLjcwNS0xLjcwNi0zLjc3LTIuNTU5LTYuMTkxLTIuNTU5LTIuNDIyIDAtNC40OTIuODYtNi4yMSAyLjU3OC0xLjY5NSAxLjY5My0yLjU0IDMuNzUtMi41NCA2LjE3MnMuODUzIDQuNDg2IDIuNTU5IDYuMTkxYzEuNzA1IDEuNzA2IDMuNzcgMi41NTkgNi4xOTEgMi41NTl6bTMuMTY0LTQuNDE0Yy0uMDUyIDAtLjExNy0uMDQtLjE5NS0uMTE3bC0yLjk2OS0yLjkzLTIuOTMgMi45NjljLS4wNTIuMDUyLS4xMy4wNzgtLjIzNC4wNzhzLS4xODItLjAyNi0uMjM0LS4wNzhsLS44Mi0uODZjLS4wNTMtLjA1Mi0uMDc5LS4xMy0uMDc5LS4yMzRzLjAyNi0uMTcuMDc4LS4xOTVsMi45NjktMi45NjktMi45NjktMi45M2MtLjE1Ni0uMTU2LS4xNTYtLjMxMiAwLS40NjhsLjgyLS44MmMuMDc5LS4wNzkuMTU3LS4xMTguMjM1LS4xMTguMDUyIDAgLjExNy4wNC4xOTUuMTE3bDIuOTY5IDIuODkgMi45NjktMi44OWMuMDc4LS4wNzguMTQzLS4xMTcuMTk1LS4xMTcuMDc4IDAgLjE1Ni4wNC4yMzQuMTE3bC44Ni44MmMuMTU2LjE1Ny4xNTYuMzEzIDAgLjQ3bC0yLjk2OSAyLjkyOSAyLjkzIDIuOTNjLjA3OC4wNzguMTE3LjE1Ni4xMTcuMjM0IDAgLjEwNC0uMDQuMTgyLS4xMTcuMjM0bC0uODIuODJjLS4wNzkuMDc5LS4xNTcuMTE4LS4yMzUuMTE4eiIgZmlsbD0iIzE0OWM5NyIvPjwvZz48L3N2Zz4K"></span>
<span class="gradient"></span>
<span class="gradient" style="display: none;"></span>
<div class="spinner" style="display: none;"></div>
<div id="location-list" class="location-list" style="display: none;"><ul data-role="autocomplete-list" class="autocomplete-list"> </ul></div>
</form>

Can you make a function to navigate pages, and on each page do the actions you require. And with each call of the function use driver.switch_to.window to ensure you are on the latest page.
Although based on your edits, it now seems the issue is that you are having trouble locating and following one of the links on the pages.
def navigate(n):
""" Move through the pages. Select the relevant buttons on each page"""
window_after = driver.window_handles[0]
driver.switch_to.window(window_after)
if n == 0:
form = driver.find_elements(by=By.XPATH, value='//input[#id="geo_nav"]')
button = driver.find_elements(by=By.XPATH, value='//button[#data-role="filter-apply"]').click()
elif n == 1:
pass
# Do something
else:
pass
# Do something else
for i in range(3):
navigate(i)
time.sleep(3)

The solution was in fact linked to the autocompletion forms. They require you to click on the autocompletion suggestions so the button is actually working.
FYI, here is the full code to autocomplete with clicking the form, deleting the content, adding the content, clicking the list and clicking the button.
def get_city_locations(service):
url='url'
#options = Options()
#options.headless = True
driver = webdriver.Firefox()#options=options)
driver.get(url)
time.sleep(2)
buttoncookie = driver.find_elements(by=By.XPATH, value='//button[#class="cf2Lf6"]')
buttoncookie[0].click()
time.sleep(1)
form2 = driver.find_elements(by=By.XPATH, value='//input[#id="sev_nav"]')
form2[0].click()
time.sleep(1)
Static.clear_text(driver)
form2[0].send_keys(service)
time.sleep(1)
autocompleteservice = driver.find_elements(by=By.XPATH, value='//li[not(#class)]')
for f in autocompleteservice:
if f.text == service:
f.click()
df_pref=pd.read_csv('arrondissement_2022.csv',sep=',')
deptlist = []
arrlist = []
idlist = []
for i in df_pref['LIBELLE']:
df_dep=df_pref[df_pref['LIBELLE']==i]
dept = df_dep.loc[df_dep.index.values[0], 'DEP']
form = driver.find_elements(by=By.XPATH, value='//input[#id="geo_nav"]')
form[0].click()
time.sleep(1)
Static.clear_text(driver)
form[0].send_keys(i)
time.sleep(3)
autocompletelocation=driver.find_elements(by=By.XPATH, value='//li[not(#class)]')
cond=0
for a in autocompletelocation:
if a.text==i:
print ('condition ok')
cond=1
a.click()
break
time.sleep(3)
button=driver.find_elements(by=By.XPATH, value='//button[#data-role="filter-apply"]')
button[0].click()
time.sleep(3)
driver.switch_to.window(driver.window_handles[-1]);
url=driver.current_url
print(dept)
print(i)
print(url)
if cond==0:
id=0
else:
id=re.findall(r"(?<=\[population\]=)(\d{7,10})",url)[0]
print(f'id = {id}')
print('\n')
deptlist.append(dept)
arrlist.append(i)
idlist.append(id)
df0 = pd.DataFrame({"dept": deptlist, "arrondissement":arrlist,"id":idlist})
df0.to_csv('arr_id.csv',sep=';',index=False)

Selenium: Can't find an element in HTML

Hi I am working on the script to automate downloads of videos from this side https://pixabay.com/videos/
I can find a class with href(href is an attribute with URL) but after that Selenium gives me a bug with any error only a result of print(xy.get_atribute("href)) is None:
my code:
from selenium import webdriver
from selenium.webdriver.firefox.service import Service
from webdriver_manager.firefox import GeckoDriverManager
from selenium.webdriver.common.by import By
from mutagen.mp3 import MP3
import requests
import time
tag = "city "
while True:
s = Service(GeckoDriverManager().install())
driver = webdriver.Firefox(service=s)
driver.minimize_window()
tester = tag.split()
print(tester)
print(len(tester))
if len(tester) == 2:
tag = tester[0] + "%20" + tester[1]
print(tag)
print("2 " + tag)
driver.get("https://pixabay.com/cs/videos/search/" + tag )
images = driver.find_elements(By.CLASS_NAME, 'item' )
print(images)
n = 0
lenght = 0
for image in images:
image = image.get_attribute("href")
print(image)
break
HTML on the side
<html lang="cs" prefix="og: http://ogp.me/ns#">
<head>.</head> <body class="" data-new-gr-C-5-check-loaded="14.1050." data-gr-ext-installed>
<noscript>_</noscript> <div id="wrapper"> > <div id="header">...</div> <div id="content" class="clearfix">
::before <div id="search-term" style="display:none">city</div> <div class="media_list">
<div style="border-bottom:1px solid #f0f1f4"> </div> <div style="background:#e8eaec" class="external-media">.</div> <div style="background: #f6f5fa"> <div style="max-width: 1824px;padding: 10px 3px 20px;margin: auto">
<h1 style="font-size: 13px;color:#bbb;margin:0 19px; position:relative;top:2px">70 videa zdarma z city</h1> <div class="related-keywords">...</div> <div class="row-masonry video video-search-results"> flex <div class="row-masonry-cell" style="flex-basis: 355.55555555555554px; flex-grow: 1.7777777777777777; flex-shrink: 1.7777777777777777; max-width: 622.2222222222222px"> <div class="row-masonry-cell-outer" style="padding-top: 56.25%"> <div class="row-masonry-cell-inner"> <div itemscope itemtype="schema.org/videoobject" class="item" data-w="1920" data-h="1980">
<meta itemprop="license" content="https://creativecommons.org/licenses/publicdomain/"> <meta itemprop="contentUrl" content="//player.vimeo.com/external/142621375.mobile.mp4?s=e9a3c9616798b6f3de74d579ea8314acc75fad72&profile_id=116"> <meta itemprop="thumbnailUrl" content="https://i.vimeocdn.com/video/539965294-5d28c2680682aa5173e86fa74acb94671783ba7e2dc2892682e897c8a158af75-d_640x360.jpg"> <meta itemprop="name" content="New York City, Manhattan, Lidé"> <meta itemprop="description" content="New York City, Manhattan, Lidé, Auta, Rozcestí, Amerika"> <meta itemprop="duration" content="TM145"> <meta itemprop="uploadDate" content="2022-02-16"> <a href="/cs/videos/new-york-city-manhattan-lidC3%A9-auta-1944/"> == $ <div class="media" data-mp4="//player.vimeo.com/external/142621375.mobile.mp4?s=e9a3c9616798b6f3de74d579ea8314acc75fad72&profile_id=116">
<img class="video-preview" src="https://i.vimeocdn.com/video/539965294-5d28c26...d 640x360.jpg" alt="New York City, Manhattan, Lidé, Auta">
<i></i> </div> </a> ►<em class="info-corner">-</em>
<em class="info-line">-</em> flex </div> </div> </div>

To extract the values of the href attributes you have to induce WebDriverWait for visibility_of_all_elements_located() and you can use either of the following locator strategies:
Using CSS_SELECTOR:
driver.get("https://pixabay.com/videos/search/madona/")
print([my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div.item>a")))])
Using XPATH:
driver.get("https://pixabay.com/videos/search/madona/")
print([my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[#class='item']/a")))])
Console Output:
['https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fthe-blessed-virgin-mary-in-front-of-the-roman-catholic-diocese-public-place-in-gm1297534089-390641862%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=96d4db58e5ed4dfa33719b6789ec2e54c6a9e93c', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fshining-star-landscape-above-the-nativity-scene-in-bethlehem-in-the-middle-of-the-gm1284414826-381542953%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=63c5e9d5424c6ed9234b7558a47c2f9bd34b0b2f', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fvirgin-mary-statue-and-stained-glass-window-cathedral-la-major-marseille-france-gm523679382-92792807%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=eba8300b6dc8d879d5c0c20ad4159667075b13a8', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fmary-and-joseph-kissing-and-touching-baby-jesus-gm1331078248-414315796%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=0d8025f74d6831ec1728a487c6c348fbb0489c0e', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fthe-blessed-virgin-mary-in-front-of-the-roman-catholic-diocese-public-place-in-gm1297536160-390641882%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=1a8d9dc7db4a76fe4fe8b15d050e950a4ef6ceee', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fmary-and-joseph-speaking-and-taking-care-of-baby-jesus-gm1331074139-414312596%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=71968733d6752db862f8f614ec81965c72c045c7', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fvirgin-mary-over-the-village-of-maaloula-in-syria-view-of-the-virgin-mary-in-the-gm1225424859-360681300%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=77ee3cc8968570cce77488fd36324f9a21aa8cdb', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fstatue-of-st-mary-in-the-church-gm1192697685-338971692%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=0bc49cf096c333282f735ce562f59c9eb17309f4', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fnotre-dame-de-paris-exterior-beautiful-statue-of-virgin-and-child-architecture-gm980707058-266394749%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=69ab20290bfcaef4b644d7f888618d569cd0c994', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fmary-and-joseph-with-baby-jesus-in-barn-gm1331079736-414317051%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=dbae2b824a22623d12eaf1127f0fd33f4de2cc41', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fchristmas-nativity-gm113747428-13537033%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=496f4633dd6cd6394f1a06959e4c6dd089d3aeff', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fsculpture-of-the-image-of-nossa-senhora-aparecida-the-patroness-of-brazil-gm1348138435-425434800%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=c745fbc9d7b69eeaa2186c9843f8d566590b5e61', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fchurch-icon-close-up-gm824010680-134889129%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=89a9e0bee0d8d49c4dad52407e63e5ab37a98b55', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fangled-detail-of-icon-image-of-virgin-mary-in-st-nicholas-orthodox-cathedral-in-nice-gm860958856-143133885%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=0fc269e8cf9d4fc2c0a6384537349b33fa98f963', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fvitaleta-chapel-aerial-view-in-the-wonderful-valley-of-orcia-tuscany-la-toscana-drone-gm1309396875-399121012%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=356539420e275a4aae53df6dfc2ec1e22a469f5a', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fburning-candles-in-the-cathedral-of-chartres-gm1358325815-431987333%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=3e6b1cd3f8922c80cb2ea4aacbc5f0ca13b48024', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Flourdes-france-sanctuary-of-our-lady-of-lourdes-a-famous-pilgrimage-place-gm1352878773-428151138%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=e29f6ee34464b11218e1bba7f72d321db571968a', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fsanctuary-of-our-lady-of-lourdes-gm1348213795-425484844%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=00fc1155414568f9c01d4d8e07b3e7d6396e9c76', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Frosaries-on-the-bridge-in-lourdes-gm1348213738-425484842%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=ffc4132d8125f9da46baff336297150fee0617e2', 'https://pixabay.com/link/?ua=t%3Devent%26ec%3Dapi_ad%26ea%3Dnavigate%26el%3Dgetty%26v%3D1%26tid%3DUA-20223345-1&next=https%3A%2F%2Fwww.istockphoto.com%2Fvideo%2Fa-statue-of-the-virgin-mary-in-lourdes-gm1348213561-425484841%3Futm_source%3Dpixabay%26utm_medium%3Daffiliate%26utm_campaign%3DSRP_video_sponsored%26utm_content%3Dhttp%253A%252F%252Fpixabay.com%252Fvideos%252Fsearch%252Fmadona%252F%26utm_term%3Dmadona&hash=198c52a01d8f0d8e2c2079ea5238cfc1f3da8d00']
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Well, first of all, you haven't actually provided what the error is, which would be helpful.
Additionally, your for loop is overwriting the image var which is just bad practice, but shouldn't break anything.
Finally, it looks like you've written an infinite loop with a non-conditional break at the end and no continue. Is this supposed to be a loop?

Python Selenium Dropdown Menu Last Element Selection

I am attempting to select an element that is not displayed in the default menu. The element is the last one and won't show up unless scrolled down. I can change the element number but it does not update the table values of the element number. Below is the code I tried. Any pointers would be greatly appreciated!! Looked through some of the questions and could not replicate any working solutions.
Screenshot:
Page Source Image
Code below:
week_dropdown = driver.find_element_by_xpath('//*[#id="SEARCH"]/div/div[3]/div[2]/div[1]/div/div')
driver.execute_script("arguments[0].click();", week_dropdown)
page_value_span = driver.find_element_by_xpath('//*[#id="SEARCH"]/div/div[3]/div[2]/div[1]/div/div/span[2]')
page_value_edit = driver.execute_script('arguments[0].innerHTML = "1";', page_value_span)
driver.execute_script("arguments[0].click();",page_value_span)
<div span="8" class="ant-row ant-row-end" style="padding-top: 10px;"><div class="style_reportOptionBlock__1Qm2T" style="margin-right: 20px;"><label class="style_vlabelfull__1uP_p">#Weeks: </label><div class="ant-select style_vSelectSearchnew__3UstT ant-select-single ant-select-show-arrow" style="width: 60px;"><div class="ant-select-selector"><span class="ant-select-selection-search"><input autocomplete="off" class="ant-select-selection-search-input" role="combobox" aria-haspopup="listbox" aria-owns="rc_select_1_list" aria-autocomplete="list" aria-controls="rc_select_1_list" aria-activedescendant="rc_select_1_list_-1" readonly="" unselectable="on" value="" id="rc_select_1" style="opacity: 0;" aria-expanded="false"></span><span class="ant-select-selection-item" title="4">4</span></div><span class="ant-select-arrow" unselectable="on" aria-hidden="true" style="user-select: none;"><span role="img" aria-label="down" class="anticon anticon-down ant-select-suffix"><svg viewBox="64 64 896 896" focusable="false" class="" data-icon="down" width="1em" height="1em" fill="currentColor" aria-hidden="true"><path d="M884 256h-75c-5.1 0-9.9 2.5-12.9 6.6L512 654.2 227.9 262.6c-3-4.1-7.8-6.6-12.9-6.6h-75c-6.5 0-10.3 7.4-6.5 12.7l352.6 486.1c12.8 17.6 39 17.6 51.7 0l352.6-486.1c3.9-5.3.1-12.7-6.4-12.7z"></path></svg></span></span></div></div>

You can try the with ActionChains
week_dropdown = driver.find_element_by_xpath("xpath for open dropdown...").click()
element = driver.find_element_by_xpath("//div[#class='ant-select-item ant-select-item-option']/div[text()='1']")
actions = ActionChains(driver)
actions.move_to_element(to_element=element)
actions.click(on_element=element).perform()
Make sure to import ActionChains
from selenium.webdriver.common.action_chains import ActionChains
You may also try with arguments[0].scrollIntoView()
week_dropdown = driver.find_element_by_xpath("xpath for open dropdown...").click()
element = driver.find_element_by_xpath("//div[#class='ant-select-item ant-select-item-option']/div[text()='1']")
driver.execute_script('arguments[0].scrollIntoView();', element)
Was this what you were looking for?

If you are able to scroll through the menu, then put a small wait first before clicking/selecting that last option. Let me know if this works ?

Unable to fetch the relevant links and discard others

I've written a script in python in combination with selenium along with BeautifulSoup to get the links leading to property details from a webpage. As the content are heavily dynamic, I made use of selenium to get the page source. When I run my script, I get lots of links including those required links.
How can I get only the relevant link from each container out of the three?
My try:
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
def fetch_info(link):
driver.get(link)
wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, "#community-search-homes .propertyWrapper > a")))
soup = BeautifulSoup(driver.page_source, "lxml")
linklist = [item.get("href") for item in soup.select("#community-search-homes .propertyWrapper > a")]
return linklist
if __name__ == '__main__':
url = "https://www.khov.com/find-new-homes/arizona/buckeye"
driver = webdriver.Chrome()
wait = WebDriverWait(driver,10)
for newlink in fetch_info(url):
print(newlink)
driver.quit()
Results I'm having:
/find-new-homes/arizona/buckeye/85396/k-hovnanian-homes/aspire-at-sienna-hills
/find-new-homes/arizona/buckeye/85396/k-hovnanian-homes/affinity-at-verrado
/find-new-homes/arizona/buckeye/85396/four-seasons/k.-hovnanian's-four-seasons-at-victory-at-verrado
/find-new-homes/arizona/scottsdale/85255/k-hovnanian-homes/summit-at-silverstone
/find-new-homes/arizona/scottsdale/85257/k-hovnanian-homes/skye
/find-new-homes/arizona/phoenix/85020/k-hovnanian-homes/pointe-16
/find-new-homes/arizona/peoria/85383/k-hovnanian-homes/fusion-ii-at-the-meadows
/find-new-homes/arizona/scottsdale/85257/k-hovnanian-homes/aire
/find-new-homes/arizona/scottsdale/85255/k-hovnanian-homes/pinnacle-at-silverstone
/find-new-homes/arizona/peoria/85383/k-hovnanian-homes/montage-at-the-meadows
/find-new-homes/arizona/sun-city/85373/four-seasons/k.-hovnanian-s-four-seasons-at-ventana-lakes
/find-new-homes/arizona/peoria/85382/k-hovnanian-homes/park-paseo
/find-new-homes/arizona/laveen/85339/k-hovnanian-homes/affinity-at-montana-vista
/find-new-homes/arizona/laveen/85339/k-hovnanian-homes/aspire-at-montana-vista
/find-new-homes/arizona/scottsdale/85255/k-hovnanian-homes/pinnacle-ii-at-silverstone
/find-new-homes/arizona/scottsdale/85255/k-hovnanian-homes/summit-ii-at-silverstone
Results I would like to get:
/find-new-homes/arizona/buckeye/85396/k-hovnanian-homes/aspire-at-sienna-hills
/find-new-homes/arizona/buckeye/85396/k-hovnanian-homes/affinity-at-verrado
/find-new-homes/arizona/buckeye/85396/four-seasons/k.-hovnanian's-four-seasons-at-victory-at-verrado
A chunk of html elements (the link I'm after is within the second line of the following elements):
<div class="propertyWrapper clear">
<span class="link-outside"></span>
<div class="propertyCarouselWrapper">
<div class="responsiveImageCarousel enabled" style="touch-action: pan-y; user-select: none; -webkit-user-drag: none; -webkit-tap-highlight-color: rgba(0, 0, 0, 0);">
<div class="prevBtn"></div>
<div class="nextBtn"></div>
<div class="images" data-detail-url="/find-new-homes/arizona/buckeye/85396/k-hovnanian-homes/aspire-at-sienna-hills">
<ul style="width: 960px; left: 0px;">
<li style="width: 320px;"><img alt="holiday exterior new homes sienna hills usp" src="https://khovcachecdn.azureedge.net/azure/sitefinitylibraries/images/default-source/images/az/aspire-at-sienna-hills/community-thumbnails/holiday-exterior-new-homes-sienna-hills-usp.jpg?sfvrsn=4&build=1019&encoder=wic&useresizingpipeline=true&w=450&h=280&mode=crop"></li>
<li style="width: 320px;"><img alt="carnival exterior new homes sienna hills usp" src="https://khovcachecdn.azureedge.net/azure/sitefinitylibraries/images/default-source/images/az/aspire-at-sienna-hills/community-thumbnails/carnival-exterior-new-homes-sienna-hills-usp.jpg?sfvrsn=4&build=1019&encoder=wic&useresizingpipeline=true&w=450&h=280&mode=crop"></li>
</ul>
</div>
<div class="pagination" style="width: 56px;"><ul><li class="active"></li><li></li></ul></div>
</div>
</div>
<div class="propertyInfoWrapper">
<div class="marker-details-container">
<h3 class="marker-details">New Homes in Buckeye, Arizona</h3>
<div class="spacer"></div>
<h4 class="propertyListingHeader">Aspire at Sienna Hills</h4>
<p class="marker-details">21007 West Almeria Road, Buckeye, AZ 85396</p>
<p class="marker-details marker-status">Final Opportunities</p>
<div class="spacer"></div>
<p class="marker-details marker-price"><span class="bold">Priced from: </span>Mid $200s</p>
<p class="marker-details"><span class="bold">Home type: </span>Single Family Homes</p>
<p class="marker-details marker-amenities"><span class="bold">Amenities: </span>Pool, Hiking Trails, Park</p>
</div>
<div class="community-tag-container">
<a href="/find-new-homes/arizona/buckeye/85396/k-hovnanian-homes/aspire-at-sienna-hills#quick-move-in-homes" onclick="KHOV.Analytics.trackEvent('Qmi_Icon_Qmi');">
<div class="community-tag">
<div class="ctaDesc quick-move-in-badge link-inside">Quick Move In Homes</div>
<div class="ctaIcon quick-move-in-badge-icon link-inside"></div>
</div>
</a>
</div>
<a href="#request-info-form-modal" class="open-inline-modal-link" onclick="KHOV.Analytics.trackEvent('Orange_Ri_Request_Info');">
<div class="button orange-color requestInfoButton link-inside" data-urlname="aspire-at-sienna-hills">Request Info</div>
</a>
</div>
</div>

You need to include the featured id as well as results. You can use Or to combine. Latest bs4 supports not.
#propertyResultsContainer .propertyWrapper :not([onclick])[href*=find], #propertyFeaturedResultsContainer .propertyWrapper :not([onclick])[href*=find]
This can also be shortened to
#propertyResultsContainer .propertyWrapper :not([onclick])[href*=find], #propertyFeaturedResultsContainer
But that shortening may be less robust.

You can just check for the desired keyword in the link and print those, and ignore the others:
if __name__ == '__main__':
url = "https://www.khov.com/find-new-homes/arizona/buckeye"
driver = webdriver.Chrome()
wait = WebDriverWait(driver,10)
for newlink in fetch_info(url):
if url.split('/')[-1] in newlink:
print(newlink)
driver.quit()
Output:
/find-new-homes/arizona/buckeye/85396/k-hovnanian-homes/aspire-at-sienna-hills
/find-new-homes/arizona/buckeye/85396/k-hovnanian-homes/affinity-at-verrado
/find-new-homes/arizona/buckeye/85396/four-seasons/k.-hovnanian's-four-seasons-at-victory-at-verrado

Would list slicing works?
def fetch_info(link):
driver.get(link)
wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, "#community-search-homes .propertyWrapper > a")))
soup = BeautifulSoup(driver.page_source, "lxml")
linklist = [item.get("href") for item in soup.select("#community-search-homes .propertyWrapper > a")][:3]
return linklist