We Keep Coding

Selenium presence_of_element_located look for children of an element

I was wondering if it's possible to look for children of an element with the presence_of_element_located function. I know I could just use the entire path, but that would make my code more confusing due to it's nature. My code would look something like this (much more complicated but this is the important bit):
currentEl = driver.find_element(By.XPATH, ("//*[#id='2']"))
func(currentEl)
def func(currentEl):
#Wait for the element to appear
WebDriverWait(driver, 10, poll_frequency=0.001, ignored_exceptions = (StaleElementReferenceException, NoSuchElementException)).until(
EC.presence_of_element_located((currentEl, (By.CLASS_NAME, "class")))
)
return
So basically I'm trying to wait for the child element with the class name "class" of the WebElement named currentEl. I'we tried many things and I'd hate to resort to just using the entire path.

Child element with class name className can be located by relative XPath .//*[contains(#class,'className')] or with relative CSS Selector .className.
So, I think your code can be modified to be
currentEl = driver.find_element(By.XPATH, ("//*[#id='2']"))
func(currentEl)
def func(currentEl):
#Wait for the element to appear
WebDriverWait(driver, 10, poll_frequency=0.001, ignored_exceptions = (StaleElementReferenceException, NoSuchElementException)).until(
EC.presence_of_element_located((currentEl, By.XPATH, ".//*[contains(#class,'className')]")))
return
Or
currentEl = driver.find_element(By.XPATH, ("//*[#id='2']"))
func(currentEl)
def func(currentEl):
#Wait for the element to appear
WebDriverWait(driver, 10, poll_frequency=0.001, ignored_exceptions = (StaleElementReferenceException, NoSuchElementException)).until(
EC.presence_of_element_located((currentEl, By.CSS_SELECTOR, ".className")))
return
UPD
I'm not sure the structure above will work.
What you can do is to pass correct locator to existing expected_conditions, as following:
locator1 = (By.XPATH, "//*[#id='2']//*[contains(#class,'className')]")
locator2 = (By.CSS_SELECTOR, "#2 .className")
def wait_for_child_element(locator):
WebDriverWait(driver, 30).wait.until(EC.presence_of_element_located(locator))
#call the method above with arguments:
wait_for_child_element(locator1)
wait_for_child_element(locator2)

Loop through the div tag containing paragraph tags

I am trying to scrape information from an automobile blog but i can't loop through the div tag containing the paragraph tags that contain the info.
driver.get("https://www.autocar.co.uk/car-news")
driver.maximize_window()
for i in range(3):
i+=1
info = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.XPATH, f'//*[#id="page"]/div[2]/div[1]/div[1]/div[2]/div/div[1]/div/div[1]/div[1]/div[{i}]/div')))
heading = info.find_element_by_tag_name('h2')
clickable = heading.find_element_by_tag_name('a')
driver.execute_script("arguments[0].click();", clickable)
# the code starts to fail around here
try:
body_info = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.CLASS_NAME, 'field-item even')))
main_text = []
for j in range(3):
j+=1
text = body_info.find_element_by_tag_name('p')
main_text.append(text)
for t in main_text:
t_info = t.text
print(f'{heading.text}\n{t_info}')
except:
print("couldn't find tag")
driver.back()

There's an issue with your code, (By.CLASS_NAME, 'field-item even').
Selenium does not have support for multiple classes or classes with space.
Simply replace space with . and that would be the CSS_SELECTOR
Try something like this:
try:
body_info = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.CSS_SELECTOR, '.field-item.even')))

It must be '.field-item even' and not 'field-item even' if you are using By.CSS_SELECTOR for presence_of_element_located().
So Replace,
body_info = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.CLASS_NAME, 'field-item even')))
with,
body_info = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.CLASS_NAME, 'field-item even')))
Official docs. https://selenium-python.readthedocs.io/api.html#locate-elements-by

To select multiple classes, you must use the class selector. You can't select multiple classes through a space, like in CSS. You need to select multiple classes using the class selector. So you must put a dot before all of the classes and not give any space between them

XPath Check if ID exists within drop down. Class check

thanks in advance. Noobie to Python here and I am trying to automate value entries into a website via Selenium and the respective XPath values.
How it is supposed to function is that I send keys of the dynamic 'ID' into the input box and the ID will pop up and I select the ID. This works. Right now I am running into the issue where the ID does not exist and the tool ends up stalling out. I know I need an If function, then execute an elif if it does not exist but I am lost when it comes to these kind of statements with XPaths and need a little bit of guidance.
I have the class XPath of the pop up value stating the ID does not exist:
<li class="vv_list_no_items vv_item_indent listNoItems">No results match "1234567890123456"</li>
The confusing part is also having dynamic IDs where "1234567890123456" can be any ID.
Current code is below, sorry for the indenting as this was grabbed out of a larger set of scripts.
try:
wait = WebDriverWait(browser, 10)
# Inputs Legal Entity
elem = wait.until(EC.element_to_be_clickable((By.XPATH,
"//*[#id='di3Form']/div[2]/div[2]/div/div[1]/div[3]/div/div[2]/div/div[1]/input"))).send_keys(
LE)
elem = wait.until(
EC.element_to_be_clickable((By.XPATH, "//*[#id='veevaBasePage']/ul[3]/li/a"))).click()
LE = None
# Inputs WWID
elem = wait.until(EC.element_to_be_clickable((By.XPATH,
"//*[#id='di3Form']/div[2]/div[2]/div/div[1]/div[4]/div/div[2]/div/div[1]/input"))).send_keys(ID)
elem = wait.until(
EC.element_to_be_clickable((By.XPATH, "//*[#id='veevaBasePage']/ul[4]/li[2]/a/em"))).click()
# Inputs Country
elem = wait.until(EC.element_to_be_clickable((By.XPATH,
"//*[#id='di3Form']/div[2]/div[2]/div/div[1]/div[5]/div/div[2]/div/div[1]/input"))).send_keys(
Country)
elem = wait.until(
EC.element_to_be_clickable((By.XPATH, "//*[#id='veevaBasePage']/ul[5]/li/a"))).click()
# Save
elem = wait.until(EC.element_to_be_clickable((By.XPATH,
"//a[#class='docInfoSaveButton save vv_button vv_primary']/span[#class='vv_button_text vv_ellipsis' and text()='Save']")))
browser.execute_script("arguments[0].click();", elem)
wait = WebDriverWait(browser, 15)
# Click dropdown menu arrow
elem = wait.until(EC.element_to_be_clickable(
(By.XPATH, "//*[#id='di3Header']/div[3]/div/div[2]/div[1]/div/div[2]/div/div/button")))
browser.execute_script("arguments[0].click();", elem)
wait = WebDriverWait(browser, 100)
# Click "Publish"
elem = wait.until(EC.element_to_be_clickable((By.XPATH,
"/html/body/div[6]/div/ul/li")))
browser.execute_script("arguments[0].click();", elem)
#Confirm Publish
elem = wait.until(EC.element_to_be_clickable((By.XPATH,
"//a[#class='save vv_button vv_primary']/span[#class='vv_button_text' and text()='Yes']")))
browser.execute_script("arguments[0].click();", elem)

You can use xpath contains, with find_elements that returns a list and have a if condition that if it is >0, then No match found string would be present in UI.
try :
no_match = "No results match" + " " + '"' + WWID + '"'
if (len(driver.find_elements(By.XPATH, "//li[contains(text(),'{}')]".format(no_match)))) > 0:
print("Code has found No results match, String ")
# do what ever you wanna do here.
else:
print("There is not locator that contains, No results match")
except:
print("Something went wrong")
pass

Select an element of a drop down -selenium -python

I try of select the sport 'Football' in a drop down of sport but impossible of click on it.
I tried with the Select() method:
driver = webdriver.Chrome()
url = "https://www.flashscore.com/"
driver.get(url)
Team = 'Paris SG'
Type = 'Teams'
sport = 'Football'
buttonSearch = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, ".header__button--search"))).click()
fill_search_bar = driver.find_element(By.CSS_SELECTOR, ".input___1NGxU3-")
fill_search_bar.clear()
fill_search_bar.send_keys(Team)
driver.find_element(By.CSS_SELECTOR, ".dropDown").click()
select_sport = Select(driver.find_element(By.XPATH,"//div[contains(#class, 'dropDown__list')]"))
select_sport.select_by_visible_text(sport)
This code return this error : UnexpectedTagNameException: Message: Select only works on <select> elements, not on <div>.
Here is my second version:
fill_search_bar = driver.find_element(By.CSS_SELECTOR, ".input___1NGxU3-")
fill_search_bar.clear()
fill_search_bar.send_keys(Team)
driver.find_element(By.CSS_SELECTOR, ".dropDown").click()
select_sport = WebDriverWait(driver, timeout=10).until(EC.element_to_be_clickable((By.XPATH,"//[#class='dropDown__list']/[contains(text(),'"+ sport +"')]"))).click()
This code return this error : selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//div[#class='dropDown__list']/div[contains(text(),'Football')]"}.
How can I solve this problem ?

I would suggest to break down the wait until class into two lines for simplicity. Its totally optional and wouldn't make much of a difference.
wait = WebDriverWait(driver, 300)
wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, ".header__button--search")))
element_to_be_clicked=driver.find_element_by_css_selector(".header__button--search")
element_to_be_clicked.click()
For the second part try using the values of the options in drop down list:
fill_search_bar.clear()
fill_search_bar.send_keys(Team)
driver.find_element_by_xpath("//div[#class='dropDown__selectedValue dropDownValueSelected___3msxRQS']").click()
select_sport=Select(driver.find_element_by_class("dropDown__list dropDownList___3V-ppVu"))
select_sport.select_by_value('1') #football has value 1 in the list

Next Page Iteration in Selenium/BeautfulSoup for Scraping E-Commerce Website

I'm scraping an E-Commerce website, Lazada using Selenium and bs4, I manage to scrape on the 1st page but I unable to iterate to the next page. What I'm tyring to achieve is to scrape the whole pages based on the categories I've selected.
Here what I've tried :
# Run the argument with incognito
option = webdriver.ChromeOptions()
option.add_argument(' — incognito')
driver = webdriver.Chrome(executable_path='chromedriver', chrome_options=option)
driver.get('https://www.lazada.com.my/')
driver.maximize_window()
# Select category item #
element = driver.find_elements_by_class_name('card-categories-li-content')[0]
webdriver.ActionChains(driver).move_to_element(element).click(element).perform()
t = 10
try:
WebDriverWait(driver,t).until(EC.visibility_of_element_located((By.ID,"a2o4k.searchlistcategory.0.i0.460b6883jV3Y0q")))
except TimeoutException:
print('Page Refresh!')
driver.refresh()
element = driver.find_elements_by_class_name('card-categories-li-content')[0]
webdriver.ActionChains(driver).move_to_element(element).click(element).perform()
print('Page Load!')
#Soup and select element
def getData(np):
soup = bs(driver.page_source, "lxml")
product_containers = soup.findAll("div", class_='c2prKC')
for p in product_containers:
title = (p.find(class_='c16H9d').text)#title
selling_price = (p.find(class_='c13VH6').text)#selling price
try:
original_price=(p.find("del", class_='c13VH6').text)#original price
except:
original_price = "-1"
if p.find("i", class_='ic-dynamic-badge ic-dynamic-badge-freeShipping ic-dynamic-group-2'):
freeShipping = 1
else:
freeShipping = 0
try:
discount = (p.find("span", class_='c1hkC1').text)
except:
discount ="-1"
if p.find(("div", {'class':['c16H9d']})):
url = "https:"+(p.find("a").get("href"))
else:
url = "-1"
nextpage_elements = driver.find_elements_by_class_name('ant-pagination-next')[0]
np=webdriver.ActionChains(driver).move_to_element(nextpage_elements).click(nextpage_elements).perform()
print("- -"*30)
toSave = [title,selling_price,original_price,freeShipping,discount,url]
print(toSave)
writerows(toSave,filename)
getData(np)

The problem might be that the driver is trying to click the button before the element is even loaded correctly.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome(PATH, chrome_options=option)
# use this code after driver initialization
# this is make the driver wait 5 seconds for the page to load.
driver.implicitly_wait(5)
url = "https://www.lazada.com.ph/catalog/?q=phone&_keyori=ss&from=input&spm=a2o4l.home.search.go.239e359dTYxZXo"
driver.get(url)
next_page_path = "//ul[#class='ant-pagination ']//li[#class=' ant-pagination-next']"
# the following code will wait 5 seconds for
# element to become clickable
# and then try clicking the element.
try:
next_page = WebDriverWait(driver, 5).until(
EC.element_to_be_clickable((By.XPATH, next_page_path)))
next_page.click()
except Exception as e:
print(e)
EDIT 1
Changed the code to make the driver wait for the element to become clickable. You can add this code inside a while loop for iterating multiple times and break the loop if the button is not found and is not clickable.