Web-scraping with selenium python - python

I'm trying to scrape a page whose classes are in this format class="jss262 jss434 jss263"
I tried to click on a box and insert a specific value but whether I use class, xpath or css selector, the element is not identified.
I believe I missed something but I'm not sure what it is. Is there a way to modify my code in order to access this element?
My code:
try:
button2 = WebDriverWait(driver, 10).until(ec.presence_of_element_located((By.CLASS_NAME, 'jss262 jss434 jss263')))
button2.click()
button2.send_keys(att)
driver.implicitly_wait(5)

By.CLASS_NAME only accepts one class, use css_selector:
button2 = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CSS_SELECTOR, '.jss262.jss434.jss263')))

Related

Change to iframe

On gmx.net I want to click on a hyperlink in an email.
But the hyperlink is enclosed with two iframes.
How do I change into it?
My approach:
WebDriverWait(driver, 10).until(expected_conditions.frame_to_be_available_and_switch_to_it((By.XPATH, '//iframe[#class="app-stack_children--active"]')))
WebDriverWait(driver, 10).until(expected_conditions.frame_to_be_available_and_switch_to_it((By.XPATH, '//iframe[#class="js-box-flex need-overlay"]')))
As I suspected, your XPath is incorrect.
You have to either use
WebDriverWait(driver, 10).until(expected_conditions.frame_to_be_available_and_switch_to_it((By.XPATH, '//iframe[#class='app-stack__children l-vertical app-stack__children--active']')))
or
WebDriverWait(driver, 10).until(expected_conditions.frame_to_be_available_and_switch_to_it((By.XPATH, '//iframe[contains(#class, 'app-stack_children--active']')))
Obviously, the first option is better than the second one. There's always a chance that contains() will match more than one element. It is less probable if using equality.

Scroll the exact element into view_selenium python

I use a python script to perform some actions on a website. One of the actions is to insert value and select the exact value from the list. The issue with the code below is that is selecting the first value that appears and not the exact value.
For example, I'm searching for 'algodón' and it should select the exact value available in the dropdown as 'Algodón' and not 'Mezcla de algodón'.
my code
button = WebDriverWait(driver, 6).until(ec.element_to_be_clickable((By.CLASS_NAME,'awsui-select-dropdown-filter')))
driver.implicitly_wait(10)
ActionChains(driver).move_to_element(button).click(button).perform()
ActionChains(driver).move_to_element(button).send_keys(a).perform()
# ------------------click checkbox
element = driver.find_element_by_class_name('awsui-select-option-checkbox')
actions = ActionChains(driver)
actions.move_to_element(element).perform()
driver.execute_script("arguments[0].scrollIntoView(true);", element)
element.click()
bulk = WebDriverWait(driver, 5).until(ec.presence_of_element_located((By.XPATH,'//*[#id="awsui-tabs-0-FILTERED_QUESTIONS_TAB-panel"]/span/div/awsui-cards/div/div[2]/div[1]/span/span/div[1]/div/div/div[3]/awsui-button')))
ActionChains(driver).move_to_element(bulk).click(bulk).perform()
How this appears in the dropdown:
What should select:
Do you have any suggestion on how should I modify the code to select the exact match?

Not able to select and click dropdown search query result with Selenium webdriver

Im trying to perform select and click action from the search box result dropdown for testing purpose. Though i dont get ant error but i'm stuck and not able to do so, search results came then disappeared immediately. Please any one help me out. Im using Python script to automate webdriver. Here is the screenshot below for reference.
. I have tried webdriverwait for same action but it gives Timeout exception error. If there is any child actions from CSS to perform let me know. Here is what i tried
search = driver.find_element_by_id('searchInput')
search.send_keys("flowers")
dropdown = WebDriverWait(driver, 4).until(
EC.presence_of_element_located((By.XPATH, "//li[text()='flowers']")))
Apart from this snippet, i want to rather just perform enter key operation, where i get query result for 'flower' on this ecomm. website.
Here is the website URL- https://paytmmall.com
Once you type flower in the input field, there are multiple options appearing based on the input provided. They are in li tags and under b tag.
Code :
driver = webdriver.Chrome(driver_path)
driver.maximize_window()
#driver.implicitly_wait(30)
wait = WebDriverWait(driver, 30)
driver.get("https://paytmmall.com/")
search = wait.until(EC.visibility_of_element_located((By.ID, "searchInput")))
search.send_keys("flowers")
time.sleep(3)
wait.until(EC.visibility_of_element_located((By.XPATH, "(//li)[4]/descendant::b[contains(text(),'flowers')]"))).click()
time.sleep is just for visibility purpose. you can remove that as well.
Also this xpath (//li)[4]/descendant::b[contains(text(),'flowers')] is based on xpath indexing , since I think you wanna select the 4th option which is flower itself. In case you wanna select a different option, you would have to write the different xpath.
In case you are looking to just select the searched item, it's better to pass enter key once you type flower in the input field.
You can use the below code for that :
search = wait.until(EC.visibility_of_element_located((By.ID, "searchInput")))
search.send_keys("flowers")
time.sleep(3)
search.send_keys(Keys.RETURN)
The suggested options are not containing the text directly in li elements, they are inside child elements inside li elements.
Try this instead:
search = driver.find_element_by_id('searchInput')
search.send_keys("flowers")
dropdown = WebDriverWait(driver, 4).until(
EC.visibility_of_element_located((By.XPATH, "//li//*[text()='flowers']")))

Selenium - Visibility of element in HTML page (Python)

I'm trying to scrape the following URL: https://www.vivareal.com.br/venda/sp/holambra/imovel-comercial_comercial/. It has two pages only. However, there is a button in the bottom which I can locate via XPATH: "//*[contains(#title, "Página anterior")]".
I'm trying to set a variable for when the button is not clickable, like:
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[contains(#title, "Página anterior")]')))
But it returns an element, even if the element is not clickable (I'm testing manually too). Is there any sugestions? Thanks!
When the button is not clickable, it has an attribute called data-disabled; otherwise that attribute is missing. The attribute has no value, but if you use the following CSS Selector it should grab that element only if there is no data-disabled attribute:
driver.find_element_by_css_selector('a[title="Página anterior"]:not([data-disabled]')

Selenium Python selecting dropdown element

I'm having a lot of trouble finding the web element corresponding to the drop down box directly to the right of Code:. I've tried using Select, finding by tag_name, css_selector, xpath, name, class_name, select_by_visible_text, and haven't been able get it. I'm guessing the issue is that the dropdown box is nested within the form tag, highlighted in blue in the picture. The closest I've gotten is this:
element = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div#code_section_srch form#expertsearchformid select#expertsearchformid:toctextcodeid")))
Can you help me understand what I'm doing wrong?
Any insight would be greatly appreciated!
You have to use quotas, escape colon or use find by ID:
div#code_section_srch form#expertsearchformid select#expertsearchformid\:toctextcodeid
div#code_section_srch form#expertsearchformid select[id='expertsearchformid:toctextcodeid']
This should work:
element = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.ID, "expertsearchformid:toctextcodeid")))
element = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "[id='expertsearchformid:toctextcodeid']")))

Categories