I have been messing around with python and selenium and was trying to make a program that could automatically checkout on bestbuy.ca . I have gotten to nearly the last stage of the checkout process but am having difficulty getting the element/clicking the buttons to continue. The only button I have been able to get to work is the add to cart button on the product page.
Here is the site as well as the HTML:
Click here for image
Some things I have tried with no success include:
driver.find_element_by_class_name().click()
For this, I used the class in button and span. I have tried using each individual class in separately and together with little success. It would usually give me a "No Element Found" error or just not click anything afterwards.
driver.driver.find_element_by_xpath().click()
For this, I tried using the element locator extension for the xpath as well as "//*[#type='submit']" with no success.
driver.driver.find_element_by_css_selector().click()
For this, I tried the element locator extension again as well as copying it from the firefox inspect with no success.
Not really sure if there are any other ways to go about doing this but any help would be appreciated.
Did you try to copy the entire class attribute from the html and use :
wait.until(EC.presence_of_element_located((By.XPATH, "//button[#class='asd asd sad asd']"))).click()
copy the class and paste it.
Try either of the following to click on the button tag.
wait=WebDriverWait(driver, 10)
wait.until(EC.element_to_be_clickable((By.XPATH, "//span[text()='Continue']/parent::button"))).click()
Or by attribute data automation
wait.until(EC.element_to_be_clickable((By.XPATH, "//button[#data-automation='review-order']"))).click()
Import
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
Related
The code I want to extract the "ul" element from
After trying suggestion, I want to get "title" in the follwing "li" tag
I'm trying to extract the following "ul" element from a webpage using selenium. Using Python, I can't figure out what the X_PATH should be, I've tried everything I could think of. Also tried the css_selection. I'm getting back nothing.
I want to iterate over the "li" elements within that specific "ul" element. If anybody could help it would be appreciated, I've literally tried everything I can think/search.
//ul[#aria-label='Chat content']/li[1]/div[1]/div[1]
To me it looks like you could just grab the ul by it's aria label and then select the first li from there. You could also just inspect the element and copy the xpath from the developers console.
It also seems to be in an iframe.
wait = WebDriverWait(driver, 30)
wait.until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, "//iframe[#class='embedded-electron-webview embedded-page-content']")))
Imports:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
Check is that element are inside some iFrame, maybe is for that, if that dont work install selenium extension in your browser and try to recreat the flow, you will receive all xpath, is a way that I use when nothing is working to me.
Using selenium and python, I'm opening a document by making click on a link which opens in the same tab. When I make a screenshot without using '--headless' it takes the document screenshot, but when I activate '--headless' the screenshot is from the previous page (the one where I make click to the document to be opened)
I have tried different browsers and ways (opening in new tab and switching but not working when using '--headless') but not working...
Any idea why '--headless' is behaving like this?
While taking a screenshot ideally you need to induce WebDriverWait for the visibility_of_element_located() of some static and visible element e.g. some <h1> / <h2> element on the desired page and then take the screenshot as follows:
WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "css_visible_element")))
driver.save_screenshot("Federico Albrieu.png")
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
I solved the problem.
I was using the action
ActionChains(driver).move_by_offset('xcoordinate', 'ycoordinate).click().perform()
to open the desired document. Apparently, while using headless, selenium can't make click in the document coordinate (?) but it can when not using headless.
I tried instead making click with element_to_be_clickable((By.XPATH, 'XPATH'))).click() and it works when using headless.
Thanks!
I'm trying to put together a small scraper for public trademark data. I have a database available that i'm using selenium and python to access.
I can do just about anything I need to be able to, but for some reason i can't actually click the "accept cookies" button on the website. The following code i use highlights the button, but it does not get rid of the popup.
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
driver = webdriver.Chrome(executable_path=DRIVER_PATH)
driver.get('https://data.inpi.fr/recherche_avancee/marques')
element = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.ID, "tarteaucitronPersonalize2"))
).click()
I have looked up similar threads on this forum, and I have tried multiple things :
- adding a waiting period, that ended up highlighting the button, so at least i know it does something
- using javascript code to do the actual click, did not work any better
- i tried calling the button via its ID, its XPATH, its CSS selector, anything i could find really
I even downloaded Selenium IDE to record my clicks to see exactly how I could replicate it, but it still only recorded a click.
I tried my best, does anyone know where my mistake lie ? I am open to using other languages, or another platform
Well it looks like I managed to solve it just minutes after posting my question !
For some reason you need to resize the window. I just added the following line of code after opening the URL and it worked first time.
driver.maximize_window()
I added this answer in case anyone stumbles upon this post and wants to avoid pulling their hair out over this !
Newbie here... 2 days into learning this.
In a learning management system, there is an element (a plus mark icon) to click which adds a form field upon each click. The goal is to click the icon, which generates a new field, and then put text into the new field. This field does NOT exist when the page loads... it's added dynamically based on the clicking of the icon.
When I try to use "driver.find_element_by_*" (have tried ID, Name and xpath), I get an error that it can't be found. I'm assuming it's because it wasn't there when the page loaded. Any way to resolve this?
By the way, I've been successful in scripting the login process and navigating through the site to get to this point. So, I have actually learned how to find other elements that are static.
Let me know if I need to provide more info or a better description.
Thanks,
Bill
Apparently I needed to have patience and let something catch up...
I added:
import time
and then:
time.sleep(3)
after the click on the icon to add the field. It's working!
You can use time.sleep(3) but that would force you to wait for the entire 3 seconds before using that element. In Selenium we use webdriver waits that polls the DOM to allow us to immediately use that element as quick as possible when it is useable.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome()
wait = WebDriverWait(driver, 10)
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR,""))).click()
First of all, I'm completely new in web scraping, html and selenium, so my question might not seem meaningful to you.
What I'm trying to do: automated click on an image on a webpage. Manual click on this image allows to display new information on a part of the webpage.
How I tried to do it: I found something specific to this image, for example:
<img src="images/btn_qmj.gif" border="0">
So I just entered in python:
xpath = '//img[#src="images/btn_qmj.gif"]'
driver.find_elements_by_xpath(xpath)
Problem: this returns me an empty list. I partially understood why, but I don't have any solution.
image of the html code inspector
I included here an image of the html tree I obtain on the web inspector. As one can see, the tree to reach my line of interest gives "/html/frameset/frame1/#document/html/body/center/table/tbody/tr[2]/td/a/img".
The issue is that I cannot access -by using an xpath- anything that comes after the #document. If I try to copy the xpath of my line of interest, it tracks it back only up to the 2nd "/html". It is like the page is subdivided, compartmentalized, with inner parts I cannot access. I suppose there is a way to access the #document content, maybe it refers to another webpage, but I'm stuck at that point.
I would be very grateful if anyone could help me on this.
Have you tried switching to the frame first?
driver.switch_to.frame('gauche')
You need to switch control to frame first to handle image on it. Please see below code for your reference
driver.switch_to_frame(frame)
WebDriverWait(driver, 20).until(
EC.visibility_of_element_located((By.XPATH, "'//img[#src="images/btn_qmj.gif"]'"))).click()
driver.switch_to_default_content()
Note :: You need to add below imports
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
The element is present inside an iframe.In order to access the element you need to switch it first.
Induce WebDriverWait() and frame_to_be_available_and_switch_to_it()
Induce WebDriverWait() and visibility_of_element_located()
WebDriverWait(driver,10).until(EC.frame_to_be_available_and_switch_to_it((By.NAME,"gauche")))
WebDriverWait(driver,10).until(EC.visibility_of_element_located((By.XPATH,"//img[#src='images/btn_qmj.gif']"))).click()
You need to import following libraries.
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By