I am trying to scrape for car prices from this website:
To get car prices, you should fill out the form and I have to choose from dropdowns using Selenium.
I am using this code to choose from dropdowns:
# Imports
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.support.ui import Select
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
year_dropdown = Select(WebDriverWait(driver, 5)
.until(EC.element_to_be_clickable((By.ID, "j_id_3q-carInfoForm-year-selectOneMenu"))))
year_dropdown.select_by_value('2015')
But after I chose the year, it just keeps loading and never stops:
Any suggestions please?
I resolved the issue by using a real chrome driver. I was using chromdriver-manager package and when I removed it and downloaded a real chrome driver, the issue was gone.
Related
I am trying to access the following website:
and apply some filters like click on and select a date. When I click on I want to select one of the two options. But I cant click with xpath neither can I send keys to to type in what I want. Can someone help me by finding how I can click on and right after that select one of the two options and click the green button so that I can click afterwards on the date?
here is what I got so far in code (Python)
%pip install selenium webdriver_manager
import requests
from datetime import datetime
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support.ui import WebDriverWait
driver = webdriver.Chrome()
driver.get(url_dist_vacinas)
print(driver.title)
driver.find_element(By.XPATH,'//*[#id="filtro-04"]/div/article/div[1]/div/div/qv-
filterpane/div/div/div/div[2]/span').click()
Try the below lines of code, this might help
driver.get("https://infoms.saude.gov.br/extensions/DEMAS_C19VAC_Distr/DEMAS_C19VAC_Distr.html")
clickReviw = WebDriverWait(driver, 40).until(EC.element_to_be_clickable((By.XPATH, "//*[text()='Tipo de vacina']")))
clickReviw.click()
sleep(4)
driver.find_element_by_xpath("//input[#placeholder='Search in listbox']").send_keys("vacina")
btn1 = driver.find_element_by_xpath("(//div[#class='qv-listbox-text qv-listbox-text-value'])[1]")
btn1.click()
btn2 = driver.find_element_by_xpath("//button[#title='Confirm selection']")
btn2.click()
Imports
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from time import sleep
I'm trying to interact with the field of Klarna Checkout on a website, but I'm not able to change the selected radio buttons nor enter information in the credit card fields. I'm using VS code, Python and selenium webdriver.
The website destination is: https://voltfashion.com/no/functional/kassen/
You need to add an item in order to see the "Klarna checkout section".
It is a Norwegian website.
I have tried some different coding solutions, but none of them worked. I didn't have any problems interacting with the other elements of the website, just the Klarna checkout section. Any suggestions?
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
Some of the code I've tried:
cardForm = driver.find_element_by_id("cardNumber")
cardForm.send_keys('1234567890987654')
and
inputCC = WebDriverWait(driver, 5).until\
(lambda driver: driver.find_element_by_xpath\
("//input[#id='cardNumber']")
)
inputCC.send_keys("1234567890987654")
Klarna screenshot (Source):
I want to create a GUI for this code that automatically downloads PDF's. But I am not sure where to start. These are 2 inputs I need to change.
drp.select_by_visible_text('**Dan Pitts**')
checkfield=wait.until(EC.element_to_be_clickable((By.XPATH,'//*[#id="FieldCheckBox-**701847**"]')))
So just 2 input boxes and then have a submit button for it to open chrome and start.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import Select
from selenium.webdriver.support.ui import WebDriverWait
import time
driver=webdriver.Chrome()
driver.get('')
username=driver.find_element_by_xpath('//*[#id="username"]')
username.send_keys('')
password=driver.find_element_by_xpath('//*[#id="password"]')
password.send_keys('')
login=driver.find_element_by_xpath('/html/body/pcs-root/pcs-site-wrapper/div/div/div/div/pcs- login/div/form/button')
login.click()
time.sleep(10)
driver.switch_to.frame('pcsIFrame')
growerlist=driver.find_element_by_xpath('//*[#id="GrowerDropDownList"]')
drp=Select(growerlist)
drp.select_by_visible_text('**Dan Pitts**')
wait=WebDriverWait(driver,30)
getfields=driver.find_element_by_xpath('//*[#id="GetFieldsButton"]')
getfields.click()
##time.sleep(4)
checkfield=wait.until(EC.element_to_be_clickable((By.XPATH,'//*[#id="FieldCheckBox-**701847**"]')))
checkfield.click()
I have put the two input boxes in and it does work for the Grower. I cant seem to get it to work with xpath though.
I am trying to scrape this website
https://script.google.com/a/macros/cprindia.org/s/AKfycbzgfCVNciFRpcpo8P7joP1wTeymj9haAQnNEkNJJ2fQ4FBXEco/exec
I am using selenium and python.I am not able to view entire page source,Basically i have to scrape the table inside it and click on next button,but the code of next and table not visible on page source.Here is my code
from selenium import webdriver
from selenium.webdriver.support.ui import Select
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
from bs4 import BeautifulSoup
from selenium import webdriver
browser = webdriver.PhantomJS()
browser.get(link)
pass1 = browser.find_element_by_xpath("/html/body/div[2]/table[2]/tbody/tr[1]/td/div/div/div[2]/div[2]")
pass1.click()
time.sleep(30)
I am getting this error,NoSuchElementException.
There are two iframes present on the page, so you need to first switch on those iframe and then you need to click on the element.
And you can apply explicit wait on the element so that the script waits until the element is visible on the page.
You can do it like:
browser = webdriver.PhantomJS()
browser.get(link)
browser.switch_to.frame(driver.find_element_by_id('sandboxFrame'))
browser.switch_to.frame(driver.find_element_by_id('userHtmlFrame'))
WebDriverWait(browser, 20).until(EC.presence_of_element_located((By.XPATH, "//div[contains(#class,'charts-custom-button-collapse-left')]//div"))).click()
Note: You have to add the following imports:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
I'm trying out the selenium framework to learn something about the web browser automation. Therefore, I decided to build an Audi model...
My code so far:
from selenium import webdriver
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
chrome_driver = webdriver.Chrome(executable_path=r"chromedriver_win32\chromedriver.exe")
chrome_driver.get("https://www.audi.de/de/brand/de/neuwagen.html")
# choose A3 model
chrome_driver.find_element_by_xpath('//*[#id="list"]/div[1]/ul/li[2]/a[2]').click()
# choose sedan version
WebDriverWait(chrome_driver, 3).until(EC.visibility_of_element_located((By.XPATH, '//*[#id="a3limo"]/div/a'))).click()
# start model configuration
WebDriverWait(chrome_driver, 3).until(EC.visibility_of_element_located((By.XPATH, '/html/body/div[1]/div[2]/div/div[6]/div[2]/div/div[1]/ul/li[1]/a'))).click()
# choose the s-line competition package
WebDriverWait(chrome_driver, 3).until(EC.visibility_of_element_located((By.XPATH, '/html/body/div[1]/div[2]/div/div[7]/div[2]/div[2]/div[3]/div[1]/div/div[1]/div/div[1]/span/span'))).click()
# accept the s-line competition package
WebDriverWait(chrome_driver, 3).until(EC.visibility_of_element_located((By.XPATH, '/html/body/div[5]/div/div/div/div/div/ul[2]/li[2]/a'))).click()
Now, the code fail already on the line start model configuration (see "Konfiguration starten" button on webpage https://www.audi.de/de/brand/de/neuwagen/a3/a3-limousine-2019.html). The xPath must be correct and the element should also be visible, so what am I doing wrong here?
Actually required link becomes visible only if to scroll page down.
Try to use this piece of code to click link:
configuration_start = chrome_driver.find_element_by_xpath('//a[#title="Konfiguration starten"]')
chrome_driver.execute_script('arguments[0].scrollIntoView();', configuration_start)
configuration_start.click()
As navigation panel has fixed position it might overlap target link, so you can change navigation panel style before handling link:
nav_panel = chrome_driver.find_element_by_xpath('//div[#data-module="main-navigation"]')
driver.execute_script('arguments[0].style.position = "absolute";', nav_panel)
The following seems to work for me. I have added waits for elements and used a simulated click of the element via javascript.
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
url = 'https://www.audi.de/de/brand/de/neuwagen.html'
d = webdriver.Chrome()
d.get(url)
WebDriverWait(d,10).until(EC.presence_of_element_located((By.CSS_SELECTOR, '[data-filterval="a3"]'))).click()
d.get(WebDriverWait(d,10).until(EC.presence_of_element_located((By.CSS_SELECTOR, '#a3limo .mf-model-details a'))).get_attribute('href'))
element = WebDriverWait(d,10).until(EC.presence_of_element_located((By.CSS_SELECTOR, '[title="Konfiguration starten"]')))
d.execute_script("arguments[0].click();", element)