I've written the following code to search the UNSPSC search page. This script works, but I'm having difficulty extracting the actual results. Adding to the code below, it should then extract "Toy balloons or balls"
from selenium import webdriver
# from selenium.common import exceptions
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
from selenium.webdriver.common.keys import Keys
opts = Options()
#opts.add_argument("--headless")
driver = webdriver.Firefox(
firefox_options=opts,
firefox_binary='path/Firefox/firefox.exe',
executable_path='./geckodriver.exe',
)
driver.get("https://www.unspsc.org/search-code/")
search = driver.find_element_by_id("dnn_ctr1535_UNSPSCSearch_txtsearchCode")
search.send_keys("60141001")
search.send_keys(Keys.ENTER)
To Extract the text induce WebDriverWait and wait for visibility_of_element_located() and use the following xpath
driver.get("https://www.unspsc.org/search-code/")
search = driver.find_element_by_id("dnn_ctr1535_UNSPSCSearch_txtsearchCode")
search.send_keys("60141001")
search.send_keys(Keys.ENTER)
print(WebDriverWait(driver,20).until(EC.visibility_of_element_located((By.XPATH,"//table[#id='dnn_ctr1535_UNSPSCSearch_gvDetailsSearchView']/tbody/tr[2]/td[2]"))).text)
You need to import following libraries.
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
Related
I am struggling with Selenium
for the url: https://pubchem.ncbi.nlm.nih.gov/compound/2078
I am trying to click the button Download, but it doesn't find the element.
My code:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.chrome.options import Options
from ipykernel import kernelapp as app
import time
options = webdriver.ChromeOptions()
driver_path = 'C:\\Users\\Idener\\Downloads\\chromedriver_win32\\chromedriver.exe'
driver = webdriver.Chrome(driver_path, options=options)
url = f"https://pubchem.ncbi.nlm.nih.gov/compound/2078"
driver.get(url)
driver.find_element_by_xpath("//*[#id='"'page-download-btn'"']").click()
enter image description here
Your XPath is not valid. You don't need so much quotes
driver.find_element_by_xpath("//*[#id='page-download-btn']").click()
You are missing a delay.
Element should clicked only when it is completely rendered and ready to accept a click event. WebDriverWait expected_conditions explicit waits should be used for that.
Also, no need to add f before URL value and '"' instead of ' in XPath expression.
The following code will work for you:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.chrome.options import Options
from ipykernel import kernelapp as app
import time
options = webdriver.ChromeOptions()
driver_path = 'C:\\Users\\Idener\\Downloads\\chromedriver_win32\\chromedriver.exe'
driver = webdriver.Chrome(driver_path, options=options)
url = "https://pubchem.ncbi.nlm.nih.gov/compound/2078"
driver.get(url)
wait = WebDriverWait(driver, 10)
wait.until(EC.element_to_be_clickable((By.ID, "page-download-btn"))).click()
Code trials:
from selenium import webdriver
from selenium.webdriver import Keys, ActionChains
from selenium.webdriver.chrome.service import Service
from time import time
from selenium.webdriver.common.actions.action_builder import ActionBuilder
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
service = Service(executable_path="C:\\Users\\aps\\Desktop\\Python\\chromedriver.exe")
driver = webdriver.Chrome(service=service)
driver.get("https://youtube.com")
search = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, "search")))
ActionChains(driver).move_to_element(search).pause(2).click_and_hold().send_keys("Iktarfa").perform()
button = driver.find_element(By.ID, "search-icon-legacy").click()
Search field is getting fetched.
But after that I am getting following error, I am out of ideas and a new learner. Please HELP!!
The locator strategy which you have used:
(By.ID, "search")
identifies multiple elements within the HTML DOM.
and the first WebElement having the property style="display: none;". Hence you see the error.
Solution
To send a character sequence to the element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "ytd-searchbox#search"))).send_keys("text")
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//ytd-searchbox[#id='search']"))).send_keys("text")
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
There is no need to add a Actions class for this scenario. It is perfectly capable of doing the search after just using the click and then searching for the element.
I used this code piece and it works perfectly fine
import time
from seleniumwire import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
svc = Service(ChromeDriverManager().install())
options = Options()
options.add_argument('--allow-running-insecure-content')
options.add_argument('--ignore-certificate-errors')
driver = webdriver.Chrome(service=svc,options=options)
driver.maximize_window()
driver.get("https://youtube.com")
search = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.ID, "search")))
driver.find_element(By.NAME,'search_query').click()
time.sleep(5)
driver.find_element(By.NAME,'search_query').send_keys('Iktarfa')
button = driver.find_element(By.ID, "search-icon-legacy").click()
time.sleep(5)
driver.quit()
I've used the sleep method, but you should in general use an intelligent wait mechanism like Explicit waits.
Output screenshot -
I am trying to access the following website:
and apply some filters like click on and select a date. When I click on I want to select one of the two options. But I cant click with xpath neither can I send keys to to type in what I want. Can someone help me by finding how I can click on and right after that select one of the two options and click the green button so that I can click afterwards on the date?
here is what I got so far in code (Python)
%pip install selenium webdriver_manager
import requests
from datetime import datetime
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support.ui import WebDriverWait
driver = webdriver.Chrome()
driver.get(url_dist_vacinas)
print(driver.title)
driver.find_element(By.XPATH,'//*[#id="filtro-04"]/div/article/div[1]/div/div/qv-
filterpane/div/div/div/div[2]/span').click()
Try the below lines of code, this might help
driver.get("https://infoms.saude.gov.br/extensions/DEMAS_C19VAC_Distr/DEMAS_C19VAC_Distr.html")
clickReviw = WebDriverWait(driver, 40).until(EC.element_to_be_clickable((By.XPATH, "//*[text()='Tipo de vacina']")))
clickReviw.click()
sleep(4)
driver.find_element_by_xpath("//input[#placeholder='Search in listbox']").send_keys("vacina")
btn1 = driver.find_element_by_xpath("(//div[#class='qv-listbox-text qv-listbox-text-value'])[1]")
btn1.click()
btn2 = driver.find_element_by_xpath("//button[#title='Confirm selection']")
btn2.click()
Imports
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from time import sleep
import selenium
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.remote import webelement
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support.ui import Select
from selenium.common.exceptions import TimeoutException, NoSuchElementException
import time
url = "https://www.mrporter.com/en-gb/mens/product/nike/shoes/low-top-sneakers/space-hippie-04-recycled-stretch-knit-sneakers/19971654707345242"
PATH = 'C:\Program Files (x86)\chromedriver.exe'
browser = webdriver.Chrome(PATH)
browser.get(url)
element_dropdown = browser.find_element_by_class_name("CombinedSelect11__field CombinedSelect11__field--selectableOption CombinedSelect11__field--nativeSelect")
select = Select(element_dropdown)
try:
select.select_by_visible_text("8")
except NoSuchElementException:
print("the item doesnt exist")
I am trying to locate the dropdown menu of the link in my code. Once the dropdown box is located I want to search by visible text for a size 8. However whatever I try it still doesn't work.
You can try using explicit wait and then perform your operation. Please take a look at the below code which I have written to replicate your scenario. It's working fine for me. Do let me know if you face any problems.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
options = webdriver.ChromeOptions()
options.add_argument("start-maximized")
driver = webdriver.Chrome(options=options)
driver.get(
'https://www.mrporter.com/en-gb/mens/product/nike/shoes/low-top-sneakers/space-hippie-04-recycled-stretch-knit-'
'sneakers/19971654707345242')
wait = WebDriverWait(driver, 30)
wait.until(EC.visibility_of_element_located((By.XPATH, '//div[text()="Select a size"]'))).click()
wait.until(EC.visibility_of_element_located((By.XPATH, '//li[#data-value="8"]'))).click()
I am trying to scrape this website
https://script.google.com/a/macros/cprindia.org/s/AKfycbzgfCVNciFRpcpo8P7joP1wTeymj9haAQnNEkNJJ2fQ4FBXEco/exec
I am using selenium and python.I am not able to view entire page source,Basically i have to scrape the table inside it and click on next button,but the code of next and table not visible on page source.Here is my code
from selenium import webdriver
from selenium.webdriver.support.ui import Select
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
from bs4 import BeautifulSoup
from selenium import webdriver
browser = webdriver.PhantomJS()
browser.get(link)
pass1 = browser.find_element_by_xpath("/html/body/div[2]/table[2]/tbody/tr[1]/td/div/div/div[2]/div[2]")
pass1.click()
time.sleep(30)
I am getting this error,NoSuchElementException.
There are two iframes present on the page, so you need to first switch on those iframe and then you need to click on the element.
And you can apply explicit wait on the element so that the script waits until the element is visible on the page.
You can do it like:
browser = webdriver.PhantomJS()
browser.get(link)
browser.switch_to.frame(driver.find_element_by_id('sandboxFrame'))
browser.switch_to.frame(driver.find_element_by_id('userHtmlFrame'))
WebDriverWait(browser, 20).until(EC.presence_of_element_located((By.XPATH, "//div[contains(#class,'charts-custom-button-collapse-left')]//div"))).click()
Note: You have to add the following imports:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC