I am trying to press a button with selenium because afterwards I need to inspect the full html of the website. This is the code that I am using:
driver = webdriver.Chrome()
driver.get('https://www.quattroruote.it/listino/audi/a4-allroad')
time.sleep(10)
html = driver.find_element_by_id('btnallestimenti')
html.click()
But I get this error:
selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable
when the page is open there are cookies and other things that shows up, is there a way to block all of them so that I can work on the html?
Thanks a lot!
As you can see the "cookies" banner is an HTML element itself and it contains a "Close" ("Chiudi") button that can be clicked.
If you inspect the page source you will find this code that relates to that button:
<button type="button" class="iubenda-cs-close-btn" tabindex="0" role="button" aria-pressed="false" style="font-size:16px!important;">Chiudi</button>
Your script needs to be modified to search the element by visible text (using the XPath) and click it to close the banner:
close_button = driver.find_element_by_xpath("//*[text()='Chiudi']")
close_button.click()
I can see that this kind of banner appears 2 times (one for cookies one for "Informativa") but once you click this one away you are redirected to the right page.
Of course you will need to test your script and adjust it to the page's behavior.
Also, be aware that every time the pages change because the devs change it your script will break and you will need to re-adjust it.
EDIT
Posting here the full code, try to use it and continue from here:
import time
from selenium.webdriver import Chrome
driver = Chrome()
driver.get("https://www.quattroruote.it/listino/audi/a4-allroad")
time.sleep(6)
driver.find_element_by_xpath("//button[text()='Accetta']").click()
time.sleep(6)
driver.switch_to.frame("promo-premium-iframe")
driver.find_element_by_xpath("//a[normalize-space()='Non sono interessato']").click()
time.sleep(6)
driver.switch_to.default_content()
driver.find_element_by_id("btnallestimenti").click()
input()
You could try to accept the cookie and proceed further, check the below lines of code.
options = Options()
options.add_argument("--disable-notifications")
driver = webdriver.Chrome(options=options,"ChromeDriver_Path")
driver.maximize_window()
driver.get('https://www.quattroruote.it/listino/audi/a4-allroad')
sleep(10)
cookie_btn = driver.find_element_by_xpath("//button[text()='Accetta']")
cookie_btn.click()
sleep(3)
WebDriverWait(driver,10).until(EC.frame_to_be_available_and_switch_to_it((By.ID,"promo-premium-iframe")))
register_btn = driver.find_element_by_xpath("//a[normalize-space()='Accedi o Registrati']")
register_btn.click()
Iframe available so just switched to the iframe, tried the to perform the registration.
Related
I'm stuck with a selenium scrape using jupyter. I'm trying to get the "download page data" from the bottom right corner of this page: https://polkadot.subscan.io/account?role=all&page=1
enter image description here
Also, here's the html code:
Download page data
I've tried copying the xpath and full xpath from the Google Chrome "inspect" tab, but it doesn't work.
Here's the code I used, but feel free to suggest anything else.
#Initiating Webdriver
s=Service('CHROMEDRIVER LOCATION')
op = webdriver.ChromeOptions()
driver = webdriver.Chrome(service=s, options=op)
link = "https://polkadot.subscan.io/account?role=all&page=1"
driver.get(link)
Ingresar = driver.find_element(By.XPATH,"//*[#id='app']/main/div/div/div[5]/div/div[3]/div[1]/div/div")
Here's the error I get:
ElementClickInterceptedException: Message: element click intercepted: Element <div data-v-24af4670="" class="label align-items-center">...</div> is not clickable at point (125, 721). Other element would receive the click: <div data-v-c018c6b4="" class="banner">...</div>
Either fixing my code, or getting a new one that works with jupyer and selenium
Try this code:
url = "https://polkadot.subscan.io/account?role=all&page=1"
driver.get(url)
driver.find_element(By.XPATH, ".//*[text()='I accept']").click()
time.sleep(5)
download_btn = driver.find_element(By.XPATH, ".//*[text()='Download page data']")
driver.execute_script("arguments[0].scrollIntoView(true);", download_btn)
download_btn.click()
I am currently dabbling in Python in combination with Selenium. Here I came across the topic of Java Script buttons. I am looking for the smartest way to find out how to execute a Java-Script code (in this case a button) the easiest way.
My goals:
Open web page http://www.easyCredit.de
Accept the Cockies there
Click the "Jetzt berechnen" button
Click the "Zu den finanziellen Angaben" button
Number 1 and 2, I got right with a lot of research. Now it's time to click the button. This can be identified quite well in the code, but still I do not succeed in clicking.
<input type="submit" class="calculator__submit-button" value="calculate now">
I have now tried it similar to the cockies, but unfortunately it does not work. I hope you can help me.
Here is my code:
from selenium import webdriver
from selenium.webdriver.common.by import By
import time
def expand_shadow_element(element):
shadow_root = browser.execute_script('return arguments[0].shadowRoot', element)
return shadow_root
# link to Chromedriver
browser = webdriver.Chrome('/usr/local/bin/chromedriver')
# maximize window
browser.maximize_window()
# URL
browser.get('https://www.easycredit.de/')
# wait
time.sleep(5)
# accept cookies
coockie_button = browser.execute_script("""return document.querySelector('#usercentrics-root').shadowRoot.querySelector("button[data-testid='uc-accept-all-button']")""")
coockie_button.click()
# wait
time.sleep(5)
# click "Jetzt berechnen"
berechnen_button = browser.execute_script("""return document.querySelector('rr-62f0ff69f069c').shadowRoot.querySelector("button[class='calculator__submit-button']")""")
berechnen_button.click()
The last part does not work.
This is the button I want to click:
The element Jetzt berechnen is within #shadow-root (open) and is an <input> element.
Solution
To click on Jetzt berechnen you can use the following line of code:
berechnen_button = browser.execute_script("""return document.querySelector('kredit-rechner').shadowRoot.querySelector('input.calculator__submit-button')""")
berechnen_button.click()
<div id="start" class="btn btn-huge btn-success"><i class="fas fa-power-off"></i> Start</div>
This is the HTML code line in the view source of a website. It is the code line for a start button that is on the website. I am using selenium with python and i want to locate this start button and click it. I have tried to use the following:
browser.find_element_by_css_selector("div.btn btn-huge btn-success").click()
browser.find_element_by_id("start").click()
and the .find_element_by_xpath() method too.. but none of them seem to work. I dont know if the code is able to locate that start button or not, but it is not clicking it so i am assuming that it is not locating it.. please tell me how i should fix this.. :) this is my complete code:
import undetected_chromedriver as uc
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
import time
browser = uc.Chrome()
browser.get('https://aternos.org/server/devialcraft')
browser.refresh()
time.sleep(1)
username = browser.find_element_by_id("user")
username.clear()
username.send_keys("Not_Breadbutter")
time.sleep(2)
password = browser.find_element_by_id("password")
password.clear()
password.send_keys("my_password")
time.sleep(2)
browser.find_element_by_id("login").click()
browser.refresh()
browser.refresh()
browser.find_element_by_css_selector("div.server-name").click()
#This is the line where i am locating that button:
time.sleep(1)
browser.find_element_by_css_selector("div.btn btn-huge btn-success").click()
time.sleep(9)
browser.quit()
I do not have the right credentials to check the HTMLDOM.
But your this line of code :
browser.find_element_by_css_selector("div.btn btn-huge btn-success").click()
is wrong cause CSS is wrongly used.
Please try :
browser.find_element_by_css_selector("div.btn.btn-huge.btn-success").click()
The reason behind this is, it is a combination of multiple class, so in Selenium we use . to combine these classes.
PS : Please check in the dev tools (Google chrome) if we have unique entry in HTML DOM or not.
Steps to check:
Press F12 in Chrome -> go to element section -> do a CTRL + F -> then paste the css and see, if your desired element is getting highlighted with 1/1 matching node.
Diagnosis :
Check if it is in an iframe.
Check if it is in shadow root.
Have you been redirected to a new tab or windows?
I have written a simple web scraping code using Selenium but I want to scrape only the portion that is present 'before scroll'
Say, if it is this page I want to scrape - https://en.wikipedia.org/wiki/Pandas_(software) - Selenium reads information till the absolute last element/text which for me is the 'Powered by Media Wiki' button on the far bottom-right of the page.
What I want Selenium to do is stop after DataFrames (see screenshot) and not scroll down to the bottom.
And I also want to know where on the page it stops. I have checked multiple sources and most of them ask for infinite scroll websites. No one asks for just the 'visible' half of a page.
This is my code now:
from selenium import webdriver
EXECUTABLE = r"chromedriver.exe"
# get the URL
url = "https://en.wikipedia.org/wiki/Pandas_(software)"
# open the chromedriver
driver = webdriver.Chrome(executable_path = EXECUTABLE)
# google window is maximized so that all webpages are rendered in the same size
driver.maximize_window()
# make the driver wait for 30 seconds before throwing a time-out exception
driver.implicitly_wait(30)
# get URL
driver.get(url)
for element in driver.find_elements_by_xpath("//*"):
try:
#stuff
except:
continue
driver.close()
Absolutely any direction is appreciated. I have tried to be as clear as possible here but let me know if any more details are required.
I don't think that is possible. Observe the DOM, all the informational elements are under one section I mean one tag div[#id='content'], which is already visible to Selenium. Even if you try with //*, div[#id='content'] is visible.
And trying to check whether the element is visible though not scrolled, will also return True. (If someone knows to do what you are asking for, even I would like to know.)
from selenium import webdriver
from selenium.webdriver.support.expected_conditions import _element_if_visible
driver = webdriver.Chrome(executable_path = 'path to chromedriver.exe')
driver.maximize_window()
driver.implicitly_wait(30)
driver.get("https://en.wikipedia.org/wiki/Pandas_(software)")
elements = driver.find_elements_by_xpath("//div[#id='content']//*")
for element in elements:
try:
if _element_if_visible(element):
print(element.get_attribute("innerText"))
except:
break
driver.quit()
I am trying to click to an object that I select with Xpath, but there seems to be problem that I could not located the element. I am trying to click accept on the page's "Terms of Use" button. The code I have written is as
driver.get(link)
accept_button = driver.find_element_by_xpath('//*[#id="terms-ok"]')
accept_button.click()
prov = driver.find_element_by_id("province-region")
prov.click()
Here is the HTML code I have:
And I am getting a "NoSuchElementException". My goal is to click this "Kabul Ediyorum" button at the bottom of the HTML code. I started to think that we have some restrictions on the tasks we could do on that page. Any ideas ?
Not really sure what the issue might be.
But you could try the following:
Try to locate the element by its visible text
accept_button = driver.find_element_by_xpath("//*[text()='Kabul Ediyorum']").click()
Try with ActionChains
For that you need to import ActionChains
from selenium.webdriver.common.action_chains import ActionChains
accept_button = driver.find_element_by_xpath("//*[text()='Kabul Ediyorum']")
actions = ActionChains(driver)
actions.click(on_element=accept_button).perform()
Also make sure you have implicitly wait
# implicitly wait
driver.implicitly_wait(10)
Or explicit wait
element = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//*[text()='Kabul Ediyorum']"))).click()
Hope this helped!