I am trying to parse a table from https://www.morningstar.de/de/screener/fund.aspx#?filtersSelectedValue=%7B%22sustainabilityRating%22:%7B%22id%22:%225%22%7D%7D&page=1&perPage=10&sortField=legalName&sortOrder=asc.
However, by opening the website with selenium I always get at first a pop-up, to close which I need to select type of user (radiobutton) and then click on "accept" button .
After I proceed these "clicks" with the help of python and selenium, the pop-up doesn't disappear sbut I can see that clicks were proceeded. It doesn't show any error (all the needed fields are selected and python script also doesn't throw anything).
Here is my code:
from selenium import webdriver
import time
browser = webdriver.Firefox()
url="https://www.morningstar.de/de/screener/fund.aspx#?filtersSelectedValue=%7B%22sustainabilityRating%22:%7B%22id%22:%225%22%7D%7D&page=1&perPage=10&sortField=legalName&sortOrder=asc"
browser.get(url)
time.sleep(10)
try:
radio_button = browser.find_elements_by_xpath('/html/body/div[2]/div[3]/div/div[2]/div/div[3]/div[1]/div[1]/fieldset/div[2]/label/span/span[1]')[0]
radio_button.click()
time.sleep(3)
accept_button=browser.find_element_by_id('_evidon-accept-button')
accept_button.click()
print("accepted")
except:
print(" something went wrong")
I need to close this pop-up in order to get access to the table, what am I doing wrong?
Example for radio button #finaprofessional > span:nth-child(1) or #finaprofessional > span:nth-child(2)
import time
from selenium import webdriver
def example():
firefox_browser = webdriver.Firefox()
firefox_browser.get("https://www.morningstar.de/de/screener/fund.aspx#?filtersSelectedValue=%7B%22sustainabilityRating%22:%7B%22id%22:%225%22%7D%7D&page=1&perPage=10&sortField=legalName&sortOrder=asc")
time.sleep(10) # wait for page to load
radio_button = firefox_browser.find_element_by_css_selector("#finaprofessional > span:nth-child(1)")
radio_button.click()
time.sleep(10)
accept_button = firefox_browser.find_element_by_id("_evidon-accept-button")
accept_button.click()
if __name__ == "__main__":
example()
Related
I'm pretty new to selenium and i searched a lot, but can't find an answer to my problem.
I want to open firefox, go on google and search for something. Store everthing in a list and print it to my console.
But everytime i open firefox with selenium, a pop-up windows opens with a confirmation to the privacy regulations of google. I dont know how to close it.
Here is what i tried:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
# create a new Firefox session
driver = webdriver.Firefox()
driver.implicitly_wait(30)
driver.maximize_window()
# Navigate to the application home page
driver.get("http://www.google.com")
main_page = driver.current_window_handle
time.sleep(3)
#Tried to find out in which browers window I am right now -> terminal says '19'
print('The actual browers window is: {}'.format(main_page))
driver.find_element_by_xpath("//*[#id='introAgreeButton']/span/span").click()
# get the search textbox
search_field = driver.find_element_by_id("lst-ib")
search_field.clear()
# enter search keyword and submit
search_field.send_keys("Flowers")
search_field.submit()
# get the list of elements which are displayed after the search
# currently on result page using find_elements_by_class_name method
lists= driver.find_elements_by_class_name("_Rm")
# get the number of elements found
print ("Found " + str(len(lists)) + " searches:")
# iterate through each element and print the text that is
# name of the search
i=0
for listitem in lists:
print (listitem.get_attribute("innerHTML"))
i=i+1
if(i>10):
break
# close the browser window
driver.quit()
Google privacy popup is contained in iframe. You must switch to iframe and then look for accept (or deny) button. Like so
wait = WebDriverWait(driver, 5)
iframe = wait.until(EC.element_to_be_clickable([By.CSS_SELECTOR, '#cnsw > iframe']))
driver.switch_to.frame(iframe)
And then when in iframe, look for Accept button a click it.
Hello and happy Holidays!
I wrote a script where I open a webpage in chrome, login, go to a certain page and then trying to print that page.
When the Chrome print preview comes up, I can not target the print button neither try to just hit enter via automation. Nothing works.
Any ideas please?
Ps: This is a the code I wrote but the login information will be incorrect
Thank you in advance!
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from shutil import which
import time
from pynput.keyboard import Key, Controller
keyboard = Controller()
chrome_path = which("chromedriver")
driver = webdriver.Chrome(executable_path="chromedriver")
name = '****'
password = '****'
driver.get(
"https://www.winbank.gr/sites/corporate/el/Pages/default.aspx")
driver.switch_to.frame(driver.find_element_by_tag_name("iframe"))
username = driver.find_element_by_name('Username')
username.send_keys(name)
passwords = driver.find_element_by_name('Password')
passwords.send_keys(password)
passwords.send_keys(Keys.ENTER)
time.sleep(10)
driver.find_element_by_xpath(
'//*[#id="ng-app"]/body/div[3]/div/div/div/div/div/div/div/div[2]/div[2]/div[1]/div[2]/div/div[2]/div/div/div[6]').click()
window_before = driver.window_handles[0]
time.sleep(6)
driver.find_element_by_xpath(
'//*[#id="ng-app"]/body/div[3]/div/div/div/div/div/div/div/div[3]/div[2]/wb-pfm-transactions/div/div[2]/div/div[22]/div[2]').click()
time.sleep(3)
driver.switch_to.window(driver.window_handles[-1])
driver.find_element_by_id('print').click()
driver.switch_to.window(driver.window_handles[-1])
print('-------------test----------')
# driver.print = "function(){};"
keyboard.press(Key.enter)
driver.find_element_by_xpath(
'/html/body/print-preview-app//print-preview-sidebar//print-preview-button-strip//cr-button[2]').click()
Usually when dealing with untargetable objects I use the tab key to navigate and enter to "click" the element currently focused, what you should do is:
1. Try to print it manually, but with using tabs to navigate and Enter to click
2. Keep track of all key presses needed
3. Write your code accordingly
Edit: Also, if you need help with automatically pressing keys please refer to this link.
My code :
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
# execute url
url = "https://www.youtube.com/user/xuanvinh1612/community"
driver_path = ('F:/chromedriver.exe')
browser = webdriver.Chrome(executable_path=driver_path)
browser.get(url)
# Auto scroll and auto click with text:'Read more'
read_mores2 = browser.find_elements_by_link_text('Read more')
for read_mores2 in read_mores2:
browser.execute_script("arguments[0].scrollIntoView();", read_mores2)
browser.execute_script("$(arguments[0]).click();", read_mores2)
# Scroll down stop when all post was showed
read_mores2 = browser.find_elements_by_link_text('Read more')
With a same code, my code can run some website(2-3 another website). But when i re-use code for auto scroll down and auto click on Youtube/community, it not working. I dont know how it not work. I need help, please.
Try this code:
It will first load all the pages, then click on all Read More.
import time
from selenium import webdriver
# execute url
url = "https://www.youtube.com/user/xuanvinh1612/community"
browser = webdriver.Chrome()
browser.get(url)
# Auto scroll and auto click with text:'Read more'
previous_count = 0
page_sections = browser.find_elements_by_css_selector('.style-scope.ytd-item-section-renderer')
current_count = len(page_sections)
print("Scrolling to enable all the pages")
while previous_count != current_count:
try:
previous_count = current_count
browser.execute_script("arguments[0].scrollIntoView();", page_sections[-1])
print("Number of total Elements found: {}".format(len(page_sections)))
finally:
# As the page load the newer elements, you need to implement logic here to wait until the loading spinner at the
# button becomes invisible (not attached to the DOM)
time.sleep(2) # WorkAround as you need to implement the above logic here
page_sections = browser.find_elements_by_css_selector('.style-scope.ytd-item-section-renderer')
current_count = len(page_sections)
print("Clicking on all Read More")
for read_more in browser.find_elements_by_css_selector('.more-button'):
browser.execute_script("arguments[0].scrollIntoView();", read_more)
browser.execute_script("arguments[0].click();", read_more)
I have the following Code that goes to a URL(www.example.com), and clicks on a link(Example 1). (This part works fine)
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("https://www.example.com")
link = driver.find_element_by_link_text('Example 1')
link.click()
Now, when we click on 'Example 1' link, it opens a confirmation window, with 2 buttons: 'Yes I am authorized user to this site' and 'No I am a new visitor to this site'
So, I wish to click on 'Yes I am authorized user to this site' and then finally enter my log-in credentials.
I have written these 2 lines, just below the above code, for clicking on that button. But these don't work.
button = driver.find_element_by_name("'Yes I am authorized user to this site'")
button.click()
If it is an alert window, you need to use the Alert command.
#import Alert
from selenium.webdriver.common.alert import Alert
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("https://www.example.com")
link = driver.find_element_by_link_text('Example 1')
link.click()
Alert(driver).accept()
#to dismiss alert
#Alert(driver).dismiss()
I think this would have solved your query.
Based on the comment conversation, I would recommend both using an XPATH search (instead of Name or Id) and waiting for elements to be clickable or loaded. When web-driving or web-scraping, pages may intentionally or accidentally load slowly and this can cause issues if you have pauses or waits either hard coded or non-existent. This snippet of code should allow you to search Google using Selenium and Chromedriver (you can modify the driver function to use Firefox or something else if you'd like):
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import ElementNotVisibleException
from selenium.webdriver.chrome.options import Options
from time import sleep
def init_driver(drvr_path):
chrome_options = Options()
chrome_options.add_argument("--disable-extensions")
driver = webdriver.Chrome(drvr_path+'chromedriver.exe',chrome_options=chrome_options)
driver.wait = WebDriverWait(driver, 5)
return driver
def lookup(query, driver=None, drvr_path=''):
driver = None
if driver is None:
driver = init_driver(drvr_path)
driver.implicitly_wait(45) # Allow up to 45 Seconds for page to load
driver.get("http://www.google.com")
try:
box = driver.wait.until(EC.presence_of_element_located((By.XPATH, """//*[#id="lst-ib"]""")))
box.send_keys(query)
sleep(3) # Let you see the window open
button = driver.wait.until(EC.element_to_be_clickable((By.XPATH,"""//*[#id="sblsbb"]/button""")))
try:
button.click()
except ElementNotVisibleException, s:
print "Error Handled: "+str(s)
button = driver.wait.until(EC.element_to_be_clickable((By.XPATH,"""//*[#id="sblsbb"]/button""")))
try:
button.click()
except:
print "Could not search Google..."
return
resp=driver.page_source.encode('utf-8')
with open(query+'.html','wb') as f:
f.write(resp)
print 'Wrote the File...'
except:
print("Box or Button not found in google.com")
driver.quit()
For example, if your Chromedriver.exe file was located in your default Python path, you could do something like: lookup('Selenium Python XPATH Examples') and it should download an HTML file of the Google Search results. If you already have a Driver initialized, you could of course pass that to it.
Hope this helps
Try this code, hope it will help you
from selenium import webdriver
import time
driver = webdriver.Chrome('path to chromedriver\chromedriver.exe')
driver.get('https://www.example.com')
driver.maximize_window()
link = driver.find_element_by_link_text('Example 1')
link.click()
handles =driver.window_handles # this will give window handles
driver.switch_to.window(handles[1])
button = driver.find_element_by_name("'Yes I am authorized user to this site'")
button.click()
How do I switch to a popup window in the below selenium program. I've looked up for all possible solutions but haven't been able to get my head around them. Please help!!
from selenium import webdriver
from splinter import Browser
from selenium.webdriver.common.keys import Keys
handle = []
driver = webdriver.Firefox()
driver.get("http://example.com/test.aspx")
driver.find_element_by_link_text("Site Actions").click()
driver.find_element_by_link_text('Edit Page').click()
select = driver.find_element_by_id('ctl00_PlaceHolderMain_ctl35_ctl00_SelectResult')
for option in select.find_elements_by_xpath('//*[#id="ctl00_PlaceHolderMain_ctl35_ctl00_SelectResult"]/option'):
if option.text != 'Channel':
option.select() # select() in earlier versions of webdriver
driver.find_element_by_id('ctl00_PlaceHolderMain_ctl35_ctl00_RemoveButton').click()
parent_h = driver.current_window_handle
#click that activates the popup.
checkIn = driver.find_element_by_id('qaCheckin_anchor').click()
# click on the link that opens a new window
handles = driver.window_handles # before the pop-up window closes
driver.remove(parent_h)
driver.switch_to_window(handles.pop())
driver.implicitly_wait(10) # seconds
driver.find_element_by_xpath('/html/body/form/div[3]/table/tbody/tr[4]/td/table/tbody/tr[3]/td[2]/input').click()
driver.find_element_by_name('btnClose2').click()
driver.close();
# do stuff in the popup
# popup window closes
driver.switch_to_window(parent_h)
# and you're back
driver.switch_to_default_content()
in terms of browser, pop up is not a window, it is an alert. so, you should use following:
driver.switch_to_alert()