All I am trying to do is: go to "http://news.google.com" and click on the "Technology" link on the side menu. Here's my code:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import Select
from selenium.common.exceptions import NoSuchElementException
from selenium.common.exceptions import NoAlertPresentException
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import unittest, time, re
class GoogleTech(unittest.TestCase):
def setUp(self):
self.driver = webdriver.Firefox()
self.driver.implicitly_wait(30)
self.base_url = "https://www.google.com/"
self.verificationErrors = []
self.accept_next_alert = True
def test_googletech(self):
driver = self.driver
driver.get("http://news.google.com")
#driver.find_element_by_xpath("//div[#id='main-wrapper']/div[#id='main-pane']/div[#class='background']/div[#class='centered']/div[#id='nav-menu-wrapper']/div[#class='browse-sidebar']/ul[#id='anchorman-two-browse-nav']/li[#class='nav-item nv-en_us:tc']/a[contains(text(),'Technology')]").click()
#driver.find_element_by_xpath("//ul[#id='anchorman-two-browse-nav']/li/a[contains(text(),'Technology')]").click()
#driver.find_element_by_xpath("//ul[#id='anchorman-two-browse-nav']/li[#class='nav-item nv-en_us:tc']/a[contains(text(),'Technology')]").click()
#driver.find_element_by_xpath("html/body/div[3]/div[1]/div/div/div[2]/div/ul/li[6]/a").click()
#driver.find_element_by_link_text("Technology").click()
wait = WebDriverWait(driver, 10)
link = wait.until(EC.presence_of_element_located((By.XPATH, "//ul[#id='anchorman-two-browse-nav']/li/a[contains(text(),'Technology')]")))
link.click()
time.sleep(10)
...
I've tried many combinations (all the commented out lines plus more!), but still cannot get it to work. I've check to make sure that the element I am using does exists (using Element Inspector) before entering it into the code. But even the "find_element_by_link_text" is giving me a NoSuchElementException exception. Can someone please tell me what I am doing wrong?
UPDATE: After doing further tests, I now have a better understanding of when this error is occuring for the news.google.com page. So, basically, the side menu, that contains the "Technology" link, is set to scrollable (I believe this error is only happening because the link is in scrollable area). The error is show up if the browser (that the Selenium script opens up when you run it), does not show the "Technology" link ie. You have to maximize the browser or scroll down to see the link. You can test this error out yourself - when the test opens up the browser, quickly resize the browser window so that the "Technology" link is not showing, and the test will fail. If your initial browser window doesn't show "Technology" link when it first opens, it will fail unless you maximize the browser window or scroll down really quickly until the "Technology" link is displayed. The error can take two forms - if you are using the wait.until method, you will get a Timeoutexception. If you are not using the wait.until method, you will see the ElementNotVisibleException. I've tested in both Chrome and Firefox and for both, I am seeing this problem. I am new to Selenium, but I believe this is not normal behavior, can someone please confirm? If this normal, then can someone please tell me how I can make sure my test runs correctly each time?
I think you want to use element_to_be_clickable as your ExpectedCondition.
That means that your driver will keep polling Selenium until it finds that the element is visible and enabled source
Try the below solution
link = wait.until(EC.element_to_be_clickable((By.XPATH, "//ul[#id='anchorman-two-browse-nav']/li/a[contains(text(),'Technology')]")))
link.click()
time.sleep(10)
Related
Hello everyone I'm learning selenium recently and as I bet is a classic newbie mistake I filled my code with time.sleep which made everything really slow. So I started to learn about webdriverWait. I took some sample code and I have tried multiple things but I still get the error of "what you clicked is blocked by this other thing" meaning that the library did not crash but it also did not do anything. Which must mean I am doing something wrong. This is an error I can avoid if use the time.sleep function.
I'm using expected conditions, BY and action chains alongside WebDriverWait, though in my last test I tried to not use Acton chains to lower the possibilities of why its failing.
I'm using a company site that is probably under NDA so I don't have a public example to show this on, I tried searching for "pages with cover openings" but I couldn't find any, so if you guys know of any that I can use to illustrate this I would also find that really helpful. What is happening is that the site loads with a "cover" animation and after a few seconds the animation goes away to reveal the Button I'm looking for.
Environment info:
Using Pycharm community latest version
Using Selenium 4.0.0b2.post1 (I also tried in 3.9 to no results)
Using ChromeDriver 89 as my google chrome version is 89 as well [Version 89.0.4389.114 for Chrome]
Here are my code snippets:
Attempt #1:
from selenium import webdriver
from selenium.webdriver.support.wait import WebDriverWait
import selenium.webdriver.support.expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.by import By
import selenium.webdriver.support.ui as ui
def test(driver, actions, delay):
cart = "//header/div[1]/div[1]/a[2]/*[1]"
cartxp = WebDriverWait(driver, delay).until(EC.visibility_of_element_located((By.XPATH, cart)))
actions.move_to_element(cartxp).perform()
cartclk = driver.find_element_by_xpath(cart).click()
Attempt #2: Using UI library
from selenium import webdriver
from selenium.webdriver.support.wait import WebDriverWait
import selenium.webdriver.support.expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.by import By
import selenium.webdriver.support.ui as ui
def test2(driver, delay, actions):
cart = "//header/div[1]/div[1]/a[2]/*[1]"
cartxp = ui.WebDriverWait(driver, delay).until(EC.visibility_of_element_located((By.XPATH, cart)))
actions.move_to_element(cartxp).perform()
cartclk = driver.find_element_by_xpath(cart).click()
Attempt #3: Not using Action chains
from selenium import webdriver
from selenium.webdriver.support.wait import WebDriverWait
import selenium.webdriver.support.expected_conditions as EC
from selenium.webdriver.common.by import By
def test3(driver, delay):
wait = WebDriverWait(driver, delay)
cart = "//header/div[1]/div[1]/a[2]/*[1]"
button = wait.until(EC.element_to_be_clickable((By.XPATH, cart)))
cartclk = driver.find_element_by_xpath(cart).click()
All of these have a main method that just has:
driver = webdriver.Chrome()
test3(driver, 30)
driver.get('Site that I cant reveal cuz of NDA')
action = ActionChains(driver)
The error I get is:
Message: element click intercepted: Element ... is not clickable at point (1183, 27). Other element would receive the click
Thanks in advance for any help!
Edit: After some comments changed my xpath to //a[#class='header__cart'] both the previous and this one leads to a single result If I use it on chrome inspect to look for the button, additionally it works as intended if I use it to actually click the button after using time.sleep() to wait out the animation
Additionally just in case tried using the try catch as they did on the suggested question. It also failed
Attempt 4: surrounding in a try-catch
def test4(driver, delay):
cart = "//a[#class='header__cart']"
try:
my_elem = WebDriverWait(driver, delay).until(EC.visibility_of_element_located((By.XPATH, cart)))
print
"Page is ready!"
except TimeoutException:
print
"Loading took too much time!"
cart_btn = driver.find_element_by_xpath(cart)
cart_btn.click()
Error: selenium.common.exceptions.ElementClickInterceptedException: Message: element click intercepted: Element <a href="/cart" class="header__cart" data-items-in-cart="0" )="">... is not clickable at point (1200, 27). Other element would receive the click
To clarify what I meant earlier this works:
cart = "//a[#class='header__cart']"
time.sleep(20)
cartclk = driver.find_element_by_xpath(cart).click()
I am trying to log on to Target's website using Selenium in Python with the Chrome WebDriver..
When I am prompted to log in, I use the following code:
self.browser.find_element_by_name("password").send_keys(pw)
self.browser.find_element_by_id("login").submit()
After the field is submitted, I am presented with this error in the DOM:
DOM Error Alert
..and this in the console:
401 Error
Note:
I have tried logging in with Selenium on Instagram, and it works.. So I know it has something to do with the structure of Target's website. Has anyone run into this issue before?
Thanks!
So I originally tried solving this issue using Chrome, but could not figure out why the page would not precede after entering the login data. I thought maybe the page was protected by some bot software, but could not find any proof.
I decided to try Safari on my MAC, and actually had success. See the below code:
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as ec
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.keys import Keys
driver = webdriver.Safari(executable_path='/usr/bin/safaridriver')
driver.get('https://www.target.com/')
action = ActionChains(driver)
driver.find_element(By.XPATH, '//*[#id="account"]').click()
WebDriverWait(driver, 30).until(ec.presence_of_element_located((By.XPATH, '//*[#id="accountNav-signIn"]')))
action.send_keys(Keys.ENTER)
action.perform()
WebDriverWait(driver, 10).until(ec.presence_of_element_located((By.XPATH, '//h2[#class="sc-hMqMXs sc-esjQYD eXTUDl"]')))
driver.find_element(By.ID, 'username').click()
driver.find_element(By.ID, 'username').send_keys('foo')
time.sleep(5)
driver.find_element(By.ID, 'password').click()
driver.find_element(By.ID, 'password').send_keys('bar')
time.sleep(5)
driver.find_element(By.XPATH, "//button[#id=\'login\']").send_keys(Keys.ENTER)
time.sleep(10)
driver.quit()
You will notice some time.sleeps which I am using to slow the program down (you can take these out).
I also tried on FireFox and Edge, but had the same problems as Chrome.
Conclusion, it seems there could be some sort of bot protection which is blocking you from using Chrome (also Edge and FireFox). Given these webdrivers are being detected as automated. Safari (I believe) does not get detected as such.
I would also suggest reading through the below post, it may offer more insight.
Website navigates to no-access page using ChromeDriver and Chrome through Selenium probably Bot Protected
I'm testing some web scraping on Instagram with Selenium and Python.
In this case I want to upload a picture.
Normally you have to click on the upload icon and choose the file from a window. How can I manage it with Selenium?
I tried:
driver.find_element_by_class_name("coreSpriteFeedCreation").send_keys('C:\\path-to-file\\file.jpg')
and also with find_element_by_xpath but I get an exception:
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot focus element
I tried also only with click() but nothing happens.
Any Idea?
EDIT
Thanks to #homersimpson comment I tried this:
actions = ActionChains(driver)
element = driver.find_element_by_class_name("coreSpriteFeedCreation")
actions.move_to_element(element)
actions.click()
actions.send_keys('C:\\path-to-file\\file.jpg')
actions.perform()
Now the window to choose the file appears. The problem is that I would like to avoid this window and give directly the path of my file.
If right understand, you are trying to avoid handling with a native window. You can try this:
# get all inputs
inputs = driver.find_elements_by_xpath("//input[#accept = 'image/jpeg']").send_keys(os.getcwd() + "/image.png")
Now you can try all of them. I don't know which of them will work.
More about os.getcwd() is here
To be able to perform this code you have to have an element like this:
<input type="file" name="fileToUpload" id="fileToUpload2" class="fileToUpload">
EDIT:
It looks like instagram turned of input fields interaction for posts. For Account image it still works, but not for posting. I assume it is was done to prevent bots to post images. Anyway, there is a solution for this problem. You can use AutoIt like this:
import autoit
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
ActionChains(driver).move_to_element( driver.find_element_by_xpath("//path/to/upload/button")).click().perform()
handle = "[CLASS:#32770; TITLE:Open]"
autoit.win_wait(handle, 60)
autoit.control_set_text(handle, "Edit1", "\\file\\path")
autoit.control_click(handle, "Button1")
I think I may have found a solution that works for me. I found if you first have the bot click the plus icon while the browser is in the mobile view.
self.driver.find_element_by_xpath("/html/body/div[1]/section/nav[2]/div/div/div[2]/div/div/div[3]")\
.click()
after that, I would immediately send my file to an input tag in the HTML and I find you may need to play around as to which one works but I find the last input tag worked for me.
self.driver.find_element_by_xpath("/html/body/div[1]/section/nav[2]/div/div/form/input")\
.send_keys("/image/path.jpg")
The one weird thing about this is you will have a popup menu on top of the page but your code will still function with this window displayed over the window you are working on.
Addition to HumbleFox's answer. To solve his problem regarding the pop-up box not closing or the file pop-up box not closing (bug)
The solution to this is to make the browser headless here's a part of my code for example:
mobile_emulation = { "deviceName": "Pixel 2" }
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_experimental_option("mobileEmulation", mobile_emulation)
chrome_options.binary_location = self.opt.binary_location
self.driver = webdriver.Chrome(executable_path=self.path, options=chrome_options)
import webbrowser
from selenium import webdriver
browser = webdriver.Chrome()
browser.get('https://www.suntrust.com/')
browser.implicitly_wait(10)
elem = browser.find_element_by_xpath('//*[#id="sign-on-3A69E29D-79E0-403E-
9352-5261239ADD89-user"]')
elem.send_keys('your-username')
I'm having two problems:
1) The window doesn't open up in full screen, meaning the username field isn't physically visible. How do I open the url in a new tab instead of a new window.
2) Other posts suggest that the element is faked by JavaScript so that webdriver can't see it.
I've tried find_element_by in all the other locators.
Your question should be answered by a simple line of code that you need to include
browser.maximize_window()
would maximise your window. Another option is to set a specific window size like
driver.set_window_size(1280, 1024)
You can use both to achieve the browser being open to a maximum size.
Another point that I would make is that, if you're a beginner, try using more of CSS Selectors instead of the Xpath. They are much faster than Xpath's. Please see a detailed post on SQA about what makes a good locator.
For your case, the CSS Selector for the sign in field would be
driver.find_element_by_css_selector('input#sign-on-3A69E29D-79E0-403E-9352-5261239ADD89-user')
For password, it would be
driver.find_element_by_css_selector('input#sign-on-3A69E29D-79E0-403E-9352-5261239ADD89-password')
For Sign On button it would be
driver.find_element_by_css_selector('button.suntrust-login-button')
Please read more about CSS Selectors and try using them more often in your code.
Here is the Answer to your Question:
The xpath you have constructed is not unique. The xpath matches exactly to 2 elements on the HTML DOM. So Selenium was trying to send_keys on the first matching element which was invisible. Hence the error element not visible. The xpath used in the following code block identifies the User ID field uniquely and sends the text:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("start-maximized")
options.add_argument("disable-infobars")
options.add_argument("--disable-extensions")
browser = webdriver.Chrome(chrome_options=options, executable_path="C:\\Utility\\BrowserDrivers\\chromedriver.exe")
browser.get('https://www.suntrust.com/')
browser.implicitly_wait(15)
elem = browser.find_element_by_xpath('//section[#role="main"]//input[#id="sign-on-3A69E29D-79E0-403E-9352-5261239ADD89-user"]')
elem.send_keys('your-username')
Let me know if this Answers your Question.
If you use absolute xpath then you can send the text in textbox.
Below code will do that
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
browser = webdriver.Chrome()
browser.maximize_window() # to open full size window
browser.get('https://www.suntrust.com/')
# browser.implicitly_wait(10)
WebDriverWait(browser, 10).until(
EC.presence_of_element_located((By.XPATH, '//*[#id="sign-on-3A69E29D-79E0-403E-9352-5261239ADD89-user"]')))
elem = browser.find_element_by_xpath("//div[#id='suntrust-login-form-herosignon']/div[2]/form/div[1]/input[1]")
elem.send_keys('your-username')
elem1 = browser.find_element_by_xpath("//div[#id='suntrust-login-form-herosignon']/div[2]/form/div[2]/input[1]")
elem1.send_keys('your-password')
I'm new to selenium and trying to automate the download of some government data. When using the code below. I manage to navigate to the right page and enter the right parmeter in the form, but then can't find a way to click the 'submit' button. I've tried find_element_by_partial_link_text("Subm").click() and I've tried find_element_by_class_name on a number of class names. Nothing works. Any ideas?
import time
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.keys import Keys
main_url="http://data.stats.gov.cn/english/easyquery.htm?cn=E0101"
driver = webdriver.Firefox()
driver.get(main_url)
time.sleep(8)
driver.find_element_by_partial_link_text("Industry").click()
time.sleep(8)
driver.find_element_by_partial_link_text("Main Economic Indicat").click()
time.sleep(8)
driver.find_element_by_id("mySelect_sj").click()
time.sleep(3)
driver.find_element_by_class_name("dtText").send_keys("last72")
time.sleep(4)
try:
driver.find_element_by_class_name("dtFoot").click()
except:
driver.find_element_by_class_name("dtFoot").submit()
Solved my own problem, the key was using
driver.find_element_by_class_name(`dtTextBtn`)
instead of
driver.find_element_by_class_name(`dtTextBtn f10`)
The latter was what I saw in the source code, but the f10 blocked selenium.