How to click the Continue button on website using selenium in python? - python

I am trying to write a code that is able to auto apply on job openings on indeed.com. I have managed to reach the last stage, however, the final click on the application form is giving me a lot of trouble. Please refer the page as below
Once logged in to my profile, I go to the relevant search page, click on the listing I am interested in and then on the final page (shown above) I am trying to click on the continue button using xpath as follows:
driver.get("https://in.indeed.com/jobs?q=data%20analyst&l=Delhi&vjk=5c0bd416675cf4e5")
driver.find_element_by_xpath('//*[#id="apply-button-container"]/div[1]/span[1]').click()
driver.find_element_by_xpath('//*[#id="form-action-continue"]')
However, this gives me an error:
Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[#id="form-action-continue"]"}
Having gone through some suggestions on the net I have even tried the following:
driver.get("https://in.indeed.com/jobs?q=data%20analyst&l=Delhi&vjk=5c0bd416675cf4e5")
driver.find_element_by_xpath('//*[#id="apply-button-container"]/div[1]/span[1]').click()
WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, '//*[#id="form-action-continue"]')))
But then this gives me a timeout error
TimeoutException: Message:
Will appreciate some help on this.

From it seems, on that form there are multiple iframes, therefore the reason for your errors.
You need to get the first iframe, switch to it, get the second iframe inside the first one, switch to it and only afterwards you'll be able to get the continue button.
Something like this should do the trick:
frame_1 = driver.find_element_by_css_selector('iframe[title="Job application form container"')
driver.switch_to.frame(frame_1)
frame_2 = driver.find_element_by_css_selector('iframe[title="Job application form"]')
driver.switch_to.frame(frame_2)
continue_btn = driver.find_element_by_css_selector('#form-action-continue')
continue_btn.click()

Once I had a similar issue and standard time.sleep() until the form (or continue button) appears helped me. Try it instead of WebDriverWait, maybe it will work.

Related

selenium-driver :looking a solution for locating and clicking a button when i open google page

python selenium driver:
No thanks
when i open a google page the, there is a small window asking if i want to sign in or not. i want to click on "No thanks" button, which is as shown above.
i have tried these methods so far, but i keep getting errors. None of the following is working.
#self.driver.find_element(By.CSS_SELECTOR, 'button.M6CB1c')
#button=self.driver.find_elements(By.XPATH, '//button')
#abc=self.driver.find_elements(By.NAME, 'ZUkOIc').click()
#self.driver.find_element(By.TAG_NAME, 'button').click()
error message for the 1st line of code:
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":".M6CB1c.rr4y5c"}
selenium.common.exceptions.NoSuchElement is caused when the element is not in the page at the current time.
TLDR;
What you're looking for is Explicit Wait in selenium. You need to use WebDriverWait with expected condition element_to_be_clickable.
When we load a page, modern pages tend to load javascript that can often manipulate DOM (html page objects). The proper way to handle this is to wait for the page or the required element to load, and then try to locate it.
The selenium waits section explains this very well with an example.
You should try this :
driver.find_element(By.XPATH, '//button[#id="W0wltc"])

How to click on a button that does not appear on page source?

I'm trying to click on a button using Selenium with Python, however the button does not appear on Page Source and thus, the instantiated driver cannot find it. Here is the link I am accessing: https://data.ntsb.gov/carol-main-public/query-builder?month=6&year=2021
And the button I want to click is "Download Summary (CSV)", which is located at the end of the page.
I have already googled this kind of trouble but I haven't found a possible solution so far.
My attempt was to find the button directly by using the following commands, but it returns a NoSuchElementException:
element = driver.find_element_by_id('exportSummaryButton')
element.click()
This exception triggered me the intuition that the page source differs from what I see on Devtools and this is exactly what happens.
Thanks in advance!

With selenium on python don't know how to shut down a banner which is preventing selenium from accessing the page content

I'm trying to open a site with Selenium (with Python) using Chrome browser, but when I do, a full screen promo banner immediately pops-up and I can't access the site content unless I close it.
On the top right there is an "x" as if it was a quit button, but actually it's an element ::before
and from its description it seems to me that it doesn't contain any button element.
If I operate manually, both clicking on the x and on the upper part of the page outside the banner, the latter closes, but I really don't understand how to access it with selenium.
The webpage I'm trying to open is https://sports.bwin.it/it/sports
Needless to say I'm quite inexperienced, so I hope this question won't sound too basic, but I wasn't able to find a solutione in the selenium docs or on the web; if someone could give me any hint I would appreciate it.
This is a screenshot from the page I'm talking about
This is part of the html code from the web page; the element I am talking about is the one pointed by the arrow;
Based on your screen shot the xpath you want to use would be something like this:
//*[#data-id='dj_sports_c_ovl_br']//span
full code would be something like this:
element = WebDriverWait(driver, 20).until(
EC.element_to_be_clickable((By.XPATH, "//*[#data-id='dj_sports_c_ovl_br']//span"))
)
element.click();

Unable to locate/click pop-up button with Selenium in Python

I'm using Selenium in Python 3 to access webpages, and I want to click on a pop-up button, but I am unable to locate it with Selenium.
What I'm describing below applies to a number of sites with a pop-up, so I'll use a simple example.
url = "https://www.google.co.uk"
from selenium import webdriver
driver = webdriver.Firefox()
driver.implicitly_wait(10)
driver.get(url)
The page has a pop-up for agreeing to cookies.
I want the script to click on the "I agree" button, but I'm unable to locate it.
I've found a few questions and posts about this online (including on Stackoverflow), but all the suggestions I found seem to fall in one of the following categories and don't seem to work for me.
Wait longer for the pop-up to actually load.
I've tried adding delays, and in fact, I'm testing this interactively, so I can wait all I want for the page to load before I try to locate the button, but it doesn't make any difference.
Use something like driver.switch_to.alert
I get a NoAlertPresentException. The pop-up doesn't seem to be an alert.
Locate the element using driver.find_element.
This doesn't work either, regardless of which approach I use (xpath, class name, text etc.). I can find elements from the page under the pop-up, but nothing from the pop-up itself. For example,
# Elements in main page (under pop-up)
driver.find_element_by_partial_link_text("Sign in") # returns FirefoxWebElement
driver.find_element_by_class_name("gb_g") # returns FirefoxWebElement
# Elements on the pop-up
driver.find_element_by_partial_link_text("I agree") # NoSuchElementException
driver.find_element_by_class_name("RveJvd snByac") # NoSuchElementException
The popup just doesn't seem to be there in the page source. In fact, if I try looking at the loaded page source from the browser, I can't find anything related to the pop-up. I understand that many sites use client-side scripts to load elements dynamically, so many elements wouldn't show up in the raw source, but that was the point of using Selenium: to load the page, interpret the scripts and access the end result.
So, what am I doing wrong? Where is the pop-up coming from, and how can I access it?

Selenium Python: Census ACS Data- unable to select Download button in window

I am attempting to scrape the Census website for ACS data. I have scripted the whole processes using Selenium except the very last click. I am using Python. I need to click a download button that is in a window that pops when the data is zipped and ready, but I can't seem to identify this button. It also seems that the button might change names based on when it was last run, for example, yui-gen2, yui-gen3, etc so I am thinking I might need to account for this someone. Although I normally only see yui-gen2.
Also, the tag seems to be in a "span" which might be adding to my difficulty honing in on the button I need to click.
Please help if you can shed any light on this for me.
code snippet:
#Refine search results to get tables
driver.find_element_by_id("prodautocomplete").send_keys("S0101")
time.sleep(2)
driver.find_element_by_id("prodsubmit").click()
driver.implicitly_wait(100)
time.sleep(2)
driver.find_element_by_id("check_all_btn_above").click()
driver.implicitly_wait(100)
time.sleep(2)
driver.find_element_by_id("dnld_btn_above").click()
driver.implicitly_wait(100)
driver.find_element_by_id("yui-gen0-button").click()
time.sleep(10)
driver.implicitly_wait(100)
driver.find_element_by_id("yui-gen2-button").click()
enter image description here
enter image description here
Instead of using the element id, which as you pointed out varies, you can use XPath as Nogoseke mentioned or CSS Selector. Be careful to not make the XPath/selector too specific or reliant on changing values, in this case the element id. Rather than using the id in XPath, try expressing the XPath in terms of the DOM structure (tags):
//*/div/div/div/span/span/span/button[contains(text(),'Download')]
TIL you can validate your XPath by using the search function, rather than by running it in Selenium. I right-clicked the webpage, "inspect element", ctrl+f, and typed in the above XPath to validate that it is the Download button.
For posterity, if the above XPath is too specific, i.e. it is reliant on too many levels of the DOM structure, you can do something shorter, like
//*button[contains(text(),'Download')]
although, this may not be specific enough and may require an additional field, since there may be multiple buttons on the page with the 'Download' text.
Given the HTML you provided, you should be able to use
driver.find_element_by_id("yui-gen2-button")
I know you said you tried it but you didn't say if it works at all or what error message you are getting. If that never works, you likely have an IFRAME that you need to switch to.
If it works sometimes but not consistently due to changing ID, you can use something like
driver.find_element_by_xpath("//button[.='Download']")
On the code inspection view on Chrome you can right click on the item you want to find and copy the xpath. You can they find your element by xpath on Selenium.

Categories