Can't find element on webpage with Selenium - python

I'm trying to build a bot that will book a class for me when I run it.
But I'm confused about how to proceed.
I'm using Python and Selenium with Chromedriver to load in the page.
The next step is to click the login button, add my data, log in and then navigate the page to find the correct class.
I'm very new to this so it might be very obvious but I can't figure it out.
I'm running this code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
d = webdriver.Chrome()
d.get("http://www.360functionalfitness.se/boka-pass/")
elem = d.find_element_by_name("showLogin")
The following error:
python3.9/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[name="showLogin"]"}
(Session info: chrome=87.0.4280.141)
I've been reading a bunch and still can't figure out how to identify the dang button.
This is the block I think I'm supposed to look at to find the relevant info:
<li ng-if="!member" class="ng-scope">
<i class="fa fa-sign-in"><span class="icon-bg bg-darkgreen"></span></i><span class="ng-binding">Logga in</span>
</li>
Any pointers would be extremely helpful.

The element with the text as Logga in is within an <iframe> so you have to:
Induce WebDriverWait for the desired frame to be available and switch to it.
Induce WebDriverWait for the desired element to be clickable.
You can use either of the following Locator Strategies:
Using CSS_SELECTOR:
driver.get("http://www.360functionalfitness.se/boka-pass/")
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe[src^='https://crossfitmalmo.gymsystem.se/member#/schedule']")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[ng-click^='showLogin'] span.ng-binding"))).click()
Using XPATH:
driver.get("http://www.360functionalfitness.se/boka-pass/")
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[starts-with(#src, 'https://crossfitmalmo.gymsystem.se/member#/schedule')]")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[starts-with(#ng-click, 'showLogin')]//span[#class='ng-binding']"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Browser Snapshot:
Reference
You can find a couple of relevant discussions in:
Ways to deal with #document under iframe
Switch to an iframe through Selenium and python
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element while trying to click Next button with selenium
selenium in python : NoSuchElementException: Message: no such element: Unable to locate element

Related

Having trouble locating an element with in an iframe using Selenium Python

I am attempting to click a button using an XPath but I am unable to locate the element. Complete noob here.
Here is the button element I copied:
<button _ngcontent-ygw-c218="" class="btn"><span _ngcontent-ygw-c218="" translate="">SHOW ALL</span></button>
Here is my programming:
ShowAll =driver.find_element_by_xpath('//*[#id="app-SelectComponents"]/div[1]/button[1]')
ShowAll.click()
I have tried the following solution I found online to no avail (I've also replaced the 'btn' with 'SHOW ALL', no luck there):
driver.switch_to.frame(driver.find_element_by_name('btn'))
ShowAll =driver.find_element_by_xpath('//*[#id="app-SelectComponents"]/div[1]/button[1]')
ShowAll.click()
driver.switch_to.default_content()
Much appreciated.
EDIT: Here is a picture for reference. What am I doing wrong or what I can do to work around this issue?
You need to add time to make sure the element is available before asking selenium to pick it.
Try this:
import time
driver.switch_to.frame(driver.find_element_by_name('btn'))
time.sleep(5) #you can change the 5 depending on the number that works
ShowAll =driver.find_element_by_xpath('//*[#id="app-SelectComponents"]/div[1]/button[1]').click()
driver.switch_to.default_content()
The element with the text as SHOW ALL is within an <iframe> so you have to:
Induce WebDriverWait for the desired frame to be available and switch to it.
Induce WebDriverWait for the desired element to be clickable.
You can use either of the following Locator Strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe#iParts")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button.btn > span[translate]"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[#id='iParts']")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[#class='btn']/span[text()='SHOW ALL']"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Reference
You can find a couple of relevant discussions in:
Ways to deal with #document under iframe
Switch to an iframe through Selenium and python
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element while trying to click Next button with selenium
selenium in python : NoSuchElementException: Message: no such element: Unable to locate element

Python XPath selector not working with selenium

I tried to automate some inputs. For this I need to input some text after the tag.
To identify the place where to input I tried XPath for fowlloing HTML code.
<span data-offset-key="1dq3m-0-0">
<br data-text="true">
</span>
Here is what I wrote in python.
buf_comp_text = 'foobar'
el_xp_comp_text = '//*[#data-text]'
...
## create post in queue (comment)
print('create post in queue - text')
post_txt = driver.find_element_by_xpath(el_xp_comp_text).send_keys(buf_comp_text)
Unfortunately I alwas get error message:
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[#data-text]"}
Any hint is appreciated.
Instead of targetting the <br> tag you need to target the <span> tag you can use the following Locator Strategy:
Using css_selector:
buf_comp_text = 'foobar'
driver.find_element_by_css_selector("span[data-offset-key]").send_keys(buf_comp_text)
Using xpath:
buf_comp_text = 'foobar'
driver.find_element_by_xpath("//span[#data-offset-key][.//br[#data-text]]").send_keys(buf_comp_text)
Ideally, to send a character sequence to the element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategy:
Using CSS_SELECTOR:
buf_comp_text = 'foobar'
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "span[data-offset-key]"))).send_keys(buf_comp_text)
Using XPATH:
buf_comp_text = 'foobar'
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//span[#data-offset-key][.//br[#data-text]]"))).send_keys(buf_comp_text)
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Reference
You can find a couple of relevant discussions in:
Ways to deal with #document under iframe
Switch to an iframe through Selenium and python
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element while trying to click Next button with selenium
selenium in python : NoSuchElementException: Message: no such element: Unable to locate element

Selenium returning NoSuchElementException but the element exists

I am having this code
driver.implicitly_wait(300) # I have tried different timers
driver.find_element_by_xpath("(//button[#aria-pressed='false'])[1]").click()
But I get this error:
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"(//button[#aria-pressed='false'])[1]"}
However, this exists in my HTML content, like:
<div class="webix_el_box" style="width:90px; height:26px">
<button aria-pressed="false" type="button" class="webix_img_btn" style="line-height:22px;">
<span class="webix_icon_btn fa-lock red" style="max-width:22px;">
</span> Read Only
</button>
</div>
I have also tried to use the XPath expression:
//button[#class='webix_img_btn']//child::span[#class='webix_icon_btn fa-lock red']
Both XPath expressions worked fine in Google Chrome.
In the website I am able to find the button using Ctrl + F and 'Read Only', can I use it in Selenium?
It is not working because of Selenium Driver is one page behind ajax call after large json data download.
To click on the element with text as Read Only you have to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div.webix_el_box > button.webix_img_btn"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[#class='webix_el_box']/button[#class='webix_img_btn' and contains(., 'Read Only')]"))).click()
Note: You have to add the following imports:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
References
You can find a couple of relevant discussions on NoSuchElementException in:
"selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element" while clicking a 'Next' button with Selenium
Selenium in Python: "NoSuchElementException: Message: no such element: Unable to locate element"
Try doing just
driver.implicitly_wait(300)
driver.find_element_by_xpath("//button[#aria-pressed='false']").click()
This will click on the first element that satisfies the requirements.

Selenium Webdriver How to scrape text?

Here's the website code:
<div class="question-row clearfix " lang="en">
<div class="qquestion qtext ">
five
</div>
</div>
I'm trying to scrape the text (in this example, it's 'five'). Here's my current code:
qtitle = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//*[#id='prompt-row'div/div"))).get_attribute("text")
However when I run it, this error code comes up:
File "C:\Users\lycop\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\support\wait.py", line 80, in until
raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:
That's all it says. Any help on solving this problem would be appreciated.
To print the text five you can use either of the following Locator Strategies:
Using css_selector and get_attribute():
print(driver.find_element_by_css_selector("div.qquestion.qtext").get_attribute("innerHTML"))
Using xpath and text attribute:
print(driver.find_element_by_xpath("//div[#class='qquestion qtext ']").text)
Ideally, to print the innerText of an element you have to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following solutions:
Using CSS_SELECTOR:
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "button.save[type='submit'][onclick]"))).get_attribute("innerHTML"))
Using XPATH:
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//button[#class='save' and text()='save'][#type='submit' and #onclick]"))).text)
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
You can find a couple of relevant discussions on NoSuchElementException in:
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element while trying to click Next button with selenium
selenium in python : NoSuchElementException: Message: no such element: Unable to locate element

Clicking on HTML <a> tag with selenium python

I try to click on html a tag with selenium but I get an error.
How can I get selenium to click on such tag?
I've tried this code but I get an error:
driver.find_element_by_id("btnCreateJE").click()
HTML code:
<a id="btnCreateJE" data-permission="true" onclick="NewManualJE()" href="#" class="btn waves-effect waves-light"> New </a>
Python selenium code:
driver.find_element_by_id("btnCreateJE").click()
I get that error:
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="btnCreateJE"]"}
(Session info: chrome=75.0.3770.142)
Make sure that the element doesn't belong to the iframe, if this is the case - you will need to switch_to_frame first
Make sure that the element doesn't belong to the Shadow DOM, if this is the case - you will have to find the relevant ShadowRoot and locate your element relatively
It might be the case the link is not immediately present in DOM, i.e. it's being added later as a result of an AJAX call. If this is the case - consider adding Explicit Wait like:
WebDriverWait(driver, 10).until(expected_conditions.presence_of_element_located((By.ID, "btnCreateJE"))).click()
The desired element is a JavaScript enabled element so to click() on the element you have to induce WebDriverWait for the element to be clickable and you can use either of the following solutions:
Using PARTIAL_LINK_TEXT:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, "New"))).click()
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a.btn.waves-effect.waves-light#btnCreateJE[onclick^='NewManualJE']"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[#class='btn waves-effect waves-light' and #id='btnCreateJE'][starts-with(#onclick, 'NewManualJE')]"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Categories