I am trying to find an element in the web page using selenium. when I "right click" on the web element and inspect, I can find the html code in the console. However when I copy the xpath of the same element from the console and try to execute it in firepath, it says "No matching nodes". Why is it so and how can I fix this ?
Here is the HTML of the element.
<input id="mobile" type="text" onchange="javascript:dispLocMob(this);
"onkeydown="javascript:dispLocMob(this);
"onkeyup="javascript:dispLocMob(this);" value="" maxlength=
"10" placeholder="Mobile Number" name="mobile">
And this is what I am doing:
receiverMobileElement = browser.find_element_by_xpath('//*[#id="mobile"]')
Please help.
Why using xpath when you can find the element by unique id?
from selenium import webdriver
browser = webdriver.Firefox()
browser.get('http://example.com')
target_element = browser.find_element_by_id('mobile')
I always say screw firebug, use the Chrome console..
From the looks of it, the best way to get this particular element would be one of the following two options (given that 'id=mobile' is unique)..
By.cssSelector("#mobile");
By.xpath("//input[#id = 'mobile']");
To use the chrome console -->
Start Chrome
Right click and inspect your element.
You should see the html code and the console appear below it. (If you can not see the console, click the three vertical dots and select 'Show Console'
Now, from here you can do some awesome stuff! You can search for the elements that you are trying to identify and see if they can be found. for example, to see if the By.cssSelector("#mobile"); will work type this
document.querySelectorAll("#mobile")
for the xpath, type this
$xBy.xpath("//input[#id = 'mobile']");
Just a suggestion... dont be concerned about firebug if your code works, and seriously try out this way in chrome!!
Related
I have a script going using Selenium and on a new window pop up I am trying to retrieve this element:
<div class="custom-control p-2">somenumber</div>
The full path to this class is
html/body.modal-open/div#modalUserPRMLead.modal.fade.show/div.modal-dialog/div.modal-content/div.modal-body/div.row.justify-content-center.p-3/div.col-md-12/div.custom-control.p-2
I have tried variants of these to grab just the text of 'somenumber'
driver.find_element(By.CSS_SELECTOR,'row justify-content-center p-3').get_attribute('innerHTML')
driver.find_element(By.XPATH("html/body.modal-open/div.col-md-12./div.custom-control.p-2")).text
driver.find_element(By.XPATH("custom-control p-2")).text
driver.find_element(By.XPATH("div.row.justify-content-center.p-3")).text
driver.find_element(By.XPATH,custom-control p-2).text
driver.find_element(By.XPATH,div.custom-control.p-2).text
The errors I get is that it cannot locate the tag from current session. It's a new window that pops up as a result of previous script, I did driver.current_url just to make sure it was capturing the link to the new window in current session and it was.
I'm not understanding how to index down to the div class and just grab the text. Any help would be greatly appreciated. The script prior to window pop for form input was much easier because I could tag everything using By.NAME or By.ID
Your CSS selector is wrong. XPaths don't really looks like XPaths...
Try
driver.find_element(By.XPATH, '//div[#class="custom-control p-2"]').text
or
driver.find_element(By.CLASS_NAME, 'custom-control').text
In case element is dynamic you might need to apply wait
You can also check basic syntax of XPath and CSS-selector
Here is the inspect result for the button that says +5 per day
>span class="text user-links entry-method-title ng-scope ng-binding" ng-include="::'views/entry-text/'+entry_method.entry_type+'.html'">
Click For a Daily Bonus Entry"
</span>
<div class="entry-method bonus template" data-remove-popovers="" id="em6129519" ng-class="{expanded: entryState.expanded == entry_method, template: entry_method.template, 'completed-entry-method': !canEnter(entry_method) && isEntered(entry_method)}" ng-repeat="entry_method in ::entry_methods">
here is the HTML given information when I inspect the link/button, I have tried to use XPath, CSS, link text, and class name and it keeps giving me an error saying it cannot identify the element. Does anyone have a suggestion for how to identify this, it is on gleam.io for a giveaway I'm trying to automate this so i don't have to log in and press this everyday. This is my first ever web interfacing project with python.
Here is my most recent try
driver.maximize_window()
time.sleep(10)
driver.execute_script("window.scrollTo(0, 1440)")
time.sleep(10)
button2 = driver.find_element_by_class_name("text user-links entry-method-title ng-scope ng-binding")
button2.click()
Similar to a previous issue, Selenium find_element_by_class_name and find_element_by_css_selector not working, you can't have spaces in your class name when using driver.find_element_by_class_name. Instead, find the element via css_selector and replace each space with a dot.
driver.find_element_by_css_selector("span.text.user-links.entry-method-title.ng-scope.ng-binding")
That'll fix what you have above, but keep in mind there are other ways to make selenium actions more reliable (eg. WebDriverWait, etc). And there may be a cleaner selector to use than the one above.
I believe the element you want to access is contained within an "iframe", thus you must first switch to iframe before you can access it using selectors.
driver.switch_to.frame(x.find_element_by_xpath("PUT IFRAME XPATH HERE"))
My goal is to be able to scrape definitions of words in python.
To begin with, I am trying to get just the first definition of the word "assist" which should be "to help". I am using dictionary.cambridge.org
//web driver goes to page
driver.get("https://dictionary.cambridge.org/dictionary/english/assist")
//to give time for the page to load
time.sleep(4)
//click "accept cookies"
driver.find_element_by_xpath("/html[#class='i-amphtml-singledoc i-amphtml-standalone']/body[#class='break default_layout amp-mode-mouse']/div[#id='onetrust-consent-sdk']/div[#id='onetrust-banner-sdk']/div[#class='ot-sdk-container']/div[#class='ot-sdk-row']/div[#id='onetrust-button-group-parent']/div[#id='onetrust-button-group']/div[#class='banner-actions-container']/button[#id='onetrust-accept-btn-handler']").click()
Up this point, everything is working correctly. However, when I try to print the first definition using "find element by xpath", I get a NoSuchElementException. I'm pretty familiar with selenium and have scraped web stuff hundreds of times before but on this webpage, I don't know what I'm doing wrong. Here's the code I am using:
print(driver.find_element_by_xpath("/html[#class='i-amphtml-singledoc i-amphtml-standalone']/body[#class='break default_layout amp-mode-mouse']/div[#class='cc fon']/div[#class='pr cc_pgwn']/div[#class='x lpl-10 lpr-10 lpt-10 lpb-25 lmax lp-m_l-20 lp-m_r-20']/div[#class='hfr-m ltab lp-m_l-15']/article[#id='page-content']/div[#class='page']/div[#class='pr dictionary'][1]/div[#class='link']/div[#class='pr di superentry']/div[#class='di-body']/div[#class='entry']/div[#class='entry-body']/div[#class='pr entry-body__el'][1]/div[#class='pos-body']/div[#class='pr dsense dsense-noh']/div[#class='sense-body dsense_b']/div[#class='def-block ddef_block ']/div[#class='ddef_h']/div[#class='def ddef_d db']").text())
To print the scrape definitions of words you can use either of the following Locator Strategies:
Using xpath and text attribute:
print(driver.find_element_by_xpath("//span[contains(#class, 'epp-xref dxref')]//following::div[1]").text)
Using xpath and innerText:
print(driver.find_element_by_xpath("//span[contains(#class, 'epp-xref dxref')]//following::div[1]").get_attribute("innerText"))
Console Output:
to help:
Instead of Absolute xpath, opt for Relative xpaths. You can refer this link
Tried with below code and it retrieved the data.
driver.get("https://dictionary.cambridge.org/dictionary/english/assist")
print(driver.find_element_by_xpath("(//div[#class='ddef_h'])[1]/div").get_attribute("innerText"))
to help:
#Thanks in advance for help. New to python, tried for hour trying to correct mistake.#
Trying to locate login button element. Attached is the image of the website with the element of the login button. please see here
Below is code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
url = "https://www.xxxxxx.com/index.php/admin/index"
username = 'xxxx'
password = 'xxxxx'
driver = webdriver.Firefox(executable_path="C:\\Users\\kk\\AppData\\Local\\Programs\\Python\\Python38-32\\geckodriver.exe")
driver.get(url)
driver.find_element_by_name(name='aname').send_keys(username)
driver.find_element_by_name(name='apass').send_keys(password)
driver.find_elements_by_xpath('//style[#type="submit"]')
Rather than finding it with a CSS selector. Why not use find_element_by_xpath()
To get the XPath of that element just right-click the HTML of the input in Inspect Element, hover over Copy and you'll see "Full XPath"
Issue is your xpath.
driver.find_elements_by_xpath('//style[#type="submit"]')
Use below:
driver.find_elements_by_xpath('//input[#type="submit"]')
or
driver.find_elements_by_xpath('//input[#value="login"]')
#This is more accurate as many input tags could have type as submit
Also, please use some sort of wait as i am not sure if page will be loading fast enough every time you launch URL.
You can identify the submit button by using any of these 2:
//input[#type="submit"] or //input[#value="login"]
They should work without any problem if you don't have any similar elements on your page (which I doubt)
But if you want to be more precise, you can mix these 2 into:
//input[#value="login" and #type="submit"]
I know the basics of python and I am trying out Selenium. This page I am on is my schoolwork website. I want to select every single element in the timeline of my homework. I tried to select the whole timeline with XPath or even just one specific homework(elem) but both have this error code:
selenium.common.exceptions.NoSuchElementException: Message: no such
element: Unable to locate element:
{"method":"xpath","selector":"//*[#id="jwdd3b016b_acnt"]/div[3]/ul/li[3]"}
(Session info: chrome=81.0.4044.122)
This is the code:
from selenium import webdriver
driver = webdriver.Chrome('C:\\Users\\tomas\\Downloads\\chromedriver_win32\\chromedriver')
driver.get('https://glnt.edupage.org/timeline/')
#then I log in but that's not important
important_stuff = driver.find_element_by_xpath('//*[#id="jwdd3b016b_acnt"]/div[3]/ul/li[3]')
I have no idea what to do. I was under the impression that when u use XPath the code should always work. Am I wrong? (not always I mean, but when the code is correct)
Edit: sorry I forgot here is the HTML of the element:
<li class="hwItem m04 d5" data-typ="timeline" data-homeworkid="timeline:2031705" data-date="2020-04-24" data-timelineid="2031705">
That should do it:
<div class="edubarMainNoSkin">
<div id="jwdd3b016b_acnt" class="hwMainListMain">
<div class="hwListElem">
<ul class="hwList timelineMode ">
I couldn't say much without seen more of the HTML.
//*[#id="jwdd3b016b_acnt"]/div[3]/ul/li[3]
id part may not be constant.
It might be inside of an iframe, so you have to switch into it.
Due to browser rendering, deepness of the div might be changed.(div[3] would be changed)
First find the correct element using developer tools.
Then while executing using webdriver put a break point and check via developer tools.