I need help selecting an element on a webpage with Selenium. I have been using Selenium on this website for about 3 weeks and so far, I can usually find an element by css selector or XPath. However, this specific section of the website is giving me a very hard time.
After I click on “reset office 365 password” a window comes up and I want to programmably put in the new password but it can’t find anything in the popup window.
Here is what the page looks like:
(I am too low of score to post pictures here) https://cdn.discordapp.com/attachments/768594779344470022/845811910577881098/unknown.png
Here is the whole element’s information:
<input type="password" tabindex="1" name="password" class="m-third pass ng-pristine ng-empty ng-invalid ng-invalid-required ng-touched" ng-model="password.value" ng-blur="password.check = false" ng-focus="password.check = true" required="" autofocus="" ng-disabled="!active">
Here is what I tried: (I tried a lot of things)
Tried clicking on the password box by using css selector – failed: Invalid selector
im_blacklistaddbutton = browser_options.browser.find_element_by_css_selector('#ng-app > div.page-container > div > div > div.vertical-tabs.j-vertical-tabs.ng-scope > div.vertical-tabs-panes.p0 > div > div > div.page-content.ng-scope > div > div > form > div > div > div.ng-isolate-scope > div.modal > div.modal-body.ng-transclude > div > reset:password > ng-form > div:nth-child(1) > div > div.validation-input > input')
im_blacklistaddbutton.send_keys(email_pd.pd)
selenium.common.exceptions.InvalidSelectorException: Message: invalid selector: An invalid or illegal selector was specified
Tried clicking on the password box by using xpath selector – failed: Namespace Error
im_blacklistaddbutton = browser_options.browser.find_element_by_xpath('//*[#id="ng-app"]/div[2]/div/div/div[3]/div[2]/div/div/div[2]/div/div/form/div/div/div[3]/div[1]/div[2]/div/reset:password/ng-form/div[1]/div/div[1]/input')
im_blacklistaddbutton.send_keys(email_pd.pd)
NamespaceError: Failed to execute 'evaluate' on 'Document': The string '//*[#id="ng-app"]/div[2]/div/div/div[3]/div[2]/div/div/div[2]/div/div/form/div/div/div[3]/div[1]/div[2]/div/reset:password/ng-form/div[1]/div/div[1]/input' contains unresolvable namespaces.
Tried waiting for the element by partial link text: It timed out
wait.until(EC.visibility_of_element_located((By.PARTIAL_LINK_TEXT, 'Generate password')))
selenium.common.exceptions.TimeoutException: Message:
Tried waiting for the element by ID name text value: It timed out
wait.until(EC.text_to_be_present_in_element((By.CLASS_NAME, 'btn m-link'), "Generate Password"))
selenium.common.exceptions.TimeoutException: Message:
Tried to switch to a window or iframe but it said that the div class of "model" is not a window or an iframe.
From here I am completely lost as to why this stupid window is not accessible. Text window - why are you so mean to me?
Here is my specific function in total:
def reset_im_oa_password():
browser_options.browser.get('https://cpx.intermedia.net/ControlPanel/Menu/AccountMenu/?frameUrl=https://cpx.intermedia.net/aspx/Office365/Home/licenses#/installed/users')
wait = WebDriverWait(browser_options.browser, 10)
try:
wait.until(EC.element_to_be_clickable((By.XPATH, 'player')))
except exceptions.TimeoutException as e:
pass
browser_options.browser.switch_to_frame('mainFrame')
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, '#ng-app > div.page-container > div > div > div.vertical-tabs.j-vertical-tabs.ng-scope > div.vertical-tabs-panes.p0 > div > div > div.page-content.ng-scope > div > div > form > div > div > div:nth-child(2) > div.table-wrap.table-fixed.j-table-wrap.s-wide.ng-isolate-scope > div.table-filter > div.table-filter-search.searchbox.ng-isolate-scope > div > span:nth-child(3) > input')))
im_blacklistaddbutton = browser_options.browser.find_element_by_css_selector('#ng-app > div.page-container > div > div > div.vertical-tabs.j-vertical-tabs.ng-scope > div.vertical-tabs-panes.p0 > div > div > div.page-content.ng-scope > div > div > form > div > div > div:nth-child(2) > div.table-wrap.table-fixed.j-table-wrap.s-wide.ng-isolate-scope > div.table-filter > div.table-filter-search.searchbox.ng-isolate-scope > div > span:nth-child(3) > input')
im_blacklistaddbutton.send_keys(email_or_user_selection.email_select)
im_blacklistaddbutton = browser_options.browser.find_element_by_css_selector('#ng-app > div.page-container > div > div > div.vertical-tabs.j-vertical-tabs.ng-scope > div.vertical-tabs-panes.p0 > div > div > div.page-content.ng-scope > div > div > form > div > div > div:nth-child(2) > div.table-wrap.table-fixed.j-table-wrap.s-wide.ng-isolate-scope > div.table-filter > div.table-filter-search.searchbox.ng-isolate-scope > div > span:nth-child(3) > button')
im_blacklistaddbutton.send_keys(Keys.ENTER)
wait.until(EC.element_to_be_clickable((By.XPATH, ("//*[starts-with(#id, 'btnResetPassword')]"))))
im_blacklistaddbutton = browser_options.browser.find_element_by_xpath(("//*[starts-with(#id, 'btnResetPassword')]"))
im_blacklistaddbutton.send_keys(Keys.ENTER)
try:
wait.until(EC.visibility_of_element_located((By.PARTIAL_LINK_TEXT, 'Generate password')))
except exceptions.TimeoutException as e:
pass
browser_options.browser.switch_to_window('model') # anything past this section will fail
wait.until(EC.visibility_of_element_located((By.CLASS_NAME, 'model')))
im_blacklistaddbutton = browser_options.browser.find_element_by_xpath('//*[#id="ng-app"]/div[2]/div/div/div[3]/div[2]/div/div/div[2]/div/div/form/div/div/div[3]/div[1]/div[2]/div/reset:password/ng-form/div[1]/div/div[1]/input')
im_blacklistaddbutton.send_keys(email_pd.pd)
return
if anyone needs the full code from the webpage let me know. Thanks
If this element is not really inside an iframe as you write, then, wait for it to become clickable, like this:
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input[type='password']")))
im_blacklistaddbutton = browser.find_element_by_css_selector("input[type='password']")
im_blacklistaddbutton.send_keys("new_password")
But make sure that css selector input[type='password'] is unique.
If not, try this one: .validation-input>input[type='password']
(Check validation-input class name si correct as it is cut on your screenshot)
If the input frame is inside iframe nothing will work until you switch to this iframe.
Because of no webpage code, right now I can't say why the element is not detectable by Selenium but you can try one thing. Right click on the element(input tag in dom shown in picture) and go to "Copy to" option and select "Copy JS Path". Then go to console tab in dev tools and paste it. Then try to set it's value to some dummy text and see if it sets the password.
jsPath.value="some password" //this should set the password
If this works, then you can set the value by using JavaScript executor of Selenium in the same way.
Related
I have 19 span with the same attribute data-checked
<span id="u25-accordion-panel--61" data-type="checkbox" data-checked style="display: none;"></span>
I want to change the attribute in all the spans, I tried with 1 but I get the following error
options = webdriver.EdgeOptions()
options.add_argument("start-maximized")
driver = webdriver.Edge(options=options)
driver.implicitly_wait(20)
driver.get(
'https://www.udemy.com/course/angular-10-fundamentos-8-app/')
elem = driver.find_element(By.CSS_SELECTOR,
'#udemy > div.ud-main-content-wrapper > div.ud-main-content > div > div > div.paid-course-landing-page__container > div.paid-course-landing-page__body > div > div:nth-child(3) > div > button')
elem.click()
elems = driver.find_elements(By.CSS_SELECTOR,
'div.accordion-panel--panel--24beS > span')
driver.execute_script("arguments[0].data-checked = 'checked';", elems[1])
driver.execute_script("arguments[0].data-checked = 'checked';", elems[1])
selenium.common.exceptions.JavascriptException: Message: javascript error: Invalid left-hand side in assignment
(Session info: MicrosoftEdge=109.0.1518.78)
To change set the value of data-checked attribute for all the accordion-panel elements as checked you can use the setAttribute() method as follows:
Code block:
driver.get('https://www.udemy.com/course/angular-10-fundamentos-8-app/')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#udemy > div.ud-main-content-wrapper > div.ud-main-content > div > div > div.paid-course-landing-page__container > div.paid-course-landing-page__body > div > div:nth-child(3) > div > button"))).click()
elements = driver.find_elements(By.CSS_SELECTOR, 'div.accordion-panel--panel--24beS > span')
for element in elements:
driver.execute_script("arguments[0].setAttribute('data-checked', 'checked')", element)
Browser Snapshot:
xxx, yyy is the things that i want to access with css selector in selenium
xxx=driver.find_element(By.CSS_SELECTOR,'#contents > div.tabWrap.pdtTabWrap.fixed > div.tabContents > section.tabCont.active >
div > div > div.prdDetailConWrap > div.prdType.prdType11 > div.imgWrap.imgCrop > img')
yyy=driver.find_element(By.CSS_SELECTOR,'#contents > div.tabWrap.pdtTabWrap.fixed > div.tabContents > section.tabCont.active >
div > div > div > div.prd_sec.prd_top_type01.sec01.mt0 > div > div.top_img_box > img')
xxx, yyy look similar
is it possible to access similar things(same start point, different middle point, same end point)
with just one line using descendant(>>)?
i ran
driver.find_element(
By.CSS_SELECTOR,'#contents > div.tabWrap.pdtTabWrap.fixed > div.tabContents > section.tabCont.active >> img')
but error occured
Try using a space instead:
driver.find_element( By.CSS_SELECTOR,'#contents > div.tabWrap.pdtTabWrap.fixed > div.tabContents > section.tabCont.active img')
You can read more here:
https://www.w3.org/TR/selectors/#descendant-combinators
I am getting Error while scraping data from a site please if anyone could help me with that
my Code
html = requests.get('https://www.cryptocompare.com/coins/btc/influence/USDT').text
soup = BeautifulSoup(html, 'html.parser')
total_commit = soup.select_one(' # col-body > div > social-influence > div.row.row-zero.influence-others > div:nth-child(2) > div > div > div > div.col-md-3.td-col.brd-right > div > div.repo-tag > span > span > a').text
print(total_commit)
error
soupsieve.util.SelectorSyntaxError: Malformed id selector at position 2
line 1:
# col-body > div > social-influence > div.row.row-zero.influence-others > div:nth-child(2) > div > div > div > div.col-md-3.td-col.brd-right > div > div.repo-tag > span > span > a
^
and also if anyone can tell me how to use the Css selectors which we copy directly from inspect element in bs4.
As mentioned by David Miró removing whitespace will fix the error but to get your goal you have to deal with selenium
Selenium will render the website and you can inspect the page_source and select your Element with bs4:
soup.select_one('div.repo-tag a')['href']
Example
from bs4 import BeautifulSoup
from selenium import webdriver
driver = webdriver.Chrome('YOUR PATH TO DRIVER')
driver.get('https://www.cryptocompare.com/coins/btc/influence/USDT')
soup=BeautifulSoup(driver.page_source, 'html.parser')
soup.select_one('div.repo-tag a')['href']
Output
https://github.com/bitcoin/bitcoin
Try removing space between # and col-body.
html = requests.get('https://www.cryptocompare.com/coins/btc/influence/USDT').text
soup = BeautifulSoup(html, 'html.parser')
total_commit = soup.select_one('#col-body > div > social-influence > div.row.row-zero.influence-others > div:nth-child(2) > div > div > div > div.col-md-3.td-col.brd-right > div > div.repo-tag > span > span > a').text
print(total_commit)
But it doesn't work because a part of the html is generated by javascript. So, you need to simulate that you are a web browser (for example with Selenium):
<div class="col-body col-body-new" id="col-body" ui-view>
<div class="loader-ccc">
<div class="loader-ccc-logo"></div>
<div class="loader-ccc-sides"></div>
</div>
In the web browser information exists:
I need to get this while loop to continue once element is visible, then click on that element.
tried using expected conditions, but I am a noob so I may be doing something wrong.
# wait until apply clickable
apply = wait(driver,10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "body > associate > div > adjust-schedule > div > list-adjustments > div > basecard > div:nth-child(3) > div > fieldset > div > table.table.table-striped > tbody > tr:nth-child(1) > td:nth-child(6) > button")))
while True:
try:
apply = driver.find_element_by_css_selector("body > associate > div > adjust-schedule > div > list-adjustments > div > basecard > div:nth-child(3) > div > fieldset > div > table.table.table-striped > tbody > tr:nth-child(1) > td:nth-child(6) > button")
apply.click()
except:
ignored_exceptions = (NoSuchElementException, StaleElementReferenceException,)
clckfilter = wait(driver, 10, ignored_exceptions=NoSuchElementException)\
.until(EC.presence_of_element_located((By.XPATH, "/html/body/associate/div/adjust-schedule/div/list-adjustments/div/basecard/div[2]/div/fieldset/div/table[1]/thead/tr/th[4]/button")))
clckfilter.click()
apply.click()
driver.find_element_by_css_selector("body > associate > div > adjust-schedule > div > list-adjustments > div > basecard > div:nth-child(3) > div > fieldset > div > table.table.table-striped > tbody > tr:nth-child(1) > td:nth-child(6) > div > button.btn.btn-primary").click()
# if: EC.presence_of_element_located(By.CSS_SELECTOR, "body > associate > div > adjust-schedule > div > list-adjustments > div > basecard > div:nth-child(3) > div > fieldset > div > table.table.table-striped > tbody > tr:nth-child(1) > td:nth-child(6) > button"): continue
If I remove the apply.click and following line, the program will loop like I need, but I can't figure out how to get it to loop until the apply button is visible. (It becomes visible once my work posts a shift)
Try this:
def click():
try:
wait = WebDriverWait(self.browser, 15)
wait.until(EC.visibility_of_element_located(“selector”))
driver.find_element_by_css_selector(“selector”).click()
except:
Click()
In the above code I have created a separate method for clicking on that element in "try" i have write the code for clicking on that element and if the element not visible than in except section the same method call again until the element visible.
I am writing an automation script for
http://computer-database.gatling.io/computers/new
When an invalid value (or no computer name) is entered and save is clicked an error indicator appears (the field turns red)
However I can't get Selenium to find the error indicator
I have not worked with this type of error before so not sure what to look for
my code
Locator
computer_name_required = (By.CSS_SELECTOR, "#main > form > fieldset > div:nth-child(1) > div")
invalid_intro_date = (By.CSS_SELECTOR, "#main > form > fieldset > div:nth-child(2) > div")
invalid_dscon_date = (By.CSS_SELECTOR, "#main > form > fieldset > div:nth-child(3) > div)")
using element is displayed
def element_displayed(self, *element):
element = self.browser.find_element(*element)
if element.is_displayed():
return True
else:
return False
The invalid name method
def invalid_name_error(self):
return self.element_displayed(*CreateAndEditPageLocators.computer_name_required)
Since the class name changes when an error occurs you could use the xPath instead of css selector
".//*[#class='clearfix error']//*[#for='name']" #for computer name error
".//*[#class='clearfix error']//*[#for='introduced']" #for introduced error
".//*[#class='clearfix error']//*[#for='discontinued']" #for discontinued error