I made a code to scrape some website. A list of IDs is iterated in the website, and it contains two conditions(If and Elif). But the problem is with the Elif. The error is it doesn't found the elif element (elem2).
I read in this question Python if elif else can't go to the elif statement Selenium the solution is a try/except, butI already used a Try/except to make works the if statement. What is a solution to make this code works with two conditions?
The code looks like this:
for item in list:
input = driver.find_element(By.ID, "busquedaRucId")
input.send_keys(item)
time.sleep(2)
elem1 = driver.find_element(By.ID, 'elem1')
elem1_displayed = elem1.is_displayed()
elem2 = driver.find_element(By.ID, 'elem2')
elem2_displayed = elem2.is_displayed()
try:
if elem1_displayed is True:
code to scrape given de first condition
elif elem2_displayed is True:
code to scrape given de second condition
except NoSuchElementException:
input = driver.find_element(By.ID, ('busquedaRucId')).clear()
Than you for any help. I'm stuck with this problem for two weeks.
I would restructure your code by wrapping the find_element function in a function which handles NoSuchElementExceptions by returning False, basically making the error silent:
def element_exists_and_displayed(driver, id):
try:
return driver.find_element(By.ID, id).is_displayed()
except NoSuchElementException:
return False
for item in list:
input = driver.find_element(By.ID, "busquedaRucId")
input.send_keys(item)
time.sleep(2)
if element_exists_and_displayed(driver, 'elem1'):
# code to scrape given first condition
pass
elif element_exists_and_displayed(driver, 'elem2'):
# code to scrape given second condition
pass
else:
driver.find_element(By.ID, ('busquedaRucId')).clear()
I'm trying to use Wikipedia to determine a person's gender based on the user's input (person's name). However, my code only goes so far before it gives an error or shuts down.
def find_gender(name):
# Using Wikipedia to find information
url = "https://www.wikipedia.org/"
driver.get(url)
driver.find_element("xpath", '//*[#id="searchInput"]').send_keys(name)
driver.find_element("xpath", '//*[#id="search-form"]/fieldset/button').click()
def get_person():
# Using first paragraph to find pronouns
description = driver.find_element("xpath", '//*[#id="mw-content-text"]/div[1]/p[2]').text
print(description + "\n")
# Checking to see if this is the name in question
answer = input(f"Is this the {name} you are searching for? Yes or no: ")
"""This is seems to be the point where the program stops"""
def get_gender():
if answer.lower() == 'yes':
pronouns = ['she', 'her', 'he', 'his']
# Turn the paragraph into an array of strings
words = description.split()
# Returns gender
for word in words:
for pronoun in pronouns:
if word == pronoun:
if word == 'she' or word == 'her':
return 'Female'
elif word == 'he' or word == 'his':
return 'Male'
else:
print("Unknown gender. Make sure that your input is a real person.")
return get_person()
# In the case of the wrong person
elif answer.lower() == 'no':
# Find the disambiguation page and switch to it
disambig = driver.find_element(By.CLASS_NAME, 'mw-disambig').get_attribute('href')
driver.get(disambig)
# Use the last name to search in HTML tags
first, last = name.split()
elements = driver.find_elements(By.PARTIAL_LINK_TEXT, last)
e_dict = {}
# Make dictionary of the persons and their references
for i in range(1, len(elements)):
e_dict[i] = [elements[i - 1].text, elements[i - 1].get_attribute('href')]
for key, value in e_dict.items():
print(f"{key}. {value[0]}")
# Ask which reference they would like to go to
ans = input(f"Which {name} would you like to visit? Pick a number: ")
def validate_input():
if ans in e_dict.keys():
driver.get(e_dict[ans][1])
get_person()
return get_gender()
else:
ans = input("Sorry, your input is invalid. Please choose again: ")
validate_input()
# In the case that input isn't yes or no
else:
answer = input("Please say yes or no: ")
get_gender()
get_person()
gender = find_gender("Lionel Messi")
Basically, the program goes to Wikipedia and types in the user's request. Then, the driver finds the first paragraph and asks the user if it's the right person. Then, depending on what the user says, the program intends to determine the gender of the person in question.
Why does the program stop there? Also, are the nested functions used properly?
Also, I did import Selenium.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.by import By
Code:
def funt():
print(Fore.GREEN, end='')
tool = input('Enter Desired Tool: ')
if tool == 'web':
try:
print(Fore.CYAN, end='')
site = input('Please Enter The Website Here: ')
response = requests.get(site)
requests.get(site)
if response.status_code == 200:
print(f'{Fore.GREEN}Online!')
sleep(1)
else:
print(f'{Fore.RED}Offline!')
sleep(1)
while True:
funt()
Location of Error is while True: .
The Error Is As Follows:
while True:
^
IndentationError: unexpected unindent
I cannot find a solution, there is not any sign of indentation in the while loop.
try expects an except block followed by it.
You can modify your code as follows:
def funt():
print(Fore.GREEN, end='')
tool = input('Enter Desired Tool: ')
if tool == 'web':
try:
print(Fore.CYAN, end='')
site = input('Please Enter The Website Here: ')
response = requests.get(site)
requests.get(site)
if response.status_code == 200:
print(f'{Fore.GREEN}Online!')
sleep(1)
else:
print(f'{Fore.RED}Offline!')
sleep(1)
except:
pass
while True:
funt()
But writing proper code requires you to handle exceptions. So, if possible write a piece of code in the except block.
You've missed except block for your try block
Here's some info on exception handling in python.
Do this:
def funt():
print(Fore.GREEN, end='')
tool = input('Enter Desired Tool: ')
if tool == 'web':
try:
print(Fore.CYAN, end='')
site = input('Please Enter The Website Here: ')
response = requests.get(site)
requests.get(site)
if response.status_code == 200:
print(f'{Fore.GREEN}Online!')
sleep(1)
else:
print(f'{Fore.RED}Offline!')
sleep(1)
# You were missing this part:
except:
print("Message")
while True:
funt()
A try block always goes together with a catch block. The purpose of the try block is to attempt to run code that may throw an exception. The catch block is what catches this exception.
I want to know if there is any way to log the number of times my page has refreshed in command prompt when running.
want it to tell me the number of times it has refreshed. Refresh is located between while true: and continue. thanks
driver = webdriver.Chrome(chrome_path)
driver.get(link)
while True:
size = driver.find_elements_by_xpath(".//*[#id='atg_store_picker']/div/div[2]/div[1]/div[1]/span[2]/a[4]")
if len(size) <= 0:
time.sleep(0.5)
print "PAGE NOT LIVE"
driver.refresh()
continue
else:`enter code here`
print 'LIVE!!!!'
break
the answer to my question was very simple...
driver = webdriver.Chrome(chrome_path)
driver.get(link)
count = 0
while True:
size = driver.find_elements_by_xpath(".//*[#id='atg_store_picker']/div/div[2]/div[1]/div[1]/span[2]/a[4]")
if len(size) <= 0:
count +=1
print 'Refresh Count:', count
time.sleep(2)
driver.refresh()
continue
else:
print 'LIVE!!!!'
break
I am trying to submit an input(type= button).But I am unable to update the value.
Any help is appreciated.
I have attached the testcase below for your reference.
search for CLICK FAILS HERE
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import Select
from selenium.common.exceptions import NoSuchElementException
import unittest, time, re,datetime,os,sys
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.action_chains import ActionChains
def is_element_present(inst,selector,value):
try:
inst.find_element(by=selector, value=value)
return True
except:
return False
class Testing(unittest.TestCase):
def setUp(self):
self.driver = webdriver.Chrome()
self.driver.implicitly_wait(5)
self.base_url = "http://new.ossmoketest.appspot.com/"
self.verificationErrors = []
def test_ing(self):
try:
driver = self.driver
driver.get(self.base_url + "/Apab4b39d4_09d7_11e1_8df9_139372201eeb/1/signin?forward=/%3F")
now = datetime.datetime.now()
start = time.clock()
for i in range(5000000):
try:
if is_element_present(driver,By.ID,"userid"): break
except: pass
else: self.fail("time out")
end = time.clock()
diff = end - start
print diff
driver.find_element_by_id("userid").clear()
driver.find_element_by_id("userid").send_keys("senthil.arumugam#orangescape.com")
driver.find_element_by_xpath("//input[#src='/static/images/signup.png']").click()
print 'finished'
start = time.clock()
for i in range(5000000):
try:
if is_element_present(driver,By.LINK_TEXT,"Logout"): break
except: pass
else: self.fail("time out")
end = time.clock()
diff = end - start
print diff
time.sleep(5)
start = time.clock()
name = "smoketest"+ str(now.minute) +str(now.second)
for i in range(5000000):
try:
if is_element_present(driver,By.LINK_TEXT,"PurchaseOrder"): break
except: pass
else: self.fail("time out")
end = time.clock()
diff = end - start
driver.find_element_by_link_text('PurchaseOrder').click()
name = "smoketest"+ str(now.minute) +str(now.second)
start = time.clock()
for i in range(5000000):
try:
if is_element_present(driver,By.ID,"Customer_Name"): break
except: pass
else: self.fail("time out")
end = time.clock()
diff = end - start
newproduct = "rexona"+ str(now.minute) +str(now.second)
newprice = str(now.minute) +str(now.second)
newprice = float(newprice)
print newprice
driver.find_element_by_xpath("//input[starts-with(#id,'New_Product')]").send_keys(newproduct)
driver.find_element_by_xpath("//input[starts-with(#id,'Price')]").clear()
time.sleep(3)
driver.find_element_by_xpath("//input[starts-with(#id,'Price')]").send_keys(Keys.CONTROL+'a'+Keys.NULL, str(newprice))
Mouse_cntrl = ActionChains(driver)
Mouse_cntrl.release(driver.find_element_by_xpath("//input[starts-with(#id,'Price')]"))
value = newprice
print value
time.sleep(2)
print 'start'
print driver.find_element_by_xpath("//input[starts-with(#id,'NewItem_NewItem')]").get_attribute('data-id')
# ------------------------CLICK FAILS HERE ------------------------------
# driver.find_element_by_xpath("//input[starts-with(#id,'NewItem_NewItem')]").click()
# driver.find_element_by_xpath("//input[starts-with(#id,'NewItem_NewItem')]").submit()
driver.find_element_by_xpath("//input[starts-with(#id,'NewItem_NewItem')]").send_keys(keys.ENTER)
# Mouse_cntrl.double_click(driver.find_element_by_xpath("//input[starts-with(#id,'NewItem_NewItem')]"))
for i in range(10):
try:
print driver.switch_to_alert().text
if driver.switch_to_alert():
driver.switch_to_alert().accept()
break
except: pass
time.sleep(1)
else:
print "alert not found"
print 'finished -- '
time.sleep(8)
driver.find_element_by_xpath("//input[starts-with(#id,'Product')]").click()
arg = newproduct
print 'end'
for i in range(60):
try:
if is_element_present(driver,By.LINK_TEXT,arg): break
except: pass
time.sleep(1)
else: self.fail("time out")
# sel.mouse_over("//html/body/ul/li/a[.=\""+arg+"\"]")
driver.find_element_by_link_text(arg).click()
start = time.clock()
time.sleep(25)
for i in range(1000000):
try:
if newprice == float(driver.find_element_by_id('Unit_Price').text):
end = time.clock()
diff = end - start
log.log(module='Smoke',testcase='Action New', result='Pass',time_taken= diff)
break
except: pass
else:
log.log(module='Smoke',testcase='Action New', result='Fail')
self.fail('New Failure')
log.log(module='Smoke',testcase='On Submit', result='Pass',time_taken= diff)
driver.find_element_by_id('Quantity').send_keys(Keys.CONTROL+'a'+Keys.NULL,"1")
time.sleep(5)
start = time.clock()
for i in range(1000000):
try:
if value == float(driver.find_element_by_id('Unit_Price').text):
end = time.clock()
diff = end - start
log.log(module='Smoke',testcase='Multiply', result='Pass',time_taken= diff)
break
except: pass
else: self.fail("time out")
for i in range(1000000):
try:
if value == float(driver.find_element_by_id('Amount').text):
end = time.clock()
diff = end - start
log.log(module='Smoke',testcase='DSUM with Parent', result='Pass',time_taken= diff)
break
except: pass
else:
end = time.clock()
diff = end - start
log.log(module='Smoke',testcase='DSUM with Parent', result='Fail',time_taken= diff)
self.fail("time out")
except:
self.driver.quit()
e = sys.exc_info()[1]
print str(e)
def tearDown(self):
self.driver.quit()
self.assertEqual([], self.verificationErrors)
if __name__ == "__main__":
unittest.main()
It has been a showstopper for my work. Any help is appreciated.Thanks
You could try substituting .click() with .send_keys("\n"), which is equivalent to "Pressing enter while focusing on an element".
So this:
driver.find_element_by_link_text('PurchaseOrder').click()
would become this:
driver.find_element_by_link_text('PurchaseOrder').send_keys("\n")
In case this is still a recurring problem for anyone else, if you have confirmed your code is correct (you've reviewed it for errors etc.) and you still find the find_element_by_...('text').click() function not working properly it is often due to your code continuing to run before the JavaScript can update the page.
A simple solution is to import time then insert the below code immediately after any click() methods:
time.sleep(2)
The duration of the sleep timer can be whatever you choose. In my case I used 2 seconds. Hopefully that helps.
I had this problem as well. Sometimes, for whatever reason webdriver didn't click the button. It was able to find the button (it didn't throw a NoSuchElementException and a WebDriverWait didn't help).
The problem with clicking the button twice was that if the first click succeed, the second one would fail (or click the submit button on the next page if it found a match!). My first attempt was to put the second click in a try/except block - this is how I found out about it clicking submit on the next page. XD And it also really slowed down my test when it couldn't find the second button.
I found some good insights at Selenium 2.0b3 IE WebDriver, Click not firing. Basically, I click on a parent element first, which seemingly does nothing. Then I click on the submit button.
If the element you click() is an url. I found that taking the href properties and using driver.get(elem.get_attribute('href')) being the cleanest.
I would try other element finders like className, cssSelector or something. xPath sometimes doesnt provide errors if the element isn't found.
So first start by finding out if the element is really found by webdriver.
You can also try to click or use the other commands two times in a row. This already solved some of such issues.
I had the same issue where a two-part drop down menu selection would not generate what it's supposed to generate with proper selections. It worked when I did imported time and use time.sleep(2) between the two "click"s. (For reference I used find_element_by_xpath to find an modify the options.)
I ran into the above issue where the same .click() is working in all browsers(IE, Chrome and Firefox) but not working for Safari. So I tried all the possible solutions from this post and none of them worked and it doesn't even throw error.
I tried below by substituting .click() with .submit(), which is an equivalent method and it worked not only for Safari but also for all other browsers.
So this:
driver.find_element_by_id('login_btn').click()
Replaced as:
driver.find_element_by_id('login_btn').submit()
If above fails to work on other browsers due to any reason keep .submit() in try and except. If it fails .click() will be triggered and executed.
Hope this helps if none of the other solutions works.
For those, click() doesn't work, use submit() if that button element(clickable element) is in a form element. Basically, in order to submit a form, we need to use submit(), click() will not work in some cases.
I foolishly did elem.click with the parentheses: elem.click().