I'm having an odd problem trying to web scrape some data from ESPN. I have the below code that sometimes works as intended, but sometimes will get hung up trying to log in. It really seems random, and I'm not sure what's going on.
import time
from selenium import webdriver
from selenium.common.exceptions import TimeoutException
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
driver = webdriver.Chrome(r'Source')
driver.get("http://games.espn.go.com/ffl/signin")
WebDriverWait(driver,1000).until(EC.presence_of_all_elements_located((By.XPATH,"(//iframe)")))
frms = driver.find_elements_by_xpath("(//iframe)")
driver.switch_to_frame(frms[2])
time.sleep(2)
WebDriverWait(driver,10).until(EC.presence_of_element_located((By.XPATH,'(//input)[1]')))
driver.find_element_by_xpath("(//input)[1]").send_keys("Username")
driver.find_element_by_xpath("(//input)[2]").send_keys("password")
driver.find_element_by_xpath("//button").click()
driver.switch_to_default_content()
time.sleep(2)
When I run the code as is, it often times times out during the second "WebDriverWait", despite the page having fully loaded in chrome. If I take that line out, I then will get an error message that reads:
"selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element:"
You check this xpath? Apparently it is not correct, you can try it on the browser console.
$x('yourxpath')
Related
I am writing script in python using selenium.
This code should switch focus to iframe on the webpage.
But instead, I get this error:
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: //iframe[#id="lightbox_iframe_4"]
here is the code:
iframe = browser.find_element("xpath", "//iframe[#id="lightbox_iframe_4"]")
browser.switch_to.frame(iframe)
Please help, how to get the iframe element?
Here are all modules:
from selenium import *
from time import sleep
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
Tried adding other modules, changing browser. to driver./webdriver.
I was having trouble with find elements by XPATH for a selenium script in Python.
Couldn’t understand the problem, i want to identify all of the elements that satisfy my xpath criteria.
So I simply compared the find_elements(By.XPATG, ‘//div’) to the same result when looking at Chrome’s Inspect tool.
The tool says 300+ results for this whereas my programme, which I got to return a count of results, only states 60? Any ideas? looks to be an thing from the browser/site side rather than my code.
Thanks
EDIT added my code:
import time
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.select import Select
from selenium.webdriver.support.ui import WebDriverWait
from selenium.common.exceptions import StaleElementReferenceException
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support import expected_conditions as EC
import login
print(time.time())
driver.get("WEBSITE URL")
WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//body")))
time.sleep(1)
links = driver.find_elements(By.XPATH,'//body')
linklist = len(links)
print(linklist)
This provides a result of around 60 however when '//body' is put in the inspect element tool, it returns 300+
Apologies but the targeted webpage is confidential so I can't post. But suggestions are appreciated so I can investigate.
Does your web page has a scroll? If yes, you may need to deal with scroll bar. Also try different conditions for WebDriverWait, for example EC.visibility_of_all_elements_located or EC.presence_of_all_elements_located
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome()
driver.get(updatedCarLinks[0]) #This has a list of links that I need to look but will but this in a for loop later
time.sleep(2)
h1 = driver.find_element_by_xpath('//h1')
print(h1)
I keep getting the following error screen. I have been looking at it on other posts and can't seem to figure out why this error is happening.
selenium.common.exceptions.WebDriverException: Message: target frame detached <--- Error
Is the selenium crome driver compatible with you chrome version? Check the versions match or is supported by the driver.
More information on what exact versions are used would be helpful
Python/coding newb, beware!
Im writing a script to download from youtube from " https://ytmp3.cc/en13/ ". I had written a click script using pyautogui but the problem being that the download button appears anywhere betweeen 1 and 15 seconds of entering the link. So i wanted to re-code it in a Selenium window to dynamically wait until the button is visible and then continue with the click script. I have tried about 15 ways but cant get it to work
library blah :
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support.expected_conditions import presence_of_element_located
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
ways i tried to get it to work:
without waits, results in NosuchElement Exception
def check_exists():
try:
while True:
a=driver.find_element(By.LINK_TEXT, "Download")
a.click()
except NoSuchElementException:
time.sleep(10)
the syntax of this is most probably wrong
i tried a couple different variations of implicit and explicit waits but couldnt get it to work. Is there some javascript on the site linked above thats messing with my code?
in a separate .py, running
#driver.find_element(By.LINK_TEXT, "Download").click()
#either the line above or below works at finding the element, and clicking it
driver.find_element_by_link_text("Download").click()
I just need help getting the line above ^ into an explicit wait, but i dont have the language knowledge of how to do it yet. i would appreciate some help, thanks in advance!
You can use webdriver wait to wait until button is displayed. Tried the below code which works for me
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
video_path = "your_video_path"
waitTime=10
driver = webdriver.Chrome("./chromedriver.exe")
driver.get("https://ytmp3.cc/")
element = driver.find_element_by_id("input")
element.send_keys(video_path)
driver.find_element_by_id("submit").click()
try:
# wait 10 seconds before looking for element
element = WebDriverWait(driver, waitTime).until(
EC.presence_of_element_located((By.LINK_TEXT,"Download"))
)
driver.find_element_by_link_text("Download").click()
finally:
print(driver.title)
You can update your required wait time in waitTime variable in seconds so if the value is 10 selenium will wait and check for elements upto 10 seconds
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from webdriver_manager.chrome import ChromeDriverManager
import chromedriver_binary
from selenium.webdriver.common.keys import Keys
import time
element = driver.find_element_by_xpath('//*[#id="username"]').send_keys(user_name)
This is how code looks like (I've tried xpath,id, class name, etc..) but I will ALWAYS get an error
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//[#id="username"]"}
(Session info: chrome=80.0.3987.87)
I am trying to get this done on here
Any ideas please?
I noticed that the given form is within Iframe so switch to iframe is required to interact with any element which is within IFrame.
Following code works for me :
driver.switch_to_frame(driver.find_element_by_xpath('(//iframe[#title="Registration form"])[1]'))
element = driver.find_element_by_xpath('//*[#id="username"]').send_keys("username")
Feel free to use explicit wait if require.