I'm using selenium in Python to try and scrape multiple pages. ID's and XPATH's keep changing per page, so I figured I'd best access them through their attribute-value combinations (see below).
I'm trying to access the text in the following element:
https://i.stack.imgur.com/ly1YU.png
which belongs to the following:
https://i.stack.imgur.com/strep.png
As I said, the ID's keep changing, so I wanted to access the element by data-fragment-name="articleDetail", or data-testid = "article-body". Can somebody help me how to do so?
Thanks in advance!
Try using the following CSS_SELECTOR
div[data-fragment-name='articleDetail'] div[data-testid='article-body']
Or XPath
//div[#data-fragment-name='articleDetail']//div[#data-testid='article-body']
The Selenium command can look like:
driver.find_element(By.CSS_SELECTOR, "div[data-fragment-name='articleDetail'] div[data-testid='article-body']")
Or
driver.find_element(By.XPATH, "//div[#data-fragment-name='articleDetail']//div[#data-testid='article-body']")
from selenium.webdriver.common.by import By
obj = driver.find_element(By.XPATH, "//div[#data-fragment-name='articleDetail']")
obj2 = driver.find_element(By.XPATH, "//div[#data-testid='article-body']")
where of course driver = webdriver.Firefox() or something like that and you already moved to the desired page.
Related
This is HTML code of that page
From there I want to access the 2nd element by using class name "maxbutton-1" as it has 3 same buttons and I can't use xpath or any constant selector so want to use the indexing with class and can't find anything to do that in python particular.
Also tried the method used in java to do same thing but it didn't worked.
Link of that same page
just trying to automate the movie downloading process for any movie.
Thank you.
To click on first, second or third button, try to change number of element:
el1 = driver.find_element_by_xpath("(//a[#class='maxbutton-1 maxbutton maxbutton-download-links'])[1]")
el2 = driver.find_element_by_xpath("(//a[#class='maxbutton-1 maxbutton maxbutton-download-links'])[2]")
el3 = driver.find_element_by_xpath("(//a[#class='maxbutton-1 maxbutton maxbutton-download-links'])[3]")
then you can extract element href/link attribute like that:
link = el.get_attribute('href')
or click it like that:
el.click()
#Thanks in advance for help. New to python, tried for hour trying to correct mistake.#
Trying to locate login button element. Attached is the image of the website with the element of the login button. please see here
Below is code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
url = "https://www.xxxxxx.com/index.php/admin/index"
username = 'xxxx'
password = 'xxxxx'
driver = webdriver.Firefox(executable_path="C:\\Users\\kk\\AppData\\Local\\Programs\\Python\\Python38-32\\geckodriver.exe")
driver.get(url)
driver.find_element_by_name(name='aname').send_keys(username)
driver.find_element_by_name(name='apass').send_keys(password)
driver.find_elements_by_xpath('//style[#type="submit"]')
Rather than finding it with a CSS selector. Why not use find_element_by_xpath()
To get the XPath of that element just right-click the HTML of the input in Inspect Element, hover over Copy and you'll see "Full XPath"
Issue is your xpath.
driver.find_elements_by_xpath('//style[#type="submit"]')
Use below:
driver.find_elements_by_xpath('//input[#type="submit"]')
or
driver.find_elements_by_xpath('//input[#value="login"]')
#This is more accurate as many input tags could have type as submit
Also, please use some sort of wait as i am not sure if page will be loading fast enough every time you launch URL.
You can identify the submit button by using any of these 2:
//input[#type="submit"] or //input[#value="login"]
They should work without any problem if you don't have any similar elements on your page (which I doubt)
But if you want to be more precise, you can mix these 2 into:
//input[#value="login" and #type="submit"]
I am trying to get Selenium to do a Youtube search for me and I got to the final step which is actually entering the text and I'm stuck. searchElem uses the id that allows me to use .click(), but I cannot use send_keys() with that id so I tried inputElem and that doesn't work either.
There isn't a more specific id or class to use for the search input so I'm not sure what to do.
Below is the error I get once I try to use send_keys() with searchElem and inputElem.
selenium.common.exceptions.ElementNotInteractableException: Message: Element <div id="search-input" class="ytd-searchbox-spt"> is not reachable by keyboard
This is my code for the WebDriver and elements within the source code.
driver = webdriver.Firefox()
driver.get('https://www.youtube.com')
searchElem = driver.find_element_by_id('search-input')
inputElem = driver.find_element_by_id('search')
searchElem.click()
searchElem.send_keys('election')
It's possible that 'search' is the ID of a different element that is being picked up, so try using the xpath instead.
inputElem = browser.find_element_by_xpath('/html/body/ytd-app/div/div/ytd-masthead/div[3]/div[2]/ytd-searchbox/form/div/div[1]/input')
inputElem.send_keys('election')
You can create the locator based on search field placeholder (this would make it unique)
For example, for english, the placeholder would be "Search" and your locator would look like:
//input[#placeholder='Search']
I am very new to using selenium but I cannot find way around a very simple task.
I need to be able to click on the element that specifies bedrooms: 2.
I have used I don't know how many references by xpath, by id, by name, by class but selenium just won't find the element. I also have tried to browse the internet but could not find solutions that help me.
Here's the sanpshot
For instance: using
driver.find_element_by_id('agatha_bedrooms1588844814480_advancedSearch1').click()
This won't work. Selenium cannot find the element. It seems that this element is within another element but I don't understand how to access it.
Could you help me please?
Thanks a lot to you.
G
The ids seem to be dynamically generated, in which case you cannot rely on them. Try with this xpath:
driver.find_element_by_xpath("//*[#name='bedrooms' and #value='2']/following::label").click()
Although it is generally good practice to work with waits. So something like:
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//*[#name='bedrooms' and #value='2']/following::label"))).click()
Ensure to have these imports for the wait to work
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
Thanks a lot, after multiple and multiple trials I could get around that way:
elemt = driver.find_element_by_xpath("//*#name='bedrooms']").find_element_by_xpath("//[#value='2']")
idvar = elemt.get_attribute("id")
elemt2 = driver.find_element_by_xpath("//label[#for='" + idvar + "']")
elemt2.click()
It seems that the checkbox was hidden under the label (?!) so that Selenium did not want to click on it.
If the checkbox is inside of an iframe, do this:
# basically just select the iframe any way you want
frame = driver.find_element_by_css_selector("iframe")
driver.switch_to.frame(frame)
driver.find_element_by_id('agatha_bedrooms1588844814480_advancedSearch1').click()
edit:
I've found solution. Kinda ugly but works lol
element = driver.find_elements_by_css_selector("input[name=bedrooms][value='2']")[0]
element.find_element_by_xpath("..").click()
You can try this xpath. hope its helps:
//*[#name='bedrooms']/following::*/*[text()='2']
from selenium import webdriver
browser=webdriver.Firefox()
browser.get("http://dollarupload.com/dl/08c646d60")
browser.find_element_by_id("reg_download").click()
elementlist=browser.find_elements_by_class_name("offer_title")
Actually I was trying to get all the class named offer_title and with that I would like to click the link.But as I can see elementlist is empty.Why?
I think the elements you want are in an iframe to a different page.
If you get the iframe element and try again from there you might have more luck.
elementlist=browser.find_element_by_tag_name("iframe").find_elements_by_class_name("offer_title")