I'm trying to locate and click on a section of a web page using Selenium so I can add a comment. I'm having a bit of trouble figuring out how to do this, though.
It seems like the class of the element changes from page-to-page. It also doesn't help that there are multiple similar elements in the document. Here's what I've come up so far:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("--user-data-dir=/Users/me/Library/Application/Support/Google/Chrome/Default")
chrome_options.add_argument('--profile-directory=Profile 1')
driver = webdriver.Chrome(chrome_options=chrome_options)
driver.get("https://link-i-want-to-visit")
comment_section = driver.find_elements_by_xpath(("//input"))
print comment_section
comment_section.click()
Here's some of the markup from the relevant page:
<input class="sc-iKpIOp igoGaM" placeholder="Add a comment…">
On each different URL on this site, the class name appears to change. How can I circumvent that limitation, click into the input field, and send my comment?
Any guidance would be much appreciated. If it helps, this input field appears to be the last on the page, but I don't know if that's relevant (semantically).
Can't you use absolute xpath not including class?
e.g. /html/body/div/div/div/div/div/div/div/div/div/p[1] is your "I'm trying to..." paragraph on this page.
Also, try //input[placeholder="Add a comment…"]
comment_section = driver.find_elements_by_xpath("//input")
This will return list.So you can't click on list. You should use driver.find_element_by_xpath("//input") to click on the element.
However for a best practice Use WebDriverWait and Wait for the element element_to_be_clickable
and then click.
comment_section=WebDriverWait(driver,30).until(EC.element_to_be_clickable((By.XPATH,"//input[#placeholder='Add a comment…']")))
comment_section.click()
OR
comment_section=WebDriverWait(driver,30).until(EC.element_to_be_clickable((By.XPATH,"//input[contains(#placeholder,'Add a comment')]")))
comment_section.click()
You need to use following imports to execute above code.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
Related
UPDATE:
So thanks to the voted answer it displayed some information not the right information, it shows 0kb out of 100 and when in the inspect element console if doing console.log($0) then the item would be displayed in console how do I fetch this
I want to create a python 3.x programme that gets my stats off of netlify and easybase using selenium. The issue I have come across already is that the element does not have a specific class name and the text widget isn't just a tag nor a tag. Here is a screenshot of the html of netlify the screenshot, and this is the code that I used
element = driver.find_element_by_name("github")
element.click()
login = driver.find_element_by_name("login")
login.send_keys(email)
password = driver.find_element_by_name("password")
password.send_keys(passwordstr)
loginbtn = driver.find_element_by_name("commit")
loginbtn.click()
getbandwidth = driver.find_element_by_xpath('//*[#id="main"]/div/div[1]/div/section/div/div/div/dl/div/dd')
print(getbandwidth.text)
getbandwidth = driver.find_element_by_xpath("//dd[#class='tw-text-xl tw-mt-4px tw-leading-none']")
You can use this to grab the first xpath with that class. Below does the same but if you want to index other elements with similar classes.
(//dd[#class='tw-text-xl tw-mt-4px tw-leading-none'])[1]
Normally we use webdriver waits to allow for the element to become visible.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
getbandwidth = WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH,"//dd[#class='tw-text-xl tw-mt-4px tw-leading-none']")))
I've looked around and I tried using different versions of WebElement userid = driver.findElement(By.xpath and WebElement email = driver.findElement(By.id("email")); but they return a syntax error when I try to use them.
For reference, this is the textbox I want to type in
and this is the html xpath I found from using inspect element on that box.
This is my first time using python & selenium so please be easy on me :)
EDIT: I'm now on the dropdown menu part of this script and I don't know how to use the select function
HTML
So, based on your HTML code, I think that the xpath that you need is this
//div[#data-section='customer-information']//input[#id='checkout_email']
Based on this xpath, you could write your code as such
sampleEmail = "MySampleEmail#Email.com"
driver.find_element(By.XPATH, "//div[#data-section='customer-information']//input[#id='checkout_email']").send_keys(sampleEmail)
if(driver.find_element(By.XPATH, "//div[#data-section='customer-information']//input[#id='checkout_email']").get_attribute('value').__len__() != sampleEmail.__len__()):
raise Exception(f'Failed to populate our e-mail textbox with our value: {sampleEmail}')
To wait for an input tag with class field__input to be able to load in and send keys to it
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.field__input"))).send_keys("ANYTHING")
Import
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
I am trying to login to a website using python so that I can get some of their text from the website.
Here is my code. There always an error at the end of the code after the id and password code.
import os
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
driver = webdriver.Chrome(executable_path=r"C:\chromedriver\chromedriver.exe")
driver.get('https://www.saramin.co.kr/zf_user/auth')
driver.implicitly_wait(3)
driver.find_element_by_name('id').send_keys('<<my_id>>')
driver.find_element_by_name('password').send_keys('<<my_password>>')
driver.find_element_by_xpath('//*[#id="frmNIDLogin"]/fieldset/input').click()
HTML source of the button:
Eventually I figured it out! Thanks for your answer though.
Here is the final code.
driver = webdriver.Chrome(executable_path="C:\chromedriver\chromedriver.exe")
browser = webdriver.Chrome('C:\chromedriver\chromedriver.exe')
driver.get('https://www.saramin.co.kr/zf_user/auth')
driver.implicitly_wait(3)
driver.find_element_by_name('id').send_keys('ID') driver.find_element_by_name('password').send_keys('PW')
driver.find_element_by_xpath( '//*[#class="btn-login"]' ).click()
The xpath is incorrect, it doesn't match anything in the page. Try
driver.find_element_by_xpath('//form[#id="login_frm"]//button[#class="btn-login"]').click()
or simply use submit() function on the <form>
form = driver.find_element_by_id('login_frm')
form.submit()
In the first case you were using 'id' ('//*[#id="frmNIDLogin"]) for click button, because 'id' changes every time page loads it was giving error. But in the second case when you used class ( '//*[#class="btn-login"]' ) it worked because it remains same every time page is loaded. Also as mentioned above the value of id in first case was wrong.
Using Python27:
I am trying to automate a search and extract method using beautifulsoup to parse and input data on this database server. So far I have managed to get Python to login to it. But now when I try to search for the input element to make a search, I can't seem to get the identifier/code correct.
The highlighted in blue code says:
<input id="QUICKSEARCH_STRING" type="text" on focus="setTimeout('focusSearchElem()',100...
The highlighted portion in blue is where I believe I need to search for that element in order to input text then search. I think the rest, like inputting the results I get from the page might be a bit easier.
My code is as follows:
from selenium import webdriver
browser = webdriver.Firefox()
browser.get('http://somewebpage')
emailElem = browser.find_element_by_id('j_username')
emailElem.send_keys('blahblahblah')
passwordElem = browser.find_element_by_id('j_password')
passwordElem.send_keys('blahblahblah!')
login_form = browser.find_element_by_xpath("//a[#id='login']").click()
searchElem = browser.find_element_by_id('search_panel')
searchElem.send_keys('blahblahblah')
I'm not sure where I'm going wrong, but I think I am close.
browser.find_element_by_id('search_panel')
I don't see any elements with id="search_panel".
Here is how I would locate the desired input element:
browser.find_element_by_id("QUICKSEARCH_STRING")
browser.find_element_by_css_selector("div.search_panel input#QUICKSEARCH_STRING")
You may need to wait for it to become present after the logging in:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
wait = WebDriverWait(driver, 10)
search_input = wait.until(EC.presence_of_element_located((By.ID, "QUICKSEARCH_STRING")))
search_input.send_keys('blahblahblah')
I'm very new to Selenium.
I'm crawling data from this page. I need to scroll down the page and click on "Load More Arguments" to get more text. This is the location to click on.
<a class="debate-more-btn" href="javascript:void(0);" onclick="loadMoreArguments('15F7E61D-89B8-443A-A21C-13FD5EAA6087');">
Load More Arguments
</a>
I have tried this code but it does not work. Should I need more code to locate to that (I think the 1 has already tell the location to click). Do you have any recommendation? Thank you in advance.
[1] btn_moreDebate = driver.find_elements_by_class_name("debate-more-btn")
[2] btn.click()
Find the link by link text, move to the element and click:
from selenium.webdriver.common.action_chains import ActionChains
link = driver.find_element_by_link_text('Load More Arguments')
ActionChains(browser).move_to_element(link).perform()
link.click()
If you get an exception while finding an element, you may need to use an Explicit Wait:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
link = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.LINK_TEXT, "Load More Arguments")))
ActionChains(browser).move_to_element(link).perform()
link.click()
If I understand your code correctly, I can see a few things wrong.
1. You're using find_elements_by_class_name. I'd recommend using find_element_by_class_name instead. elements returns a list, which isn't needed in a case where there is only one element.
2. You're using btn_moreDebate as the holder for the results of your find_elements, but then interacting with btn.
You should be able to perform the find and click in one action:
driver.find_element_by_class_name("debate-more-btn").click()