I am trying to use Selenium in Python, as as I am a beginner in doing so I cannot get send_key to work, but most probably it is straight forward and I am missting something.
Here is an example of what I have done so far:
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("https://semantria.com/demo")
item = driver.find_element_by_id("analyze_url_form")
item.send_keys("http://finance.yahoo.com/news/skystar-bio-pharmaceutical-company-provides-133000048.html")
go_button = driver.find_element_by_id("analyze_url_button")
go_button.click()
The idea is that in the https://semantria.com/demo website, there is an empty space that one can enter a website link, and then click on the Go button.
However, it looks like my code does not do this.
Am I doing something wrong? Does this website do something that I should be aware of and change my code accordingly?
Any help on this is really appreciated.
The problem is that you are sending keys to the form element, not the input element inside.
Plus, you can just send the URL with a new line at the end which is the same as you've entered the URL and pressed ENTER key which results in the form being submitted. Works for me:
item = driver.find_element_by_css_selector("form#analyze_url_form input[name=link]")
item.send_keys("http://finance.yahoo.com/news/skystar-bio-pharmaceutical-company-provides-133000048.html" + "\n")
As a bonus, here is how you can grab the sentiment value (you have to let selenium know what to wait for via WebDriverWait and Expected Conditions):
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Firefox()
driver.get("https://semantria.com/demo")
item = driver.find_element_by_css_selector("form#analyze_url_form input[name=link]")
item.send_keys("http://finance.yahoo.com/news/skystar-bio-pharmaceutical-company-provides-133000048.html" + "\n")
wait = WebDriverWait(driver, 30)
sentiment_value = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "strong.sentiment_score_value")))
print(sentiment_value.text)
Prints positive (+0.230).
Related
I am trying out Selenium for the first time so I apologize if there is an obvious mistake or problem with my code.
from selenium import webdriver
driver = webdriver.Chrome()
driver.get('https://youtube.com')
searchBox = driver.find_element_by_id('search')
searchBox.send_keys('Programming')
searchButton = driver.find_element_by_id('search-icon-legacy')
searchButton.click()
So I tried this and it loads the page fine but, it does not input any characters into the searchBox (I quadruple checked that the id was correct - copied it directly from the inspector).
NOTE:
My internet is really REALLY slow and it takes YouTube approx. 20 seconds to fully load, so I thought that was an issue so I tried;
...
driver.get('https://youtube.com')
driver.implicitly_wait(30)
searchBox = driver.find_element_by_id('search')
...
But this did not work either.
I did use XPATH instead of finding it by element ID at the start and that did not work.
I checked and copied the XPATHs and IDs directly from the inspector and nothing so far has inputted anything into the textbox.
What could be the problem? (1)
and does the webdriver wait for the page to load/find the element before doing anything after it being initialized with the driver.get('websiteAddress')? (2)
NOTE: I double checked that I was selecting the right element as well.
To send keys to the input tag with id = search. We use webdriver waits to allow the element to be usable after driver.get so the page loads correctly.
driver.get('https://youtube.com')
WebDriverWait(driver, 30).until(EC.element_to_be_clickable((By.XPATH, "//input[#id='search']"))).send_keys("Programming")
Import
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
If you don't know the waiting time:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
delay = 40
WebDriverWait(driver, delay).until(EC.presence_of_element_located((By.XPATH, "//form[#role='form']//input[#id='username']")))
Then it just waits on the element, for as log as delay is, but will continue as soon as the element is found, that the best way to wait on slow connections.
To relate elements more easily you can use ChroPath, it is an extension for google chrome / edge that allows you to see the path of an element, through cssSelector, Abs XPath, Rel XPath and the tag name, so when your code is not working you can try these other ways. Particularly this extension helps me a lot.
I was trying to get the email address and click on the refresh button from the following screenshot. But I am getting errors.
My code for this is like the following:
from selenium import webdriver
url = 'http://od.obagg.com/ '
driver = webdriver.Chrome(executable_path='chromedriver')
driver.get(url)
s = driver.find_element_by_id('//*[#id="shortid"]').get_attribute('placeholder')
print(s)
Based on the inspect, i was trying to do and tried many ways to get that email field value and click on refresh button. But still no luck.
Do anybody know any tricks to share?
It may be due to the fact the element is disabled, also, find_element_by_id('//*[#id="shortid"]') is incorrect. It can be either:
find_element_by_xpath('//*[#id="shortid"]')
find_element_by_id("shortid") ?
The following works for me:
from selenium import webdriver
from selenium.webdriver.support import expected_conditions as ec
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
driver = webdriver.Chrome(executable_path='chromedriver')
driver.maximize_window()
driver.get('http://od.obagg.com/')
wait = WebDriverWait(driver, 10)
el = wait.until(ec.visibility_of_element_located((By.ID, "shortid")))
placeholder = el.get_attribute("placeholder")
email = el.get_attribute('value')
print(placeholder, email)
# 请等待分配临时邮箱 -_ylp06tc#xxx.xxx
If you need 10 different emails, you can use:
from time import sleep
for x in range(10):
driver.find_element_by_id("refreshShortid").click()
sleep(0.15) # you may have to increase this value to give enough time to generate the new email
new_email = driver.find_element_by_id("shortid").get_attribute('value')
print(new_email)
I am trying to login to a website using python so that I can get some of their text from the website.
Here is my code. There always an error at the end of the code after the id and password code.
import os
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
driver = webdriver.Chrome(executable_path=r"C:\chromedriver\chromedriver.exe")
driver.get('https://www.saramin.co.kr/zf_user/auth')
driver.implicitly_wait(3)
driver.find_element_by_name('id').send_keys('<<my_id>>')
driver.find_element_by_name('password').send_keys('<<my_password>>')
driver.find_element_by_xpath('//*[#id="frmNIDLogin"]/fieldset/input').click()
HTML source of the button:
Eventually I figured it out! Thanks for your answer though.
Here is the final code.
driver = webdriver.Chrome(executable_path="C:\chromedriver\chromedriver.exe")
browser = webdriver.Chrome('C:\chromedriver\chromedriver.exe')
driver.get('https://www.saramin.co.kr/zf_user/auth')
driver.implicitly_wait(3)
driver.find_element_by_name('id').send_keys('ID') driver.find_element_by_name('password').send_keys('PW')
driver.find_element_by_xpath( '//*[#class="btn-login"]' ).click()
The xpath is incorrect, it doesn't match anything in the page. Try
driver.find_element_by_xpath('//form[#id="login_frm"]//button[#class="btn-login"]').click()
or simply use submit() function on the <form>
form = driver.find_element_by_id('login_frm')
form.submit()
In the first case you were using 'id' ('//*[#id="frmNIDLogin"]) for click button, because 'id' changes every time page loads it was giving error. But in the second case when you used class ( '//*[#class="btn-login"]' ) it worked because it remains same every time page is loaded. Also as mentioned above the value of id in first case was wrong.
Using Python27:
I am trying to automate a search and extract method using beautifulsoup to parse and input data on this database server. So far I have managed to get Python to login to it. But now when I try to search for the input element to make a search, I can't seem to get the identifier/code correct.
The highlighted in blue code says:
<input id="QUICKSEARCH_STRING" type="text" on focus="setTimeout('focusSearchElem()',100...
The highlighted portion in blue is where I believe I need to search for that element in order to input text then search. I think the rest, like inputting the results I get from the page might be a bit easier.
My code is as follows:
from selenium import webdriver
browser = webdriver.Firefox()
browser.get('http://somewebpage')
emailElem = browser.find_element_by_id('j_username')
emailElem.send_keys('blahblahblah')
passwordElem = browser.find_element_by_id('j_password')
passwordElem.send_keys('blahblahblah!')
login_form = browser.find_element_by_xpath("//a[#id='login']").click()
searchElem = browser.find_element_by_id('search_panel')
searchElem.send_keys('blahblahblah')
I'm not sure where I'm going wrong, but I think I am close.
browser.find_element_by_id('search_panel')
I don't see any elements with id="search_panel".
Here is how I would locate the desired input element:
browser.find_element_by_id("QUICKSEARCH_STRING")
browser.find_element_by_css_selector("div.search_panel input#QUICKSEARCH_STRING")
You may need to wait for it to become present after the logging in:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
wait = WebDriverWait(driver, 10)
search_input = wait.until(EC.presence_of_element_located((By.ID, "QUICKSEARCH_STRING")))
search_input.send_keys('blahblahblah')
from selenium import webdriver
fp = webdriver.FirefoxProfile('')
driver = webdriver.Firefox(firefox_profile=fp)
driver.set_window_size(1400, 1000)
driver.get('')
def get_list_of_all_elements(self):
list_of_elements = self.driver.find_elements_by_xpath('//*')
for ele in list_of_elements:
print ele
I want my code to print out the elements on the page, the code works except it doesnt print anything out and idk why. To be more specific, ideally I dont want every element on the page, just a set a possible images that I locate using driver.find_element_by_xpath("//img[#title='']") the title varies from image to image, and they dont have a class/type/etc so I either have to use xpath or css selector. Any help is appreciated =).
You might need to give it time to load the page by introducing an explicit wait. For instance, you can wait for at least one img element to appear on the page:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.TAG_NAME, "img"))
)
list_of_elements = self.driver.find_elements_by_css_selector('img[title]')
# ...