I want to scrape a local website using selenium and need to send Username and Password to be logged in but there is no inspect element to find these elements. The website is like this:
To send keys to popup using Selenium, you can first switch your target to the popup
obj = driver.switch_to.alert
Then send keys using
obj.send_keys(username)
obj.send_keys(str(Keys.TAB)) # send a tab to go to the next field
obj.send_keys(password)
Reference: Handle alert popup inselenium
First of all, try this URL format in driver.get():
http://username:password#the-site.com
Related
I am trying to login into openload.co using python Selenium Chrome Driver but I am getting the following error:
Message: element not interactable
I am using the following code and the error occurs in the last line of the code where I am not able to send the keys to the input tag.
from selenium import webdriver
path="path_to_chrome_driver" #add chromedriver path
driver=webdriver.Chrome(path)
from selenium.webdriver.common.keys import Keys
driver.get('https://openload.co/login')
email = driver.find_element_by_xpath('//*[#id="loginform-email"]')
email.send_keys("example#xyz.com")
I searched for the problem on stackoverflow and landed on the following link similar question which says that probably it is not pointing to correct xpath or css_selector. But I can't seem to find it.
What wrong am I doing here?
It's because there's a modal in the HTML made visible when you click the top right "Sign in" button. There's a duplication of ids. Could you try passing the password and username like this http://username:password#openload.co/ ?
#john try this, works for me:
driver.get('https://openload.co/login')
emails = driver.find_elements_by_xpath('//*[#id="loginform-email"]')
emails[1].send_keys("example#xyz.com")
Try to click on the element before sending keys.
driver.find_element_by_xpath('//*[#id="loginform-email"]').click()
Because of this the cursor will be active on the email textbox field so the element should interactivable.
There are two forms on page: first one for SignIn, second for LogIn. Both have input fields with the same #id values. You need to select form for LogIn:
email = driver.find_element_by_xpath('//h1[.="Login"]/following::*[#id="loginform-email"]')
email.send_keys("example#xyz.com")
I want to create a program where I can check my grades using python and I have the code to web scrape data, but I do not know how to log into this specific website. The website is https://hac.chicousd.org/LoginParent.aspx?page=Default.aspx and if you need it I can give my username and password. I have tried using requests and urllib and neither work. I appreciate any help given.
Try using mechanical soup. It allows you to navigate a website just like you would normally.
As pointed out in the comments, a possibility is to use selenium, a browser manipulation tool. However, you can also use requests.Sessions to send a POST request with a payload of the email, and then a GET request for whatever portal page you wish to view after:
import requests
r = requests.Session()
payload = {'portalAccountUsername':'yoursutdentemail#school.com'}
r.post('https://hac.chicousd.org/LoginParent.aspx?page=Default.aspx', data = payload)
Then, with r instance, you can send a GET request to a page on the portal that is only visible to authenticated users:
data = r.get('https://hac.chicousd.org/some_student_only_page').text
Note that the keys of the payload dictionary must all be valid <input> "name" values from the site's HTML.
As others have said, you can use selenium. You also should use time to stop the program some seconds before to put your password. First install selenium in you command prompt pip install selenuim and a webdriver (here is the code for chrome pip install chromedriver_installer). Then you could use them in your code.
import selenium
from selenium import webdriver
import time
from time import sleep
Then, you should open the web page with the web driver
browser = webdriver.Chrome('C:\\Users...\\chromedriver.exe')
browser.get('The website address')
The next step is to find the name of the elements on the web page to write your username, password, and the path for the buttons
username = browser.find_element_by_id('portalAccountUsername')
username.send_keys('your email')
next = browser.find_element_by_xpath('//*[#id="next"]')
next.click()
password = browser.find_element_by_id('portalAccountPassword')
time.sleep(2)
password.send_keys('your password')
sing_in = browser.find_element_by_xpath('//*[#id="LoginButton"]')
sing_in.click()
I have an application that generates a new browser tab when I click on a link. In the new tab I get the Chrome authentication alert. I've tried using the below code:
wd.switch_to_window(wd.window_handles[1])
a = switch_to_alert()
a.send_keys('username' + Keys.TAB + 'password')
but I get the no alert error msg
I have also tried using the ActionChain to send the keys but the alert does not receive the send_keys
Are there any alternatives?
fyi I cannot use https://username#password format as this tab is generated when I click on a link
Traceback
I suppose you mean that a link opens a new tab which has username and password fields which you input them with ‘username’ and ‘password’ respectively?
If that is the case can you on the newly opened tab first search for the elements (username and password) using an explicit wait,and only then enter values.
I had a similar issue where my web driver was stuck on the earlier page and tried inputting values into fields which didn’t exist there. Try typing your active tab’s title to confirm that’s not the case.
Sript click a link like <a href="/loginform.do">
and it's open a new pop up window with login/password fields (OK/Cancel buttons).
And webdriver for some reason determine this pop up as alert not window...
if try
for handle in browser.window_handles:
print(handle)
it's return exception: UnexpectedAlertPresentException: Alert Text: Message: Modal dialog present
if try
alert=browser.switch_to_alert()
alert.send_keys('userlogin')
it's working (userlogin inserted into first field).
And then if trying to move cursor to next field (password) by TAB key
alert.send_keys(Keys.TAB)
it's replace text at the first field to bullet symbol instead of tabbing to next field...
Not only TAB, even ENTER working in the same way
alert.send_keys(Keys.ENTER)
it's paste the same bullet symbol instead of sending request to a server
And
alert.accept()
doing nothing (no sending request to a server)
What can be done here?
It's an alert, so find_element_by_... methods are not applicable.
TAB by send_keys is not working (no tabbing).
Is the only solution to click by coordinates (using pyautogui module as an option)?
Or is it possible to do more with webdriver?
BTW using IE (IEDriverServer)
And some searching for close questions
31152912 close question, but answer is about find_element_by_name (no such attribute for 'Alert' object)
31152912, 29516740, 27322871 answers about using 'http://username:password#', unfortunatelly it's not working
So an example with login form of .htaccess password protection (advancedhtml.co.uk/password.html)
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
browser=webdriver.Ie('c:\\...\\IEDriverServer.exe')
page='http://www.advancedhtml.co.uk/password/'
browser.get(page)
alert=browser.switch_to_alert()
#alert=browser.switch_to.alert() # this should work, but it doesn't: TypeError: 'Alert' object is not callable
alert.send_keys('user')
alert.send_keys(Keys.TAB) # no tabbing, bullet symbol instead...
#alert.send_keys('password')
So this kind of login page is HTTP Basic Access Authentication. And looks the best way to handle such authentication is to sent post request (Requests package)
Need to pass browser session from Selenium to Requests
(29563335)
The code may look like this:
from selenium import webdriver
import requests
startURL='http://some_url'
browser=webdriver.Ie('c:\\path\\IEDriverServer.exe')
browser.get(startURL)
# an example of finding login link and get the url
loginLink=browser.find_element_by_xpath("//*[contains(text(), 'Log In')]")
loginURL=loginLink.get_attribute('href')
# load cookies from Selenium to Requests (so we could authenticate in current session)
cookies=browser.get_cookies()
s=requests.Session()
for cookie in cookies:
s.cookies.set(cookie['name'], cookie['value'])
# and send the post request
auth=('login', 'password')
r=s.post(url=loginURL, auth=auth)
browser.refresh() # refresh page to see the result
And for .htaccess password protection (example: advancedhtml.co.uk/password.html) it may be handled with additional click at coordinates
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import pyautogui
page='http://www.advancedhtml.co.uk/password/'
browser=webdriver.Ie('c:\\path\\IEDriverServer.exe')
browser.get(page)
alert=browser.switch_to_alert()
alert.send_keys('demo')
#alert.send_keys(Keys.TAB) # tabbing is not working
pyautogui.moveTo(620, 550) # so move mouse cursor to second field
pyautogui.click()
pyautogui.typewrite('password')
alert.accept()
#Litvin, Your approach is correct but you needed to know one more thing in this.
When you do:- obj = driver.switch_to.alert
You can send the keys in one more way as below, which I understood after walking through the Python selenium source code (here).
obj.send_keys(keysToSend="username\ue004password")
Here \ue004 is the value for TAB.
I am using selenium to scrape twitter (not using the api's just practicing selenium) which require login when it coms to following page, am using the following code to locate the login input fields and then send the username and password string:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome(executable_path="Chrome")
driver.get("https://twitter.com/login")
username = driver.find_element_by_xpath("//*[#id='page-container']/div/div[1]/form/fieldset/div[1]/input")
username.send_keys("username")
password = driver.find_element_by_xpath("//*[#id='page-container']/div/div[1]/form/fieldset/div[2]/input")
password.send_keys("password")
password.send_keys(Keys.ENTER)
driver.get("http://twitter.com/user/following")
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
the code is working except for the second field (password field) it didn't send the password strings and skips it to the next line (Keys.ENTER) and next it moves to the following page which require a login, however the cursor was at the password field which mean it already located it, strange enough there is no error message or error code when executing the script.
any thoughts what is the cause of this issue ??
am using python 2.7.6
selenium version 2.53.6
on ubuntu 14.04
thanx in advance
Instead of sending the ENTER key, just click on the Log in button. Also, your XPath is kinda brittle. I would use a more specific one or you can use the CSS selectors below.
driver.find_element_by_css_selector("#page-container input.email-input").send_keys(username);
driver.find_element_by_css_selector("#page-container input.js-password-field").send_keys(password);
driver.find_element_by_css_selector("button.submit").click();