I'm trying to do some web-scraping on instagram with selenium. specifically i'm trying to log-in by this address
https://www.instagram.com/accounts/login/
with selenium.
On this page, the input 'username' is written like this
<input class="_ph6vk _o716c" aria-describedby="" aria-label="Phone number, username, or email" aria-required="true" autocapitalize="off" autocorrect="off" maxlength="30" name="username" placeholder="Phone number, username, or email" value="" type="text">
What i'm doing in python is:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.action_chains import ActionChains
driver=webdriver.Firefox()
driver.get('https://www.instagram.com/accounts/login/')
input_username = driver.find_elements_by_xpath("//input[#name='username']")
input_username.send_keys("username")
Python returns me : AttributeError: 'list' object has no attribute 'send_keys'
So I did the same changing this:
input_username[0].send_keys("username")
And the error is:
IndexError: list index out of range
So, the array is empty. Anyone knows how to solve it?
In your case the page might not have loaded the form, so use WebDriverWait to let the element load and start scraping.
You can check for the element this way, instead of putting time.sleep(2) because it might take a long time to load it as well.
myElem = WebDriverWait(driver, delay).until(EC.presence_of_element_located((By.ID, 'IdOfMyElement')))
Also try to use the api if it's possible, scraping should be the second approach.
Try this code:
from selenium import webdriver
import time
driver=webdriver.Firefox()
driver.get('https://www.instagram.com/accounts/login/')
time.sleep(2)
user_name=driver.find_element_by_xpath('//*[#id="react-root"]/section/main/div/article/div/div[1]/div/form/div[1]/div/input')
user_name.send_keys('user_name')
password=driver.find_element_by_xpath('//*[#id="react-root"]/section/main/div/article/div/div[1]/div/form/div[2]/div/input')
password.send_keys('pa$$')
driver.find_element_by_xpath('//*[#id="react-root"]/section/main/div/article/div/div[1]/div/form/span[1]/button').click()
The form loads after the page is loaded so here's how I did it :
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
driver = webdriver.Firefox()
driver.get("https://instagram.com")
time.sleep(4)
driver.find_element_by_name("username").send_keys("foobar")
driver.find_element_by_name("password").send_keys("pass")
driver.find_element_by_name("password").send_keys(Keys.ENTER)
A bit hacky at the end
Related
I am struggling on how to get the button type element in this example, so that I can be able to test it using selenium in python.
<button class="login-button signin-button button-primary" type="submit"><div class="signin-text">Sign in</div></button>
My Code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.service import Service
import time
PATH = Service("C:\Program Files (x86)\chromedriver.exe")
driver = webdriver.Chrome(service=PATH)
driver.get("https://www.dropbox.com/login")
driver.find_element("name", "login_password").send_keys("123456")
driver.find_element("type", "submit").send_keys(Keys.ENTER)
time.sleep(5)
driver.close()
print("Test Completed Successfully ")
I think the code changed because of the selenium version? I need help guys Thank you.
There is no such selector type as "type". You can either use another locator for Submit button e.g.
driver.find_element("xpath", "//button[#type='submit']").send_keys(Keys.ENTER)
or this form submitting approach
input_field = driver.find_element("name", "login_password")
input_field.send_keys("123456")
input_field.submit()
you can try below xpath
click_button = driver.find_element(By.X_Path, "//button[contains(#class, 'login-button signin-button')]")
print(click_button.get_attribute('type'))
Im trying to get the username box from the twitter log in page, i have followed alot of online tutorials but they all use twitters old log in page, and i am not that good at coding to be able to understand how. I want to locate the username box and send a username to it.
import csv
from getpass import getpass
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.common.exceptions import NoSuchElementException
from msedge.selenium_tools import Edge, EdgeOptions
options = EdgeOptions()
options.use_chromium = True
driver = Edge(options=options)
driver.get('https://www.twitter.com/login')
driver.maximize_window()
username=driver.find_element_by_xpath('//input[#name="username"]').send_keys('Username')
time.sleep(2)
HTML code of username field on twitter is:
<input autocapitalize="none" autocomplete="username" autocorrect="off" inputmode="text" name="username" spellcheck="false" type="text" dir="auto" class="r-30o5oe r-1niwhzg r-17gur6a r-1yadl64 r-deolkf r-homxoj r-poiln3 r-7cikom r-1ny4l3l r-t60dpp r-1dz5y72 r-fdjqy7 r-13qz1uu" data-testid="ocfEnterTextTextInput" value="">
Observe name attribute with value username
so, try using this xpath //*[#name="username"]
Im trying to automize some webpages and run into the following error:
I have passed the login screen, which gives no problems, but after being redirected to the next page, python gives the following error:
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element:
I have tried to fix it by introducing the driver.implicitly.wait, the time.sleep and the WebDriverWait(driver, time).until(EC.presence_of_element_located((By.ID, "myDynamicElement")) functions, but all the options do not seem to work.
I have shown the relevant HTML code below:
<input type="Text" class="urEdf2TxtEnbl" autocomplete="off" id="DLG_VARIABLE_vsc_CommonVariablesList_VAR_5_VARINPUT_inp" ct="I" name="DLG_VARIABLE_vsc_CommonVariablesList_VAR_5_VARINPUT_inp" st="" tp="STRING" value="502309" onchange="sapUrMapi_InputField_change('DLG_VARIABLE_vsc_CommonVariablesList_VAR_5_VARINPUT_inp',event);" onblur="sapUrMapi_InputField_Blur('DLG_VARIABLE_vsc_CommonVariablesList_VAR_5_VARINPUT_inp',event);" onkeydown="sapUrMapi_InputField_keydown('DLG_VARIABLE_vsc_CommonVariablesList_VAR_5_VARINPUT_inp',event);" onkeyup="sapUrMapi_InputField_KeyUp('DLG_VARIABLE_vsc_CommonVariablesList_VAR_5_VARINPUT_inp',event);" onfocus="sapUrMapi_InputField_focus('DLG_VARIABLE_vsc_CommonVariablesList_VAR_5_VARINPUT_inp',event);" onselectstart="sapUrMapi_InputField_onselectstart('DLG_VARIABLE_vsc_CommonVariablesList_VAR_5_VARINPUT_inp',event);">
The HTML code in the region around this is shown below:
<input type="Text" class="urEdf2TxtEnbl" autocomplete="off" id="DLG_VARIABLE_vsc_CommonVariablesList_VAR_4_VARINPUT_inp" ct="I" name="DLG_VARIABLE_vsc_CommonVariablesList_VAR_4_VARINPUT_inp" st="" tp="STRING" value="" onchange="sapUrMapi_InputField_change('DLG_VARIABLE_vsc_CommonVariablesList_VAR_4_VARINPUT_inp',event);" onblur="sapUrMapi_InputField_Blur('DLG_VARIABLE_vsc_CommonVariablesList_VAR_4_VARINPUT_inp',event);" onkeydown="sapUrMapi_InputField_keydown('DLG_VARIABLE_vsc_CommonVariablesList_VAR_4_VARINPUT_inp',event);" onkeyup="sapUrMapi_InputField_KeyUp('DLG_VARIABLE_vsc_CommonVariablesList_VAR_4_VARINPUT_inp',event);" onfocus="sapUrMapi_InputField_focus('DLG_VARIABLE_vsc_CommonVariablesList_VAR_4_VARINPUT_inp',event);" onselectstart="sapUrMapi_InputField_onselectstart('DLG_VARIABLE_vsc_CommonVariablesList_VAR_4_VARINPUT_inp',event);">
What I am trying to do is to change the value: value="502309". See below for the full code:
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
driver = webdriver.Chrome(ChromeDriverManager().install())
driver.get('xxxx')
login = driver.find_element_by_xpath('xxx')
login.send_keys('xxx')
passw = driver.find_element_by_xpath('xxx')
passw.send_keys('xxx')
button = driver.find_element_by_xpath('xxx')
button.click()
driver.maximize_window()
driver.implicitly_wait(120)
mat = driver.find_element_by_id('DLG_VARIABLE_vsc_CommonVariablesList_VAR_2_VARINPUT_inp')
driver.close()
Try to print the html using beautifulsoup so you can check if the element really exist. For your reference on how to use it, see link
I don't know if this will help but i had the same problem and solved it by this:
searchbox = WebDriverWait(driver, 10).until(
EC.visibility_of_element_located(
(By.XPATH, "your_xpath_here")
The above code shows that the webdriver will wait's for an element located in your page.
Issue with your code is not wait , but rather correct element locator. As far as I can see from limited HTML DOM you have provided, part of name and id of element you want to locate inside variable mat is changing. You can locate your element using below code ( Also i have included more sophisticated way to wait for element load).
mat = WebDriverWait(driver, 120).until(EC.presence_of_element_located((By.XPATH, "//input[starts-with(#id,'DLG_VARIABLE_vsc_CommonVariablesList_VAR')]")))
I have to find Input Tag for id of a web page in the following HTML code using python Selenium:
NOTE: I was not able to paste the code because of its complexity so sorry for that. Here is the page to look out the complete HTML Tree.
I want to insert username to the mentioned below input tag:
<input type="text" id="appleId" can-field="accountName" autocomplete="off"
autocorrect="off" autocapitalize="off" aria-required="true"
required="required" aria-labelledby="appleIdFieldLabel" spellcheck="false"
autofocus="" ($focus)="appleIdFocusHandler()"
($keyup)="appleIdKeyupHandler()" ($blur)="appleIdBlurHandler()" class="si-
text-field form-textbox " placeholder="Apple ID">
Currently, I am using mentioned below code by the means of element's xpath but it's returning an empty list:
from selenium import webdriver
import time
browser = webdriver.Chrome()
browser.get('http://www.icloud.com')
time.sleep(20)
ele = browser.find_elements_by_xpath('//*[#id="appleId"]')
# ele.send_keys('abc#icloud.com')
print ele
Result: []
I have tried every other find functions too but could not get any results.
Authorization form located inside an iframe. To be able to handle input fields, you should switch to that iframe:
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait as wait
from selenium import webdriver
from selenium.webdriver.common.by import By
browser = webdriver.Chrome()
browser.get('http://www.icloud.com')
wait(browser, 10).until(EC.frame_to_be_available_and_switch_to_it("auth-frame"))
account_name = wait(browser, 10).until(EC.presence_of_element_located((By.ID, "appleId")))
account_name.send_keys('abc#icloud.com')
Using Python27:
I am trying to automate a search and extract method using beautifulsoup to parse and input data on this database server. So far I have managed to get Python to login to it. But now when I try to search for the input element to make a search, I can't seem to get the identifier/code correct.
The highlighted in blue code says:
<input id="QUICKSEARCH_STRING" type="text" on focus="setTimeout('focusSearchElem()',100...
The highlighted portion in blue is where I believe I need to search for that element in order to input text then search. I think the rest, like inputting the results I get from the page might be a bit easier.
My code is as follows:
from selenium import webdriver
browser = webdriver.Firefox()
browser.get('http://somewebpage')
emailElem = browser.find_element_by_id('j_username')
emailElem.send_keys('blahblahblah')
passwordElem = browser.find_element_by_id('j_password')
passwordElem.send_keys('blahblahblah!')
login_form = browser.find_element_by_xpath("//a[#id='login']").click()
searchElem = browser.find_element_by_id('search_panel')
searchElem.send_keys('blahblahblah')
I'm not sure where I'm going wrong, but I think I am close.
browser.find_element_by_id('search_panel')
I don't see any elements with id="search_panel".
Here is how I would locate the desired input element:
browser.find_element_by_id("QUICKSEARCH_STRING")
browser.find_element_by_css_selector("div.search_panel input#QUICKSEARCH_STRING")
You may need to wait for it to become present after the logging in:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
wait = WebDriverWait(driver, 10)
search_input = wait.until(EC.presence_of_element_located((By.ID, "QUICKSEARCH_STRING")))
search_input.send_keys('blahblahblah')