I am trying to create a bot in WhatsApp but I keep running into
NoSuchElementException
My code is below :
from selenium import webdriver
Import time
Firefox_browser = webdriver.Firefox(executable_path="/usr/bin/geckodriver")
Firefox_browser.get('https://web.whatsapp.com/')
time.sleep(15)
firefox_browser.switch_to_frame(firefox_browser.find_element_by_name("WhatsAppBot"))
firefox_browser.find_element_by_xpath('//span[#title="WhatsAppBot"]).click()
There is no element with name attribute equals WhatsAppBot on that page
As the other answer has already pointed out, what you are trying to do in the last two lines does not make sense. The find element by xpath/name function expects to be pointed at a certain element on Whatsapp. There is no element called "WhatsAppBot" on the WhatsApp webpage.
Unfortunately you did not give a lot of info to work with, however, I think what you want to achieve might be similar to this tutorial. I used this myself when I tried to play around with sending automatic messages via Whatsapp.
The following code is what I used for my own small bot and largely follows the before mentioned tutorial:
import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
# SPECIFY PATH FOR CHROME DRIVER HERE
driverPath = 'D:chromedriver_win32\\chromedriver.exe'
# SPECIFY PATH FOR USER DIRECTORY (just change "TEST" to your WIN10 username)
userPath = "--user-data-dir=C:\\Users\\TEST\\AppData\\Local\\Google\\Chrome\\User Data"
# SPECIFY MESSAGES HERE
messages = ["Testmessage 1", "Testmessage 2"]
# bypass QR code login after first login
options = Options()
options.add_argument(userPath)
options.add_experimental_option('useAutomationExtension', False)
options.add_experimental_option("excludeSwitches", ["enable-automation"])
driver = webdriver.Chrome(driverPath, options=options)
# open specified whatsapp chat
driver.maximize_window()
driver.get('https://web.whatsapp.com')
time.sleep(10)
driver.find_element_by_xpath("//*[#title='INSERT NAME OF CHAT HERE']").click()
# Send message
driver.find_element_by_xpath('//*[#id="main"]/footer/div[1]/div/div/div[2]/div[1]/div/div[2]').send_keys(messages[0])
driver.find_element_by_xpath('//*[#id="main"]/footer/div[1]/div/div/div[2]/div[2]/button').click()
driver.close()
Note that this code uses the chrome driver instead.
In the future the whatsapp layout might change, meaning the given xpaths could produce errors similar to yours because the element arrangement has changed. In that case you will need to find the new xpath of the element you want to perform tasks on (i.e. click on it) and update it in your code. You can easily do that by pressing F12 while you are on Whatsapp Web, then hit Ctrl + Shift + C and click on the element. The element is then highlighted on the right of the screen and if you right click on it you will see the option "copy -> copy xpath". That's the path you will need to paste into the find_element_by_xpath function for it to work.
Related
I am trying to use Selenium in Python to pull some data from https://www.seekingalpha.com. The front page has a "Sign-in/Join now" link. I used Selenium to click it, which brought up a popup asking for sign-in information with another "Sign in" button. It seems my code below can enter my username and password, but my attempt to click the "sign in" button didn't get the right response (it clicked on the ad below the popup box.)
I am using Python 3.5.
Here is my code:
driver = webdriver.Chrome()
url = "https://seekingalpha.com"
driver.get(url)
sleep(5)
driver.find_element_by_xpath('//*[#id ="sign-in"]').click()
sleep(5)
driver.find_element_by_xpath('//*[#id ="authentication_login_email"]').send_keys("xxxx#gmail.com")
driver.find_element_by_xpath('//*[#id ="authentication_login_password"]').send_keys("xxxxxxxxx")
driver.find_element_by_xpath('//*[#id="log-btn"]').click()
Any advice/suggestion is greatly appreciated.
EDIT: previous 'answer' was wrong so I have updated it.
Got you man, this is what you need to do:
1.) grab the latest firefox
2.) grab the latest geckodriver
3.) use a firefox driver
driver = webdriver.Firefox(executable_path=r'd:\Python_projects\geckodriver.exe')
url = "https://seekingalpha.com"
driver.get(url)
sign_in = driver.find_element_by_xpath('//*[#id ="sign-in"]')
driver.execute_script('arguments[0].click()', sign_in)
time.sleep(1)
email = driver.find_element_by_xpath('//*[#id ="authentication_login_email"]')
email.send_keys("xxxx#gmail.com")
pw = driver.find_element_by_xpath('//*[#id ="authentication_login_password"]')
pw.send_keys("xxxxxxxxx")
pw.send_keys(Keys.ENTER)
Explanation:
It is easy to detect if selenium is used or not if the browser tells that information (and it seems this page does not want to be scraped):
The webdriver read-only property of the navigator interface indicates whether the user agent is controlled by automation.
I have looked for an answer how to bypass detection and found this article.
Your best of avoiding detection when using Selenium would require you to use one of the latest builds of Firefox which don’t appear to give off any obvious sign that you are using Firefox.
Gave a shot and after launch the correct page design loaded and the login attempt resulted the same like the manual attempt.
Also with a bit more searching found that if you modify your chromedriver, you have a chance to bypass detection even with chromedriver.
Learned something new today too. \o/
An additional idea:
I have made a little experiment using embedded chromium (CEF). If you open a chrome window via selenium and you open the console and check navigator.webdriver the result will be True. If you open a CEF window however and then remote debug it, the flag will be False. I did not check edge cases with it but non-edge-case scenarios should be fine with CEF.
So what you may want to check out in the future:
1.) in command line: pip install cefpython3
2.) git clone https://github.com/cztomczak/cefpython.git
3.) open your CEF project and find hello.pyin the examples
4.) update the startup to cef.Initialize(settings={"remote_debugging_port":9222})
5.) run hello.py
(this was the initial, one time setup, you may customize it in the future, but the main thing is done, you have a browser with a debug port open)
6.) modify chrome startup to:
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.debugger_address = "127.0.0.1:9222"
driver = webdriver.Chrome(chrome_options=chrome_options, executable_path=chrome_driver_executable)
7.) now you have a driver without 'automated' signature in the browser. There may be some drawbacks like:
CEF is not super very latest, right now the latest released chrome is v76, CEF is v66.
also "some stuff" may not work, like window.Notification is not a thing in CEF
I tried code you provided and it works fine. i added selenium wait just to check other options and those also worked well i changed 2 lines instead of sleeps
driver.get(url)
wait = WebDriverWait(driver, 10)
signin = wait.until(EC.element_to_be_clickable((By.XPATH, "//*[#id ='sign-in']")))
#sleep(5)
signin.click()
#driver.find_element_by_xpath('//*[#id ="sign-in"]').click()
#sleep(5)
wait.until(EC.element_to_be_clickable((By.XPATH, "//*[#id ='authentication_login_email']")))
driver.find_element_by_xpath('//*[#id ="authentication_login_email"]').send_keys("xxxx#gmail.com")
and it does click on Sign in button. and what i found is there is captcha handling on the site when i checked console after clicked on sign in button it tell the story. I went ahead and added user agent to your script but it did not worked as well. Notice the blockscript parameter in response of login API and console errors in below screenshots. However there is no captcha on the ui -
The purpose of this script is to scrape info from my work schedule. The full script works fine when I run it on my windows laptop but when I try to run on raspian it appears the click.() on the "display_but" variable is not doing its job.The page pulls up fine and logs with no problem, and it even selects an option from a dropdown with no problem. Only when clicking the display button does an error seem to occur. The object is being found as if I print it I get a selenium web object. There are not error messages. When I use drop.click() this appears to work as correct option from dropdown is being selected. I am lost
Below are the workarounds I have tried.
1. Using Keys module to tab to the button and then submitting.
- this results in the correct button being selected but when I "press enter" using keys nothing happens.
2. I tried waiting for element to be clickable using WebDriverWait, expected conditions, and By modules
- this method also works on my windows but not on raspian
3. I have tried adding implicit waits and time.sleep
- these methods did not seem to help
Below is my code
import time
from selenium import webdriver
driver = webdriver.Chrome(executable_path="/Users/Sanch/Desktop/Drivers/chromedriver")
url = 'website'
driver.get(url)
#logs into account
username_xpath = '//*[#id="usernameInputField"]'
password_xpath = '//*[#id="passwordInputField"]'
login_xpath = '//*[#id="submitButton"]/span/input'
user_name = driver.find_element_by_xpath(username_xpath)
user_name.send_keys('username')
password = driver.find_element_by_xpath(password_xpath)
password.send_keys('password')
password.submit()
#selects option from dropdown
drop_xpath ='/html/body/associate/div/view-userschedule/div/div/div[2]/div/div[1]/select/option[2]'
drop = driver.find_element_by_xpath(drop_xpath)
drop.click()
time.sleep(3)
#clicks display button
Clicks display button (shows whatever selected in dropdown)
display_but_xpath = '/html/body/associate/div/view-userschedule/div/div/div[2]/div/div[3]/button'
display_but = driver.find_element_by_xpath(display_but_xpath)
display_but.click()
You should probably do as much of that from the browser context as possible. For example:
driver.execute_script("document.querySelector('[id=usernameInputField]').value = 'user'")
driver.execute_script("document.querySelector('[id=passwordInputField]').value = 'password'")
driver.execute_script("document.querySelector('css-for-button').click()")
Solved the problem by running the script with headless chrome instead of regular chrome. Using the code below in place of "driver = webdriver.Chrome(executable_path="/Users/Sanch/Desktop/Drivers/chromedriver")" made it so that the script ran properly.
I am not sure if it was due to the lack of computing power with raspberry pi 3+ or some other factor but everything is working properly now. Maybe someone else can shed light on why headless would work but regular chrome wouldn't. Thanks for the help everyon!
#headless driver setup and launch
chrome_options = Options()
chrome_options.add_argument("--headless")
chrome_options.add_argument("--window-size=1920x1080")
chrome_driver = "your drivers path"
driver = webdriver.Chrome(chrome_options=chrome_options, executable_path=chrome_driver)
The aim of this is to open a browser window and save the site as PDF.
I'm writing Python code that:
1) Opens a web page
2) Does a control-p to bring up the print dialog box
NOTE: I will have pre-configured the browser to save as PDF instead of defaulting as printing to a printer)
3) Does "return"
4) Enters the file name
5) Does "return" again
NOTE: In my full code, I'll be doing these steps hundreds of times
I'm having a problem early on with control-p. As a test, I'm able to send dummy text to Google's search, but I can't seem to be able to send a control-p (no error messages). I'm using Google as an easy example, but my final code will use various other sites.
Obviously I'm missing something but just can't figure out what.
I tried an alternate method of using javascript instead of ActionChains:
driver.execute_script('window.print();')
This worked in getting the print dialog but I wasn't able to feed anything else in that dialog box (like , file name and location for the pdf).
I tried PDFkit, to convert the web page into pdf. It worked on some sites, but it crashed often (depending on what the site returned), the page was sometimes poorly formatted and some sites (pinterest) just didn't render at all. For this reason, I changed method and decided to use selenium and Chrome in order for the pdf to render just like it shows in the browser.
I thought about using "element.send_keys("some text")" instead of ActionChains, but since I'm going across multiple different web sites, I don't necessarily know what element to look for.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.action_chains import ActionChains
import time
DRIVER = 'chromedriver'
driver = webdriver.Chrome(DRIVER)
URL = "http://www.google.com"
driver.get(URL)
actions = ActionChains(driver)
time.sleep(5) #Give the page time to load
actions.key_down(Keys.CONTROL)
actions.send_keys('p')
actions.key_up(Keys.CONTROL)
actions.perform()
time.sleep(5) #Sleep to see if the print dialog came up
driver.quit()
You can use autoit to achieve your requirement.
First do pip install -U pyautoit
from selenium import webdriver
import autoit
import time
DRIVER = 'chromedriver'
driver = webdriver.Chrome(DRIVER)
driver.get('http://google.com')
driver.maximize_window()
time.sleep(10)
autoit.send("^p")
time.sleep(10) # Pause to allow you to inspect the browser.
driver.quit()
Please let me know if it's working.
try this:
webdriver.ActionChains(driver).key_down(Keys.CONTROL).send_keys('P').key_up
(Keys.CONTROL).perform()
check this out :
robot.keyPress(KeyEvent.VK_CONTROL)
robot.keyPress(KeyEvent.VK_P)
// CTRL+P is now pressed
robot.keyRelease(KeyEvent.VK_P)
robot.keyRelease(KeyEvent.VK_CONTROL)
As a regular task within my job role i often have to download full Facebook accounts from within a users account. I am trying to improve my workflow and automate this if i can.
I have tried searching for this topic on the site and although many cover the login part i am yet to locate a question that deals with the popup windows of Facebook. If i am wrong i apologise and please amend the post accordingly.
As a starting point i have decided to start learning python and am using this to script the process with a little help from selenium and Chrome Driver. I have managed to write the code to login and navigate to the correct page and click the initial link 'download a copy'. I am struggling however to get the script to locate and click the 'Start My Archive' button within the popup window.
Here is the code that i have used so far including the several alternative code blocks that i have tried commented out at the bottom:
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
#Global Variables
target = "john.doe.7"
username = "john#doe.com"
password = "Password"
sleep = 5
#Finds and locates the ChromeDriver
driver = webdriver.Chrome("C:\Python35-32\ChromeDriver\chromedriver.exe")
#Set Chrome Options
chrome_options = webdriver.ChromeOptions()
prefs = {"profile.default_content_setting_values.notifications": 2}
chrome_options.add_experimental_option("prefs",prefs)
driver = webdriver.Chrome(chrome_options=chrome_options)
#Directs browser to webpage
driver.get('http://www.facebook.com');
#code block to log in user
def logmein():
search_box = driver.find_element_by_name('email')
search_box.send_keys(username)
search_box = driver.find_element_by_name('pass')
search_box.send_keys(password)
search_box.submit()
#code to go to specific URL
def GoToURL(URL):
time.sleep(sleep) # Let the user actually see something!
driver.get(URL);
logmein()
GoToURL('https://www.facebook.com/'+'settings')
link = driver.find_element_by_link_text('Download a copy')
link.click()
#driver.find_element_by.xpath("//button[contains(., 'Start My Archive')]").click()
#driver.find_element_by_css_selector('button._42ft._42fu.selected._42gz._42gy').click()
#driver.find_element_by_xpath("//*[contains(text(), 'Start My Archive')]").click()
#driver.find_element_by_css_selector('button._42ft._42fu.layerConfirm.uiOverlayButton.selected._42g-._42gy').click()
#from selenium.webdriver.common.action_chains.ActionChains import driver
#buttons = driver.find_elements_by_xpath("//title[contains(text(),'Start My Archive')]")
#actions = ActionChains(self.driver)
#time.sleep(2)
#actions.click(button)
#actions.perform()
Just add this to your code and run if it is working or not. Always use CssSelector.
Look i ran the below code in eclipse with java and I'm not aware of python so if something is wrong in syntax , i apologies. Just Try this and see.
driver.implicitly_wait(10)
Startmyarchive = driver.find_element_by_css_selector("._42ft._42fu.selected._42gz._42gy")
Startmyarchive.click()
driver.implicitly_wait(10)
Acknowledge = driver.find_element_by_css_selector("._42ft._42fu.layerConfirm.uiOverlayButton.selected._42g-._42gy")
Acknowledge.click()
driver.implicitly_wait(10)
ClickOkay = driver.find_element_by_css_selector("._42ft._42fu.layerCancel.uiOverlayButton.selected._42g-._42gy")
ClickOkay.click()
Happy Learning :-) Do reply back for any query.
I am trying to send some shortcut keys to open a new tab in chrome, using selenium in python. I open up facebook, then I log in, then I want to open a new tab where I can pass the url of one of my friend so that I can view his profile. I wrote the following code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import getpass
driver = webdriver.Chrome()
#opening browser and loging into the account
def login():
driver.get("http://www.facebook.com")
elem = driver.find_element_by_xpath("//*[#id=\"email\"]")
elem.send_keys("myUsername")
password = driver.find_element_by_name("pass")
password.send_keys("myPassword")
elem.send_keys(Keys.RETURN)
driver.implicitly_wait(5)
def scout():
scroll = driver.find_element_by_tag_name('body').send_keys(Keys.CONTROL + 't')
driver.get("http://www.facebook.com/some.friend")
driver.implicitly_wait(8)
login()
scout()
But it gives me the error:
selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
I made the browser wait implicitly but still it's unable to find the tag. Please help.
First of all, try calling WebDriverWait instead of find_element_by_css_selector in order to find elements. It calls find_element_by_<method> every half a second until given timeout. That way you don't have to gamble with implicitly_wait or time.sleep().
Regarding your open new tab problem - you cannot send ctrl + t to chromedriver, it simply wouldn't do anything (it would work on firefox though). I've encountered this problem before, and the best solution I found is to search for a link in the website itself, so if you're on Facebook you can use that F icon at the top left. Then you can send ctrl + shift + click to open that in a new tab.
from selenium.webdriver import ActionChains
actions = ActionChains(driver);
actions.move_to_element(link)
actions.send_keys(Keys.CONTROL+ Keys.SHIFT)
actions.click(link)
actions.perform()
From there you should just switch tabs using
tabs = driver.window_handles
driver.switch_to.window(tabs[<index>])