I'm trying to make an auto bump program for forums but I can't seem to interact with anything.
import webbrowser
import pyautogui
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome()
driver.get("https://ogusers.com/member.php?action=login")
#Login
inputElement = driver.find_element_by_class_name("textbox")
inputElement.send_keys("xKyle")
Can't really tell where problem lies, you need to say exactly what is happening, let's start with not giving selenium path to ChromeDriver. You need to download it from here, make sure to get driver compatible with your browser verison.
Next give path to it, if you are on windows path will look like:
r"C:\dir\dir\chromedriver.exe'
you need to give 'r' for raw data, otherwise interpreter will think that you are trying to give some weird in-string command like \n for new line
Also try not to search by class name, for 99% of time you want to use Xpath, especially if it's relative. So just click on element, and copy xpath.
For more answers you need to tell exactly what are you trying to do and when it fails :)
Related
As I learn selenium with python, I find that I may have made a mistake that leads to an exception while I adapt it to the exact flow of the web application. As such, the scripts exits with an error, and python loses control over the driver.
Fixing the mistake takes some time while a new browser is created; that also creates a lot of temporary files that are required for the browser profile (rust_mozprofile). In short, fixing the error loses me significant time.
I'm looking for an answer specifying a workflow with selenium that allows me to reuse the browser instance, make mistakes in syntax, intent or error handling, and correct them without neither reloading the script, respawning the browser, nor redoing all steps in the website.
I'd prefer a Firefox focused solution, but Chrome answers are also welcome.
Use the VS Code Debugger
Most of the issues mentioned could be avoided by first doing an exhaustive review of the website so that you start coding your script with "perfect" knowledge of the every site transition, delay, button identifier, and so on. Realistically, this would be too much work.
I will provide a simple example of the workflow I use to develop:
Example. Retrieve the first picture from #cats in Instagram
Step 1.
Start with boilerplate code that imports all relevant modules so that if I need them, I can call them immediately. A good start is:
bot_template.py:
import logging
import pickle
import time
from inspect import getsourcefile
from os.path import abspath
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.common.exceptions import NoSuchElementException, TimeoutException
from selenium.webdriver.remote.webelement import WebElement
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.firefox.options import Options
driver = webdriver.Firefox()
# your breakpoint should go in this line
driver.quit()
At this point try doing the process manually of going to Instagram and getting the first picture in the tags feed. If you did you would find that you can go to https://www.instagram.com/explore/tags/cats/ for the cat tag.
Step 2
Add a breakpoint in the line right after driver = webdriver.Firefox() which spawns Firefox. Breakpoints are added by clicking in the empty region immediately left of the line numbers or with the default shortcut (F9).
Step 3
In VSCode, Press F1 and type Start Debugging, and select Python File, or use the default shortcut (F5).
Step 4
In VSCode, Press F1 and type Focus on Debug Console View. Switch over to the debug console.
Step 5
Make mistakes in the debug console as you try to build your script. You need the bot to go to https://www.instagram.com/explore/tags/cats/, so you write: driver.Get(r'https://www.instagram.com/explore/tags/cats/'). Next thing you see is:
Error: AttributeError: 'WebDriver' object has no attribute 'Get'
Step 6
Correct your mistake. You realize the method is actually called get (lowercase). Type that again in the VSCode Debug Console. The browser will now navigate to the right URL, no need to respawn a new one due to a typo.
>>> driver.get('https://www.instagram.com/explore/tags/cats/')
Step 6
Switch over to the browser you are controlling from Python. You are free to do whatever you want to it, in whatever tempo/order you want.
You open the inspector and realize that you can use XPATH in the browser terminal $x('//div[#class="_aabd _aa8k _aanf"]') to get the array of images. Therefore, you type in the VSCode debug console:
>>> image = driver.find_element(By.XPATH, '//div[#class="_aabd _aa8k _aanf"]')
Step 7
You start trying to get the image src. Therefore, you type image.get_attribute('src') in the debug console. It does not work because you selected a div that does have a 'src' html attribute, rookie mistake. The code would have terminated in regular execution and you would have lost control over the browser, but you are on the debugger, so you retain control and your precious time. You keep trying in your debugging playground, until you figure out that you can fix this by changing the xpath. You try typing in the debug console and that gets you the image url.
>>> image = driver.find_element(By.XPATH, '//*[#class="_aabd _aa8k _aanf"]//img[#src]')
>>> src = image.get_attribute('src')
Step 8
Download the image from the url. You keep the browser and vscode session open and go look for ways to download the image. You notice you need other libraries. No problem, try typing the following in same debug console session and you will have your image downloaded.
# Changes the current dir to the project folder and downloads as 'image.png'
>>> import os
>>> import urllib
>>> os.chdir('.')
>>> urllib.urlretrieve(src, 'image.png')
Step 9
Keep building interactively until your bot does all it has to do. Commit the lines of code that gave you the desired result to your file bot_template.py. Once you are done, test the whole thing by running the script without debugging. You never had to respawn another browser session and lose time.
I have been really banging my head on this, I can't figure out my problem, if someone can help me it would be appreciated, thank you :).
I'm trying to select with Xpath the twitter search box, from what I know, my input is correct, since on google when I press Ctrl+F and search for it in the "inspect mode" it's highlighting in yellow the search box input tag. But in selenium, I wish it was.
(Using google chrome)
import csv
from getpass import getpass
import time
from time import sleep
from selenium.webdriver.common.keys import Keys
from selenium.common.exceptions import NoSuchElementException
from selenium import webdriver
#other selenium code, working.
twitterS = driver.find_element_by_xpath('//input[contains(#data-testid,"SearchBox")]')
Return
selenium.common.exceptions.NoSuchElementException:
Message: no such element: Unable to locate element:
{"method":"xpath","selector":"//input[contains(#data-testid,"SearchBox")]"}
I tried the relative path, not working either
/html/body/div[1]/div/div/div[2]/main/div/div/div/div[2]/div/div[2]/div/div/div/div[1]/div/div/div/form/div[1]/div/div/div[2]/input
Your XPath seems to be fine (I've checked it). If Selenium tells you the elements is not there it means that it's not there in the moment it tries to find it. I don't see any frame or shadow DOM there so I would say you need to wait for this element before getting it. I would also make sure if at that point you are really on the page you're expecting to see the element at. Without the code I can't really help better.
I want to be able to use pure selenium webdriver to open a zoom link in Chrome and then redirect me to the zoom.us application.
When I execute this:
from selenium import webdriver
def main():
driver = webdriver.Chrome()
driver.get("https://zoom.us/j/000-000-000")
main()
I receive a pop-up saying
https://zoom.us wants to open this application.
and I must press a button titled open zoom.us to open the app.
Is there a way to press this pop-up button through selenium. Or, is there some other way to open zoom from chromedriver?
NOTE: I only want to use selenium. I have been able to implement pyautogui to click on the button but that is not what I am looking for.
Solution for Java:
driver.switchTo().alert().accept();
Solution for Python:
driver.switch_to.alert.accept()
There are a lot of duplicated questions regarding this issue. Here is one of them, and it is quite sure that selenium is not capable of achieving such job since it only interacts with the chrome page. I previously encountered this issue as well and here is my solution to it. It might look really unprofessional, but fortunately it works.
The logic of my solution is to change the setting of chrome in order to skip the popup and directly open the application you want. However, the Chrome team has removed this feature in the latter version for some reasons, and we need to get it back manually. Then, we know that everytime when selenium starts to do the thing it opens a new Chrome page with NO customized settings just like the incognito page. Therefore we need to do something to let selenium opened a Chrome page with your customized setting, so that we can make sure that the popup, which we changed manually to skip, can be skipped successfully.
Type the following code in your terminal.
defaults write com.google.Chrome ExternalProtocolDialogShowAlwaysOpenCheckbox -bool true
This enables you to change the setting of skipping popups, which is the feature Chrome team removed.
Restart Chrome,and open the zoom (or whatever application) page to let the popup display. If you do the 1st step correctly you will be able to see there is a checkbox shown next to the "Open Zoom.us" saying if you check it chrome will open this application without asking, that is, to skip the popup for this application.
Now we need to let selenium open the Chrome with our customized setting. To do this, type "chrome://version" in the search tab of your ordinary Chrome (Not automated page opened by selenium). Go to "Profile Path", and copy this path without the last word "default". For example:
/Users/MYNAME/Library/Application Support/Google/Chrome/Default
This is my profile path, but I only copy everything except the last word Default, so this is what I need to copy.
/Users/MYNAME/Library/Application Support/Google/Chrome/
This is for Mac users, but for Windows only the path is different(starts with C:// or something), steps are same.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
option = Options()
option.add_argument('THE PATH YOU JUST COPIED')
driver = webdriver.Chrome(executable_path='YOUR PATH TO CHROMEDRIVER', options=option)
driver.get("google.com") #Or anything else
We use "options" to let selenium open a page with our customized profile. Now you will see selenium opens a Chrome page with all your account profile, settings, and it just appears like your ordinary chrome page.
Run your code. But before that, remember to quit ALL CHROME sessions manually. For Mac, make sure that there is no dot under Chrome icon indicating that Chrome is not running for any circumstances. THIS STEP IS CRITICAL otherwise selenium will open a chrome page and it just stops there.
Here are all the steps. Again, this solution is vert informal and I personally don't think it is a "solution" to this problem. I will try to figure out a better way of achieving this in the future. But I still posted this as an alternative simply because I guess it might be helpful to some extent for somebody just like me. Hope it works for you, and good luck.
Run the following:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
# Get web driver going
cp = webdriver.ChromeOptions()
cp.add_argument("--browser.download.folderList=2")
cp.add_argument("--browser.helperApps.neverAsk.saveToDisk=image/jpg")
cp.add_argument("--browser.helperApps.neverAsk.saveToDisk=image/png")
cp.add_argument("--browser.download.dir=~/Downloads/")
driver = webdriver.Chrome(chrome_options=cp)
driver.get("http://www.google.com")
# Try to open a new tab
driver.find_element_by_tag_name("body").send_keys(Keys.CONTROL, 't')
This was an attempt to open a new tab, but the code does not work. This is also the case when trying to use Firefox. For Firefox, this does work if I don't change the profile (using equivalent code), but does not work with a custom profile.
I would also like to be able to send Ctrl+S too, but it seems no commands involving a special character work (though I can still send_keys normal text, not involving special keys like Ctrl).
What can I do to be able to send Ctrl+T and Ctrl+S (especially the latter)?
You can use action chain as given below.
ActionChains(driver).key_down(Keys.CONTROL).send_keys('s').key_up(Keys.CONTROL).perform()
I have a python script where i'm using Selenium to do some testing.
Currently I am trying to select some elements in multiple list boxes.
You can check out the site here: http://www.guloggratis.dk/annonce/opret (The language is Danish, if you are wondering)
So what I want to do is select a category, then some new categories pops up and I select one of those until there are no more categories left.
I can select the first element in the first box like this:
from selenium import webdriver
browser = webdriver.Firefox()
browser.get('http://www.guloggratis.dk/annonce/opret')
browser.find_element_by_css_selector("span.pos-absolute").click()
browser.find_element_by_id("categoryId_597").click()
While this works, I don’t know why i need to do
browser.find_element_by_css_selector("span.pos-absolute").click()
Anyways if I try select a category from the list that appears in the second box like this
from selenium import webdriver
browser = webdriver.Firefox()
browser.get('http://www.guloggratis.dk/annonce/opret')
browser.find_element_by_css_selector("span.pos-absolute").click()
browser.find_element_by_id("categoryId_597").click()
browser.find_element_by_id("categoryId_598”).click()
it doesn’t work even though "categoryId_598” is an id of one of the new categories.
I appreciate any help you might provide and I look forward to hear your answer.
If you have a problem reproducing the error, make a comment and I’ll instantly respond from my phone.
Maybe because of the cookie window?
from selenium import webdriver
browser = webdriver.Firefox()
browser.get('http://www.guloggratis.dk/annonce/opret')
browser.find_element_by_id("cookieClose").click()
browser.find_element_by_id("categoryId_597").click()
browser.find_element_by_id("categoryId_598").click()
works fine for me.