I am using VSC Editor, Windows 10, the latest version of Python and all associated plugins.
The browser I am using is FireFox (latest version as well).
When I run my code, a headless browser appears and I can see that my search criteria is entered in to the text box, the search button hit and the results are displayed in the browser.
However, in my code, when I print out the results, I get an empty list [].
What am I doing wrong?
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.common.keys import Keys
driver = webdriver.Firefox()
driver.get("https://duckduckgo.com")
search_form = driver.find_element_by_id("search_form_input_homepage")
search_form.clear()
search_form.send_keys("python jobs remote")
search_form.submit()
results = driver.find_elements_by_class_name("result")
print(results[0])
driver.close()
Related
I am grinding through day 2 of me learning Python 3.7 with Selenium.
I am accessing a web page using WebDriver. I have been making progress, but am stymied now. Though I can easily disable the Firefox password manager popup window on my normal Browser (Options/Privacy and Security/Location/Settings), my script's remotely-run (think that is by definition) browser does not recognize that configuration, and the Firefox popup shows up.
The script can ignore the popup and navigate the target site until the very last page that I need to access. At that point, the HTML for that page is inaccessible, until I manually click on the Firefox popup, dismissing it. As soon as I do that, the HTML code for that web page lights up in Firefox Web Developer Inspector.
Now, that HTML code may be inaccessible for other reasons (like I said, day 2 of the learning curve), but is there some library or commands within Webdriver that allow me to automate the dismissal of that FireFox popup. It is not part of the HTML of any page, so I am at a loss.
Edit: I should mention also, the bulk of that last page's content is blank until I manually dismiss the FireFox popup.
I have added the following code, but still am getting the same popup:
from selenium import webdriver
#Using Firefox to access the Web
options = webdriver.FirefoxOptions()
options.set_preference("dom.webnotifications.enabled", False)
driver = webdriver.Firefox(options=options)
driver.maximize_window()
Second Edit: This is the current code section defining the profile, and I am still getting the pop up password manager.
import datetime
import time
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
#Using Firefox to access the Web
profile = webdriver.FirefoxProfile()
#profile.set_preference("dom.push.enabled", False)
profile.set_preference("dom.webnotifications.enabled", False)
profile.update_preferences()
driver = webdriver.Firefox(firefox_profile=profile)
driver.maximize_window()
about:config
preference security.insecure_field_warning.contextual.enabled to false
the issue am dealing with is trying to get selenium to run in the background while getting data like webpage elements xpath and ids and being able to use it while remaining activity in the background and not keep poping up browser tab in front of other running programs
You should try running your browser in headless mode. Here is a code snippet of the function that gives the instance of chrome in headless mode.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
def browser_open(headless=False):
options = Options()
options = webdriver.ChromeOptions()
options.add_argument("disable-gpu")
options.add_argument("no-default-browser-check")
options.add_argument("no-first-run")
options.add_argument("no-sandbox")
options.add_argument("window-size=1300x744")
if headless == True:
options.add_argument("headless")
chrome_browser = webdriver.Chrome(executable_path=os.path.join(os.getcwd(), "chromedriver"), chrome_options=options)
chrome_browser.maximize_window()
return chrome_browser
From a Jupyter Notebook running Python 3, I'm running the following code to try and insert the text "a search term" into the Google search box with Selenium [1]:
import os
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
# get the path of ChromeDriverServer
chrome_driver_path = os.getcwd() + "/chromedriver 2"
# create a new Chrome session
driver = webdriver.Chrome(chrome_driver_path)
# navigate to the application home page
driver.get("https://www.google.com")
# get the search textbox
search_field = driver.find_element_by_name("q")
# enter search keyword and submit
search_field.send_keys("a search term")
However, when I run the code, I just get a new browser with the Google homepage open but the text isn't inserted into the search bar. What am I missing here?
[1] Make sure to have the right webdriver installed -- I initially installed the 32-bit version but then had to install the 64-bit version, since the "2".
The reason was because my 64-bit Chrome Web driver was outdated; the most recent version as of this writing is 2.37 and can be downloaded here: https://chromedriver.storage.googleapis.com/index.html?path=2.37/
Code now works.
I'd like to ask something about Selenium library in Python.
I'm trying to open a webpage, directly log onto it, and access another webpage behind it (I wanted to navigate on the website after the login) with a Python script. I've found the following code on the Internet but I have a problem with the line:
browser = webdriver.Firefox()
It just opens a blank page in Firefox and it looks like the script get stuck with it and does nothing afterwards. I tried in the Python interpreter and it's the same, it opens a blank page in Firefox and I lose the hand (I can't enter other commands).
python interpreter blocked:
I'm using Selenium-3.3.1 and I work under CentOS 6.5.
Is it normal? Am I missing something obvious?
Here is my code:
#!usr/bash/python
from selenium import webdriver
from selenium.webdriver.support import ui
from selenium.webdriver.common.keys import Keys
def loadedPage(browser):
return browser.find_element_by_tag_name("body") != None
browser = webdriver.Firefox() #supposedly just a firefox webdrive instance creation
browser.get("http://machine/machineDir/index.php")
wait = ui.WebDriverWait(browser, 10)
wait.until(loadedPage)
username=browser.find_element_by_id("username")
username.send_keys("userTest")
passwd=browser.find_element_by_id("password")
passwd.send_keys("userTestpass")
passwd.send_keys(Keys.RETURN)
As you are using selenium 3, firefox browser can't be instantiate directly, you need to configure gecko driver for the same.
System.setProperty("webdriver.gecko.driver","path of geckodriver.exe");
I fixed it using the right version of Selenium for my old Firefox.
Firefox version: 17.0.10
Selenium version installed: 2.40
I'm trying to start a full screen page in Firefox with Selenium in Python 3. The page opening works fine, but when I send the F11 key to the browser (the Full Screen key), anything happens. Here is my code :
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
firefox = webdriver.Firefox()
firefox.get('http://localhost')
firefox.maximize_window()
body = firefox.find_element_by_tag_name('html')
body.send_keys(Keys.F11)
Does anyone know how to make my page start in full screen ? I know it's possible with Chrome, but it's harder with Firefox
This is what worked for me.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Firefox()
driver.get('http://localhost')
driver.find_element_by_xpath('/html/body').send_keys(Keys.F11)
Hope this helps
Just realized I am using Python 2.6 vs your 3. Sorry about that, but at least you know it will work on an older Python version
Selenium has a built-in method to make the window fullscreen: fullscreen_window()
Selenium API - fullscreen_window()
Invokes the window manager-specific ‘full screen’ operation
browser.get("https://www.screenku.com")
browser.fullscreen_window()
pip3 install selenium pyautogui
#!/usr/bin/env python3
import pyautogui
from selenium import webdriver
profile = webdriver.FirefoxProfile()
profile.set_preference("dom.webnotifications.enabled", False)
profile.set_preference("general.useragent.override", "Mozilla/5.0")
profile.update_preferences()
browser = webdriver.Firefox(firefox_profile=profile,executable_path = '/usr/local/bin/geckodriver')
browser.get("https://www.screenku.com")
pyautogui.press('f11')