Python Selenium Firefox on Ubuntu: New Tabs Not Working - python

I'm doing remote web crawling and scraping, and hoping not to reload a new browser window for every link on one page.
The problem is that new tabs are not opening up with my Firefox web driver.
Here's what I've tried:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.action_chains import ActionChains
from pyvirtualdisplay import Display
# launch our headless display
display = Display(visible=0, size=(800, 600))
display.start()
# launch our web driver and get a page
browser = webdriver.Firefox()
browser.get("http://www.google.com/")
# try to open a new tab
ActionChains(browser).key_down(Keys.CONTROL).send_keys("t").key_up(Keys.CONTROL).perform()
# this should print 2, but it prints 1, because new tab not opened
print len(browser.window_handles)
# clean up everything
browser.quit()
display.stop()
Specifications:
Ubuntu 14.04.2
Python 2.7.6
Selenium 2.47.1
PyVirtualDisplay 0.1.3

Based on this response from a Selenium developer, new tabs in Firefox are not supported as of August 2015. He suggested exploring Marionette but currently its dependencies cause more trouble than it's worth, at least for my use case. His solution is to just use new windows (driver.execute_script("window.open()")), instead of new tabs.

Related

selenium firefox driver no results returned

I am using VSC Editor, Windows 10, the latest version of Python and all associated plugins.
The browser I am using is FireFox (latest version as well).
When I run my code, a headless browser appears and I can see that my search criteria is entered in to the text box, the search button hit and the results are displayed in the browser.
However, in my code, when I print out the results, I get an empty list [].
What am I doing wrong?
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.common.keys import Keys
driver = webdriver.Firefox()
driver.get("https://duckduckgo.com")
search_form = driver.find_element_by_id("search_form_input_homepage")
search_form.clear()
search_form.send_keys("python jobs remote")
search_form.submit()
results = driver.find_elements_by_class_name("result")
print(results[0])
driver.close()

Selenium web driver Firefox opening blank page

I'd like to ask something about Selenium library in Python.
I'm trying to open a webpage, directly log onto it, and access another webpage behind it (I wanted to navigate on the website after the login) with a Python script. I've found the following code on the Internet but I have a problem with the line:
browser = webdriver.Firefox()
It just opens a blank page in Firefox and it looks like the script get stuck with it and does nothing afterwards. I tried in the Python interpreter and it's the same, it opens a blank page in Firefox and I lose the hand (I can't enter other commands).
python interpreter blocked:
I'm using Selenium-3.3.1 and I work under CentOS 6.5.
Is it normal? Am I missing something obvious?
Here is my code:
#!usr/bash/python
from selenium import webdriver
from selenium.webdriver.support import ui
from selenium.webdriver.common.keys import Keys
def loadedPage(browser):
return browser.find_element_by_tag_name("body") != None
browser = webdriver.Firefox() #supposedly just a firefox webdrive instance creation
browser.get("http://machine/machineDir/index.php")
wait = ui.WebDriverWait(browser, 10)
wait.until(loadedPage)
username=browser.find_element_by_id("username")
username.send_keys("userTest")
passwd=browser.find_element_by_id("password")
passwd.send_keys("userTestpass")
passwd.send_keys(Keys.RETURN)
As you are using selenium 3, firefox browser can't be instantiate directly, you need to configure gecko driver for the same.
System.setProperty("webdriver.gecko.driver","path of geckodriver.exe");
I fixed it using the right version of Selenium for my old Firefox.
Firefox version: 17.0.10
Selenium version installed: 2.40

Full screen Firefox Python Selenium

I'm trying to start a full screen page in Firefox with Selenium in Python 3. The page opening works fine, but when I send the F11 key to the browser (the Full Screen key), anything happens. Here is my code :
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
firefox = webdriver.Firefox()
firefox.get('http://localhost')
firefox.maximize_window()
body = firefox.find_element_by_tag_name('html')
body.send_keys(Keys.F11)
Does anyone know how to make my page start in full screen ? I know it's possible with Chrome, but it's harder with Firefox
This is what worked for me.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Firefox()
driver.get('http://localhost')
driver.find_element_by_xpath('/html/body').send_keys(Keys.F11)
Hope this helps
Just realized I am using Python 2.6 vs your 3. Sorry about that, but at least you know it will work on an older Python version
Selenium has a built-in method to make the window fullscreen: fullscreen_window()
Selenium API - fullscreen_window()
Invokes the window manager-specific ‘full screen’ operation
browser.get("https://www.screenku.com")
browser.fullscreen_window()
pip3 install selenium pyautogui
#!/usr/bin/env python3
import pyautogui
from selenium import webdriver
profile = webdriver.FirefoxProfile()
profile.set_preference("dom.webnotifications.enabled", False)
profile.set_preference("general.useragent.override", "Mozilla/5.0")
profile.update_preferences()
browser = webdriver.Firefox(firefox_profile=profile,executable_path = '/usr/local/bin/geckodriver')
browser.get("https://www.screenku.com")
pyautogui.press('f11')

Selenium Stucks on pyvirtualdisplay

I am running Selenium python on server where I need to hide chrome display. Python script runs most of time but sometimes it stucks when it creates new chromedriver session. Don't have any idea why it stucks sometimes.
Snippet Code:
from selenium import webdriver
from pyvirtualdisplay import Display
chromedriver = '/usr/local/bin/chromedriver'
os.environ['webdriver.chrome.driver'] = chromedriver
display = Display(visible=0, size=(800,600))
display.start()
driver = webdriver.Chrome("/usr/local/bin/chromedriver") => Stuck here
driver.get("example.com")
I just currently set up Selenium on my server. If you get your permission right, try to put this line.
options = webdriver.ChromeOptions()
options.add_argument("--no-sandbox")
driver = webdriver.Chrome(desired_capabilities=options.to_capabilities())
to turn off the sandbox.

Is it possible to use Selenium/Python without spawning a visible window? Can I use lxml with it?

I expect to be able to save a page and then use a lxml.html.parse() but I was wondering if I could do it directly off a opened page?
I'm using Ubuntu if it makes any difference.
Edit: There's a method to use xpath directly(find_element_by_xpath), so I guess I don't need lxml. But to save the page all you have to do is call the page_source method.
To answer the 'use Selenium without spawning a visible window' question, yes you can use PyVirtualDisplay on Ubunutu easily.
from pyvirtualdisplay import Display
from selenium import webdriver
display = Display(visible=0, size=(800, 600))
display.start()
# now Firefox will run in a virtual display.
# you will not see the browser.
browser = webdriver.Firefox()
browser.get('http://www.google.com')
print browser.title
browser.quit()
display.stop()
Code is from this blog post

Categories