To run my functional tests i use LiveServerTestCase.
I want to call set_speed (and other methods, set_speed is just an example) that aren't in the webdriver, but are in the selenium object.
http://selenium.googlecode.com/git/docs/api/py/selenium/selenium.selenium.html#module-selenium.selenium
my subclass of LiveServerTestCase
from selenium import webdriver
class SeleniumLiveServerTestCase(LiveServerTestCase):
#classmethod
def setUpClass(cls):
cls.driver = webdriver.Firefox()
cls.driver.implicitly_wait(7)
cls.driver.maximize_window()
# how to call selenium.selenium.set_speed() from here? how to get the ref to the selenium object?
super(SeleniumLiveServerTestCase, cls).setUpClass()
How to get that? I can't call the constructor on selenium, i think.
You don't. Setting the speed in WebDriver is not possible and the reason for this is that you generally shouldn't need to, and the 'waiting' is now done at a different level.
Before it was possible to tell Selenium, don't run this at normal speed, run it at a slower speed to allow more things to be available on page load, for slow loading pages or AJAX'ified pages.
Now, you do away with that altogether. Example:
I have a login page, I login and once logged in I see a "Welcome" message. The problem is the Welcome message is not displayed instantly and is on a time delay (using jQuery).
Pre WebDriver Code would dictate to Selenium, run this test, but slow down here so we can wait until the Welcome message appears.
Newer WebDriver code would dictate to Selenium, run this test, but when we login, wait up to 20 seconds for the Welcome Message to appearing, using explicit waits.
Now, if you really want access to "set" Selenium's speed, first off I'd recommend against it but the solution would be to dive into the older, now deprecated code.
If you use WebDriver heavily already, you can use the WebDriverBackedSelenium which can give you access to the older Selenium methods, whilst keeping the WebDriver backing the same, therefore much of your code would stay the same.
https://groups.google.com/forum/#!topic/selenium-users/6E53jIIT0TE
Second option is to dive into the old Selenium code and use it, this will change a lot of your existing code (because it is before the "WebDriver" concept was born).
The code for both Selenium RC & WebDriverBackedSelenium lives here, for the curious:
https://code.google.com/p/selenium/source/browse/py/selenium/selenium.py
Something along the lines of:
from selenium import webdriver
from selenium import selenium
driver = webdriver.Firefox()
sel = selenium('localhost', 4444, '*webdriver', 'http://www.google.com')
sel.start(driver = driver)
You'd then get access to do this:
sel.setSpeed(5000)
Related
I've discovered that even if I specify:
webdriver.Firefox(
firefox_profile=webdriver.FirefoxProfile("/my/profile/path")
)
Selenium actually clones that profile and launches the browser based on that temporary clone. I'm sure this makes sense when you're using Selenium for testing, but as I'm using it for automation, I'd like to operate directly on the existing profile.
Is this possible with Selenium? Really I'm looking for a way to maintain state between sessions (bookmarks, history, Firefox sync setup, etc.) so a persistent session makes the most sense to me. I'm doing this work in Python, but I imagine that the pattern used in other languages would be similar.
Try this
from selenium import webdriver
myprofile = webdriver.FirefoxProfile(r'C:\Users\{YOUR OWN USERNAME}\AppData\Roaming\Mozilla\Firefox\Profiles\moskcpdq.SeleniumTest')
driver = webdriver.Firefox(firefox_profile=myprofile, executable_path=r'C:\Utility\BrowserDrivers\geckodriver.exe')
driver.get('https://www.google.co.in')
print("Page Title is : %s" %driver.title)
driver.quit()
Make sure you are using the correct path for the profile!
I would like to run my script on Multiple browser using selenium.
As of now I am able to perform the operation by opening one browser at a time.
Eg:- Register to amazon.
I want to be able to Register two users to amazon at the same time.
This is the code I have as of now.
import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.select import Select
driver.get("https://www.amazon.com/ap/register?openid.pape.max_auth_age=0&openid.return_to=https%3A%2F%2Fwww.amazon.com%2F%3Fref_%3Dnav_signin&prevRID=VBHFJ50CPKFJ3PGG7RDY&openid.identity=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.assoc_handle=usflex&openid.mode=checkid_setup&openid.ns.pape=http%3A%2F%2Fspecs.openid.net%2Fextensions%2Fpape%2F1.0&prepopulatedLoginId=&failedSignInCount=0&openid.claimed_id=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&pageId=usflex&openid.ns=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0")
driver.find_element_by_xpath("""//*[#id="s2id_ID_form4a8055de_guest_register_sponsor_lookup"]/a/span[2]/b""").click()
driver.find_element_by_xpath("""//*[#id="s2id_autogen1_search"]""").send_keys(v1)
By using this I can run it for one user at one time. But I want to be able to register more than two users upto n users at the same time.
Hence, the multiple windows questions.
You could create multiple instances of the webdriver. You can then manipulate each individually. For example,
from selenium import webdriver
driver1 = webdriver.Chrome()
driver2 = webdriver.Chrome()
driver1.get("http://google.com")
driver2.get("http://yahoo.com")
This question is a bit old at this point, but I still found it applicable to something I was having trouble with today.
In order to achieve parallel processes you need to utilize multiprocessing. Essentially, this allows you to create browser instances for each function and allow each script to lock to each browser GIL separately. You can then start each of the processes in your main code and they will all execute in parallel.
If you need an explanation on how to do this, a great video can be found here
I have over 19,000 links which I need to visit to scrape data from. Each takes about 5 seconds to fully load, which means that I will need slightly more than 26 hours to scrape everything on a single webdriver.
To me, it seems that a solution is simply to start another webdriver (or few others) in a separate python notebook which goes through another portion of the links in parallel. i.e:
In first iPython notebook:
from selenium import webdriver
driver1 = webdriver.Firefox()
... scraping code looping over links 0-9500 using driver1...
In second iPython notebook:
from selenium import webdriver
driver2 = webdriver.Firefox()
... scraping code looping over links 9501-19000 using driver2...
I'm fairly new to scraping so this question may be completely elementary/ridiculous(?). However, I've tried searching for this and haven't seen anything on the topic, so I would appreciate any advice on this matter. Or any recommendations for a better/more correct way to implement this.
I've heard of multi-threading using the thread module (http://www.tutorialspoint.com/python/python_multithreading.htm), but wonder whether implementing it in this manner would have any advantage over simply creating multiple webdrivers as in the aforementioned code.
You really need to use Selenium in order to do this?
Check Scrapy with this framework you can easily send a lots of request and scrape data. Selenium is useful to get browser automation.
I am writing a bot using Python with Selenium module.When I open a webpage with my bot, since the webpage contains too many external sources than dom, it takes a lot to get all of the page loaded. I used the explicit and implicit waits to eliminate this problem since I just wanted a specific element to be loaded and not all of the webpage, it didn't work. The problem is If i run the following statement:
driver = webdriver.Firefox()
driver.get('somewebpage')
elm = WebDriverWait(driver, 5).until(ExpectedConditions.presence_of_element_located((By.ID, 'someelementID'))
elm.click()
It doesn't work since the Selenium waits for the driver.get() to fully retrieve the webpage and then, it proceeds further. Now I want to write a code that sets a timeout for driver.get(), Like:
driver.get('somewebpage').timeout(5)
Where the driver.get() stops loading the page after 5 secs and the program flow proceeds, whether the driver.get() fully loaded webpage or not.
By the way, I have searched about the feature that I said above, and came across there:
Selenium WebDriver go to page without waiting for page load
But the problem is that the answer in the above link does not say anything about the Python equivalent code.
How do I accomplish the future that I am searching for?
python equivalent code for the question mentioned in the current question (Selenium WebDriver go to page without waiting for page load):
from selenium import webdriver
profile = webdriver.FirefoxProfile()
profile.set_preference('webdriver.load.strategy', 'unstable')
driver = webdriver.Firefox(profile)
and:
driver.set_page_load_timeout(5)
There is a ton of questions on this, here is an example. Here is an example that waits until all jquery ajax calls have completed or a 5 second timeout.
from selenium.webdriver.support.ui import WebDriverWait
WebDriverWait(driver, 5).until(lambda s: s.execute_script("return jQuery.active == 0"))
It was a really tedious issue to solve. I just did the following and the problem got resolved:
driver= webdriver.Firefox()
driver.set_page_load_timeout(5)
driver.get('somewebpage')
It worked for me using Firefox driver (and Chrome driver as well).
Alright, I'm confused. So I want to scrape a page using Selenium Webdriver and Python. I've recorded a test case in the Selenium IDE. It has stuff like
Command Taget
click link=14
But I don't see how to run that in Python. The desirable end result is that I have the source of the final page.
Is there a run_test_case command? Or do I have to write individual command lines? I'm rather missing the link between the test case and the actual automation. Every site tells me how to load the initial page and how to get stuff from that page, but how do I enter values and click on stuff and get the source?
I've seen:
submitButton=driver.find_element_by_xpath("....")
submitButton.click()
Ok. And enter values? And get the source once I've submitted a page? I'm sorry that this is so general, but I really have looked around and haven't found a good tutorial that actually shows me how to do what I thought was the whole point of Selenium Webdriver.
I've never used the IDE. I just write my tests or site automation by hand.
from selenium import webdriver
browser = webdriver.Firefox()
browser.get("http://www.google.com")
print browser.page_source
You could put that in a script and just do python wd_script.py or you could open up a Python shell and type it in by hand, watch the browser open up, watch it get driven by each line. For this to work you will obviously need Firefox installed as well. Not all versions of Firefox work with all versions of Selenium. The current latest versions of each (Firefox 19, Selenium 2.31) do though.
An example showing logging into a form might look like this:
username_field = browser.find_element_by_css_selector("input[type=text]")
username_field.send_keys("my_username")
password_field = browser.find_element_by_css_selector("input[type=password]")
password_field.send_keys("sekretz")
browser.find_element_by_css_selector("input[type=submit]").click()
print browser.page_source
This kind of stuff is much easier to write if you know css well. Weird errors can be caused by trying to find elements that are being generated in JavaScript. You might be looking for them before they exist for instance. It's easy enough to tell if this is the case by putting in a time.sleep for a little while and seeing if that fixes the problem. More elegantly you can abstract some kind of general wait for element function.
If you want to run Webdriver sessions as part of a suite of integration tests then I would suggest using Python's unittest to create them. You drive the browser to the site under test, and make assertions that the actions you are taking leave the page in a state you expect. I can share some examples of how that might work as well if you are interested.