We Keep Coding

Equivalent of time.sleep() in Selenium

I realize this is a relatively simple question but I haven't found the answer yet.
I'm using driver.get() in a for loop that iterates through some urls. To help avoid my IP address getting blocked, I've implemented time.sleep(5) before the driver.get statement in the for loop.
Basically, I just want a wait period to make my scraping seem more natural.
I think time.sleep may be causing page crashes. What is the equivalent of time.sleep in selenium? From what I understand, implicitly_wait just sets the amount of time before throwing an exception, but I'm not sure that's what I want here? I want a specific amount of time for the driver to wait.

time.sleep()
The sleep() function is from the time module which suspends execution of the current thread for a given number of seconds.
Now, WebDriver being a out-of-process library which instructs the browser what to perform and at the same time the web browser being asynchronous in nature, WebDriver can't track the active, real-time state of the HTML DOM. This gives rise to some intermittent issues that arise from usage of Selenium and WebDriver those are subjected to race conditions that occur between the browser and the user’s instructions.
As of now Selenium doesn't have any identical method to time.sleep(), however there are two equavalent methods at your disposal and can be used as per the prevailing condition of your automated tests.
Implicit wait: In this case, WebDriver polls the DOM for a certain duration when trying to find any element. This can be useful when certain elements on the webpage are not available immediately and need some time to load.
def implicitly_wait(self, time_to_wait) -> None:
"""
Sets a sticky timeout to implicitly wait for an element to be found,
or a command to complete. This method only needs to be called one
time per session. To set the timeout for calls to
execute_async_script, see set_script_timeout.
:Args:
- time_to_wait: Amount of time to wait (in seconds)
:Usage:
::
driver.implicitly_wait(30)
"""
self.execute(Command.SET_TIMEOUTS, {
'implicit': int(float(time_to_wait) * 1000)})
Explicit wait: This type of wait allows your code to halt program execution, or freeze the thread, until the condition you pass it resolves. As an example:
presence_of_element_located()
visibility_of_element_located()
element_to_be_clickable()

There is no specific method in Selenium for hardcoded pauses like a time.sleep() general Python method.
As you mentioned there is an implicitly_wait and Expected Conditions explicit WebDriverWait waits but both these are NOT a hardcoded pauses.
Both the implicitly_wait WebDriverWait are used for setting the timeout - how long time to poll for some element presence or condition, so if that condition is fulfilled or the element is presented the program flow will immediately continue to the next code line.
So, if you want to put a pause you have to use some general Python method that will suspend the program / thread run like the time.sleep().

Selenium browser automation session expires when wait for 7 mins

I run selenium in python to do web testing, and I have noticed that, when I have added a longer wait in python, my selenium session is logged-out after a certain point of time.
Below Line of code I use to wait in the code
time.sleep(420)
I have tried to do some fake click during the wait period, but still, I have seen the security logout. is there any approach I can solve this issue?
I use python 3.5 and firefox web driver for testing.

What if you try to insert also a wait to Selenium.
In Java it looks like: SeleniumUtils.sleepQuietly(420);
There should also be a method for Python how to tell Selenium to wait (sleep) for certain time.
In your case, you have to say Selenium to wait, after you have called "the function" or click the button that needs to long (6 minutes) to execute.

when you use sleep() function you cannot do anything in that time. that is very useful between two lines of code where you need some time for processing like you are downloading file so, wait 5 seconds after 5 seconds add new parameters and download another file something like that.
you can surely do clicks without having sleep() function. between to click use sleep of 5 to 10 seconds like this your session will not expire it will run until you do driver.quit().

Python Selenium send request and avoid "Waiting for (website) ...."

I am launching several requests on different tabs. While one tab loads I will iteratively go to other tabs and see whether they have loaded correctly. The mechanism works great except for one thing: a lot of time is wasted "Waiting for (website)..."
The way in which I go from one tab to the other is launching an exception whenever a key element that I have to find is missing. But, in order to check for this exception (and therefore to proceed on other tabs, as it should do) what happens is that I have to wait for the request to end (so for the message "Waiting for..." to disappear).
Would it be possible not to wait? That is, would it be possible to launch the request via browser.get(..) and then immediately change tab?

Yes you can do that. You need to change the pageLoadStrategy of the driver. Below is an example of firefox
import time
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium import webdriver
cap = DesiredCapabilities.FIREFOX
cap["pageLoadStrategy"] = "none"
print(DesiredCapabilities.FIREFOX)
driver = webdriver.Firefox(capabilities=cap)
driver.get("http://tarunlalwani.com")
#execute code for tab 2
#execute code for tab 3
Nothing will wait now and it is up to you to do all the waiting. You can also use eager instead of none

Make Selenium wait 10 seconds

Yes I know the question has been asked quite often but I still don't get it. I want to make Selenium wait, no matter what. I tried these methods
driver.set_page_load_timeout(30)
driver.implicitly_wait(90)
WebDriverWait(driver, 10)
driver.set_script_timeout(30)
and other things but it does not work. I need selenium to wait 10 seconds. NO not until some element is loaded or whatever, just wait 10 seconds. I know there is this
try:
element_present = EC.presence_of_element_located((By.ID, 'whatever'))
WebDriverWait(driver, timeout).until(element_present)
except TimeoutException:
print "Timed out waiting for page to load"
I do not want that.
If waiting for some seconds is to much (not achievable) for selenium, what other (python) library's/programs would be capable to achieve this task? With Javas Selenium it does not seem to be a problem...

All the APIs you have mentioned is basically a timeout, so it's gonna wait until either some event happens or maximum time reached.
set_page_load_timeout - Sets the amount of time to wait for a page load to complete before throwing an error. If the timeout is negative, page loads can be indefinite.
implicitly_wait - Specifies the amount of time the driver should wait when searching for an element if it is not immediately present.
set_script_timeout - Sets the amount of time to wait for an asynchronous script to finish execution before throwing an error. If the timeout is negative, then the script will be allowed to run indefinitely.
For more information please visit this page. (documention is for JAVA binding, but functionality should be same for all the bindings)
So, if you want to wait selenium (or any script) 10 seconds, or whatever time. Then the best thing is to put that thread to sleep.
In python it would be
import time
time.sleep(10)
The simplest way to do this in Java is using
try {
Thread.sleep(10*1000);
} catch (InterruptedException e) {
e.printStackTrace();
}

Explicit Wait not timing out consistently in Selenium Python

I have the following code using Selenium in Python 3:
profile = webdriver.FirefoxProfile()
profile.set_preference('webdriver.load.strategy', 'unstable')
browser = webdriver.Firefox(profile)
browser.set_page_load_timeout(10)
url = 'my_url'
while True:
try:
st = time.time()
browser.get(url)
print('Finished get!')
time.sleep(2)
wait = WebDriverWait(browser, 10)
element = wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, 'div[my_attr="my_attr"]')))
print('Success after {} seconds.'.format(round(time.time()-st)))
break
except:
print('Timed out after {} seconds.'.format(round(time.time()-st)))
print('Reloading')
continue
From my understanding, using the explicit wait here (even with the unstable load strategy and page load timeout), what should happen is that the page should load, it should look for the element specified, and if either the page doesn't load within 10 seconds or the element is not found within 10ish seconds, it should time out and reload again (because of the try/except clause with the while loop).
However, what I'm finding is that it's not timing out consistently. For example, I've had instances where the loading times out after 10ish seconds the first time around, but once it reloads, it doesn't time out and instead "succeeds" after like 140 seconds. Or sometimes it doesn't time out at all and just keeps running until it succeeds. Because of the unstable load strategy, I don't think the page load itself is ever timing out (more specifically, the 'Finished get!' message always prints). But the explicit wait here that I specified also does not seem to be consistent. Is there something in my code that is overriding the timeouts? I want the timeouts to be consistent such that if either the page doesn't load or the element isn't located within 10ish seconds, I want it to timeout and reload. I don't ever want it to go on for 100+ seconds, even if it succeeds.
Note that I'm using the unstable webdriver load strategy here because the page I'm going to takes forever to completely load so I want to go straight through the code once the elements I need are found without needing the entire page to finish loading.

After some more testing, I have located the source of the problem. It's not that the waits are not working. The problem is that all the time is being taken up by the locator. I discovered this by essentially writing my own wait function and using the .find_element_by_css_selector() method, which is where all the runtime is occurring when it takes 100+ seconds. Because of the nature of my locator and the complexity of the page source, it's taking 100+ seconds sometimes for the locator to find the element when the page is nearly fully loaded. The locator time is not factored into the wait time. I presume that the only "solution" to this is to write a more efficient locator.