We Keep Coding

Can`t attach to detached selenium window in python

Cant send commands to selenium webdriver in detached session because link http://localhost:port died.
But if i put breakpoint 1 link stay alive
import multiprocessing
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
def create_driver_pool(q):
options = Options()
driver = webdriver.Chrome(options=options)
pass #breakpoint 1
return driver.command_executor._url
windows_pool = multiprocessing.Pool(processes=1)
result = windows_pool.map(create_driver_pool, [1])
print(result)
pass # breakpoint 2 for testing link
why is this happening and what can i do about it?

After some research i finally found the reason for this behavor.
Thanks https://bentyeh.github.io/blog/20190527_Python-multiprocessing.html and some googling about signals.
This is not signals at all.
I found this code in selenium.common.service
def __del__(self):
print("del detected")
# `subprocess.Popen` doesn't send signal on `__del__`;
# so we attempt to close the launched process when `__del__`
# is triggered.
try:
self.stop()
except Exception:
pass
This is handler for garbage collector function, that killing subprocess via SIGTERM
self.process.terminate()
self.process.wait()
self.process.kill()
self.process = None
But if you in the debug mode with breakpoint, garbage collector wont collect this object, and del wont start.

Restart python program whenever it encounters an exception

So I've got a selenium python script that occasionally runs into the following error at differing sections in my code:
Exception has occurred: WebDriverException
Message: unknown error: cannot determine loading status
from target frame detached
But when I encounter this error, if I re-run the program without changing anything, everything works again (so I know that it's either a problem with the website or the webdriver). In the meantime, I was wondering if there was a way to tell python to restart the program if a WebDriverException is encountered. Any help would be greatly appreciated, thank you.

You could try os.execv(), according to here, it enables a python script to be restarted, but you need to clean the buffers etc using this C-like function sys.stdout.flush()
try:
<your block of code>
except WebDriverException:
print(<Your Error>)
import os
import sys
sys.stdout.flush()
os.execv(sys.argv[0], sys.argv)

You could simply use the os module to do this: os.execv(sys.argv[0], sys.argv)
Use a main() function as a starting point for your program, and trigger that function again whenever you need to restart. Something like
def foo1():
return
def foo2():
try:
...
except:
main()
def main():
foo1()
foo2()
if __name__ == "main":
main()

If the error occurs because it does not find a certain tag, you could put a wait
element = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.ID, "myDynamicElement"))
)
or worst case
browser.implicitly_wait(5)
or
time.sleep(5)
which waits 5 seconds for the element to appear

How to handle console exit and object destruction

Given this code:
from time import sleep
class TemporaryFileCreator(object):
def __init__(self):
print 'create temporary file'
# create_temp_file('temp.txt')
def watch(self):
try:
print 'watching tempoary file'
while True:
# add_a_line_in_temp_file('temp.txt', 'new line')
sleep(4)
except (KeyboardInterrupt, SystemExit), e:
print 'deleting the temporary file..'
# delete_temporary_file('temp.txt')
sleep(3)
print str(e)
t = TemporaryFileCreator()
t.watch()
during the t.watch(), I want to close this application in the console..
I tried using CTRL+C and it works:
However, if I click the exit button:
it doesn't work.. I checked many related questions about this but it seems that I cannot find the right answer..
What I want to do:
The console can be exited while the program is still running.. to handle that, when the exit button is pressed, I want to make a cleanup of the objects (deleting of created temporary files), rollback of temporary changes, etc..
Question:
how can I handle console exit?
how can I integrate it on object destructors (__exit__())
Is it even possible? (how about py2exe?)
Note: code will be compiled on py2exe.. "hopes that the effect is the same"

You may want to have a look at signals. When a *nix terminal is closed with a running process, this process receives a couple signals. For instance this code waits for the SIGHUB hangup signal and writes a final message. This codes works under OSX and Linux. I know you are specifically asking for Windows but you might want to give it a shot or investigate what signals a Windows command prompt is emitting during shutdown.
import signal
import sys
def signal_handler(signal, frame):
with open('./log.log', 'w') as f:
f.write('event received!')
signal.signal(signal.SIGHUP, signal_handler)
print('Waiting for the final blow...')
#signal.pause() # does not work under windows
sleep(10) # so let us just wait here
Quote from the documentation:
On Windows, signal() can only be called with SIGABRT, SIGFPE, SIGILL, SIGINT, SIGSEGV, or SIGTERM. A ValueError will be raised in any other case.
Update:
Actually, the closest thing in Windows is win32api.setConsoleCtrlHandler (doc). This was already discussed here:
When using win32api.setConsoleCtrlHandler(), I'm able to receive shutdown/logoff/etc events from Windows, and cleanly shut down my app.
And if Daniel's code still works, this might be a nice way to use both (signals and CtrlHandler) for cross-platform purposes:
import os, sys
def set_exit_handler(func):
if os.name == "nt":
try:
import win32api
win32api.SetConsoleCtrlHandler(func, True)
except ImportError:
version = “.”.join(map(str, sys.version_info[:2]))
raise Exception(”pywin32 not installed for Python ” + version)
else:
import signal
signal.signal(signal.SIGTERM, func)
if __name__ == "__main__":
def on_exit(sig, func=None):
print "exit handler triggered"
import time
time.sleep(5)
set_exit_handler(on_exit)
print "Press to quit"
raw_input()
print "quit!"

If you use tempfile to create your temporary file, it will be automatically deleted when the Python process is killed.
Try it with:
>>> foo = tempfile.NamedTemporaryFile()
>>> foo.name
'c:\\users\\blah\\appdata\\local\\temp\\tmpxxxxxx'
Now check that the named file is there. You can write to and read from this file like any other.
Now kill the Python window and check that file is gone (it should be)
You can simply call foo.close() to delete it manually in your code.

Selenium webdriver + PhantomJS processes not closing

Here's just about the simplest open and close you can do with webdriver and phantom:
from selenium import webdriver
crawler = webdriver.PhantomJS()
crawler.set_window_size(1024,768)
crawler.get('https://www.google.com/')
crawler.quit()
On windows (7), every time I run my code to test something out, new instances of the conhost.exe and phantomjs.exe processes begin and never quit. Am I doing something stupid here? I figured the processes would quit when the crawler.quit() did...

Go figure. Problem resolved with a reboot.

Rebooting is not a solution for this problem. I have experimented this hack in LINUX system. Try modifying the stop() function defined in service.py
def stop(self):
"""
Cleans up the process
"""
if self._log:
self._log.close()
self._log = None
#If its dead dont worry
if self.process is None:
return
#Tell the Server to properly die in case
try:
if self.process:
self.process.stdin.close()
#self.process.kill()
self.process.send_signal(signal.SIGTERM)
self.process.wait()
self.process = None
except OSError:
# kill may not be available under windows environment
pass
Added line send_signal explicitly to give the signal to quit phantomjs process. Don't forget to add import signal statement at start of this file.

Python use signal alarm in order to limit function execution time dosnt always stop the method

I need to limit function execution time, so i followed Josh Lee answer
try:
with time_limit(10):
long_function_call()
except TimeoutException, msg:
print "Timed out!"
where long_function_call() is a selenium webdriver function that interact with a page and do some operations.
def long_function_call(self, userName, password):
driver = self.initDriver()
try:
driver.get("https://yyyy.com")
time.sleep(2)
if not self.isHttps(driver.current_url):
isHttps = False
driver.find_element_by_id("i015516").clear()
time.sleep(5)
if 'https://yyy.com' not in driver.current_url:
self.raiseFailedToLogin('yyy')
except Exception as e:
self.raiseException('yyy',e)
finally:
driver.close()
driver.quit()
return 'yyyy'
In most cases , when function execution time exceed the signal timeout signal was sent and method stopped, but in some cases the method exceed the timeout and didnt stop. it seems that selenium is hang.(the firefox is open and nothing is done in it).
I tried to pause debugger in these cases , but pause didn't show me where it hang.
If i close the selenium firefox than the debug pause stop on this method:
_read_status [httplib.py:366]
begin [httplib.py:407]
getresponse [httplib.py:1030]
do_open [urllib2.py:1180]
http_open [urllib2.py:1207]
def _read_status(self):
# Initialize with Simple-Response defaults
line = self.fp.readline()
if self.debuglevel > 0: ################Hang here
Any idea why in some cases signal alarm with selenium didnt work? (i dont think they catch interrupt).

This is a very interesting problem you are facing. I've created an example which can demonstrate some of the issues you may face when using with time_limit. If you run the code below you may expect that after 1 second a TimeoutException will be raised which will then exit python and the running thread as well as the xterm should all exit. What happens is after one second the TimeoutException is raised and you will see the "Timed out!" appear in the terminal but both the thread and xterm will continue their execution. This maybe the type of scenario you are facing with selenium. I don't know how selenium is implemented but the firefox process must be spawned in a way similar to how xterm is in the example.
simple approach
import time, os, sys, subprocess, threading
def worker():
for i in range(30):
print i
time.sleep(0.1)
try:
with time_limit(1):
threading.Thread(target=worker).start()
subprocess.check_output('/usr/bin/xterm')
except TimeoutException, msg:
print "Timed out!"
a potential solution to this problem is the following code. It will kill all children of this python program which would kill xterm in this case, and then it would kill itself. This is of course a forceful way to exit but would guarantee everything ended on timeout.
kill them all
subprocess.call('pkill -P %s' % os.getpid(), shell=True)
subprocess.call('kill -9 %s' % os.getpid(), shell=True)
taking into account your comments below another approach would be to have another thread that would perform the kill operations if an operation in the main thread exceeded the specified timeout. This is an implementation of that approach with an example call.
thread that watches and kills on timeout
import time, os
from subprocess import check_output, call
from threading import Thread
from contextlib import contextmanager
def wait_thread(active, duration):
start = time.time()
while active[0] and time.time() - start < duration:
time.sleep(0.1)
if active[0]:
call('pkill -P %s' % os.getpid(), shell=True)
call('kill -9 %s' % os.getpid(), shell=True)
#contextmanager
def wait(duration):
active = [True]
Thread(target=wait_thread, args=(active, duration)).start()
yield
active[0] = False
with wait(1):
time.sleep(0.5)
print 'finished safely before timeout'
with wait(1):
call('/usr/bin/xterm')