Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 7 years ago.
Improve this question
I was messing around with a zip file cracker and decided to use the multiprocessing module to speed the process up. It was a complete pain since it was my first time using the module and I don't even fully understand it yet. However, I got it to work.
The problem is that it doesn't complete the word list; it just stops at random puts during the word list, and if the password is found it continues to go through the word list instead of just stopping the process.
Does anyone know why it's exhibiting this behaviour?
Source Code For ZipFile Cracker
#!/usr/bin/env python3
import multiprocessing as mp
import zipfile # Handeling the zipfile
import sys # Command line arguments, and quiting application
import time # To calculate runtime
def usage(program_name):
print("Usage: {0} <path to zipfile> <dictionary>".format(program_name))
sys.exit(1)
def cracker(password):
try:
zFile.extractall(pwd=password)
print("[+] Password Found! : {0}".format(password.decode('utf-8')))
pool.close()
except:
pass
def main():
global zFile
global pool
if len(sys.argv) < 3:
usage(sys.argv[0])
zFile = zipfile.ZipFile(sys.argv[1])
print("[*] Started Cracking")
startime = time.time()
pool = mp.Pool()
for i in open(sys.argv[2], 'r', errors='ignore'):
pswd = bytes(i.strip('\n'), 'utf-8')
pool.apply_async(cracker, (pswd,))
print (pswd)
runtime = round(time.time() - startime, 5)
print ("[*] Runtime:", runtime, 'seconds')
sys.exit(0)
if __name__ == "__main__":
main()
You are terminating your program too early. To test this out, add a harmless time.sleep(10) in the cracker method and observe your program still terminating within a second.
Call join to wait for the pool to finish:
pool = mp.Pool()
for i in open(sys.argv[2], 'r', errors='ignore'):
pswd = bytes(i.strip('\n'), 'utf-8')
pool.apply_async(cracker, (pswd,))
pool.close() # Indicate that no more data is coming
pool.join() # Wait for pool to finish processing
runtime = round(time.time() - startime, 5)
print ("[*] Runtime:", runtime, 'seconds')
sys.exit(0)
Additionally, once you find the right password, calling close just indicates that no more future tasks are coming - all tasks already submitted will still be done. Instead, call terminate to kill the pool without processing any more tasks.
Furthermore, depending on the implementation details of multiprocessing.Pool, the global variable pool may not be available when you need it (and its value isn't serializable anyways). To solve this problem, you can use a callback, as in
def cracker(password):
try:
zFile.extractall(pwd=password)
except RuntimeError:
return
return password
def callback(found):
if found:
pool.terminate()
...
pool.apply_async(cracker, (pswd,), callback=cb)
Of course, since you now look at the result all the time, apply is not the right way to go. Instead, you can write your code using imap_unordered:
with open(sys.argv[2], 'r', errors='ignore') as passf, \
multiprocessing.Pool() as pool:
passwords = (line.strip('\n').encode('utf-8') for line in passf)
for found in pool.imap_unordered(cracker, passwords):
if found:
break
Instead of using globals, you may also want to open the zip file (and create a ZipFile object) in each process, by using an initializer for the pool. Even better (and way faster), forgo all of the I/O in the first place and read just the bytes you need once and then pass them on to the children.
phihag's answer is the correct solution.
I just wanted to provide an additional detail regarding calling terminate() when you've found the correct password. The pool variable in cracker() was not defined when I ran the code. So trying to invoke it from there simply threw an exception:
NameError: name 'pool' is not defined
(My fork() experience is weak, so I don't completely understand why the global zFile is copied to the child processes successfully while pool is not. Even if it were copied, it would not be the same pool in the parent process, right? So any methods invoked on it would have no effect on the real pool in the parent process. Regardless, I prefer this advice listed within the multiprocessing module's Programming guidelines section: Explicitly pass resources to child processes.)
My suggestion is to make cracker() return the password if it is correct, otherwise return None. Then pass a callback to apply_async() that records the correct password, as well as terminating the pool. Here's my take at modifying your code to do this:
#!/usr/bin/env python3
import multiprocessing as mp
import zipfile # Handeling the zipfile
import sys # Command line arguments, and quiting application
import time # To calculate runtime
import os
def usage(program_name):
print("Usage: {0} <path to zipfile> <dictionary>".format(program_name))
sys.exit(1)
def cracker(zip_file_path, password):
print('[*] Starting new cracker (pid={0}, password="{1}")'.format(os.getpid(), password))
try:
time.sleep(1) # XXX: to simulate the task taking a bit of time
with zipfile.ZipFile(zip_file_path) as zFile:
zFile.extractall(pwd=bytes(password, 'utf-8'))
return password
except:
return None
def main():
if len(sys.argv) < 3:
usage(sys.argv[0])
print('[*] Starting main (pid={0})'.format(os.getpid()))
zip_file_path = sys.argv[1]
password_file_path = sys.argv[2]
startime = time.time()
actual_password = None
with mp.Pool() as pool:
def set_actual_password(password):
nonlocal actual_password
if password:
print('[*] Found password; stopping future tasks')
pool.terminate()
actual_password = password
with open(password_file_path, 'r', errors='ignore') as password_file:
for pswd in password_file:
pswd = pswd.strip('\n')
pool.apply_async(cracker, (zip_file_path, pswd,), callback=set_actual_password)
pool.close()
pool.join()
if actual_password:
print('[*] Cracked password: "{0}"'.format(actual_password))
else:
print('[*] Unable to crack password')
runtime = round(time.time() - startime, 5)
print("[*] Runtime:", runtime, 'seconds')
sys.exit(0)
if __name__ == "__main__":
main()
Here's an implementation of the advice from #phihag's and #Equality 7-2521's answers:
#!/usr/bin/env python3
"""Brute force zip password.
Usage: brute-force-zip-password <zip archive> <passwords>
"""
import sys
from multiprocessing import Pool
from time import monotonic as timer
from zipfile import ZipFile
def init(archive): # run at the start of a worker process
global zfile
zfile = ZipFile(open(archive, 'rb')) # open file in each process once
def check(password):
assert password
try:
with zfile.open(zfile.infolist()[0], pwd=password):
return password # assume success
except Exception as e:
if e.args[0] != 'Bad password for file':
# assume all other errors happen after the password was accepted
raise RuntimeError(password) from e
def main():
if len(sys.argv) != 3:
sys.exit(__doc__) # print usage
start = timer()
# decode passwords using the preferred locale encoding
with open(sys.argv[2], errors='ignore') as file, \
Pool(initializer=init, initargs=[sys.argv[1]]) as pool: # use all CPUs
# check passwords encoded using utf-8
passwords = (line.rstrip('\n').encode('utf-8') for line in file)
passwords = filter(None, passwords) # filter empty passwords
for password in pool.imap_unordered(check, passwords, chunksize=100):
if password is not None: # found
print("Password: '{}'".format(password.decode('utf-8')))
break
else:
sys.exit('Unable to find password')
print('Runtime: %.5f seconds' % (timer() - start,))
if __name__=="__main__":
main()
Note:
each worker process has its own ZipFile object and the zip file is opened once per process: it should make it more portable (Windows support) and improve time performance
the content is not extracted: check(password) tries to open and immediately closes an archive member on success: it is safer and it should improve time performance (no need to create directories, etc)
all errors except 'Bad password for file' while decrypting the archive member are assumed to happen after the password is accepted: the rational is to avoid silencing unexpected errors -- each exception should be considered individually
check(password) expects nonempty passwords
chunksize parameter may drastically improve performance
a rare for/else syntax is used, to report cases when the password is not found
the with-statement calls pool.terminate() for you
Related
I have been told that logging can not be used in Multiprocessing. You have to do the concurrency control in case multiprocessing messes the log.
But I did some test, it seems like there is no problem using logging in multiprocessing
import time
import logging
from multiprocessing import Process, current_process, pool
# setup log
logger = logging.getLogger(__name__)
logging.basicConfig(level=logging.DEBUG,
format='%(asctime)s %(filename)s[line:%(lineno)d] %(levelname)s %(message)s',
datefmt='%a, %d %b %Y %H:%M:%S',
filename='/tmp/test.log',
filemode='w')
def func(the_time, logger):
proc = current_process()
while True:
if time.time() >= the_time:
logger.info('proc name %s id %s' % (proc.name, proc.pid))
return
if __name__ == '__main__':
the_time = time.time() + 5
for x in xrange(1, 10):
proc = Process(target=func, name=x, args=(the_time, logger))
proc.start()
As you can see from the code.
I deliberately let the subprocess write log at the same moment( 5s after start) to increase the chance of conflict. But there are no conflict at all.
So my question is can we use logging in multiprocessing?
Why so many posts say we can not ?
As Matino correctly explained: logging in a multiprocessing setup is not safe, as multiple processes (who do not know anything about the other ones existing) are writing into the same file, potentially intervening with each other.
Now what happens is that every process holds an open file handle and does an "append write" into that file. The question is under what circumstances the append write is "atomic" (that is, cannot be interrupted by e.g. another process writing to the same file and intermingling his output). This problem applies to every programming language, as in the end they'll do a syscall to the kernel. This answer answers under which circumstances a shared log file is ok.
It comes down to checking your pipe buffer size, on linux that is defined in /usr/include/linux/limits.h and is 4096 bytes. For other OSes you find here a good list.
That means: If your log line is less than 4'096 bytes (if on Linux), then the append is safe, if the disk is directly attached (i.e. no network in between). But for more details please check the first link in my answer. To test this you can do logger.info('proc name %s id %s %s' % (proc.name, proc.pid, str(proc.name)*5000)) with different lenghts. With 5000 for instance I got already mixed up log lines in /tmp/test.log.
In this question there are already quite a few solutions to this, so I won't add my own solution here.
Update: Flask and multiprocessing
Web frameworks like flask will be run in multiple workers if hosted by uwsgi or nginx. In that case, multiple processes may write into one log file. Will it have problems?
The error handling in flask is done via stdout/stderr which is then cought by the webserver (uwsgi, nginx, etc.) which needs to take care that logs are written in correct fashion (see e.g. this flask+nginx example), probably also adding process information so you can associate error lines to processes. From flasks doc:
By default as of Flask 0.11, errors are logged to your webserver’s log automatically. Warnings however are not.
So you'd still have this issue of intermingled log files if you use warn and the message exceeds the pipe buffer size.
It is not safe to write to a single file from multiple processes.
According to https://docs.python.org/3/howto/logging-cookbook.html#logging-to-a-single-file-from-multiple-processes
Although logging is thread-safe, and logging to a single file from
multiple threads in a single process is supported, logging to a single
file from multiple processes is not supported, because there is no
standard way to serialize access to a single file across multiple
processes in Python.
One possible solution would be to have each process write to its own file. You can achieve this by writing your own handler that adds process pid to the end of the file:
import logging.handlers
import os
class PIDFileHandler(logging.handlers.WatchedFileHandler):
def __init__(self, filename, mode='a', encoding=None, delay=0):
filename = self._append_pid_to_filename(filename)
super(PIDFileHandler, self).__init__(filename, mode, encoding, delay)
def _append_pid_to_filename(self, filename):
pid = os.getpid()
path, extension = os.path.splitext(filename)
return '{0}-{1}{2}'.format(path, pid, extension)
Then you just need to call addHandler:
logger = logging.getLogger('foo')
fh = PIDFileHandler('bar.log')
logger.addHandler(fh)
Use a queue for correct handling of concurrency simultaneously recovering from errors by feeding everything to the parent process via a pipe.
from logging.handlers import RotatingFileHandler
import multiprocessing, threading, logging, sys, traceback
class MultiProcessingLog(logging.Handler):
def __init__(self, name, mode, maxsize, rotate):
logging.Handler.__init__(self)
self._handler = RotatingFileHandler(name, mode, maxsize, rotate)
self.queue = multiprocessing.Queue(-1)
t = threading.Thread(target=self.receive)
t.daemon = True
t.start()
def setFormatter(self, fmt):
logging.Handler.setFormatter(self, fmt)
self._handler.setFormatter(fmt)
def receive(self):
while True:
try:
record = self.queue.get()
self._handler.emit(record)
except (KeyboardInterrupt, SystemExit):
raise
except EOFError:
break
except:
traceback.print_exc(file=sys.stderr)
def send(self, s):
self.queue.put_nowait(s)
def _format_record(self, record):
# ensure that exc_info and args
# have been stringified. Removes any chance of
# unpickleable things inside and possibly reduces
# message size sent over the pipe
if record.args:
record.msg = record.msg % record.args
record.args = None
if record.exc_info:
dummy = self.format(record)
record.exc_info = None
return record
def emit(self, record):
try:
s = self._format_record(record)
self.send(s)
except (KeyboardInterrupt, SystemExit):
raise
except:
self.handleError(record)
def close(self):
self._handler.close()
logging.Handler.close(self)
The handler does all the file writing from the parent process and uses just one thread to receive messages passed from child processes
QueueHandler is native in Python 3.2+, and safely handles multiprocessing logging.
Python docs have two complete examples: Logging to a single file from multiple processes
For those using Python < 3.2, just copy QueueHandler into your own code from: https://gist.github.com/vsajip/591589 or alternatively import logutils.
Each process (including the parent process) puts its logging on the Queue, and then a listener thread or process (one example is provided for each) picks those up and writes them all to a file - no risk of corruption or garbling.
Note: this question is basically a duplicate of How should I log while using multiprocessing in Python? so I've copied my answer from that question as I'm pretty sure it's currently the best solution.
I've been having an odd issue with PyCharm and subprocesses created by the multiprocessing library locking up forever. I'm using Windows with Python 3.5. What I'm trying to do is:
Start a background thread to block on stdin (waiting for input)
Have the main thread check occasionally for input from stdin and then delegate the work to Python processes created using multiprocessing
However, I've found that newly created multiprocessing Processes lock up forever if and only if the following conditions are met:
I'm running the code via Pycharm (both the latest and older versions)
The background thread is blocking on stdin
Here's the simplest example I can create that reproduces the problem:
import multiprocessing
import threading
import sys
def noop():
pass
def consume():
while True:
sys.stdin.readline()
if __name__ == '__main__':
# create a daemon thread to block on stdin
thread = threading.Thread(target=consume, daemon=True)
thread.start()
# create a background process
process = multiprocessing.Process(target=noop)
process.start()
I've Googled various combinations of "PyCharm stdin multiprocessing hang ..." and had no luck at finding an explanation, and I can't figure out why a thread of the main process blocking on stdin should ever cause a subprocess to also block/hang, let alone why it would only happen when running the script in PyCharm. The only think I can guess is that there might be some monkey-patching of either stdin or the multiprocessing library going on.
Has anyone else encountered this problem? Can anyone explain to me why this only occurs in PyCharm, and how I can make it work regardless of the Python editor I'm using?
I faced the same problem when I was trying to do multiple API calls to fetch data from a remote server. I replaced multiprocessing dummy with ThreadPoolExecutor. It works in the same way as dummy.
Following is a short snippet of a running code to write the response to a json file:
uids = [] # an array of the requisite parameters used in requests
with open('flight_config.json', 'w') as f:
futures = []
for i in range(chunk_index, len(uids)):
print('For uid[{}], fetching started:'.format(i))
chunk_index += 1
auth_token = get_header()
with ThreadPoolExecutor(max_workers=7) as executor:
future_to_url = {executor.submit(fetch_response_from_api, uid=uid, auth_token=auth_token): uid for uid in
uids[i]}
for future in concurrent.futures.as_completed(future_to_url):
result = future_to_url[future]
try:
data = future.result()
print(data)
except Exception as exc:
print('%r generated an exception: %s' % (result, exc))
else:
print('%r page is %d bytes' % (result, len(data)))
I wish to have a process continually monitoring RPi input, and set a variable (I have chosen a queue) to True or False to reflect the debounced value. Another process will then capture an image (from a stream). I have written some code just to check I can get multiprocessing and signalling (the queue) working ok (I'm an amature coder...).
It all works fine with threading, but multiprocessing is giving an odd error. Specifically 'multiprocessing, EOFError: EOF when reading a line'. Code outputs:-
this computer has the following number of CPU's 6
OK, started thread on separate processor, now we monitor variable
enter something, True is the key word:
Process Process-1:
Traceback (most recent call last):
File "c:\Python34\lib\multiprocessing\process.py", line 254, in _bootstrap
self.run()
File "c:\Python34\lib\multiprocessing\process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\Peter\Documents\NetBeansProjects\test_area\src\test4.py", line 16, in Wait4InputIsTrue
ValueIs = input("enter something, True is the key word: ")
EOFError: EOF when reading a line
This module monitors the 'port' (I am using the keyboard as an input):
#test4.py
from time import sleep
from multiprocessing import Lock
def Wait4InputIsTrue(TheVar, TheLock):
while True:
sleep(0.2)
TheLock.acquire()
#try:
ValueIs = input("enter something, True is the key word: ")
#except:
# ValueIs = False
if ValueIs == "True":
TheVar.put(True)
print("changed TheVar to True")
TheLock.release()
This module monitors the status, and acts on it:
#test5.py
if __name__ == "__main__":
from multiprocessing import Process, Queue, Lock, cpu_count
from time import sleep
from test4 import Wait4InputIsTrue
print("this computer has the following number of CPU's", cpu_count())
LockIt = Lock()
IsItTrue = Queue(maxsize = 3)
Wait4 = Process(target = Wait4InputIsTrue, args = (IsItTrue, LockIt))
Wait4.start()
print("OK, started thread on separate processor, now we monitor variable")
while True:
if IsItTrue.qsize():
sleep(0.1)
print("received input from separate thread:", IsItTrue.get())
Note that I have tried adding a try: to the input statement in test4.py, in which case it keeps printing "enter something, True is the key word: " indefinitely, without a cr.
I added Lock in wild attempt to fix it, makes no difference
Anyone any idea why this is happening?
Your problem can be boiled down to a simpler script:
import multiprocessing as mp
import sys
def worker():
print("Got", repr(sys.stdin.read(1)))
if __name__ == "__main__":
process = mp.Process(target=worker)
process.start()
process.join()
When run, it produces
$ python3 i.py
Got ''
Reading zero bytes means the pipe is closed and input(..) turns that into an EOFError exception.
The multiprocessing module doesn't let you read stdin. That makes sense generally because mixing stdin readers from multiple children is a risky business. In fact, digging into the implementation, multiprocessing/process.py explicitly sets stdin to devnull:
sys.stdin.close()
sys.stdin = open(os.devnull)
If you are just using stdin for test, then the solution is simple: Don't do that! If you really need user input, life is quite a bit more difficult. You can use additional queues plus code in the parent to prompt users and get input.
I am learning how to crack zip files using dictionary attacks. This is the code:
import zipfile
from threading import Thread
def extractFile(zFile, password):
try:
zFile.extractall(pwd=password)
print '[+] Found password ' + password + '\n'
except:
pass
def main():
zFile = zipfile.ZipFile('evil.zip')
passFile = open('dictionary.txt')
for line in passFile.readlines():
password = line.strip('\n')
extractFile(zFile, password)
if __name__ == '__main__':
main()
I use threading on it
import zipfile
from threading import Thread
def extractFile(zFile, password):
try:
zFile.extractall(pwd=password)
print '[+] Found password ' + password + '\n'
except:
pass
def main():
zFile = zipfile.ZipFile('evil.zip')
passFile = open('dictionary.txt')
for line in passFile.readlines():
password = line.strip('\n')
t = Thread(target=extractFile, args=(zFile, password))
t.start()
if __name__ == '__main__':
main()
However, when I time the two programmes, it takes 90 seconds to complete the first but nearly 300 seconds to complete the second. The dictionary contains 459026 entries. I am baffled as to why this happens. I also tried limiting the threads to 10,20,so on. But still the loop performs faster at each instance. Can anybody explain why this is so?? Also is there any chance to improve the program at all.
EDIT
I tried slicing as suggested by Ray as follows:
import zipfile
from threading import Thread
def chunks(l, n):
"""Yield successive n-sized chunks from l."""
for i in xrange(0, len(l), n):
yield l[i:i+n]
def extractFile(zFile, passwords):
for password in passwords:
try:
zFile.extractall(pwd=password)
print '[+] Found password ' + password + '\n'
sys.exit(0)
except:
continue
def main():
zFile = zipfile.ZipFile('evil.zip')
with open('dictionary.txt', 'rb') as pass_file:
passwords = [i.strip() for i in pass_file]
passes = list(chunks(passwords, 10))
for pas in passes:
t = Thread(target=extractFile, args=(zFile, pas))
t.start()
if __name__ == '__main__':
main()
Still takes 3-4 mins
One reason why this does not work properly for multiprocessing; you must open the zip file in each subprocess, otherwise you can be hurt by sharing filehandles. Then create only a handful (say 2 * number of cores) subprocesses, and let a single subprocess test multiple passwords.
Thus we get:
import zipfile
from multiprocessing import Process
def extract_file(passwords):
with zipfile.ZipFile('evil.zip') as zipf:
for password in passwords:
try:
zipf.extractall(pwd=password)
print('[+] Found password {}\n'.format(password))
except Exception as e:
pass
def main():
with open('dictionary.txt', 'rb') as pass_file:
passwords = [i.strip() for i in pass_file]
N_PROC = 8
for i in range(N_PROC):
p = Process(target=extract_file, args=[passwords[i::N_PROC]])
p.start()
if __name__ == '__main__':
main()
Can anybody explain why this is so??
I think that, in addition to the problem of the Global Interpreter Lock (GIL), you might be using the threads incorrectly.
Judging from the loop, you're starting a completely new thread for every password line in your file -i.e. just to make a single attempt. Starting a new thread for only a single attempt is, as you've discovered, expensive and not working out as you expected. If you do this using multiprocessing, then it'll be even slower because creating a completely new process just to for a single try is even more expensive than creating a thread.
Is there any chance to improve the program at all?
I suggest you:
break up the passwords into several sub-lists/groups (i.e. slicing)
create a thread (or process) for each of these sub-lists
let each thread/process consume a group (i.e. make multiple attempts and get more out of them)
For example, if you have 100 lines in the file, you could break it up into 4 parts (i.e. 25 passwords per sub-list) and use these to feed 4 threads/processes (i.e. one for each sub-list).
Using multiprocessing here would be advantageous because you can avoid the GIL. However, keep in mind that you'd still have multiple processes accessing the same file simultaneously, so make sure you account for this when trying to extract the file, etc.
You should take care not to overwhelm your PC cores. You might want to use a process pool (see python docs) and limit the amount of processes you create to the number of cores in your PC as a maximum (perhaps your_core_count - 1 to keep it responsive).
Then, as each process consumes a sub-list and terminates, a new process is created (or existing one re-assigned, if using a process pool) to handle yet another sub-list waiting in your queue. If one of the children completes successfully, then you might want to get the parent process to kill all the other children to avoid unnecessary resource usage.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Python: single instance of program
I need to prevent a cron job from running concurrent instances when a job takes longer to complete than the launcher interval. I'm trying to use the flock concept to achieve this, but fcntl module is not behaving the way I expect.
Can anyone tell me why this works to prevent two concurrent instances:
import sys
import time
import fcntl
file_path = '/var/lock/test.py'
file_handle = open(file_path, 'w')
try:
fcntl.lockf(file_handle, fcntl.LOCK_EX | fcntl.LOCK_NB)
print 'no other instance is running'
for i in range(5):
time.sleep(1)
print i + 1
except IOError:
print 'another instance is running exiting now'
sys.exit(0)
And why this does not work:
import sys
import time
import fcntl
def file_is_locked(file_path):
file_handle = open(file_path, 'w')
try:
fcntl.lockf(file_handle, fcntl.LOCK_EX | fcntl.LOCK_NB)
return False
except IOError:
return True
file_path = '/var/lock/test.py'
if file_is_locked(file_path):
print 'another instance is running exiting now'
sys.exit(0)
else:
print 'no other instance is running'
for i in range(5):
time.sleep(1)
print i + 1
My humble opinion (although I may be totally wrong) is that file_handle is local to the function (in the second case) and therefore, it gets destroyed once the function is done.
The following code seems to work as expected:
#!/usr/bin/env python
#http://stackoverflow.com/questions/14406562/prevent-running-concurrent-instances-of-a-python-script
import sys
import time
import fcntl
file_handle = None
def file_is_locked(file_path):
global file_handle
file_handle= open(file_path, 'w')
try:
fcntl.lockf(file_handle, fcntl.LOCK_EX | fcntl.LOCK_NB)
return False
except IOError:
return True
file_path = '/var/lock/test.py'
if file_is_locked(file_path):
print 'another instance is running exiting now'
sys.exit(0)
else:
print 'no other instance is running'
for i in range(5):
time.sleep(1)
print i + 1
Notice that the only thing I did is setting file_handle as global variable (although I copied the whole code to have a working example)
As I mentioned in my commen on #BorrajaX's answer, since it looks like you are POSIX-bound anyway, you could try using a native named semaphore.
You could use the setlock program from D. J. Bernstein's daemontools instead:
http://cr.yp.to/daemontools/setlock.html
Easiest way would be to create a file /tmp/scriptlock at the start of the script and check if that file exists before doing work. Make sure the lock file is removed though at the end of processing.