I'm quite new to Python and hope the answer to this is obvious to most of you.
I'm creating a class in Python that represents a ScanFolder.
In the __init__ of that class, I start a watchdog.observers
This observer will fire a watchdog.events.PatternMatchingEventHandler whenever a file is changed under the watched directory.
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import sys
import time
from watchdog.observers import Observer
import watchdog.events
path = sys.argv[1] if len(sys.argv) > 1 else '.'
class MyEventHandler(watchdog.events.PatternMatchingEventHandler):
def on_any_event(self, event):
print(event.src_path, event.event_type)
class ScanFolder:
'Class defining a scan folder'
def __init__(self, path):
self.path = path
self.documents = dict() # key = document label value = Document reference
self.event_handler = MyEventHandler(patterns=["*.jpg", "*.jpeg", "*.png", "*.bmp", "*.pdf"],
ignore_patterns=[],
ignore_directories=True)
self.observer = Observer()
self.observer.schedule(self.event_handler, self.path, recursive=False)
self.observer.start()
def stop(self):
self.observer.stop()
self.observer.join()
scan_folder = ScanFolder(path)
try:
while True:
time.sleep(1)
"""Here, I'll act on my scan_folder object that lists the discovered files"""
except KeyboardInterrupt:
log.warning("Ouch !!! Keyboard interrupt received.")
scan_folder.stop()
My problem is the following:
How can I have my scan_folder object modified by my scan_folder.event_handler() ?
Actually, I would like to populate the scan_folder.documents dictionary wherever a file is detected in the scan folder.
Thank you very much and sorry for my ignorance.
There are a lot of way to do it: but the simplest way is set a bound method of ScanFolder as on_any_event callback function of watchdog.events.PatternMatchingEventHandler. So your code become:
class ScanFolder:
'Class defining a scan folder'
def __init__(self, path):
self.path = path
self.documents = dict() # key = document label value = Document reference
self.event_handler = watchdog.events.PatternMatchingEventHandler(patterns=["*.jpg", "*.jpeg", "*.png", "*.bmp", "*.pdf"],
ignore_patterns=[],
ignore_directories=True)
self.event_handler.on_any_event = self.on_any_event
self.observer = Observer()
self.observer.schedule(self.event_handler, self.path, recursive=False)
self.observer.start()
def on_any_event(self, event):
print(event.src_path, event.event_type)
print("Complete ScanFolder() access")
def stop(self):
self.observer.stop()
self.observer.join()
Other way could be derive ScanFolder from watchdog.events.PatternMatchingEventHandler .... But injecting function is one of the power of python
Related
I have a script that watches for changes in file and if occurred it should trigger some actions. Those actions come from two functions defined outside Class. In Class I have defined portion of code to look for changes in file. I cannot figure out how to pass two functions in Class arguments. Here is simplified portion of my script:
import time, os
watch_file = 'my_file.txt'
def first_action():
print('First action called')
def second_action():
print('Second action called')
class Watcher():
def __init__(self, watch_file, first_action=None, second_action=None):
self._cached_stamp = 0
self.filename = watch_file
self.first_action = first_action
self.second_action = second_action
# Look for changes in 'my_file.txt'
def look(self):
stamp = os.stat(self.filename).st_mtime
if stamp != self._cached_stamp:
self._cached_stamp = stamp
# File has changed, so do something...
print('File changed')
if self.first_action is not None:
print('Call first action')
self.first_action(self)
if self.second_action is not None:
print('Call second action')
self.second_action(self)
watcher = Watcher(watch_file, first_action(), second_action())
Doing it like above calls first_action() and second_action() but not inside Class. I know it because I dont see printed 'Call first action' or 'Call second action'
I was able to correctly trigger first action with below code:
watch_file = 'my_file.txt'
def first_action():
print('First action called')
def second_action():
print('Second action called')
class Watcher():
def __init__(self, watch_file, first_action=None, *args):
self._cached_stamp = 0
self.filename = watch_file
self.first_action = first_action
self.args = args
# Look for changes in 'my_file.txt'
def look(self):
stamp = os.stat(self.filename).st_mtime
if stamp != self._cached_stamp:
self._cached_stamp = stamp
# File has changed, so do something...
print('File changed')
if self.first_action is not None:
print('Call first action')
self.first_action(*self.args)
watcher = Watcher(watch_file, first_action)
For some reason I need to specify *args even for function which does not take any argument on call. Can someone explain why *args must be used?
You did it right in the init, but not in the call of "Watcher". To pass the function itself, instead of it's return value, you have to remove the braces.
watcher = Watcher(watch_file, first_action, second_action)
And you should be fine.
I have written one python script to monitor one local folder which is having only .txt files and i want to start this script automatically if some changes happened to the folder(created,deleted or updated)
I tried to run this script and also tried to make changes in the directory, but i couldn't see any output and no error messages. It always says "Process finished with exit code 0" can any one review my code and give me some tips where to correct to get the expected out put.
import os
import sys
import time
from watchdog.observers import Observer
from watchdog.events import PatternMatchingEventHandler
#Step 1 Create the event handler
if __name__ == "__main__":
patterns = ".txt"
ignore_patterns = None
ignore_directories = False
case_sensitive = True
event_handler = PatternMatchingEventHandler(patterns, ignore_patterns, ignore_directories, case_sensitive)
#step 2 Handle all the events
def on_created(event):
print("new files has been created!")
def on_deleted(event):
print("Some files has been Deleted")
def on_modified(event):
print("Some files has been modified")
def on_moved(event):
print("Some files has been moved")
#step 3 specify to the handler that we want these functions to be called
event_handler.on_created = on_created
event_handler.on_deleted = on_deleted
event_handler.on_modified = on_modified
event_handler.on_moved = on_moved
#step 4 create an observer
path = "T:\Laboratory\Instruments\Worklists\TrackMateRacks\old"
go_recursively = True
my_observer = Observer()
my_observer.path(event_handler, path, recursive=go_recursively)
# start the observer
my_observer.start()
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
my_observer.stop()
my_observer.join()
you have to move the lines after " # start the observer" to the very left.
otherwise there will be executed nothing. that lines are part of on_moved(). but you want them to be executed if you start the script.
or
for most programs it's useful to add this line:
if __name__ == '__main__':
bevore line "# start the observer"
than your my_observer.start() will be executed, if you call your script. but if you import your script in another script, this will not be executed, but the other script can use all the functions, you created.
It seems, that you're really new to Python. You've to watch the indents, they're part of the syntax.
Exceptionally I reformat the complete code for you:
import os
import sys
import time
from watchdog.observers import Observer
from watchdog.events import PatternMatchingEventHandler
#Step 1 Create the event handler
patterns = ".txt"
ignore_patterns = None
ignore_directories = False
case_sensitive = True
event_handler = PatternMatchingEventHandler(patterns, ignore_patterns, ignore_directories, case_sensitive)
#step 2 Handle all the events
def on_created(event):
print("new files has been created!")
def on_deleted(event):
print("Some files has been Deleted")
def on_modified(event):
print("Some files has been modified")
def on_moved(event):
print("Some files has been moved")
#step 3 specify to the handler that we want these functions to be called
event_handler.on_created = on_created
event_handler.on_deleted = on_deleted
event_handler.on_modified = on_modified
event_handler.on_moved = on_moved
#step 4 create an observer
def main():
path = "T:\Laboratory\Instruments\Worklists\TrackMateRacks\old"
go_recursively = True
my_observer = Observer()
my_observer.path(event_handler, path, recursive=go_recursively)
# start the observer
my_observer.start()
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
my_observer.stop()
my_observer.join()
if __name__ == "__main__":
main()
#end of file
Good luck!
I want to watch a folder for addition, modification and deletion of file and execute a command whenever any of this event occurs.
I found this tutorial that helped https://www.michaelcho.me/article/using-pythons-watchdog-to-monitor-changes-to-a-directory
so here is the code I now have
import time
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
class Watcher:
DIRECTORY_TO_WATCH = "/Users/***/desktop/google drive/protpics"
def __init__(self):
self.observer = Observer()
def run(self):
event_handler = Handler()
self.observer.schedule(event_handler, self.DIRECTORY_TO_WATCH, recursive=True)
self.observer.start()
try:
while True:
time.sleep(5)
except:
self.observer.stop()
print("Error")
self.observer.join()
class Handler(FileSystemEventHandler):
#staticmethod
def on_my_event(event):
if event.is_directory:
return None
elif event.event_type == 'created':
#Take any action here when a file is first created.
print ("Recived created event - %s" % event.src_path)
elif event.event_type == 'modified':
# Take any action here when a file is modified.
print ("Recieved modified event - %s" % event.src_path)
if __name__ == '__main__':
W = Watcher()
W.run()
the problem now is that when I added a new file to the directory no message gets printed out. What am I doing wrong and how can I fix it?
Couldn't you figure out the difference between your code and example's? In your link, the author use on_any_event, but you are using on_my_event. There isn't a method named on_my_event.
Have a check at official document: http://pythonhosted.org/watchdog/api.html#watchdog.events.FileSystemEventHandler
I am trying to listen to filesystem changes using the watchdog module of Python. I want to monitor the files from two different directories. For a single file watch, I used PatternMatchingEventHandler from watchdog.events. I want to use the same for multiple directories.
code:
import time
from watchdog.observers import Observer
from watchdog.events import PatternMatchingEventHandler
class EventHandler(PatternMatchingEventHandler):
def on_modified(self, event):
super(EventHandler, self).on_modified(event)
print event
if __name__ == "__main__":
dir_name = ["/home/user1/first", "/home/user1/second"]
observer = Observer()
patterns = ["/home/user1/first/first.log","/home/user1/second/second.log")]
for i in xrange(len(dir_name)):
event_handler = EventHandler(patterns = patterns[i])
observer.schedule(event_handler, dir_name[i], recursive=True)
observer.start()
try:
while True:
time.sleep(0.1)
except KeyboardInterrupt:
observer.stop()
In the above code, I tried to do multiple directory observing format and create an event handler for each of the files. It's not working for me. Is there anything that I am missing here?? What is the way to do this??
Thanks.
some wrong in here
EventHandler(patterns = patterns[i])
arg patterns is a type of list, so you can use like this
patterns = [["/home/user1/first/first.log"], ["/home/user1/second/second.log"]]
EventHandler(patterns = patterns[i])
Though it does not use the watchdog library , this will the easy way just to check if the specific type of files are added or removed
if u want to check which files u can append them using any variable and store them in array
import os
import fnmatch
import threading
import time
initial_count = 0
flag = 0
files = []
path = ["/home/kirti/workspace/pythonproject6/img", "/home/kirti/workspace/pythonproject6/copy"]
def taskcount(path, flag, initial_count):
while 1:
time.sleep(3)
new_count = len(fnmatch.filter(os.listdir(path), "*.jpg"))
if new_count > initial_count:
if flag != 0:
print("Added \nCount :", new_count, "=", path)
else:
print(new_count)
if new_count < initial_count:
print("Removed \nCount :", new_count, "=", path)
initial_count = new_count
flag = 1
for j in range(len(path)):
t = threading.Thread(target=taskcount, args=(path[j], flag, initial_count))
t.start()
I am using python3, LINUX OS
With a minor modification as suggested above and some additions from myside too this is working now
import time
from watchdog.observers import Observer
from watchdog.events import PatternMatchingEventHandler
class EventHandler(PatternMatchingEventHandler):
def on_modified(self, event):
super(EventHandler, self).on_modified(event)
print(event)
if __name__ == "__main__":
dir_name = ["/home/don/test1", "/home/don/test2"]
observer = Observer()
threads=[]
patterns = [['*.log'],['*.ok']]
for i in range(len(dir_name)):
event_handler = EventHandler(patterns = patterns[i],ignore_directories=True,case_sensitive=False)
observer.schedule(event_handler, dir_name[i], recursive=True)
threads.append(observer)
observer.start()
try:
while True:
time.sleep(0.1)
except KeyboardInterrupt:
observer.stop()
observer.join()
Comparing with mine, you are lacking of observer.join() at the EOF. Try with that.
EDIT
Try this code below:
import time
from watchdog.observers import Observer
from watchdog.events import PatternMatchingEventHandler
class EventHandler(PatternMatchingEventHandler):
def on_modified(self, event):
super(EventHandler, self).on_modified(event)
print event
if __name__ == "__main__":
observer = Observer()
patterns = ["/home/user1/first/first.log","/home/user1/second/second.log"]
for pattern in patterns:
event_handler = EventHandler(patterns=pattern)
observer.schedule(event_handler, dir_name[i], recursive=True)
observer.start()
try:
while True:
time.sleep(0.1)
except KeyboardInterrupt:
observer.stop()
observer.join()
I need to run as many threads of class Observer as there are elements in list dirlist.
When I run it python console it works all right.
class Observer(Thread):
def run(self):
naptime = random.randint(1,10)
print(self.name + ' starting, running for %ss.' % naptime)
time.sleep(naptime)
print(self.name + ' done')
observers = {}
for d in dirlist:
observers[d] = Observer()
observers[d].start()
But when I try to do it from a Master thread which is supposed to spawn the Observer threads I get errors.
class Master(Thread):
def __init__(self, dirlist):
self.dirlist = dirlist
def run(self):
observers = {}
for d in dirlist:
observers[d] = Observer()
observers[d].start()
while True:
time.sleep(3600)
master_thread = Master(dirlist)
master_thread.start()
The call to Master.start results in:
RuntimeError: thread.__init__() not called
This looks strange to me.
I am unable to understand whats the difference between both cases.
Can anybody figure out a solution to my problem ?
Somehow following doesn't produce an error, and I don't understand why.
class Master(Thread):
def set(self, dirlist):
self.dirlist = dirlist
def run(self):
observers = {}
for d in dirlist:
observers[d] = Observer()
observers[d].start()
while True:
time.sleep(3600)
master_thread = Master()
master_thread.set(dirlist)
master_thread.start()
>>> master_thread.start()
RuntimeError: thread.__init__() not called
Make sure to call Thread.__init__() in your Master.__init__:
class Master(Thread):
def __init__(self, dirlist):
super(Master, self).__init__()
self.dirlist = dirlist
Well i know it's late to answer but, what the hell, i am a newbie in python but this same thing was happening to me, so i went back to read the python tutorial and it has some differences with what we both we're trying, hope it helps.
instead of this
import threading
class Master(Thread):
def set(self, dirlist):
self.dirlist = dirlist
def run(self):
observers = {}
for d in dirlist:
...
class according to the python tutorial:
class Master(threading.Thread):
this line is missing:
threading.Thread.__init__(self)
so it will end up being:
import threading
class Master(threading.Thread):
def __init__(self, dirlist):
threading.Thread.__init__(self)
self.dirlist = dirlist
def run(self):
observers = {}
for d in dirlist:
...
and that should work, at least work for me.
I hope it was helpfull.
And your second try using the set method works, because you are not overriding the
__init__ method
from Thread therefore the original init method from the parent class is used it runs as it's supposed.
Error is clear, you should call thread.__init__():
def __init__(self, dirlist):
super(Master, self).__init__()
self.dirlist = dirlist