I am trying to start a python script everytime is a certain file modified. To be accurate, I have a device on Raspberry Pi's serial ports which writes data into a text file (test.txt) . I have tried both tools - Watchdog / Pyinotify. Everytime the file is modified (triggers event Watchdog: on_modified / Pyinotify: IN_MODIFY), it makes duplicate trigger. I have tried every other method, even IN_CLOSE_WRITE as some people suggest, but this doesn't work at all.
Does someone know, how can just a single event be triggered on one file update?
My code using Pyinotify (a bit edited tutorial file):
import pyinotify,subprocess
def onChange(ev):
cmd = ['/usr/bin/env', 'python', 'doThisFile.py', ev.pathname]
subprocess.Popen(cmd).communicate()
wm = pyinotify.WatchManager()
wm.add_watch('/home/pi/test.txt', pyinotify.IN_MODIFY, onChange)
notifier = pyinotify.Notifier(wm)
notifier.loop()
or Watchdog:
#!/usr/bin/python
import time
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
import subprocess
class MyHandler(FileSystemEventHandler):
def on_modified(self, event):
subprocess.call("/home/pi/doThisFile.py")
print("Code started")
if __name__ == "__main__":
event_handler = MyHandler()
observer = Observer()
observer.schedule(event_handler, path='.', recursive=False)
observer.start()
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
observer.stop()
observer.join()
I was facing the same issue while using pyinotify. But changing IN_MODIFY to IN_CLOSE_WRITE solved the issue. You can get more info from this answer
This probably because of the text editor which was used for editing your source code.
solution: calculate the time between 2 events, for example, If the events trigger too frequently, exit the handler from one of the events.
class SubEventHandler(FileSystemEventHandler):
def __init__(self):
self._time0 = self.getTime()
def on_modified(self, event):
time1 = self.getTime()
if (time1 - self._time0) < 5:
exit(0) # not sure it will exit properly
The 2 events that you are getting are:
The file was modified
The directory was modified
If you run the demo code and then in a touch a file in the directory being watched, you'll see the following output:
2022-11-04 10:28:45 - Modified file: ./test
2022-11-04 10:28:45 - Modified directory: .
Related
The python code below is my first attempt for notification when a file is modified, however nothing happens when the file is changed.
What have I missed here?
#!/usr/bin/python3
import time
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
class MyHandler(FileSystemEventHandler):
def on_modified(self, event):
print('file changed')
if __name__ == "__main__":
event_handler = MyHandler()
observer = Observer()
observer.schedule(event_handler, path='/Users/jeff/smb/storage/wsjt-x/wsjtx_log.adi', recursive=False)
observer.start()
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
observer.stop()
observer.join()
NOTE: I found that this code works well using Linux. It's on macOS Big Sur where I'm having issues.
After clarifying your environment in comments, one of following 3 approaches might resolve your issue.
Reproduction: Watching a local file on Linux
Running in one shell A and meanwhile using another shell B to add to the observed file so.md like:
echo "---" >> so.md
results in following output:
python3 so_watchdog.py so.md
file changed <FileModifiedEvent: event_type=modified, src_path='so.md', is_directory=False>
file changed <DirModifiedEvent: event_type=modified, src_path='', is_directory=True>
Here is the slightly modified version of your script (see comments):
#!/usr/bin/python3
import time
import sys
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
class MyHandler(FileSystemEventHandler):
def on_modified(self, event):
print('file changed', event) # added event to see the difference of two events for a single modification
if __name__ == "__main__":
if len(sys.argv) < 2:
file = '/Users/jeff/smb/storage/wsjt-x/wsjtx_log.adi'
else:
file = sys.argv[1]
# below only added the file as path
event_handler = MyHandler()
observer = Observer()
observer.schedule(event_handler, path=file, recursive=False)
observer.start()
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
observer.stop()
observer.join()
Note: This was tested on Linux, with a local file.
Watching shared files (CIFS/SMB)
Since your path /Users/jeff/smb/storage/wsjt-x/wsjtx_log.adi contains smb I assume, you are watching a file on a mounted shared network location using the SMB (Server Message Blocks) protocol, also known as CIFS (Common Internet File System).
Then please follow the official documentation, About using watchdog with CIFS:
When you want to watch changes in CIFS, you need to explicitly tell watchdog to use PollingObserver, that is, instead of letting watchdog decide an appropriate observer like in the example above, do:
from watchdog.observers.polling import PollingObserver as Observe
So your import statement needs to be modified.
Watching files locally on MacOS
For MacOS the underlying file-system event API used is FSEvents or FreeBSD's kqueue, instead of Linux's inotify.
Thus the behavior might also deviate.
I don't want to enable linux onedrive service as I've noticed it can be hard on my cpu, so I would rather run it on demand. That's why I'm trying to create a little script that would run onedrive sync one time only when I access OneDrive directory. Makes sense? I hope so.
I thought it would be easy but watchdog doesn't seem to have a way to monitor for that kind of event. Is there any other way I could do this ? Am I missing something ?
Here is watchdog documentation:
https://python-watchdog.readthedocs.io/en/v0.10.2/quickstart.html#a-simple-example
import sys
import logging
from watchdog.observers import Observer
from watchdog.events import LoggingEventHandler
if __name__ == "__main__":
logging.basicConfig(level=logging.INFO,
format='%(asctime)s - %(message)s',
datefmt='%Y-%m-%d %H:%M:%S')
path = sys.argv[1] if len(sys.argv) > 1 else '.'
event_handler = LoggingEventHandler()
observer = Observer()
observer.schedule(event_handler, path, recursive=True)
observer.start()
try:
while observer.isAlive():
observer.join(1)
except KeyboardInterrupt:
observer.stop()
observer.join()
I'm running linux mint 19 in case it matters.
A good utility for executing actions when some files change is entr.
For example if you'd like to run some script upload_to_onedrive.sh every time a file changes in the directory onedrive you'd just ls ~/onedrive | entr ./upload_to_onedrive.sh
I am using watchdog in Python to monitor realtime whenever a file is created or deleted.
Following examples, I tried with the following:
import time
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
class MyHandler(FileSystemEventHandler):
def on_created(self, event):
print("File is created!")
event_handler = MyHandler()
observer = Observer()
observer.schedule(event_handler, path='C:/daten/dog.txt', recursive=False)
observer.start()
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
observer.stop()
observer.join()
Of course, the file path='C:/daten/dog.txt' does not exist when this script begins to run. But I still get error messages as
FileNotFoundError: [WinError 2] The system cannot find the file specified.
Why it's telling me it cannot find the file specified at the first place. I need it running to watch for the creation of the file after all.
Update:
Now I understand that watchdog is for monitoring a folder rather than a file.
Is there a similar package for monitoring a file or is it just better done by while and sleep statements together?
Why it's telling me it cannot find the file specified at the first place.
Because the path you give to watchdog is where it's going to hook itself to listen for events.
You can't watch a file and expect its creation event to be recorded. File creation events are posted on the parent directory, so that is what you should be watching.
In fact I don't know that watching a specific file makes any sense with watchdog, its tagline is
Directory monitoring made easy with
I try to monitor text log file for changes and want to see lines that was added
I try with watchdog it seems to work with some manually created/edited files for testing, but my log file (in the same directory) isn't detected as "changed", but it was changed. It's probably because this file is already open (maybe in a specific mode). If I close the app that uses this log file and change it manually then watchdog works ok.
How do I check if the file was changed and if the changes were written to
the console?
#!/usr/bin/python
import time
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
class MyHandler(FileSystemEventHandler):
def on_modified(self, event):
print(f'event type: {event.event_type} path : {event.src_path}')
if __name__ == "__main__":
event_handler = MyHandler()
observer = Observer()
observer.schedule(event_handler, path='C:\gory\parcienaszklo\logs', recursive=False)
observer.start()
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
observer.stop()
observer.join()
I am trying to learn python-watchdog, but I am sort of confused why the job I set up runs more than once. So, here is my set up:
#handler.py
import os
from watchdog.events import FileSystemEventHandler
from actions import run_something
def getext(filename):
return os.path.splitext(filename)[-1].lower()
class ChangeHandler(FileSystemEventHandler):
def on_any_event(self, event):
if event.is_directory:
return
if getext(event.src_path) == '.done':
run_something()
else:
print "event not directory.. exiting..."
pass
the observer is set up like so:
#observer.py
import os
import time
from watchdog.observers import Observer
from handler import ChangeHandler
BASEDIR = "/path/to/some/directory/bin"
def main():
while 1:
event_handler = ChangeHandler()
observer = Observer()
observer.schedule(event_handler, BASEDIR, recursive=True)
observer.start()
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
observer.stop()
observer.join()
if __name__ == '__main__':
main()
and finally, the actions like so:
#actions.py
import os
import subprocess
def run_something():
output = subprocess.check_output(['./run.sh'])
print output
return None
..where ./run.sh is just a shell script I would like to run when a file with an extension .done is found on /path/to/some/directory/bin
#run.sh
#!/bin/bash
echo "Job Start: $(date)"
rm -rf /path/to/some/directory/bin/job.done # remove the .done file
echo "Job Done: $(date)"
However, when I issue a python observer.py and then do a touch job.done on /path/to/some/directory/bin, I see that my shell script ./run.sh runs three times and not one..
I am confused why this runs thrice and not just once (I do delete the job.done file on my bash script)
To debug watchdog scripts, it is useful to print what watchdog is seeing as events. One file edit or CLI command, such as touch, can result in multiple watchdog events. For example, if you insert a print statement:
class ChangeHandler(FileSystemEventHandler):
def on_any_event(self, event):
print(event)
to log every event, running
% touch job.done
generates
2014-12-24 13:11:02 - <FileCreatedEvent: src_path='/home/unutbu/tmp/job.done'>
2014-12-24 13:11:02 - <DirModifiedEvent: src_path='/home/unutbu/tmp'>
2014-12-24 13:11:02 - <FileModifiedEvent: src_path='/home/unutbu/tmp/job.done'>
Above there were two events with src_path ending in job.done. Thus,
if getext(event.src_path) == '.done':
run_something()
runs twice because there is a FileCreatedEvent and a FileModifiedEvent.
You might be better off only monitoring FileModifiedEvents.