I have a file that is updated throughout the day and the name of that file changes with a time signature of when the file was updated.
e.g.
\folder\todolist\grocerylist_2015-12-30-093000.csv
How can I open this file in python without manually updating the full file path?
You can use the glob module to handle this.
import glob
file_path = glob.glob('\\folder\\todolist\\grocerylist_*.csv')[0]
If there are more files and you only need the most recent, you can sort and take the last one:
file_path = sorted(glob.glob('\\folder\\todolist\\grocerylist_*.csv'))[-1]
Perhaps you could use the python glob module to open the file with the prefix grocerylist_
import glob
path = glob.glob('/folder/todolist/grocerylist_*.csv')[0]
If you want to watch for file creation I would recommend using something like watchdog, this will always give you the latest file created that matches your pattern:
import sys
import time
from watchdog.observers import Observer
from watchdog.events import PatternMatchingEventHandler
class Handler(PatternMatchingEventHandler):
def __init__(self, regexes):
super(Handler, self).__init__(regexes)
def process(self, event):
print(event.src_path, event.event_type)
def on_created(self, event):
self.process(event)
if __name__ == '__main__':
import os
pth = sys.argv[1:]
observer = Observer()
pth = pth[0] if pth else "./"
observer.schedule(Handler([os.path.join(pth, "grocerylist_*.csv")]), path=pth[0] if pth else './')
observer.start()
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
observer.stop()
observer.join()
When a file matching your pattern is created on_created will be called which calls process and you can do whatever you like in the method:
~$ python3 watch.py &
[15] 21005
padraic#home:~$ touch grocerylist_12345.csv
padraic#home:~$ ./grocerylist_12345.csv created
If you want to open the file just do it in process.
Related
The python code below is my first attempt for notification when a file is modified, however nothing happens when the file is changed.
What have I missed here?
#!/usr/bin/python3
import time
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
class MyHandler(FileSystemEventHandler):
def on_modified(self, event):
print('file changed')
if __name__ == "__main__":
event_handler = MyHandler()
observer = Observer()
observer.schedule(event_handler, path='/Users/jeff/smb/storage/wsjt-x/wsjtx_log.adi', recursive=False)
observer.start()
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
observer.stop()
observer.join()
NOTE: I found that this code works well using Linux. It's on macOS Big Sur where I'm having issues.
After clarifying your environment in comments, one of following 3 approaches might resolve your issue.
Reproduction: Watching a local file on Linux
Running in one shell A and meanwhile using another shell B to add to the observed file so.md like:
echo "---" >> so.md
results in following output:
python3 so_watchdog.py so.md
file changed <FileModifiedEvent: event_type=modified, src_path='so.md', is_directory=False>
file changed <DirModifiedEvent: event_type=modified, src_path='', is_directory=True>
Here is the slightly modified version of your script (see comments):
#!/usr/bin/python3
import time
import sys
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
class MyHandler(FileSystemEventHandler):
def on_modified(self, event):
print('file changed', event) # added event to see the difference of two events for a single modification
if __name__ == "__main__":
if len(sys.argv) < 2:
file = '/Users/jeff/smb/storage/wsjt-x/wsjtx_log.adi'
else:
file = sys.argv[1]
# below only added the file as path
event_handler = MyHandler()
observer = Observer()
observer.schedule(event_handler, path=file, recursive=False)
observer.start()
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
observer.stop()
observer.join()
Note: This was tested on Linux, with a local file.
Watching shared files (CIFS/SMB)
Since your path /Users/jeff/smb/storage/wsjt-x/wsjtx_log.adi contains smb I assume, you are watching a file on a mounted shared network location using the SMB (Server Message Blocks) protocol, also known as CIFS (Common Internet File System).
Then please follow the official documentation, About using watchdog with CIFS:
When you want to watch changes in CIFS, you need to explicitly tell watchdog to use PollingObserver, that is, instead of letting watchdog decide an appropriate observer like in the example above, do:
from watchdog.observers.polling import PollingObserver as Observe
So your import statement needs to be modified.
Watching files locally on MacOS
For MacOS the underlying file-system event API used is FSEvents or FreeBSD's kqueue, instead of Linux's inotify.
Thus the behavior might also deviate.
I try to monitor text log file for changes and want to see lines that was added
I try with watchdog it seems to work with some manually created/edited files for testing, but my log file (in the same directory) isn't detected as "changed", but it was changed. It's probably because this file is already open (maybe in a specific mode). If I close the app that uses this log file and change it manually then watchdog works ok.
How do I check if the file was changed and if the changes were written to
the console?
#!/usr/bin/python
import time
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
class MyHandler(FileSystemEventHandler):
def on_modified(self, event):
print(f'event type: {event.event_type} path : {event.src_path}')
if __name__ == "__main__":
event_handler = MyHandler()
observer = Observer()
observer.schedule(event_handler, path='C:\gory\parcienaszklo\logs', recursive=False)
observer.start()
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
observer.stop()
observer.join()
I produced a script that detects the changes on files that are located in a specific directory. I'm trying to write all these changes to a changes.txt file. For this purpose I'm using the sys.stdout = open('changes.txt','w') instruction.
The problem is that whenever I run the script and change a file in the directory and save it, an empty file called changes.txt is created. This file is never written!
#!/usr/bin/python
import time
import sys
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
sys.stdout = open('changes.txt','w')
class MyHandler(FileSystemEventHandler):
def on_modified(self, event):
print "something happened!"
if __name__ == "__main__":
event_handler = MyHandler()
observer = Observer()
observer.schedule(event_handler, path='.', recursive=False)
observer.start()
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
observer.stop()
I'd recommend something like
#!/usr/bin/python
import time
import sys
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
class MyHandler(FileSystemEventHandler):
def __init__(self, f):
self.f = f
def on_modified(self, event):
self.f.write("something happened!\n")
self.f.flush()
if __name__ == "__main__":
with open('changes.txt','w') as f:
event_handler = MyHandler(f)
observer = Observer()
observer.schedule(event_handler, path='.', recursive=False)
observer.start()
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
observer.stop()
as you can see, the control over where your outputwill be written to has been handed to the caller (the one instanciating MyHandler) instead of the callee (on_modified).
This means you can also do
event_handler = MyHandler(sys.stdout)
and see the output instead of the output being put into the file.
An additional benefit: using a context manager you can be sure the file is closed properly, even if errors occurr.
I am trying to get the name of a file that changes periodically.
I am using watchdog to do this.
import time
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
timestr = time.strftime("%Y.%m.%d-%H.%M.%S")
class MyHandler(FileSystemEventHandler):
def on_modified(self, event):
change_log = open('change_log_' + timestr + '.txt', 'aw')
change_log.write('Time the file changed: ' + timestr + '\n')
change_log.close()
if __name__ == "__main__":
event_handler = MyHandler()
observer = Observer()
observer.schedule(event_handler, path='.', recursive=False)
observer.start()
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
observer.stop()
observer.join()
For some reason this prints out about 62 lines in the "change_log" file. This is not very useful.
What I would like to do is to print the name of the file that changed, or store it in a variable to pass to my other module.
In your example, if you need a file name, it is necessary to replace 'change_log_' with event.src_path. See the official code for more details.
You can also see the use of event.src_path in this answer as I used it in the printout.
It looks like the event object that is sent to your handler includes the information that you seek:
http://pythonhosted.org/watchdog/api.html#watchdog.events.FileSystemEvent
Use the src_path property of the event object that's passed into your FileSystemEvent subclass handler method to get the filename.
I'm intending to use Python watchdog to handle a directory where files are written to,
and I'm only interested in image files, trouble is I dont quite grok the code at this page.
This is my attempt:
from watchdog.observers import Observer
from watchdog.events import PatternMatchingEventHandler
class Beat(PatternMatchingEventHandler):
def on_create(self,event):
print event.src_path
if __name__ == "__main__":
patt = ['\w+[.]jpeg']
event_handler = Beat(patterns=patt,ignore_directories=True,)
observer = Observer()
path = "./"
observer.schedule(event_handler, path, recursive=True)
observer.start()
I'm trying to use the pattern matching class, but I'm getting nothing. How is it supposed to be used?
Based on the source code, fnmatch is being used under the hood. fnmatch can only do UNIX glob-style pattern matching. Which means you may have better luck with *.jpg than \w+[.]jpeg
You can actually use the RegexMatchingEventHandler instead of PatternMatchingEventHandler to accomplish exactly what you want to do:
from watchdog.observers import Observer
from watchdog.events import RegexMatchingEventHandler
class ExampleHandler(RegexMatchingEventHandler):
def on_create(self, event):
print(event.src_path)
if __name__ == "__main__":
pattern = '\w+\.jpeg'
event_handler = ExampleHandler(regexes=[pattern], ignore_directories=True)
observer = Observer()
path = "./"
observer.schedule(event_handler, path, recursive=True)
observer.start()
import time
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
observer.stop()
observer.join()