How to Set Up Python Watchdog for Folder Monitoring
Purpose
In this post, I will demonstrate how to use Python’s watchdog library to monitor folder changes and automatically trigger commands, replacing repetitive manual tasks with automated workflows.
The Environment
- Python 3.8+
- watchdog library
- Basic command line knowledge
The Problem
I was working on a documentation project where I had to manually rebuild the docs every time I made changes to markdown files. It looked like this:
$ make buildBuilding documentation...Done!
# edit some markdown files...
$ make buildBuilding documentation...Done!
# edit more files...
$ make buildBuilding documentation...Done!I got tired of switching between my editor and terminal to run the same command over and over. Sometimes I forgot to rebuild and wondered why my changes weren’t showing up.
What I Was Doing
My workflow was broken. I was:
- Edit a markdown file
- Switch to terminal
- Run
make build - Check the output
- Repeat 50 times a day
This context switching was killing my productivity. I needed something that would watch my files and run the build command automatically.
First Attempt: A Simple Loop
I tried a naive approach first:
import timeimport subprocessimport os
while True: result = subprocess.run(['make', 'build'], capture_output=True) print(result.stdout.decode()) time.sleep(5)But this was terrible. It ran the build every 5 seconds regardless of whether anything changed. My CPU was constantly busy, and I was getting false positives.
The Solution: Watchdog Library
I discovered the watchdog library. Here’s my first working version:
import timeimport subprocessfrom watchdog.observers import Observerfrom watchdog.events import FileSystemEventHandler
class BuildHandler(FileSystemEventHandler): def on_modified(self, event): if event.is_directory: return if event.src_path.endswith('.md'): print(f"Change detected: {event.src_path}") subprocess.run(['make', 'build'])
if __name__ == "__main__": path = "./docs" event_handler = BuildHandler() observer = Observer() observer.schedule(event_handler, path, recursive=True) observer.start()
try: while True: time.sleep(1) except KeyboardInterrupt: observer.stop() observer.join()I ran it:
$ pip install watchdog$ python watch_build.pyIt worked! But I noticed a problem immediately.
The Debounce Problem
When I saved a file in my editor, the handler triggered multiple times:
Change detected: ./docs/index.mdBuilding...Change detected: ./docs/index.mdBuilding...Change detected: ./docs/index.mdBuilding...Three builds for a single save! This happened because my editor writes to the file in multiple operations (write content, update metadata, etc.).
I needed to add debouncing:
import timeimport subprocessfrom watchdog.observers import Observerfrom watchdog.events import FileSystemEventHandlerfrom datetime import datetime
class DebouncedHandler(FileSystemEventHandler): def __init__(self, cooldown_seconds=1.0): self.cooldown = cooldown_seconds self.last_triggered = 0
def on_modified(self, event): if event.is_directory: return
now = time.time() if now - self.last_triggered < self.cooldown: return
self.last_triggered = now print(f"[{datetime.now().strftime('%H:%M:%S')}] Rebuilding...") result = subprocess.run(['make', 'build'], capture_output=True) if result.returncode == 0: print("Build successful!") else: print(f"Build failed: {result.stderr.decode()}")
if __name__ == "__main__": path = "./docs" event_handler = DebouncedHandler(cooldown_seconds=1.0) observer = Observer() observer.schedule(event_handler, path, recursive=True) observer.start()
try: while True: time.sleep(1) except KeyboardInterrupt: observer.stop() observer.join()Now it only triggers once per second at most:
[14:32:15] Rebuilding...Build successful!Handling Multiple File Types
I wanted to handle different file types differently. Markdown files should rebuild docs, Python files should run tests, and images should be converted:
import timeimport subprocessfrom watchdog.observers import Observerfrom watchdog.events import FileSystemEventHandlerfrom datetime import datetime
class SmartHandler(FileSystemEventHandler): def __init__(self, cooldown_seconds=1.0): self.cooldown = cooldown_seconds self.last_triggered = {} self.last_global = 0
def on_modified(self, event): if event.is_directory: return
path = event.src_path now = time.time()
# Global cooldown if now - self.last_global < self.cooldown: return self.last_global = now
timestamp = datetime.now().strftime('%H:%M:%S')
# Route different file types to different commands if path.endswith('.md'): print(f"[{timestamp}] Docs changed: {path}") subprocess.run(['make', 'build']) elif path.endswith(('.png', '.jpg', '.jpeg')): print(f"[{timestamp}] Image changed: {path}") subprocess.run(['python', 'scripts/optimize_images.py', path]) elif path.endswith('.py'): print(f"[{timestamp}] Python changed: {path}") subprocess.run(['pytest', 'tests/', '-q'])
if __name__ == "__main__": path = "." event_handler = SmartHandler(cooldown_seconds=1.0) observer = Observer() observer.schedule(event_handler, path, recursive=True) observer.start()
print("Watching for changes... Press Ctrl+C to stop") try: while True: time.sleep(1) except KeyboardInterrupt: observer.stop() observer.join()Common Mistakes I Made
Mistake 1: Not Filtering File Types
Initially, my handler triggered on every file change, including .pyc files and .git directory changes:
def on_modified(self, event): # This triggers on EVERYTHING subprocess.run(['make', 'build'])The fix is to filter by file extension:
def on_modified(self, event): if event.is_directory: return if not event.src_path.endswith(('.md', '.py', '.js')): return subprocess.run(['make', 'build'])Mistake 2: Watching Too Broad a Directory
I initially watched the entire project directory, which caused performance issues:
observer.schedule(event_handler, ".", recursive=True)# Watches everything including node_modules, .git, __pycache__The fix is to watch only what you need:
observer.schedule(event_handler, "./src", recursive=True)observer.schedule(event_handler, "./docs", recursive=True)Mistake 3: Not Handling Exceptions
When the build command failed, my script would crash:
subprocess.run(['make', 'build'])# If make fails, we don't knowThe fix is to capture and handle errors:
result = subprocess.run(['make', 'build'], capture_output=True)if result.returncode != 0: print(f"Error: {result.stderr.decode()}") # Optionally: send notification, log to file, etc.How Watchdog Works Under the Hood
The watchdog library uses different mechanisms depending on the operating system:
+------------------+ +-------------------+ +------------------+| File System | --> | OS Event API | --> | Watchdog || Changes | | (inotify/FSEvnt | | Observer |+------------------+ | /ReadDirectory) | +------------------+ +-------------------+ | v +-------------------+ +------------------+ | Your Handler | <-- | Event Queue | | (on_modified, | | (thread-safe) | | on_created) | +------------------+ +-------------------+On Linux, it uses inotify. On macOS, it uses FSEvents. On Windows, it uses ReadDirectoryChangesW. This means you get native performance on each platform.
Related Knowledge
Watchdog vs Other Tools
| Tool | Pros | Cons |
|---|---|---|
| watchdog | Cross-platform, Python-native, flexible | Requires Python |
| nodemon | Great for Node.js, simple setup | Node.js only |
| entr | Lightweight, Unix-native | Not cross-platform |
| fswatch | Cross-platform CLI | Requires separate installation |
Event Types Available
Watchdog provides several event types you can handle:
class MyHandler(FileSystemEventHandler): def on_created(self, event): """Called when a file or directory is created.""" pass
def on_deleted(self, event): """Called when a file or directory is deleted.""" pass
def on_modified(self, event): """Called when a file or directory is modified.""" pass
def on_moved(self, event): """Called when a file or directory is moved or renamed.""" pass
def on_any_event(self, event): """Called on any event.""" passSummary
In this post, I showed how to use Python’s watchdog library to automate repetitive development tasks. The key points are:
- Install watchdog with
pip install watchdog - Create a handler class extending
FileSystemEventHandler - Override the event methods you need (
on_modified,on_created, etc.) - Add debouncing to prevent multiple rapid triggers
- Filter by file type to avoid unnecessary work
- Handle exceptions properly
This simple 15-line script replaced three separate tools I used to have installed, and it saves me hours of context switching every week.
Final Words + More Resources
My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me
Here are also the most important links from this article along with some further resources that will help you in this scope:
Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!
Comments