Nice writeup. I work on native macOS utilities and have dealt with file monitoring quite a bit. kqueue is reliable but the per-file descriptor requirement can get tricky when you need to watch large directory trees.
For anyone looking at this space, FSEvents is the higher-level alternative Apple provides. It watches directory-level changes without needing a file descriptor per file, which scales better for broad monitoring. But kqueue gives you more granular control, like detecting attribute changes or renames specifically, which FSEvents sometimes lumps together.
In practice I've found a combination works well: FSEvents for broad directory watching to catch that something changed, then kqueue for targeted monitoring of specific files you care about.
There are benchmarks in there, and comparisons with all the other watchers out there I could find at the time I made it (see "Comparison with Similar Projects" at the very bottom of the readme).
It's a pretty long list. Tons of watchers out there with different design philosophies and shapes of problems they solve.
There are a lot of caveats to the filesystem monitoring APIs provided by kernels. Some projects (like facebook's watchman) take that as a kind of antagonism, and decide to fight back with layers and layers of fallbacks and distrust and rescans. That projects basically only makes sense as a daemon.
Other programs and libraries try to take that complexity and tame it by being super-focused on one platform or providing a lot of configuration options.
Some provide debouncing logic. This particular feature comes up from time to time, I believe, both for practical reasons and because over-reported events from the kernel subsystems, especially for some arcane events like moving a file across mount points can trigger a flurry of hard-to-associate events for the same path.
If you want to avoid dealing with under-documented filesystem event subsystems, you can also just make your own with ebpf. Especially for security-oriented systems, you'll find that the only (nearly) perfectly accurate filesystem event subsystem you can make, is the one you make from the ground up.
Love it. Are there any benchmarks available on how fast this is from file change, kernel event received, event dispatched? I would expect it to be really quick.
For anyone looking at this space, FSEvents is the higher-level alternative Apple provides. It watches directory-level changes without needing a file descriptor per file, which scales better for broad monitoring. But kqueue gives you more granular control, like detecting attribute changes or renames specifically, which FSEvents sometimes lumps together.
In practice I've found a combination works well: FSEvents for broad directory watching to catch that something changed, then kqueue for targeted monitoring of specific files you care about.
There are benchmarks in there, and comparisons with all the other watchers out there I could find at the time I made it (see "Comparison with Similar Projects" at the very bottom of the readme).
It's a pretty long list. Tons of watchers out there with different design philosophies and shapes of problems they solve.
There are a lot of caveats to the filesystem monitoring APIs provided by kernels. Some projects (like facebook's watchman) take that as a kind of antagonism, and decide to fight back with layers and layers of fallbacks and distrust and rescans. That projects basically only makes sense as a daemon.
Other programs and libraries try to take that complexity and tame it by being super-focused on one platform or providing a lot of configuration options.
Some provide debouncing logic. This particular feature comes up from time to time, I believe, both for practical reasons and because over-reported events from the kernel subsystems, especially for some arcane events like moving a file across mount points can trigger a flurry of hard-to-associate events for the same path.
If you want to avoid dealing with under-documented filesystem event subsystems, you can also just make your own with ebpf. Especially for security-oriented systems, you'll find that the only (nearly) perfectly accurate filesystem event subsystem you can make, is the one you make from the ground up.
For more software in this domain see also the excellent entr https://eradman.com/entrproject/
The OpenBSD documentation for it is top notch, as usual. No idea about the rest (I suspect they’ve all converged at this point).