We got many reports where the 'lazy check' didn't work too well for
people... so make it a bit less lazy, so it'll just work for more
people.
In practice, never skip _directories_ unless they're leaf directories;
this avoids the mtime-does-not-bubble-up problem.
b465c1f779