A few measurements of stat()'s speed

[H. S. Teoh]

On Tue, Mar 26, 2019 at 02:06:08PM -0400, Andrei Alexandrescu via Digitalmars-d wrote:
[...]
> On a Linux moderately-loaded local directory (146 files) mounted from
> an SSD drive, one failed stat() takes only about 0.5 microseconds.
> That means e.g.  if a module imports std.all (which fails 142 times),
> the overhead accountable to failed stat() calls is about 70
> microseconds, i.e. negligible.
> 
> The results change drastically when network mounts are tested. For
> sftp and sshfs mounts on a high speed local connection, one failed
> stat() takes 6-7 milliseconds, so an import like std.all (and many
> other imports liable to transitively pull others) would cause
> significant overheads.
> 
> So the question is whether many projects are likely to import files
> over network mounts, which would motivate the optimization. Please
> share your thoughts, thanks.
[...]

Does caching the contents of import directories cause significant
overhead?  If not, why not just cache it anyway, regardless of whether
the import happens across network mounts. Making excessive OS roundtrips
(calling stat() hundreds of times) should be reduced anyway.

//

On a slightly different note, why are we paying so much attention to
import speeds anyway?  We can optimize import speeds to hell and back
again until they cost practically zero time, yet the act of actually
*using* one of those imports -- ostensibly the reason you'd want to
import anything in the first place -- immediately adds a huge amount of
overhead that by far overshadows those niggly microseconds we pinched.
Ergo:

	import std.regex;
	void main() {
		version(withRegex)
			auto re = regex("a");
	}

This takes about 0.5 seconds to compile without -version=withRegex on my
machine. With -version=withRegex, it takes about *4.5 seconds* to
compile.  We have a 4 second bottleneck here and yet we're trying to
shave off microseconds elsewhere.  Why does instantiating a
single-character regex add FOUR SECONDS to compilation time?  I think
*that* is the question we should be answering.


T

-- 
People say I'm arrogant, and I'm proud of it.