Re: [PATCH] 2.6.0 - Watchdog patches (BK consistency checks)

[Andy Isaacson <adi@hexapodia.org>]

On Tue, Dec 30, 2003 at 12:56:32PM -0700, Eric D. Mudama wrote:
> On Tue, Dec 30 at 13:13, Andy Isaacson wrote:
> >On Tue, Dec 30, 2003 at 08:36:15AM -0500, Ed Tomlinson wrote:
> >The consistency check definitely should not take 15 minutes.  It should
> >be (much) less than 30 seconds.  What is the hardware you're running on?
> >
> >I'm running on an Athlon 2 GHz (XP 2400+) with 512MB and a 7200RPM IDE
> >disk, and I can do a complete clone (with full data copy and consistency
> >check) of the 2.4 tree in 1:40.  That was with cold caches; with the
> >sfile copies and "checkout:get", a half-gig isn't enough to cache
> >everything.  The consistency check is about 19 seconds (bk -r check -acv).
> 
> For what it is worth:
> 
> AMD Duron 950MHz, 768MB RAM
> 7200RPM 80GB Quantum Viper IDE drive, 26% full
> 
> phat-penguin:~/src/linux-2.5> time bk -r check -acv
> 100% |=================================================================| OK
> 42.710u 5.770s 2:04.63 38.8%    0+0k 0+0io 74078pf+0w
> 
> over 2 minutes of wall time, 42 seconds of "user" time... (if I'm reading it right), without primed disk caches.
> 
> The 2nd run, half a minute later:
> 
> phat-penguin:~/src/linux-2.5> time bk -r check -acv
> 100% |=================================================================| OK
> 41.900u 3.080s 0:45.53 98.7%    0+0k 0+0io 74078pf+0w
> 
> 
> ...would appear to show that BK's checksumming, on my system, is
> constrained near 41 seconds of calculation time, and the difference
> between the user and the wall-clock time is basically time spent
> waiting for the disk to do all its reads.
> 
> 
> I guess in that case, it'd be interesting to see what the user and
> wall times were for the original poster who reported a 15+ minute
> integrity check.

That's basically right, except that if you don't have enough memory to
keep bk's working set in memory, then you're paging and performance
starts to suck.

I didn't realize that the cpu-bound portion of the check would scale so
closely with CPU speed, but it looks like the scaling is almost dead-on;
18.4/41.9 = .439
950/2000  = .475

So I was wrong to say "less than 30 seconds".  "If you have a fast CPU
and enough memory", I guess.  But the memory matters a lot more than the
CPU.

-andy