Re: Re: NFS Patch for FSCache

[David Masover <ninja@slaphack.com>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Troy Benjegerdes wrote:
>>Reading one really big file (bigger than the memory available) over AFS, with
>>a cold cache it took very roughly 107% of the time it took with no cache; but
>>using a warm cache, it took 14% of the time it took with no cache. However,
>>this is on my particular test box, and it varies a lot from box to box.
> 
> 
> What network did that box have?
> 
> I'm finding that with OpenAFS, and memcache, read performance is
> affected greatly by the -chunksize argument to afsd. Using -chunksize 20 
> (1MB chunks) gets me around 50MB/sec, while -chunksize 18 gets
> 5-7MB/sec. (I believe that's the size of the 'fetchrpc' calls)
> 
> Another question with the afs client.. I'd really like to use the kafs
> client to mount a root filesystem, then use OpenAFS to mount /afs so
> I can have read/write support. I went so far as to patch kafs to mount 
> as type 'kafs', but then found that both clients want to listent on port
> 7000. Can I either
> 
> 	a) change the port for kafs
> 	b) get working read-write and auth support for kafs?
> 
> I'm guessing a) is much more likely.. 

How about a more hack-ish solution?

An initrd environment of some sort.  This especially makes sense if you
use a hard disk partition -- assuming the local disk is only used for
local cache, it's a good idea.

How I'd do it:

If kernel and initrd are server-side (as in a PXE boot):
- - initrd copies itself to tmpfs and then deallocates itself, so that it
can be swapped out
- - new tmpfs-based initrd has an OpenAFS client, and mounts /afs
- - root is somewhere in /afs, and the initrd pivot_root's to it.

If kernel and initrd are client-side (hard drive):
- - as above, except:
- - initrd checks for a new version of itself
- - if there is a new version of kernel/initrd, it:
 - downloads both a new kernel and a new initrd
 - saves them to disk, so we don't re-download next boot
 - kexecs the new kernel/initrd
 - assuming the upgrade worked, the new kernel/initrd will:
  - check for an update again
  - won't find one and will continue booting.

Also, we don't actually need a ramdisk -- we can use a partition.  This
is a little easier to manage, because we don't have to worry about
freeing the ramdisk or how much swap we need or things like that.

So, this is a bit harder to maintain than what you're suggesting, but
it's workable.  The admin just needs a good set of tools to rebuild the
initrd every kernel upgrade, or when he decides to add stuff to it.
But, it provides a pretty decent working environment once it's done, and
it puts everything except the AFS client on the AFS root fs.  It also
avoids using the kafs client at all -- all network stuff is done in
userspace.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQIVAwUBQow2z3gHNmZLgCUhAQJCtQ//VAFNzVo8kZFrkhtrYvq5qkmashiFOC4h
P2GA3GLNWPCjQ+UDwwhFUd89Zudu8iPDkaiiWtrOIt9I5jzjX0pjPLiamBZV1u8n
eNdNClpRUbWFNoItfczMvENftD9ZfaWtmn5CX/7IF2R1fEhEGdI5a67ibHeDTa8H
aqYGjb4KWyGYbrHbJUsHXbsCrxZdKbmp/esS6sotpFdBkWyZMFzQsGE4Insvyqrq
mfhX+o4YGql0p+UvcqJ8U823HQwpsVomnD1OE7NzAP01Y1Q0N5fUCTE92zMwqljR
6t9/FLBx0Vlp976WOQX1pSPlKy2nYL0VbuiGLfKbaWc3eHngBXdaW1qiNxC+QSK3
Nb6ojJ9QdQInwP0z0uQpWm+Od8Rv0qxuEx+kmpckOtN+0xuHH/f6DKzm1zrEMRkA
GhFTQfBda0DZ/4z6Qxrya8Je5+8NAno73DkKfFlboqd3JpJSiJ6sJRCGmtAVE5gG
1MaRUV3oXuALXd37gUNoPbCzlB5A6aZn9F4TV0jQSd388Ya97wwACDwRlAHyOLa/
yAx69UYPM+rEv0n3f5QtK8PBEIEh+W+e3VD7k1U2SiyV76WYL00BfqucdadHjqv8
dAawVzIsfv0Fl9xqUbk3BJYRokG0WzgA7B+6qS3C398/c9KTkTf1ZIdnQzVKmPL3
95k5sD1lNG4=
=X8zT
-----END PGP SIGNATURE-----