Is it possible to use a local drive to cache an NFS share

Is it possible to use a local drive to cache an NFS share

[Graeme Russ]

Hi All,

I am mounting /home over NFS which is great, but it really kills compile
times. So I have a local HDD which I have copied all my source code over
from which I do my coding and compiling.

What I would love to do is use the local drive to cache /home. Specifically
one user directory under /home - only one machine will ever be modifying
the contents of this directory.

Or should I just use rsync?

On a side note - I moved my source code from a 1TB HDD on a 3Gb/s SATA port
to an Intel 510 series 250GB SSD on a 6Gb/s SATA port. But I didn't see an
appreciable increase in compile speed.

Regards,

Graeme

Is it possible to use a local drive to cache an NFS share

[michi1 at michaelblizek.twilightparadox.com]

Hi!

On 23:00 Mon 03 Sep     , Graeme Russ wrote:
> Hi All,
> 
> I am mounting /home over NFS which is great, but it really kills compile
> times. So I have a local HDD which I have copied all my source code over
> from which I do my coding and compiling.
> 
> What I would love to do is use the local drive to cache /home. Specifically
> one user directory under /home - only one machine will ever be modifying
> the contents of this directory.

NFS is not really made for caching. Its design idea was mainly that the server
should be stateless. There is basically no way to get a notification whether a
file was changed remotely without either polling or running an external
daemon. There are 2 network file systems which support client caching: codafs
and andrewfs. However, they are rather complex to setup and I guess nobody is
really using them.

If nobody else changes the files you should be fine. But I still would not
recommend doing this, because this is basically a time bomb which might go off
after you forget what you are doing now. Also there should not be that much
performance difference for your worklead unless either your network or your
nas is very slow.

After a quick search I have found this project (not sure what it does exactly):
http://sourceforge.net/apps/mediawiki/fuse/index.php?title=FileSystems#mcachefs

> Or should I just use rsync?

How do you want to sync it back?

> On a side note - I moved my source code from a 1TB HDD on a 3Gb/s SATA port
> to an Intel 510 series 250GB SSD on a 6Gb/s SATA port. But I didn't see an
> appreciable increase in compile speed.

This does not suprise me. Compiling mostly stresses the cpu(s).

	-Michi
-- 
programing a layer 3+4 network protocol for mesh networks
see http://michaelblizek.twilightparadox.com

Is it possible to use a local drive to cache an NFS share

[Mulyadi Santosa]

Hi :)

On Mon, Sep 3, 2012 at 8:00 PM, Graeme Russ <graeme.russ@gmail.com> wrote:
> Hi All,
>
> I am mounting /home over NFS which is great, but it really kills compile
> times. So I have a local HDD which I have copied all my source code over
> from which I do my coding and compiling.
>
> What I would love to do is use the local drive to cache /home. Specifically
> one user directory under /home - only one machine will ever be modifying
> the contents of this directory.

I suggest to use ccache, if your focus is only about compiling.

Then configure ccache to use your local drive as the directory to
store the cache objects.

IMHO, that should be doable :)



-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

Is it possible to use a local drive to cache an NFS share

[Rajat Sharma]

Hi,

NFS client support FSCache support which is default in recent kernels
CONFIG_NFS_FSCACHE=y, But as I remember, it is a read cache (not even
write-through), so a compile load generates lots of new files, so
those won't really be benefited by FSCache.

But as you are pointing out, your setup is a single user access, so
probably you can relax some of the NFS consistency options, e.g. async
mount is really suitable for your case, also you can increase actimeo
values. Compiler writes will hit-page cache and with asynchronous
writebacks, it is way more faster than SSDs :).

-Rajat

On Mon, Sep 3, 2012 at 6:30 PM, Graeme Russ <graeme.russ@gmail.com> wrote:
> Hi All,
>
> I am mounting /home over NFS which is great, but it really kills compile
> times. So I have a local HDD which I have copied all my source code over
> from which I do my coding and compiling.
>
> What I would love to do is use the local drive to cache /home. Specifically
> one user directory under /home - only one machine will ever be modifying
> the contents of this directory.
>
> Or should I just use rsync?
>
> On a side note - I moved my source code from a 1TB HDD on a 3Gb/s SATA port
> to an Intel 510 series 250GB SSD on a 6Gb/s SATA port. But I didn't see an
> appreciable increase in compile speed.
>
> Regards,
>
> Graeme
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Is it possible to use a local drive to cache an NFS share

[michi1 at michaelblizek.twilightparadox.com]

Hi!

On 22:27 Mon 03 Sep     , Mulyadi Santosa wrote:
> Hi :)
> 
> On Mon, Sep 3, 2012 at 8:00 PM, Graeme Russ <graeme.russ@gmail.com> wrote:
> > Hi All,
> >
> > I am mounting /home over NFS which is great, but it really kills compile
> > times. So I have a local HDD which I have copied all my source code over
> > from which I do my coding and compiling.
> >
> > What I would love to do is use the local drive to cache /home. Specifically
> > one user directory under /home - only one machine will ever be modifying
> > the contents of this directory.
> 
> I suggest to use ccache, if your focus is only about compiling.
> 
> Then configure ccache to use your local drive as the directory to
> store the cache objects.

ccache caches the compilation result, but it still needs to reads the source
code files and write the object files...

	-Michi
-- 
programing a layer 3+4 network protocol for mesh networks
see http://michaelblizek.twilightparadox.com