tmp on NFS

tmp on NFS

[Luke (Lucas) Starrett]

[-- Attachment #1: Type: text/plain, Size: 516 bytes --]

Hi,

Can anybody give a brief history of time on why using an NFS drive for tmp is necessarily a bad thing, and why we have a sanity check for it?  We're doing this without any obvious side effects.

I'm aware of the checks added by changes like this:

patchwork.openembedded.org/patch/61107/

However, I don't see the reasoning/background documented as to exactly what is actually broken when putting tmp on NFS.  Is it time skew, problems with concurrent file access, something else?

Thanks,

Luke


[-- Attachment #2: Type: text/html, Size: 3082 bytes --]

Re: tmp on NFS

[Mike Looijmans]

I can think of various things that would go wrong with tmp on NFS. One of the 
most obvious example would be to try and change the network configuration 
while running, and needing some temporary file to manage that.\

I think the expectation is that /tmp should be accessible at all times, and 
that it's local and (at least somewhat) volatile.

I tend to mount /tmp/ in RAM on all systems. Even my desktop. Not having to do 
wait for a device IO queue when performing actions in /tmp/ can greatly 
improve the responsiveness of the system.

If your application's /tmp/ storage requirements are such that they don't fit 
in RAM, I don't think /tmp/ is the place where they should be stored.



On 07-10-15 03:37, Luke (Lucas) Starrett wrote:
> Hi,
>
> Can anybody give a brief history of time on why using an NFS drive for tmp is
> necessarily a bad thing, and why we have a sanity check for it?  We’re doing
> this without any obvious side effects.
>
> I’m aware of the checks added by changes like this:
>
> patchwork.openembedded.org/patch/61107/
>
> However, I don’t see the reasoning/background documented as to exactly what is
> actually broken when putting tmp on NFS.  Is it time skew, problems with
> concurrent file access, something else?
>
> Thanks,
>
> Luke
>
>
>



Kind regards,

Mike Looijmans
System Expert

TOPIC Embedded Products
Eindhovenseweg 32-C, NL-5683 KH Best
Postbus 440, NL-5680 AK Best
Telefoon: +31 (0) 499 33 69 79
Telefax: +31 (0) 499 33 69 70
E-mail: mike.looijmans@topicproducts.com
Website: www.topicproducts.com

Please consider the environment before printing this e-mail

Re: tmp on NFS

[Luke (Lucas) Starrett]

I should clarify, the sanity check warning I'm referring to is about the Yocto build/tmp directory not /tmp as in the system level tmp dir.

Thanks, Luke.


-----Original Message-----
From: yocto-bounces@yoctoproject.org [mailto:yocto-bounces@yoctoproject.org] On Behalf Of Mike Looijmans
Sent: Wednesday, October 07, 2015 3:01 AM
To: yocto@yoctoproject.org
Subject: Re: [yocto] tmp on NFS

I can think of various things that would go wrong with tmp on NFS. One of the most obvious example would be to try and change the network configuration while running, and needing some temporary file to manage that.\

I think the expectation is that /tmp should be accessible at all times, and that it's local and (at least somewhat) volatile.

I tend to mount /tmp/ in RAM on all systems. Even my desktop. Not having to do wait for a device IO queue when performing actions in /tmp/ can greatly improve the responsiveness of the system.

If your application's /tmp/ storage requirements are such that they don't fit in RAM, I don't think /tmp/ is the place where they should be stored.



On 07-10-15 03:37, Luke (Lucas) Starrett wrote:
> Hi,
>
> Can anybody give a brief history of time on why using an NFS drive for tmp is
> necessarily a bad thing, and why we have a sanity check for it?  We’re doing
> this without any obvious side effects.
>
> I’m aware of the checks added by changes like this:
>
> patchwork.openembedded.org/patch/61107/
>
> However, I don’t see the reasoning/background documented as to exactly what is
> actually broken when putting tmp on NFS.  Is it time skew, problems with
> concurrent file access, something else?
>
> Thanks,
>
> Luke
>
>
>



Kind regards,

Mike Looijmans
System Expert

TOPIC Embedded Products
Eindhovenseweg 32-C, NL-5683 KH Best
Postbus 440, NL-5680 AK Best
Telefoon: +31 (0) 499 33 69 79
Telefax: +31 (0) 499 33 69 70
E-mail: mike.looijmans@topicproducts.com
Website: www.topicproducts.com

Please consider the environment before printing this e-mail





-- 
_______________________________________________
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: tmp on NFS

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 874 bytes --]

On 7 October 2015 at 02:37, Luke (Lucas) Starrett <lstarret@broadcom.com>
wrote:

> I’m aware of the checks added by changes like this:
>
>
>
> patchwork.openembedded.org/patch/61107/
>
>
>
> However, I don’t see the reasoning/background documented as to exactly
> what is actually broken when putting tmp on NFS.  Is it time skew, problems
> with concurrent file access, something else?
>

Follow the link in the patch to the referenced bug, and comment #8 by Mark
Hatle explains the problems that Wind River were seeing in testing:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=5442#c8

(background: a lot of the sanity tests are from Wind River who appear to
have a pathological need to break software: what happens if I do a build
nested 20 directories deep, what happens if I've a space in the build
directory name, and so on)

Ross

[-- Attachment #2: Type: text/html, Size: 1746 bytes --]

Re: tmp on NFS

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 474 bytes --]

On 7 October 2015 at 13:59, Burton, Ross <ross.burton@intel.com> wrote:

> Follow the link in the patch to the referenced bug, and comment #8 by Mark
> Hatle explains the problems that Wind River were seeing in testing:
>

I should also add that many people buy lots of RAM and do builds with
build/tmp in a tmpfs as it gives you a noticeable performance increase, so
I dread to think how much it will hurt to do a build with build/tmp on a
network mount.

Ross

[-- Attachment #2: Type: text/html, Size: 877 bytes --]

Re: tmp on NFS

[Mike Looijmans]

On 07-10-15 15:04, Burton, Ross wrote:
>
> On 7 October 2015 at 13:59, Burton, Ross <ross.burton@intel.com
> <mailto:ross.burton@intel.com>> wrote:
>
>     Follow the link in the patch to the referenced bug, and comment #8 by Mark
>     Hatle explains the problems that Wind River were seeing in testing:
>
>
> I should also add that many people buy lots of RAM and do builds with
> build/tmp in a tmpfs as it gives you a noticeable performance increase, so I
> dread to think how much it will hurt to do a build with build/tmp on a network
> mount.

Telling Ubuntu to actually use all that RAM for some serious caching (e.g. 
allow 50% dirty pages, and extend the timeout to several minutes), and (if you 
have >14) putting cfq back into its rightful place as io scheduler made a huge 
difference for me.



Kind regards,

Mike Looijmans
System Expert

TOPIC Embedded Products
Eindhovenseweg 32-C, NL-5683 KH Best
Postbus 440, NL-5680 AK Best
Telefoon: +31 (0) 499 33 69 79
Telefax: +31 (0) 499 33 69 70
E-mail: mike.looijmans@topicproducts.com
Website: www.topicproducts.com

Please consider the environment before printing this e-mail