oe-init-build-env slowdown: chmod -R

oe-init-build-env slowdown: chmod -R

[Patrick Ohly]

Hello!

The following commit introduced a "chmod -R -st" on the build dir:

commit f0d3587706fffdcc1c708013f6d6ce296dfc5f24
Author: Alex Franco <alejandro.franco@linux.intel.com>
Date:   Fri Aug 28 17:34:04 2015 -0500

    Fix mode +st on TMPDIR when creating it
    
    A sanity check fails when TMPDIR has setuid, setgid set. It was
    proposed to fix this on TMPDIR creation instead of failing with
    a sanity test only. This patch handles removal of those special
    bits (and additonally, the sticky bit) from BUILDDIR and TMPDIR
    when these directories are created.

In my case (*), that chmod takes about 2 minutes to complete. strace
shows that it does a fchmodat on all files, even if nothing needs to be
changed. atop then shows that this keeps the disk 100% busy with writes.
I don't know what gets written (can't be the access time, I tried
mounting with noatime).

Are others seeing the same behavior? I can't rule out that my disk is
just slow due to fragmentation.

The following commands work much better for me. I can submit a patch if
others see the same thing.

        find . -perm /+st -print0 | xargs -0 --no-run-if-empty chmod -st


(*) Debian 8, Linux 3.16.0-4-amd64, coreutils 8.23-4, ext4 on a SATA
hard disk.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

Re: oe-init-build-env slowdown: chmod -R

[Jussi Kukkonen]

[-- Attachment #1: Type: text/plain, Size: 1278 bytes --]

On 4 September 2015 at 09:42, Patrick Ohly <patrick.ohly@intel.com> wrote:

> Hello!
>
> The following commit introduced a "chmod -R -st" on the build dir:
>
> commit f0d3587706fffdcc1c708013f6d6ce296dfc5f24
> Author: Alex Franco <alejandro.franco@linux.intel.com>
> Date:   Fri Aug 28 17:34:04 2015 -0500
>
>     Fix mode +st on TMPDIR when creating it
>
>     A sanity check fails when TMPDIR has setuid, setgid set. It was
>     proposed to fix this on TMPDIR creation instead of failing with
>     a sanity test only. This patch handles removal of those special
>     bits (and additonally, the sticky bit) from BUILDDIR and TMPDIR
>     when these directories are created.
>
> In my case (*), that chmod takes about 2 minutes to complete. strace
> shows that it does a fchmodat on all files, even if nothing needs to be
> changed. atop then shows that this keeps the disk 100% busy with writes.
> I don't know what gets written (can't be the access time, I tried
> mounting with noatime).
>
> Are others seeing the same behavior? I can't rule out that my disk is
> just slow due to fragmentation.
>

 d8caef4f6 in master should fix this (and yes it was a problem elsewhere
too -- an autobuilder was seen spending 12 minutes on that).

Jussi

[-- Attachment #2: Type: text/html, Size: 1825 bytes --]

Re: oe-init-build-env slowdown: chmod -R

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 575 bytes --]

On 4 September 2015 at 07:42, Patrick Ohly <patrick.ohly@intel.com> wrote:

> In my case (*), that chmod takes about 2 minutes to complete. strace
> shows that it does a fchmodat on all files, even if nothing needs to be
> changed. atop then shows that this keeps the disk 100% busy with writes.
> I don't know what gets written (can't be the access time, I tried
> mounting with noatime).
>

The autobuilder has been seen to take 14 minutes at this stage before being
killed by buildbot. :) There's a patch on the list for this, and will be
merged soon.

Ross

[-- Attachment #2: Type: text/html, Size: 1002 bytes --]