From: "Richard Purdie" <richard.purdie@linuxfoundation.org>
To: Seebs <seebs@seebs.net>
Cc: openembedded-core@lists.openembedded.org
Subject: Re: [PATCH 2/9] pseudo: Add support for ignoring paths from the pseudo DB
Date: Mon, 12 Oct 2020 21:48:55 +0100 [thread overview]
Message-ID: <2523bf7b98d0a897060bba2286c4e7442bb9ee02.camel@linuxfoundation.org> (raw)
In-Reply-To: <20201012143137.6b4cbe34@seebsdell>
On Mon, 2020-10-12 at 14:31 -0500, Seebs wrote:
> On Wed, 7 Oct 2020 11:14:42 +0100
> Richard Purdie <richard.purdie@linuxfoundation.org> wrote:
>
> > ++int pseudo_client_ignore_path_chroot(const char *path, int
> > ignore_chroot) { ++ char *env;
> > ++ if (path) {
> > ++ if (ignore_chroot && pseudo_chroot && strncmp(path,
> > pseudo_chroot, pseudo_chroot_len) == 0) ++
> > return 0; ++ env =
> > pseudo_get_value("PSEUDO_IGNORE_PATHS"); ++ if (env) {
> > ++ char *p = env;
> > ++ while (*p) {
> > ++ char *next = strchr(p, ',');
> > ++ if (!next)
> > ++ next = strchr(p, '\0');
> > ++ if ((next - p) && !strncmp(path, p,
> > next - p)) { ++
> > pseudo_debug(PDBGF_PATH | PDBGF_VERBOSE, "ignoring path: '%s'\n",
> > path); ++ return 1;
> > ++ } ++
> > if (next && *next != '\0') ++
> > p = next+1; ++ else
> > ++ break;
> > ++ }
> > ++ free(env);
> > ++ }
> > ++ }
> > ++ return 0;
>
> This seems like it might merit caching; parse PSEUDO_IGNORE_PATHS at
> startup, build a table of paths (and probably their lengths), and use
> those? Seems like there's gonna be a LOT of hits on this.
>
> Apart from that, this looks plausible. It's certainly a large change,
> but I do think that reducing the size of the database like this is
> probably a big win.
I agree caching would be useful, I'm figuring if we can make this work,
we can make it faster in due course...
The database size win is actually quite surprising. The performance
isn't too different for builds overall with this change. What is
noticeable is the ~2GB improvement in size of TMPDIR from around 42GB
to 40GB. We even have charts to prove it:
https://autobuilder.yocto.io/pub/non-release/20201007-16/testresults/buildperf-ubuntu1604/perf-ubuntu1604_master_20201007212955_c194e5fac6.html
'59,583' doesn't have the pseudo change, '59,600' does. We had a
different patch just before it which also knocked 3GB of build
dependencies off.
https://autobuilder.yocto.io/pub/non-release/20201012-6/testresults/buildperf-ubuntu1604/perf-ubuntu1604_master_20201012150127_0c0b236b4c.html
is later data showing it was a step change.
Cheers,
Richard
next prev parent reply other threads:[~2020-10-12 20:48 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-07 10:14 [PATCH 1/9] pseudo: Ignore mismatched inodes from the db Richard Purdie
2020-10-07 10:14 ` [PATCH 2/9] pseudo: Add support for ignoring paths from the pseudo DB Richard Purdie
2020-10-12 19:31 ` Seebs
2020-10-12 20:48 ` Richard Purdie [this message]
2020-10-07 10:14 ` [PATCH 3/9] pseudo: Abort on mismatch patch Richard Purdie
2020-10-07 10:14 ` [PATCH 4/9] psuedo: Add tracking of linked files for fds Richard Purdie
2020-10-12 19:27 ` Seebs
2020-10-07 10:14 ` [PATCH 5/9] pseudo: Fix xattr segfault Richard Purdie
2020-10-12 19:22 ` Seebs
2020-10-07 10:14 ` [PATCH 6/9] pseudo: Add may unlink patch Richard Purdie
2020-10-12 19:20 ` Seebs
2020-10-12 20:33 ` Richard Purdie
2020-10-07 10:14 ` [PATCH 7/9] pseudo: Add pathfix patch Richard Purdie
2020-10-07 10:14 ` [PATCH 8/9] base/bitbake.conf: Enable pseudo path filtering Richard Purdie
2020-10-07 10:14 ` [PATCH 9/9] wic: Handle new PSEUDO_IGNORE_PATHS variable Richard Purdie
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2523bf7b98d0a897060bba2286c4e7442bb9ee02.camel@linuxfoundation.org \
--to=richard.purdie@linuxfoundation.org \
--cc=openembedded-core@lists.openembedded.org \
--cc=seebs@seebs.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox