All of lore.kernel.org
 help / color / mirror / Atom feed
From: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
To: buildroot@busybox.net
Subject: [Buildroot] [PATCH v6 17/28] core/pkg-infra: Fix package file statistics for parallel build
Date: Wed, 11 Mar 2020 23:07:03 +0100	[thread overview]
Message-ID: <20200311230703.37bf0b63@windsurf> (raw)
In-Reply-To: <20200217212350.29750-18-anaumann@ultratronik.de>

Hello,

On Mon, 17 Feb 2020 22:23:39 +0100
Andreas Naumann <anaumann@ultratronik.de> wrote:

> When activating top level parallel build, multiple processes may
> try to modify the various *files-list* files in the common build/
> directory at the same time. This can cause racy build failures.
> 
> The fix here is to use flock to ensure exclusive execution of the
> statistics gathering code. For this to work it is assumed that the
> target/staging/host directories are isolated, which is true for
> per-package builds.
> For standard sequential builds, the locking is of course unnecessary
> but no conditional handling is implemented  since the runtime cost is
> expected to be next to nothing.
> 
> For flock to work, the statistics gathering code must be run in a
> single subshell. Otherwise the lockfile descriptor would be closed
> after the subshell (= the Makefile line where it was opened) ends
> and thus the lock would be released before even entering the critical
> code section.
> 
> Signed-off-by: Andreas Naumann <anaumann@ultratronik.de>

This is no longer needed: we have taken a different approach in commit
0e2be4db8ab01d479177a3a187c22525752195ae, where we make the logic
per-package, and collect the overall data in target-finalize.

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

  reply	other threads:[~2020-03-11 22:07 UTC|newest]

Thread overview: 98+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-02-17 21:23 [Buildroot] [PATCH v6 00/28] Qt5 qmake infra and per-package compatibility Andreas Naumann
2020-02-17 21:23 ` [Buildroot] [PATCH v6 01/28] package/qt5base: Do not build shared libs if BR2_STATIC_LIBS is chosen Andreas Naumann
2020-03-09 21:21   ` Thomas Petazzoni
2020-03-09 22:42     ` Peter Seiderer
2020-03-10  8:13       ` Thomas Petazzoni
2020-03-10  8:17         ` Peter Seiderer
2020-03-10 17:02     ` Andreas Naumann
2020-02-17 21:23 ` [Buildroot] [PATCH v6 02/28] core/pkg-infra: Add generic qmake package infrastructure Andreas Naumann
2020-03-09 22:10   ` Thomas Petazzoni
2020-03-10 17:10     ` Andreas Naumann
2020-02-17 21:23 ` [Buildroot] [PATCH v6 03/28] qt5: Convert straightforward packages to qmake infra Andreas Naumann
2020-03-09 22:12   ` Thomas Petazzoni
2020-03-10 18:09     ` Andreas Naumann
2020-02-17 21:23 ` [Buildroot] [PATCH v6 04/28] package/qt5base: Convert " Andreas Naumann
2020-03-09 22:29   ` Thomas Petazzoni
2020-03-10 18:20     ` Andreas Naumann
2020-02-17 21:23 ` [Buildroot] [PATCH v6 05/28] package/qwt: Partially convert " Andreas Naumann
2020-03-11 21:19   ` Thomas Petazzoni
2020-03-14 20:58     ` Andreas Naumann
2020-02-17 21:23 ` [Buildroot] [PATCH v6 06/28] package/qwt: Add missing qt5svg dependency Andreas Naumann
2020-02-17 21:56   ` Thomas Petazzoni
2020-02-18  9:43     ` Andreas Naumann
2020-02-18 14:58       ` Thomas Petazzoni
2020-02-19 19:16   ` Thomas Petazzoni
2020-02-17 21:23 ` [Buildroot] [PATCH v6 07/28] package/qt5tools: Partially convert to qmake infra Andreas Naumann
2020-03-11 21:22   ` Thomas Petazzoni
2020-02-17 21:23 ` [Buildroot] [PATCH v6 08/28] package/qt5virtualkeyboard: Convert " Andreas Naumann
2020-03-11 21:35   ` Thomas Petazzoni
2020-02-17 21:23 ` [Buildroot] [PATCH v6 09/28] package/quazip: " Andreas Naumann
2020-02-17 21:57   ` Thomas Petazzoni
2020-02-18 10:00     ` Andreas Naumann
2020-03-11 21:42       ` Thomas Petazzoni
2020-03-14 21:52         ` Andreas Naumann
2020-02-17 21:23 ` [Buildroot] [PATCH v6 10/28] package/qt5wayland: " Andreas Naumann
2020-03-11 21:48   ` Thomas Petazzoni
2020-02-17 21:23 ` [Buildroot] [PATCH v6 11/28] package/qt5webchannel: " Andreas Naumann
2020-03-11 21:49   ` Thomas Petazzoni
2020-02-17 21:23 ` [Buildroot] [PATCH v6 12/28] package/qt5webkit: " Andreas Naumann
2020-03-11 21:55   ` Thomas Petazzoni
2020-02-17 21:23 ` [Buildroot] [PATCH v6 13/28] package/qt5webkit-examples: " Andreas Naumann
2020-03-11 21:57   ` Thomas Petazzoni
2020-02-17 21:23 ` [Buildroot] [PATCH v6 14/28] package/qt5webengine: " Andreas Naumann
2020-03-11 22:01   ` Thomas Petazzoni
2020-03-16 21:15     ` Andreas Naumann
2020-02-17 21:23 ` [Buildroot] [PATCH v6 15/28] package/qt-webkit-kiosk: " Andreas Naumann
2020-03-11 22:04   ` Thomas Petazzoni
2020-03-16 21:53     ` Andreas Naumann
2020-02-17 21:23 ` [Buildroot] [PATCH v6 16/28] package/qt5cinex: " Andreas Naumann
2020-03-11 22:06   ` Thomas Petazzoni
2020-02-17 21:23 ` [Buildroot] [PATCH v6 17/28] core/pkg-infra: Fix package file statistics for parallel build Andreas Naumann
2020-03-11 22:07   ` Thomas Petazzoni [this message]
2020-03-16 21:54     ` Andreas Naumann
2020-02-17 21:23 ` [Buildroot] [PATCH v6 18/28] package/cmake: Fix for using -O in top-level make Andreas Naumann
2020-03-11 22:25   ` Thomas Petazzoni
2020-03-12 13:00     ` Arnout Vandecappelle
2020-03-12 13:22       ` Thomas Petazzoni
2020-03-16 22:15       ` Andreas Naumann
2020-03-17  6:08         ` Thomas Petazzoni
2020-03-17 19:02           ` Asaf Kahlon
2020-03-18 15:35             ` Andreas Naumann
2020-03-18 18:59               ` Asaf Kahlon
2020-03-18 21:03                 ` Andreas Naumann
2020-03-19 11:40                   ` Asaf Kahlon
2020-03-19 12:49                     ` Thomas Petazzoni
2020-03-19 13:53                       ` Asaf Kahlon
2020-03-19 14:00                         ` Thomas Petazzoni
2020-03-22 20:46                           ` Andreas Naumann
2020-02-17 21:23 ` [Buildroot] [PATCH v6 19/28] core/pkg-infra: Add per-package support in qmake infra Andreas Naumann
2020-04-10 20:25   ` Yann E. MORIN
2020-07-01 19:01   ` Yann E. MORIN
2020-07-20  6:14   ` Peter Korsgaard
2020-02-17 21:23 ` [Buildroot] [PATCH v6 20/28] qt5: Fix pkgconfig search path for per-package infrastructure Andreas Naumann
2020-06-19 10:22   ` Louis-Paul CORDIER
2020-07-01 21:15     ` Yann E. MORIN
2020-07-01 21:54       ` Yann E. MORIN
2020-07-02 17:38         ` Yann E. MORIN
2020-07-22 19:48         ` LP C
2020-07-22 20:26           ` Yann E. MORIN
2020-07-23 10:29             ` Andreas Naumann
2020-07-23 20:17               ` Yann E. MORIN
2020-07-24  7:17                 ` Andreas Naumann
2020-02-17 21:23 ` [Buildroot] [PATCH v6 21/28] package/python-sip: Fix qmake usage for per-package build Andreas Naumann
2020-03-11 22:30   ` Thomas Petazzoni
2020-03-16 22:32     ` Andreas Naumann
2020-07-01 21:56   ` Yann E. MORIN
2020-02-17 21:23 ` [Buildroot] [PATCH v6 22/28] package/python-pyqt5: " Andreas Naumann
2020-02-17 21:23 ` [Buildroot] [PATCH v6 23/28] package/qt5webengine: add libXdamage as XCB dependency Andreas Naumann
2020-03-12  8:15   ` Thomas Petazzoni
2020-03-16 22:40     ` Andreas Naumann
2020-03-22 21:58       ` Andreas Naumann
2020-03-22 22:03         ` Thomas Petazzoni
2020-02-17 21:23 ` [Buildroot] [PATCH v6 24/28] package/qt5webengine: Add optional dependency on qt5quickcontrols2 Andreas Naumann
2020-02-17 21:23 ` [Buildroot] [PATCH v6 25/28] package/qt5webengine: Add optional dependency on qt5location Andreas Naumann
2020-02-17 21:23 ` [Buildroot] [PATCH v6 26/28] package/qt5virtualkeyboard: Add optional dependency on qt5quickcontrols2 Andreas Naumann
2020-02-17 21:23 ` [Buildroot] [PATCH v6 27/28] package/qt53d: Add optional dependency on qt5multimedia Andreas Naumann
2020-02-17 21:23 ` [Buildroot] [PATCH v6 28/28] package/qt5charts: " Andreas Naumann
2020-02-18 21:33 ` [Buildroot] [PATCH v6 00/28] Qt5 qmake infra and per-package compatibility Peter Seiderer
2020-02-19 14:46   ` Andreas Naumann

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200311230703.37bf0b63@windsurf \
    --to=thomas.petazzoni@bootlin.com \
    --cc=buildroot@busybox.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.