Buildroot Archive on lore.kernel.org
 help / color / mirror / Atom feed
* [Buildroot] [PATCH v2 0/1] core: Use of percent_defconfig seems to impact performance
@ 2023-01-05  1:57 Nevo Hed via buildroot
  2023-01-05  1:57 ` [Buildroot] [PATCH 1/1] " Nevo Hed via buildroot
  0 siblings, 1 reply; 6+ messages in thread
From: Nevo Hed via buildroot @ 2023-01-05  1:57 UTC (permalink / raw)
  To: buildroot; +Cc: Nevo Hed, yann.morin.1998, thomas.petazzoni

Noticing serious performance issues on `make X_defconfig` with growth of

number of external trees.  At 7 trees it is somewhat tolerable but at 8
trees we have seen it take 80-110 seconds.

Numbers for before and after the change in commit itself.

I do not know for sure what the underlying issue is but looking at some
`strace` and `make -d` output it feels like implosion of implicit rules
being evaluated.

Note that after I submitted v1 of this patch it was pointed out to me that
a different patch is already in the works.  But since that neither that
patch nor mine had any seen any movement in months I decided to take a
stab at addressing the feedback I received from Yann on my patch:

> ... it defers the test that the defconfig exists into the shell
> rather than the Makefile

Adding the resets as suggested here
https://lists.gnu.org/archive/html/help-make/2002-11/msg00062.html seem to
help a lot but not as much as the attached patch.

--8<------------
%: %,v
%: RCS/%,v
%: RCS/%
%: s.%
%: SCCS/s.%
--8<------------

Inlined below a script to repro this issue, not sure if this functionality
should be integrated anywhere or just left in the mailing list.  By
default the script runs the scenario from 1 tree to 8 trees.  Running with
param (1-10) will allow a single run with that many trees.


--8<------------
#!/usr/bin/env bash

declare treedirs=()

declare -r cfg="pc_x86_64_bios_defconfig"

function setup_dir {
    for tree in tree{1..10}; do
        local treedir="defconf_test/${tree}"

        mkdir -p "${treedir}/configs"
        touch "${treedir}/Config.in"
        touch "${treedir}/external.mk"
        echo -e "name: ${tree^^}\ndesc: ${tree^}" > "${treedir}/external.desc"

        cp "./configs/${cfg}" "${treedir}/configs/${tree}_${cfg}"

        treedirs+=( "${treedir}" )
    done
}

function run_one {
    local count="${1}"
    local br2_external=$(IFS=':'; echo "${treedirs[*]:0:count}")
    echo "Defconfig with ${count} trees (BR2_EXTERNAL=\"${br2_external}\")"
    /usr/bin/time -f "${count},%e" /bin/make BR2_EXTERNAL="${br2_external}" "${cfg}"
}

# Main
setup_dir
if [ -n "${1}" ]; then
    run_one "${1}"
else
    for i in {1..8}; do
        run_one $i
    done
fi
--8<------------

Changes v1 -> v2:
  - Testing for defconfig existence and error when failing to find it is
    now done at the Makefile level rather than the shell (utilizing `$(or
    ...` and `$(error ...`).

Nevo Hed (1):
  core: Use of percent_defconfig seems to impact performance

 Makefile | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)

-- 
2.37.3

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 6+ messages in thread
end of thread, other threads:[~2023-01-22  9:13 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-01-05  1:57 [Buildroot] [PATCH v2 0/1] core: Use of percent_defconfig seems to impact performance Nevo Hed via buildroot
2023-01-05  1:57 ` [Buildroot] [PATCH 1/1] " Nevo Hed via buildroot
2023-01-07 20:15   ` Yann E. MORIN
2023-01-12 10:20     ` Peter Korsgaard
2023-01-21 22:27   ` Yann E. MORIN
2023-01-22  9:13     ` Yann E. MORIN

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox