Re: [Buildroot] [PATCH] utils/genrandconfig: reduce the maximum "size" of random configurations

["Yann E. MORIN" <yann.morin.1998@free.fr>]

Thomas, All,

On 2021-11-13 15:17 +0100, Thomas Petazzoni spake thusly:
> genrandconfig is used by the Buildroot autobuilders to generate
> semi-random configurations that we build test. As part of this, we use
> "make randpackageconfig" to randomize the selection of packages,
> together with a KCONFIG_PROBABILITY value, which indicates the
> probabibility for each option to be enabled. This probability is
> itself randomized, between 1% and 30% for every build.
> 
> However, with our increasing number of packages (over 2900), when we
> use a 30% probability for options to be enabled, it means a *lot* of
> options are enabled, causing very large configurations to be
> tested. These configurations are not very realistic, and they take
> ages to build on our autobuilders: we have builds that take 4, 5 or
> even 7 hours to build.
> 
> In order to test a larger number of configurations and therefore a
> larger variety of configurations, this commit reduces the maximum
> probability to 20%.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> ---
>  utils/genrandconfig | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/utils/genrandconfig b/utils/genrandconfig
> index 4fffcbad11..77c7e585f5 100755
> --- a/utils/genrandconfig
> +++ b/utils/genrandconfig
> @@ -410,7 +410,7 @@ def gen_config(args):
>              return 1
>          bounded_loop -= 1
>          subprocess.check_call(["make", "O=%s" % args.outputdir, "-C", args.buildrootdir,
> -                               "KCONFIG_PROBABILITY=%d" % randint(1, 30),
> +                               "KCONFIG_PROBABILITY=%d" % randint(1, 20),

On IRC, we discussed about using a normal distrobution rather than the
uniform distribution that randint() provides.

So, I've done a few tests with 100M iterations, adn for example, here is
the (15,5) gaussian [0]:

     1: #    0%
     2: #    0%
     3: ##    0%
     4: ####    0%
     5: ######    1%
     6: #########    1%
     7: #############    2%
     8: #################    3%
     9: #####################    4%
    10: ##########################    5%
    11: ###############################    6%
    12: ###################################    7%
    13: ######################################    7%
    14: #######################################    7%
    15: #######################################    7%
    16: ######################################    7%
    17: ###################################    7%
    18: ###############################    6%
    19: ##########################    5%
    20: #####################    4%
    21: #################    3%
    22: #############    2%
    23: #########    1%
    24: ######    1%
    25: ####    0%
    26: ##    0%
    27: #    0%
    28: #    0%
    29:     0%
    30:     0%

And here is the (10,7) gaussian (which has my preference):

     1: ###############    3%
     2: #################    3%
     3: ####################    4%
     4: #######################    4%
     5: #########################    5%
     6: ###########################    5%
     7: #############################    5%
     8: ##############################    6%
     9: ###############################    6%
    10: ###############################    6%
    11: ##############################    6%
    12: #############################    5%
    13: ###########################    5%
    14: #########################    5%
    15: #######################    4%
    16: ####################    4%
    17: #################    3%
    18: ###############    3%
    19: ############    2%
    20: ##########    2%
    21: ########    1%
    22: ######    1%
    23: ####    0%
    24: ###    0%
    25: ##    0%
    26: #    0%
    27: #    0%
    28:     0%
    29:     0%
    30:     0%

Since a gaussian can output values all over ℝ, we must limit the range
explicitly (note that the 0% above are not representing 0 occurences!):

    proba = 0
    while proba < 1 or proba > 100:
        proba = int(random.gauss(15, 5))

What do you think?

Regards,
Yann E. MORIN.

[0] Here's the ugly script to reprodce the gaussians; adapt for other
paramaters...

    #!/usr/bin/env python3
    import random

    iters = 100000  # In units of 1k

    def main():
        buckets = dict()
        for i in range(1, 31):
            buckets[i] = 0
        for i in range(1000*iters):
            r = 0
            while r < 1 or r > 30:  # 30 for the bucket size
                r = int(random.gauss(15, 5))
            buckets[r] = buckets[r]+1
        for i in range(1, 31):
            print('{:2}: '.format(i), end='')
            for j in range(int(buckets[i]/(2*iters))):
                print('#', end='')
            print('   {: 2}%'.format(int(buckets[i]/(10*iters))))

    if __name__ == '__main__':
        main()

>                                 "randpackageconfig"])
>  
>          if fixup_config(sysinfo, configfile):
> -- 
> 2.31.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot