Re: [PATCH] kernel/isolation: Asset that a housekeeping CPU comes up at boot time

Nicholas Piggin
2019-06-25T00:06:22Z

Qais Yousef's on June 24, 2019 8:57 pm:
> On 06/01/19 21:39, Nicholas Piggin wrote:
>> With the change to allow the boot CPU0 to be isolated, it is possible
>> to specify command line options that result in no housekeeping CPU
>> online at boot.
>>
>> An 8 CPU system booted with "nohz_full=0-6 maxcpus=4", for example.
>>
>> It is not easily possible at housekeeping init time to know all the
>> various SMP options that will result in an invalid configuration, so
>> this patch adds a sanity check after SMP init, to ensure that a
>> housekeeping CPU has been onlined.
>>
>> The panic is undesirable, but it's better than the alternative of an
>> obscure non deterministic failure. The panic will reliably happen
>> when advanced parameters are used incorrectly.
>>
>> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>> ---
>> kernel/sched/isolation.c | 23 +++++++++++++++++++++++
>> 1 file changed, 23 insertions(+)
>>
>> diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
>> index 123ea07a3f3b..7b9e1e0d4ec3 100644
>> --- a/kernel/sched/isolation.c
>> +++ b/kernel/sched/isolation.c
>> @@ -63,6 +63,29 @@ void __init housekeeping_init(void)
>> WARN_ON_ONCE(cpumask_empty(housekeeping_mask));
>> }
>>
>> +static int __init housekeeping_verify_smp(void)
>> +{
>> + int cpu;
>> +
>> + /*
>> + * Early housekeeping setup is done before CPUs come up, and there are
>> + * a range of options scattered around that can restrict which CPUs
>> + * come up. It is possible to pass in a combination of housekeeping
>> + * and SMP arguments that result in housekeeping assigned to an
>> + * offline CPU.
>> + *
>> + * Check that condition here after SMP comes up, and give a useful
>> + * error message rather than an obscure non deterministic crash or
>> + * hang later.
>> + */
>> + for_each_online_cpu(cpu) {
>> + if (cpumask_test_cpu(cpu, housekeeping_mask))
>> + return 0;
>> + }
>> + panic("Housekeeping: nohz_full= or isolcpus= resulted in no online CPUs for housekeeping.\n");
>
> I am hitting this panic when I boot my juno board.
>
>
> I have CONFIG_CPU_ISOLATION=y but I don't pass nohuz_full nor isolcpus in the
> commandline. I think what's going on is that housekeeping_setup() doesn't get
> called and hence housekeeping_mask isn't initialized in my case, causing this
> check to fail and trigger the panic.
>
> The below seems to 'fix' it though not sure if it's the right way forward.
> A revert obviously fixes it too but I doubt we want that :-)

That'll do it. Thanks for the report and investigation.

>
>
> diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
> index 7b9e1e0d4ec3..a9ca8628c1a2 100644
> --- a/kernel/sched/isolation.c
> +++ b/kernel/sched/isolation.c
> @@ -67,6 +67,9 @@ static int __init housekeeping_verify_smp(void)
> {
> int cpu;
>
> + if (!housekeeping_flags)
> + return 0;
> +
> /*
> * Early housekeeping setup is done before CPUs come up, and there are
> * a range of options scattered around that can restrict which CPUs
>
>
> Cheers
>
> --
> Qais Yousef
>
>
>> +}
>> +core_initcall(housekeeping_verify_smp);
>> +
>> static int __init housekeeping_setup(char *str, enum hk_flags flags)
>> {
>> cpumask_var_t non_housekeeping_mask;
>> --
>> 2.20.1
>>
>