Re: [PATCH 4/4] xen/arm: update the docs about heterogeneous computing

[Julien Grall <julien.grall@arm.com>]

On 16/02/2018 21:15, Stefano Stabellini wrote:
> On Fri, 16 Feb 2018, Julien Grall wrote:
>> On 16/02/2018 20:50, Stefano Stabellini wrote:
>>> On Fri, 16 Feb 2018, Julien Grall wrote:
>>>> Hi Stefano,
>>>>
>>>> On 15/02/18 23:17, Stefano Stabellini wrote:
>>>>> Update the documentation of the hmp-unsafe option to explain how to use
>>>>> it safely, together with the right cpu affinity setting, on big.LITTLE
>>>>> systems.
>>>>>
>>>>> Also update the warning message to point users to the docs.
>>>>>
>>>>> Signed-off-by: Stefano Stabellini <sstabellini@kernel.org>
>>>>> CC: jbeulich@suse.com
>>>>> CC: konrad.wilk@oracle.com
>>>>> CC: tim@xen.org
>>>>> CC: wei.liu2@citrix.com
>>>>> CC: andrew.cooper3@citrix.com
>>>>> CC: George.Dunlap@eu.citrix.com
>>>>> CC: ian.jackson@eu.citrix.com
>>>>>
>>>>> ---
>>>>>     docs/misc/xen-command-line.markdown | 10 +++++++++-
>>>>>     xen/arch/arm/smpboot.c              |  9 +++++----
>>>>>     2 files changed, 14 insertions(+), 5 deletions(-)
>>>>>
>>>>> diff --git a/docs/misc/xen-command-line.markdown
>>>>> b/docs/misc/xen-command-line.markdown
>>>>> index 2184cb9..a1ebeea 100644
>>>>> --- a/docs/misc/xen-command-line.markdown
>>>>> +++ b/docs/misc/xen-command-line.markdown
>>>>> @@ -1007,7 +1007,15 @@ Control Xens use of the APEI Hardware Error
>>>>> Source
>>>>> Table, should one be found.
>>>>>       Say yes at your own risk if you want to enable heterogenous
>>>>> computing
>>>>>     (such as big.LITTLE). This may result to an unstable and insecure
>>>>> -platform. When the option is disabled (default), CPUs that are not
>>>>> +platform, unless you manually specify the cpu affinity of all domains
>>>>> so
>>>>> +that all vcpus are scheduled on the same class of pcpus (big or LITTLE
>>>>> +but not both). vcpu migration between big cores and LITTLE cores is not
>>>>> +supported. Thus, if the first 4 pcpus are big and the last 4 are
>>>>> LITTLE,
>>>>> +all domains need to have either cpus = "0-3" or cpus = "4-7" in their
>>>>> VM
>>>>> +config. Moreover, dom0_vcpus_pin needs to be passed on the Xen command
>>>>> +line.
>>>>
>>>> In your example here you suggest to have all the vCPUs of a guest to
>>>> either on
>>>> big or LITTLE cores. How about giving an example where the guest can have
>>>> 2
>>>> LITTLE vCPUs and one big vCPU?
>>>
>>> I would rather discourage it at the moment, given that it requires more
>>> complex cpu affinity settings, or vcpu pinning. Also, I am afraid that
>>> without matching corresponding topology information on the guest device
>>> tree, guests might not work as expected in such a scenario.
>>>
>>> What do you think?
>>
>> You already know my view on this. I would rather strongly discourage anyone
>> pinning all vCPUs of a domain to big cores. We should avoid to provide
>> shortcuts to use that could have potentially damageable impact on their
>> platform without telling them.
> 
> Do you have a link to a doc somewhere that provides more details about
> this? We could add a link to it here to inform users. It would be
> useful.

This is quite well described in 
https://xenbits.xen.org/docs/unstable/man/xl.cfg.5.html#CPU-Allocation 
see "cpus".

> 
> 
>>> I see. What about:
>>>
>>>     See docs/misc/xen-command-line.markdown:hmp-unsafe
>>>
>>> Or:
>>>
>>>     See hmp-unsafe under
>>> https://xenbits.xen.org/docs/unstable/misc/xen-command-line.html
>>
>> I believe that people looking at big.LITTLE (and really want it) are smart
>> enough to look at the docs or the code themselves. Given how fragile is your
>> solution, I would rather avoid to help people doing bad thing.
> 
> People might know very well the hardware, and have very detailed
> information about big.LITTLE and their platform, but might not know that
> much about Xen.

We also might have student who wants to try Xen on their platform and 
have no clue how it works. So we want to provide safe information here 
and avoid to induce pinning on big cores is always safe.

I think with the tools we have, it is not very difficult provide a quick 
tuto how to do big.LITTLE in Xen. Obviously this will not be as easy as 
the solution suggested by Dario in its design document.

The most important bits is libxl provide an extensive way to set CPUs 
affinity. OS such as Linux will be able to deal with big.LITTLE even 
without DT thought scheduling might not be that good... I have no idea 
how Android detects big.LITTLE and will leave that to person who knows more.

> 
> I would like to introduce a tie between the warning message and the
> documentation provided. The tie doesn't have to be around hmp-unsafe:
> the intention wasn't really to provide a shortcut to do more damage, but
> rather to inform about the situation. In fact, I didn't mean to
> recommend the usage of hmp-unsafe.

In that case we should make clear hmp-unsafe is not recommended. When I 
read the suggested paragraph for the command line option, it feels that 
it is safe to use it if you pin your domain on big cores only.

> 
> Maybe we could say:
> 
>    See big.LITTLE under https://xenbits.xen.org/docs/unstable/misc/xen-command-line.html
> 
> or create docs/mics/arm/big.LITTLE ?

A doc would make more sense over a long paragraph in the xen-command-line.

But if we decide to no recommend the usage of hmp-unsafe (as you 
suggested above), then we should avoid to tease the user with that. 
Anyone caring enough about big.LITTLE could find easily a documentation 
to enable it on Xen.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel