Re: [RFC] Support of non-indirect grant backend on 64KB guest

[Julien Grall <julien.grall@citrix.com>]

On 19/08/2015 01:58, Jan Beulich wrote:
>>>> On 18.08.15 at 20:45, <julien.grall@citrix.com> wrote:
>> Hi Roger,
>>
>> On 18/08/2015 00:09, Roger Pau Monné wrote:
>>> Hello,
>>>
>>> El 18/08/15 a les 8.29, Julien Grall ha escrit:
>>>> Hi,
>>>>
>>>> Firstly, this patch is not ready at all and mostly here for collecting
>> comment about the way to do it. It's not clean so no need to complain about
>> the coding style.
>>>>
>>>> The qdisk backend in QEMU is not supporting indirect grant, this is means
>> that a request can only support 11 * 4KB = 44KB.
>>>>
>>>> When using 64KB page, a Linux block request (struct *request) may contain up
>> to 64KB of data. This is because the block segment size must at least be the
>> size of a Linux page.
>>>>
>>>> So when indirect is not supported by the backend, we are not able to fitall
>> the data in a single request. We therefore need to create a second request to
>> copy the rest of the data.
>>>>
>>>> I've wrote a patch last week which make 64KB guest booting with qdisk.
>> Although, I'm not sure this is the right way to do it. I would appreciate
>> ifone of the block maintainers give me insight about it.
>>>
>>> Maybe I'm missing some key data, but I see two ways to solve this, the
>>> first one is the one you describe above, and consists in allowing
>>> blkfront to split a request into multiple ring slots. The other solution
>>> would be to add indirect descriptors support to Qdisk, has this been
>>> looked into?
>>>
>>> AFAICT it looks more interesting, and x86 can also benefit from it.
>>> Since I would like to prevent adding more cruft to blkfront, I rather
>>> prefer 64KB guests to require indirect descriptors in order to run.
>>
>> Actually supporting indirect in Qdisk was one of our idea. While I agree
>> this is a good improvement in general we put aside this idea for various
>> reasons.
>>
>> The first one is openStack is using by default Qdisk backend, so Linux
>> 64KB guest wouldn't be able to boot on current version of Xen. This is
>> the only blocker in order use 64KB guests, everything else is working.
>> Having the indirect grant support in QEMU for Xen 4.6 is not realistic,
>> there is only a month left and we are already in feature.
>>
>> That would mean that any new distribution using Linux 64KB would not
>> work out-of-box on Xen.
>>
>> Furthermore, not supporting non-indirect grant in the frontend means
>> that any userspace backend won't be supported for Linux 64KB guests.
>>
>> Overall, I think we have to support non-indirect with Linux 64KB guests.
>> Many (but not all) distribution will only support 64KB pages, so we
>> can't wait until Xen 4.7 to get something running. Not that I rule out
>> the requirement for the user to upgrade the QEMU version in order to run
>> 64KB guests.
>
> To be honest, none of this really reads like a good reason for not
> following Roger's suggestion. All it points out that there is a new
> feature that's not fully cooked yet. Distros wanting to support such
> guests should be willing to either backport the necessary qemu
> patch(es) or update qemu. Uglifying blkfront (via other than an
> experimental, out of tree patch) doesn't look like a good idea to me.

With your suggested approach, modifiying QEMU, you would have to push a 
patch in DOM0 distributions (which may be different as the guests) in 
order to boot a such guests. On the other hand, patching Linux, will 
make a guest booting out-of-box without having the cloud provider using 
aarch64 platform fixing their DOM0 in order to boot a such a guest.

> And then, if blkfront was to be made capable, it would seem to me
> that the better route would be to have it use its native page size for
> the blkif instantiation, and extend the blkif protocol so the frontend
> can communicate its page size to the backend. The current
> backend-page-size == frontend-page-size restriction would then
> become <= (or maybe could even go away altogether).

And then we could also improve memory performance right now rather than 
having a first version upstream... if we take this approach, it will 
take another year to upstream the 64KB guest support in Linux.
Having a fully working, high-performing 64KB guest support is huge and 
we can't do everything at one time.

Having 64KB grant is my next plan but it requires changes in both Linux 
and Xen. Although, this can't be done in a short timeline and we need to 
have a first approach done today.

Major distributions will be shipped with 64KB page supports only 
(including the one from your company Suse), they are targeting Linux 
4.2/4.3 and are willing to take patches in their kernel.

Now, if we decide to not support non-indirect grant for 64KB guests, we 
also have to reach distributions using 4KB pages in order to update 
their QEMU.

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel