Re: [Qemu-devel] [RFC PATCH RDMA support v5: 03/12] comprehensive protocol documentation

["Michael R. Hines" <mrhines@linux.vnet.ibm.com>]

On 04/11/2013 11:44 AM, Michael S. Tsirkin wrote:
> On Thu, Apr 11, 2013 at 11:18:56AM -0400, Michael R. Hines wrote:
>> First of all,
> I know it's a hard habit to break but could you
> please stop stop top-posting?
>
>> this whole argument should not even exist for the
>> following reason:
>>
>> Page registrations are supposed to be *rare* - once a page is registered, it
>> is registered for life. There is nothing in the design that says a page must
>> be "unregistered" and I do not believe anybody is proposing that.
> Hmm proposing what? Of course you need to unregister pages
> eventually otherwise your pinned memory on the destination
> will just grow indefinitely. People are often doing
> registration caches to help reduce the overhead,
> but never unregistering seems too aggressive.
>
> You mean the chunk-based thing just delays the agony
> until all guest memory is pinned for RDMA anyway?
> Wait, is it registered for life on the source too?
>
> Well this kind of explains why qemu was dying on OOM,
> doesn't it?
>
>> Second, this means that my previous analysis showing that
>> performance was reduced
>> was also incorrect because most of the RDMA transfers were against
>> pages during
>> the bulk phase round, which incorrectly makes dynamic page
>> registration look bad.
>> I should have done more testing *after* the bulk phase round,
>> and I apologize for not doing that.
>>
>> Indeed when I do such a test (with the 'stress' command) the cost of
>> page registration disappears
>> because most of the registrations have already completed a long time ago.
>>
>> Thanks, Paolo for reminding us about the bulk-phase behavior to being with.
>>
>> Third, this means that optimizing this protocol would not be helpful
>> and that we should
>> follow the "keep it simple" approach because during steady-state
>> phase of the migration
>> most of the pages should have already been registered.
>>
>> - Michael
>
> But if you mean to say that the current chunk based code
> is useless, then I'd have to agree.
>
Well, you asked me to write an overcommit solution, so I wrote one. =)

Second, there is *no need* for a fast registration protocol, as I've 
summarized,
because most of the page registrations are supposed have already completed
before the steady-state iterative phase of the migration already begins
(which will be further optimized in a later patch). You're complaining 
about a non-issue.

- Michael