Re: [PATCH] qemu: qemu_fopen_fd: differentiate between reader and writer user

[Anthony Liguori <anthony@codemonkey.ws>]

Uri Lublin wrote:
> Anthony Liguori wrote:
>> Uri Lublin wrote:
>>>
>>> That is true, but in the case I mentioned above it would take the 
>>> management tool some time (guest down time) to realize what happens, 
>>> and to send "cont" to the SRC. With end-of-migration messages SRC 
>>> discovers DST fails and immediately continues.
>>> I agree those messages add some complexity, and slow things a bit 
>>> for the good/average case.
>>
>> It's the classic general's dilemma.  If SRC waits for DST to send an 
>> ACK, DST still doesn't know whether SRC received the ACK so it 
>> doesn't know whether it's truly safe to continue.
>>
>> This is why migration doesn't quit SRC immediately, and leaves SRC in 
>> the stopped state.  It's because the only safe way to handle this is 
>> with a third party that is reliable.
>>
>
> In the scenario above (with ACK/GO messages), SRC _does_ know that DST 
> have failed (as it does not receive ACK). With ACK/GO messages we only 
> need third party involvement to handle a scenario where GO does not 
> reach DST. Without ACK/GO messages we need third party involvement for 
> almost any state-load function failure. In other words the 
> risk/exposure is smaller with ACK/GO messages.

I think this is a scenario where we have to be careful about layering in 
the design.  The core migration protocol is a mechanism.  The goal is to 
not implement policy.  Having an exchange of ack/go messages may 
increase reliability but they don't do so in a perfect way.  How many 
times you go back and forth therefore becomes a policy which is based on 
how important reliability is to you trading off latency.  If you have a 
high latency network, the round trip cost of an ack/go message may 
introduce unwanted latency (which translates to VM downtime).  Moreover, 
if you have a third party orchestrating everything, it's totally 
unnecessary downtime.

This is not to say there is no place for QEMU to support policies.  They 
should be layered in such a way that they don't burden everyone though.  
The idea behind using migration protocols is to help facilitate this.

I think the tcp: protocol should remain a pure migration-over-tcp 
transport.  I think there is room for implementing another migration 
protocol that was maybe geared toward more average users.  An ack/go 
message may be appropriate for this.  I really think it should also have 
a daemon associated with it that could automatically spawn QEMU 
instances.  I've always felt the ssh: protocol should provide this but 
it proved less popular than I expected it to be.

Anyway, my point is that if you want an ack/go message, you should 
encapsulate the existing protocol within another protocol (that has it's 
own versioning) and introduce a new transport.


> Since in both cases we must have a third party involvement in the 
> worst case, and since on the good/normal case those messages slow down 
> the migration process a bit (and complicate the code a bit), I do not 
> mind dropping those messages. I just wanted to make sure we all 
> understand their benefit. We can always add them later if we'll "miss" 
> them (if we'll find out they are more useful then we think now).
>
> In any case, we need to think of a way to get the migration status on 
> the destination. A minimum is to term_printf a message specifying that 
> status.

What's the use case for this?  In what circumstances would you have no 
idea of what was happening on DST such that you'd need to get this from 
the SRC?  The problem with the old migration code is that while there 
were a lot of error status, in practice, there was only one or two that 
would ever happen.

Regards,

Anthony Liguori