Re: Question on skip_emulated_instructions()

[Yoshiaki Tamura <tamura.yoshiaki@lab.ntt.co.jp>]

Avi Kivity wrote:
 > On 04/08/2010 11:30 AM, Yoshiaki Tamura wrote:
 >>
 >> If I transferred a VM after I/O operations, let's say the VM sent an
 >> TCP ACK to the client, and if a hardware failure occurred to the
 >> primary during the VM transferring *but the client received the TCP
 >> ACK*, the secondary will resume from the previous state, and it may
 >> need to receive some data from the client. However, because the client
 >> has already receiver TCP　ACK, it won't resend the data to the
 >> secondary. It looks this data is going to be dropped. Am I missing
 >> some point here?
 >>
 >
 > I think you should block I/O not at the cpu/device boundary (that's
 > inefficient as many cpu I/O instructions don't necessarily cause
 > externally visible I/O) but at the device level. Whenever the network
 > device wants to send out a packet, halt the guest (letting any I/O
 > instructions complete), synchronize the secondary, and then release the
 > pending I/O. This ensures that the secondary has all of the data prior
 > to the ack being sent out.

Although I was thinking to clean up my current code, maybe I should post the 
current status for explanation now.  As you mentioned, I'm capturing I/O at the 
device level, by inserting a hook inside of PIO/MMIO handler in virtio-blk, 
virtio-net and e1000 emulator.  Since it's implemented naively, it'll stop 
(meaning I/O instructions will be delayed) until transferring the VM is done.
So what I can do here is,

1. Let I/O instructions to complete both at qemu and kvm.
2. Transfer the guest state.
# VCPU and device model thinks I/O emulation is already done.
3. Finally release the pending output to the real world.

>>>> If the responses to the mmio or pio request are exactly the same,
>>>> then the replay will happen exactly the same.
>>
>>
>> I agree. What I'm wondering is how can we guarantee that the responses
>> are the same...
>
> I don't think you can in the general case. But if you gate output at the
> device level, instead of the instruction level, the problem goes away, no?

Yes, it should.
To implement this, we need to make No.3 to be called asynchronously.  If qemu is 
already handling I/O asynchronously, it would be relatively easy to make this.