[Qemu-devel] Pending Disk I/O requests during live VM migration

[Qemu-devel] Pending Disk I/O requests during live VM migration

[Kaushik Bhandankar]

Hello

I am trying to implement functionality in fully-virtualized xen wherein 
the responses to the pending disk I/O requests during live VM migration 
get trasnsferred to the new VMM form the old VMM. Specifically, I am 
looking at IDE disk (tools/ioemu/hw/ide.c)

Basically, ide.c:pci_ide_save() saves the state of the IDE disk in a 
QEMUFile and this file is sent over the network (can somebody point me 
to the code where this happens ??)) so that the new VMM (where the VM 
has migrated) invokes ide.c:pci_ide_load() to retrieve the IDE Disk 
contents form the file. As of now, the pending disk I/O requests do not 
get saved in this file so these pending disk I/O requests are not 
available when executing pci_ide_load().

I am still trying to figure out where the pending Disk I/O requests get 
stored in the VMM so that during live VM migration, these requests can 
be put in the QEMUFile (As mentioned above) so that they can be reissued 
by the new VMM. Much better would be to send the responses to these 
pending disk I/O requests from the old to new VMM. Can somebody help me 
out with this as I am newbie here....??

register_savevm() in ioemu/vl.c is simply used to register the save & 
load handlers......... register_savevm() is called in 
ioemu/hw/ide.c:pci_piix_ide_init() to register pci_ide_save() and 
pci_ide_load() as the save & load handlers for IDE disks....... 
qemu_savevm() in ioemu/vl.c walks the list of the registered save 
handlers whereas qemu_loadvm() in vl.c expects a load handler to have 
been registered for the corresponding save handlers for each chunk of 
the save file.

Any help in locating the code would be greatly appreciated.

-Kaushik

Re: [Qemu-devel] Pending Disk I/O requests during live VM migration

[Blue Swirl]

On 10/4/07, Kaushik Bhandankar <kaushikb@cc.gatech.edu> wrote:
> I am trying to implement functionality in fully-virtualized xen wherein
> the responses to the pending disk I/O requests during live VM migration
> get trasnsferred to the new VMM form the old VMM. Specifically, I am
> looking at IDE disk (tools/ioemu/hw/ide.c)

You should ask Xen related questions in xen lists. At least in Qemu
pending I/O are flushed before VM save in vl.c:5522.

Re: [Qemu-devel] Pending Disk I/O requests during live VM migration

[Kaushik Bhandankar]

Blue Swirl wrote:
> On 10/4/07, Kaushik Bhandankar <kaushikb@cc.gatech.edu> wrote:
>   
>> I am trying to implement functionality in fully-virtualized xen wherein
>> the responses to the pending disk I/O requests during live VM migration
>> get trasnsferred to the new VMM form the old VMM. Specifically, I am
>> looking at IDE disk (tools/ioemu/hw/ide.c)
>>     
>
> You should ask Xen related questions in xen lists. At least in Qemu
> pending I/O are flushed before VM save in vl.c:5522.
>   
Thanks Blue...

somebody in the xen-devel list suggested to mail this list about it.... 
Anyway, I have some IDE related questions which somebody on this list 
might answer

1) I guess the IDE disk code is in ioemu/hw/ide.c. But where is the IDE 
disk controller code located ??
2) I do not really understand BMDMA in ioemu/hw/ide.c. google doesnt 
seem to be helping much here. Any documentation about the QEMU IDE disks 
would be greatly appreciated.
3) vl.c:5522 seems to be not the correct code (I guess our vl.c codes 
are different).. Could you point me to the routine in vl.c where the 
pending disk IO requests are flushed before VM save (I guess the VM save 
happens in qemu_savevm_state() in vl.c wherein it invokes all the 
registered save handlers)?
4) Somebody told me that the pending disk IO requests are stored 
somewhere in quemu's I/O disk model but I am not sure exactly 
where....Can somebody help me with this ?

Thanks,
Kaushik

Re: [Qemu-devel] Pending Disk I/O requests during live VM migration

[Blue Swirl]

On 10/4/07, Kaushik Bhandankar <kaushikb@cc.gatech.edu> wrote:
> 1) I guess the IDE disk code is in ioemu/hw/ide.c. But where is the IDE
> disk controller code located ??

Same file, see ide_ioport_write().

> 2) I do not really understand BMDMA in ioemu/hw/ide.c. google doesnt
> seem to be helping much here. Any documentation about the QEMU IDE disks
> would be greatly appreciated.

No idea.

> 3) vl.c:5522 seems to be not the correct code (I guess our vl.c codes
> are different).. Could you point me to the routine in vl.c where the
> pending disk IO requests are flushed before VM save (I guess the VM save
> happens in qemu_savevm_state() in vl.c wherein it invokes all the
> registered save handlers)?

It's in:
void do_savevm(const char *name)

> 4) Somebody told me that the pending disk IO requests are stored
> somewhere in quemu's I/O disk model but I am not sure exactly
> where....Can somebody help me with this ?

Qemu uses asynchronous IO, grepping aio or AIO should find something.

Re: [Qemu-devel] Pending Disk I/O requests during live VM migration

[Kaushik Bhandankar]

In ioemu/vl.c: do_savevm(), before saving the current VM state for live 
VM migration (qemu_savevm_state()), it calls qemu_aio_flush() in 
ioemu/block-raw.c. Now qemu_aio_flush() internally calls qemu_aio_poll() 
where we loop through the linked list of RawAIOCB structures and remove 
the request (qemu_aio_release()) only if the request is CANCELED or NOT 
IN PROGRESS.

I am not really sure then why ioemu/hw/ide.c:pci_ide_save() has a 
comment saying "If a transfer is pending, we do not save it yet". 
Looking at qemu_aio_poll() doesn't seem to indicate so...

Also, looking at /usr/include/aio.h sees to suggest that the pending 
requests get stored in the aio_buf of "struct aiocb" structure. Do the 
responses to these requests go in the same buffer (in which case the 
aio_buf and aio_nbytes would get overwritten by the response) ??

any help in this regard would be helful.

-Kaushik

Blue Swirl wrote:
> On 10/4/07, Kaushik Bhandankar <kaushikb@cc.gatech.edu> wrote:
>   
>> 1) I guess the IDE disk code is in ioemu/hw/ide.c. But where is the IDE
>> disk controller code located ??
>>     
>
> Same file, see ide_ioport_write().
>
>   
>> 2) I do not really understand BMDMA in ioemu/hw/ide.c. google doesnt
>> seem to be helping much here. Any documentation about the QEMU IDE disks
>> would be greatly appreciated.
>>     
>
> No idea.
>
>   
>> 3) vl.c:5522 seems to be not the correct code (I guess our vl.c codes
>> are different).. Could you point me to the routine in vl.c where the
>> pending disk IO requests are flushed before VM save (I guess the VM save
>> happens in qemu_savevm_state() in vl.c wherein it invokes all the
>> registered save handlers)?
>>     
>
> It's in:
> void do_savevm(const char *name)
>
>   
>> 4) Somebody told me that the pending disk IO requests are stored
>> somewhere in quemu's I/O disk model but I am not sure exactly
>> where....Can somebody help me with this ?
>>     
>
> Qemu uses asynchronous IO, grepping aio or AIO should find something.
>
>
>   


-- 
"question = ( to ) ? be : ! be;"
                -- Wm. Shakespeare