Re: [Qemu-devel] [Xen-devel] [PATCHv3] QEMU(upstream): Disable xen's use of O_DIRECT by default as it results in crashes.

[George Dunlap <george.dunlap@eu.citrix.com>]

On 03/19/2013 11:14 AM, Paolo Bonzini wrote:
> Il 19/03/2013 11:51, George Dunlap ha scritto:
>> On 03/19/2013 10:43 AM, Paolo Bonzini wrote:
>>>>> Even for successful migration, it would also be bad for downtime (QEMU
>>>>> isn't exactly lightning-fast to start).  And even if failure weren't
>>>>> catastrophic, it would be a pity to transfer a few gigs of memory and
>>>>> then find out that QEMU isn't present in the destination. :)
>>>>
>>>> Well, if qemu isn't present at the destination, that's definitely user
>>>> error. :-)  In any case, I know that he migrate can resume if it
>>>> fails, so I suspect that the qemu is just paused on the sending side
>>>> until the migration is known to complete.  As long as the last write
>>>> was flushed to the NFS server before the receiver opens the file, we
>>>> should be safe.
>>>
>>> Note that the close really must happen before the next open.  Otherwise
>>> the file metadata might not be up-to-date on the destination, too.
>>
>> By "file metadata" I assume you mean "metadata about the virtual disk
>> within the file", not "metadata about the file within the filesystem",
>> right?  That's good to know, I'll keep that in mind.
>
> Actually especially the former (I'm calling them respectively "image
> metadata" and "file metadata").  File metadata could also be a problem,
> but I think it might just work except in cases like on-line resizing
> during migration.
>
>> Even if it's true that at the moment qemu doesn't write the file
>> metadata until it closes the file, that just means we'd have to add a
>> hook to the callback to save qemu state, to sync the metadata at that
>> point, right?
>
> Unfortunately no.  The problem is in the loading side's kernel, on which
> you do not have any control.  If the loading side doesn't use O_DIRECT,
> any attempt to invalidate the metadata in userspace or on the source is
> futile, because there is no way to invalidate the page cache's copy of
> that metadata.

Yes, I meant "the only further thing we would have to do". The entire 
discussion relies on the assumption that the receiving side doesn't open 
the file until after the sending side has issued the qemu state save. 
So as long as both the virtual blocks and the image metadata have been 
synced to the NFS server at that point, we should be all right.  If at 
the moment the image metadata is *not* synced at that point, it seems 
like we should be able to make it so relatively easily.

  -George