Re: [Qemu-devel] [PATCH 1/2] Reopen files after migration

[Anthony Liguori <anthony@codemonkey.ws>]

On 11/09/2011 03:10 PM, Juan Quintela wrote:
> Anthony Liguori<anthony@codemonkey.ws>  wrote:
>> On 11/09/2011 01:16 PM, Juan Quintela wrote:
>>> We need to invalidate the Read Cache on the destination, otherwise we
>>> have corruption.  Easy way to reproduce it is:
>>>
>>> - create an qcow2 images
>>> - start qemu on destination of migration (qemu .... -incoming tcp:...)
>>> - start qemu on source of migration and do one install.
>>> - migrate at the end of install (when lot of disk IO has happened).
>>>
>>> Destination of migration has a local copy of the L1/L2 tables that existed
>>> at the beginning, before the install started.  We have disk corruption at
>>> this point.  The solution (for NFS) is to just re-open the file.  Operations
>>> have to happen in this order:
>>>
>>> - source of migration: flush()
>>> - destination: close(file);
>>> - destination: open(file)
>>>
>>> it is not necesary that source of migration close the file.
>>>
>>> Signed-off-by: Juan Quintela<quintela@redhat.com>
>>
>> Couple thoughts:
>>
>> 1) Pretty sure this would break -snapshot.  I do test migration with
>> -snapshot so please don't break it.
>
> Can you give me one example?  I don't know how to use -snapshot with migration.

This is totally unsafe but has always worked for me.  On the same box:

$ qemu -hda foo.img -snapshot

$ qemu -hda foo.img -snapshot -incoming tcp:localhost:1025

This is not the *only* way I test migration but it's very convenient for sniff 
testing.  The problem with your patch is that it assumes that once you've opened 
a file, the name still exists.  But that is not universally true.  It needs to 
degrade in a useful way.

I think just deferring open is probably the best strategy.

>
>> 2) I don't think this is going to work very well with encrypted drives.
>
> To be hones, no clue.

Deferring open addresses this is a nice way I think.

>> Perhaps we could do something like:
>>
>> http://mid.gmane.org/1284213896-12705-2-git-send-email-aliguori@us.ibm.com
>
> That is something like I wanted to know.
>
>> And do reopen as a default implementation.  That way we don't have to
>> do reopen for formats that don't need it (raw)
>
> Kevin told me that know that we allow online resize, we should also
> update that for raw, but I haven't tested to be sure one way or another.
>
>> or can flush caches without reopening the file (qed).
>
> qcow2 could be told to flush caches, it is that the code is not there.
> It shouldn't be _that_ difficult.  But I am not able to understand
> anymore block_open<->  block_file_open relationship.
>
>> It doesn't fix NFS close-to-open, but I think the right way to do that
>> is to defer the open, not to reopen.
>
> Fully agree here, that would be another way to fix it.  See that in my
> other answer I showed that Markus already have problems with ide + cmos,
> so I think that we should have:

I've posted patches that delay the geometry guess until the device model is 
initialized.  That avoids this particular problem.

Regards,

Anthony Liguori

>
> - initialization done before we open files/block/<whatever you call it>
> - open files/block/...
> - late initialization that uses that (almost nothing needs to be here
>    and should be easy to audit).
>
> About NFS, iSCSI, FC, my understanding is that if you use anything
> different than cache=none you are playing with fire, and will get burned
> sooner or later (it took quite a bit for Christoph to make me understand
> that, but now I fully agree with him).
>
> Later, Juan.