Re: [Qemu-devel] [PATCH v11 14/15] rdma: introduce MIG_STATE_NONE and change MIG_STATE_SETUP state transition

["Michael R. Hines" <mrhines@linux.vnet.ibm.com>]

On 06/25/2013 10:17 AM, Juan Quintela wrote:
> "Michael R. Hines" <mrhines@linux.vnet.ibm.com> wrote:
>> On 06/25/2013 06:13 AM, Paolo Bonzini wrote:
>>> Il 25/06/2013 11:49, Juan Quintela ha scritto:
>>>> mrhines@linux.vnet.ibm.com wrote:
>>>>> From: "Michael R. Hines" <mrhines@us.ibm.com>
>>>>>
>>>>> As described in the previous patch, until now, the MIG_STATE_SETUP
>>>>> state was not really a 'formal' state. It has been used as a 'zero' state
>>>>> (what we're calling 'NONE' here) and QEMU has been unconditionally transitioning
>>>>> into this state when the QMP migration command was called. Instead we want to
>>>>> introduce MIG_STATE_NONE, which is our starting state in the state machine, and
>>>>> then immediately transition into the MIG_STATE_SETUP state when the QMP migrate
>>>>> command is issued.
>>>>>
>>>>> In order to do this, we must delay the transition into MIG_STATE_ACTIVE until
>>>>> later in the migration_thread(). This is done to be able to timestamp the amount of
>>>>> time spent in the SETUP state for proper accounting to the user during
>>>>> an RDMA migration.
>>>>>
>>>>> Furthermore, the management software, until now, has never been aware of the
>>>>> existence of the SETUP state whatsoever. This must change, because, timing of this
>>>>> state implies that the state actually exists.
>>>>>
>>>>> These two patches cannot be separated because the 'query_migrate' QMP
>>>>> switch statement needs to know how to handle this new state transition.
>>>>>
>>>>> Tested-by: Michael R. Hines <mrhines@us.ibm.com>
>>>>> Signed-off-by: Michael R. Hines <mrhines@us.ibm.com>
>>>>> @@ -316,6 +321,7 @@ static void migrate_fd_cancel(MigrationState *s)
>>>>>    {
>>>>>        DPRINTF("cancelling migration\n");
>>>>>    +    migrate_set_state(s, MIG_STATE_SETUP, MIG_STATE_CANCELLED);
>>>>>        migrate_set_state(s, MIG_STATE_ACTIVE, MIG_STATE_CANCELLED);
>>>>>    }
>>>> This chunk is wrong.
>>>>
>>>> we can call qme_migrate_cancel() at any point,  and it is going to be
>>>> called normally from MIG_STATE_ACTIVE.
>>>>
>>>>       migrate_set_satet(s, s->state,  MIG_STATE_CANCELLED)
>>>>
>>>> should do the trick.  Or something like that,  what do you think?
>>> I don't like the three-arguments migrate_set_state, but I don't have any
>>> better idea.
>>>
>>> With Juan's modification, it is fine (but not reviewed-by me :)).  While
>>> you resend, the first 13 patches of v10 can be merged (pull request).
>>> You can then rebase the last three on top.
>> So, I should send a v11 before Juan applies and generates the pull request?
>>
>> Did I understand correctly?
>>
>>> Michael, did you look at the "debugging/getting the protocol ready" mode
>>> where all pages are unpinned?
>>>
>>> Paolo
>>>
>>
>> We did not come up with a design on how such a mode would be
>> exposed to the user. Implementing this in migration-rdma.c is
>> just a few lines of code, but without a clear use case in this patch
>> series, I've not chosen expose it to the user yet without some kind
>> of agreement with you guys.
>>
>> Recommendations?
> Dunno.  I have tried to use the thing:
>
> I am using -incoming x-rdma:deus:4444
>
> (qemu) char device redirected to /dev/pts/2 (label serial0)
> librdmacm: Warning: couldn't read ABI version.
> librdmacm: Warning: assuming: 4
> librdmacm: Fatal: unable to get RDMA device list
> RDMA ERROR: could not create rdma event channel
> -incoming x-rdma:deus:4444: RDMA ERROR: could not create rdma event channel
>
>
> I also used "0" instead of deus (the machine name).
>
> The only thing that I did appart from intsalling librdmacm-devel was:
>
>
> # cat /etc/udev/rules.d/80-infinibad.rules
> KERNEL="rdma_cm", NAME="infiniband/%k", MODE="0666"
>
> As per README.  Something else I need to do to use the tcp
> implementations?
>
> Later,  Juan.
>
>

There's a simple bug found by Visilis affecting localhost migration 
which I believe also
affects using '0' as a hostname in the migration.....

Both of these should be fixed in my next version. Will send out today.
(I never tried migrating with 0.0.0.0 until now).

- Michael


>
>