Re: [Qemu-devel] [PATCH COLO-Frame (Base) v20 16/17] docs: Add documentation for COLO feature

[Hailiang Zhang <zhang.zhanghailiang@huawei.com>]

On 2016/10/5 21:37, Eric Blake wrote:
> On 09/29/2016 03:46 AM, zhanghailiang wrote:
>> Introduce the design of COLO, and how to test it.
>>
>> Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
>> ---
>>   docs/COLO-FT.txt | 190 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>   1 file changed, 190 insertions(+)
>>   create mode 100644 docs/COLO-FT.txt
>>
>
>> +
>> +== Background ==
>> +Virtual machine (VM) replication is a well known technique for providing
>> +application-agnostic software-implemented hardware fault tolerance
>> +"non-stop service".
>
> Do you want s/tolerance/tolerance, also known as/ ?
>

Yes, that is more appropriate.

>
>> +== Architecture ==
>> +
>> +The architecture of COLO is shown in the bellow diagram.
>
> s/bellow diagram/diagram below/
>

>> +It consists of a pair of networked physical nodes:
>> +The primary node running the PVM, and the secondary node running the SVM
>> +to maintain a valid replica of the PVM.
>> +PVM and SVM execute in parallel and generate output of response packets for
>> +client requests according to the application semantics.
>> +
>> +The incoming packets from the client or external network are received by the
>> +primary node, and then forwarded to the secondary node, so that Both the PVM
>
> s/Both/both/
>

>> +and the SVM are stimulated with the same requests.
>> +
>> +COLO receives the outbound packets from both the PVM and SVM and compares them
>> +before allowing the output to be sent to clients.
>> +
>> +The SVM is qualified as a valid replica of the PVM, as long as it generates
>> +identical responses to all client requests. Once the differences in the outputs
>> +are detected between the PVM and SVM, COLO withholds transmission of the
>> +outbound packets until it has successfully synchronized the PVM state to the SVM.
>> +
>
>> +== Components introduction ==
>> +
>> +You can see there are several components in COLO's diagram of architecture.
>> +Their functions are described as bellow.
>
> s/as bellow/below/
>

>> +
>> +HeartBeat:
>> +Runs on both the primary and secondary nodes, to periodically check platform
>> +availability. When the primary node suffers a hardware fail-stop failure,
>> +the heartbeat stops responding, the secondary node will trigger a failover
>> +as soon as it determines the absence.
>> +
>> +COLO disk Manager:
>> +When primary VM writes data into image, the colo disk manger captures this data
>> +and send it to secondary VM’s which makes sure the context of secondary VM's
>
> s/send/sends/
>

>> +image is consentient with the context of primary VM 's image.
>
> s/consentient/consistent/
> s/VM 's/VM's/
>

>> +For more details, please refer to docs/block-replication.txt.
>> +
>> +Checkpoint/Failover Controller:
>> +Modifications of save/restore flow to realize continuous migration,
>> +to make sure the state of VM in Secondary side always be consistent with VM in
>
> s/always be/is always/
>

>> +Primary side.
>> +
>> +COLO Proxy:
>> +Delivers packets to Primary and Seconday, and then compare the responses from
>> +both side. Then decide whether to start a checkpoint according to some rules.
>> +
>> +Note:
>> + a. HeartBeat is not been realized, so you need to trigger failover process
>
> s/is/has/
> s/realized/implemented yet/
>
> Is this note going to be stale once heartbeat is implemented?
>

Yes, but we're not sure if it is suitable to implement it in qemu.

>> +    by using 'x-colo-lost-heartbeat' command.
>> + b. COLO proxy compents is work-in-process, it only support periodic checkpoint
>
> s/compents is/components are a/
>

>> +    mode now, just as Micro-checkpointing.
>> +
>
>> +3. On Primary VM's QEMU monitor, issue command:
>> +{'execute':'qmp_capabilities'}
>> +{ 'execute': 'human-monitor-command',
>> +  'arguments': {'command-line': 'drive_add -n buddy driver=replication,mode=primary,file.driver=nbd,file.host=xx.xx.xx.xx,file.port=8889,file.export=colo-disk0,node-name=node0'}}
>
> It would be really nice if we could get this done through QMP
> blockdev-add instead of HMP drive_add.
>

You are right, but this command doesn't support nbd drive yet in upstream.
I saw Max had send a patch-set to support it. I will update this after his
patches been merged.

>> +
>> +Before issuing '{ "execute": "x-colo-lost-heartbeat" }' command, we have to
>> +issue block related command to stop block replication.
>> +Primary:
>> +  Remove the nbd child from the quorum:
>> +  { 'execute': 'x-blockdev-change', 'arguments': {'parent': 'colo-disk0', 'child': 'children.1'}}
>> +  { 'execute': 'human-monitor-command','arguments': {'command-line': 'drive_del blk-buddy0'}}
>> +  Note: there is no qmp command to remove the blockdev now
>
> Don't we have x-blockdev-del?
>

Yes, we can use this command, I'll fix it in next version.

>> +
>> +Secondary:
>> +  The primary host is down, so we should do the following thing:
>> +  { 'execute': 'nbd-server-stop' }
>> +
>> +== TODO ==
>> +1. Support continuously VM replication.
>
> s/continuously/continuous/
>
>> +2. Support shared storage.
>> +3. Develop the heartbeat part.
>> +4. Reduce checkpoint VM’s downtime while do checkpoint.
>
> s/do/doing/
>
>>

All the above typos and grammatical mistake  will be fixed in next version, thanks!

Hailiang
>