From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:57845) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SFle5-000515-D6 for qemu-devel@nongnu.org; Thu, 05 Apr 2012 08:18:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1SFldp-0002g3-IB for qemu-devel@nongnu.org; Thu, 05 Apr 2012 08:18:31 -0400 Received: from mail-qc0-f173.google.com ([209.85.216.173]:40254) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SFldp-0002f6-As for qemu-devel@nongnu.org; Thu, 05 Apr 2012 08:18:17 -0400 Received: by mail-qc0-f173.google.com with SMTP id c20so791325qcs.4 for ; Thu, 05 Apr 2012 05:18:12 -0700 (PDT) From: Ori Mamluk MIME-Version: 1.0 Date: Thu, 5 Apr 2012 15:17:58 +0300 Message-ID: <24dda3a956e11856b0439825366f186e@mail.gmail.com> Content-Type: multipart/alternative; boundary=20cf300fafbf9abfd604bced88b9 Subject: [Qemu-devel] [RFC PATCH v3 6/9] repagent: Updated documentation in qemu-repagent.txt. Renamed rephub read command to remoteIo. List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: Kevin Wolf , Roni Luxenberg , Stefan Hajnoczi , dlaor@redhat.com, Anthony Liguori , Oded Kedem , Yair Kuszpet , Paolo Bonzini --20cf300fafbf9abfd604bced88b9 Content-Type: text/plain; charset=ISO-8859-1 Sent 'qemu-devel@nongnu.org'as repagent patch V2 --- block/repagent/qemu-repagent.txt | 147 +++++++++++++++----------------------- block/repagent/repagent.c | 71 ++++++++++-------- block/repagent/repagent.h | 4 +- block/repagent/repagent_client.c | 4 +- block/repagent/repcmd.h | 4 +- block/repagent/rephub_cmds.h | 27 ++++--- 6 files changed, 119 insertions(+), 138 deletions(-) diff --git a/block/repagent/qemu-repagent.txt b/block/repagent/qemu-repagent.txt index e3b0c1e..f8def3f 100644 --- a/block/repagent/qemu-repagent.txt +++ b/block/repagent/qemu-repagent.txt @@ -1,104 +1,73 @@ - repagent - replication agent - a Qemu module for enabling continuous async replication of VM volumes + repagent - replication agent - a Qemu module for enabling continuous async replication of VM volumes Introduction - This document describes a feature in Qemu - a replication agent (AKA Repagent). - The Repagent is a new module that exposes an API to an external replication system (AKA Rephub). - This API allows a Rephub to communicate with a Qemu VM and continuously replicate its volumes. - The imlementation of a Rephub is outside of the scope of this document. There may be several various Rephub - implenetations using the same repagent in Qemu. + This document describes a feature in Qemu - a replication agent (named Repagent). + The Repagent is a new module that exposes an API to an external replication system (AKA Rephub). + This API allows a Rephub to communicate with a Qemu VM and continuously replicate its volumes. + The imlementation of a Rephub is outside of the scope of this document. There may be several various Rephub + implenetations using the same repagent in Qemu. + The Repagent is storage driver that acts like a filter driver. + It can be regarded as a 'plugin' that is activated when the management system enables replication. Main feature of Repagent - Repagent does the following: - * Report volumes - report a list of all volumes in a VM to the Rephub. - * Report writes to a volume - send all writes made to a protected volume to the Rephub. - The reporting of an IO is asyncronuous - i.e. the IO is not delayed by the Repagent to get any acknowledgement from the Rephub. - It is only copied to the Rephub. - * Read a protected volume - allows the Rephub to read a protected volume, to enable the protected hub to syncronize the content of a protected volume. + Repagent has the following main features: + * Report volumes - report a list of all volumes in a VM to the Rephub. + * Mirror writes - Report writes to a volume - send all writes made to a protected volume to the Rephub. + The reporting of an IO is asyncronuous - i.e. the IO is not delayed by the Repagent to get any acknowledgement from the Rephub. + It is only copied to the Rephub. + * Remote IO - Read/write a volume - allows the Rephub to read a protected volume, to enable the protected hub to syncronize + the content of a protected volume. + Also used to read/write to a recovery volume - the replica of a protected volume. Description of the Repagent module Build and run options - New configure option: --enable-replication - New command line option: - -repagent [hub IP/name] - Enable replication support for disks - hub is the ip or name of the machine running the replication hub. + New configure option: --enable-repagent + New command line option: + -repagent [hub IP/name] + Enable replication support for disks + hub is the ip or name of the machine running the replication hub. Module APIs - The Repagent module interfaces two main components: - 1. The Rephub - An external API based on socket messages - 2. The generic block layer- block.c - - Rephub message API - The external replication API is a message based API. - We won't go into the structure of the messages here - just the sematics. - - Messages list - (The updated list and comments are in Rephub_cmds.h) - - Messages from the Repagent to the Rephub: - * Protected write - The Repagent sends each write to a protected volume to the hub with the IO status. - In case the status is bad the write content is not sent - * Report VM volumes - The agent reports all the volumes of the VM to the hub. - * Read Volume Response - A response to a Read Volume Request - Sends the data read from a protected volume to the hub - * Agent shutdown - Notifies the hub that the agent is about to shutdown. - This allows a graceful shutdown. Any disconnection of an agent without - sending this command will result in a full sync of the VM volumes. - - Messages from the Rephub to the Repagent: - * Start protect - The hub instructs the agent to start protecting a volume. When a volume is protected - all its writes are sent to to the hub. - With this command the hub also assigns a volume ID to the given volume name. - * Read volume request - The hub issues a read IO to a protected volume. - This command is used during sync - when the hub needs to read unsyncronized - sections of a protected volume. - This command is a request, the read data is returned by the read volume response message (see above). - block.c API - The API to the generic block storage layer contains 3 functionalities: - 1. Handle writes to protected volumes - In bdrv_co_do_writev, each write is reported to the Repagent module. - 2. Handle each new volume that registers - In bdrv_open - each new bottom-level block driver that registers is reported. - 2. Read from a volume - Repagent calls bdrv_aio_readv to handle read requests coming from the hub. + The Repagent module interfaces two main components: + 1. The Rephub - An external API based on socket messages + See detailed comments about each message in rephub_cmds.h + 2. The generic block layer- block.c + Repagent is a block driver. Most of the block driver functions are just a pass-through + to the next driver. + Writes are mirrors to the hub for replication + Open function is used for registering each volume in Repagent. General description of a Rephub - a replication system the repagent connects to - This section describes in high level a sample Rephub - a replication system that uses the repagent API - to replicate disks. - It describes a simple Rephub that comntinuously maintains a mirror of the volumes of a VM. - - Say we have a VM we want to protect - call it PVM, say it has 2 volumes - V1, V2. - Our Rephub is called SingleRephub - a Rephub protecting a single VM. - - Preparations - 1. The user chooses a host to rub SingleRephub - a different host than PVM, call it Host2 - 2. The user creates two volumes on Host2 - same sizes of V1 and V2, call them V1R (V1 recovery) and V2R. - 3. The user runs SingleRephub process on Host2, and gives V1R and V2R as command line arguments. - From now on SingleRephub waits for the protected VM repagent to connect. - 4. The user runs the protected VM PVM - and uses the switch -repagent . - - Runtime - 1. The repagent module connects to SingleRephub on startup. - 2. repagent reports V1 and V2 to SingleRephub. - 3. SingleRephub starts to perform an initial synchronization of the protected volumes- - it reads each protected volume (V1 and V2) - using read volume requests - and copies the data into the - recovery volume V1R and V2R. - 4. SingleRephub enters 'protection' mode - each write to the protected volume is sent by the repagent to the Rephub, - and the Rephub performs the write on the matching recovery volume. - - * Note that during stage 3 writes to the protected volumes are not ignored - they're kept in a bitmap, - and will be read again when stage 3 ends, in an interative convergin process. - - This flow continuously maintains an updated recovery volume. - If the protected system is damaged, the user can create a new VM on Host2 with the replicated volumes attached to it. - The new VM is a replica of the protected system. + This section describes in high level a sample Rephub - a replication system that uses the repagent API + to replicate disks. + It describes a simple Rephub that comntinuously maintains a mirror of the volumes of a VM. + + Say we have a VM we want to protect - call it PVM, say it has 2 volumes - V1, V2. + Our Rephub is called SingleRephub - a Rephub protecting a single VM. + + Preparations + 1. The user chooses a host to rub SingleRephub - a different host than PVM, call it Host2 + 2. The user creates two volumes on Host2 - same sizes of V1 and V2, call them V1R (V1 recovery) and V2R. + 3. The user runs SingleRephub process on Host2, and gives V1R and V2R as command line arguments. + From now on SingleRephub waits for the protected VM repagent to connect. + 4. The user runs the protected VM PVM - and uses the switch -repagent . + + Runtime + 1. The repagent module connects to SingleRephub on startup. + 2. repagent reports V1 and V2 to SingleRephub. + 3. SingleRephub starts to perform an initial synchronization of the protected volumes- + it reads each protected volume (V1 and V2) - using read volume requests - and copies the data into the + recovery volume V1R and V2R. + 4. SingleRephub enters 'protection' mode - each write to the protected volume is sent by the repagent to the Rephub, + and the Rephub performs the write on the matching recovery volume. + + * Note that during stage 3 writes to the protected volumes are not ignored - they're kept in a bitmap, + and will be read again when stage 3 ends, in an interative convergin process. + + This flow continuously maintains an updated recovery volume. + If the protected system is damaged, the user can create a new VM on Host2 with the replicated volumes attached to it. + The new VM is a replica of the protected system. diff --git a/block/repagent/repagent.c b/block/repagent/repagent.c index c3dd593..bdc0117 100644 --- a/block/repagent/repagent.c +++ b/block/repagent/repagent.c @@ -29,7 +29,7 @@ struct RepAgentState { typedef struct RepagentReadVolIo { QEMUIOVector qiov; - RepCmdReadVolReq rep_cmd; + RepCmdRemoteIoReq rep_cmd; uint8_t *buf; struct timeval start_time; } RepagentReadVolIo; @@ -38,7 +38,7 @@ static int repagent_get_volume_by_driver( BlockDriverState *bs); static int repagent_get_volume_by_name(const char *name); static void repagent_report_volumes_to_hub(void); -static void repagent_vol_read_done(void *opaque, int ret); +static void repagent_remote_io_done(void *opaque, int ret); static struct timeval tsub(struct timeval t1, struct timeval t2); RepAgentState g_rep_agent = { 0 }; @@ -242,15 +242,15 @@ static int repagent_get_volume_by_id(uint64_t vol_id) return -1; } -int repaget_read_vol(RepCmdReadVolReq *pcmd, uint8_t *pdata) +int repagent_remote_io(RepCmdRemoteIoReq *pcmd, uint8_t *pdata) { int index = repagent_get_volume_by_id(pcmd->volume_id); int size_bytes = pcmd->size_sectors * 512; if (index < 0) { printf("Vol read - Could not find vol id %llx\n", (unsigned long long int) pcmd->volume_id); - RepCmdReadVolRes *p_res_cmd = (RepCmdReadVolRes *) repcmd_new( - REPHUB_CMD_READ_VOL_RES, 0, NULL); + RepCmdRemoteIoRes *p_res_cmd = (RepCmdRemoteIoRes *) repcmd_new( + REPHUB_CMD_REMOTE_IO_RES, 0, NULL); p_res_cmd->req_id = pcmd->req_id; p_res_cmd->volume_id = pcmd->volume_id; p_res_cmd->io_status = -1; @@ -264,58 +264,67 @@ int repaget_read_vol(RepCmdReadVolReq *pcmd, uint8_t *pdata) (unsigned long long int) pcmd->offset_sectors, pcmd->size_sectors); { - RepagentReadVolIo *read_xact = calloc(1, sizeof(RepagentReadVolIo)); + RepagentReadVolIo *io_xaction = calloc(1, sizeof(RepagentReadVolIo)); /* BlockDriverAIOCB *acb; */ - ZERO_MEM_OBJ(read_xact); + ZERO_MEM_OBJ(io_xaction); - qemu_iovec_init(&read_xact->qiov, 1); + qemu_iovec_init(&io_xaction->qiov, 1); /*read_xact->buf = qemu_blockalign(g_rep_agent.volumes[index]->driver_ptr, size_bytes); */ - read_xact->buf = (uint8_t *) g_malloc(size_bytes); - read_xact->rep_cmd = *pcmd; - qemu_iovec_add(&read_xact->qiov, read_xact->buf, size_bytes); + io_xaction->buf = (uint8_t *) g_malloc(size_bytes); + io_xaction->rep_cmd = *pcmd; + qemu_iovec_add(&io_xaction->qiov, io_xaction->buf, size_bytes); - gettimeofday(&read_xact->start_time, NULL); + gettimeofday(&io_xaction->start_time, NULL); /* orim TODO - use the returned acb to cancel the request on shutdown */ - /*acb = */bdrv_aio_readv(g_rep_agent.volumes[index]->driver_ptr, - read_xact->rep_cmd.offset_sectors, &read_xact->qiov, - read_xact->rep_cmd.size_sectors, repagent_vol_read_done, - read_xact); + /*acb = */ + if (pcmd->is_read) { + bdrv_aio_readv(g_rep_agent.volumes[index]->driver_ptr, + io_xaction->rep_cmd.offset_sectors, &io_xaction->qiov, + io_xaction->rep_cmd.size_sectors, repagent_remote_io_done, + io_xaction); + } else { + bdrv_aio_writev(g_rep_agent.volumes[index]->driver_ptr, + io_xaction->rep_cmd.offset_sectors, &io_xaction->qiov, + io_xaction->rep_cmd.size_sectors, repagent_remote_io_done, + io_xaction); + } } return TRUE; } -static void repagent_vol_read_done(void *opaque, int ret) +static void repagent_remote_io_done(void *opaque, int ret) { struct timeval t2; - RepagentReadVolIo *read_xact = (RepagentReadVolIo *) opaque; + RepagentReadVolIo *io_xaction = (RepagentReadVolIo *) opaque; uint8_t *pdata = NULL; - RepCmdReadVolRes *pcmd = (RepCmdReadVolRes *) repcmd_new( - REPHUB_CMD_READ_VOL_RES, read_xact->rep_cmd.size_sectors * 512, + RepCmdRemoteIoRes *pcmd = (RepCmdRemoteIoRes *) repcmd_new( + REPHUB_CMD_REMOTE_IO_RES, io_xaction->rep_cmd.size_sectors * 512, &pdata); - pcmd->req_id = read_xact->rep_cmd.req_id; - pcmd->volume_id = read_xact->rep_cmd.volume_id; + pcmd->req_id = io_xaction->rep_cmd.req_id; + pcmd->volume_id = io_xaction->rep_cmd.volume_id; pcmd->io_status = -1; - printf("Protected vol read - volId %llu, offset %llu, size %u\n", - (unsigned long long int) read_xact->rep_cmd.volume_id, - (unsigned long long int) read_xact->rep_cmd.offset_sectors, - read_xact->rep_cmd.size_sectors); + printf("Remote IO request - volId %llu, offset %llu, size %u, is_read %u\n", + (unsigned long long int) io_xaction->rep_cmd.volume_id, + (unsigned long long int) io_xaction->rep_cmd.offset_sectors, + io_xaction->rep_cmd.size_sectors, + io_xaction->rep_cmd.is_read); gettimeofday(&t2, NULL); if (ret >= 0) { /* Read response - send the data to the hub */ - t2 = tsub(t2, read_xact->start_time); - printf("Read prot vol done. Took %u seconds, %u us.", + t2 = tsub(t2, io_xaction->start_time); + printf("Remote IO done. Took %u seconds, %u us.", (uint32_t) t2.tv_sec, (uint32_t) t2.tv_usec); pcmd->io_status = 0; /* Success */ /* orim todo optimize - don't copy, use the qiov buffer */ - qemu_iovec_to_buffer(&read_xact->qiov, pdata); + qemu_iovec_to_buffer(&io_xaction->qiov, pdata); } else { printf("readv failed: %s\n", strerror(-ret)); } @@ -323,9 +332,9 @@ static void repagent_vol_read_done(void *opaque, int ret) repagent_client_send((RepCmd *) pcmd); /*qemu_vfree(read_xact->buf); */ - g_free(read_xact->buf); + g_free(io_xaction->buf); - g_free(read_xact); + g_free(io_xaction); } static struct timeval tsub(struct timeval t1, struct timeval t2) diff --git a/block/repagent/repagent.h b/block/repagent/repagent.h index 310db0f..0f69820 100644 --- a/block/repagent/repagent.h +++ b/block/repagent/repagent.h @@ -30,7 +30,7 @@ typedef struct RepAgentState RepAgentState; typedef struct RepCmdStartProtect RepCmdStartProtect; typedef struct RepCmdDataStartProtect RepCmdDataStartProtect; -struct RepCmdReadVolReq; +struct RepCmdRemoteIoReq; /* orim temporary */ extern int use_repagent; @@ -45,7 +45,7 @@ void repagent_deregister_drive(const char *drive_path, BlockDriverState *driver_ptr); int repaget_start_protect(RepCmdStartProtect *pcmd, RepCmdDataStartProtect *pcmd_data); -int repaget_read_vol(struct RepCmdReadVolReq *pcmd, uint8_t *pdata); +int repagent_remote_io(struct RepCmdRemoteIoReq *pcmd, uint8_t *pdata); void repagent_client_connected(void); diff --git a/block/repagent/repagent_client.c b/block/repagent/repagent_client.c index 9ed8485..ee4aeb7 100644 --- a/block/repagent/repagent_client.c +++ b/block/repagent/repagent_client.c @@ -125,8 +125,8 @@ void repagent_process_cmd(RepCmd *pcmd, uint8_t *pdata, void *clientPtr) (RepCmdDataStartProtect *) pdata); } break; - case REPHUB_CMD_READ_VOL_REQ: { - is_free_data = repaget_read_vol((RepCmdReadVolReq *) pcmd, pdata); + case REPHUB_CMD_REMOTE_IO_REQ: { + is_free_data = repagent_remote_io((RepCmdRemoteIoReq *) pcmd, pdata); } break; default: diff --git a/block/repagent/repcmd.h b/block/repagent/repcmd.h index 8c6cf1b..0a7f297 100644 --- a/block/repagent/repcmd.h +++ b/block/repagent/repcmd.h @@ -36,8 +36,8 @@ enum RepCmds { REPHUB_CMD_PROTECTED_WRITE = 2, REPHUB_CMD_REPORT_VM_VOLUMES = 3, REPHUB_CMD_START_PROTECT = 4, - REPHUB_CMD_READ_VOL_REQ = 5, - REPHUB_CMD_READ_VOL_RES = 6, + REPHUB_CMD_REMOTE_IO_REQ = 5, + REPHUB_CMD_REMOTE_IO_RES = 6, REPHUB_CMD_AGENT_SHUTDOWN = 7, }; diff --git a/block/repagent/rephub_cmds.h b/block/repagent/rephub_cmds.h index d1bad06..cb737e6 100644 --- a/block/repagent/rephub_cmds.h +++ b/block/repagent/rephub_cmds.h @@ -97,40 +97,43 @@ typedef struct RepCmdDataStartProtect { /********************************************************* - * RepCmd Read Volume Request + * RepCmd Remote IO Request * - * REPHUB_CMD_READ_VOL_REQ + * REPHUB_CMD_REMOTE_IO_REQ * Direction: hub->agent * - * The hub issues a read IO to a protected volume. - * This command is used during sync - when the hub needs - * to read unsyncronized sections of a protected volume. + * The hub issues an IO to a volume. + * This command is used for: + * - Reading protected volume during sync + * - Read/write to a recovery volume during + * protect and failover or failover test * This command is a request, the read data is returned - * by the response command REPHUB_CMD_READ_VOL_RES + * by the response command REPHUB_CMD_REMOTE_IO_RES *********************************************************/ -typedef struct RepCmdReadVolReq { +typedef struct RepCmdRemoteIoReq { RepCmdHdr hdr; int req_id; int size_sectors; uint64_t volume_id; uint64_t offset_sectors; -} RepCmdReadVolReq; + int is_read; +} RepCmdRemoteIoReq; /********************************************************* * RepCmd Read Volume Response * - * REPHUB_CMD_READ_VOL_RES + * REPHUB_CMD_REMOTE_IO_RES * Direction: agent->hub * - * A response to REPHUB_CMD_READ_VOL_REQ. + * A response to REPHUB_CMD_REMOTE_IO_REQ. * Sends the data read from a protected volume *********************************************************/ -typedef struct RepCmdReadVolRes { +typedef struct RepCmdRemoteIoRes { RepCmdHdr hdr; int req_id; int io_status; uint64_t volume_id; -} RepCmdReadVolRes; +} RepCmdRemoteIoRes; /********************************************************* * RepCmd Agent shutdown -- 1.7.6.5 --20cf300fafbf9abfd604bced88b9 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

Sent 'qemu-devel@nongnu.org'as repagent patch V2

=A0

---

block/repagent= /qemu-repagent.txt |=A0 147 +++++++++++++++-----------------------

block/repagent/repagent.c=A0=A0=A0=A0=A0=A0=A0 |=A0=A0 7= 1 ++++++++++--------

block/repagent/repagent.h=A0=A0=A0=A0=A0=A0=A0 |=A0= =A0=A0 4 +-

block/repagent/repagent_client.c |= =A0=A0=A0 4 +-

block/repagent/repcmd.h=A0=A0=A0= =A0=A0=A0=A0=A0=A0 |=A0=A0=A0 4 +-

block/repagent/rephub_cmds.h=A0=A0=A0=A0 |=A0=A0 27 ++++---

6 files changed, 119 insertions(+), 138 deletions(-)

=A0

diff --git a/block/repagent/= qemu-repagent.txt b/block/repagent/qemu-repagent.txt

index e3b0c1e..f8def3f 100644

--- a/block/repagent/qemu-repagent.txt

+++ b/= block/repagent/qemu-repagent.txt

@@ -1,104 +1,73 = @@

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 repagent - = replication agent - a Qemu module for enabling continuous async replication= of VM volumes

+=A0=A0=A0 repagent - replication = agent - a Qemu module for enabling continuous async replication of VM volum= es

=A0Introduction

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 This document de= scribes a feature in Qemu - a replication agent (AKA Repagent).

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 The Repagent is a n= ew module that exposes an API to an external replication system (AKA Rephub= ).

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 This API al= lows a Rephub to communicate with a Qemu VM and continuously replicate its = volumes.

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= The imlementation of a Rephub is outside of the scope of this document. Th= ere may be several various Rephub

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 implenetati= ons using the same repagent in Qemu.

+=A0=A0=A0 T= his document describes a feature in Qemu - a replication agent (named Repag= ent).

+=A0=A0=A0 The Repagent is a new module tha= t exposes an API to an external replication system (AKA Rephub).

+=A0=A0=A0 This API allows a Rephub to communicate w= ith a Qemu VM and continuously replicate its volumes.

+=A0=A0=A0 The imlementation of a Rephub is outside of the scope of th= is document. There may be several various Rephub

+=A0=A0=A0 implenetations using the same repagent in= Qemu.

+=A0=A0=A0 The Repagent is storage driver = that acts like a filter driver.

+=A0=A0=A0 It can= be regarded as a 'plugin' that is activated when the management sy= stem enables replication.

=A0Main feature of Repag= ent

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Repa= gent does the following:

-=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0 * Report volumes - report a list of all volumes in a VM = to the Rephub.

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 * Report wr= ites to a volume - send all writes made to a protected volume to the Rephub= .

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 The reporting of an IO is asyncr= onuous - i.e. the IO is not delayed by the Repagent to get any acknowledgem= ent from the Rephub.

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 It is only copied to the Rephub.

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 * Read a prot= ected volume - allows the Rephub to read a protected volume, to enable the = protected hub to syncronize the content of a protected volume.

+=A0=A0=A0 Repagent has the following main features:=

+=A0=A0=A0 * Report volumes - report a list of a= ll volumes in a VM to the Rephub.

+=A0=A0=A0 * Mi= rror writes - Report writes to a volume - send all writes made to a protect= ed volume to the Rephub.

+=A0=A0=A0=A0=A0=A0=A0 The reporting of an IO is asy= ncronuous - i.e. the IO is not delayed by the Repagent to get any acknowled= gement from the Rephub.

+=A0=A0=A0=A0=A0=A0=A0 It= is only copied to the Rephub.

+=A0=A0=A0 * Remote IO - Read/write a=A0 volume - al= lows the Rephub to read a protected volume, to enable the protected hub to = syncronize

+=A0=A0=A0=A0=A0=A0 the content of a p= rotected volume.

+=A0=A0=A0=A0=A0=A0 Also used to read/write to a rec= overy volume - the replica of a protected volume.

=A0Description of the Repagent module

=A0Build and run options

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 New configure option: --enable= -replication

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0 New command line option:

-=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0 -repagent [hub IP/name]

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Enable replication support for disks

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0 hub is the ip or name of the machine running th= e replication hub.

+=A0=A0=A0 New configure option: --enable-repagent

+=A0=A0=A0 New command line option:

+=A0=A0=A0 -repagent [hub IP/name]

+= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Enable replicatio= n support for disks

+=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0 hub is the ip or name of the machine running the replication hub.

=A0Module APIs

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 The Repagent mod= ule interfaces two main components:

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 1. The Reph= ub - An external API based on socket messages

-= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 2. The generic block layer- block.c=

-

-=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0 Rephub message API

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 The external replication API is a mess= age based API.

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 We won't go into= the structure of the messages here - just the sematics.

-

-=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Messages= list

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0 (The updated list and comments are in Rephub_cmds.h)

=

-

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0 Messages from the Repagent to the Rephub:

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 * = Protected write

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 The Repagent sends each write to a pro= tected volume to the hub with the IO status.

-=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 In case the status is bad the write content = is not sent

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0 * Report VM volumes

-=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0 The agent reports all the volumes of the VM to the hub.

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0 * Read Volume Response

-=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0 A response to a Read Volume Request

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Sends the data read from a protected v= olume to the hub

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 * Agent shutdown

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Notifies the hub that the agent is abo= ut to shutdown.

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0 This allows a graceful shutdown. Any disconnection of an agent without<= /p>

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 sending this comm= and will result in a full sync of the VM volumes.

-

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0 Messages from the Rephub to the Repagent:

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0 * Start protect

-=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0 The hub instructs the agent to start protecting a volume. When= a volume is protected

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 all its writes ar= e sent to to the hub.

-=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0 With this command the hub also assigns a volume ID to the given v= olume name.

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0 * Read volume request

-=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0 The hub issues a read IO to a protected volume.

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 This command is u= sed during sync - when the hub needs to read unsyncronized

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 sections of a protected volum= e.

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 This command is a= request, the read data is returned by the read volume response message (se= e above).

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0 block.c API

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 The API to the generic block storage l= ayer contains 3 functionalities:

-=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 1.= Handle writes to protected volumes

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 In bdrv_co_= do_writev, each write is reported to the Repagent module.

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0 2. Handle each new volume that registers

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 In bdrv_ope= n - each new bottom-level block driver that registers is reported.

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 2. Read from a volume

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Repagent ca= lls bdrv_aio_readv to handle read requests coming from the hub.

+=A0=A0=A0 The Repagent module interfaces two main component= s:

+=A0=A0=A0 1. The Rephub - An external API based on socket messages

+=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 See detailed comments ab= out each message in rephub_cmds.h

+=A0=A0=A0 2. T= he generic block layer- block.c

+=A0=A0=A0=A0=A0=A0=A0 Repagent is a block driver. M= ost of the block driver functions are just a pass-through

+=A0=A0=A0=A0=A0=A0=A0 to the next driver.

+=A0=A0=A0=A0=A0=A0=A0 Writes are mirrors to the hub for replication

+=A0=A0=A0=A0=A0=A0=A0 Open function is used for reg= istering each volume in Repagent.

=A0

=A0General description of a Re= phub=A0 - a replication system the repagent connects to

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 This sectio= n describes in high level a sample Rephub - a replication system that uses = the repagent API

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0 to replicate disks.

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 It describes a simple Rephub that = comntinuously maintains a mirror of the volumes of a VM.

-

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0 Say we have a VM we want to protect - call it PVM, say it has 2 volumes= - V1, V2.

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Our Rephub = is called SingleRephub - a Rephub protecting a single VM.

-

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0 Preparations

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0 1. The user chooses a host to rub SingleRephub - a different host= than PVM, call it Host2

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 2. The user= creates two volumes on Host2 - same sizes of V1 and V2, call them V1R (V1 = recovery) and V2R.

-=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0 3. The user runs SingleRephub process on Host2, and gives V1R = and V2R as command line arguments.

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 From now on SingleRephub waits for the= protected VM repagent to connect.

-=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0 4. The user runs the protected VM PVM - and use= s the switch -repagent <Host2 IP>.

-

-=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0 Runtime

-=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0 1. The repagent module connects to SingleRephub on start= up.

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 2. r= epagent reports V1 and V2 to SingleRephub.

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 3. SingleRe= phub starts to perform an initial synchronization of the protected volumes-=

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 it reads each protected volume (= V1 and V2) - using read volume requests - and copies the data into the

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 recovery volume V1R and V2R.

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 4. SingleRephub e= nters 'protection' mode - each write to the protected volume is sen= t by the repagent to the Rephub,

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 and the Rephub performs the write on t= he matching recovery volume.

-

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 * Note that during stage 3= writes to the protected volumes are not ignored - they're kept in a bi= tmap,

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 and will be read again when stage 3 en= ds, in an interative convergin process.

-

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 This flow conti= nuously maintains an updated recovery volume.

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 If the prot= ected system is damaged, the user can create a new VM on Host2 with the rep= licated volumes attached to it.

-=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0 The new VM is a replica of the protected system.

+=A0=A0=A0 This section describes in high level a sa= mple Rephub - a replication system that uses the repagent API

+=A0=A0=A0 to replicate disks.

+= =A0=A0=A0 It describes a simple Rephub that comntinuously maintains a mirro= r of the volumes of a VM.

+

+=A0=A0=A0 Say we have a= VM we want to protect - call it PVM, say it has 2 volumes - V1, V2.

+=A0=A0=A0 Our Rephub is called SingleRephub - a Rephub= protecting a single VM.

+

+=A0=A0=A0 Preparations<= /p>

+=A0=A0=A0 1. The user chooses a host to rub Sing= leRephub - a different host than PVM, call it Host2

+=A0=A0=A0 2. The user creates two volumes on Host2 - same sizes of V1 a= nd V2, call them V1R (V1 recovery) and V2R.

+=A0=A0=A0 3. The user runs SingleRephub process on = Host2, and gives V1R and V2R as command line arguments.

+=A0=A0=A0=A0=A0=A0=A0 From now on SingleRephub waits for the protec= ted VM repagent to connect.

+=A0=A0=A0 4. The user runs the protected VM PVM - a= nd uses the switch -repagent <Host2 IP>.

+<= /p>

+=A0=A0=A0 Runtime

+=A0= =A0=A0 1. The repagent module connects to SingleRephub on startup.

+=A0=A0=A0 2. repagent reports V1 and V2 to SingleRe= phub.

+=A0=A0=A0 3. SingleRephub starts to perfor= m an initial synchronization of the protected volumes-

+=A0=A0=A0=A0=A0=A0=A0 it reads each protected volume (V1 and V2) - u= sing read volume requests - and copies the data into the

+=A0=A0=A0=A0=A0=A0=A0 recovery volume V1R and V2R.<= /p>

+=A0=A0=A0 4. SingleRephub enters 'protection= ' mode - each write to the protected volume is sent by the repagent to = the Rephub,

+=A0=A0=A0=A0=A0=A0=A0 and the Rephub performs the write on the matching re= covery volume.

+

+=A0= =A0=A0 * Note that during stage 3 writes to the protected volumes are not i= gnored - they're kept in a bitmap,

+=A0=A0=A0=A0=A0=A0=A0 and will be read again when s= tage 3 ends, in an interative convergin process.

= +

+=A0=A0=A0 This flow continuously maintains an = updated recovery volume.

+=A0=A0=A0 If the protected system is damaged, the u= ser can create a new VM on Host2 with the replicated volumes attached to it= .

+=A0=A0=A0 The new VM is a replica of the prote= cted system.

=A0

diff --git a/block/repagent/repagent.c b/block/repagent/repagent.c

index c3dd593..bdc0117 100644

--- a/block/repagent/repagent.c

+++ b/block/repagent/repagent.c

@@ -29,7 +29,7 @@ struct RepAgentState {

<= /p>

=A0typedef struct RepagentReadVolIo {

=A0=A0=A0=A0 QEMUIOVector qiov;

-=A0=A0=A0 RepCmd= ReadVolReq rep_cmd;

+=A0=A0=A0 RepCmdRemoteIoReq = rep_cmd;

=A0=A0=A0=A0 uint8_t *buf;

=A0=A0=A0=A0 struct timeval start_time;

} RepagentReadVolIo;

@@ -= 38,7 +38,7 @@ static int repagent_get_volume_by_driver(

=A0=A0=A0=A0=A0=A0=A0=A0 BlockDriverState *bs);

static int repagent_get_volume_by_name(const char *name);

static void repagent_report_volumes_to_hub(void);

-static void repagent_vol_read_done(void *opaque, = int ret);

+static void repagent_remote_io_done(vo= id *opaque, int ret);

static struct timeval tsub(struct timeval t1, struc= t timeval t2);

=A0Rep= AgentState g_rep_agent =3D { 0 };

@@ -242,15 +242= ,15 @@ static int repagent_get_volume_by_id(uint64_t vol_id)

=A0=A0=A0=A0 return -1;

}=

-int repaget_read_vo= l(RepCmdReadVolReq *pcmd, uint8_t *pdata)

+int re= pagent_remote_io(RepCmdRemoteIoReq *pcmd, uint8_t *pdata)

{

=A0=A0=A0=A0 int index = =3D repagent_get_volume_by_id(pcmd->volume_id);

=A0=A0=A0=A0 int size_bytes =3D pcmd->size_sectors * 512;

=A0=A0=A0=A0 if (index < 0) {

=A0=A0=A0=A0=A0=A0=A0=A0 printf("Vol read - Cou= ld not find vol id %llx\n",

=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 (unsigned long long int) pcmd->volume_= id);

-=A0=A0=A0=A0=A0=A0=A0 RepCmdReadVolRes *p_r= es_cmd =3D (RepCmdReadVolRes *) repcmd_new(

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 REPHU= B_CMD_READ_VOL_RES, 0, NULL);

+=A0=A0=A0=A0=A0=A0= =A0RepCmdRemoteIoRes *p_res_cmd =3D (RepCmdRemoteIoRes *) repcmd_new(

<= p class=3D"MsoNormal">+=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 REPHUB= _CMD_REMOTE_IO_RES, 0, NULL);

=A0=A0=A0=A0=A0=A0=A0=A0 p_res_cmd->req_id =3D pc= md->req_id;

=A0=A0=A0=A0=A0=A0=A0=A0 p_res_cmd= ->volume_id =3D pcmd->volume_id;

=A0=A0=A0= =A0=A0=A0=A0=A0 p_res_cmd->io_status =3D -1;

@@ -264,58 +264,67 @@ int repaget_read_vol(RepCmdReadVolReq *pcmd, uint8_t = *pdata)

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 (uns= igned long long int) pcmd->offset_sectors, pcmd->size_sectors);

=A0=A0=A0=A0=A0{

-=A0=A0= =A0=A0=A0=A0=A0 RepagentReadVolIo *read_xact =3D calloc(1, sizeof(RepagentR= eadVolIo));

+=A0=A0=A0=A0=A0=A0=A0 RepagentReadVo= lIo *io_xaction =3D calloc(1, sizeof(RepagentReadVolIo));

=A0/*=A0=A0=A0=A0=A0=A0= =A0 BlockDriverAIOCB *acb; */

-=A0=A0=A0=A0=A0=A0=A0 ZERO_MEM_OBJ(read_xact);

+=A0=A0=A0=A0=A0=A0=A0 ZERO_MEM_OBJ(io_xaction);

-=A0=A0=A0=A0=A0=A0=A0 q= emu_iovec_init(&read_xact->qiov, 1);

+=A0= =A0=A0=A0=A0=A0=A0 qemu_iovec_init(&io_xaction->qiov, 1);

=A0=A0=A0=A0=A0=A0=A0=A0=A0/*read_xact->buf =3D

=A0=A0=A0=A0=A0=A0=A0=A0 qemu_blockalign(g_rep_agent.volumes[index]->d= river_ptr, size_bytes); */

-=A0=A0=A0=A0=A0=A0=A0= read_xact->buf =3D (uint8_t *) g_malloc(size_bytes);

-=A0=A0=A0=A0=A0=A0=A0 read_xact->rep_cmd =3D *pc= md;

-=A0=A0=A0=A0=A0=A0=A0 qemu_iovec_add(&re= ad_xact->qiov, read_xact->buf, size_bytes);

+=A0=A0=A0=A0=A0=A0=A0 io_xaction->buf =3D (uint8_t *) g_malloc(size_by= tes);

+=A0=A0=A0=A0=A0=A0=A0 io_xaction->rep_cmd =3D *p= cmd;

+=A0=A0=A0=A0=A0=A0=A0 qemu_iovec_add(&i= o_xaction->qiov, io_xaction->buf, size_bytes);

-=A0=A0=A0=A0=A0=A0=A0 gettimeofday(&re= ad_xact->start_time, NULL);

+=A0=A0=A0=A0=A0=A0=A0 gettimeofday(&io_xaction-= >start_time, NULL);

=A0=A0=A0=A0=A0=A0=A0=A0 /= * orim TODO - use the returned acb to cancel the request on shutdown */

=

-=A0=A0=A0=A0=A0=A0=A0 /*acb =3D */bdrv_aio_readv(g_= rep_agent.volumes[index]->driver_ptr,

-=A0=A0 =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0read_= xact->rep_cmd.offset_sectors, &read_xact->qiov,

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 read_xact->rep_c= md.size_sectors, repagent_vol_read_done,

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 read_xact);

+=A0=A0=A0=A0=A0=A0=A0 /*acb =3D */

+= =A0=A0=A0=A0=A0=A0=A0 if (pcmd->is_read) {

+= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 bdrv_aio_readv(g_rep_agent.volumes[index]= ->driver_ptr,

+=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0 io_xaction->rep_cmd.offset_sectors, &io_xaction->qiov,

=

+=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0 io_xaction->rep_cmd.size_sectors, repagent_remote_io_done,

+=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 io_xaction);

=

+=A0=A0=A0=A0=A0=A0=A0 } else {

+=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 bdrv_aio_writev(g_rep_agent.volume= s[index]->driver_ptr,

+=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 io_xaction->rep_cmd.offset_sectors,= &io_xaction->qiov,

+=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0 io_xaction->rep_cmd.size_sectors, repagent_remote_io_done,

+=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0 io_xaction);

+=A0=A0=A0=A0=A0=A0=A0 }

=A0=A0=A0=A0 }

=A0=A0= =A0=A0=A0return TRUE;

}

-static void repagent_vol_read_done(void *opa= que, int ret)

+static void repagent_remote_io_done(void *opaque, int ret)

{

=A0=A0=A0=A0 struct timeval t2;

=

-=A0=A0=A0 RepagentReadVolIo *read_xact =3D (Repagen= tReadVolIo *) opaque;

+=A0=A0=A0 RepagentReadVolIo *io_xaction =3D (Repage= ntReadVolIo *) opaque;

=A0=A0=A0=A0 uint8_t *pdat= a =3D NULL;

-=A0=A0=A0 RepCmdReadVolRes *pcmd =3D= (RepCmdReadVolRes *) repcmd_new(

-=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 REPHUB_CMD_READ_V= OL_RES, read_xact->rep_cmd.size_sectors * 512,

+=A0=A0=A0 RepCmdRemoteIoRes *pcmd =3D (RepCmdRemoteIoRes *) repcmd_new(

+=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 REPHUB_CMD_REMO= TE_IO_RES, io_xaction->rep_cmd.size_sectors * 512,

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 &pdata);

-=A0=A0=A0 pcmd->req_id =3D read_xact->rep_cm= d.req_id;

-=A0=A0=A0 pcmd->volume_id =3D read_= xact->rep_cmd.volume_id;

+=A0=A0=A0 pcmd->req_id =3D io_xaction->rep_cmd.req_id;

+=A0=A0=A0 pcmd->volume_id =3D io_xaction->rep_cmd.vol= ume_id;

=A0=A0=A0=A0 pcmd->io_status =3D -1;

-=A0=A0=A0 printf("Protected vol read - volId %= llu, offset %llu, size %u\n",

-=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0 (unsigned long long int) read_xact->rep_cmd.volume= _id,

-=A0=A0=A0=A0=A0=A0=A0 =A0=A0=A0=A0(unsigned long long int) read_xact->r= ep_cmd.offset_sectors,

-=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0 read_xact->rep_cmd.size_sectors);

+= =A0=A0=A0 printf("Remote IO request - volId %llu, offset %llu, size %u= , is_read %u\n",

+=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 (unsigned long lo= ng int) io_xaction->rep_cmd.volume_id,

+=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0 (unsigned long long int) io_xaction->rep_cmd= .offset_sectors,

+=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0 io_xaction->rep_cmd.size_sectors,

+=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 io_xaction->re= p_cmd.is_read);

=A0=A0=A0=A0 gettimeofday(&t2= , NULL);

=A0=A0=A0=A0= =A0if (ret >=3D 0) {

=A0=A0=A0=A0=A0=A0=A0=A0 /* Read response - send the data to the hub */

=

-=A0=A0=A0=A0=A0=A0=A0 t2 =3D tsub(t2, read_xact->= ;start_time);

-=A0=A0=A0=A0=A0=A0=A0 printf("= ;Read prot vol done. Took %u seconds, %u us.",

+=A0=A0=A0=A0=A0=A0=A0 t2 =3D tsub(t2, io_xaction-&g= t;start_time);

+=A0=A0=A0=A0=A0=A0=A0 printf(&quo= t;Remote IO done. Took %u seconds, %u us.",

= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 (uint32_t) t2.tv_sec, (uin= t32_t) t2.tv_usec);

=A0=A0=A0=A0=A0=A0=A0=A0= =A0pcmd->io_status =3D 0;=A0=A0=A0 /* Success */

=A0=A0=A0=A0=A0=A0=A0=A0 /* orim todo optimize - don't copy, use the= qiov buffer */

-=A0=A0=A0=A0=A0=A0=A0 qemu_iovec= _to_buffer(&read_xact->qiov, pdata);

+=A0=A0=A0=A0=A0=A0=A0 qemu_iovec_to_buffer(&io_= xaction->qiov, pdata);

=A0=A0=A0=A0 } else {

=A0=A0=A0=A0=A0=A0=A0=A0 printf("readv failed= : %s\n", strerror(-ret));

=A0=A0=A0=A0 }

@@ -323,9 +332,9 @@ static void re= pagent_vol_read_done(void *opaque, int ret)

=A0= =A0=A0=A0 repagent_client_send((RepCmd *) pcmd);

=

=A0=A0=A0=A0=A0/*qemu_vfree(read_xact->buf); */

-=A0=A0=A0 g_free(read_xact->buf);

+=A0=A0= =A0 g_free(io_xaction->buf);

-=A0=A0=A0 g_free(read_xact);

+=A0=A0=A0 g_free(io_xaction);

}

=A0static str= uct timeval tsub(struct timeval t1, struct timeval t2)

diff --git a/block/repagent/repagent.h b/block/repagent/repagent.h

index 310db0f..0f69820 100644

--- a/block/repagent/repagent.h

+++ b/block/r= epagent/repagent.h

@@ -30,7 +30,7 @@

typedef struct RepAgentState RepAgentState;

typ= edef struct RepCmdStartProtect RepCmdStartProtect;

typedef struct RepCmdDataStartProtect RepCmdDataStartProtect;

-struct RepCmdReadVolReq;

+struct RepCmdRemoteIoR= eq;

=A0/* orim tempor= ary */

extern int use_repagent;

@@ -45,7 +45,7 @@ void repagent_deregister_drive(const char *drive_path,

=A0=A0=A0=A0=A0=A0=A0=A0 BlockDriverState *driver_p= tr);

int repaget_start_protect(RepCmdStartProtec= t *pcmd,

=A0=A0=A0=A0=A0=A0=A0=A0 RepCmdDataStartProtect *pcmd_data);

-int repaget_read_vol(struct RepCmdReadVolReq *pcmd, uint8_t *p= data);

+int repagent_remote_io(struct RepCmdRemot= eIoReq *pcmd, uint8_t *pdata);

void repagent_client_connected(void);

=A0

di= ff --git a/block/repagent/repagent_client.c b/block/repagent/repagent_clien= t.c

index 9ed8485..ee4aeb7 100644

--- a/block/repagen= t/repagent_client.c

+++ b/block/repagent/repagent= _client.c

@@ -125,8 +125,8 @@ void repagent_proce= ss_cmd(RepCmd *pcmd, uint8_t *pdata, void *clientPtr)

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 (Re= pCmdDataStartProtect *) pdata);

=A0=A0=A0=A0 }

=A0=A0=A0=A0=A0=A0=A0=A0 break;

-=A0=A0=A0 case REPHUB_CMD_READ_VOL_REQ: {

-=A0=A0=A0=A0=A0=A0=A0 is_free_data =3D repaget_read_vol((RepCmdReadVolReq = *) pcmd, pdata);

+=A0=A0=A0 case REPHUB_CMD_REMOT= E_IO_REQ: {

+=A0=A0=A0=A0=A0=A0=A0 is_free_data = =3D repagent_remote_io((RepCmdRemoteIoReq *) pcmd, pdata);

=A0=A0=A0=A0 }

=A0=A0=A0= =A0=A0=A0=A0=A0 break;

=A0=A0=A0=A0 default:

<= p class=3D"MsoNormal">diff --git a/block/repagent/repcmd.h b/block/repagent= /repcmd.h

index 8c6cf1b..0a7f297 100644

--- a/block/repagent/repcmd.h

+++ b/block/repagent/repcmd.h

@@ -36,8 +36,8 = @@ enum RepCmds {

=A0=A0=A0=A0 REPHUB_CMD_PROTECT= ED_WRITE=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 =3D 2,

=A0=A0=A0=A0 REPHUB_CMD_REPORT_VM_VOLUMES=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0 =3D 3,

=A0=A0=A0=A0 REPH= UB_CMD_START_PROTECT=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 =3D 4,

-=A0=A0=A0 REPHUB_CMD_READ_VOL_REQ=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 =3D 5,

-=A0=A0=A0 REPHUB_CMD_READ_VOL_RES=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 =3D 6,

+=A0=A0=A0 = REPHUB_CMD_REMOTE_IO_REQ=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 = =3D 5,

+=A0=A0=A0 REPHUB_CMD_REMOTE_IO_RES=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 =3D 6,

=A0=A0=A0=A0 REPHUB_CMD_AGENT_SHUTDOWN=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 =3D 7,

};

diff --git a/block/repagent/= rephub_cmds.h b/block/repagent/rephub_cmds.h

index d1bad06..cb737e6 100644

--- a/block/repagent/rephub_cmds.h

+++ b/bloc= k/repagent/rephub_cmds.h

@@ -97,40 +97,43 @@ type= def struct RepCmdDataStartProtect {

=A0

=A0/*********************************************************

- * RepCmd Read Volume Request

+ * RepCmd Remote IO Request

=A0 *

- * REPHUB_CMD_READ_= VOL_REQ

+ * REPHUB_CMD_REMOTE_IO_REQ

=A0 * Direction: hub->agent

=A0= *

- * The hub issues a read IO to a protected volume.

- * This command is used during sync - when the hub needs

- * to read unsyncronized sections of a protected volume.

+ * The hub issues an IO to a volume.

+ * This co= mmand is used for:

+ * - Reading protected volume= during sync

+ * - Read/write to a recovery volum= e during

+ *=A0=A0=A0=A0 protect and failover or failover tes= t

=A0 * This command is a request, the read data = is returned

- * by the response command REPHUB_CM= D_READ_VOL_RES

+ * by the response command REPHUB_CMD_REMOTE_IO_RES=

=A0 ********************************************= *************/

-typedef struct RepCmdReadVolReq {=

+typedef struct RepCmdRemoteIoReq {

=A0=A0=A0=A0 RepCmdHdr hdr;

=A0=A0=A0= =A0 int req_id;

=A0=A0=A0=A0 int size_sectors;

=A0=A0=A0=A0 uint64_t volume_id;

=A0=A0=A0=A0 uint64_t offset_sectors;

-} RepCmdReadVolReq;

+=A0=A0=A0 int i= s_read;

+} RepCmdRemoteIoReq;

=A0/*********************************************************

=A0 * RepCmd Read Volume Response

= =A0 *

- * REPHUB_CMD_READ_VOL_RES

+ * REPHUB_CMD_REMOTE_IO_RES

=A0 * Direction: agent->hub

=A0 *

- * A response to REPHUB_CMD_READ_VOL_= REQ.

+ * A response to REPHUB_CMD_REMOTE_IO_REQ.<= /p>

=A0 * Sends the data read from a protected volume

=A0 *********************************************************/

-typedef struct RepCmdReadVolRes {

+typedef struct RepCmdRemoteIoRes {

=A0=A0=A0=A0 RepCmdHdr hdr;

=A0=A0=A0=A0 int req_id;

=A0=A0=A0=A0 int io_st= atus;

=A0=A0=A0=A0 uint64_t volume_id;

-} RepCmdReadVolRes;

+} RepCmdRemoteIoRes;

=A0/***********************************************= **********

=A0 * RepCmd Agent shutdown

--

1.7.6.5

--20cf300fafbf9abfd604bced88b9--