[PATCH 00/10 V7] Remus/Libxl: Network buffering support

[Lai Jiangshan <laijs@cn.fujitsu.com>]

This patch series adds support for network buffering in the Remus
codebase in libxl. 

Changes in V7:
  Applied missing comments(by IanJ).
  Applied Shriram comments.

  merge netbufering tangled setup/teardown code into one patch.
  (2/6/8 in V6 => 5 in V7. 9/10 in V6 => 7 in V7)

Changes in V6:
  Applied Ian Jackson's comments of V5 series.
  the [PATCH 2/4 V5] is split by small functionalities.

  [PATCH 4/4 V5] --> [PATCH 13/13] netbuffer is default enabled.

Changes in V5:

Merge hotplug script patch (2/5) and hotplug script setup/teardown
patch (3/5) into a single patch.

Changes in V4:

[1/5] Remove check for libnl command line utils in autoconf checks

[2/5] minor nits

[3/5] define LIBXL_HAVE_REMUS_NETBUF in libxl.h

[4/5] clean ups. Make the usleep in checkpoint callback asynchronous

[5/5] minor nits

Changes in V3:
[1/5] Fix redundant checks in configure scripts
      (based on Ian Campbell's suggestions)

[2/5] Introduce locking in the script, during IFB setup.
      Add xenstore paths used by netbuf scripts
      to xenstore-paths.markdown

[3/5] Hotplug scripts setup/teardown invocations are now asynchronous
      following IanJ's feedback.  However, the invocations are still
      sequential. 

[5/5] Allow per-domain specification of netbuffer scripts in xl remus
      commmand.

And minor nits throughout the series based on feedback from
the last version

Changes in V2:
[1/5] Configure script will automatically enable/disable network
      buffer support depending on the availability of the appropriate
      libnl3 version. [If libnl3 is unavailable, a warning message will be
      printed to let the user know that the feature has been disabled.]

      use macros from pkg.m4 instead of pkg-config commands
      removed redundant checks for libnl3 libraries.

[3,4/5] - Minor nits.

Version 1:

[1/5] Changes to autoconf scripts to check for libnl3. Add linker flags
      to libxl Makefile.

[2/5] External script to setup/teardown network buffering using libnl3's
      CLI. This script will be invoked by libxl before starting Remus.
      The script's main job is to bring up an IFB device with plug qdisc
      attached to it.  It then re-routes egress traffic from the guest's
      vif to the IFB device.

[3/5] Libxl code to invoke the external setup script, followed by netlink
      related setup to obtain a handle on the output buffers attached
      to each vif.

[4/5] Libxl interaction with network buffer module in the kernel via
      libnl3 API.

[5/5] xl cmdline switch to explicitly enable network buffering when
      starting remus.


  Few things to note(by shriram): 

    a) Based on previous email discussions, the setup/teardown task has
    been moved to a hotplug style shell script which can be customized as
    desired, instead of implementing it as C code inside libxl.

    b) Libnl3 is not available on NetBSD. Nor is it available on CentOS
   (Linux).  So I have made network buffering support an optional feature
   so that it can be disabled if desired.

   c) NetBSD does not have libnl3. So I have put the setup script under
   tools/hotplug/Linux folder.

thanks
Lai



Shriram Rajagopalan (8):
  remus: add libnl3 dependency to autoconf scripts
  tools/libxl: update libxl_domain_remus_info
  tools/libxl: introduce a new structure libxl__remus_state
  remus: introduce a function to check whether network buffering is
    enabled
  remus: Remus network buffering core and APIs to setup/teardown
  remus: implement the APIs to buffer/release packages
  libxl: use the APIs to setup/teardown network buffering
  libxl: rename remus_failover_cb() to remus_replication_failure_cb()
  libxl: control network buffering in remus callbacks
  libxl: network buffering cmdline switch

 README                                 |    4 +
 config/Tools.mk.in                     |    3 +
 docs/man/xl.conf.pod.5                 |    6 +
 docs/man/xl.pod.1                      |   11 +-
 docs/misc/xenstore-paths.markdown      |    4 +
 tools/configure.ac                     |   15 +
 tools/hotplug/Linux/Makefile           |    1 +
 tools/hotplug/Linux/remus-netbuf-setup |  183 +++++++++++
 tools/libxl/Makefile                   |   11 +
 tools/libxl/libxl.c                    |   48 ++-
 tools/libxl/libxl.h                    |   13 +
 tools/libxl/libxl_dom.c                |  118 ++++++--
 tools/libxl/libxl_internal.h           |   54 +++-
 tools/libxl/libxl_netbuffer.c          |  561 ++++++++++++++++++++++++++++++++
 tools/libxl/libxl_nonetbuffer.c        |   56 ++++
 tools/libxl/libxl_remus.c              |   64 ++++
 tools/libxl/libxl_types.idl            |    2 +
 tools/libxl/xl.c                       |    4 +
 tools/libxl/xl.h                       |    1 +
 tools/libxl/xl_cmdimpl.c               |   28 ++-
 tools/libxl/xl_cmdtable.c              |    3 +
 tools/remus/README                     |    6 +
 22 files changed, 1155 insertions(+), 41 deletions(-)
 create mode 100644 tools/hotplug/Linux/remus-netbuf-setup
 create mode 100644 tools/libxl/libxl_netbuffer.c
 create mode 100644 tools/libxl/libxl_nonetbuffer.c
 create mode 100644 tools/libxl/libxl_remus.c