[PATCH v10 0/5] Remus netbuffer: Network buffering support

[Yang Hongyang <yanghy@cn.fujitsu.com>]

This patch series adds support for network buffering in the Remus
codebase in libxl.

This is a rebased version of v10 series.The first patch was applied,
but can not found on current master, if it's not necessary, please
let me know.

the code is also hosted on github:

url: https://github.com/laijs/xen
branch: remus-0605

Changes V10:
  Restructured the whole patch series.
  Introduce the remus device abstract layer.
  Make remus checkpoint asynchronous.

Changes in V9:
  Use async exec script api to exec scripts.

Changes in V8:
  Applied some comments(by IanJ).
  Merge some struct definitions to it's implementation.
  (2/3/5 in V7 => 3 in V8)

Changes in V7:
  Applied missing comments(by IanJ).
  Applied Shriram comments.

  merge netbufering tangled setup/teardown code into one patch.
  (2/6/8 in V6 => 5 in V7. 9/10 in V6 => 7 in V7)

Changes in V6:
  Applied Ian Jackson's comments of V5 series.
  the [PATCH 2/4 V5] is split by small functionalities.

  [PATCH 4/4 V5] --> [PATCH 13/13] netbuffer is default enabled.

Changes in V5:

Merge hotplug script patch (2/5) and hotplug script setup/teardown
patch (3/5) into a single patch.

Changes in V4:

[1/5] Remove check for libnl command line utils in autoconf checks

[2/5] minor nits

[3/5] define LIBXL_HAVE_REMUS_NETBUF in libxl.h

[4/5] clean ups. Make the usleep in checkpoint callback asynchronous

[5/5] minor nits

Changes in V3:
[1/5] Fix redundant checks in configure scripts
      (based on Ian Campbell's suggestions)

[2/5] Introduce locking in the script, during IFB setup.
      Add xenstore paths used by netbuf scripts
      to xenstore-paths.markdown

[3/5] Hotplug scripts setup/teardown invocations are now asynchronous
      following IanJ's feedback.  However, the invocations are still
      sequential. 

[5/5] Allow per-domain specification of netbuffer scripts in xl remus
      commmand.

And minor nits throughout the series based on feedback from
the last version

Changes in V2:
[1/5] Configure script will automatically enable/disable network
      buffer support depending on the availability of the appropriate
      libnl3 version. [If libnl3 is unavailable, a warning message will be
      printed to let the user know that the feature has been disabled.]

      use macros from pkg.m4 instead of pkg-config commands
      removed redundant checks for libnl3 libraries.

[3,4/5] - Minor nits.

Version 1:

[1/5] Changes to autoconf scripts to check for libnl3. Add linker flags
      to libxl Makefile.

[2/5] External script to setup/teardown network buffering using libnl3's
      CLI. This script will be invoked by libxl before starting Remus.
      The script's main job is to bring up an IFB device with plug qdisc
      attached to it.  It then re-routes egress traffic from the guest's
      vif to the IFB device.

[3/5] Libxl code to invoke the external setup script, followed by netlink
      related setup to obtain a handle on the output buffers attached
      to each vif.

[4/5] Libxl interaction with network buffer module in the kernel via
      libnl3 API.

[5/5] xl cmdline switch to explicitly enable network buffering when
      starting remus.


  Few things to note(by shriram): 

    a) Based on previous email discussions, the setup/teardown task has
    been moved to a hotplug style shell script which can be customized as
    desired, instead of implementing it as C code inside libxl.

    b) Libnl3 is not available on NetBSD. Nor is it available on CentOS
   (Linux).  So I have made network buffering support an optional feature
   so that it can be disabled if desired.

   c) NetBSD does not have libnl3. So I have put the setup script under
   tools/hotplug/Linux folder.

thanks

Shriram Rajagopalan (1):
  libxl: network buffering cmdline switch

Yang Hongyang (4):
  libxl: introduce asynchronous execution API
  remus: add libnl3 dependency for network buffering support
  remus: introduce remus device
  remus: implement remus network buffering for nic devices

 README                                 |   4 +
 config/Tools.mk.in                     |   4 +
 docs/man/xl.conf.pod.5                 |   6 +
 docs/man/xl.pod.1                      |  11 +-
 docs/misc/xenstore-paths.markdown      |   4 +
 tools/configure.ac                     |  15 +
 tools/hotplug/Linux/Makefile           |   1 +
 tools/hotplug/Linux/remus-netbuf-setup | 183 +++++++++++
 tools/libxl/Makefile                   |  15 +
 tools/libxl/libxl.c                    |  52 +++-
 tools/libxl/libxl.h                    |  13 +
 tools/libxl/libxl_aoutils.c            |  89 ++++++
 tools/libxl/libxl_device.c             |  78 ++---
 tools/libxl/libxl_dom.c                | 132 +++++++-
 tools/libxl/libxl_internal.h           | 151 ++++++++-
 tools/libxl/libxl_netbuffer.c          | 550 +++++++++++++++++++++++++++++++++
 tools/libxl/libxl_nonetbuffer.c        |  98 ++++++
 tools/libxl/libxl_remus_device.c       | 323 +++++++++++++++++++
 tools/libxl/libxl_save_msgs_gen.pl     |   2 +-
 tools/libxl/libxl_types.idl            |   3 +
 tools/libxl/xl.c                       |   4 +
 tools/libxl/xl.h                       |   1 +
 tools/libxl/xl_cmdimpl.c               |  28 +-
 tools/libxl/xl_cmdtable.c              |   3 +
 tools/remus/README                     |   6 +
 25 files changed, 1694 insertions(+), 82 deletions(-)
 create mode 100644 tools/hotplug/Linux/remus-netbuf-setup
 create mode 100644 tools/libxl/libxl_netbuffer.c
 create mode 100644 tools/libxl/libxl_nonetbuffer.c
 create mode 100644 tools/libxl/libxl_remus_device.c

-- 
1.9.1