[PATCH] tools/python/xend: VBD QoS policy bits

[PATCH] tools/python/xend: VBD QoS policy bits

[William Pitcock]

Add the ability to define VBD QoS policy in the xend layer.

Consider the following vbd entry:

vbd = [
   'phy:/dev/server/virtualmachine1-disk,xvda1,w,credit=5000/s@50ms',
]

This means that a VM may perform 5000 I/O operations per second, with
credit being replenished every 50 milliseconds.

The 'credit' xenstore value is by the blkback driver to ratelimit I/O
operations for the specific device.

Signed-off-by: William Pitcock <nenolod@dereferenced.org>

---

 README.sxpcfg            |    1 +
 xen/xend/server/blkif.py |   43 ++++++++++++++++++++++++++++++++++++++++---
 2 files changed, 41 insertions(+), 3 deletions(-)

diff --git a/tools/python/README.sxpcfg b/tools/python/README.sxpcfg
--- a/tools/python/README.sxpcfg
+++ b/tools/python/README.sxpcfg
@@ -82,6 +82,7 @@
   - uname
   - dev (ioemu:, .. etc)
   - mode (r, w. w!)
+  - credit
 
 vif
   - type
diff --git a/tools/python/xen/xend/server/blkif.py b/tools/python/xen/xend/server/blkif.py
--- a/tools/python/xen/xend/server/blkif.py
+++ b/tools/python/xen/xend/server/blkif.py
@@ -42,6 +42,37 @@
 
         return os.access(auxbin.scripts_dir() + '/block-%s' % protocol, os.X_OK)
 
+    def _calculateRateLimit(self, credstr):
+        """Calculate the rate limit, given a string like: 5000/s@50ms.
+        If this fails, the limit is unlimited.
+        """
+        credit_per_interval = 0xffffffffL
+        interval_usecs = 0L
+
+        credit_re = re.compile("^([0-9]+)/s(@([0-9]+)([mu]?)s)?$")
+
+        m = credit_re.match(credstr)
+        if m:
+            credit_per_sec = m.group(1)
+
+            if m.group(2) is None:
+                interval_usecs = 50000L      # 50ms default
+            else:
+                interval_usecs = long(m.group(5))
+                if m.group(3) == '':
+                    interval_usecs *= 1000 * 1000
+                elif m.group(3) == 'm':
+                    interval_usecs *= 1000
+
+            credit_per_interval = (credit_per_sec * interval_usecs) / 1000000L
+
+            # overflow / underflow checking: default to unlimited rate
+            if credit_per_interval == 0 or credit_per_interval > 0xffffffffL or \
+               interval_usecs == 0 or interval_usecs > 0xffffffffL:
+                credit_per_interval = 0xffffffffL
+                interval_usecs     = 0L
+
+        return "%lu,%lu" % (credit_per_interval, interval_usecs)
 
     def getDeviceDetails(self, config):
         """@see DevController.getDeviceDetails"""
@@ -91,6 +122,10 @@
         if security.on() == xsconstants.XS_POLICY_USE:
             self.do_access_control(config, uname)
 
+        cred = config.get('credit', '')
+        if cred:
+            back['credit'] = self._calculateRateLimit(cred)
+
         (device_path, devid) = blkif.blkdev_name_to_number(dev)
         if devid is None:
             raise VmError('Unable to find number for device (%s)' % (dev))
@@ -153,12 +188,12 @@
         config = DevController.getDeviceConfiguration(self, devid, transaction)
         if transaction is None:
             devinfo = self.readBackend(devid, 'dev', 'type', 'params', 'mode',
-                                       'uuid', 'bootable')
+                                       'uuid', 'bootable', 'credit')
         else:
             devinfo = self.readBackendTxn(transaction, devid,
                                           'dev', 'type', 'params', 'mode', 'uuid',
-                                          'bootable')
-        dev, typ, params, mode, uuid, bootable = devinfo
+                                          'bootable', 'credit')
+        dev, typ, params, mode, uuid, bootable, credit = devinfo
         
         if dev:
             if transaction is None:
@@ -178,6 +213,8 @@
             config['uuid'] = uuid
         if bootable != None:
             config['bootable'] = int(bootable)
+        if credit:
+            config['credit'] = credit
 
         proto = self.readFrontend(devid, 'protocol')
         if proto:

Re: [PATCH] tools/python/xend: VBD QoS policy bits

[Christoph Egger]

On Friday 14 August 2009 11:48:47 William Pitcock wrote:
> Add the ability to define VBD QoS policy in the xend layer.
>
> Consider the following vbd entry:
>
> vbd = [
>    'phy:/dev/server/virtualmachine1-disk,xvda1,w,credit=5000/s@50ms',
> ]
>
> This means that a VM may perform 5000 I/O operations per second, with
> credit being replenished every 50 milliseconds.

Does this enforce the VBD to do 5000 IO operations or none at all or is
this considered as an upper limit ?

> The 'credit' xenstore value is by the blkback driver to ratelimit I/O
> operations for the specific device.
>
> Signed-off-by: William Pitcock <nenolod@dereferenced.org>




-- 
---to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen
Geschaeftsfuehrer: Thomas M. McCoy, Giuliano Meroni
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632

Re: [PATCH] tools/python/xend: VBD QoS policy bits

[William Pitcock]

Hi,

----- "Christoph Egger" <Christoph.Egger@amd.com> wrote:

> Does this enforce the VBD to do 5000 IO operations or none at all or
> is
> this considered as an upper limit ?

That is dependent on the dom0 kernel, obviously... but my kernel-side
patch basically keeps the specified IOPS target as the upper limit, while
also allowing some level of initial burstability.

The goal here is to stop domUs which have gone swap-happy from degrading
system performance (especially with say, SANs) while the domU's owner is
not around to fix the problem... basically instead of the entire storage
volume's resources being consumed, the requests going to the physical volume
become ratelimited.

I submitted some patches earlier (around March, then May) which worked a
little bit differently, and feel that this way is more self-explanatory and
generally robust.

William