Re: [PATCH] build/utils: Add BB_TASK_IONICE_LEVEL support

["Avery, Brian" <brian.avery@intel.com>]

Hi,

It mostly appears as CFQ scheduler (Completely Fair Queuing).  Since we tell them to set it to cfq, it might be nice to tell them to do this by doing sudo sh -c “echo cfq > /sys/block/<device>/queu/scheduler; where device is the backing block device e.g. sda,sdb,…

-b

From: Scott Rifenbark <srifenbark@gmail.com<mailto:srifenbark@gmail.com>>
Date: Tuesday, October 27, 2015 at 6:34 AM
To: Richard Purdie <richard.purdie@linuxfoundation.org<mailto:richard.purdie@linuxfoundation.org>>
Cc: bitbake-devel <bitbake-devel@lists.openembedded.org<mailto:bitbake-devel@lists.openembedded.org>>, Brian Avery <brian.avery@intel.com<mailto:brian.avery@intel.com>>, "Witt, Randy E" <randy.e.witt@intel.com<mailto:randy.e.witt@intel.com>>, "Flanagan, Elizabeth" <elizabeth.flanagan@intel.com<mailto:elizabeth.flanagan@intel.com>>, "Graydon, Tracy" <tracy.graydon@intel.com<mailto:tracy.graydon@intel.com>>, Michael Halstead <michael@yoctoproject.org<mailto:michael@yoctoproject.org>>
Subject: Re: [PATCH] build/utils: Add BB_TASK_IONICE_LEVEL support

Hi,

See http://www.yoctoproject.org/docs/2.0/bitbake-user-manual/bitbake-user-manual.html#var-BB_TASK_IONICE_LEVEL for the new variable description in the BitBake User Manual.  Note that I did not include the future implementation information at the bottom of Richard's explanation.  Also, there is a bit about the 'cfq scheduler' near the end.  I did not know how to properly format this string (e.g. CFG Scheduler).

Let me know of any changes or fixes.

Thanks,
Scott

On Sat, Oct 24, 2015 at 3:19 AM, Richard Purdie <richard.purdie@linuxfoundation.org<mailto:richard.purdie@linuxfoundation.org>> wrote:
Similarly to BB_TASK_NICE_LEVEL, add BB_TASK_IONICE_LEVEL which allows the ioprio
of tasks to be adjusted. This is in response to various qemu runtime timeouts
which have been witnessed on the autobuilder, seemingly due to IO starvation (we
already use NICE_LEVEL to adjust tasks). This has a fairly urgent need to deal
with certain 'random' failures we're seeing on the autobuilders in testing.

The format of the data in the variable is BB_TASK_IONICE_LEVEL = "<class>.<prio>".

For <class>, 2 is best effort (the default), 1 is real time and 3 is idle. You'd
need superuser privileges to use realtime. The <prio> value is a default of 4,
and can be set between 0 and 7 with 7 being lowest priority and 0 the highest.
The user can set this freely with normal privileges

Note that in order for this to take effect, you need the cfq scheduler selected
for the backing block device.

We could use nice wrapper functions for ioprio from modules like psutil however
that would complicate bitbake dependencies. This version has some magic numbers
but works on the main 32 and 64 bit x86 build architectures and can easily be
extended if ever needed. When we move to python 3.x, we can likely replace this
with standard calls.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org<mailto:richard.purdie@linuxfoundation.org>>

diff --git a/bitbake/lib/bb/build.py b/bitbake/lib/bb/build.py
index 948c395..22428a6 100644
--- a/bitbake/lib/bb/build.py
+++ b/bitbake/lib/bb/build.py
@@ -413,6 +413,13 @@ def _exec_task(fn, task, d, quieterr):
         nice = int(nice) - curnice
         newnice = os.nice(nice)
         logger.debug(1, "Renice to %s " % newnice)
+    ionice = localdata.getVar("BB_TASK_IONICE_LEVEL", True)
+    if ionice:
+        try:
+            cls, prio = ionice.split(".", 1)
+            bb.utils.ioprio_set(os.getpid(), int(cls), int(prio))
+        except:
+            bb.warn("Invalid ionice level %s" % ionice)

     bb.utils.mkdirhier(tempdir)

diff --git a/bitbake/lib/bb/utils.py b/bitbake/lib/bb/utils.py
index eb219c1..86a5328 100644
--- a/bitbake/lib/bb/utils.py
+++ b/bitbake/lib/bb/utils.py
@@ -1310,3 +1310,27 @@ def signal_on_parent_exit(signame):
     result = cdll['libc.so.6'].prctl(PR_SET_PDEATHSIG, signum)
     if result != 0:
         raise PrCtlError('prctl failed with error code %s' % result)
+
+#
+# Manually call the ioprio syscall. We could depend on other libs like psutil
+# however this gets us enough of what we need to bitbake for now without the
+# dependency
+#
+_unamearch = os.uname()[4]
+IOPRIO_WHO_PROCESS = 1
+IOPRIO_CLASS_SHIFT = 13
+
+def ioprio_set(who, cls, value):
+    NR_ioprio_set = None
+    if _unamearch == "x86_64":
+      NR_ioprio_set = 251
+    elif _unamearch[0] == "i" and _unamearch[2:3] == "86":
+      NR_ioprio_set = 289
+
+    if NR_ioprio_set:
+        ioprio = value | (cls << IOPRIO_CLASS_SHIFT)
+        rc = cdll['libc.so.6'].syscall(NR_ioprio_set, IOPRIO_WHO_PROCESS, who, ioprio)
+        if rc != 0:
+            raise ValueError("Unable to set ioprio, syscall returned %s" % rc)
+    else:
+        bb.warn("Unable to set IO Prio for arch %s" % _unamearch)