Re: [PATCH RFC] ioctl based CAT interface

[Luiz Capitulino <lcapitulino@redhat.com>]

On Mon, 16 Nov 2015 17:26:57 +0100
Peter Zijlstra <peterz@infradead.org> wrote:

> On Mon, Nov 16, 2015 at 11:18:42AM -0500, Luiz Capitulino wrote:
> > Peter, I'm giving a serious try on the cgroups patches and would be
> > glad to be enlightened if I'm missing something. But I don't see how
> > what you're proposing would solve the problem.
> > 
> > My understanding of CAT is that if I want to reserve 80% of the cache
> > in socket-1 to $thread-A I also have to:
> > 
> >  1. Create another mask reserving 20% of the cache in socket-1
> >  2. Assign that mask to all other threads that may run in socket-1
> > 
> > If I'm right about this, then when a task with 20% reservation migrates
> > to socket-2 it will only access 20% of the cache there even though there
> > should be no restrictions in socket-2's cache.
> 
> Uh what? Task-A was bound to socket-1, it will never get to socket-2.

Sure, but you're going to allow other threads besides Task-A to execute
in socket-1 too, right? In this case, my understanding of CAT is that
those threads will require a mask to restrict them to only 20% of the
cache.

> Clearly I'm not getting these examples you're throwing around.

It's the "Isolated Bitmask" example from the Intel Manual p. 616
(You'll find the excerpt below, but Figure 17-27 is easier to understand).

My understanding of this example is that in order to isolate a portion
of the cache to Task-A I also have to create a second mask which excludes
bits reserved to Task-A. This second mask is assigned to all other tasks
that will share the L3 cache with Task-A.

"""
Figure 17-27 also shows three examples of sets of Cache Capacity Bitmasks. For simplicity these are represented
as 8-bit vectors, though this may vary depending on the implementation and how the mask is mapped to the avail-
able cache capacity. The first example shows the default case where all 4 Classes of Service (the total number of
COS are implementation-dependent) have full access to the cache. The second case shows an overlapped case,
which would allow some lower-priority threads share cache space with the highest priority threads. The third case
shows various non-overlapped partitioning schemes. As a matter of software policy for extensibility COS0 should
typically be considered and configured as the highest priority COS, followed by COS1, and so on, though there is
no hardware restriction enforcing this mapping. When the system boots all threads are initialized to COS0, which
has full access to the cache by default.
"""

> Also, I explicitly do not want tasks that can migrate between sockets to
> have different performance profiles across those sockets.

I think we can solve this problem with the ioctl interface, if that's
what you mean.