Re: [PATCH RFC] ioctl based CAT interface

[Marcelo Tosatti <mtosatti@redhat.com>]

On Mon, Nov 16, 2015 at 10:07:56AM +0100, Peter Zijlstra wrote:
> On Fri, Nov 13, 2015 at 03:33:04PM -0200, Marcelo Tosatti wrote:
> > On Fri, Nov 13, 2015 at 05:51:00PM +0100, Peter Zijlstra wrote:
> > > On Fri, Nov 13, 2015 at 02:39:33PM -0200, Marcelo Tosatti wrote:
> > > > + * 	* one tcrid entry can be in different locations 
> > > > + * 	  in different sockets.
> > > 
> > > NAK on that without cpuset integration.
> > > 
> > > I do not want freely migratable tasks having radically different
> > > performance profiles depending on which CPU they land.
> > 
> > Ok, so, configuration:
> > 
> > 
> > Socket-1				Socket-2
> > 
> > pinned thread-A with			100% L3 free
> > 80% of L3 
> > reserved
> > 
> > 
> > So it is a problem if a thread running on socket-2 is scheduled to 
> > socket-1 because performance is radically different, fine.
> > 
> > Then one way to avoid that is to not allow freely migratable tasks
> > to move to Socket-1. Fine.
> > 
> > Then you want to use cpusets for that.
> > 
> > Can you fill in the blanks what is missing here?
> 
> I'm still not seeing what the problem with CAT-cgroup is.
> 
> /cgroups/cpuset/
>   socket-1/cpus = $socket-1
>            tasks = $thread-A
> 
>   socket-2/cpus = $socket-2
>            tasks = $thread-B
> 
> /cgroups/cat/
>   group-A/bitmap = 0x3F / 0xFF
>   group-A/tasks = $thread-A
> 
>   group-B/bitmap = 0xFF / 0xFF
>   group-B/tasks = $thread-B
> 
> 
> That gets you thread-A on socket-1 with 6/8 of the L3 and thread-B on
> socket-2 with 8/8 of the L3.

Going that route, might as well expose the region shared with HW
to userspace and let userspace handle the problem of contiguous free regions,
which means the cgroups bitmask maps one-to-one to HW bitmap.

All is necessary then is to modify the Intel patches to

1) Support bitmaps per socket.
2) Remove hierarchical support.

3) Lazy enforcement (which can be done later as an improvement).