Re: L3 cache way-locking memory zone

From: Marek Bykowski <marek.bykowski@gmail.com>
To: Sudeep Holla <sudeep.holla@arm.com>
Cc: linux-arm-kernel@lists.infradead.org
Subject: Re: L3 cache way-locking memory zone
Date: Tue, 16 Apr 2019 12:32:16 +0200	[thread overview]
Message-ID: <20190416123216.04a89bd3@gmail.com> (raw)
In-Reply-To: <20190415140645.GA24249@e107155-lin>

Hi Sudeep,

To me it is quite generic but I don't have an experience working on a
broad range of ARM products. 

It works more less along this: one needs to set up an SMC service
allowing querying the L3 way-locking registers/configuration from
within kernel. The L3 way-locking is configured itself statically in
SPL Uboot running EL3, secure state. The static configuration is a
result of the constraint on when and how the registers can be accessed,
namely "normal"/memory access to HN-F cannot be in-flight while the
write to these registers occur - to my best knowledge we cannot meet
that with the system/Linux up and running. The secure state results
from the L3 lock registers being only accessed from the secure. 

After then when Linux boots it does the SMC call to query whether there
is the L3 way-locking in place and what base addresses and the sizes
and it sets up the zone upon it.

The following are some confs/printouts from the Linux having 2G of RAM,
and 24M of LLC with 16M configured to locked:

zone_size[0] 131072 zhole_size[0] 0
zone_size[1] 4096 zhole_size[1] 0
zone_size[2] 389120 zhole_size[2] 0
zone_size[3] 0 zhole_size[3] 0

L3LOCKED zone: 64 pages used for memmap
L3LOCKED zone: 4096 pages, LIFO batch:0

mem_init() DMA->managed_pages 129024
mem_init() L3LOCK->managed_pages 4032
mem_init() Normal->managed_pages 383040
mem_init() Movable->managed_pages 0

And after releasing free pages to the buddy allocator

mem_init() DMA->managed_pages 123787
mem_init() L3LOCK->managed_pages 4096
mem_init() Normal->managed_pages 379305
mem_init() Movable->managed_pages 0

# cat /proc/buddyinfo (reformatted to squeeze)
Node 0, zone DMA    4  4  3  4  2  3  2  4  2  3  122 
Node 0, zone L3LOCK 0  0  0  0  0  0  0  0  0  0  4  
Node 0, zone Normal 21 3  7  2  2  1  1  1  1  2  306 

# cat /proc/slabinfo
[...]
Node 0, zone    L3LOCK
  pages free     4096
        min      135
        low      168
        high     201
   node_scanned  0
        spanned  4096
        present  4096
        managed  4096

Then we can allocate to the L3LOCK zone with 'kmalloc(size,
GFP_L3LOCK)'. 

We have used the L3LOCK to benchmark the DMA that presented the better
results when copying from L3 locked to L3 locked than from "unlocked"
to "unlocked".

Thanks,
Marek

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel