Re: [PATCH v13 2/2] x86/sgx: Add an attribute for the amount of SGX memory in a NUMA node

[Jarkko Sakkinen <jarkko@kernel.org>]

On Wed, 2021-12-08 at 11:38 -0800, Dave Hansen wrote:
> I reworked the changelog quite a bit, addressing some of Borislav's
> questions.  No code changes, though.
> 
> The result is below.  I've retained Greg's ack.  I'll stick this in
> tip/x86/sgx if this looks OK to everyone.
> 
> ---
> 
> From: Jarkko Sakkinen <jarkko@kernel.org>
> 
> == Problem ==
> 
> The amount of SGX memory on a system is determined by the BIOS and it
> varies wildly between systems.  It can be as small as dozens of MB's
> and as large as many GB's on servers.  Just like how applications need
> to know how much regular RAM is available, enclave builders need to
> know how much SGX memory an enclave can consume.
> 
> == Solution ==
> 
> Introduce a new sysfs file:
> 
> 	/sys/devices/system/node/nodeX/x86/sgx_total_bytes
> 
> to enumerate the amount of SGX memory available in each NUMA node.
> This serves the same function for SGX as /proc/meminfo or
> /sys/devices/system/node/nodeX/meminfo does for normal RAM.
> 
> 'sgx_total_bytes' is needed today to help drive the SGX selftests.
> SGX-specific swap code is exercised by creating overcommitted enclaves
> which are larger than the physical SGX memory on the system.  They
> currently use a CPUID-based approach which can diverge from the actual
> amount of SGX memory available.  'sgx_total_bytes' ensures that the
> selftests can work efficiently and do not attempt stupid things like
> creating a 100,000 MB enclave on a system with 128 MB of SGX memory.
> 
> == Implementation Details ==
> 
> Introduce CONFIG_HAVE_ARCH_NODE_DEV_GROUP opt-in flag to expose an
> arch specific attribute group, and add an attribute for the amount of
> SGX memory in bytes to each NUMA node:
> 
> == ABI Design Discussion ==
> 
> As opposed to the per-node ABI, a single, global ABI was considered.
> However, this would prevent enclaves from being able to size
> themselves so that they fit on a single NUMA node.  Essentially, a
> single value would rule out NUMA optimizations for enclaves.
> 
> Create a new "x86/" directory inside each "nodeX/" sysfs directory.
> 'sgx_total_bytes' is expected to be the first of at least a few
> sgx-specific files to be placed in the new directory.  Just scanning
> /proc/meminfo, these are the no-brainers that we have for RAM, but we
> need for SGX:
> 
> 	MemTotal:       xxxx kB // sgx_total_bytes (implemented here)
> 	MemFree:        yyyy kB // sgx_free_bytes
> 	SwapTotal:      zzzz kB // sgx_swapped_bytes
> 
> So, at *least* three.  I think we will eventually end up needing
> something more along the lines of a dozen.  A new directory (as
> opposed to being in the nodeX/ "root") directory avoids cluttering the
> root with several "sgx_*" files.
> 
> Place the new file in a new "nodeX/x86/" directory because SGX is
> highly x86-specific.  It is very unlikely that any other architecture
> (or even non-Intel x86 vendor) will ever implement SGX.  Using "sgx/"
> as opposed to "x86/" was also considered.  But, there is a real chance
> this can get used for other arch-specific purposes.
> 
> [ dhansen: rewrite changelog ]
> 
> Signed-off-by: Jarkko Sakkinen <jarkko@kernel.org>
> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Link: https://lkml.kernel.org/r/20211116162116.93081-2-jarkko@kernel.org
> ---
>  Documentation/ABI/stable/sysfs-devices-node |  6 ++++++
>  arch/Kconfig                                |  4 ++++
>  arch/x86/Kconfig                            |  1 +
>  arch/x86/kernel/cpu/sgx/main.c              | 20 ++++++++++++++++++++
>  arch/x86/kernel/cpu/sgx/sgx.h               |  1 +
>  drivers/base/node.c                         |  3 +++
>  include/linux/numa.h                        |  4 ++++
>  7 files changed, 39 insertions(+)

Thank you! Looks good to me.

/Jarkko