Re: [PATCH] x86/sgx: Add a basic NUMA allocation scheme to sgx_alloc_epc_page()

[Jarkko Sakkinen <jarkko@kernel.org>]

On Sun, Feb 21, 2021 at 04:54:33PM -0800, Dave Hansen wrote:
> > +/* Nodes with one or more EPC sections. */
> > +static nodemask_t sgx_numa_mask;
> 
> I'd also add that this is for optimization only.
> 
> > +/* Array of lists of EPC sections for each NUMA node. */
> > +struct list_head *sgx_numa_nodes;
> 
> I'd much prefer:
> 
> /*
>  * Array with one list_head for each possible NUMA node.  Each
>  * list contains all the sgx_epc_section's which are on that
>  * node.
>  */
> 
> Otherwise, it's hard to imagine what this structure looks like.

OK.

> >  /*
> >   * These variables are part of the state of the reclaimer, and must be accessed
> >   * with sgx_reclaimer_lock acquired.
> > @@ -473,6 +479,26 @@ static struct sgx_epc_page *__sgx_alloc_epc_page_from_section(struct sgx_epc_sec
> >  	return page;
> >  }
> >  
> > +static struct sgx_epc_page *__sgx_alloc_epc_page_from_node(int nid)
> > +{
> > +	struct sgx_epc_section *section;
> > +	struct sgx_epc_page *page;
> > +
> > +	if (WARN_ON_ONCE(nid < 0 || nid >= MAX_NUMNODES))
> > +		return NULL;
> > +
> > +	if (!node_isset(nid, sgx_numa_mask))
> > +		return NULL;
> > +
> > +	list_for_each_entry(section, &sgx_numa_nodes[nid], section_list) {
> > +		page = __sgx_alloc_epc_page_from_section(section);
> > +		if (page)
> > +			return page;
> > +	}
> > +
> > +	return NULL;
> > +}
> > +
> >  /**
> >   * __sgx_alloc_epc_page() - Allocate an EPC page
> >   *
> > @@ -485,13 +511,17 @@ static struct sgx_epc_page *__sgx_alloc_epc_page_from_section(struct sgx_epc_sec
> >   */
> >  struct sgx_epc_page *__sgx_alloc_epc_page(void)
> >  {
> > +	int current_nid = numa_node_id();
> >  	struct sgx_epc_section *section;
> >  	struct sgx_epc_page *page;
> >  	int i;
> >  
> > +	page = __sgx_alloc_epc_page_from_node(current_nid);
> > +	if (page)
> > +		return page;
> 
> Comments, please.
> 
> 	/* Try to allocate EPC from the current node, first: */
> 
> then:
> 
> 	/* Search all EPC sections, ignoring locality: */
> 
> >  	for (i = 0; i < sgx_nr_epc_sections; i++) {
> >  		section = &sgx_epc_sections[i];
> > -
> >  		page = __sgx_alloc_epc_page_from_section(section);
> >  		if (page)
> >  			return page;
> 
> This still has the problem that it exerts too much pressure on the
> low-numbered sgx_epc_sections[].  If a node's sections are full, it
> always tries to go after sgx_epc_sections[0].

I have a better idea. See below.

> It can be in another patch, but I think the *minimal* thing we can do
> here for a NUMA allocator is to try to at least balance the allocations.
> 
> Instead of having a for-each-section loop, I'd make it for-each-node ->
> for-each-section.  Something like:
> 
> 	for (i = 0; i < num_possible_nodes(); i++) {
> 		node = (numa_node_id() + i) % num_possible_nodes()
> 		
> 		if (!node_isset(nid, sgx_numa_mask))
> 			continue;
> 
> 		list_for_each_entry(section, &sgx_numa_nodes[nid],
> 				    section_list) {
> 			__sgx_alloc_epc_page_from_section(section)
> 		}
> 	}
> 	
> Then you have a single loop instead of a "try local then a fall back".
> 
> Also, that "node++" thing might be able to use next_online_node().
> 
> > @@ -665,8 +695,12 @@ static bool __init sgx_page_cache_init(void)
> >  {
> >  	u32 eax, ebx, ecx, edx, type;
> >  	u64 pa, size;
> > +	int nid;
> >  	int i;
> >  
> > +	nodes_clear(sgx_numa_mask);
> > +	sgx_numa_nodes = kmalloc_array(MAX_NUMNODES, sizeof(*sgx_numa_nodes), GFP_KERNEL);
> 
> MAX_NUMNODES will always be the largest compile-time constant.  That's
> 4k, IIRC.  num_possible_nodes() might be as small as 1 if NUMA is off.

Right.

> >  	for (i = 0; i < ARRAY_SIZE(sgx_epc_sections); i++) {
> >  		cpuid_count(SGX_CPUID, i + SGX_CPUID_EPC, &eax, &ebx, &ecx, &edx);
> >  
> > @@ -690,6 +724,22 @@ static bool __init sgx_page_cache_init(void)
> >  		}
> >  
> >  		sgx_nr_epc_sections++;
> > +
> > +		nid = numa_map_to_online_node(phys_to_target_node(pa));
> > +
> > +		if (nid == NUMA_NO_NODE) {
> > +			pr_err(FW_BUG "unable to map EPC section %d to online node.\n", nid);
> > +			nid = 0;
> 
> Could we dump out the physical address there?  I think that's even more
> informative than a section number.

Yes.

> > +		} else if (WARN_ON_ONCE(nid < 0 || nid >= MAX_NUMNODES)) {
> > +			nid = 0;
> > +		}
> 
> I'm not sure we really need to check for these.  If we're worried about
> the firmware returning these, I'd expect numa_map_to_online_node() to
> sanity check them for us.

Yes, let's remove it.

> > +		if (!node_isset(nid, sgx_numa_mask)) {
> > +			INIT_LIST_HEAD(&sgx_numa_nodes[nid]);
> > +			node_set(nid, sgx_numa_mask);
> > +		}
> > +
> > +		list_add_tail(&sgx_epc_sections[i].section_list, &sgx_numa_nodes[nid]);
> >  	}
> >  
> >  	if (!sgx_nr_epc_sections) {
> > diff --git a/arch/x86/kernel/cpu/sgx/sgx.h b/arch/x86/kernel/cpu/sgx/sgx.h
> > index 5fa42d143feb..4bc31bc4bacf 100644
> > --- a/arch/x86/kernel/cpu/sgx/sgx.h
> > +++ b/arch/x86/kernel/cpu/sgx/sgx.h
> > @@ -45,6 +45,7 @@ struct sgx_epc_section {
> >  	spinlock_t lock;
> >  	struct list_head page_list;
> >  	unsigned long free_cnt;
> > +	struct list_head section_list;
> 
> Maybe name this numa_section_list.

Instead, let's just:

1. Have a global sgx_free_epc_list and remove sgx_epc_section.
   Pages from this are allocated from this in LIFO fashion.
2. Instead add struct list_head node_list and use that for node
   associated pages.
3. Replace 'int section' with 'int node'.

This will remove one layer of abstraction and provide better fallback
scheme.

E.g. allocate:

1. Check node_list of current node.
2. As a fallback check sgx_free_epc_list.
3. list_del() for two lists.

/Jarkko