Re: [PATCH v5 09/18] x86/sgx: Store struct sgx_encl when allocating new VA pages

Huang, Kai
2023-10-05T14:49:09Z

On Wed, 2023-10-04 at 23:22 -0500, Haitao Huang wrote:
> On Wed, 04 Oct 2023 16:13:41 -0500, Huang, Kai <kai.huang@intel.com> wrote:
>
> > On Wed, 2023-10-04 at 10:03 -0500, Haitao Huang wrote:
> > > On Tue, 03 Oct 2023 15:07:42 -0500, Huang, Kai <kai.huang@intel.com>
> > > wrote:
> > >
> > > > On Tue, 2023-10-03 at 01:45 -0500, Haitao Huang wrote:
> > > > > >
> > > > > > Btw, probably a dumb question:
> > > > > >
> > > > > > Theoretically if you only need to find a victim enclave you don't
> > > need
> > > > > > to put VA
> > > > > > pages to the unreclaimable list, because those VA pages will be
> > > freed
> > > > > > anyway
> > > > > > when enclave is killed. So keeping VA pages in the list is for>
> > > > > accounting all
> > > > > > the pages that the cgroup is having?
> > > > >
> > > > > Yes basically tracking them in cgroups as they are allocated.
> > > > >
> > > > > VAs and SECS may also come and go as swapping/unswapping happens.
> > > But
> > > > > if acgroup is OOM, and all reclaimables are gone (swapped out), it'd
> > > > > have toreclaim VAs/SECs in the same cgroup starting from the front
> > > of
> > > > > the LRUlist. To reclaim a VA/SECS, it identifies the enclave from
> > > the
> > > > > owner ofthe VA/SECS page and kills it, as killing enclave is the
> > > only
> > > > > way toreclaim VA/SECS pages.
> > > >
> > > > To kill enclave you just need to track SECS in the unreclaimable
> > > list.
> > > > Only when you want to account the total EPC pages via some list you
> > > > _probably_
> > > > need to track VA as well. But I am not quite sure about this either.
> > >
> > > There is a case where even SECS is paged out for an enclave with all
> > > reclaimables out.
> >
> > Yes. But this essentially means these enclaves are not active, thus
> > shouldn't
> > be the victim of OOM?
> >
>
> But there are VA pages for the enclave at that moment. So it can be
> candidate for OOM victim.

Yes. I am not familiar with how does OOM choose victim, but it seems choosing
inactive enclaves seems more reasonable.

[...]

> > > There were some discussion on paging out VAs without killing enclaves
> > > but
> > > it'd be complicated and not implemented yet.
> >
> > No we don't involve swapping VA pages now. It's a separate topic.
> >
> Only mentioned it as a kind of constraints impacting current design.
>
> Another potential alternative: we don't reclaim SECS either until OOM and
> only track SECS pages for cgroups. But that would change current behavior.
> And I'm not sure about other consequences, e.g., enclaves theoretically
> can allocate pages (including VA pages) in different cgroups/processes, so
> we may still end up tracking all VA pages for cgroups or we track SECS
> page in all cgroups in which enclave allocated any pages. Let me know your
> thoughts.

Let's not change current behaviour. I seriously doubt that is needed.

So it seems to me that what we need is just some way to let the OOM find some
victim enclave. I am not sure whether "tracking EPC pages in some lists" has
anything to do with cgroup accounting EPC pages, so will take a look the rest of
the patches.