public inbox for cgroups@vger.kernel.org
 help / color / mirror / Atom feed
From: Haitao Huang <haitao.huang@linux.intel.com>
To: jarkko@kernel.org, dave.hansen@linux.intel.com, tj@kernel.org,
	linux-kernel@vger.kernel.org, linux-sgx@vger.kernel.org,
	x86@kernel.org, cgroups@vger.kernel.org, tglx@linutronix.de,
	mingo@redhat.com, bp@alien8.de, hpa@zytor.com,
	sohil.mehta@intel.com
Cc: zhiquan1.li@intel.com, kristen@linux.intel.com,
	seanjc@google.com, zhanb@microsoft.com, anakrish@microsoft.com,
	mikko.ylinen@linux.intel.com, yangjie@microsoft.com
Subject: [PATCH v5 03/18] x86/sgx: Add sgx_epc_lru_lists to encapsulate LRU lists
Date: Fri, 22 Sep 2023 20:06:42 -0700	[thread overview]
Message-ID: <20230923030657.16148-4-haitao.huang@linux.intel.com> (raw)
In-Reply-To: <20230923030657.16148-1-haitao.huang@linux.intel.com>

From: Sean Christopherson <sean.j.christopherson@intel.com>

Introduce a data structure to wrap the existing reclaimable list and its
spinlock. Each cgroup later will have one instance of this structure to
track EPC pages allocated for processes associated with the same cgroup.
Just like the global SGX reclaimer (ksgxd), an EPC cgroup reclaims pages
from the reclaimable list in this structure when its usage reaches near
its limit.

Currently, ksgxd does not track the VA, SECS pages. They are considered
as 'unreclaimable' pages that are only deallocated when their respective
owning enclaves are destroyed and all associated resources released.

When an EPC cgroup can not reclaim any more reclaimable EPC pages to
reduce its usage below its limit, the cgroup must also reclaim those
unreclaimables by killing their owning enclaves. The VA and SECS pages
later are also tracked in an 'unreclaimable' list added to this structure
to support this OOM killing of enclaves.

Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
Co-developed-by: Kristen Carlson Accardi <kristen@linux.intel.com>
Signed-off-by: Kristen Carlson Accardi <kristen@linux.intel.com>
Co-developed-by: Haitao Huang <haitao.huang@linux.intel.com>
Signed-off-by: Haitao Huang <haitao.huang@linux.intel.com>
Cc: Sean Christopherson <seanjc@google.com>
---
V4:
- Removed unneeded comments for the spinlock and the non-reclaimables.
(Kai, Jarkko)
- Revised the commit to add introduction comments for unreclaimables and
multiple LRU lists.(Kai)
- Reordered the patches: delay all changes for unreclaimables to
later, and this one becomes the first change in the SGX subsystem.

V3:
- Removed the helper functions and revised commit messages.
---
 arch/x86/kernel/cpu/sgx/sgx.h | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/arch/x86/kernel/cpu/sgx/sgx.h b/arch/x86/kernel/cpu/sgx/sgx.h
index d2dad21259a8..018414b2abe8 100644
--- a/arch/x86/kernel/cpu/sgx/sgx.h
+++ b/arch/x86/kernel/cpu/sgx/sgx.h
@@ -83,6 +83,20 @@ static inline void *sgx_get_epc_virt_addr(struct sgx_epc_page *page)
 	return section->virt_addr + index * PAGE_SIZE;
 }
 
+/*
+ * Tracks EPC pages reclaimable by the reclaimer (ksgxd).
+ */
+struct sgx_epc_lru_lists {
+	spinlock_t lock;
+	struct list_head reclaimable;
+};
+
+static inline void sgx_lru_init(struct sgx_epc_lru_lists *lrus)
+{
+	spin_lock_init(&lrus->lock);
+	INIT_LIST_HEAD(&lrus->reclaimable);
+}
+
 struct sgx_epc_page *__sgx_alloc_epc_page(void);
 void sgx_free_epc_page(struct sgx_epc_page *page);
 
-- 
2.25.1


      parent reply	other threads:[~2023-09-23  3:06 UTC|newest]

Thread overview: 126+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-09-23  3:06 [PATCH v5 00/18] Add Cgroup support for SGX EPC memory Haitao Huang
     [not found] ` <20230923030657.16148-1-haitao.huang-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
2023-09-23  3:06   ` [PATCH v5 01/18] cgroup/misc: Add per resource callbacks for CSS events Haitao Huang
     [not found]     ` <20230923030657.16148-2-haitao.huang-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
2023-09-25 17:09       ` Jarkko Sakkinen
2023-09-25 17:09         ` Jarkko Sakkinen
2023-09-26  3:04         ` Haitao Huang
     [not found]           ` <op.2buytfetwjvjmi-yDQzE4XY+yVaPPhiJ6yCxLKMmGWinSIL2HeeBUIffwg@public.gmane.org>
2023-09-26 13:10             ` Jarkko Sakkinen
2023-09-26 13:10               ` Jarkko Sakkinen
2023-09-26 13:13               ` Jarkko Sakkinen
2023-09-26 13:13                 ` Jarkko Sakkinen
2023-09-27  1:56                 ` Haitao Huang
2023-10-02 22:47                   ` Jarkko Sakkinen
2023-10-02 22:55                     ` Jarkko Sakkinen
2023-10-04 15:45                       ` Haitao Huang
2023-10-04 17:18                         ` Tejun Heo
2023-09-27  9:20     ` Huang, Kai
2023-10-03 14:29       ` Haitao Huang
2023-10-17 18:55     ` Michal Koutný
2023-09-23  3:06   ` [PATCH v5 02/18] cgroup/misc: Add SGX EPC resource type and export APIs for SGX driver Haitao Huang
     [not found]     ` <20230923030657.16148-3-haitao.huang-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
2023-09-25 18:50       ` Tejun Heo
2023-09-25 18:50         ` Tejun Heo
2023-09-28  3:59     ` Huang, Kai
2023-10-03  7:00       ` Haitao Huang
2023-10-03 19:33         ` Huang, Kai
2023-09-23  3:06   ` [PATCH v5 04/18] x86/sgx: Use sgx_epc_lru_lists for existing active page list Haitao Huang
2023-09-23  3:06   ` [PATCH v5 05/18] x86/sgx: Store reclaimable EPC pages in sgx_epc_lru_lists Haitao Huang
2023-09-27 10:14     ` Huang, Kai
2023-09-23  3:06   ` [PATCH v5 06/18] x86/sgx: Introduce EPC page states Haitao Huang
     [not found]     ` <20230923030657.16148-7-haitao.huang-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
2023-09-25 17:11       ` Jarkko Sakkinen
2023-09-25 17:11         ` Jarkko Sakkinen
2023-09-27 10:28     ` Huang, Kai
2023-10-03  4:49       ` Haitao Huang
2023-10-03 20:03         ` Huang, Kai
2023-10-04 15:24           ` Haitao Huang
2023-10-04 21:05             ` Huang, Kai
2023-09-23  3:06   ` [PATCH v5 07/18] x86/sgx: Introduce RECLAIM_IN_PROGRESS state Haitao Huang
2023-09-25 17:13     ` Jarkko Sakkinen
2023-09-25 17:13       ` Jarkko Sakkinen
2023-09-27 10:42     ` Huang, Kai
2023-09-23  3:06   ` [PATCH v5 08/18] x86/sgx: Use a list to track to-be-reclaimed pages Haitao Huang
2023-09-28  9:28     ` Huang, Kai
2023-10-03  5:09       ` Haitao Huang
2023-09-23  3:06   ` [PATCH v5 09/18] x86/sgx: Store struct sgx_encl when allocating new VA pages Haitao Huang
2023-09-27 11:14     ` Huang, Kai
2023-09-27 15:35       ` Haitao Huang
2023-09-27 21:21         ` Huang, Kai
2023-09-29 15:06           ` Haitao Huang
2023-10-02 11:05             ` Huang, Kai
2023-09-27 11:35     ` Huang, Kai
2023-10-03  6:45       ` Haitao Huang
2023-10-03 20:07         ` Huang, Kai
2023-10-04 15:03           ` Haitao Huang
2023-10-04 21:13             ` Huang, Kai
2023-10-05  4:22               ` Haitao Huang
2023-10-05  6:49                 ` Huang, Kai
2023-09-23  3:06   ` [PATCH v5 10/18] x86/sgx: Add EPC page flags to identify owner types Haitao Huang
2023-09-23  3:06   ` [PATCH v5 11/18] x86/sgx: store unreclaimable pages in LRU lists Haitao Huang
2023-09-27 11:57     ` Huang, Kai
2023-10-03  5:42       ` Haitao Huang
2023-09-28  9:41     ` Huang, Kai
2023-10-03  5:15       ` Haitao Huang
2023-10-03 20:12         ` Huang, Kai
2023-09-23  3:06   ` [PATCH v5 12/18] x86/sgx: Add EPC OOM path to forcefully reclaim EPC Haitao Huang
2023-10-09 23:45     ` Huang, Kai
2023-10-10  0:23       ` Sean Christopherson
2023-10-10  0:50         ` Huang, Kai
2023-10-10  1:34           ` Huang, Kai
2023-10-10 16:49             ` Haitao Huang
2023-10-11  0:51               ` Huang, Kai
2023-10-12 13:27                 ` Haitao Huang
2023-10-16 10:57                   ` Huang, Kai
2023-10-16 19:52                     ` Haitao Huang
2023-10-16 21:09                       ` Huang, Kai
2023-10-17  0:10                         ` Haitao Huang
2023-10-17  1:34                           ` Huang, Kai
2023-10-17 12:58                             ` Haitao Huang
2023-10-17 18:54                               ` Michal Koutný
2023-10-17 19:13                                 ` Michal Koutný
2023-10-18  4:39                                   ` Haitao Huang
2023-10-18  4:37                                 ` Haitao Huang
2023-10-18 13:55                                   ` Dave Hansen
2023-10-18 15:26                                     ` Haitao Huang
2023-10-18 15:37                                       ` Dave Hansen
2023-10-18 15:52                                         ` Michal Koutný
2023-10-18 16:25                                           ` Haitao Huang
2023-10-16 21:32                       ` Sean Christopherson
2023-10-17  0:09                         ` Haitao Huang
2023-10-17 15:43                           ` Sean Christopherson
2023-10-17 11:49                         ` Mikko Ylinen
2023-10-11  1:14               ` Huang, Kai
2023-10-16 11:02                 ` Huang, Kai
2023-10-10  1:42         ` Haitao Huang
2023-10-10  2:23           ` Huang, Kai
2023-10-10 13:26             ` Haitao Huang
2023-10-11  0:01               ` Sean Christopherson
2023-10-11 15:02                 ` Haitao Huang
2023-10-10  1:04       ` Haitao Huang
2023-10-10  1:18         ` Huang, Kai
2023-10-10  1:38           ` Haitao Huang
2023-10-10  2:12             ` Huang, Kai
2023-10-10 17:05               ` Haitao Huang
2023-10-11  0:31                 ` Huang, Kai
2023-10-11 16:04                   ` Haitao Huang
2023-09-23  3:06   ` [PATCH v5 13/18] x86/sgx: Expose sgx_reclaim_pages() for use by EPC cgroup Haitao Huang
2023-10-05 12:24     ` Huang, Kai
2023-10-05 19:23       ` Haitao Huang
2023-10-05 20:25         ` Huang, Kai
2023-09-23  3:06   ` [PATCH v5 14/18] x86/sgx: Add helper to grab pages from an arbitrary EPC LRU Haitao Huang
2023-09-23  3:06   ` [PATCH v5 15/18] x86/sgx: Prepare for multiple LRUs Haitao Huang
2023-10-05 12:30     ` Huang, Kai
2023-10-05 19:33       ` Haitao Huang
2023-10-05 20:38         ` Huang, Kai
2023-09-23  3:06   ` [PATCH v5 16/18] x86/sgx: Limit process EPC usage with misc cgroup controller Haitao Huang
     [not found]     ` <20230923030657.16148-17-haitao.huang-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
2023-09-25 17:15       ` Jarkko Sakkinen
2023-09-25 17:15         ` Jarkko Sakkinen
2023-10-05 21:01     ` Huang, Kai
2023-10-10  0:12     ` Huang, Kai
2023-10-10  0:16     ` Huang, Kai
2023-10-10  0:26     ` Huang, Kai
2023-10-22 18:26       ` Haitao Huang
2023-10-10  9:19     ` Huang, Kai
2023-10-10  9:32     ` Huang, Kai
2023-10-17 18:54     ` Michal Koutný
2023-10-19 16:05       ` Haitao Huang
2023-09-23  3:06   ` [PATCH v5 17/18] Docs/x86/sgx: Add description for cgroup support Haitao Huang
2023-09-23  3:06   ` [PATCH v5 18/18] selftests/sgx: Add scripts for EPC cgroup testing Haitao Huang
2023-09-23  3:06 ` Haitao Huang [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230923030657.16148-4-haitao.huang@linux.intel.com \
    --to=haitao.huang@linux.intel.com \
    --cc=anakrish@microsoft.com \
    --cc=bp@alien8.de \
    --cc=cgroups@vger.kernel.org \
    --cc=dave.hansen@linux.intel.com \
    --cc=hpa@zytor.com \
    --cc=jarkko@kernel.org \
    --cc=kristen@linux.intel.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-sgx@vger.kernel.org \
    --cc=mikko.ylinen@linux.intel.com \
    --cc=mingo@redhat.com \
    --cc=seanjc@google.com \
    --cc=sohil.mehta@intel.com \
    --cc=tglx@linutronix.de \
    --cc=tj@kernel.org \
    --cc=x86@kernel.org \
    --cc=yangjie@microsoft.com \
    --cc=zhanb@microsoft.com \
    --cc=zhiquan1.li@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox