* [PATCH] powerpc/book3s_hv_uvmem: Check for failed page migration
@ 2020-12-03 5:08 Alistair Popple
2020-12-04 10:18 ` Bharata B Rao
2020-12-08 14:20 ` Ram Pai
0 siblings, 2 replies; 6+ messages in thread
From: Alistair Popple @ 2020-12-03 5:08 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Alistair Popple, bharata
migrate_vma_pages() may still clear MIGRATE_PFN_MIGRATE on pages which
are not able to be migrated. Drivers may safely copy data prior to
calling migrate_vma_pages() however a remote mapping must not be
established until after migrate_vma_pages() has returned as the
migration could still fail.
UV_PAGE_IN_in both copies and maps the data page, therefore it should
only be called after checking the results of migrate_vma_pages().
Signed-off-by: Alistair Popple <alistair@popple.id.au>
---
arch/powerpc/kvm/book3s_hv_uvmem.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/arch/powerpc/kvm/book3s_hv_uvmem.c b/arch/powerpc/kvm/book3s_hv_uvmem.c
index 84e5a2dc8be5..08aa6a90c525 100644
--- a/arch/powerpc/kvm/book3s_hv_uvmem.c
+++ b/arch/powerpc/kvm/book3s_hv_uvmem.c
@@ -762,7 +762,10 @@ static int kvmppc_svm_page_in(struct vm_area_struct *vma,
goto out_finalize;
}
- if (pagein) {
+ *mig.dst = migrate_pfn(page_to_pfn(dpage)) | MIGRATE_PFN_LOCKED;
+ migrate_vma_pages(&mig);
+
+ if ((*mig.src & MIGRATE_PFN_MIGRATE) && pagein) {
pfn = *mig.src >> MIGRATE_PFN_SHIFT;
spage = migrate_pfn_to_page(*mig.src);
if (spage) {
@@ -773,8 +776,6 @@ static int kvmppc_svm_page_in(struct vm_area_struct *vma,
}
}
- *mig.dst = migrate_pfn(page_to_pfn(dpage)) | MIGRATE_PFN_LOCKED;
- migrate_vma_pages(&mig);
out_finalize:
migrate_vma_finalize(&mig);
return ret;
--
2.20.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH] powerpc/book3s_hv_uvmem: Check for failed page migration
2020-12-03 5:08 [PATCH] powerpc/book3s_hv_uvmem: Check for failed page migration Alistair Popple
@ 2020-12-04 10:18 ` Bharata B Rao
2020-12-04 16:52 ` Ram Pai
2020-12-06 23:55 ` Alistair Popple
2020-12-08 14:20 ` Ram Pai
1 sibling, 2 replies; 6+ messages in thread
From: Bharata B Rao @ 2020-12-04 10:18 UTC (permalink / raw)
To: Alistair Popple; +Cc: linuxppc-dev, Ram Pai
On Thu, Dec 03, 2020 at 04:08:12PM +1100, Alistair Popple wrote:
> migrate_vma_pages() may still clear MIGRATE_PFN_MIGRATE on pages which
> are not able to be migrated. Drivers may safely copy data prior to
> calling migrate_vma_pages() however a remote mapping must not be
> established until after migrate_vma_pages() has returned as the
> migration could still fail.
>
> UV_PAGE_IN_in both copies and maps the data page, therefore it should
> only be called after checking the results of migrate_vma_pages().
>
> Signed-off-by: Alistair Popple <alistair@popple.id.au>
> ---
> arch/powerpc/kvm/book3s_hv_uvmem.c | 7 ++++---
> 1 file changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/arch/powerpc/kvm/book3s_hv_uvmem.c b/arch/powerpc/kvm/book3s_hv_uvmem.c
> index 84e5a2dc8be5..08aa6a90c525 100644
> --- a/arch/powerpc/kvm/book3s_hv_uvmem.c
> +++ b/arch/powerpc/kvm/book3s_hv_uvmem.c
> @@ -762,7 +762,10 @@ static int kvmppc_svm_page_in(struct vm_area_struct *vma,
> goto out_finalize;
> }
>
> - if (pagein) {
> + *mig.dst = migrate_pfn(page_to_pfn(dpage)) | MIGRATE_PFN_LOCKED;
> + migrate_vma_pages(&mig);
> +
> + if ((*mig.src & MIGRATE_PFN_MIGRATE) && pagein) {
> pfn = *mig.src >> MIGRATE_PFN_SHIFT;
> spage = migrate_pfn_to_page(*mig.src);
> if (spage) {
> @@ -773,8 +776,6 @@ static int kvmppc_svm_page_in(struct vm_area_struct *vma,
> }
> }
>
> - *mig.dst = migrate_pfn(page_to_pfn(dpage)) | MIGRATE_PFN_LOCKED;
> - migrate_vma_pages(&mig);
> out_finalize:
> migrate_vma_finalize(&mig);
> return ret;
Reviewed-by: Bharata B Rao <bharata@linux.ibm.com>
Did you actually hit this scenario with secure VMs where a UV-paged-in
page was later found to be not migratable?
Regards,
Bharata.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] powerpc/book3s_hv_uvmem: Check for failed page migration
2020-12-04 10:18 ` Bharata B Rao
@ 2020-12-04 16:52 ` Ram Pai
2020-12-07 0:13 ` Alistair Popple
2020-12-06 23:55 ` Alistair Popple
1 sibling, 1 reply; 6+ messages in thread
From: Ram Pai @ 2020-12-04 16:52 UTC (permalink / raw)
To: Bharata B Rao; +Cc: Alistair Popple, linuxppc-dev
On Fri, Dec 04, 2020 at 03:48:41PM +0530, Bharata B Rao wrote:
> On Thu, Dec 03, 2020 at 04:08:12PM +1100, Alistair Popple wrote:
> > migrate_vma_pages() may still clear MIGRATE_PFN_MIGRATE on pages which
> > are not able to be migrated. Drivers may safely copy data prior to
> > calling migrate_vma_pages() however a remote mapping must not be
> > established until after migrate_vma_pages() has returned as the
> > migration could still fail.
> >
> > UV_PAGE_IN_in both copies and maps the data page, therefore it should
> > only be called after checking the results of migrate_vma_pages().
> >
> > Signed-off-by: Alistair Popple <alistair@popple.id.au>
> > ---
> > arch/powerpc/kvm/book3s_hv_uvmem.c | 7 ++++---
> > 1 file changed, 4 insertions(+), 3 deletions(-)
> >
> > diff --git a/arch/powerpc/kvm/book3s_hv_uvmem.c b/arch/powerpc/kvm/book3s_hv_uvmem.c
> > index 84e5a2dc8be5..08aa6a90c525 100644
> > --- a/arch/powerpc/kvm/book3s_hv_uvmem.c
> > +++ b/arch/powerpc/kvm/book3s_hv_uvmem.c
> > @@ -762,7 +762,10 @@ static int kvmppc_svm_page_in(struct vm_area_struct *vma,
> > goto out_finalize;
> > }
> >
> > - if (pagein) {
> > + *mig.dst = migrate_pfn(page_to_pfn(dpage)) | MIGRATE_PFN_LOCKED;
> > + migrate_vma_pages(&mig);
> > +
> > + if ((*mig.src & MIGRATE_PFN_MIGRATE) && pagein) {
> > pfn = *mig.src >> MIGRATE_PFN_SHIFT;
> > spage = migrate_pfn_to_page(*mig.src);
> > if (spage) {
> > @@ -773,8 +776,6 @@ static int kvmppc_svm_page_in(struct vm_area_struct *vma,
> > }
> > }
> >
> > - *mig.dst = migrate_pfn(page_to_pfn(dpage)) | MIGRATE_PFN_LOCKED;
> > - migrate_vma_pages(&mig);
> > out_finalize:
> > migrate_vma_finalize(&mig);
> > return ret;
This patch certainly looks like the problem, that has been hurting
us for a while. Let me run this patch through my SVM tests. Looks very
promising.
BTW: The code does a similar thing while paging out. It pages out from the
UV, and then does the migration. Is there a bug there aswell?
RP
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] powerpc/book3s_hv_uvmem: Check for failed page migration
2020-12-04 10:18 ` Bharata B Rao
2020-12-04 16:52 ` Ram Pai
@ 2020-12-06 23:55 ` Alistair Popple
1 sibling, 0 replies; 6+ messages in thread
From: Alistair Popple @ 2020-12-06 23:55 UTC (permalink / raw)
To: bharata; +Cc: linuxppc-dev, Ram Pai
On Friday, 4 December 2020 9:18:41 PM AEDT Bharata B Rao wrote:
>
> Reviewed-by: Bharata B Rao <bharata@linux.ibm.com>
>
> Did you actually hit this scenario with secure VMs where a UV-paged-in
> page was later found to be not migratable?
No, this was found by inspection. I have no way of testing this but we had a
similar issue in Nouveau and I think you would have a similar issue here
although it might be hard to hit.
migrate_vma_pages() will fail a page migration if a CPU thread has raced and
established a non-zero page PTE for the address. See migrate_vma_insert_page()
for the implementation. It will also fail if something else has taken a
reference on the page after calling migrate_vma_setup(), but that is less
likely as any existing pages will have been isolated.
- Alistair
> Regards,
> Bharata.
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] powerpc/book3s_hv_uvmem: Check for failed page migration
2020-12-04 16:52 ` Ram Pai
@ 2020-12-07 0:13 ` Alistair Popple
0 siblings, 0 replies; 6+ messages in thread
From: Alistair Popple @ 2020-12-07 0:13 UTC (permalink / raw)
To: Ram Pai; +Cc: linuxppc-dev, Bharata B Rao
On Saturday, 5 December 2020 3:52:44 AM AEDT Ram Pai wrote:
> On Fri, Dec 04, 2020 at 03:48:41PM +0530, Bharata B Rao wrote:
> > On Thu, Dec 03, 2020 at 04:08:12PM +1100, Alistair Popple wrote:
> This patch certainly looks like the problem, that has been hurting
> us for a while. Let me run this patch through my SVM tests. Looks very
> promising.
>
> BTW: The code does a similar thing while paging out. It pages out from the
> UV, and then does the migration. Is there a bug there aswell?
As specified the migrate_pages_vma() API can fail to migrate device private
pages. However the fix was less obvious to me, and in practice I don't think it
will ever fail for device private pages as you don't have the same races to
establish the page and device private pages can't be pinned.
It might be worth adding some kind of warning though in case this ever
changes.
- Alistair
> RP
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] powerpc/book3s_hv_uvmem: Check for failed page migration
2020-12-03 5:08 [PATCH] powerpc/book3s_hv_uvmem: Check for failed page migration Alistair Popple
2020-12-04 10:18 ` Bharata B Rao
@ 2020-12-08 14:20 ` Ram Pai
1 sibling, 0 replies; 6+ messages in thread
From: Ram Pai @ 2020-12-08 14:20 UTC (permalink / raw)
To: Alistair Popple; +Cc: linuxppc-dev, bharata
On Thu, Dec 03, 2020 at 04:08:12PM +1100, Alistair Popple wrote:
> migrate_vma_pages() may still clear MIGRATE_PFN_MIGRATE on pages which
> are not able to be migrated. Drivers may safely copy data prior to
> calling migrate_vma_pages() however a remote mapping must not be
> established until after migrate_vma_pages() has returned as the
> migration could still fail.
>
> UV_PAGE_IN_in both copies and maps the data page, therefore it should
> only be called after checking the results of migrate_vma_pages().
>
> Signed-off-by: Alistair Popple <alistair@popple.id.au>
> ---
> arch/powerpc/kvm/book3s_hv_uvmem.c | 7 ++++---
> 1 file changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/arch/powerpc/kvm/book3s_hv_uvmem.c b/arch/powerpc/kvm/book3s_hv_uvmem.c
> index 84e5a2dc8be5..08aa6a90c525 100644
> --- a/arch/powerpc/kvm/book3s_hv_uvmem.c
> +++ b/arch/powerpc/kvm/book3s_hv_uvmem.c
> @@ -762,7 +762,10 @@ static int kvmppc_svm_page_in(struct vm_area_struct *vma,
> goto out_finalize;
> }
>
> - if (pagein) {
> + *mig.dst = migrate_pfn(page_to_pfn(dpage)) | MIGRATE_PFN_LOCKED;
> + migrate_vma_pages(&mig);
> +
> + if ((*mig.src & MIGRATE_PFN_MIGRATE) && pagein) {
> pfn = *mig.src >> MIGRATE_PFN_SHIFT;
> spage = migrate_pfn_to_page(*mig.src);
> if (spage) {
> @@ -773,8 +776,6 @@ static int kvmppc_svm_page_in(struct vm_area_struct *vma,
> }
> }
>
> - *mig.dst = migrate_pfn(page_to_pfn(dpage)) | MIGRATE_PFN_LOCKED;
> - migrate_vma_pages(&mig);
> out_finalize:
> migrate_vma_finalize(&mig);
> return ret;
Though this patch did not solve the specific problem, I am running into,
my tests did not expose any regression.
Tested-by: Ram Pai <linuxram@us.ibm.com>
Reviewed-by: Ram Pai <linuxram@us.ibm.com>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-12-08 14:22 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-12-03 5:08 [PATCH] powerpc/book3s_hv_uvmem: Check for failed page migration Alistair Popple
2020-12-04 10:18 ` Bharata B Rao
2020-12-04 16:52 ` Ram Pai
2020-12-07 0:13 ` Alistair Popple
2020-12-06 23:55 ` Alistair Popple
2020-12-08 14:20 ` Ram Pai
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).