[PATCH] mm: avoid tail page refcounting on non-THP compound pages

[PATCH] mm: avoid tail page refcounting on non-THP compound pages

[Kirill A. Shutemov]

THP uses tail page refcounting to be able to split huge page at any
time. Tail page refcounting is not needed for rest users of compound
pages and it's harmful because of overhead.

We try to exclude non-THP pages from tail page refcounting using
__compound_tail_refcounted() check. It excludes most common non-THP
compound pages: SL*B and hugetlb, but it doesn't catch rest of
__GFP_COMP users -- drivers.

And it's not only about overhead.

Drivers might want to use compound pages to get refcounting semantics
suitable for mapping high-order pages to userspace. But tail page
refcounting breaks it.

Tail page refcounting uses ->_mapcount in tail pages to store GUP pins
on them. It means GUP pins would affect page_mapcount() for tail pages.
It's not a problem for THP, because it never maps tail pages. But unlike
THP, drivers map parts of compound pages with PTEs and it makes
page_mapcount() be called for tail pages.

In particular, GUP pins would shift PSS up and affect /proc/kpagecount
for such pages. But, I'm not aware about anything which can lead to
crash or other serious misbehaviour.

Since currently all THP pages are anonymous and all drivers pages are
not, we can fix the __compound_tail_refcounted() check by requiring
PageAnon() to enable tail page refcounting.

Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
 include/linux/mm.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/linux/mm.h b/include/linux/mm.h
index 4a3a38522ab4..16fe322b66ea 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -456,7 +456,7 @@ static inline int page_count(struct page *page)
 
 static inline bool __compound_tail_refcounted(struct page *page)
 {
-	return !PageSlab(page) && !PageHeadHuge(page);
+	return PageAnon(page) && !PageSlab(page) && !PageHeadHuge(page);
 }
 
 /*
-- 
2.1.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] mm: avoid tail page refcounting on non-THP compound pages

[Hugh Dickins]

On Thu, 26 Mar 2015, Kirill A. Shutemov wrote:

> THP uses tail page refcounting to be able to split huge page at any
> time. Tail page refcounting is not needed for rest users of compound
> pages and it's harmful because of overhead.
> 
> We try to exclude non-THP pages from tail page refcounting using
> __compound_tail_refcounted() check. It excludes most common non-THP
> compound pages: SL*B and hugetlb, but it doesn't catch rest of
> __GFP_COMP users -- drivers.
> 
> And it's not only about overhead.
> 
> Drivers might want to use compound pages to get refcounting semantics
> suitable for mapping high-order pages to userspace. But tail page
> refcounting breaks it.
> 
> Tail page refcounting uses ->_mapcount in tail pages to store GUP pins
> on them. It means GUP pins would affect page_mapcount() for tail pages.
> It's not a problem for THP, because it never maps tail pages. But unlike
> THP, drivers map parts of compound pages with PTEs and it makes
> page_mapcount() be called for tail pages.
> 
> In particular, GUP pins would shift PSS up and affect /proc/kpagecount
> for such pages. But, I'm not aware about anything which can lead to
> crash or other serious misbehaviour.
> 
> Since currently all THP pages are anonymous and all drivers pages are
> not, we can fix the __compound_tail_refcounted() check by requiring
> PageAnon() to enable tail page refcounting.
> 
> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>

Acked-by: Hugh Dickins <hughd@google.com>

> ---
>  include/linux/mm.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/include/linux/mm.h b/include/linux/mm.h
> index 4a3a38522ab4..16fe322b66ea 100644
> --- a/include/linux/mm.h
> +++ b/include/linux/mm.h
> @@ -456,7 +456,7 @@ static inline int page_count(struct page *page)
>  
>  static inline bool __compound_tail_refcounted(struct page *page)
>  {
> -	return !PageSlab(page) && !PageHeadHuge(page);
> +	return PageAnon(page) && !PageSlab(page) && !PageHeadHuge(page);
>  }
>  
>  /*
> -- 
> 2.1.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] mm: avoid tail page refcounting on non-THP compound pages

[Hugh Dickins]

On Wed, 25 Mar 2015, Hugh Dickins wrote:
> On Thu, 26 Mar 2015, Kirill A. Shutemov wrote:
> 
> > THP uses tail page refcounting to be able to split huge page at any
> > time. Tail page refcounting is not needed for rest users of compound
> > pages and it's harmful because of overhead.
> > 
> > We try to exclude non-THP pages from tail page refcounting using
> > __compound_tail_refcounted() check. It excludes most common non-THP
> > compound pages: SL*B and hugetlb, but it doesn't catch rest of
> > __GFP_COMP users -- drivers.
> > 
> > And it's not only about overhead.
> > 
> > Drivers might want to use compound pages to get refcounting semantics
> > suitable for mapping high-order pages to userspace. But tail page
> > refcounting breaks it.
> > 
> > Tail page refcounting uses ->_mapcount in tail pages to store GUP pins
> > on them. It means GUP pins would affect page_mapcount() for tail pages.
> > It's not a problem for THP, because it never maps tail pages. But unlike
> > THP, drivers map parts of compound pages with PTEs and it makes
> > page_mapcount() be called for tail pages.
> > 
> > In particular, GUP pins would shift PSS up and affect /proc/kpagecount
> > for such pages. But, I'm not aware about anything which can lead to
> > crash or other serious misbehaviour.
> > 
> > Since currently all THP pages are anonymous and all drivers pages are
> > not, we can fix the __compound_tail_refcounted() check by requiring
> > PageAnon() to enable tail page refcounting.
> > 
> > Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> 
> Acked-by: Hugh Dickins <hughd@google.com>

Oh, hold on a moment: does this actually build in a tree without your
page-flags.h consolidation?  It didn't when I tried to add a PageAnon
test there for my series against v3.19, has something changed in v4.0?

> 
> > ---
> >  include/linux/mm.h | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/include/linux/mm.h b/include/linux/mm.h
> > index 4a3a38522ab4..16fe322b66ea 100644
> > --- a/include/linux/mm.h
> > +++ b/include/linux/mm.h
> > @@ -456,7 +456,7 @@ static inline int page_count(struct page *page)
> >  
> >  static inline bool __compound_tail_refcounted(struct page *page)
> >  {
> > -	return !PageSlab(page) && !PageHeadHuge(page);
> > +	return PageAnon(page) && !PageSlab(page) && !PageHeadHuge(page);
> >  }
> >  
> >  /*
> > -- 
> > 2.1.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] mm: avoid tail page refcounting on non-THP compound pages

[Kirill A. Shutemov]

On Wed, Mar 25, 2015 at 03:48:48PM -0700, Hugh Dickins wrote:
> On Wed, 25 Mar 2015, Hugh Dickins wrote:
> > On Thu, 26 Mar 2015, Kirill A. Shutemov wrote:
> > 
> > > THP uses tail page refcounting to be able to split huge page at any
> > > time. Tail page refcounting is not needed for rest users of compound
> > > pages and it's harmful because of overhead.
> > > 
> > > We try to exclude non-THP pages from tail page refcounting using
> > > __compound_tail_refcounted() check. It excludes most common non-THP
> > > compound pages: SL*B and hugetlb, but it doesn't catch rest of
> > > __GFP_COMP users -- drivers.
> > > 
> > > And it's not only about overhead.
> > > 
> > > Drivers might want to use compound pages to get refcounting semantics
> > > suitable for mapping high-order pages to userspace. But tail page
> > > refcounting breaks it.
> > > 
> > > Tail page refcounting uses ->_mapcount in tail pages to store GUP pins
> > > on them. It means GUP pins would affect page_mapcount() for tail pages.
> > > It's not a problem for THP, because it never maps tail pages. But unlike
> > > THP, drivers map parts of compound pages with PTEs and it makes
> > > page_mapcount() be called for tail pages.
> > > 
> > > In particular, GUP pins would shift PSS up and affect /proc/kpagecount
> > > for such pages. But, I'm not aware about anything which can lead to
> > > crash or other serious misbehaviour.
> > > 
> > > Since currently all THP pages are anonymous and all drivers pages are
> > > not, we can fix the __compound_tail_refcounted() check by requiring
> > > PageAnon() to enable tail page refcounting.
> > > 
> > > Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> > 
> > Acked-by: Hugh Dickins <hughd@google.com>
> 
> Oh, hold on a moment: does this actually build in a tree without your
> page-flags.h consolidation?  It didn't when I tried to add a PageAnon
> test there for my series against v3.19, has something changed in v4.0?

No. I haven't tried to build it without my patchset, but it seems it
wouldn't.

Just check: it would build for me on top of [PATCH 01/16], you've acked.

-- 
 Kirill A. Shutemov

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] mm: avoid tail page refcounting on non-THP compound pages

[Hugh Dickins]

On Thu, 26 Mar 2015, Kirill A. Shutemov wrote:
> On Wed, Mar 25, 2015 at 03:48:48PM -0700, Hugh Dickins wrote:
> > On Wed, 25 Mar 2015, Hugh Dickins wrote:
> > > On Thu, 26 Mar 2015, Kirill A. Shutemov wrote:
> > > > 
> > > > Since currently all THP pages are anonymous and all drivers pages are
> > > > not, we can fix the __compound_tail_refcounted() check by requiring
> > > > PageAnon() to enable tail page refcounting.
> > > > 
> > > > Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> > > 
> > > Acked-by: Hugh Dickins <hughd@google.com>
> > 
> > Oh, hold on a moment: does this actually build in a tree without your
> > page-flags.h consolidation?  It didn't when I tried to add a PageAnon
> > test there for my series against v3.19, has something changed in v4.0?
> 
> No. I haven't tried to build it without my patchset, but it seems it
> wouldn't.
> 
> Just check: it would build for me on top of [PATCH 01/16], you've acked.

Yes, I'm happy with your 1/16 (which is already there in mmotm),
it's just that I'd imagined this __compound_tail_refcounted() fix
should go to v4.0 (if not stable too: you've decided against, okay).

What do you think, should Andrew hold it back for v4.1, or should
your page-flags.h accelerate into v4.0 as precondition for this fix?

Either is fine with me; but if the latter, then a week's exposure
in linux-next first would probably be best.

Hugh

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH] mm: avoid tail page refcounting on non-THP compound pages

[Kirill A. Shutemov]

THP uses tail page refcounting to be able to split huge page at any
time. Tail page refcounting is not needed for rest users of compound
pages and it's harmful because of overhead.

We try to exclude non-THP pages from tail page refcounting using
__compound_tail_refcounted() check. It excludes most common non-THP
compound pages: SL*B and hugetlb, but it doesn't catch rest of
__GFP_COMP users -- drivers.

And it's not only about overhead.

Drivers might want to use compound pages to get refcounting semantics
suitable for mapping high-order pages to userspace. But tail page
refcounting breaks it.

Tail page refcounting uses ->_mapcount in tail pages to store GUP pins
on them. It means GUP pins would affect page_mapcount() for tail pages.
It's not a problem for THP, because it never maps tail pages. But unlike
THP, drivers map parts of compound pages with PTEs and it makes
page_mapcount() be called for tail pages.

In particular, GUP pins would shift PSS up and affect /proc/kpagecount
for such pages. But, I'm not aware about anything which can lead to
crash or other serious misbehaviour.

Since currently all THP pages are anonymous and all drivers pages are
not, we can fix the __compound_tail_refcounted() check by requiring
PageAnon() to enable tail page refcounting.

Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
 include/linux/mm.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/linux/mm.h b/include/linux/mm.h
index 4a3a38522ab4..16fe322b66ea 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -456,7 +456,7 @@ static inline int page_count(struct page *page)
 
 static inline bool __compound_tail_refcounted(struct page *page)
 {
-	return !PageSlab(page) && !PageHeadHuge(page);
+	return PageAnon(page) && !PageSlab(page) && !PageHeadHuge(page);
 }
 
 /*
-- 
2.1.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] mm: avoid tail page refcounting on non-THP compound pages

[Kirill A. Shutemov]

Please ignore. I've sent wrong patch.

-- 
 Kirill A. Shutemov

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>