All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Huang\, Ying" <ying.huang@intel.com>
To: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	linux-mm@kvack.org, linux-kernel@vger.kernel.org,
	"Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>,
	Andrea Arcangeli <aarcange@redhat.com>,
	Michal Hocko <mhocko@suse.com>,
	Johannes Weiner <hannes@cmpxchg.org>,
	Shaohua Li <shli@kernel.org>, Hugh Dickins <hughd@google.com>,
	Minchan Kim <minchan@kernel.org>, Rik van Riel <riel@redhat.com>,
	Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>,
	Zi Yan <zi.yan@cs.rutgers.edu>,
	Daniel Jordan <daniel.m.jordan@oracle.com>
Subject: Re: [PATCH -mm -v4 05/21] mm, THP, swap: Support PMD swap mapping in free_swap_and_cache()/swap_free()
Date: Wed, 11 Jul 2018 09:28:13 +0800	[thread overview]
Message-ID: <87h8l6h87m.fsf@yhuang-dev.intel.com> (raw)
In-Reply-To: <836c95a7-5f03-6d9e-6f0a-839b5fb8ba99@linux.intel.com> (Dave Hansen's message of "Tue, 10 Jul 2018 07:07:39 -0700")

Dave Hansen <dave.hansen@linux.intel.com> writes:

> On 07/10/2018 12:13 AM, Huang, Ying wrote:
>> Dave Hansen <dave.hansen@linux.intel.com> writes:
>>> The code non-resuse was, and continues to be, IMNHO, one of the largest
>>> sources of bugs with the original THP implementation.  It might be
>>> infeasible to do here, but let's at least give it as much of a go as we can.
>> 
>> I totally agree that we should unify the code path for huge and normal
>> page/swap if possible.  One concern is code size for !CONFIG_THP_SWAP.
>
> I've honestly never heard that as an argument before.  In general, our
> .c files implement *full* functionality: the most complex case.  The
> headers #ifdef that functionality down because of our .config or
> architecture.
>
> The thing that matters here is debugging and reviewing the _complicated_
> case, IMNHO.

I agree with your point here.  I will try it and measure the code size
change too.

>> The original method is good for that.  The new method may introduce some
>> huge swap related code that is hard to be eliminated for
>> !CONFIG_THP_SWAP.  Andrew Morton pointed this out for the patchset of
>> the first step of the THP swap optimization.
>> 
>> This may be mitigated at least partly via,
>> 
>> `
>> #ifdef CONFIG_THP_SWAP
>> #define nr_swap_entries(nr)          (nr)
>> #else
>> #define nr_swap_entries(nr)          1
>> #endif
>> 
>> void do_something(swp_entry_t entry, int __nr_entries)
>> {
>>         int i, nr_entries = nr_swap_entries(__nr_entries);
>> 
>>         if (nr_entries = SWAPFILE_CLUSTER)
>>                 ; /* huge swap specific */
>>         else
>>                 ; /* normal swap specific */
>> 
>>         for (i = 0; i < nr_entries; i++) {
>>                 ; /* do something for each entry */
>>         }
>> 
>>         /* ... */
>> }
>> `
>
> While that isn't perfect, it's better than the current state of things.
>
> While you are refactoring things, I think you also need to take a good
> look at roughly chopping this series in half by finding another stopping
> point.  You've done a great job so far of trickling this functionality
> in so far, but 21 patches is quite a bit, and the set is only going to
> get larger.

Yes.  The patchset is too large.  I will try to reduce it if possible.
At least [21/21] can be separated.  [02/21] may be sent separately
too.  Other parts are hard, THP swapin and creating/supporting PMD swap
mapping need to be in one patchset.

Best Regards,
Huang, Ying

WARNING: multiple messages have this Message-ID (diff)
From: "Huang\, Ying" <ying.huang@intel.com>
To: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Andrew Morton <akpm@linux-foundation.org>, <linux-mm@kvack.org>,
	<linux-kernel@vger.kernel.org>,
	"Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>,
	Andrea Arcangeli <aarcange@redhat.com>,
	Michal Hocko <mhocko@suse.com>,
	Johannes Weiner <hannes@cmpxchg.org>,
	"Shaohua Li" <shli@kernel.org>, Hugh Dickins <hughd@google.com>,
	Minchan Kim <minchan@kernel.org>, Rik van Riel <riel@redhat.com>,
	Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>,
	Zi Yan <zi.yan@cs.rutgers.edu>,
	Daniel Jordan <daniel.m.jordan@oracle.com>
Subject: Re: [PATCH -mm -v4 05/21] mm, THP, swap: Support PMD swap mapping in free_swap_and_cache()/swap_free()
Date: Wed, 11 Jul 2018 09:28:13 +0800	[thread overview]
Message-ID: <87h8l6h87m.fsf@yhuang-dev.intel.com> (raw)
In-Reply-To: <836c95a7-5f03-6d9e-6f0a-839b5fb8ba99@linux.intel.com> (Dave Hansen's message of "Tue, 10 Jul 2018 07:07:39 -0700")

Dave Hansen <dave.hansen@linux.intel.com> writes:

> On 07/10/2018 12:13 AM, Huang, Ying wrote:
>> Dave Hansen <dave.hansen@linux.intel.com> writes:
>>> The code non-resuse was, and continues to be, IMNHO, one of the largest
>>> sources of bugs with the original THP implementation.  It might be
>>> infeasible to do here, but let's at least give it as much of a go as we can.
>> 
>> I totally agree that we should unify the code path for huge and normal
>> page/swap if possible.  One concern is code size for !CONFIG_THP_SWAP.
>
> I've honestly never heard that as an argument before.  In general, our
> .c files implement *full* functionality: the most complex case.  The
> headers #ifdef that functionality down because of our .config or
> architecture.
>
> The thing that matters here is debugging and reviewing the _complicated_
> case, IMNHO.

I agree with your point here.  I will try it and measure the code size
change too.

>> The original method is good for that.  The new method may introduce some
>> huge swap related code that is hard to be eliminated for
>> !CONFIG_THP_SWAP.  Andrew Morton pointed this out for the patchset of
>> the first step of the THP swap optimization.
>> 
>> This may be mitigated at least partly via,
>> 
>> `
>> #ifdef CONFIG_THP_SWAP
>> #define nr_swap_entries(nr)          (nr)
>> #else
>> #define nr_swap_entries(nr)          1
>> #endif
>> 
>> void do_something(swp_entry_t entry, int __nr_entries)
>> {
>>         int i, nr_entries = nr_swap_entries(__nr_entries);
>> 
>>         if (nr_entries = SWAPFILE_CLUSTER)
>>                 ; /* huge swap specific */
>>         else
>>                 ; /* normal swap specific */
>> 
>>         for (i = 0; i < nr_entries; i++) {
>>                 ; /* do something for each entry */
>>         }
>> 
>>         /* ... */
>> }
>> `
>
> While that isn't perfect, it's better than the current state of things.
>
> While you are refactoring things, I think you also need to take a good
> look at roughly chopping this series in half by finding another stopping
> point.  You've done a great job so far of trickling this functionality
> in so far, but 21 patches is quite a bit, and the set is only going to
> get larger.

Yes.  The patchset is too large.  I will try to reduce it if possible.
At least [21/21] can be separated.  [02/21] may be sent separately
too.  Other parts are hard, THP swapin and creating/supporting PMD swap
mapping need to be in one patchset.

Best Regards,
Huang, Ying

  reply	other threads:[~2018-07-11  1:28 UTC|newest]

Thread overview: 100+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-06-22  3:51 [PATCH -mm -v4 00/21] mm, THP, swap: Swapout/swapin THP in one piece Huang, Ying
2018-06-22  3:51 ` Huang, Ying
2018-06-22  3:51 ` [PATCH -mm -v4 01/21] mm, THP, swap: Enable PMD swap operations for CONFIG_THP_SWAP Huang, Ying
2018-07-07 21:11   ` Dan Williams
2018-07-09  5:40     ` Huang, Ying
2018-07-09  5:40       ` Huang, Ying
2018-07-09  6:08       ` Dan Williams
2018-07-09  6:34         ` Huang, Ying
2018-07-09  6:34           ` Huang, Ying
2018-07-09 15:59   ` Dave Hansen
2018-07-10  1:08     ` Huang, Ying
2018-07-10  1:08       ` Huang, Ying
2018-06-22  3:51 ` [PATCH -mm -v4 02/21] mm, THP, swap: Make CONFIG_THP_SWAP depends on CONFIG_SWAP Huang, Ying
2018-07-07 21:12   ` Dan Williams
2018-07-09  6:34     ` Huang, Ying
2018-07-09  6:34       ` Huang, Ying
2018-07-09 16:00   ` Dave Hansen
2018-07-10  1:19     ` Huang, Ying
2018-07-10  1:19       ` Huang, Ying
2018-07-10  1:59       ` Dave Hansen
2018-07-10  5:26         ` Huang, Ying
2018-07-10  5:26           ` Huang, Ying
2018-06-22  3:51 ` [PATCH -mm -v4 03/21] mm, THP, swap: Support PMD swap mapping in swap_duplicate() Huang, Ying
2018-06-29  6:04   ` Matthew Wilcox
2018-07-02  5:19     ` Huang, Ying
2018-07-02  5:19       ` Huang, Ying
2018-07-07 23:22   ` Dan Williams
2018-07-09  7:38     ` Huang, Ying
2018-07-09  7:38       ` Huang, Ying
2018-07-09 16:51   ` Dave Hansen
2018-07-10  6:44     ` Huang, Ying
2018-07-10  6:44       ` Huang, Ying
2018-07-10 13:50       ` Dave Hansen
2018-07-11  0:59         ` Huang, Ying
2018-07-11  0:59           ` Huang, Ying
2018-06-22  3:51 ` [PATCH -mm -v4 04/21] mm, THP, swap: Support PMD swap mapping in swapcache_free_cluster() Huang, Ying
2018-07-09 17:11   ` Dave Hansen
2018-07-10  6:53     ` Huang, Ying
2018-07-10  6:53       ` Huang, Ying
2018-07-10 13:54       ` Dave Hansen
2018-07-11  1:08         ` Huang, Ying
2018-07-11  1:08           ` Huang, Ying
2018-06-22  3:51 ` [PATCH -mm -v4 05/21] mm, THP, swap: Support PMD swap mapping in free_swap_and_cache()/swap_free() Huang, Ying
2018-07-05 18:33   ` Daniel Jordan
2018-07-06 12:49     ` Huang, Ying
2018-07-06 12:49       ` Huang, Ying
2018-07-09 17:19   ` Dave Hansen
2018-07-10  7:13     ` Huang, Ying
2018-07-10  7:13       ` Huang, Ying
2018-07-10 14:07       ` Dave Hansen
2018-07-11  1:28         ` Huang, Ying [this message]
2018-07-11  1:28           ` Huang, Ying
2018-06-22  3:51 ` [PATCH -mm -v4 06/21] mm, THP, swap: Support PMD swap mapping when splitting huge PMD Huang, Ying
2018-06-22  3:51 ` [PATCH -mm -v4 07/21] mm, THP, swap: Support PMD swap mapping in split_swap_cluster() Huang, Ying
2018-06-22  3:51 ` [PATCH -mm -v4 08/21] mm, THP, swap: Support to read a huge swap cluster for swapin a THP Huang, Ying
2018-06-29  6:21   ` Matthew Wilcox
2018-07-02  6:02     ` Huang, Ying
2018-07-02  6:02       ` Huang, Ying
2018-07-04  0:12   ` Daniel Jordan
2018-07-04  2:24     ` Huang, Ying
2018-07-04  2:24       ` Huang, Ying
2018-06-22  3:51 ` [PATCH -mm -v4 09/21] mm, THP, swap: Swapin a THP as a whole Huang, Ying
2018-06-22  3:51 ` [PATCH -mm -v4 10/21] mm, THP, swap: Support to count THP swapin and its fallback Huang, Ying
2018-06-22  3:51 ` [PATCH -mm -v4 11/21] mm, THP, swap: Add sysfs interface to configure THP swapin Huang, Ying
2018-06-22  3:51 ` [PATCH -mm -v4 12/21] mm, THP, swap: Support PMD swap mapping in swapoff Huang, Ying
2018-06-22  3:51 ` [PATCH -mm -v4 13/21] mm, THP, swap: Support PMD swap mapping in madvise_free() Huang, Ying
2018-06-22  3:51 ` [PATCH -mm -v4 14/21] mm, cgroup, THP, swap: Support to move swap account for PMD swap mapping Huang, Ying
2018-07-09 17:20   ` Daniel Jordan
2018-07-10  7:49     ` Huang, Ying
2018-07-10  7:49       ` Huang, Ying
2018-07-10 22:49       ` Daniel Jordan
2018-06-22  3:51 ` [PATCH -mm -v4 15/21] mm, THP, swap: Support to copy PMD swap mapping when fork() Huang, Ying
2018-06-22  3:51 ` [PATCH -mm -v4 16/21] mm, THP, swap: Free PMD swap mapping when zap_huge_pmd() Huang, Ying
2018-06-22  3:51 ` [PATCH -mm -v4 17/21] mm, THP, swap: Support PMD swap mapping for MADV_WILLNEED Huang, Ying
2018-06-22  3:51 ` [PATCH -mm -v4 18/21] mm, THP, swap: Support PMD swap mapping in mincore() Huang, Ying
2018-06-22  3:51 ` [PATCH -mm -v4 19/21] mm, THP, swap: Support PMD swap mapping in common path Huang, Ying
2018-06-22  3:51 ` [PATCH -mm -v4 20/21] mm, THP, swap: create PMD swap mapping when unmap the THP Huang, Ying
2018-06-22  3:51 ` [PATCH -mm -v4 21/21] mm, THP: Avoid to split THP when reclaim MADV_FREE THP Huang, Ying
2018-06-28  4:51 ` [PATCH -mm -v4 00/21] mm, THP, swap: Swapout/swapin THP in one piece Andrew Morton
2018-06-28  5:29   ` Huang, Ying
2018-06-28  5:29     ` Huang, Ying
2018-06-28  5:31     ` Andrew Morton
2018-06-28  5:31       ` Andrew Morton
2018-06-28  5:35       ` Huang, Ying
2018-06-28  5:35         ` Huang, Ying
2018-06-28  6:18         ` Andrew Morton
2018-06-28  6:18           ` Andrew Morton
2018-06-28  9:03           ` Matthew Wilcox
2018-06-28  9:03             ` Matthew Wilcox
2018-06-29  1:17             ` Huang, Ying
2018-06-29  1:17               ` Huang, Ying
2018-06-29  5:57               ` Matthew Wilcox
2018-07-02  5:19                 ` Huang, Ying
2018-07-02  5:19                   ` Huang, Ying
2018-07-04  2:11   ` Sergey Senozhatsky
2018-07-04  2:20     ` Huang, Ying
2018-07-04  2:20       ` Huang, Ying
2018-07-04  2:27       ` Sergey Senozhatsky
2018-07-04  2:59         ` Huang, Ying
2018-07-04  2:59           ` Huang, Ying

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87h8l6h87m.fsf@yhuang-dev.intel.com \
    --to=ying.huang@intel.com \
    --cc=aarcange@redhat.com \
    --cc=akpm@linux-foundation.org \
    --cc=daniel.m.jordan@oracle.com \
    --cc=dave.hansen@linux.intel.com \
    --cc=hannes@cmpxchg.org \
    --cc=hughd@google.com \
    --cc=kirill.shutemov@linux.intel.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mhocko@suse.com \
    --cc=minchan@kernel.org \
    --cc=n-horiguchi@ah.jp.nec.com \
    --cc=riel@redhat.com \
    --cc=shli@kernel.org \
    --cc=zi.yan@cs.rutgers.edu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.