Re: [PATCH RFC 02/13] mm/huge_memory: fix rejection of swap cache folios with a mapping

["Zi Yan" <ziy@nvidia.com>]

On Sat Aug 8, 2026 at 2:14 PM EDT, Kairui Song wrote:
> On Sun, Aug 9, 2026 at 2:02 AM Zi Yan <ziy@nvidia.com> wrote:
>>
>> On Fri Aug 7, 2026 at 5:17 PM EDT, Kairui Song via B4 Relay wrote:
>> > From: Kairui Song <kasong@tencent.com>
>> >
>> > A folio in the swap cache cannot be split if it has a mapping (shmem).
>> > The split code only checks for this in __folio_freeze_and_split_unmapped,
>> > after the folio ref has been frozen and the NR_SHMEM_THPS/NR_FILE_THPS
>> > counters have been decremented, and returns -EINVAL without unfreezing
>> > the folio or restoring the counters. That error path is fragile: if it
>> > is ever taken, the folio is left frozen and stuck, the counters are
>> > skewed, and the VM_WARN_ON_ONCE_FOLIO would fire for a state that is
>> > actually legitimate.
>> >
>> > Check for this case up front in folio_check_splittable and return
>> > -EINVAL before any state is modified. Under DEBUG_VM, the existing
>> > "Tried to split an unsplittable folio" warning in __folio_split
>> > reports the rejection.
>>
>> Should we return -EBUSY instead? -EINVAL means the caller should not
>> split a swapcache shmem with a mapping and the caller needs to avoid
>> that. The Fixes tag tells me a caller can split a swapcache shmem with a
>> mapping, so with -EINVAL, we will want to add checks at callers to avoid
>> it from happening.
>>
>
> I can drop the Fixes tags. I meant that there is already some

If it can happen, we want to fix it.

> defensive code that trying to catch it and return -EINVAL, however,
> that defensive code itself is flawed. If this situation occurs due to
> a bug or future misuse, the flawed code will causes the folio to get
> stuck in a frozen state. The defensive code should at least not make
> things worse.
>
> Fortunately I think no one needs to split a hybrid shmem swap cache
> folio, returning -EINVAL here may help catch any potential future
> misuse.

Sounds reasonable to me.

-- 
Best Regards,
Yan, Zi