From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CCFDBC77B70 for ; Thu, 6 Apr 2023 01:07:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234351AbjDFBHY (ORCPT ); Wed, 5 Apr 2023 21:07:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45200 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234461AbjDFBHP (ORCPT ); Wed, 5 Apr 2023 21:07:15 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 17F176EB9; Wed, 5 Apr 2023 18:07:03 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id EDE60642A8; Thu, 6 Apr 2023 01:07:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 52240C4339B; Thu, 6 Apr 2023 01:07:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1680743222; bh=OryYMC+pWp2Pf+zbFTCcasL4aTIPZve2waQFDDTWLsQ=; h=Date:To:From:Subject:From; b=sZYJsoxUaZPW5wbPk9NLFbr2zl/s+RgCxipkAhNHMUnLD7l+JEeOYhfhrmWZnXeo6 Rc9pkODYwjbM+MR2Ksn0UsU5bhB1hPQ1/siwwXvLFdL/UdpETQPeZVMWz1U4cXxcfL BGZkuI1Zx30I4ps3mzax6JuZ6ozsNlUob0AAoDjw= Date: Wed, 05 Apr 2023 18:07:01 -0700 To: mm-commits@vger.kernel.org, willy@infradead.org, stable@vger.kernel.org, rcampbell@nvidia.com, jhubbard@nvidia.com, hch@infradead.org, david@redhat.com, apopple@nvidia.com, akpm@linux-foundation.org From: Andrew Morton Subject: [merged mm-hotfixes-stable] mm-take-a-page-reference-when-removing-device-exclusive-entries.patch removed from -mm tree Message-Id: <20230406010702.52240C4339B@smtp.kernel.org> Precedence: bulk Reply-To: linux-kernel@vger.kernel.org List-ID: X-Mailing-List: mm-commits@vger.kernel.org The quilt patch titled Subject: mm: take a page reference when removing device exclusive entries has been removed from the -mm tree. Its filename was mm-take-a-page-reference-when-removing-device-exclusive-entries.patch This patch was dropped because it was merged into the mm-hotfixes-stable branch of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm ------------------------------------------------------ From: Alistair Popple Subject: mm: take a page reference when removing device exclusive entries Date: Thu, 30 Mar 2023 12:25:19 +1100 Device exclusive page table entries are used to prevent CPU access to a page whilst it is being accessed from a device. Typically this is used to implement atomic operations when the underlying bus does not support atomic access. When a CPU thread encounters a device exclusive entry it locks the page and restores the original entry after calling mmu notifiers to signal drivers that exclusive access is no longer available. The device exclusive entry holds a reference to the page making it safe to access the struct page whilst the entry is present. However the fault handling code does not hold the PTL when taking the page lock. This means if there are multiple threads faulting concurrently on the device exclusive entry one will remove the entry whilst others will wait on the page lock without holding a reference. This can lead to threads locking or waiting on a folio with a zero refcount. Whilst mmap_lock prevents the pages getting freed via munmap() they may still be freed by a migration. This leads to warnings such as PAGE_FLAGS_CHECK_AT_FREE due to the page being locked when the refcount drops to zero. Fix this by trying to take a reference on the folio before locking it. The code already checks the PTE under the PTL and aborts if the entry is no longer there. It is also possible the folio has been unmapped, freed and re-allocated allowing a reference to be taken on an unrelated folio. This case is also detected by the PTE check and the folio is unlocked without further changes. Link: https://lkml.kernel.org/r/20230330012519.804116-1-apopple@nvidia.com Fixes: b756a3b5e7ea ("mm: device exclusive memory access") Signed-off-by: Alistair Popple Reviewed-by: Ralph Campbell Reviewed-by: John Hubbard Acked-by: David Hildenbrand Cc: Matthew Wilcox (Oracle) Cc: Christoph Hellwig Cc: Signed-off-by: Andrew Morton --- mm/memory.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) --- a/mm/memory.c~mm-take-a-page-reference-when-removing-device-exclusive-entries +++ a/mm/memory.c @@ -3563,8 +3563,21 @@ static vm_fault_t remove_device_exclusiv struct vm_area_struct *vma = vmf->vma; struct mmu_notifier_range range; - if (!folio_lock_or_retry(folio, vma->vm_mm, vmf->flags)) + /* + * We need a reference to lock the folio because we don't hold + * the PTL so a racing thread can remove the device-exclusive + * entry and unmap it. If the folio is free the entry must + * have been removed already. If it happens to have already + * been re-allocated after being freed all we do is lock and + * unlock it. + */ + if (!folio_try_get(folio)) + return 0; + + if (!folio_lock_or_retry(folio, vma->vm_mm, vmf->flags)) { + folio_put(folio); return VM_FAULT_RETRY; + } mmu_notifier_range_init_owner(&range, MMU_NOTIFY_EXCLUSIVE, 0, vma->vm_mm, vmf->address & PAGE_MASK, (vmf->address & PAGE_MASK) + PAGE_SIZE, NULL); @@ -3577,6 +3590,7 @@ static vm_fault_t remove_device_exclusiv pte_unmap_unlock(vmf->pte, vmf->ptl); folio_unlock(folio); + folio_put(folio); mmu_notifier_invalidate_range_end(&range); return 0; _ Patches currently in -mm which might be from apopple@nvidia.com are