From: Jason Gunthorpe <jgg@ziepe.ca>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: Alistair Popple <apopple@nvidia.com>,
ajd@linux.ibm.com, catalin.marinas@arm.com,
fbarrat@linux.ibm.com, iommu@lists.linux.dev,
jhubbard@nvidia.com, kevin.tian@intel.com, kvm@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org, linux-mm@kvack.org,
linuxppc-dev@lists.ozlabs.org, mpe@ellerman.id.au,
nicolinc@nvidia.com, npiggin@gmail.com, robin.murphy@arm.com,
seanjc@google.com, will@kernel.org, x86@kernel.org,
zhi.wang.linux@gmail.com
Subject: Re: [PATCH 3/4] mmu_notifiers: Call arch_invalidate_secondary_tlbs() when invalidating TLBs
Date: Tue, 18 Jul 2023 15:29:26 -0300 [thread overview]
Message-ID: <ZLbaBpfaAqigFzIT@ziepe.ca> (raw)
In-Reply-To: <20230718111759.5642b4c4ffd72ddd9c8aa29f@linux-foundation.org>
On Tue, Jul 18, 2023 at 11:17:59AM -0700, Andrew Morton wrote:
> On Tue, 18 Jul 2023 17:56:17 +1000 Alistair Popple <apopple@nvidia.com> wrote:
>
> > The arch_invalidate_secondary_tlbs() is an architecture specific mmu
> > notifier used to keep the TLB of secondary MMUs such as an IOMMU in
> > sync with the CPU page tables. Currently it is called from separate
> > code paths to the main CPU TLB invalidations. This can lead to a
> > secondary TLB not getting invalidated when required and makes it hard
> > to reason about when exactly the secondary TLB is invalidated.
> >
> > To fix this move the notifier call to the architecture specific TLB
> > maintenance functions for architectures that have secondary MMUs
> > requiring explicit software invalidations.
> >
> > This fixes a SMMU bug on ARM64. On ARM64 PTE permission upgrades
> > require a TLB invalidation. This invalidation is done by the
> > architecutre specific ptep_set_access_flags() which calls
> > flush_tlb_page() if required. However this doesn't call the notifier
> > resulting in infinite faults being generated by devices using the SMMU
> > if it has previously cached a read-only PTE in it's TLB.
>
> This sounds like a pretty serious bug. Can it happen in current
> released kernels? If so, is a -stable backport needed?
There are currently no in-kernel drivers using the IOMMU SVA API, so
the impact for -stable is sort of muted. But it is serious if you are
unlucky to hit it.
Jason
WARNING: multiple messages have this Message-ID (diff)
From: Jason Gunthorpe <jgg@ziepe.ca>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: kevin.tian@intel.com, x86@kernel.org, ajd@linux.ibm.com,
kvm@vger.kernel.org, catalin.marinas@arm.com,
Alistair Popple <apopple@nvidia.com>,
will@kernel.org, linux-kernel@vger.kernel.org, npiggin@gmail.com,
zhi.wang.linux@gmail.com, linux-mm@kvack.org,
iommu@lists.linux.dev, nicolinc@nvidia.com, jhubbard@nvidia.com,
fbarrat@linux.ibm.com, seanjc@google.com,
linuxppc-dev@lists.ozlabs.org,
linux-arm-kernel@lists.infradead.org, robin.murphy@arm.com
Subject: Re: [PATCH 3/4] mmu_notifiers: Call arch_invalidate_secondary_tlbs() when invalidating TLBs
Date: Tue, 18 Jul 2023 15:29:26 -0300 [thread overview]
Message-ID: <ZLbaBpfaAqigFzIT@ziepe.ca> (raw)
In-Reply-To: <20230718111759.5642b4c4ffd72ddd9c8aa29f@linux-foundation.org>
On Tue, Jul 18, 2023 at 11:17:59AM -0700, Andrew Morton wrote:
> On Tue, 18 Jul 2023 17:56:17 +1000 Alistair Popple <apopple@nvidia.com> wrote:
>
> > The arch_invalidate_secondary_tlbs() is an architecture specific mmu
> > notifier used to keep the TLB of secondary MMUs such as an IOMMU in
> > sync with the CPU page tables. Currently it is called from separate
> > code paths to the main CPU TLB invalidations. This can lead to a
> > secondary TLB not getting invalidated when required and makes it hard
> > to reason about when exactly the secondary TLB is invalidated.
> >
> > To fix this move the notifier call to the architecture specific TLB
> > maintenance functions for architectures that have secondary MMUs
> > requiring explicit software invalidations.
> >
> > This fixes a SMMU bug on ARM64. On ARM64 PTE permission upgrades
> > require a TLB invalidation. This invalidation is done by the
> > architecutre specific ptep_set_access_flags() which calls
> > flush_tlb_page() if required. However this doesn't call the notifier
> > resulting in infinite faults being generated by devices using the SMMU
> > if it has previously cached a read-only PTE in it's TLB.
>
> This sounds like a pretty serious bug. Can it happen in current
> released kernels? If so, is a -stable backport needed?
There are currently no in-kernel drivers using the IOMMU SVA API, so
the impact for -stable is sort of muted. But it is serious if you are
unlucky to hit it.
Jason
WARNING: multiple messages have this Message-ID (diff)
From: Jason Gunthorpe <jgg@ziepe.ca>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: Alistair Popple <apopple@nvidia.com>,
ajd@linux.ibm.com, catalin.marinas@arm.com,
fbarrat@linux.ibm.com, iommu@lists.linux.dev,
jhubbard@nvidia.com, kevin.tian@intel.com, kvm@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org, linux-mm@kvack.org,
linuxppc-dev@lists.ozlabs.org, mpe@ellerman.id.au,
nicolinc@nvidia.com, npiggin@gmail.com, robin.murphy@arm.com,
seanjc@google.com, will@kernel.org, x86@kernel.org,
zhi.wang.linux@gmail.com
Subject: Re: [PATCH 3/4] mmu_notifiers: Call arch_invalidate_secondary_tlbs() when invalidating TLBs
Date: Tue, 18 Jul 2023 15:29:26 -0300 [thread overview]
Message-ID: <ZLbaBpfaAqigFzIT@ziepe.ca> (raw)
In-Reply-To: <20230718111759.5642b4c4ffd72ddd9c8aa29f@linux-foundation.org>
On Tue, Jul 18, 2023 at 11:17:59AM -0700, Andrew Morton wrote:
> On Tue, 18 Jul 2023 17:56:17 +1000 Alistair Popple <apopple@nvidia.com> wrote:
>
> > The arch_invalidate_secondary_tlbs() is an architecture specific mmu
> > notifier used to keep the TLB of secondary MMUs such as an IOMMU in
> > sync with the CPU page tables. Currently it is called from separate
> > code paths to the main CPU TLB invalidations. This can lead to a
> > secondary TLB not getting invalidated when required and makes it hard
> > to reason about when exactly the secondary TLB is invalidated.
> >
> > To fix this move the notifier call to the architecture specific TLB
> > maintenance functions for architectures that have secondary MMUs
> > requiring explicit software invalidations.
> >
> > This fixes a SMMU bug on ARM64. On ARM64 PTE permission upgrades
> > require a TLB invalidation. This invalidation is done by the
> > architecutre specific ptep_set_access_flags() which calls
> > flush_tlb_page() if required. However this doesn't call the notifier
> > resulting in infinite faults being generated by devices using the SMMU
> > if it has previously cached a read-only PTE in it's TLB.
>
> This sounds like a pretty serious bug. Can it happen in current
> released kernels? If so, is a -stable backport needed?
There are currently no in-kernel drivers using the IOMMU SVA API, so
the impact for -stable is sort of muted. But it is serious if you are
unlucky to hit it.
Jason
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2023-07-18 18:29 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-07-18 7:56 [PATCH 0/4] Invalidate secondary IOMMU TLB on permission upgrade Alistair Popple
2023-07-18 7:56 ` Alistair Popple
2023-07-18 7:56 ` Alistair Popple
2023-07-18 7:56 ` [PATCH 1/4] mm_notifiers: Rename invalidate_range notifier Alistair Popple
2023-07-18 7:56 ` Alistair Popple
2023-07-18 7:56 ` Alistair Popple
2023-07-18 17:57 ` Jason Gunthorpe
2023-07-18 17:57 ` Jason Gunthorpe
2023-07-18 17:57 ` Jason Gunthorpe
2023-07-18 18:36 ` Andrew Morton
2023-07-18 18:36 ` Andrew Morton
2023-07-18 18:36 ` Andrew Morton
2023-07-18 23:49 ` Alistair Popple
2023-07-18 23:49 ` Alistair Popple
2023-07-18 23:49 ` Alistair Popple
2023-07-18 7:56 ` [PATCH 2/4] arm64/smmu: Use TLBI ASID when invalidating entire range Alistair Popple
2023-07-18 7:56 ` Alistair Popple
2023-07-18 7:56 ` Alistair Popple
2023-07-18 7:56 ` [PATCH 3/4] mmu_notifiers: Call arch_invalidate_secondary_tlbs() when invalidating TLBs Alistair Popple
2023-07-18 7:56 ` Alistair Popple
2023-07-18 7:56 ` Alistair Popple
2023-07-18 18:17 ` Andrew Morton
2023-07-18 18:17 ` Andrew Morton
2023-07-18 18:17 ` Andrew Morton
2023-07-18 18:29 ` Jason Gunthorpe [this message]
2023-07-18 18:29 ` Jason Gunthorpe
2023-07-18 18:29 ` Jason Gunthorpe
2023-07-18 7:56 ` [PATCH 4/4] mmu_notifiers: Don't invalidate secondary TLBs as part of mmu_notifier_invalidate_range_end() Alistair Popple
2023-07-18 7:56 ` Alistair Popple
2023-07-18 7:56 ` Alistair Popple
2023-07-19 3:04 ` [PATCH 0/4] Invalidate secondary IOMMU TLB on permission upgrade Anshuman Khandual
2023-07-19 3:04 ` Anshuman Khandual
2023-07-19 3:04 ` Anshuman Khandual
2023-07-19 3:14 ` Tian, Kevin
2023-07-19 3:14 ` Tian, Kevin
2023-07-19 3:14 ` Tian, Kevin
2023-07-19 5:42 ` Alistair Popple
2023-07-19 5:42 ` Alistair Popple
2023-07-19 5:42 ` Alistair Popple
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZLbaBpfaAqigFzIT@ziepe.ca \
--to=jgg@ziepe.ca \
--cc=ajd@linux.ibm.com \
--cc=akpm@linux-foundation.org \
--cc=apopple@nvidia.com \
--cc=catalin.marinas@arm.com \
--cc=fbarrat@linux.ibm.com \
--cc=iommu@lists.linux.dev \
--cc=jhubbard@nvidia.com \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=mpe@ellerman.id.au \
--cc=nicolinc@nvidia.com \
--cc=npiggin@gmail.com \
--cc=robin.murphy@arm.com \
--cc=seanjc@google.com \
--cc=will@kernel.org \
--cc=x86@kernel.org \
--cc=zhi.wang.linux@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.