Re: [PATCH v8 7/7] iommu/riscv: Paging domain support

[Zong Li <zong.li@sifive.com>]

On Tue, Jul 23, 2024 at 9:08 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
>
> On Fri, Jul 19, 2024 at 02:33:13PM +0800, Zong Li wrote:
>
> > > +static void riscv_iommu_free_paging_domain(struct iommu_domain *iommu_domain)
> > > +{
> > > +       struct riscv_iommu_domain *domain = iommu_domain_to_riscv(iommu_domain);
> > > +       const unsigned long pfn = virt_to_pfn(domain->pgd_root);
> > > +
> > > +       WARN_ON(!list_empty(&domain->bonds));
> >
> > Hi Tomasz,
> > I recently hit the issue here when I removed a device. I think we need
> > to unlink the device's bond in domain before releasing domain.
> > Do you mind pick the following modification in this patch? Thanks.
> >
> > diff --git a/drivers/iommu/riscv/iommu.c b/drivers/iommu/riscv/iommu.c
> > index 859cdd455576..f0707147e706 100644
> > --- a/drivers/iommu/riscv/iommu.c
> > +++ b/drivers/iommu/riscv/iommu.c
> > @@ -1541,6 +1541,8 @@ static void riscv_iommu_release_device(struct device *dev)
> >  {
> >         struct riscv_iommu_info *info = dev_iommu_priv_get(dev);
> >
> > +       riscv_iommu_bond_unlink(info->domain, dev);
> > +
> >         synchronize_rcu();
> >         kfree(info);
> >  }
>
> That does not seem the right place to put that, or something else has
> gone wrong becuase:
>
> > >  static const struct iommu_ops riscv_iommu_ops = {
> > > +       .pgsize_bitmap = SZ_4K,
> > >         .of_xlate = riscv_iommu_of_xlate,
> > >         .identity_domain = &riscv_iommu_identity_domain,
> > >         .blocked_domain = &riscv_iommu_blocking_domain,
> > >         .release_domain = &riscv_iommu_blocking_domain,
>
> The above use of release_domain will attach to the blocking domain
> prior to calling release_device and when doing so everything about any
> previous domain should have been fully cleaned up.
>
> So info->domain should be the blocking domain during release_device
> (if not that is another bug)
>
> And attaching the blocking domain should have cleared the
> domain->bonds of any prior paging domain.

Yes, you are right, there is no issue. I mixed up my codebase, which
doesn't support release_domain yet. Please ignore this, and thanks for
pointing it out.

>
> Jason