All of lore.kernel.org
 help / color / mirror / Atom feed
From: Jason Gunthorpe <jgg@nvidia.com>
To: Andrew Jones <andrew.jones@oss.qualcomm.com>
Cc: linux-riscv@lists.infradead.org, iommu@lists.linux.dev,
	linux-kernel@vger.kernel.org, tomasz.jeznach@linux.dev,
	tjeznach@rivosinc.com, joro@8bytes.org, will@kernel.org,
	robin.murphy@arm.com, pjw@kernel.org, palmer@dabbelt.com,
	anup@brainfault.org, tglx@kernel.org, kevin.tian@intel.com,
	fangyu.yu@linux.alibaba.com
Subject: Re: [PATCH v4 02/21] iommufd: Add iommufd_sw_map_msi()
Date: Thu, 20 Aug 2026 19:09:08 -0300	[thread overview]
Message-ID: <20260820220908.GE244917@nvidia.com> (raw)
In-Reply-To: <20260820214150.545737-3-andrew.jones@oss.qualcomm.com>

On Thu, Aug 20, 2026 at 11:41:31PM +0200, Andrew Jones wrote:
> Add a descriptor-free counterpart to iommufd_sw_msi(). The existing
> function is tied to a struct msi_desc and stores the result in the
> descriptor. This variant returns the IOVA directly so callers can
> pre-map MSI targets before any descriptor has been allocated.

I haven't grasped why it is like this?

The cover letter says:

  ARM can map a fixed doorbell PA per ITS and cache its IOVA on the
  descriptor because affinity changes only hardware routing, not the
  composed address. An IMSIC target PA changes with affinity, and MSI
  composition may run in atomic context, so mapping the selected target on
  demand is not an option. This series pre-maps the supervisor IMSIC page
  for every possible CPU into a domain-local PA-to-IOVA table when remapped
  IRQs are first allocated, allowing composition to select the target with
  an O(1) lookup. If iommufd replaces a paging domain while IRQs remain
  allocated, the incoming domain's table is rebuilt before it is attached.

There are a few confusiong things with this statement:
 1) Okay the IMSIC PA changes dynamically but since it can be
    premapped the PAs required is fixed and known. ARM doesn't change
    the PA dynamically?
 2) Why do you say mapping on demand is not possible? ARM's
    iommu_dma_prepare_msi() is not called in an atomic context and
    does do the iommu mapping.

What I rather expected was for riscv to have a PA window that is very
big and not just one page, eg adjust iommu_dma_prepare_msi() so you
can pass in the entire PA space that you need for the affinity
changes. Maybe this is a list of phys_addr_t ?

Then keep with the ARM flow where everything happens at the same
times as today. Instead of just mapping one page you map the entire
list.

I'm not keen on this design where things get pre-mapped into the
domain by the iommu driver, the ARM version is much easier to
understand :\

Jason

WARNING: multiple messages have this Message-ID (diff)
From: Jason Gunthorpe <jgg@nvidia.com>
To: Andrew Jones <andrew.jones@oss.qualcomm.com>
Cc: linux-riscv@lists.infradead.org, iommu@lists.linux.dev,
	linux-kernel@vger.kernel.org, tomasz.jeznach@linux.dev,
	tjeznach@rivosinc.com, joro@8bytes.org, will@kernel.org,
	robin.murphy@arm.com, pjw@kernel.org, palmer@dabbelt.com,
	anup@brainfault.org, tglx@kernel.org, kevin.tian@intel.com,
	fangyu.yu@linux.alibaba.com
Subject: Re: [PATCH v4 02/21] iommufd: Add iommufd_sw_map_msi()
Date: Thu, 20 Aug 2026 19:09:08 -0300	[thread overview]
Message-ID: <20260820220908.GE244917@nvidia.com> (raw)
In-Reply-To: <20260820214150.545737-3-andrew.jones@oss.qualcomm.com>

On Thu, Aug 20, 2026 at 11:41:31PM +0200, Andrew Jones wrote:
> Add a descriptor-free counterpart to iommufd_sw_msi(). The existing
> function is tied to a struct msi_desc and stores the result in the
> descriptor. This variant returns the IOVA directly so callers can
> pre-map MSI targets before any descriptor has been allocated.

I haven't grasped why it is like this?

The cover letter says:

  ARM can map a fixed doorbell PA per ITS and cache its IOVA on the
  descriptor because affinity changes only hardware routing, not the
  composed address. An IMSIC target PA changes with affinity, and MSI
  composition may run in atomic context, so mapping the selected target on
  demand is not an option. This series pre-maps the supervisor IMSIC page
  for every possible CPU into a domain-local PA-to-IOVA table when remapped
  IRQs are first allocated, allowing composition to select the target with
  an O(1) lookup. If iommufd replaces a paging domain while IRQs remain
  allocated, the incoming domain's table is rebuilt before it is attached.

There are a few confusiong things with this statement:
 1) Okay the IMSIC PA changes dynamically but since it can be
    premapped the PAs required is fixed and known. ARM doesn't change
    the PA dynamically?
 2) Why do you say mapping on demand is not possible? ARM's
    iommu_dma_prepare_msi() is not called in an atomic context and
    does do the iommu mapping.

What I rather expected was for riscv to have a PA window that is very
big and not just one page, eg adjust iommu_dma_prepare_msi() so you
can pass in the entire PA space that you need for the affinity
changes. Maybe this is a list of phys_addr_t ?

Then keep with the ARM flow where everything happens at the same
times as today. Instead of just mapping one page you map the entire
list.

I'm not keen on this design where things get pre-mapped into the
domain by the iommu driver, the ARM version is much easier to
understand :\

Jason

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

  reply	other threads:[~2026-08-20 22:09 UTC|newest]

Thread overview: 50+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-08-20 21:41 [PATCH v4 00/21] iommu/riscv: Enable MSI remapping, IOMMU_DMA and VFIO Andrew Jones
2026-08-20 21:41 ` Andrew Jones
2026-08-20 21:41 ` [PATCH v4 01/21] iommufd: Convert struct iommufd_sw_msi_maps to a growable bitmap Andrew Jones
2026-08-20 21:41   ` Andrew Jones
2026-08-20 21:41 ` [PATCH v4 02/21] iommufd: Add iommufd_sw_map_msi() Andrew Jones
2026-08-20 21:41   ` Andrew Jones
2026-08-20 22:09   ` Jason Gunthorpe [this message]
2026-08-20 22:09     ` Jason Gunthorpe
2026-08-20 21:41 ` [PATCH v4 03/21] iommu/dma: Add iommu_dma_sw_map_msi() Andrew Jones
2026-08-20 21:41   ` Andrew Jones
2026-08-20 21:41 ` [PATCH v4 04/21] iommu/dma: Add iommu_dma_map_msi() Andrew Jones
2026-08-20 21:41   ` Andrew Jones
2026-08-20 21:41 ` [PATCH v4 05/21] iommu: Document MSI mapping during domain replacement Andrew Jones
2026-08-20 21:41   ` Andrew Jones
2026-08-20 21:41 ` [PATCH v4 06/21] genirq/msi: Provide DOMAIN_BUS_MSI_REMAP Andrew Jones
2026-08-20 21:41   ` Andrew Jones
2026-08-20 21:41 ` [PATCH v4 07/21] irqchip/riscv-imsic: Compose MSI updates through the hierarchy Andrew Jones
2026-08-20 21:41   ` Andrew Jones
2026-08-20 21:41 ` [PATCH v4 08/21] iommu/riscv: Add IRQ domain for interrupt remapping Andrew Jones
2026-08-20 21:41   ` Andrew Jones
2026-08-20 21:41 ` [PATCH v4 09/21] iommu/riscv: Refresh platform MSI domain before IR setup Andrew Jones
2026-08-20 21:41   ` Andrew Jones
2026-08-20 21:41 ` [PATCH v4 10/21] iommu/riscv: Prepare info->domain for concurrent RCU read access Andrew Jones
2026-08-20 21:41   ` Andrew Jones
2026-08-20 21:41 ` [PATCH v4 11/21] iommu/riscv: Reserve an MSI IOVA window for iommufd Andrew Jones
2026-08-20 21:41   ` Andrew Jones
2026-08-20 21:41 ` [PATCH v4 12/21] iommu/riscv: Pre-map IMSIC MSI targets Andrew Jones
2026-08-20 21:41   ` Andrew Jones
2026-08-20 21:41 ` [PATCH v4 13/21] iommu/riscv: Preserve MSI IOVA state across domain replacement Andrew Jones
2026-08-20 21:41   ` Andrew Jones
2026-08-20 21:52   ` Jason Gunthorpe
2026-08-20 21:52     ` Jason Gunthorpe
2026-08-20 21:41 ` [PATCH v4 14/21] iommu/riscv: Gate direct identity boundary switches with live MSIs Andrew Jones
2026-08-20 21:41   ` Andrew Jones
2026-08-20 21:41 ` [PATCH v4 15/21] iommu/riscv: Remap IMSIC targets during MSI composition Andrew Jones
2026-08-20 21:41   ` Andrew Jones
2026-08-20 21:41 ` [PATCH v4 16/21] iommu/dma: Enable IOMMU_DMA for 64-bit RISC-V Andrew Jones
2026-08-20 21:41   ` Andrew Jones
2026-08-20 21:41 ` [PATCH v4 17/21] iommu/riscv: Report cache coherency capability Andrew Jones
2026-08-20 21:41   ` Andrew Jones
2026-08-20 21:47   ` Jason Gunthorpe
2026-08-20 21:47     ` Jason Gunthorpe
2026-08-20 21:41 ` [PATCH v4 18/21] vfio: enable IOMMU_TYPE1 for RISC-V Andrew Jones
2026-08-20 21:41   ` Andrew Jones
2026-08-20 21:41 ` [PATCH v4 19/21] RISC-V: KVM: Enable KVM_VFIO interfaces on RISC-V arch Andrew Jones
2026-08-20 21:41   ` Andrew Jones
2026-08-20 21:41 ` [PATCH v4 20/21] riscv: defconfig: Enable IOMMUFD and VFIO Andrew Jones
2026-08-20 21:41   ` Andrew Jones
2026-08-20 21:41 ` [PATCH v4 21/21] selftests/vfio: Allow building on RISC-V Andrew Jones
2026-08-20 21:41   ` Andrew Jones

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260820220908.GE244917@nvidia.com \
    --to=jgg@nvidia.com \
    --cc=andrew.jones@oss.qualcomm.com \
    --cc=anup@brainfault.org \
    --cc=fangyu.yu@linux.alibaba.com \
    --cc=iommu@lists.linux.dev \
    --cc=joro@8bytes.org \
    --cc=kevin.tian@intel.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-riscv@lists.infradead.org \
    --cc=palmer@dabbelt.com \
    --cc=pjw@kernel.org \
    --cc=robin.murphy@arm.com \
    --cc=tglx@kernel.org \
    --cc=tjeznach@rivosinc.com \
    --cc=tomasz.jeznach@linux.dev \
    --cc=will@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.