dri-devel Archive on lore.kernel.org
 help / color / mirror / Atom feed
From: Steven Price <steven.price@arm.com>
To: Lorenzo Pieralisi <lpieralisi@kernel.org>,
	"Aneesh Kumar K.V" <aneesh.kumar@kernel.org>
Cc: "Marc Zyngier" <maz@kernel.org>,
	linux-coco@lists.linux.dev,
	"Andrew Morton" <akpm@linux-foundation.org>,
	"Baoquan He" <baoquan.he@linux.dev>,
	"Mike Rapoport" <rppt@kernel.org>,
	"Pasha Tatashin" <pasha.tatashin@soleen.com>,
	"Pratyush Yadav" <pratyush@kernel.org>,
	"Catalin Marinas" <catalin.marinas@arm.com>,
	"Christian König" <christian.koenig@amd.com>,
	"Jason Gunthorpe" <jgg@ziepe.ca>,
	"Joerg Roedel" <joro@8bytes.org>,
	"Marek Szyprowski" <m.szyprowski@samsung.com>,
	"Robin Murphy" <robin.murphy@arm.com>,
	"Sumit Semwal" <sumit.semwal@linaro.org>,
	"Suzuki K Poulose" <suzuki.poulose@arm.com>,
	"Thomas Gleixner" <tglx@kernel.org>,
	"Will Deacon" <will@kernel.org>,
	"Russell King" <linux@armlinux.org.uk>,
	"Benjamin Gaignard" <benjamin.gaignard@collabora.com>,
	"Brian Starkey" <Brian.Starkey@arm.com>,
	"John Stultz" <jstultz@google.com>,
	"Mark Rutland" <mark.rutland@arm.com>,
	"Radu Rendec" <radu@rendec.net>,
	"T.J. Mercier" <tjmercier@google.com>,
	"Madhavan Srinivasan" <maddy@linux.ibm.com>,
	"Michael Ellerman" <mpe@ellerman.id.au>,
	"Nicholas Piggin" <npiggin@gmail.com>,
	"Christophe Leroy" <chleroy@kernel.org>,
	"Ritesh Harjani" <ritesh.list@gmail.com>,
	"Shrikanth Hegde" <sshegde@linux.ibm.com>,
	"Alexander Gordeev" <agordeev@linux.ibm.com>,
	"Gerald Schaefer" <gerald.schaefer@linux.ibm.com>,
	"Heiko Carstens" <hca@linux.ibm.com>,
	"Vasily Gorbik" <gor@linux.ibm.com>,
	"Christian Borntraeger" <borntraeger@linux.ibm.com>,
	"Sven Schnelle" <svens@linux.ibm.com>,
	"Ingo Molnar" <mingo@redhat.com>,
	"Borislav Petkov" <bp@alien8.de>,
	"Dave Hansen" <dave.hansen@linux.intel.com>,
	x86@kernel.org, "H . Peter Anvin" <hpa@zytor.com>,
	"Kiryl Shutsemau" <kas@kernel.org>,
	"Rick Edgecombe" <rick.p.edgecombe@intel.com>,
	"K . Y . Srinivasan" <kys@microsoft.com>,
	"Haiyang Zhang" <haiyangz@microsoft.com>,
	"Wei Liu" <wei.liu@kernel.org>,
	"Dexuan Cui" <decui@microsoft.com>,
	"Long Li" <longli@microsoft.com>,
	"Paolo Bonzini" <pbonzini@redhat.com>,
	"Vitaly Kuznetsov" <vkuznets@redhat.com>,
	"Andy Lutomirski" <luto@kernel.org>,
	"Peter Zijlstra" <peterz@infradead.org>,
	dri-devel@lists.freedesktop.org, iommu@lists.linux.dev,
	linaro-mm-sig@lists.linaro.org,
	linux-arm-kernel@lists.infradead.org,
	linux-media@vger.kernel.org, linux-mm@kvack.org,
	kvmarm@lists.linux.dev, patches@lists.linux.dev
Subject: Re: [PATCH v9 14/14] irqchip/gic-v3-its: Preallocate VPE L1 tables
Date: Mon, 16 Nov 2026 15:32:25 +0000	[thread overview]
Message-ID: <f2e47b68-1aee-4d17-8b63-e67075e7a42e@arm.com> (raw)
In-Reply-To: <avrgNBXDFMGL6dqO@red-moon>

On 16/11/2026 10:51, Lorenzo Pieralisi wrote:
> On Mon, Nov 16, 2026 at 11:51:04AM +0530, Aneesh Kumar K.V wrote:
>> Marc Zyngier <maz@kernel.org> writes:
>>
>>> On Fri, 06 Nov 2026 15:45:29 +0000,
>>> Aneesh Kumar K.V <aneesh.kumar@kernel.org> wrote:
>>>>
>>>> Marc Zyngier <maz@kernel.org> writes:
>>>>
>>>>> On Wed, 04 Nov 2026 16:45:55 +0000,
>>>>> "Aneesh Kumar K.V (Arm)" <aneesh.kumar@kernel.org> wrote:
>>>>>>
>>>>>> VPE L1 tables are allocated from the CPU-starting callback, where sleeping is
>>>>>> not allowed.  Remove the page allocation from this atomic path.
>>>>>>
>>>>>> Use a blocking allocation for the boot CPU and record the resulting table
>>>>>> order.  Allocate a page of that order from the sleepable CPU hotplug prepare
>>>>>> stage for each secondary CPU.  Consume the preallocated page only when the CPU
>>>>>> cannot inherit an existing redistributor or ITS table.
>>>>>>
>>>>>> Reclaim unused preallocations after the CPU reaches the online state, and from
>>>>>> the prepare-state teardown when CPU bring-up aborts.
>>>>>
>>>>> No. The way to do it is to move the ITS probing out of the
>>>>> non-sleepable CPU hotplug path.
>>>>>
>>>>> I proposed a potential hack at [1] (which I find far more palatable
>>>>> than this proposal), but the real fix is to probe the ITSs as
>>>>> devices.
>>>>>
>>>>
>>>> Could you share a bit more detail on what you have in mind?
>>>>
>>>> Are you suggesting that we create a platform device for each ITS and
>>>> initialize it through the driver's probe callback, while the rest of the
>>>> GIC initialization stays in the existing early boot path?
>>>>
>>>> Would that mean the main GIC is operational during early boot, while
>>>> ITS-dependent features, such as MSI delivery and direct injection, become
>>>> available after the ITS devices probe?
>>>>
>>>> Is that the split you have in mind?
>>>
>>> Yes. Then once this is done, all the tables become normal DMA
>>> allocations, for which there should already be adequate support.
>>>
>>
>> Thanks for confirming. Can we handle the ITS platform-device conversion
>> and the switch to DMA allocations in a separate series?
>>
>> The existing VPE L1 allocation path calls set_memory_decrypted() from
>> the non-sleepable CPU-starting callback, even though
>> set_memory_decrypted() can sleep. This issue predates the shared-granule
>> allocator changes.
> 
> If I may ask, why does the existing code ends up calling set_memory_decrypt(),
> or reworded, why does the VPE L1 allocation path goes through the ITS specific
> allocation function rather than using alloc_pages() (as it was before
> b08e2f42e86b5) ?

Because it needs to be accessible to the GIC "hardware". In this case
the GIC is emulated by the hypervisor and the hypervisor for a realm can
only see the unprotected memory.

Now I agree this is a little academic in practise because Linux (as a
hypervisor) will not expose RVPEID - so allocate_vpe_l1_table() will
early out. IIUC this would only actually be relevant in the case of
nested virtualisation within a realm guest - which is a long way off if
it ever happens.

> Why did b08e2f42e86b5 patched allocate_vpe_l1_table() allocation and moved
> it to using its_alloc_pages() ?

In that commit I simply changed all the allocation paths which were
allocating memory to be visible to the GIC to use the new allocator. As
we "know" that the GIC can only access "decrypted" memory. Admittedly
that could easily change in the future - but my expectation is that it
will only change with the switch to GICv5.

Of course what I missed was that this specific allocation uses
GFP_ATOMIC and that set_memory_decrypted() can sleep. This wouldn't have
shown up on any testing because of the early exit:

        if (!gic_rdists->has_rvpeid)
                return 0;

Thanks,
Steve

> Thanks,
> Lorenzo
> 
>>
>> Since fixing this properly needs a broader ITS rework, can we handle
>> that separately and drop the preallocation patch from this series?
>>
>> -aneesh


  reply	other threads:[~2026-11-16 15:32 UTC|newest]

Thread overview: 33+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-04 16:45 [PATCH v9 00/14] coco: guest: Add a shared-granule allocator for host-shared memory Aneesh Kumar K.V (Arm)
2026-11-04 16:45 ` [PATCH v9 01/14] mm: Add an allocator for CoCo shared memory Aneesh Kumar K.V (Arm)
2026-11-05 16:52   ` sashiko-bot
2026-11-12 16:49     ` Aneesh Kumar K.V
2026-11-06 14:41   ` Suzuki K Poulose
2026-11-06 15:30     ` Aneesh Kumar K.V
2026-11-06 15:34       ` Suzuki K Poulose
2026-11-04 16:45 ` [PATCH v9 02/14] mm: Zero memory during shared memory transitions Aneesh Kumar K.V (Arm)
2026-11-05 16:52   ` sashiko-bot
2026-11-12 17:29     ` Aneesh Kumar K.V
2026-11-04 16:45 ` [PATCH v9 03/14] irqchip/gic-v3-its: Resolve the default NUMA node explicitly Aneesh Kumar K.V (Arm)
2026-11-04 16:45 ` [PATCH v9 04/14] irqchip/gic-v3-its: Allocate shared tables using CoCo shared memory allocator Aneesh Kumar K.V (Arm)
2026-11-04 16:45 ` [PATCH v9 05/14] dma-contiguous: Derive shared alignment from DMA attributes Aneesh Kumar K.V (Arm)
2026-11-04 16:45 ` [PATCH v9 06/14] dma-pool: Allocate CoCo atomic pools using CoCo shared memory allocator Aneesh Kumar K.V (Arm)
2026-11-05 16:52   ` sashiko-bot
2026-11-13  3:12     ` Aneesh Kumar K.V
2026-11-04 16:45 ` [PATCH v9 07/14] dma-direct: Align CoCo shared DMA allocations to the shared granule size Aneesh Kumar K.V (Arm)
2026-11-04 16:45 ` [PATCH v9 08/14] swiotlb: Align shared IO TLB pools " Aneesh Kumar K.V (Arm)
2026-11-05 16:52   ` sashiko-bot
2026-11-04 16:45 ` [PATCH v9 09/14] swiotlb: Reject misaligned restricted DMA pools for CoCo guests Aneesh Kumar K.V (Arm)
2026-11-04 16:45 ` [PATCH v9 10/14] dma-buf: system_heap: Limit scatterlist entries to the buffer size Aneesh Kumar K.V (Arm)
2026-11-04 16:45 ` [PATCH v9 11/14] dma-buf: system_heap: Allocate shared buffers using CoCo shared memory allocator Aneesh Kumar K.V (Arm)
2026-11-04 16:45 ` [PATCH v9 12/14] swiotlb: Make rounded shared pool capacity allocatable Aneesh Kumar K.V (Arm)
2026-11-04 16:45 ` [PATCH v9 13/14] mm: Assert CoCo shared allocations may sleep Aneesh Kumar K.V (Arm)
2026-11-04 16:45 ` [PATCH v9 14/14] irqchip/gic-v3-its: Preallocate VPE L1 tables Aneesh Kumar K.V (Arm)
2026-11-05 16:20   ` Marc Zyngier
2026-11-06 15:45     ` Aneesh Kumar K.V
2026-11-12 22:22       ` Marc Zyngier
2026-11-16  6:21         ` Aneesh Kumar K.V
2026-11-16 10:51           ` Lorenzo Pieralisi
2026-11-16 15:32             ` Steven Price [this message]
2026-11-16 16:38               ` Lorenzo Pieralisi
2026-11-05 16:52   ` sashiko-bot

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=f2e47b68-1aee-4d17-8b63-e67075e7a42e@arm.com \
    --to=steven.price@arm.com \
    --cc=Brian.Starkey@arm.com \
    --cc=agordeev@linux.ibm.com \
    --cc=akpm@linux-foundation.org \
    --cc=aneesh.kumar@kernel.org \
    --cc=baoquan.he@linux.dev \
    --cc=benjamin.gaignard@collabora.com \
    --cc=borntraeger@linux.ibm.com \
    --cc=bp@alien8.de \
    --cc=catalin.marinas@arm.com \
    --cc=chleroy@kernel.org \
    --cc=christian.koenig@amd.com \
    --cc=dave.hansen@linux.intel.com \
    --cc=decui@microsoft.com \
    --cc=dri-devel@lists.freedesktop.org \
    --cc=gerald.schaefer@linux.ibm.com \
    --cc=gor@linux.ibm.com \
    --cc=haiyangz@microsoft.com \
    --cc=hca@linux.ibm.com \
    --cc=hpa@zytor.com \
    --cc=iommu@lists.linux.dev \
    --cc=jgg@ziepe.ca \
    --cc=joro@8bytes.org \
    --cc=jstultz@google.com \
    --cc=kas@kernel.org \
    --cc=kvmarm@lists.linux.dev \
    --cc=kys@microsoft.com \
    --cc=linaro-mm-sig@lists.linaro.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-coco@lists.linux.dev \
    --cc=linux-media@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=linux@armlinux.org.uk \
    --cc=longli@microsoft.com \
    --cc=lpieralisi@kernel.org \
    --cc=luto@kernel.org \
    --cc=m.szyprowski@samsung.com \
    --cc=maddy@linux.ibm.com \
    --cc=mark.rutland@arm.com \
    --cc=maz@kernel.org \
    --cc=mingo@redhat.com \
    --cc=mpe@ellerman.id.au \
    --cc=npiggin@gmail.com \
    --cc=pasha.tatashin@soleen.com \
    --cc=patches@lists.linux.dev \
    --cc=pbonzini@redhat.com \
    --cc=peterz@infradead.org \
    --cc=pratyush@kernel.org \
    --cc=radu@rendec.net \
    --cc=rick.p.edgecombe@intel.com \
    --cc=ritesh.list@gmail.com \
    --cc=robin.murphy@arm.com \
    --cc=rppt@kernel.org \
    --cc=sshegde@linux.ibm.com \
    --cc=sumit.semwal@linaro.org \
    --cc=suzuki.poulose@arm.com \
    --cc=svens@linux.ibm.com \
    --cc=tglx@kernel.org \
    --cc=tjmercier@google.com \
    --cc=vkuznets@redhat.com \
    --cc=wei.liu@kernel.org \
    --cc=will@kernel.org \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox