From: Drew Fustini <fustini@kernel.org>
To: Troy Mitchell <troy.mitchell@linux.dev>
Cc: Paul Walmsley <pjw@kernel.org>,
Palmer Dabbelt <palmer@dabbelt.com>,
Albert Ou <aou@eecs.berkeley.edu>,
Alexandre Ghiti <alex@ghiti.fr>,
linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org,
spacemit@lists.linux.dev,
Anirudh Srinivasan <asrinivasan@oss.tenstorrent.com>
Subject: Re: [PATCH v3] riscv: mm: fix SWIOTLB initialization for systems with DRAM above 4GB
Date: Wed, 29 Jul 2026 10:20:27 -0700 [thread overview]
Message-ID: <amo2W_XZDiJ_2sLg@thelio> (raw)
In-Reply-To: <20260727-fix-riscv-swiotlb-v3-1-59479b23736c@linux.dev>
On Mon, Jul 27, 2026 at 01:08:44AM -0700, Troy Mitchell wrote:
> On RISC-V platforms where the entire physical memory (DRAM) resides
> above the 32-bit address space (i.e., above dma32_phys_limit), the
> current SWIOTLB initialization logic fails.
>
> This patch addresses two interconnected issues on such platforms:
>
> 1. Incorrect 32-bit DMA bounce assumption:
> The existing condition `max_pfn > PFN_DOWN(dma32_phys_limit)` assumes
> that a 32-bit DMA bounce buffer is required simply because the maximum
> PFN exceeds the 32-bit limit. However, if all DRAM starts above 4GB,
> no memory exists below the limit to satisfy this allocation. Fix
> this by adding a check to ensure `memblock_start_of_DRAM()` is actually
> below the 32-bit limit before enforcing 32-bit SWIOTLB.
>
> 2. kmalloc() bounce buffer allocation failure on non-coherent systems:
> For non-coherent DMA, kmalloc() buffers whose sizes are not
> cache-line-aligned still require bouncing, even if 32-bit DMA bouncing
> is skipped. Without the `SWIOTLB_ANY` flag, swiotlb_init() defaults to
> allocating from low memory, which fails completely when DRAM only exists
> in high memory. By appending `SWIOTLB_ANY` to swiotlb_flags, the allocator
> is permitted to allocate this bounce buffer from high memory.
>
> With this patch, systems with non-coherent DMA and DRAM entirely above
> 4GB can successfully map the software IO TLB in high memory and boot
> normally.
>
> Tested-by: Anirudh Srinivasan <asrinivasan@oss.tenstorrent.com>
> Signed-off-by: Troy Mitchell <troy.mitchell@linux.dev>
> ---
> Changes in v3:
> - clarify when kmalloc() buffers require bouncing for non-coherent DMA
> - Link to v2: https://patch.msgid.link/20260429-fix-riscv-swiotlb-v2-1-fa99dfdfc94d@linux.dev
>
> Changes in v2:
> - add Anirudh's TB tag
> - Link to v1: https://lore.kernel.org/r/20260331-fix-riscv-swiotlb-v1-1-74dd5e6be0f1@linux.dev
Maybe fixes tag should be added like this?
Fixes: dcb2743d1e70 ("riscv: mm: still create swiotlb buffer for kmalloc() bouncing if required")
Aside from that, LGTM and resolves the issue for Linux running on the
X280 clusters in the Tenstorrent Blackhole.
Reviewed-by: Drew Fustini <fustini@kernel.org>
Thanks,
Drew
WARNING: multiple messages have this Message-ID (diff)
From: Drew Fustini <fustini@kernel.org>
To: Troy Mitchell <troy.mitchell@linux.dev>
Cc: Paul Walmsley <pjw@kernel.org>,
Palmer Dabbelt <palmer@dabbelt.com>,
Albert Ou <aou@eecs.berkeley.edu>,
Alexandre Ghiti <alex@ghiti.fr>,
linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org,
spacemit@lists.linux.dev,
Anirudh Srinivasan <asrinivasan@oss.tenstorrent.com>
Subject: Re: [PATCH v3] riscv: mm: fix SWIOTLB initialization for systems with DRAM above 4GB
Date: Wed, 29 Jul 2026 10:20:27 -0700 [thread overview]
Message-ID: <amo2W_XZDiJ_2sLg@thelio> (raw)
In-Reply-To: <20260727-fix-riscv-swiotlb-v3-1-59479b23736c@linux.dev>
On Mon, Jul 27, 2026 at 01:08:44AM -0700, Troy Mitchell wrote:
> On RISC-V platforms where the entire physical memory (DRAM) resides
> above the 32-bit address space (i.e., above dma32_phys_limit), the
> current SWIOTLB initialization logic fails.
>
> This patch addresses two interconnected issues on such platforms:
>
> 1. Incorrect 32-bit DMA bounce assumption:
> The existing condition `max_pfn > PFN_DOWN(dma32_phys_limit)` assumes
> that a 32-bit DMA bounce buffer is required simply because the maximum
> PFN exceeds the 32-bit limit. However, if all DRAM starts above 4GB,
> no memory exists below the limit to satisfy this allocation. Fix
> this by adding a check to ensure `memblock_start_of_DRAM()` is actually
> below the 32-bit limit before enforcing 32-bit SWIOTLB.
>
> 2. kmalloc() bounce buffer allocation failure on non-coherent systems:
> For non-coherent DMA, kmalloc() buffers whose sizes are not
> cache-line-aligned still require bouncing, even if 32-bit DMA bouncing
> is skipped. Without the `SWIOTLB_ANY` flag, swiotlb_init() defaults to
> allocating from low memory, which fails completely when DRAM only exists
> in high memory. By appending `SWIOTLB_ANY` to swiotlb_flags, the allocator
> is permitted to allocate this bounce buffer from high memory.
>
> With this patch, systems with non-coherent DMA and DRAM entirely above
> 4GB can successfully map the software IO TLB in high memory and boot
> normally.
>
> Tested-by: Anirudh Srinivasan <asrinivasan@oss.tenstorrent.com>
> Signed-off-by: Troy Mitchell <troy.mitchell@linux.dev>
> ---
> Changes in v3:
> - clarify when kmalloc() buffers require bouncing for non-coherent DMA
> - Link to v2: https://patch.msgid.link/20260429-fix-riscv-swiotlb-v2-1-fa99dfdfc94d@linux.dev
>
> Changes in v2:
> - add Anirudh's TB tag
> - Link to v1: https://lore.kernel.org/r/20260331-fix-riscv-swiotlb-v1-1-74dd5e6be0f1@linux.dev
Maybe fixes tag should be added like this?
Fixes: dcb2743d1e70 ("riscv: mm: still create swiotlb buffer for kmalloc() bouncing if required")
Aside from that, LGTM and resolves the issue for Linux running on the
X280 clusters in the Tenstorrent Blackhole.
Reviewed-by: Drew Fustini <fustini@kernel.org>
Thanks,
Drew
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
next prev parent reply other threads:[~2026-07-29 17:20 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-07-27 8:08 [PATCH v3] riscv: mm: fix SWIOTLB initialization for systems with DRAM above 4GB Troy Mitchell
2026-07-27 8:08 ` Troy Mitchell
2026-07-29 17:20 ` Drew Fustini [this message]
2026-07-29 17:20 ` Drew Fustini
2026-07-29 17:46 ` Paul Walmsley
2026-07-29 17:46 ` Paul Walmsley
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=amo2W_XZDiJ_2sLg@thelio \
--to=fustini@kernel.org \
--cc=alex@ghiti.fr \
--cc=aou@eecs.berkeley.edu \
--cc=asrinivasan@oss.tenstorrent.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-riscv@lists.infradead.org \
--cc=palmer@dabbelt.com \
--cc=pjw@kernel.org \
--cc=spacemit@lists.linux.dev \
--cc=troy.mitchell@linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.