* [RFC PATCH 0/1] mtdchar: kvmalloc() for large read/write buffers?
@ 2026-11-19 9:00 ping.li
2026-11-19 9:08 ` [RFC PATCH 1/1] mtd: mtdchar: use kvmalloc() for large read/write buffers Ping Li
2026-11-19 9:50 ` [RFC PATCH 0/1] mtdchar: kvmalloc() for large read/write buffers? Miquel Raynal
0 siblings, 2 replies; 3+ messages in thread
From: ping.li @ 2026-11-19 9:00 UTC (permalink / raw)
To: miquel.raynal, richard, vigneshr; +Cc: linux-mtd, linux-kernel, Ping Li
From: Ping Li <leeonion.muyu@gmail.com>
While investigating a throughput regression on one of our platforms
(long uptime, fragmented memory, large read()/write() on /dev/mtdX)
I noticed an inconsistency inside mtdchar:
- the ioctl paths (MEMREAD/MEMWRITE) allocate their bounce
buffers with kvmalloc() (83208e106a8e8),
- but the read()/write() paths still use mtd_kmalloc_up_to(),
which halves the buffer size until kmalloc() succeeds.
The halving behavior is not a correctness problem: the loop in
mtdchar_read()/mtdchar_write() makes progress with whatever chunk
size it gets. But under fragmentation a multi-megabyte transfer
silently degrades into many small mtd_read()/mtd_write() calls,
which is visible when the low-level driver cost scales with the
number of transactions (e.g. drivers that forward each call to a
remote controller over a bus).
This RFC proposes to align the read()/write() paths with the ioctl
paths by using kvmalloc(), keeping the KMALLOC_MAX_SIZE cap.
The open question I would like to sort out before submitting a
proper patch:
How concerned should we be about low-level drivers receiving a
vmalloc'd buffer?
Looking at the tree, most drivers only memcpy() the buffer, and
the controllers that DMA directly into it (brcmnand, gpmi,
cadence, atmel, ...) already check is_vmalloc_addr()/
virt_addr_valid() and fall back to a bounce path. So vmalloc'd
buffers are supported everywhere today -- just slower on those
controllers. With mtd_kmalloc_up_to() the same controllers get a
kmalloc'd (smaller) buffer instead, so either way there is a
slowdown; it is mostly a question of which one userspace prefers:
few large transactions through a bounce path, or many small
transactions without one.
If the direction is agreed, I will follow up with a proper patch
and numbers from our platform.
Ping Li (1):
mtd: mtdchar: use kvmalloc() for large read/write buffers
drivers/mtd/mtdchar.c | 28 ++++++++++++++++++++--------
1 file changed, 20 insertions(+), 8 deletions(-)
base-commit: aebb7274476403040d582c61478df8f4a7bf9cd3
--
2.34.1
^ permalink raw reply [flat|nested] 3+ messages in thread
* [RFC PATCH 1/1] mtd: mtdchar: use kvmalloc() for large read/write buffers
2026-11-19 9:00 [RFC PATCH 0/1] mtdchar: kvmalloc() for large read/write buffers? ping.li
@ 2026-11-19 9:08 ` Ping Li
2026-11-19 9:50 ` [RFC PATCH 0/1] mtdchar: kvmalloc() for large read/write buffers? Miquel Raynal
1 sibling, 0 replies; 3+ messages in thread
From: Ping Li @ 2026-11-19 9:08 UTC (permalink / raw)
To: miquel.raynal, richard, vigneshr; +Cc: linux-mtd, linux-kernel, Ping Li
mtdchar_read() and mtdchar_write() use mtd_kmalloc_up_to() for their
bounce buffer. Under memory fragmentation this helper halves the
requested size until kmalloc() succeeds, so a userspace read()/write()
of a few megabytes may silently degrade into many small mtd_read()/
mtd_write() calls: more per-chunk overhead, and device drivers whose
backend cost scales with the number of transactions pay a visible
throughput penalty.
The ioctl paths (MEMREAD/MEMWRITE) already use kvmalloc() for the same
purpose (83208e106a8e8 "mtdchar: use kvmalloc() for potentially large
allocations"). Use kvmalloc() in the read/write paths too: it tries
kmalloc() first and falls back to vmalloc() when the buddy allocator
cannot serve the request contiguously, so the buffer keeps its original
size instead of shrinking.
The size is still capped at KMALLOC_MAX_SIZE, as mtd_kmalloc_up_to()
did.
Most low-level MTD drivers only memcpy() the buffer, so vmalloc'd
buffers are fine there; controllers that DMA directly into the buffer
(brcmnand, gpmi, cadence, atmel, ...) check is_vmalloc_addr()/
virt_addr_valid() and fall back to a bounce path already.
Signed-off-by: Ping Li <leeonion.muyu@gmail.com>
---
drivers/mtd/mtdchar.c | 28 ++++++++++++++++++++--------
1 file changed, 20 insertions(+), 8 deletions(-)
diff --git a/drivers/mtd/mtdchar.c b/drivers/mtd/mtdchar.c
index bf01e6ac7293..442fac076940 100644
--- a/drivers/mtd/mtdchar.c
+++ b/drivers/mtd/mtdchar.c
@@ -148,7 +148,13 @@ static ssize_t mtdchar_read(struct file *file, char __user *buf, size_t count,
if (!count)
return 0;
- kbuf = mtd_kmalloc_up_to(mtd, &size);
+ /*
+ * Cap the buffer like mtd_kmalloc_up_to() used to; kvmalloc()
+ * then falls back to vmalloc() when the fragmented buddy
+ * allocator cannot serve the request contiguously.
+ */
+ size = min_t(size_t, count, KMALLOC_MAX_SIZE);
+ kbuf = kvmalloc(size, GFP_KERNEL);
if (!kbuf)
return -ENOMEM;
@@ -192,7 +198,7 @@ static ssize_t mtdchar_read(struct file *file, char __user *buf, size_t count,
if (!ret || mtd_is_bitflip_or_eccerr(ret)) {
*ppos += retlen;
if (copy_to_user(buf, kbuf, retlen)) {
- kfree(kbuf);
+ kvfree(kbuf);
return -EFAULT;
}
else
@@ -204,13 +210,13 @@ static ssize_t mtdchar_read(struct file *file, char __user *buf, size_t count,
count = 0;
}
else {
- kfree(kbuf);
+ kvfree(kbuf);
return ret;
}
}
- kfree(kbuf);
+ kvfree(kbuf);
return total_retlen;
} /* mtdchar_read */
@@ -237,7 +243,13 @@ static ssize_t mtdchar_write(struct file *file, const char __user *buf, size_t c
if (!count)
return 0;
- kbuf = mtd_kmalloc_up_to(mtd, &size);
+ /*
+ * Cap the buffer like mtd_kmalloc_up_to() used to; kvmalloc()
+ * then falls back to vmalloc() when the fragmented buddy
+ * allocator cannot serve the request contiguously.
+ */
+ size = min_t(size_t, count, KMALLOC_MAX_SIZE);
+ kbuf = kvmalloc(size, GFP_KERNEL);
if (!kbuf)
return -ENOMEM;
@@ -245,7 +257,7 @@ static ssize_t mtdchar_write(struct file *file, const char __user *buf, size_t c
len = min_t(size_t, count, size);
if (copy_from_user(kbuf, buf, len)) {
- kfree(kbuf);
+ kvfree(kbuf);
return -EFAULT;
}
@@ -292,12 +304,12 @@ static ssize_t mtdchar_write(struct file *file, const char __user *buf, size_t c
buf += retlen;
}
else {
- kfree(kbuf);
+ kvfree(kbuf);
return ret;
}
}
- kfree(kbuf);
+ kvfree(kbuf);
return total_retlen;
} /* mtdchar_write */
--
2.34.1
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [RFC PATCH 0/1] mtdchar: kvmalloc() for large read/write buffers?
2026-11-19 9:00 [RFC PATCH 0/1] mtdchar: kvmalloc() for large read/write buffers? ping.li
2026-11-19 9:08 ` [RFC PATCH 1/1] mtd: mtdchar: use kvmalloc() for large read/write buffers Ping Li
@ 2026-11-19 9:50 ` Miquel Raynal
1 sibling, 0 replies; 3+ messages in thread
From: Miquel Raynal @ 2026-11-19 9:50 UTC (permalink / raw)
To: ping.li; +Cc: richard, vigneshr, linux-mtd, linux-kernel
Hi Ping Li,
On 19/11/2026 at 17:00:54 +08, "ping.li" <leeonion.muyu@gmail.com> wrote:
> From: Ping Li <leeonion.muyu@gmail.com>
>
> While investigating a throughput regression on one of our platforms
> (long uptime, fragmented memory, large read()/write() on /dev/mtdX)
> I noticed an inconsistency inside mtdchar:
>
> - the ioctl paths (MEMREAD/MEMWRITE) allocate their bounce
> buffers with kvmalloc() (83208e106a8e8),
> - but the read()/write() paths still use mtd_kmalloc_up_to(),
> which halves the buffer size until kmalloc() succeeds.
>
> The halving behavior is not a correctness problem: the loop in
> mtdchar_read()/mtdchar_write() makes progress with whatever chunk
> size it gets. But under fragmentation a multi-megabyte transfer
> silently degrades into many small mtd_read()/mtd_write() calls,
> which is visible when the low-level driver cost scales with the
> number of transactions (e.g. drivers that forward each call to a
> remote controller over a bus).
>
> This RFC proposes to align the read()/write() paths with the ioctl
> paths by using kvmalloc(), keeping the KMALLOC_MAX_SIZE cap.
>
> The open question I would like to sort out before submitting a
> proper patch:
>
> How concerned should we be about low-level drivers receiving a
> vmalloc'd buffer?
>
> Looking at the tree, most drivers only memcpy() the buffer, and
> the controllers that DMA directly into it (brcmnand, gpmi,
> cadence, atmel, ...) already check is_vmalloc_addr()/
> virt_addr_valid() and fall back to a bounce path. So vmalloc'd
> buffers are supported everywhere today -- just slower on those
> controllers. With mtd_kmalloc_up_to() the same controllers get a
> kmalloc'd (smaller) buffer instead, so either way there is a
> slowdown; it is mostly a question of which one userspace prefers:
> few large transactions through a bounce path, or many small
> transactions without one.
>
> If the direction is agreed, I will follow up with a proper patch
> and numbers from our platform.
I would be curious to know how this gets faster and why? Because the
preferred path is obviously via DMA to avoid keeping the CPU busy for
such transfers, in which case you have no other choice than kmalloc'ing
your buffer for the low-level driver.
I am not generally opposed to the idea. I actually think it's best to
have a unified allocation method if that is possible (without strong
preference myself, but perhaps Richard will share a more educated
opinion?). The main problem is how to handle the legacy cases. Typically
Sashiko raised that some devices use __pa() over the buffer virtual
address, which is absolutely horrible in 2027. Those will have to be
fixed in your serie (as prerequisites). So be aware that the series is
gonna be larger than what you expect.
Ideally, I believe the low-level drivers should tell the MTD layer
whether they expect continuous buffers > PAGE_SIZE or not. We could
therefore have our own devm_mtd_kvmalloc(mtd, ...) which attaches the
buffer to the MTD device, tries kzalloc() if requested by the mtd device
(some kind of mtd->expects_dma_bufs boolean set during probe, typically
based on the availability of DMA callbacks), fallbacks to vmalloc()
otherwise.
Thanks,
Miquèl
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2026-11-19 9:50 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-19 9:00 [RFC PATCH 0/1] mtdchar: kvmalloc() for large read/write buffers? ping.li
2026-11-19 9:08 ` [RFC PATCH 1/1] mtd: mtdchar: use kvmalloc() for large read/write buffers Ping Li
2026-11-19 9:50 ` [RFC PATCH 0/1] mtdchar: kvmalloc() for large read/write buffers? Miquel Raynal
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox