Re: [PATCH 0/8] MUSE: Userspace backed MTD v3

Richard Weinberger
2021-02-01T13:14:58Z

*friendly FUSE maintainer ping* :-)

On Mon, Jan 25, 2021 at 12:24 AM Richard Weinberger <richard@nod.at> wrote:
>
> I'm happy to announce the first non-RFC version of this patch set.
> Over the xmas holidays I found some time to experiment with various userspace
> implementations of MTDs and gave the kernel side more fine-tuning.
>
> Rationale:
> ----------
>
> When working with flash devices a common task is emulating them to run various
> tests or inspect dumps from real hardware. To achieve that we have plenty of
> emulators in the MTD subsystem: mtdram, block2mtd, nandsim.
>
> Each of them implements an ad-hoc MTD and have various drawbacks.
> Over the last years some developers tried to extend them but these attempts
> often got rejected because they added just more adhoc feature instead of
> addressing overall problems.
>
> MUSE is a novel approach to address the need of advanced MTD emulators.
> Advanced means in this context supporting different (vendor specific) image
> formats, different ways for fault injection (fuzzing) and recoding/replaying
> IOs to emulate power cuts.
>
> The core goal of MUSE is having the complexity on the userspace side and
> only a small MTD driver in kernelspace.
> While playing with different approaches I realized that FUSE offers everything
> we need. So MUSE is a little like CUSE except that it does not implement a
> bare character device but an MTD.
>
> Notes:
> ------
>
> - OOB support is currently limited. Currently MUSE has no support for processing
> in- and out-band in the same MTD operation. It is good enough to make JFFS2
> happy. This limitation is because FUSE has no support more than one variable
> length buffer in a FUSE request.
> At least I didn’t find a good way to pass more than one buffer to a request.
> Maybe FUSE folks can correct me. :-)
>
> - Every MTD read/write operation maps 1:1 to a MUSE_READ/WRITE opcode.
> Since FUSE requests are not cheap, the amount of read/written bytes in a MTD
> operation as a huge impact on the performance. Especially when NOR style MTDs
> are implemented in userspace a large writebufsize should be requested to gain
> good write performance.
> On the other hand, MTD operations with lengths larger than writesize are *not*
> split up into multiple MUSE_READ/WRITE requests. This means that userspace
> has to split them manually when doing power-cut emulation.
>
> - MUSE is not super fast. On my i5 workstation nandsim is almost twice as fast
> as a NAND flash in userspace. But MUSE is still magnitudes faster than any
> real world MTD out there. So it is good enough for the use cases I have in
> mind.
>
> Changelog:
> ----------
>
> Changes since v2 (RFC):
> - OOB support
> - MUSE_READ/WRITE opcodes are no longer a min IO MTD unit
> - MTD partitions support via mtdparts string
> - More code cleanup
> - Code rebased to 5.11-rc4
>
> Changes since v1 (RFC):
> - Rewrote IO path, fuse_direct_io() is no longer used.
> Instead of cheating fuse_direct_io() use custom ops to implement
> reading and writing. That way MUSE no longer needs a dummy file object
> nor a fuse file object.
> In MTD all IO is synchronous and operations on kernel buffers, this
> makes IO processing simple for MUSE.
> - Support for bad blocks.
> - No more (ab)use of FUSE ops such as FUSE_FSYNC.
> - Major code cleanup.
>
> This series can also be found at:
> git://git.kernel.org/pub/scm/linux/kernel/git/rw/misc.git muse_v3
>
> Richard Weinberger (8):
> fuse: Export fuse_simple_request
> fuse: Export IO helpers
> fuse: Make cuse_parse_one a common helper
> mtd: Add MTD_MUSE flag
> mtd: Allow passing a custom cmdline to cmdline line parser
> fuse: Add MUSE specific defines FUSE interface
> fuse: Implement MUSE - MTD in userspace
> MAINTAINERS: Add entry for MUSE
>
> Documentation/ABI/testing/sysfs-class-mtd | 8 +
> MAINTAINERS | 7 +
> drivers/mtd/parsers/cmdlinepart.c | 73 +-
> fs/fuse/Kconfig | 15 +
> fs/fuse/Makefile | 2 +
> fs/fuse/cuse.c | 58 +-
> fs/fuse/dev.c | 1 +
> fs/fuse/file.c | 16 +-
> fs/fuse/fuse_i.h | 18 +
> fs/fuse/helper.c | 70 ++
> fs/fuse/muse.c | 1086 +++++++++++++++++++++
> include/linux/mtd/partitions.h | 2 +
> include/uapi/linux/fuse.h | 76 ++
> include/uapi/mtd/mtd-abi.h | 1 +
> 14 files changed, 1346 insertions(+), 87 deletions(-)
> create mode 100644 fs/fuse/helper.c
> create mode 100644 fs/fuse/muse.c
>
> --
> 2.26.2
>

--
Thanks,
//richard