public inbox for u-boot@lists.denx.de
 help / color / mirror / Atom feed
From: Qu Wenruo <wqu@suse.com>
To: u-boot@lists.denx.de
Cc: marek.behun@nic.cz, linux-btrfs@vger.kernel.org,
	jnhuang95@gmail.com, linux-erofs@lists.ozlabs.org,
	trini@konsulko.com, joaomarcos.costa@bootlin.com,
	thomas.petazzoni@bootlin.com, miquel.raynal@bootlin.com
Subject: [PATCH 0/8] u-boot: fs: add generic unaligned read handling
Date: Tue, 28 Jun 2022 15:28:00 +0800	[thread overview]
Message-ID: <cover.1656401086.git.wqu@suse.com> (raw)

[BACKGROUND]
Unlike FUSE/Kernel which always pass aligned read range, U-boot fs code
just pass the request range to underlying fses.

Under most case, this works fine, as U-boot only really needs to read
the whole file (aka, 0 for both offset and len, len will be later
determined using file size).

But if some advanced user/script wants to extract kernel/initramfs from
combined image, we may need to do unaligned read in that case.

[ADVANTAGE]
This patchset will handle unaligned read range in _fs_read():

- Get blocksize of the underlying fs

- Read the leading block contianing the unaligned range
  The full block will be stored in a local buffer, then only copy
  the bytes in the unaligned range into the destination buffer.

  If the first block covers the whole range, we just call it aday.

- Read the aligned range if there is any

- Read the tailing block containing the unaligned range
  And copy the covered range into the destination.

[DISADVANTAGE]
There are mainly two problems:

- Extra memory allocation for every _fs_read() call
  For the leading and tailing block.

- Extra path resolving
  All those supported fs will have to do extra path resolving up to 2
  times (one for the leading block, one for the tailing block).
  This may slow down the read.

[SUPPORTED FSES]

- Btrfs (manually tested*)
- Ext4 (manually tested)
- FAT (manually tested)
- Erofs
- sandboxfs
- ubifs

*: Failed to get the test cases run, thus have to go sandbox mode, and
attach an image with target fs, load the target file (with unaligned
range) and compare the result using md5sum.

For EXT4/FAT, they may need extra cleanup, as their existing unaligned
range handling is no longer needed anymore, cleaning them up should free 
more code lines than the added one.

Just not confident enough to modify them all by myself.

[UNSUPPORTED FSES]
- Squashfs
  They don't support non-zero offset, thus it can not handle the tailing
  block reading.
  Need extra help to add block aligned offset support.

- Semihostfs
  It's using hardcoded trap to do system calls, not sure how it would
  work for stat() call.

Extra testing/feedback is always appreciated.


Qu Wenruo (8):
  fs: fat: unexport file_fat_read_at()
  fs: always get the file size in _fs_read()
  fs: btrfs: move the unaligned read code to _fs_read() for btrfs
  fs: ext4: rely on _fs_read() to pass block aligned range into
    ext4fs_read_file()
  fs: fat: rely on higher layer to get block aligned read range
  fs: sandboxfs: add sandbox_fs_get_blocksize()
  fs: ubifs: rely on higher layer to do unaligned read
  fs: erofs: add unaligned read range handling

 arch/sandbox/cpu/os.c  |  11 +++
 fs/btrfs/btrfs.c       |  24 +++---
 fs/btrfs/inode.c       |  84 ++------------------
 fs/erofs/internal.h    |   1 +
 fs/erofs/super.c       |   6 ++
 fs/ext4/ext4fs.c       |  11 +++
 fs/fat/fat.c           |  17 ++++-
 fs/fs.c                | 169 +++++++++++++++++++++++++++++++++++++++--
 fs/sandbox/sandboxfs.c |  14 ++++
 fs/ubifs/ubifs.c       |  13 ++--
 include/btrfs.h        |   1 +
 include/erofs.h        |   1 +
 include/ext4fs.h       |   1 +
 include/fat.h          |   3 +-
 include/os.h           |   8 ++
 include/sandboxfs.h    |   1 +
 include/ubifs_uboot.h  |   1 +
 17 files changed, 258 insertions(+), 108 deletions(-)

-- 
2.36.1


             reply	other threads:[~2022-06-28  7:28 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-06-28  7:28 Qu Wenruo [this message]
2022-06-28  7:28 ` [PATCH RFC 1/8] fs: fat: unexport file_fat_read_at() Qu Wenruo
2022-06-30 10:06   ` Simon Glass
2022-06-28  7:28 ` [PATCH RFC 2/8] fs: always get the file size in _fs_read() Qu Wenruo
2022-06-28 12:36   ` Huang Jianan
2022-06-28 12:43     ` Qu Wenruo
2022-06-28  7:28 ` [PATCH RFC 3/8] fs: btrfs: move the unaligned read code to _fs_read() for btrfs Qu Wenruo
2022-06-28  7:28 ` [PATCH RFC 4/8] fs: ext4: rely on _fs_read() to pass block aligned range into ext4fs_read_file() Qu Wenruo
2022-06-28  7:28 ` [PATCH RFC 5/8] fs: fat: rely on higher layer to get block aligned read range Qu Wenruo
2022-06-28  7:28 ` [PATCH RFC 6/8] fs: sandboxfs: add sandbox_fs_get_blocksize() Qu Wenruo
2022-06-30 10:06   ` Simon Glass
2022-06-30 10:12     ` Qu Wenruo
2022-06-28  7:28 ` [PATCH RFC 7/8] fs: ubifs: rely on higher layer to do unaligned read Qu Wenruo
2022-06-28  7:28 ` [PATCH RFC 8/8] fs: erofs: add unaligned read range handling Qu Wenruo
2022-06-28 13:37 ` [PATCH 0/8] u-boot: fs: add generic unaligned read handling Sean Anderson
2022-06-28 14:17 ` Tom Rini
2022-06-29  1:40   ` Qu Wenruo
2022-06-29 12:53     ` Tom Rini

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cover.1656401086.git.wqu@suse.com \
    --to=wqu@suse.com \
    --cc=jnhuang95@gmail.com \
    --cc=joaomarcos.costa@bootlin.com \
    --cc=linux-btrfs@vger.kernel.org \
    --cc=linux-erofs@lists.ozlabs.org \
    --cc=marek.behun@nic.cz \
    --cc=miquel.raynal@bootlin.com \
    --cc=thomas.petazzoni@bootlin.com \
    --cc=trini@konsulko.com \
    --cc=u-boot@lists.denx.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox