From: Damien Le Moal <damien.lemoal@opensource.wdc.com>
To: linux-fsdevel@vger.kernel.org
Cc: Johannes Thumshirn <johannes.thumshirn@wdc.com>,
Jorgen Hansen <Jorgen.Hansen@wdc.com>
Subject: [PATCH 0/7] Reduce zonefs memory usage
Date: Tue, 10 Jan 2023 22:08:23 +0900 [thread overview]
Message-ID: <20230110130830.246019-1-damien.lemoal@opensource.wdc.com> (raw)
This series improves memory usage by switching to using dynamically
allocated inodes and dentries, similarly to regular file systems. This
drastically reduces the memory consumption of zonefs when the file
system is mounted. E.g., for a 26 TB SMR HDD with over 95000 zones,
memory usage is decreased from about 130 MB down to a little over 5 MB.
Since zonefs does not have persistent metadata of its own and relies
completely on the device zone state management, information such as the
zone write pointer position (used to determine an inode size) is kept in
memory at all time, even when an inode is evicted from the inode cache.
Dentries can be trivially searched for and created dynamically as needed
due to the static file tree structure of a zonefs volume (fully
determined by a zoned device zone configuration).
The patch series is organized as follows. Patch 1 fixes a problem with
synchronous write error detection and error recovery. Patch 2
reorganizes the code (splitting out file operation code into a new file)
and patch 3 simplifies the IO error recovery code. Patch 4 and 5 cleanup
inode zone information and split that information out of the zonefs
inode structure. Finally, Patch 6 implements dynamic inode and dentry
allocation and maanagement operations. Ptch 7 introduces a small
optimization to reduce file open latency.
Damien Le Moal (7):
zonefs: Detect append writes at invalid locations
zonefs: Reorganize code
zonefs: Simplify IO error handling
zonefs: Reduce struct zonefs_inode_info size
zonefs: Separate zone information from inode information
zonefs: Dynamically create file inodes when needed
zonefs: Cache zone group directory inodes
fs/zonefs/Makefile | 2 +-
fs/zonefs/file.c | 878 ++++++++++++++++++++
fs/zonefs/super.c | 1923 ++++++++++++++++----------------------------
fs/zonefs/trace.h | 20 +-
fs/zonefs/zonefs.h | 110 ++-
5 files changed, 1684 insertions(+), 1249 deletions(-)
create mode 100644 fs/zonefs/file.c
--
2.39.0
next reply other threads:[~2023-01-10 13:08 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-01-10 13:08 Damien Le Moal [this message]
2023-01-10 13:08 ` [PATCH 1/7] zonefs: Detect append writes at invalid locations Damien Le Moal
2023-01-11 12:23 ` Johannes Thumshirn
2023-01-11 22:08 ` Damien Le Moal
2023-01-12 5:18 ` Damien Le Moal
2023-01-12 7:33 ` Johannes Thumshirn
2023-01-12 8:26 ` Damien Le Moal
2023-01-10 13:08 ` [PATCH 2/7] zonefs: Reorganize code Damien Le Moal
2023-01-11 16:12 ` Johannes Thumshirn
2023-01-10 13:08 ` [PATCH 3/7] zonefs: Simplify IO error handling Damien Le Moal
2023-01-13 10:09 ` Johannes Thumshirn
2023-01-10 13:08 ` [PATCH 4/7] zonefs: Reduce struct zonefs_inode_info size Damien Le Moal
2023-01-13 10:11 ` Johannes Thumshirn
2023-01-10 13:08 ` [PATCH 5/7] zonefs: Separate zone information from inode information Damien Le Moal
2023-01-13 11:30 ` Johannes Thumshirn
2023-01-10 13:08 ` [PATCH 6/7] zonefs: Dynamically create file inodes when needed Damien Le Moal
2023-01-16 11:46 ` Johannes Thumshirn
2023-01-16 12:17 ` Damien Le Moal
2023-01-10 13:08 ` [PATCH 7/7] zonefs: Cache zone group directory inodes Damien Le Moal
2023-01-16 11:49 ` Johannes Thumshirn
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230110130830.246019-1-damien.lemoal@opensource.wdc.com \
--to=damien.lemoal@opensource.wdc.com \
--cc=Jorgen.Hansen@wdc.com \
--cc=johannes.thumshirn@wdc.com \
--cc=linux-fsdevel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).