linux-fsdevel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH 0/7] Reduce zonefs memory usage
@ 2023-01-10 13:08 Damien Le Moal
  2023-01-10 13:08 ` [PATCH 1/7] zonefs: Detect append writes at invalid locations Damien Le Moal
                   ` (6 more replies)
  0 siblings, 7 replies; 20+ messages in thread
From: Damien Le Moal @ 2023-01-10 13:08 UTC (permalink / raw)
  To: linux-fsdevel; +Cc: Johannes Thumshirn, Jorgen Hansen

This series improves memory usage by switching to using dynamically
allocated inodes and dentries, similarly to regular file systems. This
drastically reduces the memory consumption of zonefs when the file
system is mounted. E.g., for a 26 TB SMR HDD with over 95000 zones,
memory usage is decreased from about 130 MB down to a little over 5 MB.

Since zonefs does not have persistent metadata of its own and relies
completely on the device zone state management, information such as the 
zone write pointer position (used to determine an inode size) is kept in
memory at all time, even when an inode is evicted from the inode cache. 
Dentries can be trivially searched for and created dynamically as needed
due to the static file tree structure of a zonefs volume (fully
determined by a zoned device zone configuration).

The patch series is organized as follows. Patch 1 fixes a problem with
synchronous write error detection and error recovery. Patch 2
reorganizes the code (splitting out file operation code into a new file)
and patch 3 simplifies the IO error recovery code. Patch 4 and 5 cleanup
inode zone information and split that information out of the zonefs
inode structure. Finally, Patch 6 implements dynamic inode and dentry
allocation and maanagement operations. Ptch 7 introduces a small
optimization to reduce file open latency.

Damien Le Moal (7):
  zonefs: Detect append writes at invalid locations
  zonefs: Reorganize code
  zonefs: Simplify IO error handling
  zonefs: Reduce struct zonefs_inode_info size
  zonefs: Separate zone information from inode information
  zonefs: Dynamically create file inodes when needed
  zonefs: Cache zone group directory inodes

 fs/zonefs/Makefile |    2 +-
 fs/zonefs/file.c   |  878 ++++++++++++++++++++
 fs/zonefs/super.c  | 1923 ++++++++++++++++----------------------------
 fs/zonefs/trace.h  |   20 +-
 fs/zonefs/zonefs.h |  110 ++-
 5 files changed, 1684 insertions(+), 1249 deletions(-)
 create mode 100644 fs/zonefs/file.c

-- 
2.39.0


^ permalink raw reply	[flat|nested] 20+ messages in thread

end of thread, other threads:[~2023-01-16 12:18 UTC | newest]

Thread overview: 20+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-01-10 13:08 [PATCH 0/7] Reduce zonefs memory usage Damien Le Moal
2023-01-10 13:08 ` [PATCH 1/7] zonefs: Detect append writes at invalid locations Damien Le Moal
2023-01-11 12:23   ` Johannes Thumshirn
2023-01-11 22:08     ` Damien Le Moal
2023-01-12  5:18       ` Damien Le Moal
2023-01-12  7:33         ` Johannes Thumshirn
2023-01-12  8:26           ` Damien Le Moal
2023-01-10 13:08 ` [PATCH 2/7] zonefs: Reorganize code Damien Le Moal
2023-01-11 16:12   ` Johannes Thumshirn
2023-01-10 13:08 ` [PATCH 3/7] zonefs: Simplify IO error handling Damien Le Moal
2023-01-13 10:09   ` Johannes Thumshirn
2023-01-10 13:08 ` [PATCH 4/7] zonefs: Reduce struct zonefs_inode_info size Damien Le Moal
2023-01-13 10:11   ` Johannes Thumshirn
2023-01-10 13:08 ` [PATCH 5/7] zonefs: Separate zone information from inode information Damien Le Moal
2023-01-13 11:30   ` Johannes Thumshirn
2023-01-10 13:08 ` [PATCH 6/7] zonefs: Dynamically create file inodes when needed Damien Le Moal
2023-01-16 11:46   ` Johannes Thumshirn
2023-01-16 12:17     ` Damien Le Moal
2023-01-10 13:08 ` [PATCH 7/7] zonefs: Cache zone group directory inodes Damien Le Moal
2023-01-16 11:49   ` Johannes Thumshirn

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).