linux-fsdevel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH 00/11] ntfsplus: ntfs filesystem remake
@ 2025-10-20  2:07 Namjae Jeon
  2025-10-20  2:07 ` [PATCH 01/11] ntfsplus: in-memory, on-disk structures and headers Namjae Jeon
                   ` (7 more replies)
  0 siblings, 8 replies; 20+ messages in thread
From: Namjae Jeon @ 2025-10-20  2:07 UTC (permalink / raw)
  To: viro, brauner, hch, hch, tytso, willy, jack, djwong, josef,
	sandeen, rgoldwyn, xiang, dsterba, pali, ebiggers, neil, amir73il
  Cc: linux-fsdevel, linux-kernel, iamjoonsoo.kim, cheol.lee, jay.sim,
	gunho.lee, Namjae Jeon

Introduction
============

The NTFS filesystem[1] still remains the default filesystem for Windows
and The well-maintained NTFS driver in the Linux kernel enhances
interoperability with Windows devices, making it easier for Linux users
to work with NTFS-formatted drives. Currently, ntfs support in Linux was
the long-neglected NTFS Classic (read-only), which has been removed from
the Linux kernel, leaving the poorly maintained ntfs3. ntfs3 still has
many problems and is poorly maintained, so users and distributions are
still using the old legacy ntfs-3g.


What is ntfsplus?
=================

The remade ntfs called ntfsplus is an implementation that supports write
and the essential requirements(iomap, no buffer-head, utilities, xfstests
test result) based on read-only classic NTFS.
The old read-only ntfs code is much cleaner, with extensive comments,
offers readability that makes understanding NTFS easier. This is why
ntfsplus was developed on old read-only NTFS base.
The target is to provide current trends(iomap, no buffer head, folio),
enhanced performance, stable maintenance, utility support including fsck.


Key Features
============

- Write support:
   Implement write support on classic read-only NTFS. Additionally,
   integrate delayed allocation to enhance write performance through
   multi-cluster allocation and minimized fragmentation of cluster bitmap.

- Switch to using iomap:
   Use iomap for buffered IO writes, reads, direct IO, file extent mapping,
   readpages, writepages operations.

- Stop using the buffer head:
   The use of buffer head in old ntfs and switched to use folio instead.
   As a result, CONFIG_BUFFER_HEAD option enable is removed in Kconfig also.

- Public utilities include fsck[2]:
   While ntfs-3g includes ntfsprogs as a component, it notably lacks
   the fsck implementation. So we have launched a new ntfs utilitiies
   project called ntfsprogs-plus by forking from ntfs-3g after removing
   unnecessary ntfs fuse implementation. fsck.ntfs can be used for ntfs
   testing with xfstests as well as for recovering corrupted NTFS device.

- Performance Enhancements:

   - ntfsplus vs. ntfs3:

     * Performance was benchmarked using iozone with various chunk size.
        - In single-thread(1T) write tests, ntfsplus show approximately
          3~5% better performance.
        - In multi-thread(4T) write tests, ntfsplus show approximately
          35~110% better performance.
        - Read throughput is identical for both ntfs implementations.

     1GB file      size:4096           size:16384           size:65536
     MB/sec   ntfsplus | ntfs3    ntfsplus | ntfs3    ntfsplus | ntfs3
     ─────────────────────────────────────────────────────────────────
     read          399 | 399           426 | 424           429 | 430
     ─────────────────────────────────────────────────────────────────
     write(1T)     291 | 276           325 | 305           333 | 317
     write(4T)     105 | 50            113 | 78            114 | 99.6


     * File list browsing performance. (about 12~14% faster)

                  files:100000        files:200000        files:400000
     Sec      ntfsplus | ntfs3    ntfsplus | ntfs3    ntfsplus | ntfs3
     ─────────────────────────────────────────────────────────────────
     ls -lR       7.07 | 8.10        14.03 | 16.35       28.27 | 32.86


     * mount time.

             parti_size:1TB      parti_size:2TB      parti_size:4TB
     Sec      ntfsplus | ntfs3    ntfsplus | ntfs3    ntfsplus | ntfs3
     ─────────────────────────────────────────────────────────────────
     mount        0.38 | 2.03         0.39 | 2.25         0.70 | 4.51

   The following are the reasons why ntfsplus performance is higher
    compared to ntfs3:
     - Use iomap aops.
     - Delayed allocation support.
     - Optimize zero out for newly allocated clusters.
     - Optimize runlist merge overhead with small chunck size.
     - pre-load mft(inode) blocks and index(dentry) blocks to improve
       readdir + stat performance.
     - Load lcn bitmap on background.

- Stability improvement:
   a. Pass more xfstests tests:
      ntfsplus passed 287 tests, significantly higher than ntfs3's 218.
      ntfsplus implement fallocate, idmapped mount and permission, etc,
      resulting in a significantly high number of xfstests passing compared
      to ntfs3.
   b. Bonnie++ issue[3]:
      The Bonnie++ benchmark fails on ntfs3 with a "Directory not empty"
      error during file deletion. ntfs3 currently iterates directory
      entries by reading index blocks one by one. When entries are deleted
      concurrently, index block merging or entry relocation can cause
      readdir() to skip some entries, leaving files undeleted in
      workloads(bonnie++) that mix unlink and directory scans.
      ntfsplus implement leaf chain traversal in readdir to avoid entry skip
      on deletion.

- Journaling support:
   ntfs3 does not provide full journaling support. It only implement journal
   replay[4], which in our testing did not function correctly. My next task
   after upstreaming will be to add full journal support to ntfsplus.


The feature comparison summary
==============================

Feature                               ntfsplus   ntfs3
===================================   ========   ===========
Write support                         Yes        Yes
iomap support                         Yes        No
No buffer head                        Yes        No
Public utilities(mkfs, fsck, etc.)    Yes        No
xfstests passed                       287        218
Idmapped mount                        Yes        No
Delayed allocation                    Yes        No
Bonnie++                              Pass       Fail
Journaling                            Planned    Inoperative
===================================   ========   ===========

References
==========
[1] https://en.wikipedia.org/wiki/NTFS
[2] https://github.com/ntfsprogs-plus/ntfsprogs-plus
[3] https://lore.kernel.org/ntfs3/CAOZgwEd7NDkGEpdF6UQTcbYuupDavaHBoj4WwTy3Qe4Bqm6V0g@mail.gmail.com/
[4] https://marc.info/?l=linux-fsdevel&m=161738417018673&q=mbox


Namjae Jeon (11):
  ntfsplus: in-memory, on-disk structures and headers
  ntfsplus: add super block operations
  ntfsplus: add inode operations
  ntfsplus: add directory operations
  ntfsplus: add file operations
  ntfsplus: add iomap and address space operations
  ntfsplus: add attrib operatrions
  ntfsplus: add runlist handling and cluster allocator
  ntfsplus: add reparse and ea operations
  ntfsplus: add misc operations
  ntfsplus: add Kconfig and Makefile

 fs/Kconfig                |    1 +
 fs/Makefile               |    1 +
 fs/ntfsplus/Kconfig       |   45 +
 fs/ntfsplus/Makefile      |   18 +
 fs/ntfsplus/aops.c        |  631 +++++
 fs/ntfsplus/aops.h        |   92 +
 fs/ntfsplus/attrib.c      | 5373 +++++++++++++++++++++++++++++++++++++
 fs/ntfsplus/attrib.h      |  159 ++
 fs/ntfsplus/attrlist.c    |  276 ++
 fs/ntfsplus/attrlist.h    |   21 +
 fs/ntfsplus/bitmap.c      |  193 ++
 fs/ntfsplus/bitmap.h      |   90 +
 fs/ntfsplus/collate.c     |  173 ++
 fs/ntfsplus/collate.h     |   37 +
 fs/ntfsplus/compress.c    | 1565 +++++++++++
 fs/ntfsplus/dir.c         | 1226 +++++++++
 fs/ntfsplus/dir.h         |   33 +
 fs/ntfsplus/ea.c          |  712 +++++
 fs/ntfsplus/ea.h          |   25 +
 fs/ntfsplus/file.c        | 1056 ++++++++
 fs/ntfsplus/index.c       | 2114 +++++++++++++++
 fs/ntfsplus/index.h       |  127 +
 fs/ntfsplus/inode.c       | 3705 +++++++++++++++++++++++++
 fs/ntfsplus/inode.h       |  354 +++
 fs/ntfsplus/layout.h      | 2288 ++++++++++++++++
 fs/ntfsplus/lcnalloc.c    |  993 +++++++
 fs/ntfsplus/lcnalloc.h    |  127 +
 fs/ntfsplus/logfile.c     |  773 ++++++
 fs/ntfsplus/logfile.h     |  316 +++
 fs/ntfsplus/mft.c         | 2630 ++++++++++++++++++
 fs/ntfsplus/mft.h         |   93 +
 fs/ntfsplus/misc.c        |  221 ++
 fs/ntfsplus/misc.h        |  218 ++
 fs/ntfsplus/mst.c         |  195 ++
 fs/ntfsplus/namei.c       | 1606 +++++++++++
 fs/ntfsplus/ntfs.h        |  172 ++
 fs/ntfsplus/ntfs_iomap.c  |  704 +++++
 fs/ntfsplus/ntfs_iomap.h  |   22 +
 fs/ntfsplus/reparse.c     |  550 ++++
 fs/ntfsplus/reparse.h     |   15 +
 fs/ntfsplus/runlist.c     | 1995 ++++++++++++++
 fs/ntfsplus/runlist.h     |   91 +
 fs/ntfsplus/super.c       | 2716 +++++++++++++++++++
 fs/ntfsplus/unistr.c      |  471 ++++
 fs/ntfsplus/upcase.c      |   73 +
 fs/ntfsplus/volume.h      |  241 ++
 include/uapi/linux/ntfs.h |   23 +
 47 files changed, 34560 insertions(+)
 create mode 100644 fs/ntfsplus/Kconfig
 create mode 100644 fs/ntfsplus/Makefile
 create mode 100644 fs/ntfsplus/aops.c
 create mode 100644 fs/ntfsplus/aops.h
 create mode 100644 fs/ntfsplus/attrib.c
 create mode 100644 fs/ntfsplus/attrib.h
 create mode 100644 fs/ntfsplus/attrlist.c
 create mode 100644 fs/ntfsplus/attrlist.h
 create mode 100644 fs/ntfsplus/bitmap.c
 create mode 100644 fs/ntfsplus/bitmap.h
 create mode 100644 fs/ntfsplus/collate.c
 create mode 100644 fs/ntfsplus/collate.h
 create mode 100644 fs/ntfsplus/compress.c
 create mode 100644 fs/ntfsplus/dir.c
 create mode 100644 fs/ntfsplus/dir.h
 create mode 100644 fs/ntfsplus/ea.c
 create mode 100644 fs/ntfsplus/ea.h
 create mode 100644 fs/ntfsplus/file.c
 create mode 100644 fs/ntfsplus/index.c
 create mode 100644 fs/ntfsplus/index.h
 create mode 100644 fs/ntfsplus/inode.c
 create mode 100644 fs/ntfsplus/inode.h
 create mode 100644 fs/ntfsplus/layout.h
 create mode 100644 fs/ntfsplus/lcnalloc.c
 create mode 100644 fs/ntfsplus/lcnalloc.h
 create mode 100644 fs/ntfsplus/logfile.c
 create mode 100644 fs/ntfsplus/logfile.h
 create mode 100644 fs/ntfsplus/mft.c
 create mode 100644 fs/ntfsplus/mft.h
 create mode 100644 fs/ntfsplus/misc.c
 create mode 100644 fs/ntfsplus/misc.h
 create mode 100644 fs/ntfsplus/mst.c
 create mode 100644 fs/ntfsplus/namei.c
 create mode 100644 fs/ntfsplus/ntfs.h
 create mode 100644 fs/ntfsplus/ntfs_iomap.c
 create mode 100644 fs/ntfsplus/ntfs_iomap.h
 create mode 100644 fs/ntfsplus/reparse.c
 create mode 100644 fs/ntfsplus/reparse.h
 create mode 100644 fs/ntfsplus/runlist.c
 create mode 100644 fs/ntfsplus/runlist.h
 create mode 100644 fs/ntfsplus/super.c
 create mode 100644 fs/ntfsplus/unistr.c
 create mode 100644 fs/ntfsplus/upcase.c
 create mode 100644 fs/ntfsplus/volume.h
 create mode 100644 include/uapi/linux/ntfs.h

-- 
2.34.1


^ permalink raw reply	[flat|nested] 20+ messages in thread
* Re: [PATCH 00/11] ntfsplus: ntfs filesystem remake
@ 2025-10-22 21:24 Lassebq
  0 siblings, 0 replies; 20+ messages in thread
From: Lassebq @ 2025-10-22 21:24 UTC (permalink / raw)
  To: linkinjeon
  Cc: amir73il, brauner, cheol.lee, djwong, dsterba, ebiggers,
	gunho.lee, hch, hch, iamjoonsoo.kim, jack, jay.sim, josef,
	linux-fsdevel, linux-kernel, neil, pali, rgoldwyn, sandeen, tytso,
	viro, willy, xiang

Hello

I wanted to apply these patches on top of my kernel build, but I 
noticed that not every diff is included in the thread. And I think some 
code in patches appears to be a out of date or target an older kernel? 
Like there is usage of inode_generic_drop instead of generic_drop_inode.

Is there a reason for that? I'm new to kernel mailing lists so I'm not 
exactly sure why that's the case or how it works.

Appreciate if you could clear things up for a newbie like me.

Ivan



^ permalink raw reply	[flat|nested] 20+ messages in thread
[parent not found: <7YXJ4T.2LNCI9JPXKCZ2@gmail.com>]

end of thread, other threads:[~2025-10-26  5:37 UTC | newest]

Thread overview: 20+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-10-20  2:07 [PATCH 00/11] ntfsplus: ntfs filesystem remake Namjae Jeon
2025-10-20  2:07 ` [PATCH 01/11] ntfsplus: in-memory, on-disk structures and headers Namjae Jeon
2025-10-20  2:07 ` [PATCH 02/11] ntfsplus: add super block operations Namjae Jeon
2025-10-20  2:07 ` [PATCH 03/11] ntfsplus: add inode operations Namjae Jeon
2025-10-20  2:07 ` [PATCH 04/11] ntfsplus: add directory operations Namjae Jeon
2025-10-20  2:07 ` [PATCH 05/11] ntfsplus: add file operations Namjae Jeon
2025-10-20 18:33 ` [PATCH 00/11] ntfsplus: ntfs filesystem remake Pali Rohár
2025-10-21  1:49   ` Namjae Jeon
2025-10-21 22:19     ` Pali Rohár
2025-10-22  2:13       ` Namjae Jeon
2025-10-22 18:52         ` Pali Rohár
2025-10-22 22:32           ` Namjae Jeon
2025-10-21  0:17 ` Bagas Sanjaya
2025-10-21  1:55   ` Namjae Jeon
2025-10-26  5:37     ` Bagas Sanjaya
2025-10-22  6:30 ` David Sterba
2025-10-22  8:33   ` Namjae Jeon
2025-10-22 18:57   ` Pali Rohár
  -- strict thread matches above, loose matches on Subject: below --
2025-10-22 21:24 Lassebq
     [not found] <7YXJ4T.2LNCI9JPXKCZ2@gmail.com>
2025-10-23  7:20 ` Namjae Jeon

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).