All of lore.kernel.org
 help / color / mirror / Atom feed
From: Ayhan Aydin <nsd.project.dev@gmail.com>
To: linux-kernel@vger.kernel.org, linux-mm@kvack.org
Cc: nsd.project.dev@gmail.com
Subject: [RFC PATCH 3/3] Documentation: Add NSD filesystem documentation
Date: Sat, 25 Jul 2026 21:26:28 +0300	[thread overview]
Message-ID: <20260725182628.221603-4-nsd.project.dev@gmail.com> (raw)
In-Reply-To: <20260725182628.221603-1-nsd.project.dev@gmail.com>

Add initial documentation for the Neural Storage Driver
covering architecture, sysfs interface, and performance.

Signed-off-by: Ayhan Aydin <nsd.project.dev@gmail.com>
---
 Documentation/filesystems/nsd.rst | 77 +++++++++++++++++++++++++++++++
 1 file changed, 77 insertions(+)
 create mode 100644 Documentation/filesystems/nsd.rst

diff --git a/Documentation/filesystems/nsd.rst b/Documentation/filesystems/nsd.rst
new file mode 100644
index 0000000..a78018a
--- /dev/null
+++ b/Documentation/filesystems/nsd.rst
@@ -0,0 +1,77 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+========================================
+Neural Storage Driver (NSD) - v1.0.0
+========================================
+
+Overview
+========
+NSD is a learning prefetcher for the Linux kernel page cache. It monitors
+I/O patterns via a hook in :c:func:`filemap_read` and prefetches pages
+ahead of the application using :c:func:`page_cache_sync_readahead`.
+
+Unlike the kernel existing readahead (fixed window), NSD builds
+a synaptic Markov chain model of access patterns at 4 KB region
+granularity. It detects sequential strides, repeating patterns, and
+learned transitions.
+
+Architecture
+============::
+
+    Application
+         |
+         v
+    filemap_read()  <-- NSD hook
+         |
+         +---> nsd_notify_read()
+                   |
+                   +---> Synaptic Table (Markov chain)
+                   |        |
+                   |        +---> Region hash + stride predictor
+                   |
+                   +---> Prefetch Worker
+                            |
+                            +---> page_cache_sync_readahead(WILLNEED)
+
+Sysfs Interface
+===============
+/sys/kernel/nsd/::
+
+  stats          Current statistics (hit rate, prefetch count, etc.)
+  observe_only   0=active, 1=observe-only (no prefetch)
+  penalty        Enable penalty weakening on waste
+  waste_track    Enable waste tracking
+
+Statistics fields::
+
+  prefetched    Total pages prefetched
+  used          Prefetched pages that were actually accessed
+  wasted        Prefetched pages never accessed
+  hit_rate_real Used / Prefetched ratio
+  stride_preds  Stride predictions made
+  chain_depth   Average chain prediction depth
+  synapse_ents  Current synapse table entries
+  ring_events   Pending ring buffer events
+
+Module Parameters
+=================
+  observe_only  Start in observe-only mode (default: false)
+  penalty       Enable penalty on waste (default: true)
+  waste_track   Enable waste tracking (default: false)
+
+Performance
+===========
+Tested on: x86_64, SATA SSD, kernel 7.0.0::
+
+  Workload                     Improvement
+  ---------------------------  -----------
+  SQLite FTS (4 GB table)      -18.8% query time
+  Sequential 64K buffered      +22.6% throughput
+  Random 4K buffered           +1.1% (noise)
+
+The prefetcher achieves 98% real hit rate on SSD workloads.
+
+See Also
+========
+Documentation/filesystems/fscache.rst
+Documentation/filesystems/squashfs.rst
\ No newline at end of file
-- 
2.43.0



      parent reply	other threads:[~2026-07-25 18:26 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-07-25 18:26 [RFC PATCH 0/3] Neural Storage Driver - learning page cache prefetcher Ayhan Aydin
2026-07-25 18:26 ` [RFC PATCH 1/3] mm/filemap: Add NSD prefetch hook point Ayhan Aydin
2026-07-25 18:26 ` [RFC PATCH 2/3] nsd: Core prediction engine Ayhan Aydin
2026-07-25 18:26 ` Ayhan Aydin [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260725182628.221603-4-nsd.project.dev@gmail.com \
    --to=nsd.project.dev@gmail.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.