linux-mm.kvack.org archive mirror
 help / color / mirror / Atom feed
* [RFC PATCH 0/5] BLOG: per-task logging contexts with Ceph consumer
@ 2025-10-24  8:42 Alex Markuze
  2025-10-24  8:42 ` [RFC PATCH 1/5] sched, fork: Wire BLOG contexts into task lifecycle Alex Markuze
                   ` (7 more replies)
  0 siblings, 8 replies; 19+ messages in thread
From: Alex Markuze @ 2025-10-24  8:42 UTC (permalink / raw)
  To: ceph-devel, linux-kernel, linux-mm
  Cc: Liam.Howlett, amarkuze, akpm, bsegall, david, dietmar.eggemann,
	idryomov, mingo, juri.lelli, kees, lorenzo.stoakes, mgorman,
	mhocko, rppt, peterz, rostedt, surenb, vschneid, vincent.guittot,
	vbabka, xiubli, Slava.Dubeyko

Motivation: improve observability in production by providing subsystemsawith 
a logger that keeps up with their verbouse unstructured logs and aggregating
logs at the process context level, akin to userspace TLS. 

Binary LOGging (BLOG) introduces a task-local logging context: each context
owns a single 512 KiB fragment that cycles through “ready → in use → queued for
readers → reset → ready” without re-entering the allocator. Writers copy the
raw parameters they already have; readers format them later when the log is
inspected.

BLOG borrows ideas from ftrace (captureabinary  data now, format later) but 
unlike ftrace there is no global ring. Each module registers its own logger,
manages its own buffers, and keeps the state small enough for production use.

To host the per-module pointers we extend `struct task_struct` with one
additional `void *`, in line with other task extensions already in the kernel.
Each module keeps independent batches: `alloc_batch` for contexts with
refcount 0 and `log_batch` for contexts that have been filled and are waiting
for readers. The batching layer and buffer management were migrated from the
existing Ceph SAN logging code, so the behaviour is battle-tested; we simply
made the buffer inline so every composite stays within a single 512 KiB
allocation.

The patch series lands the BLOG library first, then wires the task lifecycle,
and finally switches Ceph’s `bout*` logging macros to BLOG so we exercise the
new path.

Patch summary:
  1. sched, fork: wire BLOG contexts into task lifecycle
     - Adds `struct blog_tls_ctx *blog_contexts[BLOG_MAX_MODULES]` to
       `struct task_struct`.
     - Fork/exit paths initialise and recycle contexts automatically.

  2. lib: introduce BLOG (Binary LOGging) subsystem
     - Adds `lib/blog/` sources and headers under `include/linux/blog/`.
     - Each composite (`struct blog_tls_pagefrag`) consists of the TLS
       metadata, the pagefrag state, and an inline buffer sized at
       `BLOG_PAGEFRAG_SIZE - sizeof(struct blog_tls_pagefrag)`.

  3. ceph: add BLOG scaffolding
     - Introduces `include/linux/ceph/ceph_blog.h` and `fs/ceph/blog_client.c`.
     - Ceph registers a logger and maintains a client-ID map for the reader
       callback.

  4. ceph: add BLOG debugfs support
     - Adds `fs/ceph/blog_debugfs.c` so filled contexts can be drained.

  5. ceph: activate BLOG logging
     - Switches `bout*` macros to BLOG, making Ceph the first consumer.

With these patches, Ceph now writes its verbose logging to task-local buffers
managed by BLOG, and the infrastructure is ready for other subsystems that need
allocation-free, module-owned logging.

Alex Markuze (5):
  sched, fork: Wire BLOG contexts into task lifecycle
  lib: Introduce BLOG (Binary LOGging) subsystem
  ceph: Add BLOG scaffolding
  ceph: Add BLOG debugfs support
  ceph: Activate BLOG logging

 fs/ceph/Makefile                   |   2 +
 fs/ceph/addr.c                     | 130 ++---
 fs/ceph/blog_client.c              | 244 +++++++++
 fs/ceph/blog_debugfs.c             | 361 +++++++++++++
 fs/ceph/caps.c                     | 242 ++++-----
 fs/ceph/crypto.c                   |  18 +-
 fs/ceph/debugfs.c                  |  33 +-
 fs/ceph/dir.c                      |  88 ++--
 fs/ceph/export.c                   |  20 +-
 fs/ceph/file.c                     | 130 ++---
 fs/ceph/inode.c                    | 182 +++----
 fs/ceph/ioctl.c                    |   6 +-
 fs/ceph/locks.c                    |  22 +-
 fs/ceph/mds_client.c               | 278 +++++-----
 fs/ceph/mdsmap.c                   |   8 +-
 fs/ceph/quota.c                    |   2 +-
 fs/ceph/snap.c                     |  66 +--
 fs/ceph/super.c                    |  82 +--
 fs/ceph/xattr.c                    |  42 +-
 include/linux/blog/blog.h          | 515 +++++++++++++++++++
 include/linux/blog/blog_batch.h    |  54 ++
 include/linux/blog/blog_des.h      |  46 ++
 include/linux/blog/blog_module.h   | 329 ++++++++++++
 include/linux/blog/blog_pagefrag.h |  33 ++
 include/linux/blog/blog_ser.h      | 275 ++++++++++
 include/linux/ceph/ceph_blog.h     | 124 +++++
 include/linux/ceph/ceph_debug.h    |   6 +-
 include/linux/sched.h              |   7 +
 kernel/fork.c                      |  37 ++
 lib/Kconfig                        |   2 +
 lib/Makefile                       |   2 +
 lib/blog/Kconfig                   |  56 +++
 lib/blog/Makefile                  |  15 +
 lib/blog/blog_batch.c              | 311 ++++++++++++
 lib/blog/blog_core.c               | 772 ++++++++++++++++++++++++++++
 lib/blog/blog_des.c                | 385 ++++++++++++++
 lib/blog/blog_module.c             | 781 +++++++++++++++++++++++++++++
 lib/blog/blog_pagefrag.c           | 124 +++++
 38 files changed, 5163 insertions(+), 667 deletions(-)
 create mode 100644 fs/ceph/blog_client.c
 create mode 100644 fs/ceph/blog_debugfs.c
 create mode 100644 include/linux/blog/blog.h
 create mode 100644 include/linux/blog/blog_batch.h
 create mode 100644 include/linux/blog/blog_des.h
 create mode 100644 include/linux/blog/blog_module.h
 create mode 100644 include/linux/blog/blog_pagefrag.h
 create mode 100644 include/linux/blog/blog_ser.h
 create mode 100644 include/linux/ceph/ceph_blog.h
 create mode 100644 lib/blog/Kconfig
 create mode 100644 lib/blog/Makefile
 create mode 100644 lib/blog/blog_batch.c
 create mode 100644 lib/blog/blog_core.c
 create mode 100644 lib/blog/blog_des.c
 create mode 100644 lib/blog/blog_module.c
 create mode 100644 lib/blog/blog_pagefrag.c

-- 
2.34.1



^ permalink raw reply	[flat|nested] 19+ messages in thread

end of thread, other threads:[~2025-11-03 22:37 UTC | newest]

Thread overview: 19+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-10-24  8:42 [RFC PATCH 0/5] BLOG: per-task logging contexts with Ceph consumer Alex Markuze
2025-10-24  8:42 ` [RFC PATCH 1/5] sched, fork: Wire BLOG contexts into task lifecycle Alex Markuze
2025-10-24 17:44   ` Steven Rostedt
2025-10-29 18:57   ` Viacheslav Dubeyko
2025-10-24  8:42 ` [RFC PATCH 2/5] lib: Introduce BLOG (Binary LOGging) subsystem Alex Markuze
2025-10-30 18:47   ` Viacheslav Dubeyko
2025-10-24  8:42 ` [RFC PATCH 3/5] ceph: Add BLOG scaffolding Alex Markuze
2025-11-03 22:37   ` Viacheslav Dubeyko
2025-10-24  8:42 ` [RFC PATCH 4/5] ceph: Add BLOG debugfs support Alex Markuze
2025-11-03 21:07   ` Viacheslav Dubeyko
2025-10-24  8:42 ` [RFC PATCH 5/5] ceph: Activate BLOG logging Alex Markuze
2025-11-03 21:00   ` Viacheslav Dubeyko
2025-10-24 15:32 ` [RFC PATCH 0/5] BLOG: per-task logging contexts with Ceph consumer David Hildenbrand
2025-10-24 17:53 ` Steven Rostedt
2025-10-25 10:50   ` Alex Markuze
2025-10-25 14:59     ` Steven Rostedt
2025-10-25 17:54       ` Alex Markuze
2025-10-27 14:54         ` Steven Rostedt
2025-10-28 17:07 ` Viacheslav Dubeyko

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).