[PATCH 00/24] nilfs2: introduce debugging subsystem implementation

[PATCH 00/24] nilfs2: introduce debugging subsystem implementation

[Vyacheslav Dubeyko]

Hi,

This patch set implements debugging subsystem in NILFS2 driver.

The debugging subsystem includes:
(1) debugging subsystem API;
(2) subsystems and printing opportunity flags;
(3) kernel configuration options.

The debugging subsystem API includes:
(1) nilfs2_debug() method.
    It is a basic debugging method. It expects a set of
    flags, format string and variable arguments list.

(2) nilfs2_hexdump() method.
    It is a method for writing in system log hexdumps
    of internal structures. It expects a set of flags,
    prefix string, pointer on memory and size of pointed data.

(3) NILFS_ERR_DBG() macros.
    It is a macros that prints out file name, line number,
    function name and error code.

Every module of NILFS2 driver has dedicated flag. These flags
give opportunity to enable debugging output only from required
modules of NILFS2 driver. There are several special output
flags:
(1) DBG_SHOW_ERR flag.
    This flag requests writing in system log messages about
    internal errors of NILFS2 driver.

(2) DBG_DUMP_STACK flag.
    This flag requests output of dump stack.

(3) DBG_SPAM flag.
    This flag requsts output from frequently called functions
    or detailed debugging output from function's body.

(4) DBG_HEX_DUMP flag.
    This flag requests writing in system log hexdumps of internal
    structures.

The debugging subsystem of NILFS2 driver has DBG_MASK that defines
set of flags for debugging output. Namely, this mask is used for
comparing requested flag with set of flags that should be printed out.
Such selection mechanism is used for the case of printk() calls.

Also, it is available opportunity of dynamic debugging by means of
pr_debug() method using. In such case the behaviour of pr_debug() is
controlled via writing to a control file in the 'debugfs' filesystem.

The debugging subsystem is configured by means of kernel
configuration options:
(1) NILFS2_USE_PR_DEBUG
    This option enables using of pr_debug() instead of printk() and
    print_hex_dump_bytes() instead of print_hex_dump().

(2) NILFS2_DEBUG_SHOW_ERRORS
    This option enables writing in system log messages about internal
    errors of NILFS2 driver.

(3) NILFS2_DEBUG_DUMP_STACK
    This option enables output of dump stack. Usually, every
    function in NILFS2 driver begins from debugging output of
    function name, file, line and input arguments' value.
    In the case of enabling this option debugging output
    will include dump stack too.

(4) NILFS2_DEBUG_SHOW_SPAM
    This option enables output from frequently called functions or
    detailed debugging output from function's body.

(5) NILFS2_DEBUG_HEXDUMP
    This option enables writing in system log hexdumps of internal
    structures.

(6) NILFS2_DEBUG_BASE_OPERATIONS
    This option enables base operations subsystem debugging output.

(7) NILFS2_DEBUG_MDT_FILES
    This option enables metadata (MDT) files subsystem debugging output.

(8) NILFS2_DEBUG_SEGMENTS_SUBSYSTEM
    This option enables segments subsystem debugging output.

(9) NILFS2_DEBUG_GC_SUBSYSTEM
    This option enables GC subsystem debugging output.

(10) NILFS2_DEBUG_RECOVERY_SUBSYSTEM
     This option enables recovery subsystem debugging output.

(11) NILFS2_DEBUG_BLOCK_MAPPING
     This option enables block mapping subsystem debugging output.

(12) NILFS2_DEBUG_BUFFER_MANAGEMENT
     This option enables buffer/page management subsystem debugging
     output.

With the best regards,
Vyacheslav Dubeyko.
---
 Vyacheslav Dubeyko (1):
    nilfs2: introduce debugging subsystem implementation

 fs/nilfs2/Kconfig     |  145 +++++++++++++
 fs/nilfs2/alloc.c     |  140 +++++++++++--
 fs/nilfs2/bmap.c      |  109 +++++++++-
 fs/nilfs2/btnode.c    |   45 +++-
 fs/nilfs2/btree.c     |  515 +++++++++++++++++++++++++++++++++++++++++-----
 fs/nilfs2/cpfile.c    |  204 +++++++++++++++----
 fs/nilfs2/dat.c       |  130 ++++++++++--
 fs/nilfs2/debug.h     |  291 ++++++++++++++++++++++++++
 fs/nilfs2/dir.c       |   71 ++++++-
 fs/nilfs2/direct.c    |  117 +++++++++--
 fs/nilfs2/file.c      |   29 ++-
 fs/nilfs2/gcinode.c   |   23 ++-
 fs/nilfs2/ifile.c     |   55 ++++-
 fs/nilfs2/inode.c     |  271 +++++++++++++++++++++----
 fs/nilfs2/ioctl.c     |  275 +++++++++++++++++++------
 fs/nilfs2/mdt.c       |  128 +++++++++---
 fs/nilfs2/namei.c     |  183 +++++++++++++----
 fs/nilfs2/nilfs.h     |    1 +
 fs/nilfs2/page.c      |   52 ++++-
 fs/nilfs2/recovery.c  |  160 ++++++++++++---
 fs/nilfs2/segbuf.c    |   90 +++++++-
 fs/nilfs2/segment.c   |  541 +++++++++++++++++++++++++++++++++++++++++++------
 fs/nilfs2/sufile.c    |  172 +++++++++++++---
 fs/nilfs2/super.c     |  192 ++++++++++++++----
 fs/nilfs2/the_nilfs.c |  130 +++++++++---
 25 files changed, 3537 insertions(+), 532 deletions(-)
--
1.7.9.5



--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 00/24] nilfs2: introduce debugging subsystem implementation

[Ryusuke Konishi]

On Mon, 17 Jun 2013 16:21:41 +0400, Vyacheslav Dubeyko wrote:
> Hi,
> 
> This patch set implements debugging subsystem in NILFS2 driver.
> 
> The debugging subsystem includes:
> (1) debugging subsystem API;
> (2) subsystems and printing opportunity flags;
> (3) kernel configuration options.
> 
> The debugging subsystem API includes:
> (1) nilfs2_debug() method.
>     It is a basic debugging method. It expects a set of
>     flags, format string and variable arguments list.
> 
> (2) nilfs2_hexdump() method.
>     It is a method for writing in system log hexdumps
>     of internal structures. It expects a set of flags,
>     prefix string, pointer on memory and size of pointed data.
> 
> (3) NILFS_ERR_DBG() macros.
>     It is a macros that prints out file name, line number,
>     function name and error code.
> 
> Every module of NILFS2 driver has dedicated flag. These flags
> give opportunity to enable debugging output only from required
> modules of NILFS2 driver. There are several special output
> flags:
> (1) DBG_SHOW_ERR flag.
>     This flag requests writing in system log messages about
>     internal errors of NILFS2 driver.
> 
> (2) DBG_DUMP_STACK flag.
>     This flag requests output of dump stack.
> 
> (3) DBG_SPAM flag.
>     This flag requsts output from frequently called functions
>     or detailed debugging output from function's body.
> 
> (4) DBG_HEX_DUMP flag.
>     This flag requests writing in system log hexdumps of internal
>     structures.
> 
> The debugging subsystem of NILFS2 driver has DBG_MASK that defines
> set of flags for debugging output. Namely, this mask is used for
> comparing requested flag with set of flags that should be printed out.
> Such selection mechanism is used for the case of printk() calls.
> 
> Also, it is available opportunity of dynamic debugging by means of
> pr_debug() method using. In such case the behaviour of pr_debug() is
> controlled via writing to a control file in the 'debugfs' filesystem.
> 
> The debugging subsystem is configured by means of kernel
> configuration options:
> (1) NILFS2_USE_PR_DEBUG
>     This option enables using of pr_debug() instead of printk() and
>     print_hex_dump_bytes() instead of print_hex_dump().
> 
> (2) NILFS2_DEBUG_SHOW_ERRORS
>     This option enables writing in system log messages about internal
>     errors of NILFS2 driver.
> 
> (3) NILFS2_DEBUG_DUMP_STACK
>     This option enables output of dump stack. Usually, every
>     function in NILFS2 driver begins from debugging output of
>     function name, file, line and input arguments' value.
>     In the case of enabling this option debugging output
>     will include dump stack too.
> 
> (4) NILFS2_DEBUG_SHOW_SPAM
>     This option enables output from frequently called functions or
>     detailed debugging output from function's body.
> 
> (5) NILFS2_DEBUG_HEXDUMP
>     This option enables writing in system log hexdumps of internal
>     structures.
> 
> (6) NILFS2_DEBUG_BASE_OPERATIONS
>     This option enables base operations subsystem debugging output.
> 
> (7) NILFS2_DEBUG_MDT_FILES
>     This option enables metadata (MDT) files subsystem debugging output.
> 
> (8) NILFS2_DEBUG_SEGMENTS_SUBSYSTEM
>     This option enables segments subsystem debugging output.
> 
> (9) NILFS2_DEBUG_GC_SUBSYSTEM
>     This option enables GC subsystem debugging output.
> 
> (10) NILFS2_DEBUG_RECOVERY_SUBSYSTEM
>      This option enables recovery subsystem debugging output.
> 
> (11) NILFS2_DEBUG_BLOCK_MAPPING
>      This option enables block mapping subsystem debugging output.
> 
> (12) NILFS2_DEBUG_BUFFER_MANAGEMENT
>      This option enables buffer/page management subsystem debugging
>      output.
> 
> With the best regards,
> Vyacheslav Dubeyko.

Looks like you are trying to reinvent the wheel.

Please consider using trace events kernel API (See
Documentation/trace/events.txt and include/trace/events/xxxx.h etc),
and do not try to add own debug/tracing framework.  With the trace
events framekwork, you will be able to add flexible and switchable
debug functionalities without abusing NILFS2_DEBUG_XXXX kernel options
and printk variants.

As I usually comment to you, start small, with a simple but powerful
change, and try to keep your patchset as simple as you can.

Regards,
Ryusuke Konishi

> ---
>  Vyacheslav Dubeyko (1):
>     nilfs2: introduce debugging subsystem implementation
> 
>  fs/nilfs2/Kconfig     |  145 +++++++++++++
>  fs/nilfs2/alloc.c     |  140 +++++++++++--
>  fs/nilfs2/bmap.c      |  109 +++++++++-
>  fs/nilfs2/btnode.c    |   45 +++-
>  fs/nilfs2/btree.c     |  515 +++++++++++++++++++++++++++++++++++++++++-----
>  fs/nilfs2/cpfile.c    |  204 +++++++++++++++----
>  fs/nilfs2/dat.c       |  130 ++++++++++--
>  fs/nilfs2/debug.h     |  291 ++++++++++++++++++++++++++
>  fs/nilfs2/dir.c       |   71 ++++++-
>  fs/nilfs2/direct.c    |  117 +++++++++--
>  fs/nilfs2/file.c      |   29 ++-
>  fs/nilfs2/gcinode.c   |   23 ++-
>  fs/nilfs2/ifile.c     |   55 ++++-
>  fs/nilfs2/inode.c     |  271 +++++++++++++++++++++----
>  fs/nilfs2/ioctl.c     |  275 +++++++++++++++++++------
>  fs/nilfs2/mdt.c       |  128 +++++++++---
>  fs/nilfs2/namei.c     |  183 +++++++++++++----
>  fs/nilfs2/nilfs.h     |    1 +
>  fs/nilfs2/page.c      |   52 ++++-
>  fs/nilfs2/recovery.c  |  160 ++++++++++++---
>  fs/nilfs2/segbuf.c    |   90 +++++++-
>  fs/nilfs2/segment.c   |  541 +++++++++++++++++++++++++++++++++++++++++++------
>  fs/nilfs2/sufile.c    |  172 +++++++++++++---
>  fs/nilfs2/super.c     |  192 ++++++++++++++----
>  fs/nilfs2/the_nilfs.c |  130 +++++++++---
>  25 files changed, 3537 insertions(+), 532 deletions(-)
> --
> 1.7.9.5
> 
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 00/24] nilfs2: introduce debugging subsystem implementation

[Vyacheslav Dubeyko]

Hi Ryusuke,

On Tue, 2013-06-18 at 02:02 +0900, Ryusuke Konishi wrote:

[snip]
> 
> Looks like you are trying to reinvent the wheel.
> 

I don't think so. :) But trying to reinvent the wheel is in background
of the technical progress at whole, from my viewpoint. And old way
doesn't mean bad way. :)

> Please consider using trace events kernel API (See
> Documentation/trace/events.txt and include/trace/events/xxxx.h etc),
> and do not try to add own debug/tracing framework.  With the trace
> events framekwork, you will be able to add flexible and switchable
> debug functionalities without abusing NILFS2_DEBUG_XXXX kernel options
> and printk variants.
> 

So, I suppose that I need to describe my vision and my intentions in
more details.

Yes, I agree that tracepoints framework should be used. But I think that
tracepoints framework is more suitable for performance analysis than for
investigation of an issue by means of debugging. Debugging and
tracepoints frameworks have different goals, from my understanding. So,
I think that NILFS2 should have as debugging as tracepoints
opportunities. The debugging framework can be used for gathering
information about an issue's environment on reporters' side and for deep
issue investigation. The tracepoint framework can be used for
performance analysis. And, moreover, I am thinking about adding
tracepoints into NILFS2 and I am going to do it.

The main goal of this patch set is to provide a simple way to gather
information about the issue on reporter's side. In my current approach
it is possible to ask a reporter about collecting additional details by
means of such simple steps: (1) Enable kernel configuration options for
necessary subsystems and debugging output opportunities; (2) Rebuild the
kernel; (3) Restart kernel and reproduce the issue; (4) Send content of
system log by e-mail. I think that such way is really simple and
efficient.

Ok, you dislike the way of using kernel configuration options for
subsystems and using printk variants. One of the suggested way of
debugging output in this patch set is using
pr_debug()/print_hex_dump_bytes() methods. The behaviour of these
methods are controlled via writing to a control file in the 'debugfs'
filesystem. So, you can configure in very flexible way what will be
printed out (please, see Documentation/dynamic-debug-howto.txt). As a
result, it is possible to use only pr_debug()/print_hex_dump_bytes()
instead of printk() variants. Moreover, most of the suggested kernel
configuration options will be not necessary in such case.

This patch set suggests such debugging output opportunities: (1) output
with information about internal errors; (2) output with current values
of variables/function arguments; (3) output with hexdumps of on-disk or
in-core data structures; (4) dump stack output.

I think that information about internal errors that takes place in
NILFS2 driver is very important output. Such information can provide a
clue for understanding a possible reason of an issue and a way of
further investigation. But information about internal errors is
preliminary and rough knowledge that can be used for localization of the
error. So, from my point of view, it doesn't make sense to restrict such
output by any subsystem. As a result, I use simple printk() for this
output.

Debugging output and hexdump output are very important information also.
These outputs should be made by pr_debug()/print_hex_dump_bytes(), from
my point of view. I think that dump stack can be useful information in
some situations. But if it will be used only dynamic version of printk()
then flags of for subsystems will be unnecessary. As a result, there
isn't opportunity for restricting of the dump stack output.

So, as a resume, I think that NILFS2 needs in debugging subsystem and
tracepoints subsystem. It is a complementary subsystems, from my
viewpoint.

With the best regards,
Vyacheslav Dubeyko.

> As I usually comment to you, start small, with a simple but powerful
> change, and try to keep your patchset as simple as you can.

> Regards,
> Ryusuke Konishi
> 


--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html