[RFC PATCH] Reduce noise about missing devices.

[RFC PATCH] Reduce noise about missing devices.

[Petr Rockai]

Hi,

this is not intended to be directly applied, but more to start a
discussion. One problem with our handling of missing devices is the huge
amount of log noise each missing device produces. In a single run of
lvconvert --repair, I get multiple screens of "read error" and "Couldn't
find device with uuid"...

The attached patch reduces that noise by only reporting these things
once per run. I have used two different approaches for read errors and
for uuid's. Neither is perfect...

Another possible approach (one that I would probably prefer) is to add a
global log_error_once to log.c that would maintain a dm_hash_table of
error messages and downgrade already-reported errors to log_debug or
log_very_verbose automatically. A few error paths could use this.

The downside of all these approaches is that this is sort of global
knowledge, but I guess that's hard to avoid. Presumably, adding a call
to erase the hashtable to _init_logging would solve most of the problems
with this. I suspect that threading a toolcontext pointer through
everything so it could be passed to logging functions is not a viable
option.

About the device read errors, I am ambivalent what exactly to report as
log_error. It makes sense to only report first error per device, but it
also makes sense to report all unique errors per device. The global
log_error_once approach would give the latter, which I think may be
actually preferable.

Opinions?

Yours,
   Petr.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: reduce-dup-errors.diff
Type: text/x-diff
Size: 3559 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/lvm-devel/attachments/20100427/824c0fca/attachment.bin>

[RFC PATCH] Reduce noise about missing devices.

[Petr Rockai]

Hi,

Petr Rockai <prockai@redhat.com> writes:
> Another possible approach (one that I would probably prefer) is to add a
> global log_error_once to log.c that would maintain a dm_hash_table of
> error messages and downgrade already-reported errors to log_debug or
> log_very_verbose automatically. A few error paths could use this.
here's a patch that implements this. I think this is fairly reasonable
and could be checked in. Some minor function renaming might be
necessary.

Yours,
    Petr.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: log_error_once.diff
Type: text/x-diff
Size: 5980 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/lvm-devel/attachments/20100428/a4a1610f/attachment.bin>

[RFC PATCH] Reduce noise about missing devices.

[Alasdair G Kergon]

On Tue, Apr 27, 2010 at 11:32:06PM +0200, Peter Rockai wrote:
> this is not intended to be directly applied, but more to start a
> discussion. One problem with our handling of missing devices is the huge
> amount of log noise each missing device produces. In a single run of
> lvconvert --repair, I get multiple screens of "read error" and "Couldn't
> find device with uuid"...
 
Well my first question before suppressing messages would be: Are all the
repeated accesses that generate the messages actually necessary?
Or are the messages just a symptom of other problems which should be
addressed instead?

Alasdair

[RFC PATCH] Reduce noise about missing devices.

[Takahiro Yasui]

On 05/05/10 15:01, Alasdair G Kergon wrote:
> On Tue, Apr 27, 2010 at 11:32:06PM +0200, Peter Rockai wrote:
>> this is not intended to be directly applied, but more to start a
>> discussion. One problem with our handling of missing devices is the huge
>> amount of log noise each missing device produces. In a single run of
>> lvconvert --repair, I get multiple screens of "read error" and "Couldn't
>> find device with uuid"...
>  
> Well my first question before suppressing messages would be: Are all the
> repeated accesses that generate the messages actually necessary?
> Or are the messages just a symptom of other problems which should be
> addressed instead?

I checked how many times a broken device was accessed by the lvconvert
command when 'lvconvert --repair' is executed using the following lvm
environment:

  VG: 3PVs
  LV: mirror composed of 2 legs and 1 disk log

  vg00-lv00 (253:3)
   |-vg00-lv00_mimage_1 (253:2)
   |  `- (8:48)
   |-vg00-lv00_mimage_0 (253:1)
   |  `- (8:32)
   `-vg00-lv00_mlog (253:0)
      `- (8:64)

When a primary leg is broken, 'lvconvert --repair' accesses the primary
leg by '47 times.' I think what we should do is to suppress 'access to
a broken device' instead of suppressing 'displaying error messages.'

Suppress error messages might be friendly to some users, but other users
might not notice how often a broken device is accessed by the lvconvert
command.

Do we need to access a broken device so many times? If lvm commands
detects errors on a device a few times, my opinion is that lvm
commands should quit accessing the device.

Thanks,
Taka

[RFC PATCH] Reduce noise about missing devices.

[Alasdair G Kergon]

On Thu, May 06, 2010 at 07:42:47PM -0400, Takahiro Yasui wrote:
> Do we need to access a broken device so many times? If lvm commands
> detects errors on a device a few times, my opinion is that lvm
> commands should quit accessing the device.
 
Agree.

Alasdair