From: NeilBrown <neilb@suse.de>
To: Marco Schindler <marco.schindler@gmail.com>
Cc: linux-raid@vger.kernel.org
Subject: Re: frequent disk activity with mdadm-3.3
Date: Thu, 18 Sep 2014 20:03:47 +1000 [thread overview]
Message-ID: <20140918200347.762845bb@notabene.brown> (raw)
In-Reply-To: <DDAF4EEF-EFA9-4B0E-BB97-E5D8B9788A70@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 2938 bytes --]
On Mon, 15 Sep 2014 12:52:07 +0200 Marco Schindler
<marco.schindler@gmail.com> wrote:
>
> On 15.09.2014, at 02:18, NeilBrown <neilb@suse.de> wrote:
>
> > It would help to get "udevadm monitor" info to correlate with this.
> > Presumably some uevent is generated when the spindown happens. udev might
> > respond to this by reading from the device, which defeats the purpose...
> >
> >
> >>
> >> here’s the output of blktrace -d /dev/sda during that time.
> >> https://dl.dropboxusercontent.com/u/3464720/blktrace.tar.bz2
> >
> > That suggest that something is reading the metadata from the device almost
> > constantly. Mostly a 'kworker' thread. I don't know what would cause that.
> >
> > Let's look at the 'udevadm monitor' trace first and see what that shows.
>
> here’s the output of udevadm monitor during the spindown cycle while mdadm-3.3 is installed.
>
> monitor will print the received events for:
> UDEV - the event which udev sends out after rule processing
> KERNEL - the kernel uevent
>
> KERNEL[299719.336261] change /devices/pci0000:00/0000:00:06.0/0000:09:00.0/host0/port-0:1/end_device-0:1/target0:0:1/0:0:1:0/block/sdb (block)
> UDEV [299720.646760] change /devices/pci0000:00/0000:00:06.0/0000:09:00.0/host0/port-0:1/end_device-0:1/target0:0:1/0:0:1:0/block/sdb (block)
> KERNEL[299780.202901] change /devices/pci0000:00/0000:00:06.0/0000:09:00.0/host0/port-0:0/end_device-0:0/target0:0:0/0:0:0:0/block/sda (block)
> UDEV [299781.567308] change /devices/pci0000:00/0000:00:06.0/0000:09:00.0/host0/port-0:0/end_device-0:0/target0:0:0/0:0:0:0/block/sda (block)
> KERNEL[299841.090818] change /devices/pci0000:00/0000:00:06.0/0000:09:00.0/host0/port-0:1/end_device-0:1/target0:0:1/0:0:1:0/block/sdb (block)
> UDEV [299842.407035] change /devices/pci0000:00/0000:00:06.0/0000:09:00.0/host0/port-0:1/end_device-0:1/target0:0:1/0:0:1:0/block/sdb (block)
>
> please note that the issue immediately disappears when downgrading to mdadm-3.2 without touching anything else.
> I see the udev rules have been updated in mdadm-3.3..
Getting a "change" even on spindown is causing the problem I suspect.
A change in 3.3.1 causes "mdadm -I" to be run on a device when it 'changes'.
That will read from the device which will wake it up.
(commit 25392f5fc59f96fb76 - revert it and the symptom will probably go away).
I really think the "bug" here is that the change event is emitted on
'spindown', but maybe the bug is that the exact meaning of 'change' isn't
well documented.
I can probably get "mdadm -I" to use O_EXCL which will fail on devices
already in an array, but I'm not sure that is a complete solution. You could
still get wakeups on other devices.
Can you rung the 'udevadm monitor' again, but this time with '--property'.
Maybe there is some property associated with spindown events which we can use
to ignore them.
NeilBrown
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]
next prev parent reply other threads:[~2014-09-18 10:03 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-09-11 22:05 frequent disk activity with mdadm-3.3 Marco Schindler
2014-09-11 22:24 ` NeilBrown
2014-09-11 22:45 ` Marco Schindler
2014-09-12 12:15 ` Marco Schindler
2014-09-15 0:18 ` NeilBrown
2014-09-15 10:52 ` Marco Schindler
2014-09-18 10:03 ` NeilBrown [this message]
2014-09-18 10:38 ` Marco Schindler
2014-09-18 11:09 ` NeilBrown
2014-09-18 11:22 ` Marco Schindler
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20140918200347.762845bb@notabene.brown \
--to=neilb@suse.de \
--cc=linux-raid@vger.kernel.org \
--cc=marco.schindler@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).