Re: Fwd: (user) Help needed: mdadm seems to constantly touch my disks

[NeilBrown <neilb@suse.com>]

[-- Attachment #1: Type: text/plain, Size: 2296 bytes --]

On Wed, Dec 21 2016, Jure Erznožnik wrote:

> Mr Brown,
>
> Let me begin with: please give me your paypal address or something so
> that I can at least buy you a beer or something

Thanks :-)
My first inclination is to say "no thanks" as I am very adequately
compensated by SUSE, and it is part of my role at SUSE to ensure the
upstream kernel remains healthy.  Encouraging a health community is part
of that (and I often learn something while helping people fix things).

But my second inclination is to recognize that gratitude is an important
part of human interactions, and that a community is strong when
gratitude is appropriately given and received.  It is not my place to
direct others how they should show gratitude.
So I'll tell you my paypal address is neil@brown.name and that I'm more
likely to enjoy hot chocolate than beer, but I'll also emphasize that
there is no expectation attached to this information. :-)

>
>
> Your analysis and discovery that iSCSI is the origin of writes got me
> thinking: how can he see that on md0 device if that device has two
> more layers (bcache + LVM) before iSCSI even comes into play. Maybe
> the system propagates the origin down the block devices or something,
> totally not relevant here. So I embarked on a journey of total data
> destruction by disabling one layer at a time. I started by simply
> detaching bcache as that was the first thing on the list - and was
> non-destructive to boot :)
>
> I have found the culprit:
> It is bcache that does the one second writes. I have yet to find the
> exact parameters that influence this behaviour, but the output of
> writeback_rate_debug is EXTREMELY clear: it's writing a bit of data
> each second, reducing the dirty cache by that tiny amount. This is
> what causes the write "amplification" resulting in clicks long after a
> write has been done - because bcache only writes tiny amounts each
> second instead of flushing the entire cache at once when the time
> comes.

Now that we have an understanding of what is happening, I can recommend
that you increase /sys/block/md0/md/safe_mode_delay.  It is measured in
seconds.  If you make it larger than the period of the bcache writes, it
should stop the 'ticking' you mentioned.

NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]