* [GIT PULL] MD update for 4.8
From: Shaohua Li @ 2016-07-28 17:36 UTC (permalink / raw)
To: torvalds; +Cc: linux-kernel, linux-raid, neilb
Hi,
Please pull MD update for 4.8. The patches are based on 4.7 rc initially, I
merged them to lastest upstream to fix merge conflict against block layer
changes:
- A bunch of patches from Neil Brown to fix RCU usage
- Two performance improvement patches from Tomasz Majchrzak
- Alexey Obitotskiy fixes module refcount issue
- Arnd Bergmann fixes time granularity
- Cong Wang fixes a list corruption issue
- Guoqing Jiang fixes a deadlock in md-cluster
- A null pointer deference fix from me
- Song Liu fixes misuse of raid6 rmw
- Other trival/cleanup fixes from Guoqing Jiang and Xiao Ni
Thanks,
Shaohua
The following changes since commit 194dc870a5890e855ecffb30f3b80ba7c88f96d6:
Add braces to avoid "ambiguous ‘else’" compiler warnings (2016-07-27 20:03:31 -0700)
are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/shli/md.git for-linus
for you to fetch changes up to 3f35e210ed4617a68b6baa9b7ac6c72bf7e313d9:
Merge branch 'mymd/for-next' into mymd/for-linus (2016-07-28 09:34:14 -0700)
----------------------------------------------------------------
Alexey Obitotskiy (1):
Fix kernel module refcount handling
Arnd Bergmann (1):
md: use seconds granularity for error logging
Cong Wang (1):
md: use a mutex to protect a global list
Guoqing Jiang (2):
md-cluster: fix deadlock issue when add disk to an recoverying array
md: simplify the code with md_kick_rdev_from_array
NeilBrown (17):
md: disconnect device from personality before trying to remove it.
md/raid1, raid10: don't recheck "Faulty" flag in read-balance.
md/raid10: fix refounct imbalance when resyncing an array with a replacement device.
md/raid10: add rcu protection in raid10_status.
md/raid10: add rcu protection to rdev access in raid10_sync_request.
md/raid10: add rcu protection to rdev access during reshape.
md/raid10: minor code improvement in fix_read_error()
md/raid10: simplify print_conf a little.
md/raid1: small cleanup in raid1_end_read/write_request
md/raid1: small code cleanup in end_sync_write
md/raid1: add rcu protection to rdev in fix_read_error
md/raid5: add rcu protection to rdev accesses in handle_failed_sync.
md/raid5: add rcu protection to rdev accesses in want_replace
md/raid5: add rcu protection to rdev accesses in raid5_status.
md/multipath: add rcu protection to rdev access in multipath_status.
md: be extra careful not to take a reference to a Faulty device.
md: reduce the number of synchronize_rcu() calls when multiple devices fail.
Shaohua Li (2):
MD: fix null pointer deference
Merge branch 'mymd/for-next' into mymd/for-linus
Song Liu (1):
right meaning of PARITY_ENABLE_RMW and PARITY_PREFER_RMW
Tomasz Majchrzak (3):
raid1/raid10: slow down resync if there is non-resync activity pending
md: add missing sysfs_notify on array_state update
raid10: improve random reads performance
Xiao Ni (1):
MD:Update superblock when err == 0 in size_store
drivers/md/md.c | 74 +++++++++++----
drivers/md/md.h | 10 +-
drivers/md/multipath.c | 29 +++---
drivers/md/raid1.c | 130 ++++++++++++-------------
drivers/md/raid10.c | 250 ++++++++++++++++++++++++++++++-------------------
drivers/md/raid10.h | 3 +-
drivers/md/raid5.c | 45 +++++----
7 files changed, 328 insertions(+), 213 deletions(-)
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: RAID5 Performance
From: Peter Grandi @ 2016-07-28 17:45 UTC (permalink / raw)
To: Linux RAID
In-Reply-To: <0eee845d-b833-0a92-8fe6-a2522218a0e6@websitemanagers.com.au>
[ ... ]
> My concern is that even if I solve *this* bottleneck (ie, the
> 530 model SSD being too busy), that there will be another
> bottleneck afterwards
There is always another bottleneck ;-).
[ ... ]
> I'm not sure, but I think I've had one of the 480GB drives
> fail, and 3 of the smaller 60GB and 80GB drives fail. So far,
> only the 480G failure was "catastrophic", the others were
> still operating . All were replaced by Intel.
[ ... ]
When flash SSD drives run over their "expected" write amount
they behave differently:
http://www.sabi.co.uk/blog/15-one.html#150406b
Intel flash SSDs apparently do the following:
* They switch immediately to read-only.
* On the next power up they refuse even to *read*.
BTW, as to the Samsung SM863 there is a relatively recent "group
test" here:
http://www.storagereview.com/samsung_sm863_ssd_review
The average and max latency graphs are interesting, especially
those under "Preconditioning Curve".
^ permalink raw reply
* Re: RAID5 Performance
From: Doug Dumitru @ 2016-07-28 18:45 UTC (permalink / raw)
To: Peter Grandi; +Cc: Linux RAID
In-Reply-To: <22426.16080.579831.750402@tree.ty.sabi.co.uk>
... boy this thread is getting long.
A couple of points.
* I am one of a reasonably small group of people that have actually
written an FTL and have it in production use. FTLs in SSDs are some
of the most closely guarded "implementations" I have ever seen. I am
not sure if my FTL matches others, as I have not seen the others, but
the patent office thinks my version is unique enough (not that it
really matters).
* Many SSDs, even consumer models, and even models without battery
backup, usually enforce correct serialization. If you have a single
drive in a laptop, this is what is important. If you have an array in
a server, and if the power to the SSDs are protected, then this also
protects your data. You need to separate the failures you are trying
to protect. SuperCaps on an SSD that is behind redundant power
supplies on redundant UPSs is perhaps not the best place to spend your
money. Likewise, if you have an HA link, the write is not ACKed until
the other node gets the data at least into the memory buffer. Are you
trying to engineer against multiple failures at multiple sites. You
need to decide on the level of redundancy of redundancy of redundancy.
* Assumptions that FTLs wear sync writes at the ratio of the write
block to the erase block sizes are usually wrong. Older "dumb" flash
like CF and SD cards sometimes work like this, but even there they
have gotten better. An easier assumption is that "normal" SSDs will
have write amplification at the inverse of free space percentage. So
your consumer drive with 8% free has 1/.08=12.5:1 write amplification.
This is why data center drives have more free space. "Better" FTLs,
when working with 100% random workloads can lower this to just over
50% of this value. My FTL sees 5.45:1 write amp on a 100% random
workload steady state at 10% free.
* Real workloads are sometimes that same as random workloads and
sometimes very different. Some FTLs can exploit the patterns in a
real file system workload and some cannot. For example, my FTL sees
1.3:1 write amp with the JEDEC 128GB client trace at 10% free versus
the typical 9:1 for most consumer SSDs on the same trace with about
the same free space.
* Additional games are possible if you start to reach "into" the
blocks. Compression that only saves you 10% might not seem like much,
but if it moves the free space from 8% to 18%, it matters a lot. The
above examples were without compression. Compression can also make
file system write overhead "go away". It is not uncommon for a
journal and directory entry write to compress 80+% even though the
data is binary. This makes old hard drive optimizations like
"-noatime" unnecessary.
* You can do a lot more with an FTL if you move it "in front" of raid.
This basically eliminates the raid read/modify/write operation and
overhead entirely. It does introduce a new "write hole" aspect to the
array, but this can be plugged with nvRAM hardware.
Happy Hunting.
Doug Dumitru
^ permalink raw reply
* Re: SOLVED [was Re: GPT corruption on Primary Header, backup OK, fixing primary nuked array -- help?]
From: Andreas Dröscher @ 2016-07-28 20:51 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <0d00162b-6726-924a-a84d-016871563d09@youngman.org.uk>
[-- Attachment #1: Type: text/plain, Size: 2325 bytes --]
Am 28.07.16 um 14:53 schrieb Anthony Youngman:
> On 28/07/16 00:10, David C. Rankin wrote:
>> On 07/27/2016 08:04 AM, Anthony Youngman wrote:
>>> WD Blacks? Do they support SCT/ERC? I think these are desktop drives (like my
>>> Barracudas) so you WILL get bitten by the timeout problem if anything goes
>>> wrong. Do you know what you're doing here?
>> Yes, WD Blacks, and yes, at least for the last 16 years I've managed, somehow,
>> to provide a complete open-source backend for my law office. So I would answer
>> the 2nd question in the affirmative as well. You can poo-poo drive X verses
>> drive Y all you want, but I get a consistent 5 years out of each WD black and
>> plan on a replacement cycle of 1/2 that. Go with what works for you.
>>
> I'll just say I don't think the past 16 years is a good guide at all ... (but I
> will add I'm doing exactly the same as you - two 3TB desktop drives in a mirror
> :-).
>
> The timeout problem seems to be relatively recent. MOST 1TB or less drives don't
> seem to have an issue. It's bigger drives that will bite you.
>
All drives are tailored to a use case: price, power consumption (e.g. WD Green),
desktop performance (WD Black) and Raid (WD Red or WD Enterprise Storage - also
black label). One of the key feature of raid drives is TLER (Time Limited error
recovery). Note: the name my vary by brand.
My WD-ES drive shows:
smartctl -l scterc /dev/sda
smartctl 6.6 2016-05-07 r4319 [...] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org
SCT Error Recovery Control:
Read: 70 (7.0 seconds)
Write: 70 (7.0 seconds)
server ~ #
That means that a drive will report a media error after 7 seconds and leave it
to the raid controller / raid subsystem to recover it. Linux-Raid usually
re-covers and re-writes the bad block from the remaining drives, fixing the
issue (The drives firmware relocates the sector).
Non raid optimized drives may spend a long time trying to recover such a sector.
Hence the raid controller will not simply fix the sector but fail the entire
drive for not responding. For this reason, an array can fail, that would have
not with proper.
The issue can be relaxed by tuning SCT or /sys/block/sda/device/timeout.
- Andreas
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3828 bytes --]
^ permalink raw reply
* Re: SOLVED [was Re: GPT corruption on Primary Header, backup OK, fixing primary nuked array -- help?]
From: Phil Turmel @ 2016-07-28 21:25 UTC (permalink / raw)
To: Anthony Youngman, David C. Rankin, mdraid
In-Reply-To: <0d00162b-6726-924a-a84d-016871563d09@youngman.org.uk>
On 07/28/2016 08:53 AM, Anthony Youngman wrote:
> I'll just say I don't think the past 16 years is a good guide at all ...
> (but I will add I'm doing exactly the same as you - two 3TB desktop
> drives in a mirror :-).
>
> The timeout problem seems to be relatively recent. MOST 1TB or less
> drives don't seem to have an issue. It's bigger drives that will bite you.
This is not a recent issue. I was first bitten by this in the summer of
2011 when upgrading some Seagate drives from 1T to 2T:
http://marc.info/?l=linux-raid&m=133761065622164&w=2
Prior to this, the problem existed in the sense that SCTERC wasn't
commonly disabled on powerup for desktop drives. More recent desktop
drives don't support it at all.
Run "smartctl -l scterc /dev/sdX" on your drives shortly after power-up.
You should see a short time setting like this:
> SCT Error Recovery Control:
> Read: 70 (7.0 seconds)
> Write: 70 (7.0 seconds
If you see anything else, you will need boot time scripting or udev
scripts that will either enable the above, or reset the drive's kernel
timeout. If you have this problem and fail to script the corrections,
your array will eventually go BOOM, even though your drives aren't
actually failed.
Phil
^ permalink raw reply
* To add, or not to add, a bio REQ_ROTATIONAL flag
From: Eric Wheeler @ 2016-07-29 0:50 UTC (permalink / raw)
To: linux-block; +Cc: dm-devel, linux-raid, linux-kernel, linux-bcache
Hello all,
With the many SSD caching layers being developed (bcache, dm-cache,
dm-writeboost, etc), how could we flag a bio from userspace to indicate
whether the bio is preferred to hit spinning disks instead of an SSD?
Unnecessary promotions, evections, and writeback increase the write burden
on the caching layer and burns out SSDs too fast (TBW), thus requring
equipment replacement.
Is there already a mechanism for this that could be added to the various
caching mechanisms' promote/demote/bypass logic?
For example, I would like to prevent backups from influencing the cache
eviction logic. Neither do I wish to evict cache due to a bio from a
backup process, nor do I wish a bio from the backup process to be cached
on the SSD.
We would want to bypass the cache for IO that is somehow flagged to bypass
block-layer caches and use the rotational disk unless the referenced block
already exists on the SSD.
There might be two cases here that would be ideal to unify without
touching filesystem code:
1) open() of a block device
2) open() on a file such that a filesystem must flag the bio
I had considered writing something to detect FADV_SEQUENTIAL/FADV_NOREUSE
or `ionice -c3` on a process hitting bcache and modifying
check_should_bypass()/should_writeback() to behave as such.
However, just because FADV_SEQUENTIAL is flagged doesn't mean the cache
should bypass. Filesystems can fragment, and while the file being read
may be read sequentially, the blocks on which it resides may not be.
Same thing for higher-level block devices such as dm-thinp where one might
sequentially read a thin volume but its _tdata might not be in linear
order. This may imply that we need a new way to flag cache bypass from
userspace that is neither io-priority nor fadvise driven.
So what are our options? What might be the best way to do this?
If fadvise is the better option, how can a block device driver lookup the
fadvise advice from a given bio struct? Can we add an FADV_NOSSD flag
since FADV_SEQUENTIAL may be insufficent? Are FADV_NOREUSE/FADV_DONTNEED
reasonable candidates?
Perhaps ionice could be used used, but the concept of "priority"
doesn't exactly encompass the concept of cache-bypass---so is something
else needed?
Other ideas?
--
Eric Wheeler
^ permalink raw reply
* Re: To add, or not to add, a bio REQ_ROTATIONAL flag
From: Wols Lists @ 2016-07-29 1:04 UTC (permalink / raw)
To: Eric Wheeler, linux-block
Cc: dm-devel, linux-raid, linux-kernel, linux-bcache
In-Reply-To: <alpine.LRH.2.11.1607281603530.10662@mail.ewheeler.net>
On 29/07/16 01:50, Eric Wheeler wrote:
> Hello all,
>
> With the many SSD caching layers being developed (bcache, dm-cache,
> dm-writeboost, etc), how could we flag a bio from userspace to indicate
> whether the bio is preferred to hit spinning disks instead of an SSD?
>
> Unnecessary promotions, evections, and writeback increase the write burden
> on the caching layer and burns out SSDs too fast (TBW), thus requring
> equipment replacement.
What's the spec of these devices? How long are they expected to last?
Other recent posts on this (linux-raid) mailing list refer to tests on
SSDs that indicates their typical life is way beyond their nominal life,
and that in normal usage they are actually likely to outlive "spinning
rust".
http://techreport.com/review/24841/introducing-the-ssd-endurance-experiment
http://techreport.com/review/27909/the-ssd-endurance-experiment-theyre-all-dead/3
Looking at the results, the FIRST drives only started failing once
they'd written some 700 Terabytes. How long is it going to take you to
write that much data over a SATA3 link?
Cheers,
Wol
^ permalink raw reply
* Re: To add, or not to add, a bio REQ_ROTATIONAL flag
From: Martin K. Petersen @ 2016-07-29 1:16 UTC (permalink / raw)
To: Eric Wheeler
Cc: linux-block, dm-devel, linux-raid, linux-kernel, linux-bcache
In-Reply-To: <alpine.LRH.2.11.1607281603530.10662@mail.ewheeler.net>
>>>>> "Eric" == Eric Wheeler <bcache@lists.ewheeler.net> writes:
Eric,
Eric> However, just because FADV_SEQUENTIAL is flagged doesn't mean the
Eric> cache should bypass. Filesystems can fragment, and while the file
Eric> being read may be read sequentially, the blocks on which it
Eric> resides may not be. Same thing for higher-level block devices
Eric> such as dm-thinp where one might sequentially read a thin volume
Eric> but its _tdata might not be in linear order. This may imply that
Eric> we need a new way to flag cache bypass from userspace that is
Eric> neither io-priority nor fadvise driven.
Why conflate the two? Something being a background task is orthogonal to
whether it is being read sequentially or not.
Eric> So what are our options? What might be the best way to do this?
For the SCSI I/O hints I use the idle I/O priority to classify
backups. Works fine.
Eric> Are FADV_NOREUSE/FADV_DONTNEED reasonable candidates?
FADV_DONTNEED was intended for this. There have been patches posted in
the past that tied the loop between the fadvise flags and the bio. I
would like to see those revived.
Eric> Perhaps ionice could be used used, but the concept of "priority"
Eric> doesn't exactly encompass the concept of cache-bypass---so is
Eric> something else needed?
The idle class explicitly does not have a priority.
--
Martin K. Petersen Oracle Linux Engineering
^ permalink raw reply
* Re: [PATCH] md: Prevent IO hold during accessing to failed raid5 array
From: Obitotskiy, Aleksey @ 2016-07-29 9:07 UTC (permalink / raw)
To: shli@kernel.org; +Cc: linux-raid@vger.kernel.org
In-Reply-To: <20160719224603.GB79792@kernel.org>
Hello,
I would like to know what the status of this patch.
Maybe I should provide more info about?
Regards,
Aleksey
On Tue, 2016-07-19 at 15:46 -0700, Shaohua Li wrote:
> On Fri, Jul 15, 2016 at 03:24:27PM +0200, Alexey Obitotskiy wrote:
> >
> > After array enters in failed state (e.g. number of failed drives
> > becomes more then accepted for raid5 level) it sets error flags
> > (one of this flags is MD_CHANGE_PENDING). This flag prevents to
> > finish all new or non-finished IOs to array and hold them in
> > pending state. In some cases this can leads to deadlock situation.
> >
> > For example udev handle array state changes (drives becomes faulty)
> > and blkid started but unable to finish reads due to IO hold.
> > At the same time we unable to get exclusive access to array
> > (to stop array in our case) because another external application
> > still use this array (blkid in our case).
> >
> > Fix makes possible to return IO with errors immediately.
> > So external application can finish working with array and
> > give exclusive access to other applications.
> >
> > Signed-off-by: Alexey Obitotskiy <aleksey.obitotskiy@intel.com>
> > ---
> > drivers/md/raid5.c | 4 +++-
> > 1 file changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
> > index 6c1149d..99471b6 100644
> > --- a/drivers/md/raid5.c
> > +++ b/drivers/md/raid5.c
> > @@ -4692,7 +4692,9 @@ finish:
> > }
> >
> > if (!bio_list_empty(&s.return_bi)) {
> > - if (test_bit(MD_CHANGE_PENDING, &conf->mddev-
> > >flags)) {
> > + if (test_bit(MD_CHANGE_PENDING, &conf->mddev-
> > >flags) &&
> > + (s.failed <= conf->max_degraded ||
> > + conf->mddev->external ==
> > 0)) {
> > spin_lock_irq(&conf->device_lock);
> > bio_list_merge(&conf->return_bi,
> > &s.return_bi);
> > spin_unlock_irq(&conf->device_lock);
^ permalink raw reply
* Re: To add, or not to add, a bio REQ_ROTATIONAL flag
From: Kai Krakow @ 2016-07-29 16:28 UTC (permalink / raw)
To: linux-bcache; +Cc: linux-raid, linux-block, dm-devel, linux-kernel
In-Reply-To: <579AABAA.2020200@youngman.org.uk>
Am Fri, 29 Jul 2016 02:04:42 +0100
schrieb Wols Lists <antlists@youngman.org.uk>:
> On 29/07/16 01:50, Eric Wheeler wrote:
> > Hello all,
> >
> > With the many SSD caching layers being developed (bcache, dm-cache,
> > dm-writeboost, etc), how could we flag a bio from userspace to
> > indicate whether the bio is preferred to hit spinning disks instead
> > of an SSD?
> >
> > Unnecessary promotions, evections, and writeback increase the write
> > burden on the caching layer and burns out SSDs too fast (TBW), thus
> > requring equipment replacement.
>
> What's the spec of these devices? How long are they expected to last?
>
> Other recent posts on this (linux-raid) mailing list refer to tests on
> SSDs that indicates their typical life is way beyond their nominal
> life, and that in normal usage they are actually likely to outlive
> "spinning rust".
Well, using SSD as a caching layer is probably everything else but
normal usage. Caching involves writing a lot of data given the fact
that it usually backs huge storage pools and is there to eliminate/hide
inefficient usage patterns of rotational media. Caching layers like
bcache do its best to turn writes to the device into optimal write
patterns at best - but still, it writes a lot of data.
> http://techreport.com/review/24841/introducing-the-ssd-endurance-experiment
>
> http://techreport.com/review/27909/the-ssd-endurance-experiment-theyre-all-dead/3
>
> Looking at the results, the FIRST drives only started failing once
> they'd written some 700 Terabytes. How long is it going to take you to
> write that much data over a SATA3 link?
I had a Crucial MX100 128GB SSD as bcache (writeback mode) in the past.
It lasted exactly 12-13 months when I was forced to replace it: The
lifetime of the SSD reached around 95% according to smartctl. The specs
say, it has a lifetime guarantee of 85TB written. I guess it could have
lasted longer due to bcache's optimized access patterns - but I didn't
want to test it. Bigger SSDs usually last a lot longer, so now I'm
using a Samsung 850 Evo 500GB which has a guarantee of 150TBW.
According to tests, it probably goes a lot higher before failing (like
250TBW estimated) but I don't want to test this is writeback mode.
So the idea of reducing writes to the caching layer, or better
eliminate useless writes altogether, would be very welcome.
--
Regards,
Kai
Replies to list-only preferred.
^ permalink raw reply
* xosview
From: Anthony Youngman @ 2016-07-29 16:52 UTC (permalink / raw)
To: mdraid; +Cc: mike.romberg
I don't know how many of you use this ancient nifty utility, but I've
been using it for as long as I can remember. Unfortunately, the raid
monitor code no longer works ... :-(
I emailed the maintainer privately and he was pleased that I'd got in
touch even though it was with the bad news, and he'd like to fix it, but
he has no system with raid to test it on.
It works by parsing /proc/mdstat. From what I can see of the code, it
would be very easy to test it by pointing it at a fake mdstat. I've got
three arrays on a two disk mirror, so that's easy for me to test, but
I'd like to test it on other setups.
So if people wouldn't mind, could you email your mdstat files?
Preferably on the list so people can see what has and has not been sent
- obviously I'd like standard setups like raid10, raid5, raid6, both
named and numbered. And if people have them, mdstats showing broken
arrays, rebuilds, complicated setups with lvm, etc.
Dunno about other people, but I have xosview running on my desktop all
the time (I feel naked without it :-) so if that can give me a
continuous monitor of my raid state that's great. And hopefully, if
other people use it, we might reduce the number of "I didn't realise my
raid was degraded, and then it suffered another failure" type emails.
Cheers,
Wol
^ permalink raw reply
* Re: xosview
From: Glenn Enright @ 2016-07-29 23:23 UTC (permalink / raw)
To: Anthony Youngman; +Cc: mdraid, mike.romberg
In-Reply-To: <ffd06b83-0692-d5a4-1daa-126abf53edfa@youngman.org.uk>
On 30 July 2016 at 04:52, Anthony Youngman <antlists@youngman.org.uk> wrote:
>
> I don't know how many of you use this ancient nifty utility, but I've been using it for as long as I can remember. Unfortunately, the raid monitor code no longer works ... :-(
>
> I emailed the maintainer privately and he was pleased that I'd got in touch even though it was with the bad news, and he'd like to fix it, but he has no system with raid to test it on.
>
> It works by parsing /proc/mdstat. From what I can see of the code, it would be very easy to test it by pointing it at a fake mdstat. I've got three arrays on a two disk mirror, so that's easy for me to test, but I'd like to test it on other setups.
>
> So if people wouldn't mind, could you email your mdstat files? Preferably on the list so people can see what has and has not been sent - obviously I'd like standard setups like raid10, raid5, raid6, both named and numbered. And if people have them, mdstats showing broken arrays, rebuilds, complicated setups with lvm, etc.
>
> Dunno about other people, but I have xosview running on my desktop all the time (I feel naked without it :-) so if that can give me a continuous monitor of my raid state that's great. And hopefully, if other people use it, we might reduce the number of "I didn't realise my raid was degraded, and then it suffered another failure" type emails.
>
> Cheers,
> Wol
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
There are a number of good examples on
https://raid.wiki.kernel.org/index.php/Mdstat
This is from my desktop... (boot root and swap)
$ cat /proc/mdstat
Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5]
[raid4] [raid10]
md1 : active raid1 sdb2[1] sda2[0]
4194240 blocks [2/2] [UU]
md2 : active raid1 sdb3[1] sda3[0]
62914496 blocks [2/2] [UU]
md0 : active raid1 sda1[0] sdb1[1]
204736 blocks [2/2] [UU]
unused devices: <none>
Best
--Glenn
^ permalink raw reply
* Re: [PATCH] md: Prevent IO hold during accessing to failed raid5 array
From: Shaohua Li @ 2016-07-30 21:01 UTC (permalink / raw)
To: Obitotskiy, Aleksey; +Cc: linux-raid@vger.kernel.org
In-Reply-To: <1469783178.2660.21.camel@intel.com>
On Fri, Jul 29, 2016 at 09:07:43AM +0000, Obitotskiy, Aleksey wrote:
> Hello,
>
> I would like to know what the status of this patch.
> Maybe I should provide more info about?
I'm in vacation, so response is slow, sorry. Please reorganize the patch log
and mention this is for external managed array. What's the s.failed <=
conf->max_degraded check for?
Thanks,
Shaohua
>
> Regards,
> Aleksey
>
> On Tue, 2016-07-19 at 15:46 -0700, Shaohua Li wrote:
> > On Fri, Jul 15, 2016 at 03:24:27PM +0200, Alexey Obitotskiy wrote:
> > >
> > > After array enters in failed state (e.g. number of failed drives
> > > becomes more then accepted for raid5 level) it sets error flags
> > > (one of this flags is MD_CHANGE_PENDING). This flag prevents to
> > > finish all new or non-finished IOs to array and hold them in
> > > pending state. In some cases this can leads to deadlock situation.
> > >
> > > For example udev handle array state changes (drives becomes faulty)
> > > and blkid started but unable to finish reads due to IO hold.
> > > At the same time we unable to get exclusive access to array
> > > (to stop array in our case) because another external application
> > > still use this array (blkid in our case).
> > >
> > > Fix makes possible to return IO with errors immediately.
> > > So external application can finish working with array and
> > > give exclusive access to other applications.
> > >
> > > Signed-off-by: Alexey Obitotskiy <aleksey.obitotskiy@intel.com>
> > > ---
> > > drivers/md/raid5.c | 4 +++-
> > > 1 file changed, 3 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
> > > index 6c1149d..99471b6 100644
> > > --- a/drivers/md/raid5.c
> > > +++ b/drivers/md/raid5.c
> > > @@ -4692,7 +4692,9 @@ finish:
> > > }
> > >
> > > if (!bio_list_empty(&s.return_bi)) {
> > > - if (test_bit(MD_CHANGE_PENDING, &conf->mddev-
> > > >flags)) {
> > > + if (test_bit(MD_CHANGE_PENDING, &conf->mddev-
> > > >flags) &&
> > > + (s.failed <= conf->max_degraded ||
> > > + conf->mddev->external ==
> > > 0)) {
> > > spin_lock_irq(&conf->device_lock);
> > > bio_list_merge(&conf->return_bi,
> > > &s.return_bi);
> > > spin_unlock_irq(&conf->device_lock);
^ permalink raw reply
* Re: [PATCH] raid10: increment write counter after bio is split
From: Shaohua Li @ 2016-07-30 21:01 UTC (permalink / raw)
To: Tomasz Majchrzak
Cc: linux-raid, aleksey.obitotskiy, pawel.baldysiak,
artur.paszkiewicz
In-Reply-To: <1469694505-20018-1-git-send-email-tomasz.majchrzak@intel.com>
On Thu, Jul 28, 2016 at 10:28:25AM +0200, Tomasz Majchrzak wrote:
> md pending write counter must be incremented after bio is split,
> otherwise it gets decremented too many times in end bio callback and
> becomes negative.
>
> Signed-off-by: Tomasz Majchrzak <tomasz.majchrzak@intel.com>
> Reviewed-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Applied, thanks!
^ permalink raw reply
* Re: [PATCH] raid5: fix incorrectly counter of conf->empty_inactive_list_nr
From: Shaohua Li @ 2016-07-30 21:02 UTC (permalink / raw)
To: ZhengYuan Liu; +Cc: neilb, liuzhengyuang521, linux-raid, linux-kernel
In-Reply-To: <1469686934-1937-1-git-send-email-liuzhengyuan@kylinos.cn>
On Thu, Jul 28, 2016 at 02:22:14PM +0800, ZhengYuan Liu wrote:
> The counter conf->empty_inactive_list_nr is only used for determine if the
> raid5 is congested which is deal with in function raid5_congested().
> It was increased in get_free_stripe() when conf->inactive_list got to be
> empty and decreased in release_inactive_stripe_list() when splice
> temp_inactive_list to conf->inactive_list. However, this may have a
> problem when raid5_get_active_stripe or stripe_add_to_batch_list was called,
> because these two functions may call list_del_init(&sh->lru) to delete sh from
> "conf->inactive_list + hash" which may cause "conf->inactive_list + hash" to
> be empty when atomic_inc_not_zero(&sh->count) got false. So a check should be
> done at these two point and increase empty_inactive_list_nr accordingly.
> Otherwise the counter may get to be negative number which would influence
> async readahead from VFS.
>
> Signed-off-by: ZhengYuan Liu <liuzhengyuan@kylinos.cn>
Applied, thanks!
^ permalink raw reply
* [PATCH 1/3] MD: hold mddev lock for .quiesce in md_do_sync
From: shli @ 2016-07-30 23:54 UTC (permalink / raw)
To: linux-raid; +Cc: Shaohua Li, NeilBrown
From: Shaohua Li <shli@fb.com>
.quiesce is called with mddev lock hold at most places. There are few
exceptions. Calling .quesce without the lock hold could create races. For
example, the .quesce of raid1 can't be recursively. The purpose of the patches
is to fix a race in raid5-cache. The raid5-cache .quesce will write md
superblock and should be called with mddev lock hold.
Cc: NeilBrown <neilb@suse.com>
Signed-off-by: Shaohua Li <shli@fb.com>
---
drivers/md/md.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/md/md.c b/drivers/md/md.c
index 2c3ab6f..0550445 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -7945,8 +7945,10 @@ void md_do_sync(struct md_thread *thread)
* region.
*/
if (mddev->bitmap) {
+ mddev_lock_nointr(mddev);
mddev->pers->quiesce(mddev, 1);
mddev->pers->quiesce(mddev, 0);
+ mddev_unlock(mddev);
}
}
--
2.7.4
^ permalink raw reply related
* [PATCH 2/3] MD: hold mddev lock to change bitmap location
From: shli @ 2016-07-30 23:54 UTC (permalink / raw)
To: linux-raid; +Cc: Shaohua Li, NeilBrown
In-Reply-To: <515fa68e5c4784b08f2ce99c082c923f6b02a3c9.1469922791.git.shli@fb.com>
From: Shaohua Li <shli@fb.com>
Changing the location changes a lot of things. Holding the lock to avoid race.
This makes the .quiesce called with mddev lock hold too.
Cc: NeilBrown <neilb@suse.com>
Signed-off-by: Shaohua Li <shli@fb.com>
---
drivers/md/bitmap.c | 47 +++++++++++++++++++++++++++++++++--------------
1 file changed, 33 insertions(+), 14 deletions(-)
diff --git a/drivers/md/bitmap.c b/drivers/md/bitmap.c
index 6fff794..13041ee 100644
--- a/drivers/md/bitmap.c
+++ b/drivers/md/bitmap.c
@@ -2183,19 +2183,29 @@ location_show(struct mddev *mddev, char *page)
static ssize_t
location_store(struct mddev *mddev, const char *buf, size_t len)
{
+ int rv;
+ rv = mddev_lock(mddev);
+ if (rv)
+ return rv;
if (mddev->pers) {
- if (!mddev->pers->quiesce)
- return -EBUSY;
- if (mddev->recovery || mddev->sync_thread)
- return -EBUSY;
+ if (!mddev->pers->quiesce) {
+ rv = -EBUSY;
+ goto out;
+ }
+ if (mddev->recovery || mddev->sync_thread) {
+ rv = -EBUSY;
+ goto out;
+ }
}
if (mddev->bitmap || mddev->bitmap_info.file ||
mddev->bitmap_info.offset) {
/* bitmap already configured. Only option is to clear it */
- if (strncmp(buf, "none", 4) != 0)
- return -EBUSY;
+ if (strncmp(buf, "none", 4) != 0) {
+ rv = -EBUSY;
+ goto out;
+ }
if (mddev->pers) {
mddev->pers->quiesce(mddev, 1);
bitmap_destroy(mddev);
@@ -2214,21 +2224,25 @@ location_store(struct mddev *mddev, const char *buf, size_t len)
/* nothing to be done */;
else if (strncmp(buf, "file:", 5) == 0) {
/* Not supported yet */
- return -EINVAL;
+ rv = -EINVAL;
+ goto out;
} else {
- int rv;
if (buf[0] == '+')
rv = kstrtoll(buf+1, 10, &offset);
else
rv = kstrtoll(buf, 10, &offset);
if (rv)
- return rv;
- if (offset == 0)
- return -EINVAL;
+ goto out;
+ if (offset == 0) {
+ rv = -EINVAL;
+ goto out;
+ }
if (mddev->bitmap_info.external == 0 &&
mddev->major_version == 0 &&
- offset != mddev->bitmap_info.default_offset)
- return -EINVAL;
+ offset != mddev->bitmap_info.default_offset) {
+ rv = -EINVAL;
+ goto out;
+ }
mddev->bitmap_info.offset = offset;
if (mddev->pers) {
struct bitmap *bitmap;
@@ -2245,7 +2259,7 @@ location_store(struct mddev *mddev, const char *buf, size_t len)
mddev->pers->quiesce(mddev, 0);
if (rv) {
bitmap_destroy(mddev);
- return rv;
+ goto out;
}
}
}
@@ -2257,6 +2271,11 @@ location_store(struct mddev *mddev, const char *buf, size_t len)
set_bit(MD_CHANGE_DEVS, &mddev->flags);
md_wakeup_thread(mddev->thread);
}
+ rv = 0;
+out:
+ mddev_unlock(mddev);
+ if (rv)
+ return rv;
return len;
}
--
2.7.4
^ permalink raw reply related
* [PATCH 3/3] MD: hold mddev lock for md-cluster receive thread
From: shli @ 2016-07-30 23:54 UTC (permalink / raw)
To: linux-raid; +Cc: Shaohua Li, NeilBrown, Guoqing Jiang
In-Reply-To: <515fa68e5c4784b08f2ce99c082c923f6b02a3c9.1469922791.git.shli@fb.com>
From: Shaohua Li <shli@fb.com>
md-cluster receive thread calls .quiesce too, let it hold mddev lock.
Cc: NeilBrown <neilb@suse.com>
Cc: Guoqing Jiang <gqjiang@suse.com>
Signed-off-by: Shaohua Li <shli@fb.com>
---
drivers/md/md-cluster.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/md/md-cluster.c b/drivers/md/md-cluster.c
index 41573f1..f420060 100644
--- a/drivers/md/md-cluster.c
+++ b/drivers/md/md-cluster.c
@@ -567,11 +567,13 @@ static void recv_daemon(struct md_thread *thread)
struct cluster_msg msg;
int ret;
+ mddev_lock_nointr(thread->mddev);
mutex_lock(&cinfo->recv_mutex);
/*get CR on Message*/
if (dlm_lock_sync(message_lockres, DLM_LOCK_CR)) {
pr_err("md/raid1:failed to get CR on MESSAGE\n");
mutex_unlock(&cinfo->recv_mutex);
+ mddev_unlock(thread->mddev);
return;
}
@@ -599,6 +601,7 @@ static void recv_daemon(struct md_thread *thread)
if (unlikely(ret != 0))
pr_info("unlock msg failed return %d\n", ret);
mutex_unlock(&cinfo->recv_mutex);
+ mddev_unlock(thread->mddev);
}
/* lock_token()
--
2.7.4
^ permalink raw reply related
* Re: [PATCH 1/3] MD: hold mddev lock for .quiesce in md_do_sync
From: yizhan @ 2016-07-31 6:03 UTC (permalink / raw)
To: shli, linux-raid; +Cc: Shaohua Li, NeilBrown
In-Reply-To: <515fa68e5c4784b08f2ce99c082c923f6b02a3c9.1469922791.git.shli@fb.com>
I tested these patch and fixed the bug I reported before[1].
[1] WARNING: CPU: 4 PID: 10512 at drivers/md/raid5-cache.c:728
r5l_do_reclaim+0x415/0x430 [raid456]
Thanks
Yi Zhang
On 07/31/2016 07:54 AM, shli@kernel.org wrote:
> From: Shaohua Li <shli@fb.com>
>
> .quiesce is called with mddev lock hold at most places. There are few
> exceptions. Calling .quesce without the lock hold could create races. For
> example, the .quesce of raid1 can't be recursively. The purpose of the patches
> is to fix a race in raid5-cache. The raid5-cache .quesce will write md
> superblock and should be called with mddev lock hold.
>
> Cc: NeilBrown <neilb@suse.com>
> Signed-off-by: Shaohua Li <shli@fb.com>
> ---
> drivers/md/md.c | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/drivers/md/md.c b/drivers/md/md.c
> index 2c3ab6f..0550445 100644
> --- a/drivers/md/md.c
> +++ b/drivers/md/md.c
> @@ -7945,8 +7945,10 @@ void md_do_sync(struct md_thread *thread)
> * region.
> */
> if (mddev->bitmap) {
> + mddev_lock_nointr(mddev);
> mddev->pers->quiesce(mddev, 1);
> mddev->pers->quiesce(mddev, 0);
> + mddev_unlock(mddev);
> }
> }
>
^ permalink raw reply
* Re: To add, or not to add, a bio REQ_ROTATIONAL flag
From: Eric Wheeler @ 2016-08-01 2:58 UTC (permalink / raw)
To: Martin K. Petersen
Cc: linux-block, dm-devel, linux-raid, linux-kernel, linux-bcache,
hurikhan77, antlists, Dan Williams, Jason B. Akers, Kapil Karkra,
Jens Axboe, Jeff Moyer, david
In-Reply-To: <yq1y44l2qhr.fsf@sermon.lab.mkp.net>
[+cc from "Enable use of Solid State Hybrid Drives"
https://lkml.org/lkml/2014/10/29/698 ]
On Thu, 28 Jul 2016, Martin K. Petersen wrote:
> >>>>> "Eric" == Eric Wheeler <bcache@lists.ewheeler.net> writes:
> Eric> [...] This may imply that
> Eric> we need a new way to flag cache bypass from userspace [...]
> Eric> So what are our options? What might be the best way to do this?
[...]
> Eric> Are FADV_NOREUSE/FADV_DONTNEED reasonable candidates?
>
> FADV_DONTNEED was intended for this. There have been patches posted in
> the past that tied the loop between the fadvise flags and the bio. I
> would like to see those revived.
That sounds like a good start, this looks about right from 2014:
https://lkml.org/lkml/2014/10/29/698
https://lwn.net/Articles/619058/
I read through the thread and have summarized the relevant parts here
with additional commentary below the summary:
/* Summary
They were seeking to do basically the same in 2014 thing we want with
stacked block caching drivers today: hint to the IO layer so the (ATA 3.2)
driver can decide whether a block should hit the cache or spinning disk.
This was done by adding bitflags to ioprio for IOPRIO_ADV_ advice.
There are two arguments throughout the thread: one that the cache hint
should be per-process (ionice) and the other, that hints should be per
inode via fadvise (and maybe madvise). Dan Williams noted with respect to
fadvise for their implementation that "It's straightforward to add, but I
think "80%" of the benefit can be had by just having a per-thread cache
priority."
Kapil Karkra extended the page flags so the ioprio advice bits can be
copied into bio->bi_rw, to which Jens said "is a bit...icky. I see why
it's done, though, it requires the least amount of plumbing."
Martin K. Petersen provides a matrix of desires for actual use cases here:
https://lkml.org/lkml/2014/10/29/1014
and asks "Are there actually people asking for sub-file granularity? I
didn't get any requests for that in the survey I did this summer. [...] In
any case I thought it was interesting that pretty much every use case that
people came up with could be adequately described by a handful of I/O
classes."
Further, Jens notes that "I think we've needed a proper API for passing in
appropriate hints on a per-io basis for a LONG time. [...] We've tried
(and failed) in the past to define a set of hints that make sense. It'd be
a shame to add something that's specific to a given transport/technology.
That said, this set of hints do seem pretty basic and would not
necessarily be a bad place to start. But they are still very specific to
this use case."
*/
So, perhaps it is time to plan the hint API and figure out how to plumb
it. These are some design considerations based on the thread:
a. People want per-process cache hinting (ionice, or some other tool).
b. Per inode+range hinting would be useful to some (fadvise, ioctl, etc)
c. Don't use page flags to convey cache hints---or find a clean way to do so.
d. Per IO hints would be useful to both stacking and hardware drivers.
e. Cache layers will implement their own device assignment choice based
on the caching hint; for example, an IO flagged to miss the cache might
hit if already in cache due to unrelated IO and such a determination would
be made per-cache-implementation.
I can see this go two ways:
1. A dedicated implementation for cache hinting.
2. An API for generalized hinting, upon which cache hinting may be
implemented.
To consider #2, what hinting is wanted from processes and inodes down to
bio's? Does it justify an entire API for generalized hinting, or do we
just need a cache hinting implementation? If we do want #2, then what are
all of the features wanted by the community so it can be designed as such?
If #1 is sufficient, then what is the preferred mechanism and
implementation for cache hinting?
In either direction, how can those hints pass down to bio's in an
appropriate way (ie, not page flags)?
With the interest of a cache hinting implementation independent of
transport/technology, I have been playing with an idea to use two per-IO
"TTL" counters, both of which tend toward zero; I've not yet started an
implementation:
cacheskip:
Decrement until zero to skip cache layers (slow medium)
Ignore cachedepth until cacheskip==0.
cachedepth:
Initialize to positive, negative, or zero value. Once zero, no
special treatment is given to the IO. When less than zero, prefer the
slower medium. When greater than zero, prefer the faster medium.
Inc/decrement toward zero each time the IO passes through a
caching layer.
Independent of how we might apply these counters to a pid/inode, the cache
layers might look something like this:
cachedepth description
0 direct IO
+-1 pagecache
+-2 som arbitrary
+-3 caching
+-4 driver
+-n ...
Layers beyond the pagecache are assigned arbitrarily by the driver
stacking order implemented by the end user. For example, if passing
through dm-cache, then dm-cache would use its own preference logic to
decide whether it should cache or not if cachedepth is zero. If nonzero,
it would cache/bypass appropriately and then inc/decrements cachedepth
toward zero after making its decision. Understandably, extenuating
circumstances may require a layer to ignore the hint---such as a
bypass-hinted IO that gets cached because it is already hot.
Consider the following scenarios for this contrived cache stack:
1. pagecache
2. dm-cache
3. bcache
4. HBA supporting cache hints (ATA 3.2, perhaps)
cacheskip cachedepth description
-------------------------------------------
0 0 use pagecache; lower layers do what they want
1 0 skip pagecache (direct IO); lower layers do what they want
0 -1 same as previous
2 1 skip pagecache, dmcache; prefer bcache-ssd
0 -3 skip pagecache; dmcache bypass; bcache bypass
1 2 skip pagecache; prefer dmcache-ssd, prefer bcache-ssd
3 1 hint to prefer HBA cache only
This would empower the user to decide where caching should begin, and for
how many layers caching should hint for slow(-) or fast(+) backing devices
before letting the IO stack make its own hintless choice. Hopefully this
lets each layer make their own choices that best fit their implementation.
Note that this would not support multi-device tiering as written. If some
layer supports multiple IO performance tiers (more than 2) at the same
layer, then this hinting algorithm is insufficient unless a
cache-layer-specific datastructure could be passed with the IO hinting
request. Also, an eviction hint is not supported by this model.
Please comment with your thoughts. I look forward to feedback and
implementation ideas for what would be the best way to plumb cache hinting
for whatever implementation is chosen.
--
Eric Wheeler
^ permalink raw reply
* Re: [PATCH 3/3] MD: hold mddev lock for md-cluster receive thread
From: Guoqing Jiang @ 2016-08-01 8:38 UTC (permalink / raw)
To: shli, linux-raid; +Cc: Shaohua Li, NeilBrown
In-Reply-To: <7763e508fb97d44bd61e826912055617b8be2c2d.1469922791.git.shli@fb.com>
Hi,
On 07/31/2016 07:54 AM, shli@kernel.org wrote:
> From: Shaohua Li <shli@fb.com>
>
> md-cluster receive thread calls .quiesce too, let it hold mddev lock.
I'd suggest hold on for the patchset, I can find lock problem easily with
the patchset applied. Take a resyncing clusteed raid1 as example.
md127_raid1 thread held reconfig_mutex then update sb, so it needs dlm
token lock. Meanwhile md127_resync thread got token lock and wants
EX on ack lock but recv_daemon can't release ack lock since recv_daemon
doesn't get reconfig_mutex.
etalinux135:~ # ps aux|grep md|grep D
root 2028 0.0 0.0 0 0 ? D 16:24 0:00
[md127_raid1]
root 2041 0.0 0.0 0 0 ? D 16:24 0:00
[md127_resync]
betalinux135:~ # cat /proc/2028/stack
[<ffffffffa05f2660>] metadata_update_start+0xa0/0xb0 [md_cluster]
[<ffffffffa06cb1ce>] md_update_sb.part.50+0x8e/0x810 [md_mod]
[<ffffffffa06ccb8c>] md_check_recovery+0x23c/0x4f0 [md_mod]
[<ffffffffa06f6312>] raid1d+0x42/0x7d0 [raid1]
[<ffffffffa06c6d10>] md_thread+0x130/0x150 [md_mod]
[<ffffffff810995ed>] kthread+0xbd/0xe0
[<ffffffff815e96bf>] ret_from_fork+0x3f/0x70
[<ffffffff81099530>] kthread+0x0/0xe0
[<ffffffffffffffff>] 0xffffffffffffffff
betalinux135:~ # cat /proc/2041/stack
[<ffffffffa05f24cb>] dlm_lock_sync+0x6b/0x80 [md_cluster]
[<ffffffffa05f2708>] __sendmsg+0x98/0x130 [md_cluster]
[<ffffffffa05f282d>] sendmsg+0x1d/0x30 [md_cluster]
[<ffffffffa05f2b31>] resync_info_update+0x81/0xb0 [md_cluster]
[<ffffffffa06f3ad7>] sync_request+0xa57/0xaf0 [raid1]
[<ffffffffa06ca2dd>] md_do_sync+0x90d/0xe80 [md_mod]
[<ffffffffa06c6d10>] md_thread+0x130/0x150 [md_mod]
[<ffffffff810995ed>] kthread+0xbd/0xe0
[<ffffffff815e96bf>] ret_from_fork+0x3f/0x70
[<ffffffff81099530>] kthread+0x0/0xe0
[<ffffffffffffffff>] 0xffffffffffffffff
Thanks,
Guoqing
^ permalink raw reply
* Re: WARNING: CPU: 4 PID: 10512 at drivers/md/raid5-cache.c:728 r5l_do_reclaim+0x415/0x430 [raid456]
From: Guoqing Jiang @ 2016-08-01 9:01 UTC (permalink / raw)
To: Yi Zhang, linux-raid
In-Reply-To: <2099656990.9407283.1469619529240.JavaMail.zimbra@redhat.com>
On 07/27/2016 07:38 AM, Yi Zhang wrote:
> Hello everyone
>
> I'm testing raid5-cache recently and found below issue on 4.7.0-rc7.
>
> [ 902.701162] md: bind<sdb1>
> [ 902.701248] md: bind<sdd1>
> [ 902.702566] md: bind<sde1>
> [ 902.702625] md: bind<sdf1>
> [ 902.703306] md: bind<sdc1>
> [ 902.705899] md/raid:md0: not clean -- starting background reconstruction
> [ 902.705972] md/raid:md0: device sdf1 operational as raid disk 3
> [ 902.705974] md/raid:md0: device sde1 operational as raid disk 2
> [ 902.705975] md/raid:md0: device sdd1 operational as raid disk 1
> [ 902.705976] md/raid:md0: device sdb1 operational as raid disk 0
> [ 902.706312] md/raid:md0: allocated 4374kB
> [ 902.706371] md/raid:md0: raid level 6 active with 4 out of 4 devices, algorithm 2
This is level 6 not 4 as you said in the end.
> [ 902.706372] RAID conf printout:
> [ 902.706380] --- level:6 rd:4 wd:4
> [ 902.706381] disk 0, o:1, dev:sdb1
> [ 902.706382] disk 1, o:1, dev:sdd1
> [ 902.706383] disk 2, o:1, dev:sde1
> [ 902.706384] disk 3, o:1, dev:sdf1
> [ 902.706393] md/raid456: discard support disabled due to uncertainty.
> [ 902.706394] Set raid456.devices_handle_discard_safely=Y to override.
> [ 902.706396] md/raid:md0: using device sdc1 as journal
> [ 902.707354] created bitmap (1 pages) for device md0
> [ 902.707401] md0: bitmap initialized from disk: read 1 pages, set 8 of 8 bits
> [ 902.781331] md0: detected capacity change from 0 to 1071644672
> [ 902.781465] md: resync of RAID array md0
> [ 902.781468] md: minimum _guaranteed_ speed: 1000 KB/sec/disk.
> [ 902.781469] md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for resync.
> [ 902.781475] md: using 128k window, over a total of 523264k.
> [ 902.829618] md: couldn't update array info. -22
> [ 902.876797] md: couldn't update array info. -22
> [ 902.896926] md: couldn't update array info. -22
Errors happened when update array info, not sure it have relationship with
the WARNING or not.
> [ 909.185914] md: md0: resync done.
> [ 909.262779] RAID conf printout:
> [ 909.262782] --- level:6 rd:4 wd:4
> [ 909.262783] disk 0, o:1, dev:sdb1
> [ 909.262784] disk 1, o:1, dev:sdd1
> [ 909.262785] disk 2, o:1, dev:sde1
> [ 909.262786] disk 3, o:1, dev:sdf1
> [ 912.795104] md/raid:md0: Disk failure on sdb1, disabling device.
> md/raid:md0: Operation continuing on 3 devices.
> [ 912.851893] RAID conf printout:
> [ 912.851895] --- level:6 rd:4 wd:3
> [ 912.851897] disk 0, o:0, dev:sdb1
> [ 912.851898] disk 1, o:1, dev:sdd1
> [ 912.851899] disk 2, o:1, dev:sde1
> [ 912.851900] disk 3, o:1, dev:sdf1
> [ 912.855130] RAID conf printout:
> [ 912.855133] --- level:6 rd:4 wd:3
> [ 912.855135] disk 1, o:1, dev:sdd1
> [ 912.855136] disk 2, o:1, dev:sde1
> [ 912.855137] disk 3, o:1, dev:sdf1
> [ 912.885778] md: unbind<sdb1>
> [ 912.894205] md: export_rdev(sdb1)
> [ 1548.098231] md: bind<sdb1>
> [ 1548.452714] RAID conf printout:
> [ 1548.452717] --- level:6 rd:4 wd:3
> [ 1548.452718] disk 0, o:1, dev:sdb1
> [ 1548.452719] disk 1, o:1, dev:sdd1
> [ 1548.452720] disk 2, o:1, dev:sde1
> [ 1548.452721] disk 3, o:1, dev:sdf1
> [ 1548.486067] ------------[ cut here ]------------
> [ 1548.486076] WARNING: CPU: 4 PID: 10512 at drivers/md/raid5-cache.c:728 r5l_do_reclaim+0x415/0x430 [raid456]
> [ 1548.486077] Modules linked in: fuse btrfs vfat msdos fat ext4 jbd2 mbcache binfmt_misc raid456 async_raid6_recov async_memcpy async_pq async_xor xor async_tx raid6_pq xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun ipt_REJECT nf_reject_ipv4 ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sb_edac edac_core snd_hda_codec_realtek snd_hda_codec_hdmi snd_hda_codec_generic x86_pkg_temp_thermal snd_hda_intel intel_powerclamp coretemp s
nd_hda_codec
> [ 1548.486117] kvm_intel kvm snd_hda_core irqbypass crct10dif_pclmul crc32_pclmul snd_hwdep ghash_clmulni_intel snd_seq aesni_intel lrw snd_seq_device gf128mul glue_helper snd_pcm ablk_helper hp_wmi iTCO_wdt snd_timer sparse_keymap rfkill cryptd iTCO_vendor_support mei_me ioatdma snd mei shpchp pcspkr sg soundcore i2c_i801 tpm_infineon lpc_ich mfd_core dca nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod nouveau video mxm_wmi i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm isci e1000e ata_generic libsas pata_acpi ahci libahci scsi_transport_sas firewire_ohci libata firewire_core ptp crc32c_intel serio_raw pps_core i2c_core crc_itu_t wmi fjes dm_mirror dm_region_hash dm_log dm_mod
> [ 1548.486152] CPU: 4 PID: 10512 Comm: md0_resync Tainted: G W 4.7.0-rc7 #1
> [ 1548.486153] Hardware name: Hewlett-Packard HP Z620 Workstation/158A, BIOS J61 v03.69 03/25/2014
> [ 1548.486155] 0000000000000286 000000002369dfe1 ffff88079d713b40 ffffffff8134caec
> [ 1548.486157] 0000000000000000 0000000000000000 ffff88079d713b80 ffffffff8108c351
> [ 1548.486159] 000002d8a6ae4d80 000000008f0c904f ffff8807f64b0000 ffff8807c4ab6888
> [ 1548.486161] Call Trace:
> [ 1548.486166] [<ffffffff8134caec>] dump_stack+0x63/0x87
> [ 1548.486169] [<ffffffff8108c351>] __warn+0xd1/0xf0
> [ 1548.486172] [<ffffffff8108c48d>] warn_slowpath_null+0x1d/0x20
> [ 1548.486175] [<ffffffffa097d805>] r5l_do_reclaim+0x415/0x430 [raid456]
> [ 1548.486178] [<ffffffff811f9c50>] ? kfree+0x120/0x170
> [ 1548.486181] [<ffffffffa097e33b>] r5l_quiesce+0x7b/0xa0 [raid456]
> [ 1548.486183] [<ffffffffa0970da0>] raid5_quiesce+0x50/0x2a0 [raid456]
> [ 1548.486186] [<ffffffff810d0250>] ? prepare_to_wait_event+0xf0/0xf0
> [ 1548.486189] [<ffffffff8156d00e>] md_do_sync+0xe7e/0xf60
> [ 1548.486192] [<ffffffff810c4370>] ? enqueue_entity+0x2a0/0xcc0
> [ 1548.486194] [<ffffffff810c201d>] ? update_curr+0xed/0x180
> [ 1548.486196] [<ffffffff810c026e>] ? account_entity_dequeue+0xae/0xd0
> [ 1548.486197] [<ffffffff810c27f6>] ? dequeue_entity+0x266/0x980
> [ 1548.486200] [<ffffffff810b5ef5>] ? check_preempt_curr+0x75/0x90
> [ 1548.486202] [<ffffffff810b5f29>] ? ttwu_do_wakeup+0x19/0xe0
> [ 1548.486204] [<ffffffff810c2fce>] ? dequeue_task_fair+0xbe/0x830
> [ 1548.486206] [<ffffffff810c3995>] ? put_prev_entity+0x35/0x730
> [ 1548.486209] [<ffffffff8102c6d9>] ? __switch_to+0x219/0x5c0
> [ 1548.486211] [<ffffffff81098d43>] ? kernel_sigaction+0x43/0xe0
> [ 1548.486214] [<ffffffff81566aa6>] md_thread+0x136/0x150
> [ 1548.486216] [<ffffffff81566970>] ? find_pers+0x70/0x70
> [ 1548.486219] [<ffffffff810ab698>] kthread+0xd8/0xf0
> [ 1548.486221] [<ffffffff816e873f>] ret_from_fork+0x1f/0x40
> [ 1548.486223] [<ffffffff810ab5c0>] ? kthread_park+0x60/0x60
> [ 1548.486224] ---[ end trace 242e9516e36acb1e ]---
> [ 1548.543855] md: recovery of RAID array md0
> [ 1548.543857] md: minimum _guaranteed_ speed: 1000 KB/sec/disk.
> [ 1548.543859] md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for recovery.
> [ 1548.543864] md: using 128k window, over a total of 523264k.
> [ 1548.543868] md: md0: recovery done.
> [ 1548.574961] RAID conf printout:
> [ 1548.574964] --- level:6 rd:4 wd:4
> [ 1548.574966] disk 0, o:1, dev:sdb1
> [ 1548.574967] disk 1, o:1, dev:sdd1
> [ 1548.574968] disk 2, o:1, dev:sde1
> [ 1548.574969] disk 3, o:1, dev:sdf1
>
> Steps I used:
>
> mdadm --create --run /dev/md0 --level 4 --metadata 1.2 --raid-devices 4
> /dev/sdb1 /dev/sd[d-f]1 --write-journal /dev/sdc1 --bitmap=internal
> --bitmap-chunk=64M --chunk 512
> mdadm --wait /dev/md0
> mkfs.ext4 /dev/md0
> mdadm /dev/md0 -f /dev/sdb1
> mdadm /dev/md0 -r /dev/sdb1
> mdadm /dev/md0 -a /dev/sdb1
I tried above cmds in my VM machine (with set level as 6), but can't
see the calltrace with below versions. Maybe it is a regression issue,
if so, I would suggest to do bisect to narrow down the issue.
linux135:~ # uname -r
4.4.13-46-default
linux135:~ # mdadm --version
mdadm - v3.4-47-g8718fc3 - 11th May 2016
Thanks,
Guoqing
^ permalink raw reply
* Re: WARNING: CPU: 4 PID: 10512 at drivers/md/raid5-cache.c:728 r5l_do_reclaim+0x415/0x430 [raid456]
From: Yi Zhang @ 2016-08-01 12:32 UTC (permalink / raw)
To: Guoqing Jiang; +Cc: linux-raid
In-Reply-To: <579F0E52.8090704@suse.com>
Best Regards,
Yi Zhang
----- Original Message -----
From: "Guoqing Jiang" <gqjiang@suse.com>
To: "Yi Zhang" <yizhan@redhat.com>, linux-raid@vger.kernel.org
Sent: Monday, August 1, 2016 4:54:42 PM
Subject: Re: WARNING: CPU: 4 PID: 10512 at drivers/md/raid5-cache.c:728 r5l_do_reclaim+0x415/0x430 [raid456]
On 07/27/2016 07:38 PM, Yi Zhang wrote:
> Hello everyone
>
> I'm testing raid5-cache recently and found below issue on 4.7.0-rc7.
>
> [ 902.701162] md: bind<sdb1>
> [ 902.701248] md: bind<sdd1>
> [ 902.702566] md: bind<sde1>
> [ 902.702625] md: bind<sdf1>
> [ 902.703306] md: bind<sdc1>
> [ 902.705899] md/raid:md0: not clean -- starting background reconstruction
> [ 902.705972] md/raid:md0: device sdf1 operational as raid disk 3
> [ 902.705974] md/raid:md0: device sde1 operational as raid disk 2
> [ 902.705975] md/raid:md0: device sdd1 operational as raid disk 1
> [ 902.705976] md/raid:md0: device sdb1 operational as raid disk 0
> [ 902.706312] md/raid:md0: allocated 4374kB
> [ 902.706371] md/raid:md0: raid level*6* active with 4 out of 4 devices, algorithm 2
This is level 6 not 4.
Yi: Sorry for that, but both level 6 and leve4 can be reproduced on my environment.
> [ 902.706372] RAID conf printout:
> [ 902.706380] --- level:6 rd:4 wd:4
> [ 902.706381] disk 0, o:1, dev:sdb1
> [ 902.706382] disk 1, o:1, dev:sdd1
> [ 902.706383] disk 2, o:1, dev:sde1
> [ 902.706384] disk 3, o:1, dev:sdf1
> [ 902.706393] md/raid456: discard support disabled due to uncertainty.
> [ 902.706394] Set raid456.devices_handle_discard_safely=Y to override.
> [ 902.706396] md/raid:md0: using device sdc1 as journal
> [ 902.707354] created bitmap (1 pages) for device md0
> [ 902.707401] md0: bitmap initialized from disk: read 1 pages, set 8 of 8 bits
> [ 902.781331] md0: detected capacity change from 0 to 1071644672
> [ 902.781465] md: resync of RAID array md0
> [ 902.781468] md: minimum _guaranteed_ speed: 1000 KB/sec/disk.
> [ 902.781469] md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for resync.
> [ 902.781475] md: using 128k window, over a total of 523264k.
> [ 902.829618] md: couldn't update array info. -22
> [ 902.876797] md: couldn't update array info. -22
> [ 902.896926] md: couldn't update array info. -22
Errors happened when update array info, not sure it have relationship with
the WARNING or not.
> [ 909.185914] md: md0: resync done.
> [ 909.262779] RAID conf printout:
> [ 909.262782] --- level:6 rd:4 wd:4
> [ 909.262783] disk 0, o:1, dev:sdb1
> [ 909.262784] disk 1, o:1, dev:sdd1
> [ 909.262785] disk 2, o:1, dev:sde1
> [ 909.262786] disk 3, o:1, dev:sdf1
> [ 912.795104] md/raid:md0: Disk failure on sdb1, disabling device.
> md/raid:md0: Operation continuing on 3 devices.
> [ 912.851893] RAID conf printout:
> [ 912.851895] --- level:6 rd:4 wd:3
> [ 912.851897] disk 0, o:0, dev:sdb1
> [ 912.851898] disk 1, o:1, dev:sdd1
> [ 912.851899] disk 2, o:1, dev:sde1
> [ 912.851900] disk 3, o:1, dev:sdf1
> [ 912.855130] RAID conf printout:
> [ 912.855133] --- level:6 rd:4 wd:3
> [ 912.855135] disk 1, o:1, dev:sdd1
> [ 912.855136] disk 2, o:1, dev:sde1
> [ 912.855137] disk 3, o:1, dev:sdf1
> [ 912.885778] md: unbind<sdb1>
> [ 912.894205] md: export_rdev(sdb1)
> [ 1548.098231] md: bind<sdb1>
> [ 1548.452714] RAID conf printout:
> [ 1548.452717] --- level:6 rd:4 wd:3
> [ 1548.452718] disk 0, o:1, dev:sdb1
> [ 1548.452719] disk 1, o:1, dev:sdd1
> [ 1548.452720] disk 2, o:1, dev:sde1
> [ 1548.452721] disk 3, o:1, dev:sdf1
> [ 1548.486067] ------------[ cut here ]------------
> [ 1548.486076] WARNING: CPU: 4 PID: 10512 at drivers/md/raid5-cache.c:728 r5l_do_reclaim+0x415/0x430 [raid456]
> [ 1548.486077] Modules linked in: fuse btrfs vfat msdos fat ext4 jbd2 mbcache binfmt_misc raid456 async_raid6_recov async_memcpy async_pq async_xor xor async_tx raid6_pq xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun ipt_REJECT nf_reject_ipv4 ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sb_edac edac_core snd_hda_codec_realtek snd_hda_codec_hdmi snd_hda_codec_generic x86_pkg_temp_thermal snd_hda_intel intel_powerclamp coretemp s
nd_hda_codec
> [ 1548.486117] kvm_intel kvm snd_hda_core irqbypass crct10dif_pclmul crc32_pclmul snd_hwdep ghash_clmulni_intel snd_seq aesni_intel lrw snd_seq_device gf128mul glue_helper snd_pcm ablk_helper hp_wmi iTCO_wdt snd_timer sparse_keymap rfkill cryptd iTCO_vendor_support mei_me ioatdma snd mei shpchp pcspkr sg soundcore i2c_i801 tpm_infineon lpc_ich mfd_core dca nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod nouveau video mxm_wmi i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm isci e1000e ata_generic libsas pata_acpi ahci libahci scsi_transport_sas firewire_ohci libata firewire_core ptp crc32c_intel serio_raw pps_core i2c_core crc_itu_t wmi fjes dm_mirror dm_region_hash dm_log dm_mod
> [ 1548.486152] CPU: 4 PID: 10512 Comm: md0_resync Tainted: G W 4.7.0-rc7 #1
> [ 1548.486153] Hardware name: Hewlett-Packard HP Z620 Workstation/158A, BIOS J61 v03.69 03/25/2014
> [ 1548.486155] 0000000000000286 000000002369dfe1 ffff88079d713b40 ffffffff8134caec
> [ 1548.486157] 0000000000000000 0000000000000000 ffff88079d713b80 ffffffff8108c351
> [ 1548.486159] 000002d8a6ae4d80 000000008f0c904f ffff8807f64b0000 ffff8807c4ab6888
> [ 1548.486161] Call Trace:
> [ 1548.486166] [<ffffffff8134caec>] dump_stack+0x63/0x87
> [ 1548.486169] [<ffffffff8108c351>] __warn+0xd1/0xf0
> [ 1548.486172] [<ffffffff8108c48d>] warn_slowpath_null+0x1d/0x20
> [ 1548.486175] [<ffffffffa097d805>] r5l_do_reclaim+0x415/0x430 [raid456]
> [ 1548.486178] [<ffffffff811f9c50>] ? kfree+0x120/0x170
> [ 1548.486181] [<ffffffffa097e33b>] r5l_quiesce+0x7b/0xa0 [raid456]
> [ 1548.486183] [<ffffffffa0970da0>] raid5_quiesce+0x50/0x2a0 [raid456]
> [ 1548.486186] [<ffffffff810d0250>] ? prepare_to_wait_event+0xf0/0xf0
> [ 1548.486189] [<ffffffff8156d00e>] md_do_sync+0xe7e/0xf60
> [ 1548.486192] [<ffffffff810c4370>] ? enqueue_entity+0x2a0/0xcc0
> [ 1548.486194] [<ffffffff810c201d>] ? update_curr+0xed/0x180
> [ 1548.486196] [<ffffffff810c026e>] ? account_entity_dequeue+0xae/0xd0
> [ 1548.486197] [<ffffffff810c27f6>] ? dequeue_entity+0x266/0x980
> [ 1548.486200] [<ffffffff810b5ef5>] ? check_preempt_curr+0x75/0x90
> [ 1548.486202] [<ffffffff810b5f29>] ? ttwu_do_wakeup+0x19/0xe0
> [ 1548.486204] [<ffffffff810c2fce>] ? dequeue_task_fair+0xbe/0x830
> [ 1548.486206] [<ffffffff810c3995>] ? put_prev_entity+0x35/0x730
> [ 1548.486209] [<ffffffff8102c6d9>] ? __switch_to+0x219/0x5c0
> [ 1548.486211] [<ffffffff81098d43>] ? kernel_sigaction+0x43/0xe0
> [ 1548.486214] [<ffffffff81566aa6>] md_thread+0x136/0x150
> [ 1548.486216] [<ffffffff81566970>] ? find_pers+0x70/0x70
> [ 1548.486219] [<ffffffff810ab698>] kthread+0xd8/0xf0
> [ 1548.486221] [<ffffffff816e873f>] ret_from_fork+0x1f/0x40
> [ 1548.486223] [<ffffffff810ab5c0>] ? kthread_park+0x60/0x60
> [ 1548.486224] ---[ end trace 242e9516e36acb1e ]---
> [ 1548.543855] md: recovery of RAID array md0
> [ 1548.543857] md: minimum _guaranteed_ speed: 1000 KB/sec/disk.
> [ 1548.543859] md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for recovery.
> [ 1548.543864] md: using 128k window, over a total of 523264k.
> [ 1548.543868] md: md0: recovery done.
> [ 1548.574961] RAID conf printout:
> [ 1548.574964] --- level:6 rd:4 wd:4
> [ 1548.574966] disk 0, o:1, dev:sdb1
> [ 1548.574967] disk 1, o:1, dev:sdd1
> [ 1548.574968] disk 2, o:1, dev:sde1
> [ 1548.574969] disk 3, o:1, dev:sdf1
>
> Steps I used:
>
> mdadm --create --run /dev/md0 --level 4 --metadata 1.2 --raid-devices 4
> /dev/sdb1 /dev/sd[d-f]1 --write-journal /dev/sdc1 --bitmap=internal
> --bitmap-chunk=64M --chunk 512
> mdadm --wait /dev/md0
> mkfs.ext4 /dev/md0
> mdadm /dev/md0 -f /dev/sdb1
> mdadm /dev/md0 -r /dev/sdb1
> mdadm /dev/md0 -a /dev/sdb1
I tried above cmds in my VM machine (with set level as 6), but can't
see the calltrace with below versions.
linux135:~ # uname -r
4.4.13-46-default
linux135:~ # mdadm --version
mdadm - v3.4-47-g8718fc3 - 11th May 2016
Yi: Could you try with 4.7.0, it's very easy to reproduce on my side
below is my env:
#lsblk /dev/sd[b-f]
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sdb 8:16 0 931.5G 0 disk
└─sdb1 8:17 0 10G 0 part
└─md0 9:0 0 30G 0 raid4
sdc 8:32 0 186.3G 0 disk
└─sdc1 8:33 0 10G 0 part
└─md0 9:0 0 30G 0 raid4
sdd 8:48 0 111.8G 0 disk
└─sdd1 8:49 0 10G 0 part
└─md0 9:0 0 30G 0 raid4
sde 8:64 0 931.5G 0 disk
└─sde1 8:65 0 10G 0 part
└─md0 9:0 0 30G 0 raid4
sdf 8:80 0 931.5G 0 disk
└─sdf1 8:81 0 10G 0 part
└─md0 9:0 0 30G 0 raid4
# uname -r
4.7.0
# mdadm --version
mdadm - v3.4-63-g52209d6 - 21st July 2016
^ permalink raw reply
* One question about raid10 resync
From: Xiao Ni @ 2016-08-01 14:02 UTC (permalink / raw)
To: linux-raid; +Cc: Shaohua Li, Jes Sorensen
In-Reply-To: <1870990013.9606108.1470059688027.JavaMail.zimbra@redhat.com>
Hi all
I have a question about raid10 resync. The fbio->bi_next isn't null in function
sync_request_write. But the bios' bi_next that read the data from copies is set
to NULL before sending down to underlying devices in raid10_sync_request.
I tried to find where links those bios that don't belong to a r10_bio. But I can't
find it. Is it a bug?
Best Regards
Xiao
^ permalink raw reply
* Re: xosview
From: Bill Hudacek @ 2016-08-01 18:50 UTC (permalink / raw)
To: mdraid
In-Reply-To: <ffd06b83-0692-d5a4-1daa-126abf53edfa@youngman.org.uk>
Anthony Youngman wrote on 07/29/2016 12:52 PM:
> So if people wouldn't mind, could you email your mdstat files?
> Preferably on the list so people can see what has and has not been sent
> - obviously I'd like standard setups like raid10, raid5, raid6, both
> named and numbered. And if people have them, mdstats showing broken
> arrays, rebuilds, complicated setups with lvm, etc.
>
RAID 6 across 5 disks, of which 1 is a spare (in external cabinet),
and two disks in RAID-1 (for OS, inside the tower):
> cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4]
md0 : active raid6 sdc1[0] sdf1[3] sdd1[1] sdg1[4](S) sde1[2]
3071737856 blocks super 1.2 level 6, 1024k chunk, algorithm 2
[4/4] [UUUU]
bitmap: 0/12 pages [0KB], 65536KB chunk
md126 : active raid1 sdb1[1] sda1[0]
2099136 blocks super 1.0 [2/2] [UU]
bitmap: 0/1 pages [0KB], 65536KB chunk
md127 : active raid1 sdb2[1] sda2[0]
234921984 blocks super 1.2 [2/2] [UU]
bitmap: 1/2 pages [4KB], 65536KB chunk
unused devices: <none>
I don't have any failure mdstat output saved, sorry...
--
/Bill
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox