* Re: Recovery on new 2TB disk: finish=7248.4min (raid1)
From: Nix @ 2017-05-01 20:12 UTC (permalink / raw)
To: Wols Lists
Cc: Roman Mamedov, John Stoffel, Mateusz Korniak, Ron Leach,
linux-raid
In-Reply-To: <59061BFF.2070503@youngman.org.uk>
On 30 Apr 2017, Wols Lists verbalised:
> On 30/04/17 17:10, Nix wrote:
>> This is a one-off with tooling to manage it: from my perspective, I just
>> kick off the autobuilders etc and they'll automatically use transient
>> space for objdirs. (And obviously this is all scripted so it is no
>> harder than making or removing directories would be: typing 'mktransient
>> foo' to automatically create a dir in transient space and set up a bind
>> mount to it -- persisted across boots -- in the directory' foo' is
>> literally a few letters more than typing 'mkdir foo'.)
>
> This sounds like me with tmpfs. Okay, mine don't persist across reboots,
> but if it's in your build scripts, can't they create a tmpfs and do the
> builds in that?
Even though I have 128GiB RAM, I don't want a tmpfs for everything
(though I use it for a lot, oh yes). Some of the transient stuff is
things like QEMU CoW disk images that I don't want to lose on every
reboot (though rebuilding them is not too annoying, it is not totally
ignorable either); other stuff is things like vapoursynth intermediates
which can easily exceed 500GiB, which are only ever read once but
nonetheless just won't fit.
--
NULL && (void)
^ permalink raw reply
* Re: [PATCH 25/27] block: remove the discard_zeroes_data flag
From: Bart Van Assche @ 2017-05-01 20:45 UTC (permalink / raw)
To: agk@redhat.com, lars.ellenberg@linbit.com, snitzer@redhat.com,
hch@lst.de, martin.petersen@oracle.com,
philipp.reisner@linbit.com, axboe@kernel.dk, shli@kernel.org
Cc: linux-scsi@vger.kernel.org, dm-devel@redhat.com,
drbd-dev@lists.linbit.com, linux-block@vger.kernel.org,
target-devel@vger.kernel.org, linux-raid@vger.kernel.org
In-Reply-To: <20170405172125.22600-26-hch@lst.de>
On Wed, 2017-04-05 at 19:21 +0200, Christoph Hellwig wrote:
> Now that we use the proper REQ_OP_WRITE_ZEROES operation everywhere we can
> kill this hack.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> Reviewed-by: Martin K. Petersen <martin.petersen@oracle.com>
> Reviewed-by: Hannes Reinecke <hare@suse.com>
> [ ... ]
> diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c
> index c754ae33bf7b..d2f089cfa9ae 100644
> --- a/drivers/target/target_core_device.c
> +++ b/drivers/target/target_core_device.c
> @@ -851,7 +851,7 @@ bool target_configure_unmap_from_queue(struct se_dev_attrib *attrib,
> attrib->unmap_granularity = q->limits.discard_granularity / block_size;
> attrib->unmap_granularity_alignment = q->limits.discard_alignment /
> block_size;
> - attrib->unmap_zeroes_data = q->limits.discard_zeroes_data;
> + attrib->unmap_zeroes_data = 0;
> return true;
> }
> EXPORT_SYMBOL(target_configure_unmap_from_queue);
Hello Christoph,
Sorry that I hadn't noticed this before but I think that this patch
introduces a significant performance regressions for LIO users. Before
this patch the LBPRZ flag was reported correctly to initiator systems
through the thin provisioning VPD. With this patch applied that flag
will always be reported as zero, forcing initiators to submit WRITE
commands with zeroed data buffers instead of submitting the SCSI UNMAP
command to block devices for which discard_zeroes_data was set. From
target_core_spc.c:
/* Thin Provisioning VPD */
static sense_reason_t spc_emulate_evpd_b2(struct se_cmd *cmd, unsigned char *buf)
{
[ ... ]
/*
* The unmap_zeroes_data set means that the underlying device supports
* REQ_DISCARD and has the discard_zeroes_data bit set. This satisfies
* the SBC requirements for LBPRZ, meaning that a subsequent read
* will return zeroes after an UNMAP or WRITE SAME (16) to an LBA
* See sbc4r36 6.6.4.
*/
if (((dev->dev_attrib.emulate_tpu != 0) ||
(dev->dev_attrib.emulate_tpws != 0)) &&
(dev->dev_attrib.unmap_zeroes_data != 0))
buf[5] |= 0x04;
[ ... ]
}
Please advise how to proceed.
Thanks,
Bart.
^ permalink raw reply
* Re: Recovery on new 2TB disk: finish=7248.4min (raid1)
From: Nix @ 2017-05-01 21:13 UTC (permalink / raw)
To: Roman Mamedov; +Cc: John Stoffel, Mateusz Korniak, Ron Leach, linux-raid
In-Reply-To: <20170430214729.2a02cce5@natsu>
[linux-raid list: sorry, this is getting quite off-topic, though I'm
finding the argument(?) quite fascinating. I can take it off-list if
you like.]
On 30 Apr 2017, Roman Mamedov told this:
> On Sun, 30 Apr 2017 17:10:22 +0100
> Nix <nix@esperi.org.uk> wrote:
>
>> > It's not like the difference between the so called "fast" and "slow" parts is
>> > 100- or even 10-fold. Just SSD-cache the entire thing (I prefer lvmcache not
>> > bcache) and go.
>>
>> I'd do that if SSDs had infinite lifespan. They really don't. :)
>> lvmcache doesn't cache everything, only frequently-referenced things, so
>> the problem is not so extreme there -- but
>
> Yes I was concerned the lvmcache will over-use the SSD by mistakenly caching
> streaming linear writes and the like -- and it absolutely doesn't. (it can
> during the initial fill-up of the cache, but not afterwards).
Yeah, it's hopeless to try to minimize SSD writes during initial cache
population. Of course you'll write to the SSD a lot then. That's the point.
> Get an MLC-based SSD if that gives more peace of mind, but tests show even the
> less durable TLC-based ones have lifespan measuring in hundreds of TB.
> http://techreport.com/review/27909/the-ssd-endurance-experiment-theyre-all-dead
That was a fascinating and frankly quite reassuring article, thank you! :)
> One SSD that I have currently has 19 TB written to it over its entire 4.5 year
> lifespan. Over the past few months of being used as lvmcache for a 14 TB
> bulk data array and a separate /home FS, new writes average at about 16 GB/day.
That's a lot less than I expect, alas. Busy machine, lots of busy source
trees and large transient writes -- and without some careful management
the SSD capacity would not be larger than the expected working set
forever. That's what the fast/slow division is for.
> Given a VERY conservative 120 TBW endurance estimate, this SSD should last me
> all the way into year 2034 at least.
The lifetime estimate on mine says three years to failure, at present
usage rates (datacenter-quality SSDs are neat, they give you software
that tells you things like this). I'll probably replace it with one
rated for higher write loads next time. They're still beyond my price
point right now, but in three years they should be much cheaper!)
>> the fact that it has to be set up anew for *each LV* is a complete killer
>> for me, since I have encrypted filesystems and things that *have* to be on
>> separate LVs and I really do not want to try to figure out the right balance
>> between distinct caches, thanks (oh and also you have to get the metadata
>> size right, and if you get it wrong and it runs out of space all hell breaks
>> loose, AIUI). bcaching the whole block device avoids all this pointless
>> complexity. bcache just works.
>
> Oh yes I wish they had a VG-level lvmcache. Still, it feels more mature than
> bcache, the latter barely has any userspace management and monitoring tools
I was worried about that, but y'know you hardly need them. You set it up
and it just works. (Plus, you can do things like temporarily turn the
cache *off* during e.g. initial population, have it ignore low-priority
I/O, streaming reads etc, none of which lvmcache could do last time I
looked. And nearly all the /sys knobs are persistently written to the
bcache superblock so you only need to tweak them once.)
I far prefer that to LVM's horribly complicated tools, which I frankly
barely understand by this point. The manpages randomly intermingle
ordinary LV, snapshotting, RAID, caching, clustering, and options only
useful for other use cases in an almighty tangle, relying on examples at
the bottom of the manpage to try to indicate which options are useful
where. Frankly they should be totally reorganized to be much more like
mdadm's -- divided into nice neat sections or at least with some sort of
by-LV-type options chart.
As for monitoring, the stats in /sys knock LVM's completely out of the
park, with continuously-updated stats on multiple time horizons.
To me, LVM feels both overdesigned and seriously undercooked for this
use case, definitely not ready for serious use as a cache.
> (having to fiddle with "echo > /sys/..." and "cat /sys/..." is not the state
> of something you'd call a finished product).
You mean, like md? :)
I like /sys. It's easy to explore and you can use your standard fs tools
on it. The only downside is the inability to comment anything :( but
that's what documentation is for. (Oh, also, if you need ordering or
binary data, /sys is the wrong tool. But for configuration interfaces
that is rarely true.)
> And the killer for me was that
> there is no way to stop using bcache on a partition, once it's a "bcache
> backing device" there is no way to migrate back to a raw partition, you're
> stuck with it.
That doesn't really matter, since you can turn the cache off completely
and persistently with
echo none > /sys/block/bcache$num/bcache/cache_mode
and as soon as you do, the cache device is no longer required for the
bcache to work (though if you had it in use for writeback caching,
you'll have some fscking to do), and it imposes no overhead that I can
discern.
(The inability to use names with bcache devices *is* annoying: LVM and
indeed md beats it there.)
>> This is a one-off with tooling to manage it: from my perspective, I just
>> kick off the autobuilders etc and they'll automatically use transient
>> space for objdirs. (And obviously this is all scripted so it is no
>> harder than making or removing directories would be: typing 'mktransient
>> foo' to automatically create a dir in transient space and set up a bind
>> mount to it -- persisted across boots -- in the directory' foo' is
>> literally a few letters more than typing 'mkdir foo'.)
>
> Sorry for being rather blunt initially, still IMO the amount if micromanagement
> required (and complexity introduced) is staggering compared to the benefits
I was worried about that, but it's almost entirely scripted, so "none to
speak of". The only admin overhead I see in my daily usage is a single
"sync-vms" command every time I yum update my more write-insane test
virtual machines. (I don't like writing 100GiB to the SSD ten times a
day, so I run those VMs via CoW onto the RAID-0 transient fs, and write
them back to their real filesystems on the cached/journalled array after
big yum updates or when I do something else I want long-term
preservation for. That happens every few weeks, at most.)
Everything else is automated: my autobuilders make transient bind-mounts
onto the RAID-0 as needed, video transcoding drops stuff in there
automatically, and backups run with ionice -c3 so they don't flood the
cache either. I probably don't run mktransient by hand more than once a
month.
I'd be more worried about the complexity required to just figure out the
space needed for half a dozen sets of lvmcache metadata and cache
filesystems. (How do you know how much cache you'll need for each fs in
advance, anyway? That seems like a much harder question to answer than
"will I want to cache this at all".)
> reaped -- and it all appears to stem from underestimating the modern SSDs.
> I'd suggest just get one and try "killing" it with your casual daily usage,
When did I say I was a casual daily user? Build-and-test cycles with
tens to hundreds of gigs of writes daily are routine, and video
transcoding runs with half-terabyte to a terabyte of writes happen quite
often. I care about the content of those writes for about ten minutes
(one write, one read) and then couldn't care less about them: they're
entirely transient. Dumping them to an SSD cache, or indeed to the md
journal, is just pointless. I'm dropping some of them onto tmpfs, but
some are just too large for that.
I didn't say this was a setup useful for everyone! My workload happens
to have a lot of large briefly-useful writes in it, and a lot of
archival data that I don't want to waste space caching. It's the *other*
stuff, that doesn't fit into those categories, that I want to cache and
RAID-journal (and, for that matter, run routine backups of, so my own
backup policies told me what data fell into which category.)
As for modern SSDs... I think my Intel S3510 is a modern SSD, if not a
write-workload-focused one (my supplier lied to me and claimed it was
write-focused, and the spec sheet that said otherwise did not become
apparent until after I bought it, curses).
I'll switch to a write-focused 1.2TiB S3710, or the then-modern
equivalent, when the S3510 burns out. *That* little monster is rated for
14 petabytes of writes before failure... but it also costs over a
thousand pounds right now, and I already have a perfectly good SSD, so
why not use it until it dies? I'd agree that when using something like
the S3710 I'm going to stop caring about writes, because if you try to
write that much to rotating rust it's going to wear out too. But the
480GiB S3510, depending on which spec sheets I read, is either rated for
290TiB or 876TiB of writes before failure, and given the Intel SSD
"suicide-pill" self-bricking wearout failure mode described in the
article you cited above, I think being a bit cautious is worthwhile.
290TiB is only the equivalent of thirteen complete end-to-end writes to
the sum of all my RAID arrays... so no, I'm not treating it like it has
infinite write endurance. Its own specs say it doesn't. (This is also
why only the fast array is md-journalled.)
(However, I do note that the 335 tested in the endurance test above is
only rated for 20GiB of daily writes for three years, which comes to
only 22TiB total writes, but in the tests it bricked itself after
*720TiB*. So it's quite possible my S3510 will last vastly longer than
its own diagnostic tools estimate, well into the petabytes. I do hope it
does! I'm just not *trusting* that it does. A bit of fiddling and
scripting at setup time is quite acceptable for that peace of mind.
It wouldn't be worth it if this was a lot of work on an ongoing basis,
but it's nearly none.)
> you'll find (via TBW numbers you will see in SMART compared even to vendor
> spec'd ones, not to mention what tech sites' field tests show) that you just
> can't, not until deep into a dozen of years later into the future.
I'm very impressed by modern SSD write figures, and suspect that in a
few years they will be comparable to rotating rust's. They're just not,
yet. Not quite, and my workload falls squarely into the 'not quite' gap.
Given how easy it was for me to script my way around this problem, I
didn't mind much. With a hardware RAID array, it would have been much
more difficult! md's unmatched flexibility shines yet again.
--
NULL && (void)
^ permalink raw reply
* Big-endian RAID5 recovery problem
From: Adam Thompson @ 2017-05-01 21:39 UTC (permalink / raw)
To: MUUG Roundtable, linux-raid
So I've got 4 IDE HDDs, each with 3 RAID partitions on them, that were
part of a RAID array in a now-very-dead NAS.
Of course, I need to get data off them that wasn't backed up anywhere
else.
I've got a 4-port USB3 PCIe card, and 4 IDE/SATA USB adapters, and all
the hardware seems to work. So far, so good.
The problem is that the disks use the v0.90 metadata format, and they
came from a big-endian system, not a little-endian system. MD
superblocks *since* v0.90 are endian-agnostic, but back in v0.90, the
superblock was byte-order specific.
mdadm(8) on an Intel processor refuses to acknowledge the existence of
the superblock. Testdisk detects it and correctly identifies it as a
Big-endian v0.90 superblock.
I'm reluctant to blindly do a forced --create on the four disks, because
I'm not 100% certain of the RAID topology; there are at least two RAID
devices, one of which was hidden from the user, so I have no a-priori
knowledge of its RAID level or layout.
The filesystems on the md(4) devices are, AFAIK, all XFS, and so should
(hopefully) not have any endianness issues.
I can't find any modern big-endian Linux systems... looks like all the
ARM distros run in little-endian mode.
Any suggestions on the best way to move forward?
Thanks,
-Adam
^ permalink raw reply
* Re: Recovery on new 2TB disk: finish=7248.4min (raid1)
From: Anthony Youngman @ 2017-05-01 21:44 UTC (permalink / raw)
To: Nix, Roman Mamedov; +Cc: John Stoffel, Mateusz Korniak, Ron Leach, linux-raid
In-Reply-To: <8737cojn54.fsf@esperi.org.uk>
On 01/05/17 22:13, Nix wrote:
> [linux-raid list: sorry, this is getting quite off-topic, though I'm
> finding the argument(?) quite fascinating. I can take it off-list if
> you like.]
1) I'm finding it fascinating, too ... :-)
2) We get a reasonable amount of other stuff which I think is off-topic
for raid but is cc'd to the list because it's relevant to linux
3) At some point it would be nice to have this sort of stuff on the
wiki, so people have up-to-date performance figures. Snag is, someone's
got to run a load of benchmarks, document it, and do a lot of work for
little personal benefit ...
Cheers,
Wol
^ permalink raw reply
* Re: Recovery on new 2TB disk: finish=7248.4min (raid1)
From: Roman Mamedov @ 2017-05-01 21:46 UTC (permalink / raw)
To: Nix; +Cc: John Stoffel, Mateusz Korniak, Ron Leach, linux-raid
In-Reply-To: <8737cojn54.fsf@esperi.org.uk>
On Mon, 01 May 2017 22:13:59 +0100
Nix <nix@esperi.org.uk> wrote:
> > (having to fiddle with "echo > /sys/..." and "cat /sys/..." is not the state
> > of something you'd call a finished product).
>
> You mean, like md? :)
You must be kidding. On the contrary I was looking to present md as the
example of how it should be done, with its all-encompassing and extremely
capable 'mdadm' tool -- and a complete lack of a similar tool for bcache.
> I'd be more worried about the complexity required to just figure out the
> space needed for half a dozen sets of lvmcache metadata and cache
> filesystems.
Metadata can be implicitly created and auto-managed in recent lvm versions
http://fibrevillage.com/storage/460-how-to-create-lvm-cache-logical-volume
("Automatic pool metadata LV"). If not, the rule of thumb suggested everywhere
is 1/1000 of the cache volume size; I doubled that just in case, and looks
like I didn't have to, as my metadata partitions are only about 9.5% full each.
As for half a dozen sets, I'd reconsider the need for those, as well as the
entire fast/slow HDD tracks separation, just SSD-cache everything and let it
figure out to not cache streaming writes on your video transcodes, or even bulk
writes during your compile tests (while still caching the filesystem metadata).
However the world of pain begins if you want to have multiple guest VMs each
with its disk as a separate LV. One solution (that doesn't sound too clean but
perhaps could work), is stacked LVM, i.e. a PV of a different volume group made
on top of a cached LV.
--
With respect,
Roman
^ permalink raw reply
* Re: Recovery on new 2TB disk: finish=7248.4min (raid1)
From: Anthony Youngman @ 2017-05-01 21:53 UTC (permalink / raw)
To: Roman Mamedov, Nix; +Cc: John Stoffel, Mateusz Korniak, Ron Leach, linux-raid
In-Reply-To: <20170502024657.6d33fd88@natsu>
On 01/05/17 22:46, Roman Mamedov wrote:
> On Mon, 01 May 2017 22:13:59 +0100
> Nix <nix@esperi.org.uk> wrote:
>
>>> (having to fiddle with "echo > /sys/..." and "cat /sys/..." is not the state
>>> of something you'd call a finished product).
>> You mean, like md? :)
> You must be kidding. On the contrary I was looking to present md as the
> example of how it should be done, with its all-encompassing and extremely
> capable 'mdadm' tool -- and a complete lack of a similar tool for bcache.
>
That's what I understood you to mean, but you are aware that SOME raid
management still has to be done with echo > /sys/... ?
So mdadm isn't perfect, not by a long chalk, yet :-)
Cheers,
Wol
^ permalink raw reply
* Re: Big-endian RAID5 recovery problem
From: Anthony Youngman @ 2017-05-01 21:59 UTC (permalink / raw)
To: Adam Thompson, MUUG Roundtable, linux-raid
In-Reply-To: <5c6e6e5d93d4056839e4f370e00a8e08@mail.athompso.net>
On 01/05/17 22:39, Adam Thompson wrote:
>
> I'm reluctant to blindly do a forced --create on the four disks, because
> I'm not 100% certain of the RAID topology; there are at least two RAID
> devices, one of which was hidden from the user, so I have no a-priori
> knowledge of its RAID level or layout.
>
Get hold of lsdrv, and see what that tells you. (Look at the raid wiki
for details.) I don't know if it will have endian issues, but if it
doesn't an expert will probably be able to chime straight in and tell
you the create command.
The other thing is, read up on overlays because, if you overlay those
disks, you will be able to "create" without actually writing to the
disks. That way you can test - and even do a complete backup and
recovery - without ever actually writing to, and altering, the original
disks.
Cheers,
Wol
^ permalink raw reply
* Re: Recovery on new 2TB disk: finish=7248.4min (raid1)
From: Roman Mamedov @ 2017-05-01 22:03 UTC (permalink / raw)
To: Anthony Youngman
Cc: Nix, John Stoffel, Mateusz Korniak, Ron Leach, linux-raid
In-Reply-To: <6398208a-1345-bdec-4be6-a8bd98bfb089@youngman.org.uk>
On Mon, 1 May 2017 22:53:14 +0100
Anthony Youngman <antlists@youngman.org.uk> wrote:
> That's what I understood you to mean, but you are aware that SOME raid
> management still has to be done with echo > /sys/... ?
>
> So mdadm isn't perfect, not by a long chalk, yet :-)
Well, why not post some examples of what you find yourself doing often
via /sys, that's not available in mdadm (maybe as a new thread).
One that I remember is the "want_replacement" mechanism, which was initially
only available via "echo > /sys/..." but quickly got added to mdadm as
"--replace".
People (and various outdated wikis) also tend to suggest using "echo check..."
or "echo repair...", but those are available in mdadm as well, via "--action=".
Lastly, I change the "stripe_cache_size" via /sys/, but that's fine-tuning,
which feels OK to do via sysfs parameters, whereas needing to use sysfs for the
most basic operations of managing the storage system does not.
--
With respect,
Roman
^ permalink raw reply
* Re: Big-endian RAID5 recovery problem
From: Adam Thompson @ 2017-05-01 22:33 UTC (permalink / raw)
To: Anthony Youngman; +Cc: MUUG Roundtable, linux-raid
In-Reply-To: <0ab1ea1b-8035-512e-0b0a-819e15929f41@youngman.org.uk>
On 2017-05-01 16:59, Anthony Youngman wrote:
> Get hold of lsdrv, and see what that tells you. (Look at the raid wiki
> for details.) I don't know if it will have endian issues, but if it
> doesn't an expert will probably be able to chime straight in and tell
> you the create command.
Ah! And that took me straight to the "asking for help" page.
The raw data is here:
https://gist.github.com/anonymous/321b6db3160c259c4a4dd549817a3d07
To summarize:
* smartctl either fails to run or shows nothing wrong (depending on the
vintage of drive, maybe?);
* mdadm --examine fails to read the superblock because of the endianness
issue (see
https://raid.wiki.kernel.org/index.php/RAID_superblock_formats#The_version-0.90_Superblock_Format)
* lsdrv fails to report any useful MD topology information I could see
(other than confirming that each md device had four members, one
partition on each drive)
I also see 3 "FD" type partitions on each disk, but lsdrv only
identifies *2* of them as belonging to an MD array. Not sure what's up
with that.
> The other thing is, read up on overlays because, if you overlay those
> disks, you will be able to "create" without actually writing to the
> disks. That way you can test - and even do a complete backup and
> recovery - without ever actually writing to, and altering, the
> original disks.
Currently reading, thanks. Didn't know overlays could be used for block
devices.
Spinning up a QEMU instance of Linux-PPC or Linux-MIPS with the disks in
pass-through mode has also been mentioned, but... ugh. Anecdotal
reports from the web suggest that doing so would just be opening up a
second rabbit hole in addition to the one I'm already headed down.
-Adam
^ permalink raw reply
* [GIT PULL] MD update for 4.12
From: Shaohua Li @ 2017-05-01 22:50 UTC (permalink / raw)
To: torvalds; +Cc: linux-kernel, linux-raid, neilb
Hi,
Please pull MD update for 4.12. There are conflicts between MD tree and block
tree, so I did a merge before the pull request. This update includes:
- Add Partial Parity Log (ppl) feature found in Intel IMSM raid array by Artur
Paszkiewicz. This feature is another way to close RAID5 writehole. The Linux
implementation is also available for normal RAID5 array if specific superblock
bit is set.
- A number of md-cluser fixes and enabling md-cluster array resize from Guoqing
Jiang
- A bunch of patches from Ming Lei and Neil Brown to rewrite MD bio handling
related code. Now MD doesn't directly access bio bvec, bi_phys_segments and
uses modern bio API for bio split.
- Improve RAID5 IO pattern to improve performance for hard disk based RAID5/6
from me.
- Several patches from Song Liu to speed up raid5-cache recovery and allow
raid5 cache feature disabling in runtime.
- Fix a performance regression in raid1 resync from Xiao Ni.
- Other cleanup and fixes from various people.
Thanks,
Shaohua
The following changes since commit 85724edecbdc19f53ed4b902fc3a32e4d1b61c9b:
Merge tag 'leds_for_4.12' of git://git.kernel.org/pub/scm/linux/kernel/git/j.anaszewski/linux-leds (2017-05-01 13:12:49 -0700)
are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/shli/md.git for-linus
for you to fetch changes up to e265eb3a30543a237b2ebc4e0422ac82e55b07e4:
Merge branch 'md-next' into md-linus (2017-05-01 14:09:21 -0700)
----------------------------------------------------------------
Artur Paszkiewicz (12):
md: superblock changes for PPL
raid5: separate header for log functions
raid5-ppl: Partial Parity Log write logging implementation
md: add sysfs entries for PPL
raid5-ppl: load and recover the log
raid5-ppl: support disk hot add/remove with PPL
raid5-ppl: runtime PPL enabling or disabling
raid5-ppl: move no_mem_stripes to struct ppl_conf
raid5-ppl: use resize_stripes() when enabling or disabling ppl
raid5-ppl: partial parity calculation optimization
raid5-ppl: use a single mempool for ppl_io_unit and header_page
uapi: fix linux/raid/md_p.h userspace compilation error
Christophe JAILLET (1):
md-cluster: Fix a memleak in an error handling path
Dan Carpenter (1):
raid5-ppl: silence a misleading warning message
Dennis Yang (1):
md: update slab_cache before releasing new stripes when stripes resizing
Guoqing Jiang (9):
md-cluster: use sync way to handle METADATA_UPDATED msg
md-cluster: add CHANGE_CAPACITY message type
md-cluster: introduce cluster_check_sync_size
md-cluster: add the support for resize
md: move bitmap_destroy to the beginning of __md_stop
md/raid10: refactor some codes from raid10_write_request
md/raid10: reset the 'first' at the end of loop
md/raid10: wait up frozen array in handle_write_completed
md: clear WantReplacement once disk is removed
Jason Yan (1):
md/raid5-cache: fix payload endianness problem in raid5-cache
Lidong Zhong (1):
md/raid1/10: remove unused queue
Ming Lei (15):
md: raid1/raid10: don't handle failure of bio_add_page()
md: move two macros into md.h
md: prepare for managing resync I/O pages in clean way
md: raid1: simplify r1buf_pool_free()
md: raid1: don't use bio's vec table to manage resync pages
md: raid1: retrieve page from pre-allocated resync page array
md: raid1: use bio helper in process_checks()
block: introduce bio_copy_data_partial
md: raid1: move 'offset' out of loop
md: raid1: improve write behind
md: raid10: refactor code of read reshape's .bi_end_io
md: raid10: don't use bio's vec table to manage resync pages
md: raid10: retrieve page from preallocated resync page array
md: raid10: avoid direct access to bvec table in handle_reshape_read_error
md: raid1: kill warning on powerpc_pseries
NeilBrown (30):
md/raid5: use md_write_start to count stripes, not bios
md/raid5: simplfy delaying of writes while metadata is updated.
md/raid5: call bio_endio() directly rather than queueing for later.
md/raid5: use bio_inc_remaining() instead of repurposing bi_phys_segments as a counter
md/raid5: remove over-loading of ->bi_phys_segments.
Revert "md/raid5: limit request size according to implementation limits"
md/raid1, raid10: move rXbio accounting closer to allocation.
md/raid10: stop using bi_phys_segments
md/raid1: stop using bi_phys_segment
md/raid5: don't test ->writes_pending in raid5_remove_disk
md: factor out set_in_sync()
md: close a race with setting mddev->in_sync
percpu-refcount: support synchronous switch to atomic mode.
MD: use per-cpu counter for writes_pending
md/raid6: Fix anomily when recovering a single device in RAID6.
md: MD_CLOSING needs to be cleared after called md_set_readonly or do_md_stop
md/raid1: avoid reusing a resync bio after error handling.
md/raid1: simplify the splitting of requests.
md/raid1: simplify alloc_behind_master_bio()
Revert "block: introduce bio_copy_data_partial"
md/raid1: simplify handle_read_error().
md/raid1: factor out flush_bio_list()
md/raid10: simplify the splitting of requests.
md/raid10: simplify handle_read_error()
md/raid5: make chunk_aligned_read() split bios more cleanly.
md/linear: improve bio splitting.
md/raid0: fix up bio splitting.
md: allow creation of mdNNN arrays via md_mod/parameters/new_array
md: support disabling of create-on-open semantics.
md: handle read-only member devices better.
Shaohua Li (7):
md/raid5: prioritize stripes for writeback
md/raid5-cache: bump flush stripe batch size
md/raid5: sort bios
block: remove bio_clone_bioset_partial()
md/raid1: skip data copy for behind io for discard request
md/raid10: skip spare disk as 'first' disk
Merge branch 'md-next' into md-linus
Song Liu (4):
md/r5cache: improve recovery with read ahead page pool
md/r5cache: handle R5LOG_PAYLOAD_FLUSH in recovery
md/r5cache: generate R5LOG_PAYLOAD_FLUSH
md/raid5: use consistency_policy to remove journal feature
Xiao Ni (1):
md/raid1: Use a new variable to count flighting sync requests
Zhilong Liu (2):
md: fix several trivial typos in comments
md.c:didn't unlock the mddev before return EINVAL in array_size_store
Documentation/admin-guide/md.rst | 32 +-
Documentation/md/md-cluster.txt | 2 +-
Documentation/md/raid5-ppl.txt | 44 ++
block/bio.c | 61 +-
drivers/md/Makefile | 2 +-
drivers/md/bitmap.c | 59 +-
drivers/md/bitmap.h | 3 +
drivers/md/linear.c | 75 ++-
drivers/md/md-cluster.c | 223 ++++++-
drivers/md/md-cluster.h | 1 +
drivers/md/md.c | 414 ++++++++++---
drivers/md/md.h | 71 ++-
drivers/md/raid0.c | 78 +--
drivers/md/raid1.c | 679 +++++++++-----------
drivers/md/raid1.h | 13 +-
drivers/md/raid10.c | 736 +++++++++-------------
drivers/md/raid10.h | 1 +
drivers/md/raid5-cache.c | 362 ++++++++---
drivers/md/raid5-log.h | 115 ++++
drivers/md/raid5-ppl.c | 1271 ++++++++++++++++++++++++++++++++++++++
drivers/md/raid5.c | 643 ++++++++++++-------
drivers/md/raid5.h | 106 +---
include/linux/bio.h | 11 +-
include/linux/percpu-refcount.h | 1 +
include/uapi/linux/raid/md_p.h | 45 +-
lib/percpu-refcount.c | 17 +
26 files changed, 3582 insertions(+), 1483 deletions(-)
create mode 100644 Documentation/md/raid5-ppl.txt
create mode 100644 drivers/md/raid5-log.h
create mode 100644 drivers/md/raid5-ppl.c
^ permalink raw reply
* Re: [PATCH] md/raid5: make use of spin_lock_irq over local_irq_disable + spin_lock
From: Shaohua Li @ 2017-05-01 23:11 UTC (permalink / raw)
To: Julia Cartwright
Cc: Sebastian Andrzej Siewior, linux-raid, linux-rt-users,
linux-kernel, Alexander GQ Gerasiov
In-Reply-To: <20170428174102.30003-1-julia@ni.com>
On Fri, Apr 28, 2017 at 12:41:02PM -0500, Julia Cartwright wrote:
> On mainline, there is no functional difference, just less code, and
> symmetric lock/unlock paths.
>
> On PREEMPT_RT builds, this fixes the following warning, seen by
> Alexander GQ Gerasiov, due to the sleeping nature of spinlocks.
>
> BUG: sleeping function called from invalid context at kernel/locking/rtmutex.c:993
> in_atomic(): 0, irqs_disabled(): 1, pid: 58, name: kworker/u12:1
> CPU: 5 PID: 58 Comm: kworker/u12:1 Tainted: G W 4.9.20-rt16-stand6-686 #1
> Hardware name: Supermicro SYS-5027R-WRF/X9SRW-F, BIOS 3.2a 10/28/2015
> Workqueue: writeback wb_workfn (flush-253:0)
> Call Trace:
> dump_stack+0x47/0x68
> ? migrate_enable+0x4a/0xf0
> ___might_sleep+0x101/0x180
> rt_spin_lock+0x17/0x40
> add_stripe_bio+0x4e3/0x6c0 [raid456]
> ? preempt_count_add+0x42/0xb0
> raid5_make_request+0x737/0xdd0 [raid456]
>
> Reported-by: Alexander GQ Gerasiov <gq@redlab-i.ru>
> Tested-by: Alexander GQ Gerasiov <gq@redlab-i.ru>
> Signed-off-by: Julia Cartwright <julia@ni.com>
applied, thanks!
> ---
> Hey All-
>
> While this fixes a problem on RT primarily, the patch is equally applicable
> upstream, as such probably makes sense to be pulled through the md tree. It
> may also make sense to be pulled directly into rt-devel.
>
> Alexander-
>
> I turned your "I confirm the fix" to a 'Tested-by', let me know if that's a problem.
>
> Thanks,
>
> Julia
>
> drivers/md/raid5.c | 17 +++++++----------
> 1 file changed, 7 insertions(+), 10 deletions(-)
>
> diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
> index fa2c4de32a64..54dc2995aeee 100644
> --- a/drivers/md/raid5.c
> +++ b/drivers/md/raid5.c
> @@ -110,8 +110,7 @@ static inline void unlock_device_hash_lock(struct r5conf *conf, int hash)
> static inline void lock_all_device_hash_locks_irq(struct r5conf *conf)
> {
> int i;
> - local_irq_disable();
> - spin_lock(conf->hash_locks);
> + spin_lock_irq(conf->hash_locks);
> for (i = 1; i < NR_STRIPE_HASH_LOCKS; i++)
> spin_lock_nest_lock(conf->hash_locks + i, conf->hash_locks);
> spin_lock(&conf->device_lock);
> @@ -121,9 +120,9 @@ static inline void unlock_all_device_hash_locks_irq(struct r5conf *conf)
> {
> int i;
> spin_unlock(&conf->device_lock);
> - for (i = NR_STRIPE_HASH_LOCKS; i; i--)
> - spin_unlock(conf->hash_locks + i - 1);
> - local_irq_enable();
> + for (i = NR_STRIPE_HASH_LOCKS - 1; i; i--)
> + spin_unlock(conf->hash_locks + i);
> + spin_unlock_irq(conf->hash_locks);
> }
>
> /* bio's attached to a stripe+device for I/O are linked together in bi_sector
> @@ -732,12 +731,11 @@ static bool is_full_stripe_write(struct stripe_head *sh)
>
> static void lock_two_stripes(struct stripe_head *sh1, struct stripe_head *sh2)
> {
> - local_irq_disable();
> if (sh1 > sh2) {
> - spin_lock(&sh2->stripe_lock);
> + spin_lock_irq(&sh2->stripe_lock);
> spin_lock_nested(&sh1->stripe_lock, 1);
> } else {
> - spin_lock(&sh1->stripe_lock);
> + spin_lock_irq(&sh1->stripe_lock);
> spin_lock_nested(&sh2->stripe_lock, 1);
> }
> }
> @@ -745,8 +743,7 @@ static void lock_two_stripes(struct stripe_head *sh1, struct stripe_head *sh2)
> static void unlock_two_stripes(struct stripe_head *sh1, struct stripe_head *sh2)
> {
> spin_unlock(&sh1->stripe_lock);
> - spin_unlock(&sh2->stripe_lock);
> - local_irq_enable();
> + spin_unlock_irq(&sh2->stripe_lock);
> }
>
> /* Only freshly new full stripe normal write stripe can be added to a batch list */
> --
> 2.12.0
>
^ permalink raw reply
* Re: Recovery on new 2TB disk: finish=7248.4min (raid1)
From: Nix @ 2017-05-01 23:26 UTC (permalink / raw)
To: Roman Mamedov; +Cc: John Stoffel, Mateusz Korniak, Ron Leach, linux-raid
In-Reply-To: <20170502024657.6d33fd88@natsu>
On 1 May 2017, Roman Mamedov outgrape:
> On Mon, 01 May 2017 22:13:59 +0100
> Nix <nix@esperi.org.uk> wrote:
>> I'd be more worried about the complexity required to just figure out the
>> space needed for half a dozen sets of lvmcache metadata and cache
>> filesystems.
>
> Metadata can be implicitly created and auto-managed in recent lvm versions
> http://fibrevillage.com/storage/460-how-to-create-lvm-cache-logical-volume
> ("Automatic pool metadata LV"). If not, the rule of thumb suggested everywhere
Useful.
> is 1/1000 of the cache volume size; I doubled that just in case, and looks
> like I didn't have to, as my metadata partitions are only about 9.5% full each.
Still seems like something that shouldn't need to be in a separate LV at
all (though .1% is not something to be worried about: it just seems
inelegant to me).
> As for half a dozen sets, I'd reconsider the need for those, as well as the
> entire fast/slow HDD tracks separation, just SSD-cache everything and let it
> figure out to not cache streaming writes on your video transcodes, or even bulk
> writes during your compile tests (while still caching the filesystem metadata).
>
> However the world of pain begins if you want to have multiple guest VMs each
> with its disk as a separate LV. One solution (that doesn't sound too clean but
> perhaps could work), is stacked LVM, i.e. a PV of a different volume group made
> on top of a cached LV.
Yeah. Or, as I have, multiple LVs for encrypted filesystems with
distinct passphrases and mount lifetimes, all of which I want to cache.
(And I want the cache to cache the *encrypted* data, obviously, but both
LVM and bcache will do that.)
I think LVM needs something like bcache's cache sets -- i.e. the ability
to have one cache for multiple LVs (this would be even more flexible
than doing it at the VG level, since you retain the current ability to
cache only some LVs without having to figure out how much cache each
needs).
--
NULL && (void)
^ permalink raw reply
* Re: [PATCH] md/raid10: skip spare disk as 'first' disk
From: Guoqing Jiang @ 2017-05-02 1:22 UTC (permalink / raw)
To: Shaohua Li, linux-raid; +Cc: NeilBrown
In-Reply-To: <7adb3f88bd6d611511ad14dcf8da6898e5f8168d.1493666464.git.shli@fb.com>
On 05/02/2017 03:22 AM, Shaohua Li wrote:
> Commit 6f287ca(md/raid10: reset the 'first' at the end of loop) ignores
> a case in reshape, the first rdev could be a spare disk, which shouldn't
> be accounted as the first disk since it doesn't include the offset info.
Acked-by: Guoqing Jiang <gqjiang@suse.com>
Thanks,
Guoqing
> Fix: 6f287ca(md/raid10: reset the 'first' at the end of loop)
> Cc: Guoqing Jiang <gqjiang@suse.com>
> Cc: NeilBrown <neilb@suse.com>
> Signed-off-by: Shaohua Li <shli@fb.com>
> ---
> drivers/md/raid10.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c
> index 2883b72..cce23be 100644
> --- a/drivers/md/raid10.c
> +++ b/drivers/md/raid10.c
> @@ -4079,8 +4079,8 @@ static int raid10_start_reshape(struct mddev *mddev)
> diff = 0;
> if (first || diff < min_offset_diff)
> min_offset_diff = diff;
> + first = 0;
> }
> - first = 0;
> }
>
> if (max(before_length, after_length) > min_offset_diff)
^ permalink raw reply
* Re: Big-endian RAID5 recovery problem
From: Phil Turmel @ 2017-05-02 2:47 UTC (permalink / raw)
To: Adam Thompson, Anthony Youngman; +Cc: MUUG Roundtable, linux-raid
In-Reply-To: <8aa82733243a09e48ba87d6282d8f791@mail.athompso.net>
On 05/01/2017 06:33 PM, Adam Thompson wrote:
> Spinning up a QEMU instance of Linux-PPC or Linux-MIPS with the disks in
> pass-through mode has also been mentioned, but... ugh. Anecdotal
> reports from the web suggest that doing so would just be opening up a
> second rabbit hole in addition to the one I'm already headed down.
Actually, this is a very good idea. If you can successfully assemble
within one of these VMs, you could use the --update=metadata option to
convert to v1.0 superblocks. Cleanly shut down, and then the arrays
would be usable in the host.
Phil
^ permalink raw reply
* [mdadm PATCH 0/2] unify stat and fstat into functions
From: Zhilong Liu @ 2017-05-02 3:26 UTC (permalink / raw)
To: Jes.Sorensen; +Cc: linux-raid, Zhilong Liu
Hi, Jes;
This two patches mainly unified the stat and fstat operations,
declare new function for them and decrease the repeated codes.
Also fix one bug in WaitClean(), it forgot to check the block
device.
Thanks,
Zhilong
Zhilong Liu (2):
util: unify fstat operations into function
util: unify stat operations into function
Assemble.c | 24 +++++++-----------------
Build.c | 30 +++++++-----------------------
Create.c | 23 ++++++++++-------------
Grow.c | 10 ++++------
Incremental.c | 52 +++++++++++++++-------------------------------------
Manage.c | 13 ++-----------
Monitor.c | 16 ++++------------
bitmap.c | 10 +---------
mdadm.h | 2 ++
super-ddf.c | 10 ++++------
super-intel.c | 23 +++++++----------------
util.c | 34 ++++++++++++++++++++++++++++++++++
12 files changed, 97 insertions(+), 150 deletions(-)
--
2.10.2
^ permalink raw reply
* [mdadm PATCH 1/2] util: unify fstat operations into function
From: Zhilong Liu @ 2017-05-02 3:26 UTC (permalink / raw)
To: Jes.Sorensen; +Cc: linux-raid, Zhilong Liu
In-Reply-To: <20170502032646.20220-1-zlliu@suse.com>
declare new function to integrate repeated fstat operation,
the fd and devname are necessary arguments, the dev_t *rdev
is optional. according to parse the pointer of dev_t *rdev,
if valid, assigned device number to rdev, if NULL, ignores.
Signed-off-by: Zhilong Liu <zlliu@suse.com>
---
Assemble.c | 17 +++++------------
Build.c | 5 +++--
Create.c | 23 ++++++++++-------------
Grow.c | 10 ++++------
Incremental.c | 33 ++++++++++++---------------------
Manage.c | 2 +-
bitmap.c | 10 +---------
mdadm.h | 1 +
super-intel.c | 13 +++----------
util.c | 17 +++++++++++++++++
10 files changed, 57 insertions(+), 74 deletions(-)
diff --git a/Assemble.c b/Assemble.c
index d6beb23..58bf96d 100644
--- a/Assemble.c
+++ b/Assemble.c
@@ -149,6 +149,7 @@ static int select_devices(struct mddev_dev *devlist,
struct mdinfo *content = NULL;
int report_mismatch = ((inargv && c->verbose >= 0) || c->verbose > 0);
struct domainlist *domains = NULL;
+ dev_t rdev;
tmpdev = devlist; num_devs = 0;
while (tmpdev) {
@@ -169,7 +170,6 @@ static int select_devices(struct mddev_dev *devlist,
tmpdev = tmpdev ? tmpdev->next : NULL) {
char *devname = tmpdev->devname;
int dfd;
- struct stat stb;
struct supertype *tst;
struct dev_policy *pol = NULL;
int found_container = 0;
@@ -204,14 +204,7 @@ static int select_devices(struct mddev_dev *devlist,
pr_err("cannot open device %s: %s\n",
devname, strerror(errno));
tmpdev->used = 2;
- } else if (fstat(dfd, &stb)< 0) {
- /* Impossible! */
- pr_err("fstat failed for %s: %s\n",
- devname, strerror(errno));
- tmpdev->used = 2;
- } else if ((stb.st_mode & S_IFMT) != S_IFBLK) {
- pr_err("%s is not a block device.\n",
- devname);
+ } else if (fstat_md_is_blkdev(dfd, devname, &rdev)){
tmpdev->used = 2;
} else if (must_be_container(dfd)) {
if (st) {
@@ -234,7 +227,7 @@ static int select_devices(struct mddev_dev *devlist,
devname);
tmpdev->used = 2;
} else if (auto_assem &&
- !conf_test_metadata(tst->ss->name, (pol = devid_policy(stb.st_rdev)),
+ !conf_test_metadata(tst->ss->name, (pol = devid_policy(rdev)),
tst->ss->match_home(tst, c->homehost) == 1)) {
if (report_mismatch)
pr_err("%s has metadata type %s for which auto-assembly is disabled\n",
@@ -261,7 +254,7 @@ static int select_devices(struct mddev_dev *devlist,
tst->ss->name, devname);
tmpdev->used = 2;
} else if (auto_assem && st == NULL &&
- !conf_test_metadata(tst->ss->name, (pol = devid_policy(stb.st_rdev)),
+ !conf_test_metadata(tst->ss->name, (pol = devid_policy(rdev)),
tst->ss->match_home(tst, c->homehost) == 1)) {
if (report_mismatch)
pr_err("%s has metadata type %s for which auto-assembly is disabled\n",
@@ -484,7 +477,7 @@ static int select_devices(struct mddev_dev *devlist,
/* Collect domain information from members only */
if (tmpdev && tmpdev->used == 1) {
if (!pol)
- pol = devid_policy(stb.st_rdev);
+ pol = devid_policy(rdev);
domain_merge(&domains, pol, tst?tst->ss->name:NULL);
}
dev_policy_free(pol);
diff --git a/Build.c b/Build.c
index 11ba12f..25aa4ea 100644
--- a/Build.c
+++ b/Build.c
@@ -42,6 +42,7 @@ int Build(char *mddev, struct mddev_dev *devlist,
*/
int i;
struct stat stb;
+ dev_t rdev;
int subdevs = 0, missing_disks = 0;
struct mddev_dev *dv;
int bitmap_fd;
@@ -126,8 +127,8 @@ int Build(char *mddev, struct mddev_dev *devlist,
array.nr_disks = s->raiddisks;
array.raid_disks = s->raiddisks;
array.md_minor = 0;
- if (fstat(mdfd, &stb) == 0)
- array.md_minor = minor(stb.st_rdev);
+ if (fstat_md_is_blkdev(mdfd, mddev, &rdev) == 0)
+ array.md_minor = minor(rdev);
array.not_persistent = 1;
array.state = 0; /* not clean, but no errors */
if (s->assume_clean)
diff --git a/Create.c b/Create.c
index 6ca0924..0f57257 100644
--- a/Create.c
+++ b/Create.c
@@ -89,8 +89,8 @@ int Create(struct supertype *st, char *mddev,
char *maxdisc = NULL;
int dnum, raid_disk_num;
struct mddev_dev *dv;
+ dev_t rdev;
int fail = 0, warn = 0;
- struct stat stb;
int first_missing = subdevs * 2;
int second_missing = subdevs * 2;
int missing_disks = 0;
@@ -325,11 +325,8 @@ int Create(struct supertype *st, char *mddev,
dname, strerror(errno));
exit(2);
}
- if (fstat(dfd, &stb) != 0 ||
- (stb.st_mode & S_IFMT) != S_IFBLK) {
+ if (fstat_md_is_blkdev(dfd, dname, NULL)) {
close(dfd);
- pr_err("%s is not a block device\n",
- dname);
exit(2);
}
close(dfd);
@@ -640,8 +637,8 @@ int Create(struct supertype *st, char *mddev,
* with, but it chooses to trust me instead. Sigh
*/
info.array.md_minor = 0;
- if (fstat(mdfd, &stb) == 0)
- info.array.md_minor = minor(stb.st_rdev);
+ if (fstat_md_is_blkdev(mdfd, mddev, &rdev) == 0)
+ info.array.md_minor = minor(rdev);
info.array.not_persistent = 0;
if (((s->level == 4 || s->level == 5) &&
@@ -840,7 +837,6 @@ int Create(struct supertype *st, char *mddev,
for (dnum = 0, raid_disk_num = 0, dv = devlist; dv;
dv = (dv->next) ? (dv->next) : moved_disk, dnum++) {
int fd;
- struct stat stb2;
struct mdinfo *inf = &infos[dnum];
if (dnum >= total_slots)
@@ -896,9 +892,10 @@ int Create(struct supertype *st, char *mddev,
dv->devname);
goto abort_locked;
}
- fstat(fd, &stb2);
- inf->disk.major = major(stb2.st_rdev);
- inf->disk.minor = minor(stb2.st_rdev);
+ if (fstat_md_is_blkdev(fd, dv->devname, &rdev))
+ return 1;
+ inf->disk.major = major(rdev);
+ inf->disk.minor = minor(rdev);
}
if (fd >= 0)
remove_partitions(fd);
@@ -919,8 +916,8 @@ int Create(struct supertype *st, char *mddev,
if (!have_container) {
/* getinfo_super might have lost these ... */
- inf->disk.major = major(stb2.st_rdev);
- inf->disk.minor = minor(stb2.st_rdev);
+ inf->disk.major = major(rdev);
+ inf->disk.minor = minor(rdev);
}
break;
case 2:
diff --git a/Grow.c b/Grow.c
index c6967ed..5949db0 100755
--- a/Grow.c
+++ b/Grow.c
@@ -109,7 +109,7 @@ int Grow_Add_device(char *devname, int fd, char *newdev)
*/
struct mdinfo info;
- struct stat stb;
+ dev_t rdev;
int nfd, fd2;
int d, nd;
struct supertype *st = NULL;
@@ -145,9 +145,7 @@ int Grow_Add_device(char *devname, int fd, char *newdev)
free(st);
return 1;
}
- fstat(nfd, &stb);
- if ((stb.st_mode & S_IFMT) != S_IFBLK) {
- pr_err("%s is not a block device!\n", newdev);
+ if (fstat_md_is_blkdev(nfd, newdev, &rdev)) {
close(nfd);
free(st);
return 1;
@@ -198,8 +196,8 @@ int Grow_Add_device(char *devname, int fd, char *newdev)
*/
info.disk.number = d;
- info.disk.major = major(stb.st_rdev);
- info.disk.minor = minor(stb.st_rdev);
+ info.disk.major = major(rdev);
+ info.disk.minor = minor(rdev);
info.disk.raid_disk = d;
info.disk.state = (1 << MD_DISK_SYNC) | (1 << MD_DISK_ACTIVE);
st->ss->update_super(st, &info, "linear-grow-new", newdev,
diff --git a/Incremental.c b/Incremental.c
index 28f1f77..6512af8 100644
--- a/Incremental.c
+++ b/Incremental.c
@@ -87,6 +87,7 @@ int Incremental(struct mddev_dev *devlist, struct context *c,
* start the array (auto-readonly).
*/
struct stat stb;
+ dev_t rdev;
struct mdinfo info, dinfo;
struct mdinfo *sra = NULL, *d;
struct mddev_ident *match;
@@ -175,21 +176,11 @@ int Incremental(struct mddev_dev *devlist, struct context *c,
/* 2/ Find metadata, reject if none appropriate (check
* version/name from args) */
- if (fstat(dfd, &stb) < 0) {
- if (c->verbose >= 0)
- pr_err("fstat failed for %s: %s.\n",
- devname, strerror(errno));
- goto out;
- }
- if ((stb.st_mode & S_IFMT) != S_IFBLK) {
- if (c->verbose >= 0)
- pr_err("%s is not a block device.\n",
- devname);
+ if (fstat_md_is_blkdev(dfd, devname, &rdev))
goto out;
- }
- dinfo.disk.major = major(stb.st_rdev);
- dinfo.disk.minor = minor(stb.st_rdev);
+ dinfo.disk.major = major(rdev);
+ dinfo.disk.minor = minor(rdev);
policy = disk_policy(&dinfo);
have_target = policy_check_path(&dinfo, &target_array);
@@ -341,8 +332,8 @@ int Incremental(struct mddev_dev *devlist, struct context *c,
}
dinfo = info;
- dinfo.disk.major = major(stb.st_rdev);
- dinfo.disk.minor = minor(stb.st_rdev);
+ dinfo.disk.major = major(rdev);
+ dinfo.disk.minor = minor(rdev);
if (add_disk(mdfd, st, &info, &dinfo) != 0) {
pr_err("failed to add %s to new array %s: %s.\n",
devname, chosen_name, strerror(errno));
@@ -444,8 +435,8 @@ int Incremental(struct mddev_dev *devlist, struct context *c,
goto out_unlock;
}
}
- info.disk.major = major(stb.st_rdev);
- info.disk.minor = minor(stb.st_rdev);
+ info.disk.major = major(rdev);
+ info.disk.minor = minor(rdev);
/* add disk needs to know about containers */
if (st->ss->external)
sra->array.level = LEVEL_CONTAINER;
@@ -868,12 +859,12 @@ static int array_try_spare(char *devname, int *dfdp, struct dev_policy *pol,
* Return 0 on success, or some exit code on failure, probably 1.
*/
int rv = 1;
- struct stat stb;
+ dev_t rdev;
struct map_ent *mp, *map = NULL;
struct mdinfo *chosen = NULL;
int dfd = *dfdp;
- if (fstat(dfd, &stb) != 0)
+ if (fstat_md_is_blkdev(dfd, devname, &rdev))
return 1;
/*
@@ -1045,8 +1036,8 @@ static int array_try_spare(char *devname, int *dfdp, struct dev_policy *pol,
devlist.writemostly = FlagDefault;
devlist.failfast = FlagDefault;
devlist.devname = chosen_devname;
- sprintf(chosen_devname, "%d:%d", major(stb.st_rdev),
- minor(stb.st_rdev));
+ sprintf(chosen_devname, "%d:%d", major(rdev),
+ minor(rdev));
devlist.disposition = 'a';
close(dfd);
*dfdp = -1;
diff --git a/Manage.c b/Manage.c
index 8966e33..69c68ba 100644
--- a/Manage.c
+++ b/Manage.c
@@ -1515,7 +1515,7 @@ int Manage_subdevs(char *devname, int fd,
struct stat stb;
tfd = dev_open(dv->devname, O_RDONLY);
if (tfd >= 0) {
- fstat(tfd, &stb);
+ fstat_md_is_blkdev(tfd, dv->devname, &rdev);
close(tfd);
} else {
int open_err = errno;
diff --git a/bitmap.c b/bitmap.c
index 16a6b73..2cf7938 100644
--- a/bitmap.c
+++ b/bitmap.c
@@ -183,7 +183,6 @@ static int
bitmap_file_open(char *filename, struct supertype **stp, int node_num)
{
int fd;
- struct stat stb;
struct supertype *st = *stp;
fd = open(filename, O_RDONLY|O_DIRECT);
@@ -193,14 +192,7 @@ bitmap_file_open(char *filename, struct supertype **stp, int node_num)
return -1;
}
- if (fstat(fd, &stb) < 0) {
- pr_err("failed to determine bitmap file/device type: %s\n",
- strerror(errno));
- close(fd);
- return -1;
- }
-
- if ((stb.st_mode & S_IFMT) == S_IFBLK) {
+ if (fstat_md_is_blkdev(fd, filename, NULL) == 0) {
/* block device, so we are probably after an internal bitmap */
if (!st)
st = guess_super(fd);
diff --git a/mdadm.h b/mdadm.h
index 1bbacfe..1d63932 100644
--- a/mdadm.h
+++ b/mdadm.h
@@ -1433,6 +1433,7 @@ extern int check_raid(int fd, char *name);
extern int check_partitions(int fd, char *dname,
unsigned long long freesize,
unsigned long long size);
+extern int fstat_md_is_blkdev(int fd, char *devname, dev_t *rdev);
extern int get_mdp_major(void);
extern int get_maj_min(char *dev, int *major, int *minor);
diff --git a/super-intel.c b/super-intel.c
index 0aed57c..c16edfb 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -6557,7 +6557,7 @@ count_volumes_list(struct md_list *devlist, char *homehost,
for (tmpdev = devlist; tmpdev; tmpdev = tmpdev->next) {
char *devname = tmpdev->devname;
- struct stat stb;
+ dev_t rdev;
struct supertype *tst;
int dfd;
if (tmpdev->used > 1)
@@ -6573,14 +6573,7 @@ count_volumes_list(struct md_list *devlist, char *homehost,
dprintf("cannot open device %s: %s\n",
devname, strerror(errno));
tmpdev->used = 2;
- } else if (fstat(dfd, &stb)< 0) {
- /* Impossible! */
- dprintf("fstat failed for %s: %s\n",
- devname, strerror(errno));
- tmpdev->used = 2;
- } else if ((stb.st_mode & S_IFMT) != S_IFBLK) {
- dprintf("%s is not a block device.\n",
- devname);
+ } else if (fstat_md_is_blkdev(dfd, devname, &rdev)) {
tmpdev->used = 2;
} else if (must_be_container(dfd)) {
struct supertype *cst;
@@ -6602,7 +6595,7 @@ count_volumes_list(struct md_list *devlist, char *homehost,
if (cst)
cst->ss->free_super(cst);
} else {
- tmpdev->st_rdev = stb.st_rdev;
+ tmpdev->st_rdev = rdev;
if (tst->ss->load_super(tst,dfd, NULL)) {
dprintf("no RAID superblock on %s\n",
devname);
diff --git a/util.c b/util.c
index 21a63c9..6294fff 100644
--- a/util.c
+++ b/util.c
@@ -706,6 +706,23 @@ int check_raid(int fd, char *name)
return 1;
}
+int fstat_md_is_blkdev(int fd, char *devname, dev_t *rdev)
+{
+ struct stat stb;
+
+ if (fstat(fd, &stb) != 0) {
+ pr_err("fstat failed for %s: %s\n", devname, strerror(errno));
+ return 1;
+ }
+ if ((S_IFMT & stb.st_mode) != S_IFBLK) {
+ pr_err("%s is not a block device.\n", devname);
+ return 1;
+ }
+ if (rdev)
+ *rdev = stb.st_rdev;
+ return 0;
+}
+
int ask(char *mesg)
{
char *add = "";
--
2.10.2
^ permalink raw reply related
* [mdadm PATCH 2/2] util: unify stat operations into function
From: Zhilong Liu @ 2017-05-02 3:26 UTC (permalink / raw)
To: Jes.Sorensen; +Cc: linux-raid, Zhilong Liu
In-Reply-To: <20170502032646.20220-1-zlliu@suse.com>
declare new function to integrate repeated stat operations,
the devname is necessary and the *rdev is optional argument,
parse the pointer of dev_t *rdev, if valid, assigned device
number to rdev, if NULL, ignores.
Signed-off-by: Zhilong Liu <zlliu@suse.com>
---
Assemble.c | 7 ++-----
Build.c | 25 ++++---------------------
Incremental.c | 21 ++++-----------------
Manage.c | 11 +----------
Monitor.c | 16 ++++------------
mdadm.h | 1 +
super-ddf.c | 10 ++++------
super-intel.c | 10 ++++------
util.c | 17 +++++++++++++++++
9 files changed, 41 insertions(+), 77 deletions(-)
diff --git a/Assemble.c b/Assemble.c
index 58bf96d..55267cc 100644
--- a/Assemble.c
+++ b/Assemble.c
@@ -510,15 +510,12 @@ static int select_devices(struct mddev_dev *devlist,
/* Now reject spares that don't match domains of identified members */
for (tmpdev = devlist; tmpdev; tmpdev = tmpdev->next) {
- struct stat stb;
if (tmpdev->used != 3)
continue;
- if (stat(tmpdev->devname, &stb)< 0) {
- pr_err("fstat failed for %s: %s\n",
- tmpdev->devname, strerror(errno));
+ if (stat_md_is_blkdev(tmpdev->devname, &rdev)) {
tmpdev->used = 2;
} else {
- struct dev_policy *pol = devid_policy(stb.st_rdev);
+ struct dev_policy *pol = devid_policy(rdev);
int dt = domain_test(domains, pol, NULL);
if (inargv && dt != 0)
/* take this spare as domains match
diff --git a/Build.c b/Build.c
index 25aa4ea..0dcfa48 100644
--- a/Build.c
+++ b/Build.c
@@ -41,7 +41,6 @@ int Build(char *mddev, struct mddev_dev *devlist,
* cc = chunk size factor: 0==4k, 1==8k etc.
*/
int i;
- struct stat stb;
dev_t rdev;
int subdevs = 0, missing_disks = 0;
struct mddev_dev *dv;
@@ -65,16 +64,8 @@ int Build(char *mddev, struct mddev_dev *devlist,
missing_disks++;
continue;
}
- if (stat(dv->devname, &stb)) {
- pr_err("Cannot find %s: %s\n",
- dv->devname, strerror(errno));
- return 1;
- }
- if ((stb.st_mode & S_IFMT) != S_IFBLK) {
- pr_err("%s is not a block device.\n",
- dv->devname);
+ if (stat_md_is_blkdev(dv->devname, NULL))
return 1;
- }
}
if (s->raiddisks != subdevs) {
@@ -162,16 +153,8 @@ int Build(char *mddev, struct mddev_dev *devlist,
if (strcmp("missing", dv->devname) == 0)
continue;
- if (stat(dv->devname, &stb)) {
- pr_err("Weird: %s has disappeared.\n",
- dv->devname);
+ if (stat_md_is_blkdev(dv->devname, &rdev))
goto abort;
- }
- if ((stb.st_mode & S_IFMT)!= S_IFBLK) {
- pr_err("Weird: %s is no longer a block device.\n",
- dv->devname);
- goto abort;
- }
fd = open(dv->devname, O_RDONLY|O_EXCL);
if (fd < 0) {
pr_err("Cannot open %s: %s\n",
@@ -187,8 +170,8 @@ int Build(char *mddev, struct mddev_dev *devlist,
disk.state = (1<<MD_DISK_SYNC) | (1<<MD_DISK_ACTIVE);
if (dv->writemostly == FlagSet)
disk.state |= 1<<MD_DISK_WRITEMOSTLY;
- disk.major = major(stb.st_rdev);
- disk.minor = minor(stb.st_rdev);
+ disk.major = major(rdev);
+ disk.minor = minor(rdev);
if (ioctl(mdfd, ADD_NEW_DISK, &disk)) {
pr_err("ADD_NEW_DISK failed for %s: %s\n",
dv->devname, strerror(errno));
diff --git a/Incremental.c b/Incremental.c
index 6512af8..772461e 100644
--- a/Incremental.c
+++ b/Incremental.c
@@ -86,8 +86,7 @@ int Incremental(struct mddev_dev *devlist, struct context *c,
* - if number of OK devices match expected, or -R and there are enough,
* start the array (auto-readonly).
*/
- struct stat stb;
- dev_t rdev;
+ dev_t rdev, rdev2;
struct mdinfo info, dinfo;
struct mdinfo *sra = NULL, *d;
struct mddev_ident *match;
@@ -109,18 +108,8 @@ int Incremental(struct mddev_dev *devlist, struct context *c,
struct createinfo *ci = conf_get_create_info();
- if (stat(devname, &stb) < 0) {
- if (c->verbose >= 0)
- pr_err("stat failed for %s: %s.\n",
- devname, strerror(errno));
- return rv;
- }
- if ((stb.st_mode & S_IFMT) != S_IFBLK) {
- if (c->verbose >= 0)
- pr_err("%s is not a block device.\n",
- devname);
+ if (stat_md_is_blkdev(devname, &rdev))
return rv;
- }
dfd = dev_open(devname, O_RDONLY);
if (dfd < 0) {
if (c->verbose >= 0)
@@ -159,10 +148,8 @@ int Incremental(struct mddev_dev *devlist, struct context *c,
if (!devlist) {
devlist = conf_get_devs();
for (;devlist; devlist = devlist->next) {
- struct stat st2;
- if (stat(devlist->devname, &st2) == 0 &&
- (st2.st_mode & S_IFMT) == S_IFBLK &&
- st2.st_rdev == stb.st_rdev)
+ if (stat_md_is_blkdev(devlist->devname, &rdev2) == 0 &&
+ rdev2 == rdev)
break;
}
}
diff --git a/Manage.c b/Manage.c
index 69c68ba..56ad8ce 100644
--- a/Manage.c
+++ b/Manage.c
@@ -1512,24 +1512,16 @@ int Manage_subdevs(char *devname, int fd,
*/
rdev = makedev(mj, mn);
} else {
- struct stat stb;
tfd = dev_open(dv->devname, O_RDONLY);
if (tfd >= 0) {
fstat_md_is_blkdev(tfd, dv->devname, &rdev);
close(tfd);
} else {
int open_err = errno;
- if (stat(dv->devname, &stb) != 0) {
- pr_err("Cannot find %s: %s\n",
- dv->devname, strerror(errno));
- goto abort;
- }
- if ((stb.st_mode & S_IFMT) != S_IFBLK) {
+ if (stat_md_is_blkdev(dv->devname, &rdev)) {
if (dv->disposition == 'M')
/* non-fatal. Also improbable */
continue;
- pr_err("%s is not a block device.\n",
- dv->devname);
goto abort;
}
if (dv->disposition == 'r')
@@ -1546,7 +1538,6 @@ int Manage_subdevs(char *devname, int fd,
goto abort;
}
}
- rdev = stb.st_rdev;
}
switch(dv->disposition){
default:
diff --git a/Monitor.c b/Monitor.c
index 1f15377..78363dd 100644
--- a/Monitor.c
+++ b/Monitor.c
@@ -993,23 +993,13 @@ static void link_containers_with_subarrays(struct state *list)
/* Not really Monitor but ... */
int Wait(char *dev)
{
- struct stat stb;
char devnm[32];
- char *tmp;
int rv = 1;
int frozen_remaining = 3;
- if (stat(dev, &stb) != 0) {
- pr_err("Cannot find %s: %s\n", dev,
- strerror(errno));
+ if (stat_md_is_blkdev(dev, NULL))
return 2;
- }
- tmp = stat2devnm(&stb);
- if (!tmp) {
- pr_err("%s is not a block device.\n", dev);
- return 2;
- }
- strcpy(devnm, tmp);
+ strcpy(devnm, dev);
while(1) {
struct mdstat_ent *ms = mdstat_read(1, 0);
@@ -1068,6 +1058,8 @@ int WaitClean(char *dev, int sock, int verbose)
int rv = 1;
char devnm[32];
+ if (stat_md_is_blkdev(dev, NULL))
+ return 2;
fd = open(dev, O_RDONLY);
if (fd < 0) {
if (verbose)
diff --git a/mdadm.h b/mdadm.h
index 1d63932..f45e5f4 100644
--- a/mdadm.h
+++ b/mdadm.h
@@ -1434,6 +1434,7 @@ extern int check_partitions(int fd, char *dname,
unsigned long long freesize,
unsigned long long size);
extern int fstat_md_is_blkdev(int fd, char *devname, dev_t *rdev);
+extern int stat_md_is_blkdev(char *devname, dev_t *rdev);
extern int get_mdp_major(void);
extern int get_maj_min(char *dev, int *major, int *minor);
diff --git a/super-ddf.c b/super-ddf.c
index 796eaa5..ad77464 100644
--- a/super-ddf.c
+++ b/super-ddf.c
@@ -3490,7 +3490,7 @@ static int validate_geometry_ddf_bvd(struct supertype *st,
char *dev, unsigned long long *freesize,
int verbose)
{
- struct stat stb;
+ dev_t rdev;
struct ddf_super *ddf = st->sb;
struct dl *dl;
unsigned long long maxsize;
@@ -3526,13 +3526,11 @@ static int validate_geometry_ddf_bvd(struct supertype *st,
return 1;
}
/* This device must be a member of the set */
- if (stat(dev, &stb) < 0)
- return 0;
- if ((S_IFMT & stb.st_mode) != S_IFBLK)
+ if (stat_md_is_blkdev(dev, NULL))
return 0;
for (dl = ddf->dlist ; dl ; dl = dl->next) {
- if (dl->major == (int)major(stb.st_rdev) &&
- dl->minor == (int)minor(stb.st_rdev))
+ if (dl->major == (int)major(rdev) &&
+ dl->minor == (int)minor(rdev))
break;
}
if (!dl) {
diff --git a/super-intel.c b/super-intel.c
index c16edfb..0fef01e 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -6850,7 +6850,7 @@ static int validate_geometry_imsm_volume(struct supertype *st, int level,
unsigned long long *freesize,
int verbose)
{
- struct stat stb;
+ dev_t rdev;
struct intel_super *super = st->sb;
struct imsm_super *mpb;
struct dl *dl;
@@ -6915,13 +6915,11 @@ static int validate_geometry_imsm_volume(struct supertype *st, int level,
}
/* This device must be a member of the set */
- if (stat(dev, &stb) < 0)
- return 0;
- if ((S_IFMT & stb.st_mode) != S_IFBLK)
+ if (stat_md_is_blkdev(dev, &rdev))
return 0;
for (dl = super->disks ; dl ; dl = dl->next) {
- if (dl->major == (int)major(stb.st_rdev) &&
- dl->minor == (int)minor(stb.st_rdev))
+ if (dl->major == (int)major(rdev) &&
+ dl->minor == (int)minor(rdev))
break;
}
if (!dl) {
diff --git a/util.c b/util.c
index 6294fff..c466078 100644
--- a/util.c
+++ b/util.c
@@ -723,6 +723,23 @@ int fstat_md_is_blkdev(int fd, char *devname, dev_t *rdev)
return 0;
}
+int stat_md_is_blkdev(char *devname, dev_t *rdev)
+{
+ struct stat stb;
+
+ if (stat(devname, &stb) != 0) {
+ pr_err("stat failed for %s: %s\n", devname, strerror(errno));
+ return 1;
+ }
+ if ((S_IFMT & stb.st_mode) != S_IFBLK) {
+ pr_err("%s is not a block device.\n", devname);
+ return 1;
+ }
+ if (rdev)
+ *rdev = stb.st_rdev;
+ return 0;
+}
+
int ask(char *mesg)
{
char *add = "";
--
2.10.2
^ permalink raw reply related
* Re: Recovery on new 2TB disk: finish=7248.4min (raid1)
From: Wols Lists @ 2017-05-02 6:10 UTC (permalink / raw)
To: Roman Mamedov; +Cc: Nix, John Stoffel, Mateusz Korniak, Ron Leach, linux-raid
In-Reply-To: <20170502030307.4d7a77dd@natsu>
On 01/05/17 23:03, Roman Mamedov wrote:
>> > That's what I understood you to mean, but you are aware that SOME raid
>> > management still has to be done with echo > /sys/... ?
>> >
>> > So mdadm isn't perfect, not by a long chalk, yet :-)
> Well, why not post some examples of what you find yourself doing often
> via /sys, that's not available in mdadm (maybe as a new thread).
I *should* do, rather than I *do* do, but your everyday general
maintenance tasks, like scrubbing?
I'm not aware of *any* day-to-day tasks that you do using mdadm - oh and
if you don't scrub it's not an uncommon cause of arrays crashing...
Cheers,
Wol
^ permalink raw reply
* Re: [PATCH 25/27] block: remove the discard_zeroes_data flag
From: Nicholas A. Bellinger @ 2017-05-02 6:43 UTC (permalink / raw)
To: Bart Van Assche
Cc: axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org,
linux-raid-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
snitzer-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org,
martin.petersen-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org,
philipp.reisner-63ez5xqkn6DQT0dZR+AlfA@public.gmane.org,
linux-block-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
dm-devel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org,
target-devel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-scsi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
lars.ellenberg-63ez5xqkn6DQT0dZR+AlfA@public.gmane.org,
shli-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org,
hch-jcswGhMUV9g@public.gmane.org,
agk-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org,
drbd-dev-cunTk1MwBs8qoQakbn7OcQ@public.gmane.org
In-Reply-To: <1493671519.2665.15.camel-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org>
On Mon, 2017-05-01 at 20:45 +0000, Bart Van Assche wrote:
> On Wed, 2017-04-05 at 19:21 +0200, Christoph Hellwig wrote:
> > Now that we use the proper REQ_OP_WRITE_ZEROES operation everywhere we can
> > kill this hack.
> >
> > Signed-off-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>
> > Reviewed-by: Martin K. Petersen <martin.petersen-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
> > Reviewed-by: Hannes Reinecke <hare-IBi9RG/b67k@public.gmane.org>
> > [ ... ]
> > diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c
> > index c754ae33bf7b..d2f089cfa9ae 100644
> > --- a/drivers/target/target_core_device.c
> > +++ b/drivers/target/target_core_device.c
> > @@ -851,7 +851,7 @@ bool target_configure_unmap_from_queue(struct se_dev_attrib *attrib,
> > attrib->unmap_granularity = q->limits.discard_granularity / block_size;
> > attrib->unmap_granularity_alignment = q->limits.discard_alignment /
> > block_size;
> > - attrib->unmap_zeroes_data = q->limits.discard_zeroes_data;
> > + attrib->unmap_zeroes_data = 0;
> > return true;
> > }
> > EXPORT_SYMBOL(target_configure_unmap_from_queue);
>
> Hello Christoph,
>
> Sorry that I hadn't noticed this before but I think that this patch
> introduces a significant performance regressions for LIO users. Before
> this patch the LBPRZ flag was reported correctly to initiator systems
> through the thin provisioning VPD. With this patch applied that flag
> will always be reported as zero, forcing initiators to submit WRITE
> commands with zeroed data buffers instead of submitting the SCSI UNMAP
> command to block devices for which discard_zeroes_data was set. From
> target_core_spc.c:
>
> /* Thin Provisioning VPD */
> static sense_reason_t spc_emulate_evpd_b2(struct se_cmd *cmd, unsigned char *buf)
> {
> [ ... ]
> /*
> * The unmap_zeroes_data set means that the underlying device supports
> * REQ_DISCARD and has the discard_zeroes_data bit set. This satisfies
> * the SBC requirements for LBPRZ, meaning that a subsequent read
> * will return zeroes after an UNMAP or WRITE SAME (16) to an LBA
> * See sbc4r36 6.6.4.
> */
> if (((dev->dev_attrib.emulate_tpu != 0) ||
> (dev->dev_attrib.emulate_tpws != 0)) &&
> (dev->dev_attrib.unmap_zeroes_data != 0))
> buf[5] |= 0x04;
> [ ... ]
> }
>
According to sd_config_discard(), it's SD_LBP_WS16, SD_LBP_WS10 and
SD_LBP_ZERO that where ever setting unmap_zeros_data = 1 to begin with.
For UNMAP, q->limits.discard_zeroes_data was never set.
That said, it's pretty much implied that supporting DISCARD means
subsequent READs return zeros, so target_configure_unmap_from_queue()
should be setting attrib->unmap_zeroes_data = 1, or just dropping it
all-together.
Post -rc1, I'll push a patch to do the latter.
^ permalink raw reply
* Re: [PATCH 25/27] block: remove the discard_zeroes_data flag
From: Nicholas A. Bellinger @ 2017-05-02 7:16 UTC (permalink / raw)
To: Bart Van Assche
Cc: axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org,
linux-raid-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
snitzer-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org,
martin.petersen-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org,
philipp.reisner-63ez5xqkn6DQT0dZR+AlfA@public.gmane.org,
linux-block-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
dm-devel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org,
target-devel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-scsi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
lars.ellenberg-63ez5xqkn6DQT0dZR+AlfA@public.gmane.org,
shli-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org,
hch-jcswGhMUV9g@public.gmane.org,
agk-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org,
drbd-dev-cunTk1MwBs8qoQakbn7OcQ@public.gmane.org
In-Reply-To: <1493707425.23202.77.camel-XoQW25Eq2zviZyQQd+hFbcojREIfoBdhmpATvIKMPHk@public.gmane.org>
On Mon, 2017-05-01 at 23:43 -0700, Nicholas A. Bellinger wrote:
> On Mon, 2017-05-01 at 20:45 +0000, Bart Van Assche wrote:
> > On Wed, 2017-04-05 at 19:21 +0200, Christoph Hellwig wrote:
> > > Now that we use the proper REQ_OP_WRITE_ZEROES operation everywhere we can
> > > kill this hack.
> > >
> > > Signed-off-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>
> > > Reviewed-by: Martin K. Petersen <martin.petersen-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
> > > Reviewed-by: Hannes Reinecke <hare-IBi9RG/b67k@public.gmane.org>
> > > [ ... ]
> > > diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c
> > > index c754ae33bf7b..d2f089cfa9ae 100644
> > > --- a/drivers/target/target_core_device.c
> > > +++ b/drivers/target/target_core_device.c
> > > @@ -851,7 +851,7 @@ bool target_configure_unmap_from_queue(struct se_dev_attrib *attrib,
> > > attrib->unmap_granularity = q->limits.discard_granularity / block_size;
> > > attrib->unmap_granularity_alignment = q->limits.discard_alignment /
> > > block_size;
> > > - attrib->unmap_zeroes_data = q->limits.discard_zeroes_data;
> > > + attrib->unmap_zeroes_data = 0;
> > > return true;
> > > }
> > > EXPORT_SYMBOL(target_configure_unmap_from_queue);
> >
> > Hello Christoph,
> >
> > Sorry that I hadn't noticed this before but I think that this patch
> > introduces a significant performance regressions for LIO users. Before
> > this patch the LBPRZ flag was reported correctly to initiator systems
> > through the thin provisioning VPD. With this patch applied that flag
> > will always be reported as zero, forcing initiators to submit WRITE
> > commands with zeroed data buffers instead of submitting the SCSI UNMAP
> > command to block devices for which discard_zeroes_data was set. From
> > target_core_spc.c:
> >
> > /* Thin Provisioning VPD */
> > static sense_reason_t spc_emulate_evpd_b2(struct se_cmd *cmd, unsigned char *buf)
> > {
> > [ ... ]
> > /*
> > * The unmap_zeroes_data set means that the underlying device supports
> > * REQ_DISCARD and has the discard_zeroes_data bit set. This satisfies
> > * the SBC requirements for LBPRZ, meaning that a subsequent read
> > * will return zeroes after an UNMAP or WRITE SAME (16) to an LBA
> > * See sbc4r36 6.6.4.
> > */
> > if (((dev->dev_attrib.emulate_tpu != 0) ||
> > (dev->dev_attrib.emulate_tpws != 0)) &&
> > (dev->dev_attrib.unmap_zeroes_data != 0))
> > buf[5] |= 0x04;
> > [ ... ]
> > }
> >
>
> According to sd_config_discard(), it's SD_LBP_WS16, SD_LBP_WS10 and
> SD_LBP_ZERO that where ever setting unmap_zeros_data = 1 to begin with.
> For UNMAP, q->limits.discard_zeroes_data was never set.
>
> That said, it's pretty much implied that supporting DISCARD means
> subsequent READs return zeros, so target_configure_unmap_from_queue()
> should be setting attrib->unmap_zeroes_data = 1, or just dropping it
> all-together.
>
> Post -rc1, I'll push a patch to do the latter.
>
Or, another options is use bdev_write_zeroes_sectors() to determine when
dev_attrib->unmap_zeroes_data should be set.
^ permalink raw reply
* Re: [PATCH 25/27] block: remove the discard_zeroes_data flag
From: hch-jcswGhMUV9g @ 2017-05-02 7:23 UTC (permalink / raw)
To: Nicholas A. Bellinger
Cc: axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org,
linux-raid-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
snitzer-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org,
martin.petersen-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org,
philipp.reisner-63ez5xqkn6DQT0dZR+AlfA@public.gmane.org,
linux-block-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
dm-devel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org,
target-devel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-scsi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
Bart Van Assche,
lars.ellenberg-63ez5xqkn6DQT0dZR+AlfA@public.gmane.org,
shli-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org,
hch-jcswGhMUV9g@public.gmane.org,
agk-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org,
drbd-dev-cunTk1MwBs8qoQakbn7OcQ@public.gmane.org
In-Reply-To: <1493709373.23202.79.camel-XoQW25Eq2zviZyQQd+hFbcojREIfoBdhmpATvIKMPHk@public.gmane.org>
On Tue, May 02, 2017 at 12:16:13AM -0700, Nicholas A. Bellinger wrote:
> Or, another options is use bdev_write_zeroes_sectors() to determine when
> dev_attrib->unmap_zeroes_data should be set.
Yes, that in combination with your patch to use bdev_write_zeroes_sectors
for zeroing from write same seems like the right fix.
^ permalink raw reply
* Re: [RndTbl] Big-endian RAID5 recovery problem
From: Trevor Cordes @ 2017-05-02 7:29 UTC (permalink / raw)
To: Adam Thompson; +Cc: Continuation of Round Table discussion, linux-raid
In-Reply-To: <5c6e6e5d93d4056839e4f370e00a8e08@mail.athompso.net>
On 2017-05-01 Adam Thompson wrote:
> The problem is that the disks use the v0.90 metadata format, and they
> came from a big-endian system, not a little-endian system. MD
> superblocks *since* v0.90 are endian-agnostic, but back in v0.90, the
> superblock was byte-order specific.
This may sound crazy, but conceivably you could (using the md source
code) find where the superblock lives (just the first copy should be
enough), hex edit the superblock and (again using the source) swap
bytes around to change the endian-ness. Then make sure any future
mdadm commands are just using the first superblock (which I think is
the default).
If there's some kind of checksum involved, then the endian-ness may
affect it (??), and fudging that may be quite difficult (compared to
finding a big-endian box out there to borrow instead!).
No matter what you do, I would instantly dd each real disk into files
in your own big fs and work on the copies. No reason to ever write to
the originals, keep them as backups. I *think* you can use mdadm on
plain files(??) otherwise I guess you could subpartition a big disk.
Ya, it will be cheesy to run RAID5 all off of 1 disk, but in this case
performance won't be critical.
Lastly, I would make sure to start the array in such a way that for
sure it won't resync, so either get all 3 preset perfectly, or leave 1
as missing, or I think there's a command to not start if it would
result in a dirty array.
^ permalink raw reply
* 40335 linux-raid
From: citydesk @ 2017-05-02 8:11 UTC (permalink / raw)
To: linux-raid
[-- Attachment #1: 6794.zip --]
[-- Type: application/zip, Size: 4207 bytes --]
^ permalink raw reply
* Re: Big-endian RAID5 recovery problem
From: Roman Mamedov @ 2017-05-02 8:59 UTC (permalink / raw)
To: Adam Thompson; +Cc: MUUG Roundtable, linux-raid
In-Reply-To: <5c6e6e5d93d4056839e4f370e00a8e08@mail.athompso.net>
On Mon, 01 May 2017 16:39:07 -0500
Adam Thompson <athompso@athompso.net> wrote:
> I can't find any modern big-endian Linux systems... looks like all the
> ARM distros run in little-endian mode.
Here are QEMU images for debian-mips (should be big-endian, as opposed to
debian-mipsel): https://people.debian.org/~aurel32/qemu/mips/
Of course it will run purely in software, but most likely more than fast enough
to copy away the data.
Not entirely sure that particular emulated MIPS system support more than 4
drives), but it appears that a starting point could be (man qemu-system):
Instead of -hda, -hdb, -hdc, -hdd, you can use:
qemu-system-i386 -drive file=file,index=0,media=disk
qemu-system-i386 -drive file=file,index=1,media=disk
qemu-system-i386 -drive file=file,index=2,media=disk
qemu-system-i386 -drive file=file,index=3,media=disk
with indexes 0..5, as you need the boot disk, all 4 drives, and one more as
the backup destination.
May or may not be the best way, but IMO beats trying to hex-edit the
superblock right away.
--
With respect,
Roman
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox