* Re: [PATCH] mdadm: Check bitmap first when reshape raid1 to raid0
From: Xiao Ni @ 2015-10-27 6:24 UTC (permalink / raw)
To: Neil Brown; +Cc: Jes Sorensen, linux-raid
In-Reply-To: <877fm9dy5i.fsf@notabene.neil.brown.name>
----- Original Message -----
> From: "Neil Brown" <neilb@suse.de>
> To: "Xiao Ni" <xni@redhat.com>
> Cc: "Jes Sorensen" <jes.sorensen@redhat.com>, linux-raid@vger.kernel.org
> Sent: Tuesday, October 27, 2015 7:10:17 AM
> Subject: Re: [PATCH] mdadm: Check bitmap first when reshape raid1 to raid0
>
> On Mon, Oct 26 2015, Xiao Ni wrote:
>
> >>
> >
> > Thanks for printing out the mistake. I checked the nearby code and found
> > I missed close the cfd. Is this I missed? But I'm wondering that it doesn't
> > close it in the following code. Now it's just closed when it's failed to
> > reshape raid1/raid10 to raid0.
>
> That is a very sensible question to ask - thanks.
> After a quick look - it does seem very strange. That handling of 'cfd'
> and 'fd' seems quite odd.
>
> If you would like to work out what should happen and find a simple way
> to fix the code, that would be great. But I don't insist.
I'll try it and re-send a patch then.
>
> The patch below is quite acceptable as it is a simple solution to a
> simple problem and appear consistent with nearby code. So if you gave
> it a proper description, formatted it properly and posted it in a way
> that didn't mess up all the white space, I would very likely accept it.
>
> However:
> - maybe we should just remove the bitmap rather than complain if we
> find one. After all, we are removing other things (extra devices).
> A raid0 can never had a bitmap, so when converting to a raid0, it
> does make sense to remove the bitmap.
Yes, :) it's better to remove the bitmap than check it.
>
> - The test you used only checks for an internal bitmap, not an external
> one. External bitmaps are hardly ever used except by people who know
> exactly what they are doing, so I'm not too fussed about handling
> them perfectly, but testing for and removing an external bitmap
> might make sense.
>
> Thanks,
> NeilBrown
As you said, I haven't used external bitmaps before. I'll check how to handle
them and add it to the patch too.
Best Regards
Xiao
^ permalink raw reply
* Re: Converting 4 disk RAID10 to RAID5
From: Adam Goryachev @ 2015-10-27 6:32 UTC (permalink / raw)
To: Anugraha Sinha, Phil Turmel, linux-raid
In-Reply-To: <562F1789.9080000@gmail.com>
On 27/10/15 17:19, Anugraha Sinha wrote:
> Dear Adam,
>
> On 10/27/2015 8:55 AM, Adam Goryachev wrote:
>>
>> mdadm --grow --bitmap=none /dev/md0
>> root@testraid:~# cat /proc/mdstat
>> Personalities : [raid10] [raid0] [raid6] [raid5] [raid4]
>> md0 : active raid5 vdf1[4] vdd1[3](S) vde1[2] vdc1[0]
>> 2093056 blocks super 1.2 level 5, 512k chunk, algorithm 5 [3/3]
>> [UUU]
>>
>> unused devices: <none>
>>
>> So, still 3 disk raid5 with one spare, but seems to be insync, so either
>> it was really quick (possible since they are small drives) or it didn't
>> need to do a sync??
>>
>> mdadm --grow --level=5 --raid-devices=4 /dev/md0
>> mdadm: Need to backup 3072K of critical section..
>>
>> cat /proc/mdstat
>> Personalities : [raid10] [raid0] [raid6] [raid5] [raid4]
>> md0 : active raid5 vdf1[4] vdd1[3] vde1[2] vdc1[0]
>> 2093056 blocks super 1.2 level 5, 512k chunk, algorithm 5 [4/4]
>> [UUUU]
>> resync=DELAYED
>>
>> unused devices: <none>
>>
>> OK, so now how to make it resync?
>>
>> Here I'm stuck...
>> I've tried:
>> mdadm --misc /dev/md0 --action=check
>> mdadm --misc /dev/md0 --action=repair
>>
>> Nothing seems to be happening.
>>
>> BTW, I had the array mounted during my testing, as ideally that is what
>> I will do with the live machine. Worst case scenario (on the live
>> machine) I can afford to lose all the data, as it is only an extra
>> backup of the other backup machine, but it would mean a few TB's of data
>> across a slow WAN....
>>
>> Any suggestions on getting this to progress? Did I do something wrong?
>>
>> Thanks for the suggestion, it certainly looks promising so far.
>
> Why dont you stop your array once and do something like this?
> mdadm --stop /dev/md0
> mdadm --assemble /dev/md0 --run --force --update=resync /dev/vdf1
> /dev/vdd1 /dev/vde1 vdc1
>
> This will restart your array with the required raid-level and also
> start the resyncing process.
I got:
mdadm: Failed to restore critical section for reshape, sorry.
Possibly you needed to specify the --backup-file
Personalities : [raid10] [raid0] [raid6] [raid5] [raid4]
md0 : inactive vdd1[3](S) vdf1[4](S) vde1[2](S) vdc1[0](S)
4186112 blocks super 1.2
unused devices: <none>
Related dmesg output:
[27217.316713] md: md0 stopped.
[27217.316727] md: unbind<vdc1>
[27217.316732] md: export_rdev(vdc1)
[27217.316769] md: unbind<vdf1>
[27217.316772] md: export_rdev(vdf1)
[27217.316789] md: unbind<vdd1>
[27217.316791] md: export_rdev(vdd1)
[27217.316806] md: unbind<vde1>
[27217.316809] md: export_rdev(vde1)
[27248.819955] md: md0 stopped.
[27248.855348] md: bind<vdc1>
[27248.868655] md: bind<vde1>
[27248.872681] md: bind<vdf1>
[27248.876477] md: bind<vdd1>
Any further suggestions?
I have no problem with doing the whole process offline, but I assume it
will take a lot longer on the live machine (4TB drives), and didn't want
to leave it unmounted for a long time.
Regards,
Adam
--
Adam Goryachev Website Managers www.websitemanagers.com.au
^ permalink raw reply
* Re: Converting 4 disk RAID10 to RAID5
From: Phil Turmel @ 2015-10-27 12:13 UTC (permalink / raw)
To: Adam Goryachev, Anugraha Sinha, linux-raid
In-Reply-To: <562F1A61.6010709@websitemanagers.com.au>
On 10/27/2015 02:32 AM, Adam Goryachev wrote:
> On 27/10/15 17:19, Anugraha Sinha wrote:
\
>> This will restart your array with the required raid-level and also
>> start the resyncing process.
You shouldn't have needed to stop it.
> I got:
> mdadm: Failed to restore critical section for reshape, sorry.
> Possibly you needed to specify the --backup-file
Probably just need "mdadm /dev/md0 --grow --continue --backup-file=...."
where .... is a file location outside the array. You may also need
--invalid-backup
Phil
^ permalink raw reply
* [UU_] Won't go away
From: Michael Munger @ 2015-10-27 12:17 UTC (permalink / raw)
To: linux-raid
It's my understanding, that an array running with [UU_] is degraded and
there is a disk problem.
I have been trying to create a RAID array using this command:
mdadm --create --verbose /dev/md0 --level=5 --raid-devices=3 /dev/sdb1
/dev/sdc1 /dev/sdd1
However, once it create it, it get this in /proc/mdstat:
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sdd1[3] sdc1[1] sdb1[0]
3906764800 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2]
[UU_]
[>....................] recovery = 0.1% (2611780/1953382400)
finish=1737.7min speed=18709K/sec
bitmap: 0/15 pages [0KB], 65536KB chunk
I have done the following to "fix" the drive:
1. Smartmontools smartctl long and short tests. All passed.
2. Zeroed the first 1GB of the drive (overkill, I know I could just do
the first 512) with dd if=/dev/zero of=/dev/sdb count=250000 bs=4k
3. Re-partitioned the drive.
4. Re-create.
No joy.
Then, I swapped this hard drive out for another drive.
I am still getting [UU_].
Thoughts?
--
Michael Munger, dCAP, MCPS, MCNPS, MBSS
High Powered Help, Inc.
Microsoft Certified Professional
Microsoft Certified Small Business Specialist
Digium Certified Asterisk Professional
michael@highpoweredhelp.com
--
Michael Munger, dCAP, MCPS, MCNPS, MBSS
High Powered Help, Inc.
Microsoft Certified Professional
Microsoft Certified Small Business Specialist
Digium Certified Asterisk Professional
michael@highpoweredhelp.com
^ permalink raw reply
* Re: [UU_] Won't go away
From: Phil Turmel @ 2015-10-27 12:23 UTC (permalink / raw)
To: Michael Munger, linux-raid
In-Reply-To: <562F6B56.9040901@highpoweredhelp.com>
On 10/27/2015 08:17 AM, Michael Munger wrote:
> It's my understanding, that an array running with [UU_] is degraded and
> there is a disk problem.
>
> I have been trying to create a RAID array using this command:
>
> mdadm --create --verbose /dev/md0 --level=5 --raid-devices=3 /dev/sdb1
> /dev/sdc1 /dev/sdd1
>
> However, once it create it, it get this in /proc/mdstat:
>
> Personalities : [raid6] [raid5] [raid4]
> md0 : active raid5 sdd1[3] sdc1[1] sdb1[0]
> 3906764800 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2]
> [UU_]
> [>....................] recovery = 0.1% (2611780/1953382400)
> finish=1737.7min speed=18709K/sec
> bitmap: 0/15 pages [0KB], 65536KB chunk
What's to fix? Did you try waiting for the recovery to finish?
Creating an array generally isn't instant -- the redundancy has to be
computed over the entire array.
Now, you might have speed limits that are making it slower than possible
(see man md), but multiple hours on big devices is perfectly normal.
Phil
^ permalink raw reply
* Re: [UU_] Won't go away
From: Robin Hill @ 2015-10-27 12:24 UTC (permalink / raw)
To: Michael Munger; +Cc: linux-raid
In-Reply-To: <562F6B56.9040901@highpoweredhelp.com>
[-- Attachment #1: Type: text/plain, Size: 1685 bytes --]
On Tue Oct 27, 2015 at 08:17:26AM -0400, Michael Munger wrote:
> It's my understanding, that an array running with [UU_] is degraded and
> there is a disk problem.
>
> I have been trying to create a RAID array using this command:
>
> mdadm --create --verbose /dev/md0 --level=5 --raid-devices=3 /dev/sdb1
> /dev/sdc1 /dev/sdd1
>
> However, once it create it, it get this in /proc/mdstat:
>
> Personalities : [raid6] [raid5] [raid4]
> md0 : active raid5 sdd1[3] sdc1[1] sdb1[0]
> 3906764800 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2]
> [UU_]
> [>....................] recovery = 0.1% (2611780/1953382400)
> finish=1737.7min speed=18709K/sec
> bitmap: 0/15 pages [0KB], 65536KB chunk
>
> I have done the following to "fix" the drive:
>
> 1. Smartmontools smartctl long and short tests. All passed.
> 2. Zeroed the first 1GB of the drive (overkill, I know I could just do
> the first 512) with dd if=/dev/zero of=/dev/sdb count=250000 bs=4k
> 3. Re-partitioned the drive.
> 4. Re-create.
>
> No joy.
>
> Then, I swapped this hard drive out for another drive.
>
> I am still getting [UU_].
>
> Thoughts?
This is perfectly normal. A RAID-5 array is created in degraded mode,
then the final disk is synchronised. This makes the array available
immediately, with the synchonisation happening in the background.
Just leave it to complete the recovery process.
Cheers,
Robin
--
___
( ' } | Robin Hill <robin@robinhill.me.uk> |
/ / ) | Little Jim says .... |
// !! | "He fallen in de water !!" |
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]
^ permalink raw reply
* Re: [UU_] Won't go away
From: Phil Turmel @ 2015-10-27 12:34 UTC (permalink / raw)
To: Michael Munger; +Cc: Linux-RAID
In-Reply-To: <562F6DEA.3010409@highpoweredhelp.com>
Hi Michael,
{Added the list back and fixed the top-posting. Convention on
kernel.org is to reply-to-all, trim replies, and bottom post or interleave.}
On 10/27/2015 08:28 AM, Michael Munger wrote:
> On 10/27/2015 08:23 AM, Phil Turmel wrote:
>> On 10/27/2015 08:17 AM, Michael Munger wrote:
>>> Personalities : [raid6] [raid5] [raid4]
>>> md0 : active raid5 sdd1[3] sdc1[1] sdb1[0]
>>> 3906764800 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2]
>>> [UU_]
>>> [>....................] recovery = 0.1% (2611780/1953382400)
>>> finish=1737.7min speed=18709K/sec
>>> bitmap: 0/15 pages [0KB], 65536KB chunk
>> What's to fix? Did you try waiting for the recovery to finish?
>> Creating an array generally isn't instant -- the redundancy has to be
>> computed over the entire array.
>>
>> Now, you might have speed limits that are making it slower than possible
>> (see man md), but multiple hours on big devices is perfectly normal.
> I thought [UU_] indicated a physical drive problem, not logical array
> issue. I'll wait until 2pm when this is supposed to finish and re-check.
I mentioned the speed limits because 20MB/sec is kinda low for modern
drives. You can probably speed that up.
> It will be GREAT if that's all it was... my being impatient!
>
> PS... lsdrv is a great utility. thanks for making it available on Github.
You're welcome :-)
Phil
^ permalink raw reply
* Re: Replacing a RAID1 drive that has not failed.
From: Doug Herr @ 2015-10-27 16:45 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <n0mbam$gmk$1@ger.gmane.org>
Thanks to both Wol and Eddie for pointing me to the --replace option and
for not simply telling me to RTFM. Part of my confusion was based on my
last attempt (years back) to find out if three drive RAID 1 was even an
option. Just as before I found stuff that laughed at the mere idea of a
three drive mirror. Seems people were getting too tied up on the word
"mirror", which is not really a proper name for it in my view. The
mirror copy of myself when looking into a mirror is *not* the same in me
in so many ways. Oh, and now I think I understand that hardware RAID
rarely or never allows it but Linux md RAID has it now and maybe had it
the whole time.
That said, I really should have slogged thru the man page better. It
does seem that --replace includes the key feature that I want:
"the device remains in service during the recovery process to
increase resilience against multiple failures."
So, just in case somebody wants to see if they can find anything I
missed, here is my current plan...
1. Partition new drive (plugged in via external SATA dock)
#fdisk /dev/sdc
(make it an exact match of sda/sdb unless it turns out to be smaller,
in which case I can shrink /boot to make room.)
2. Replace sda partition with sdc partition for each RAID.
(Could do all at once but feels safer to do one at a time.)
#mdadm /dev/md123 --replace /dev/sda7 --with /dev/sdc7
#watch /proc/mdstat
#mdadm /dev/md125 --replace /dev/sda6 --with /dev/sdc6
#watch /proc/mdstat
#mdadm /dev/md126 --replace /dev/sda3 --with /dev/sdc3
#watch /proc/mdstat
#mdadm /dev/md1 --replace /dev/sda5 --with /dev/sdc5
#watch /proc/mdstat
#mdadm /dev/md4 --replace /dev/sda2 --with /dev/sdc2
#watch /proc/mdstat
3. All RAIDs should be "active" and in "[UU]" status, with no sda partitions.
#cat /proc/mdstat
(Consider if this should only be done later/never so that sda can
be a form of backup in case of issues during the post replace reboot.)
4. Prevent the old partitions from ever being pulled into the RAIDs.
#mdadm --zero-superblock /dev/sda7
#mdadm --zero-superblock /dev/sda6
#mdadm --zero-superblock /dev/sda3
#mdadm --zero-superblock /dev/sda5
#mdadm --zero-superblock /dev/sda2
5. Make a new /boot on sdc.
(using rsync since the language is taken from my daily cron and this is a
tad quicker if I end up running it more than once.)
(fstab line needing update: UUID=52b14d98-b284-41a0-a36f-459ae3ae12a7 /boot ext4 defaults 1 2)
#mkfs.ext4 /dev/sdc1
#mkdir /bootnew
#mount /dev/sdc1 /bootnew
#rsync -a --delete /boot/ /bootnew
#grub-install /dev/sdc
#blkid /dev/sdc1
#vim /etc/fstab
#umount /bootnew ; rmdir /bootnew
6. Power down and swap sdc into sda "slot".
7. Make sure that all RAIDs are "active" and in "[UU]" status:
#cat /proc/mdstat
--
Doug Herr
^ permalink raw reply
* Re: [PATCH] md-cluster: Only one thread should request DLM lock
From: Neil Brown @ 2015-10-27 20:48 UTC (permalink / raw)
To: Goldwyn Rodrigues, linux-raid; +Cc: gqjiang, Goldwyn Rodrigues
In-Reply-To: <562A099E.2060709@suse.de>
[-- Attachment #1: Type: text/plain, Size: 2854 bytes --]
On Fri, Oct 23 2015, Goldwyn Rodrigues wrote:
> On 10/22/2015 09:11 PM, Neil Brown wrote:
>> rgoldwyn@suse.de writes:
>>
>>> From: Goldwyn Rodrigues <rgoldwyn@suse.com>
>>>
>>> If a DLM lock is in progress, requesting the same DLM lock will
>>> result in -EBUSY. Use a mutex to make sure only one thread requests
>>> for dlm_lock() function at a time.
>>>
>>> This will fix the error -EBUSY returned from DLM's
>>> validate_lock_args().
>>
>> I can see that we only want one thread calling dlm_lock() with a given
>> 'struct dlm_lock_resource' at a time, otherwise nasty things could
>> happen.
>>
>> However if such a race is possible, then aren't there other possibly
>> complications.
>
> This is specific to the duration of dlm_lock() function only and not the
> entire lifetime of the resource. If one thread has requested dlm_lock()
> and another thread comes in and calls dlm_lock() on the same resource,
> we will get -EBUSY on the second one because the lock is already requested.
>
> Our dlm_unlock_sync() call is also a dlm_lock_sync(), and eventually
> dlm_lock() call, with a NULL lock.
>
>>
>> Suppose two threads try to lock the same resource.
>> Presumably one will try to lock the resource, then the next one (when it
>> gets the mutex) will discover that it already has the resource, but will
>> think it has exclusive access - maybe?
>
> I am not sure if I understand this. DLM locks are supposed to be at the
> node level as opposed to thread level.
I think this is exactly my point. I think we need some extra
thread-level locking.
For example suppose some thread calls sendmsg() which takes the token
lock, and then while that is happening metadata_update_start() gets
called.
It will try to take the token lock, but as the node already has the
lock, it will succeed trivially. Then two threads on the one node both
think they have the lock which will almost certainly lead to confusion.
So we need to hold some mutex the entire time that sendmsg() is running,
and need to hold that same mutex when calling metadata_update_start().
Once we have that, there is not need for the mutex you introduced which
is just held while claiming the lock.
It could be that we can use ->reconfig_mutex for a lot of this.
Certainly we always hold ->reconfig_mutex while performing a metadata
update.
We probably don't want to take it just for ->resync_info_update().
I'm not sure if it would be best to have a per-resource mutex which we
take in dlm_lock_sync() and drop in dlm_unlock_sync(), or if we want the
locking at a higher level.
Probably ->reconfig_mutex is already used where we need higher-level
locking.
So if you change you patch to unlock in dlm_unlock_sync() rather than
at the end of dlm_lock_sync(), then I think it will make sense.
Thanks,
NeilBrown
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]
^ permalink raw reply
* Re: Replacing a RAID1 drive that has not failed.
From: Wols Lists @ 2015-10-27 21:31 UTC (permalink / raw)
To: Doug Herr, linux-raid
In-Reply-To: <n0o9n3$gmk$2@ger.gmane.org>
On 27/10/15 16:45, Doug Herr wrote:
> 1. Partition new drive (plugged in via external SATA dock)
> #fdisk /dev/sdc
> (make it an exact match of sda/sdb unless it turns out to be smaller,
> in which case I can shrink /boot to make room.)
man gdisk.
Look at the "u" option in the "expert" menu - it'll probably save you
some work :-)
Cheers,
Wol
^ permalink raw reply
* Re: [PATCH] md-cluster: Only one thread should request DLM lock
From: Goldwyn Rodrigues @ 2015-10-27 23:28 UTC (permalink / raw)
To: Neil Brown, linux-raid; +Cc: gqjiang, Goldwyn Rodrigues
In-Reply-To: <87twpcca1w.fsf@notabene.neil.brown.name>
On 10/27/2015 03:48 PM, Neil Brown wrote:
> On Fri, Oct 23 2015, Goldwyn Rodrigues wrote:
>
>> On 10/22/2015 09:11 PM, Neil Brown wrote:
>>> rgoldwyn@suse.de writes:
>>>
>>>> From: Goldwyn Rodrigues <rgoldwyn@suse.com>
>>>>
>>>> If a DLM lock is in progress, requesting the same DLM lock will
>>>> result in -EBUSY. Use a mutex to make sure only one thread requests
>>>> for dlm_lock() function at a time.
>>>>
>>>> This will fix the error -EBUSY returned from DLM's
>>>> validate_lock_args().
>>>
>>> I can see that we only want one thread calling dlm_lock() with a given
>>> 'struct dlm_lock_resource' at a time, otherwise nasty things could
>>> happen.
>>>
>>> However if such a race is possible, then aren't there other possibly
>>> complications.
>>
>> This is specific to the duration of dlm_lock() function only and not the
>> entire lifetime of the resource. If one thread has requested dlm_lock()
>> and another thread comes in and calls dlm_lock() on the same resource,
>> we will get -EBUSY on the second one because the lock is already requested.
>>
>> Our dlm_unlock_sync() call is also a dlm_lock_sync(), and eventually
>> dlm_lock() call, with a NULL lock.
>>
>>>
>>> Suppose two threads try to lock the same resource.
>>> Presumably one will try to lock the resource, then the next one (when it
>>> gets the mutex) will discover that it already has the resource, but will
>>> think it has exclusive access - maybe?
>>
>> I am not sure if I understand this. DLM locks are supposed to be at the
>> node level as opposed to thread level.
>
> I think this is exactly my point. I think we need some extra
> thread-level locking.
> For example suppose some thread calls sendmsg() which takes the token
> lock, and then while that is happening metadata_update_start() gets
> called.
> It will try to take the token lock, but as the node already has the
> lock, it will succeed trivially. Then two threads on the one node both
> think they have the lock which will almost certainly lead to confusion.
Yes, this is the other problem I was talking about which led to the call
trace originating in unlock_comm(). These are two separate problems, but
your proposal of using a single mutex should resolve both. I was
thinking more in terms of finer grained locking but it looks like an
overkill here.
>
> So we need to hold some mutex the entire time that sendmsg() is running,
> and need to hold that same mutex when calling metadata_update_start().
> Once we have that, there is not need for the mutex you introduced which
> is just held while claiming the lock.
We may have to add flags to detect where the call is coming from, but
yes that should be fine. I will come up with a patch soon.
>
> It could be that we can use ->reconfig_mutex for a lot of this.
> Certainly we always hold ->reconfig_mutex while performing a metadata
> update.
> We probably don't want to take it just for ->resync_info_update().
Agree here.
>
> I'm not sure if it would be best to have a per-resource mutex which we
> take in dlm_lock_sync() and drop in dlm_unlock_sync(), or if we want the
> locking at a higher level.
> Probably ->reconfig_mutex is already used where we need higher-level
> locking.
> So if you change you patch to unlock in dlm_unlock_sync() rather than
> at the end of dlm_lock_sync(), then I think it will make sense.
It is just as good as using a single mutex for all communication, so I
would favour using a single one.
Thanks for your comments.
--
Goldwyn
^ permalink raw reply
* [PATCH] raid5-cache: use crc32c checksum
From: Shaohua Li @ 2015-10-27 23:48 UTC (permalink / raw)
To: linux-raid; +Cc: Kernel-team, songliubraving, hch, bart.vanassche, neilb
crc32c has lower overhead with cpu acceleration. It's a shame I didn't
use it in first post, sorry. This changes disk format, but we are still
ok in current stage.
Signed-off-by: Shaohua Li <shli@fb.com>
---
drivers/md/raid5-cache.c | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/drivers/md/raid5-cache.c b/drivers/md/raid5-cache.c
index 0c398ad..e61f4de 100644
--- a/drivers/md/raid5-cache.c
+++ b/drivers/md/raid5-cache.c
@@ -16,7 +16,7 @@
#include <linux/blkdev.h>
#include <linux/slab.h>
#include <linux/raid/md_p.h>
-#include <linux/crc32.h>
+#include <linux/crc32c.h>
#include <linux/random.h>
#include "md.h"
#include "raid5.h"
@@ -242,7 +242,7 @@ static void r5l_submit_current_io(struct r5l_log *log)
block = page_address(io->meta_page);
block->meta_size = cpu_to_le32(io->meta_offset);
- crc = crc32_le(log->uuid_checksum, (void *)block, PAGE_SIZE);
+ crc = crc32c_le(log->uuid_checksum, (void *)block, PAGE_SIZE);
block->checksum = cpu_to_le32(crc);
log->current_io = NULL;
@@ -448,7 +448,7 @@ int r5l_write_stripe(struct r5l_log *log, struct stripe_head *sh)
if (test_bit(STRIPE_LOG_TRAPPED, &sh->state))
continue;
addr = kmap_atomic(sh->dev[i].page);
- sh->dev[i].log_checksum = crc32_le(log->uuid_checksum,
+ sh->dev[i].log_checksum = crc32c_le(log->uuid_checksum,
addr, PAGE_SIZE);
kunmap_atomic(addr);
}
@@ -839,7 +839,7 @@ static int r5l_read_meta_block(struct r5l_log *log,
le64_to_cpu(mb->position) != ctx->pos)
return -EINVAL;
- crc = crc32_le(log->uuid_checksum, (void *)mb, PAGE_SIZE);
+ crc = crc32c_le(log->uuid_checksum, (void *)mb, PAGE_SIZE);
if (stored_crc != crc)
return -EINVAL;
@@ -914,7 +914,7 @@ static int r5l_recovery_flush_one_stripe(struct r5l_log *log,
if (!test_bit(R5_Wantwrite, &sh->dev[disk_index].flags))
continue;
addr = kmap_atomic(sh->dev[disk_index].page);
- checksum = crc32_le(log->uuid_checksum, addr, PAGE_SIZE);
+ checksum = crc32c_le(log->uuid_checksum, addr, PAGE_SIZE);
kunmap_atomic(addr);
if (checksum != sh->dev[disk_index].log_checksum)
goto error;
@@ -1004,7 +1004,7 @@ static int r5l_log_write_empty_meta_block(struct r5l_log *log, sector_t pos,
mb->meta_size = cpu_to_le32(sizeof(struct r5l_meta_block));
mb->seq = cpu_to_le64(seq);
mb->position = cpu_to_le64(pos);
- crc = crc32_le(log->uuid_checksum, (void *)mb, PAGE_SIZE);
+ crc = crc32c_le(log->uuid_checksum, (void *)mb, PAGE_SIZE);
mb->checksum = cpu_to_le32(crc);
if (!sync_page_io(log->rdev, pos, PAGE_SIZE, page, WRITE_FUA, false)) {
@@ -1095,7 +1095,7 @@ static int r5l_load_log(struct r5l_log *log)
}
stored_crc = le32_to_cpu(mb->checksum);
mb->checksum = 0;
- expected_crc = crc32_le(log->uuid_checksum, (void *)mb, PAGE_SIZE);
+ expected_crc = crc32c_le(log->uuid_checksum, (void *)mb, PAGE_SIZE);
if (stored_crc != expected_crc) {
create_super = true;
goto create;
@@ -1144,7 +1144,7 @@ int r5l_init_log(struct r5conf *conf, struct md_rdev *rdev)
log->need_cache_flush = (rdev->bdev->bd_disk->queue->flush_flags != 0);
- log->uuid_checksum = crc32_le(~0, (void *)rdev->mddev->uuid,
+ log->uuid_checksum = crc32c_le(~0, (void *)rdev->mddev->uuid,
sizeof(rdev->mddev->uuid));
mutex_init(&log->io_mutex);
--
2.4.6
^ permalink raw reply related
* Re: Converting 4 disk RAID10 to RAID5
From: Adam Goryachev @ 2015-10-28 1:57 UTC (permalink / raw)
To: Phil Turmel, Anugraha Sinha, linux-raid
In-Reply-To: <562F6A77.20502@turmel.org>
On 27/10/15 23:13, Phil Turmel wrote:
> On 10/27/2015 02:32 AM, Adam Goryachev wrote:
>> On 27/10/15 17:19, Anugraha Sinha wrote:
> \
>>> This will restart your array with the required raid-level and also
>>> start the resyncing process.
> You shouldn't have needed to stop it.
>
>> I got:
>> mdadm: Failed to restore critical section for reshape, sorry.
>> Possibly you needed to specify the --backup-file
> Probably just need "mdadm /dev/md0 --grow --continue --backup-file=...."
>
> where .... is a file location outside the array. You may also need
> --invalid-backup
>
> Phil
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
Thank you, that did it (if I don't do the stop):
mdadm --grow --continue --backup-file=/tmp/nofile /dev/md0
mdadm: Need to backup 3072K of critical section..
mdadm: Reshape is delayed, but cannot wait carefully with this kernel.
You might experience problems until other reshapes complete.
Just in case someone else is looking later, this is how to convert from
4 disk RAID10 to 4 disk RAID5.
Starting again from scratch, I followed this process:
mdadm --create --level=10 --raid-devices=4 /dev/md0 /dev/vd[cdef]1
mdadm --grow --bitmap=internal /dev/md0
#This gets me to my current live config....
# Remove the bitmap since it causes problems later
mdadm --grow --bitmap=none /dev/md0
# Grow to RAID0 which basically just drops two devices, it finishes
immediately.
mdadm --grow --level=0 --raid-devices=2 /dev/md0
# Grow to RAID5, only add one device
mdadm --grow --level=5 --raid-devices=3 /dev/md0 -a /dev/vdd1
# A resync is completed, looks good/normal
# Grow to add the 4th device
mdadm --grow --raid-devices=4 /dev/md0 -a /dev/vdf1
# Problems, stuck with resync=DELAYED, we were not asked to supply a
backup file for this...
# Just ask mdadm to continue, and now supply a backup file
mdadm --grow --continue --backup-file=/tmp/nofile /dev/md0
# All good. Add the bitmap again
mdadm --grow /dev/md0 --bitmap=internal
It would seem to me that this is generally a relatively "dangerous"
exercise, you are very vulnerable to failed disks due to using RAID0,
and then migrating to RAID5, any read/write failure during that process
could cause significant data loss. Finally, the grow to add the 4th disk
should be fine, unless you have a problem with the same block on two
devices.
Any final comments/suggestions before I start the process "for real"?
Thanks,
Adam
--
Adam Goryachev Website Managers www.websitemanagers.com.au
^ permalink raw reply
* Re: Replacing a RAID1 drive that has not failed.
From: Roman Mamedov @ 2015-10-28 5:28 UTC (permalink / raw)
To: Wols Lists; +Cc: Doug Herr, linux-raid
In-Reply-To: <562FED15.5040804@youngman.org.uk>
[-- Attachment #1: Type: text/plain, Size: 729 bytes --]
On Tue, 27 Oct 2015 21:31:01 +0000
Wols Lists <antlists@youngman.org.uk> wrote:
> On 27/10/15 16:45, Doug Herr wrote:
> > 1. Partition new drive (plugged in via external SATA dock)
> > #fdisk /dev/sdc
> > (make it an exact match of sda/sdb unless it turns out to be smaller,
> > in which case I can shrink /boot to make room.)
>
> man gdisk.
>
> Look at the "u" option in the "expert" menu - it'll probably save you
> some work :-)
Unfamiliar with 'gdisk', but when I need the exact same set of partitions on
another drive, I use sfdisk:
# sfdisk -d /dev/sda > sda.sf
# sfdisk /dev/sdb < sda.sf
You could even do it in one step:
# sfdisk -d /dev/sda | sfdisk /dev/sdb
--
With respect,
Roman
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply
* [PATCH] md/raid5: fix locking in handle_stripe_clean_event()
From: Roman Gushchin @ 2015-10-28 8:52 UTC (permalink / raw)
To: linux-kernel; +Cc: Roman Gushchin, Neil Brown, Shaohua Li, linux-raid, stable
After commit 566c09c53455 ("raid5: relieve lock contention in get_active_stripe()")
__find_stripe() is called under conf->hash_locks + hash.
But handle_stripe_clean_event() calls remove_hash() under
conf->device_lock.
Under some cirscumstances the hash chain can be circuited,
and we get an infinite loop with disabled interrupts and locked hash
lock in __find_stripe(). This leads to hard lockup on multiple CPUs
and following system crash.
I was able to reproduce this behavior on raid6 over 6 ssd disks.
The devices_handle_discard_safely option should be set to enable trim
support. The following script was used:
for i in `seq 1 32`; do
dd if=/dev/zero of=large$i bs=10M count=100 &
done
Signed-off-by: Roman Gushchin <klamm@yandex-team.ru>
Cc: Neil Brown <neilb@suse.de>
Cc: Shaohua Li <shli@kernel.org>
Cc: linux-raid@vger.kernel.org
Cc: <stable@vger.kernel.org> # 3.10 - 3.19
---
drivers/md/raid5.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index e421016..5fa7549 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -3060,6 +3060,8 @@ static void handle_stripe_clean_event(struct r5conf *conf,
}
if (!discard_pending &&
test_bit(R5_Discard, &sh->dev[sh->pd_idx].flags)) {
+ int hash = sh->hash_lock_index;
+
clear_bit(R5_Discard, &sh->dev[sh->pd_idx].flags);
clear_bit(R5_UPTODATE, &sh->dev[sh->pd_idx].flags);
if (sh->qd_idx >= 0) {
@@ -3073,9 +3075,9 @@ static void handle_stripe_clean_event(struct r5conf *conf,
* no updated data, so remove it from hash list and the stripe
* will be reinitialized
*/
- spin_lock_irq(&conf->device_lock);
+ spin_lock_irq(conf->hash_locks + hash);
remove_hash(sh);
- spin_unlock_irq(&conf->device_lock);
+ spin_unlock_irq(conf->hash_locks + hash);
if (test_bit(STRIPE_SYNC_REQUESTED, &sh->state))
set_bit(STRIPE_HANDLE, &sh->state);
--
2.4.3
^ permalink raw reply related
* Re: Converting 4 disk RAID10 to RAID5
From: Phil Turmel @ 2015-10-28 13:56 UTC (permalink / raw)
To: Adam Goryachev, Anugraha Sinha, linux-raid
In-Reply-To: <56302B95.3000006@websitemanagers.com.au>
On 10/27/2015 09:57 PM, Adam Goryachev wrote:
>
> Any final comments/suggestions before I start the process "for real"?
Do a 'check' scrub before the raid10 to raid0 conversion.
Great report!
Phil
^ permalink raw reply
* Re: [PATCH] raid5-cache: use crc32c checksum
From: Bart Van Assche @ 2015-10-28 15:26 UTC (permalink / raw)
To: Shaohua Li, linux-raid; +Cc: Kernel-team, songliubraving, hch, neilb
In-Reply-To: <05c660271fe0d5f42bc053ef89c5e20ff57cf955.1445989535.git.shli@fb.com>
On 10/27/2015 04:48 PM, Shaohua Li wrote:
> crc32c has lower overhead with cpu acceleration. It's a shame I didn't
> use it in first post, sorry. This changes disk format, but we are still
> ok in current stage.
Hello Shaohua,
Although this patch looks fine to me I think the (void *) casts in the
crc32c_le() calls can be left out. Had you considered to include that
change in this patch ?
Thanks,
Bart.
^ permalink raw reply
* Re: [PATCH] raid5-cache: use crc32c checksum
From: Shaohua Li @ 2015-10-28 15:41 UTC (permalink / raw)
To: Bart Van Assche; +Cc: linux-raid, Kernel-team, songliubraving, hch, neilb
In-Reply-To: <5630E942.90501@sandisk.com>
On Wed, Oct 28, 2015 at 08:26:58AM -0700, Bart Van Assche wrote:
> On 10/27/2015 04:48 PM, Shaohua Li wrote:
> >crc32c has lower overhead with cpu acceleration. It's a shame I didn't
> >use it in first post, sorry. This changes disk format, but we are still
> >ok in current stage.
>
> Hello Shaohua,
>
> Although this patch looks fine to me I think the (void *) casts in
> the crc32c_le() calls can be left out. Had you considered to include
> that change in this patch ?
Sure, thanks!
From 558985bc46db90762447e0bc9515e2a84272e636 Mon Sep 17 00:00:00 2001
Message-Id: <558985bc46db90762447e0bc9515e2a84272e636.1446046795.git.shli@fb.com>
From: Shaohua Li <shli@fb.com>
Date: Wed, 28 Oct 2015 08:37:27 -0700
Subject: [PATCH] raid5-cache: use crc32c checksum
crc32c has lower overhead with cpu acceleration. It's a shame I didn't
use it in first post, sorry. This changes disk format, but we are still
ok in current stage.
V2: delete unnecessary type conversion as pointed out by Bart
Signed-off-by: Shaohua Li <shli@fb.com>
---
drivers/md/raid5-cache.c | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/drivers/md/raid5-cache.c b/drivers/md/raid5-cache.c
index 0c398ad..e9227e8 100644
--- a/drivers/md/raid5-cache.c
+++ b/drivers/md/raid5-cache.c
@@ -16,7 +16,7 @@
#include <linux/blkdev.h>
#include <linux/slab.h>
#include <linux/raid/md_p.h>
-#include <linux/crc32.h>
+#include <linux/crc32c.h>
#include <linux/random.h>
#include "md.h"
#include "raid5.h"
@@ -242,7 +242,7 @@ static void r5l_submit_current_io(struct r5l_log *log)
block = page_address(io->meta_page);
block->meta_size = cpu_to_le32(io->meta_offset);
- crc = crc32_le(log->uuid_checksum, (void *)block, PAGE_SIZE);
+ crc = crc32c_le(log->uuid_checksum, block, PAGE_SIZE);
block->checksum = cpu_to_le32(crc);
log->current_io = NULL;
@@ -448,7 +448,7 @@ int r5l_write_stripe(struct r5l_log *log, struct stripe_head *sh)
if (test_bit(STRIPE_LOG_TRAPPED, &sh->state))
continue;
addr = kmap_atomic(sh->dev[i].page);
- sh->dev[i].log_checksum = crc32_le(log->uuid_checksum,
+ sh->dev[i].log_checksum = crc32c_le(log->uuid_checksum,
addr, PAGE_SIZE);
kunmap_atomic(addr);
}
@@ -839,7 +839,7 @@ static int r5l_read_meta_block(struct r5l_log *log,
le64_to_cpu(mb->position) != ctx->pos)
return -EINVAL;
- crc = crc32_le(log->uuid_checksum, (void *)mb, PAGE_SIZE);
+ crc = crc32c_le(log->uuid_checksum, mb, PAGE_SIZE);
if (stored_crc != crc)
return -EINVAL;
@@ -914,7 +914,7 @@ static int r5l_recovery_flush_one_stripe(struct r5l_log *log,
if (!test_bit(R5_Wantwrite, &sh->dev[disk_index].flags))
continue;
addr = kmap_atomic(sh->dev[disk_index].page);
- checksum = crc32_le(log->uuid_checksum, addr, PAGE_SIZE);
+ checksum = crc32c_le(log->uuid_checksum, addr, PAGE_SIZE);
kunmap_atomic(addr);
if (checksum != sh->dev[disk_index].log_checksum)
goto error;
@@ -1004,7 +1004,7 @@ static int r5l_log_write_empty_meta_block(struct r5l_log *log, sector_t pos,
mb->meta_size = cpu_to_le32(sizeof(struct r5l_meta_block));
mb->seq = cpu_to_le64(seq);
mb->position = cpu_to_le64(pos);
- crc = crc32_le(log->uuid_checksum, (void *)mb, PAGE_SIZE);
+ crc = crc32c_le(log->uuid_checksum, mb, PAGE_SIZE);
mb->checksum = cpu_to_le32(crc);
if (!sync_page_io(log->rdev, pos, PAGE_SIZE, page, WRITE_FUA, false)) {
@@ -1095,7 +1095,7 @@ static int r5l_load_log(struct r5l_log *log)
}
stored_crc = le32_to_cpu(mb->checksum);
mb->checksum = 0;
- expected_crc = crc32_le(log->uuid_checksum, (void *)mb, PAGE_SIZE);
+ expected_crc = crc32c_le(log->uuid_checksum, mb, PAGE_SIZE);
if (stored_crc != expected_crc) {
create_super = true;
goto create;
@@ -1144,7 +1144,7 @@ int r5l_init_log(struct r5conf *conf, struct md_rdev *rdev)
log->need_cache_flush = (rdev->bdev->bd_disk->queue->flush_flags != 0);
- log->uuid_checksum = crc32_le(~0, (void *)rdev->mddev->uuid,
+ log->uuid_checksum = crc32c_le(~0, rdev->mddev->uuid,
sizeof(rdev->mddev->uuid));
mutex_init(&log->io_mutex);
--
2.4.6
^ permalink raw reply related
* Re: Replacing a RAID1 drive that has not failed.
From: Doug Herr @ 2015-10-28 15:54 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <20151028102817.4ec30b5c@natsu>
On Wed, 28 Oct 2015 10:28:17 +0500, Roman Mamedov wrote:
> On Tue, 27 Oct 2015 21:31:01 +0000
> Wols Lists <antlists@youngman.org.uk> wrote:
>
>> On 27/10/15 16:45, Doug Herr wrote:
>> > 1. Partition new drive (plugged in via external SATA dock)
>> > #fdisk /dev/sdc
>> > (make it an exact match of sda/sdb unless it turns out to be smaller,
>> > in which case I can shrink /boot to make room.)
>>
>> man gdisk.
>>
>> Look at the "u" option in the "expert" menu - it'll probably save you
>> some work :-)
>
> Unfamiliar with 'gdisk', but when I need the exact same set of
partitions on
> another drive, I use sfdisk:
>
> # sfdisk -d /dev/sda > sda.sf
> # sfdisk /dev/sdb < sda.sf
>
> You could even do it in one step:
>
> # sfdisk -d /dev/sda | sfdisk /dev/sdb
Thanks much to all who have replied and all who post here.
Yup, the sfdisk copy is one that I am ready to use if needed but your
replies have triggered better googling and I am now planning to use fdisk
(I am in Fedora 22) to manually partition the new disk. I don't think I
have any need for GPT yet but what I realized is that my current set of
disks was created when fdisk was still starting at sector 63 where now it
uses 2048. I don't think it matters much for my current set of Hitachi
HDT721010SLA360 drives. It might not even matter for the slightly newer
"E7K1000" that is on the way to me, but I do want to stick with the
current standards as best I can. I have room to tweak it since I have a
non-RAID /boot partition that is easily large enough to lend space as
needed.
I do plan to play with GPT to learn about it and to be more ready for the
next computer if it is the default at that point.
Oh, I also added a step in my plan. Adding my new partitions as spares
first will alert me of size errors (if smaller) *before* any of the
replacement jobs kick off:
For each:
# mdadm /dev/md4 --add-spare /dev/sdc2
If the above does not complain then move forward for each:
# mdadm /dev/md3 --replace /dev/sda2 --with /dev/sdc2
And fear not, I will be updating my backups before moving forward with
this.
--
Doug Herr
^ permalink raw reply
* Re: How to recover after md crash during reshape?
From: Andras Tantos @ 2015-10-28 16:31 UTC (permalink / raw)
To: Phil Turmel; +Cc: Linux-RAID
In-Reply-To: <562D5F91.5040300@turmel.org>
Thanks again Phil!
I'm almost there...
>> [ 5859.527778] EXT4-fs (md1): bad geometry: block count 1831419920
>> exceeds size of device (1831419760 blocks)
>
>Yep. You'll need to use the --size option on a create. Note that it
>specifies the amount of each device to use, not the overall array size.
>According to "man mdadm", its units is k == 1024 bytes. Use the exact
>size from your original => --size=1465135936
When I try to do that, I get the following message:
root@bazsalikom:~# mdadm --create --assume-clean --verbose
--metadata=1.0 --raid-devices=7 --size=1465135936 --chunk=64 --level=6
/dev/md1 /dev/sde2 /dev/sdc2 /dev/sdf1 /dev/sdd1 /dev/sdb1 /dev/sdg1
/dev/sdh2
mdadm: layout defaults to left-symmetric
mdadm: /dev/sde2 appears to contain an ext2fs file system
size=-1216020180K mtime=Wed Dec 8 11:55:07 1954
mdadm: /dev/sde2 appears to be part of a raid array:
level=raid6 devices=7 ctime=Wed Oct 28 09:17:55 2015
mdadm: /dev/sdc2 appears to contain an ext2fs file system
size=-1264254912K mtime=Sat Jul 18 15:26:57 2015
mdadm: /dev/sdc2 appears to be part of a raid array:
level=raid6 devices=7 ctime=Wed Oct 28 09:17:55 2015
mdadm: /dev/sdf1 is smaller than given size. 1465135808K <
1465135936K + metadata
mdadm: /dev/sdd1 is smaller than given size. 1465135808K <
1465135936K + metadata
mdadm: /dev/sdb1 is smaller than given size. 1465135808K <
1465135936K + metadata
mdadm: /dev/sdg1 appears to be part of a raid array:
level=raid6 devices=7 ctime=Wed Oct 28 09:17:55 2015
mdadm: /dev/sdh2 appears to be part of a raid array:
level=raid6 devices=7 ctime=Wed Oct 28 09:17:55 2015
mdadm: create aborted
To be able to re-assemble the array, I *have* to specify metadata
version 0.9:
root@bazsalikom:~# mdadm --create --assume-clean --verbose
--metadata=0.9 --raid-devices=7 --size=1465135936 --chunk=64 --level=6
/dev/md1 /dev/sde2 /dev/sdc2 /dev/sdf1 /dev/sdd1 /dev/sdb1 /dev/sdg1
/dev/sdh2
mdadm: layout defaults to left-symmetric
mdadm: /dev/sde2 appears to contain an ext2fs file system
size=-1216020180K mtime=Wed Dec 8 11:55:07 1954
mdadm: /dev/sde2 appears to be part of a raid array:
level=raid6 devices=7 ctime=Wed Oct 28 09:17:55 2015
mdadm: /dev/sdc2 appears to contain an ext2fs file system
size=-1264254912K mtime=Sat Jul 18 15:26:57 2015
mdadm: /dev/sdc2 appears to be part of a raid array:
level=raid6 devices=7 ctime=Wed Oct 28 09:17:55 2015
mdadm: /dev/sdf1 appears to be part of a raid array:
level=raid6 devices=7 ctime=Wed Oct 28 09:17:55 2015
mdadm: /dev/sdd1 appears to be part of a raid array:
level=raid6 devices=7 ctime=Wed Oct 28 09:17:55 2015
mdadm: /dev/sdb1 appears to be part of a raid array:
level=raid6 devices=7 ctime=Wed Oct 28 09:17:55 2015
mdadm: /dev/sdg1 appears to be part of a raid array:
level=raid6 devices=7 ctime=Wed Oct 28 09:17:55 2015
mdadm: /dev/sdh2 appears to be part of a raid array:
level=raid6 devices=7 ctime=Wed Oct 28 09:17:55 2015
mdadm: largest drive (/dev/sdg1) exceeds size (1465135936K) by more
than 1%
Continue creating array? y
mdadm: array /dev/md1 started.
Is this a problem? Can I upgrade my array to 1.0 metadata? Should I?
Andras
^ permalink raw reply
* Re: How to recover after md crash during reshape?
From: Phil Turmel @ 2015-10-28 16:42 UTC (permalink / raw)
To: Andras Tantos; +Cc: Linux-RAID
In-Reply-To: <5630F862.5040308@tantosonline.com>
On 10/28/2015 12:31 PM, Andras Tantos wrote:
> Thanks again Phil!
>
> I'm almost there...
>
>>> [ 5859.527778] EXT4-fs (md1): bad geometry: block count 1831419920
>>> exceeds size of device (1831419760 blocks)
>>
>>Yep. You'll need to use the --size option on a create. Note that it
>>specifies the amount of each device to use, not the overall array size.
>>According to "man mdadm", its units is k == 1024 bytes. Use the exact
>>size from your original => --size=1465135936
>
> When I try to do that, I get the following message:
>
> root@bazsalikom:~# mdadm --create --assume-clean --verbose
> --metadata=1.0 --raid-devices=7 --size=1465135936 --chunk=64 --level=6
> /dev/md1 /dev/sde2 /dev/sdc2 /dev/sdf1 /dev/sdd1 /dev/sdb1 /dev/sdg1
> /dev/sdh2
> mdadm: layout defaults to left-symmetric
> mdadm: /dev/sde2 appears to contain an ext2fs file system
> size=-1216020180K mtime=Wed Dec 8 11:55:07 1954
> mdadm: /dev/sde2 appears to be part of a raid array:
> level=raid6 devices=7 ctime=Wed Oct 28 09:17:55 2015
> mdadm: /dev/sdc2 appears to contain an ext2fs file system
> size=-1264254912K mtime=Sat Jul 18 15:26:57 2015
> mdadm: /dev/sdc2 appears to be part of a raid array:
> level=raid6 devices=7 ctime=Wed Oct 28 09:17:55 2015
> mdadm: /dev/sdf1 is smaller than given size. 1465135808K <
> 1465135936K + metadata
> mdadm: /dev/sdd1 is smaller than given size. 1465135808K <
> 1465135936K + metadata
> mdadm: /dev/sdb1 is smaller than given size. 1465135808K <
> 1465135936K + metadata
> mdadm: /dev/sdg1 appears to be part of a raid array:
> level=raid6 devices=7 ctime=Wed Oct 28 09:17:55 2015
> mdadm: /dev/sdh2 appears to be part of a raid array:
> level=raid6 devices=7 ctime=Wed Oct 28 09:17:55 2015
> mdadm: create aborted
>
> To be able to re-assemble the array, I *have* to specify metadata
> version 0.9:
>
> root@bazsalikom:~# mdadm --create --assume-clean --verbose
> --metadata=0.9 --raid-devices=7 --size=1465135936 --chunk=64 --level=6
> /dev/md1 /dev/sde2 /dev/sdc2 /dev/sdf1 /dev/sdd1 /dev/sdb1 /dev/sdg1
> /dev/sdh2
> mdadm: layout defaults to left-symmetric
> mdadm: /dev/sde2 appears to contain an ext2fs file system
> size=-1216020180K mtime=Wed Dec 8 11:55:07 1954
> mdadm: /dev/sde2 appears to be part of a raid array:
> level=raid6 devices=7 ctime=Wed Oct 28 09:17:55 2015
> mdadm: /dev/sdc2 appears to contain an ext2fs file system
> size=-1264254912K mtime=Sat Jul 18 15:26:57 2015
> mdadm: /dev/sdc2 appears to be part of a raid array:
> level=raid6 devices=7 ctime=Wed Oct 28 09:17:55 2015
> mdadm: /dev/sdf1 appears to be part of a raid array:
> level=raid6 devices=7 ctime=Wed Oct 28 09:17:55 2015
> mdadm: /dev/sdd1 appears to be part of a raid array:
> level=raid6 devices=7 ctime=Wed Oct 28 09:17:55 2015
> mdadm: /dev/sdb1 appears to be part of a raid array:
> level=raid6 devices=7 ctime=Wed Oct 28 09:17:55 2015
> mdadm: /dev/sdg1 appears to be part of a raid array:
> level=raid6 devices=7 ctime=Wed Oct 28 09:17:55 2015
> mdadm: /dev/sdh2 appears to be part of a raid array:
> level=raid6 devices=7 ctime=Wed Oct 28 09:17:55 2015
> mdadm: largest drive (/dev/sdg1) exceeds size (1465135936K) by more
> than 1%
> Continue creating array? y
> mdadm: array /dev/md1 started.
>
> Is this a problem? Can I upgrade my array to 1.0 metadata? Should I?
Hmm. Interesting. Your version of mdadm is insisting on reserving much
more space between end of content and the v1.0 metadata than when using
v0.90 metadata.
I'm curious how much. Please show the output of "cat /proc/partitions".
If you stop the array cleanly and then manually re-assemble with
--update=metadata, you might get around it. (Specify all of the devices
explicitly to ensure you don't get burned by v0.90's problems with last
partitions.)
You definitely don't want to stay on v0.90, but you may need to for now
to get out of trouble.
Phil
^ permalink raw reply
* Re: How to recover after md crash during reshape?
From: Andras Tantos @ 2015-10-28 17:10 UTC (permalink / raw)
To: Phil Turmel; +Cc: Linux-RAID
In-Reply-To: <5630FB05.5070902@turmel.org>
Phil,
>> To be able to re-assemble the array, I *have* to specify metadata
>> version 0.9:
>>
>> Is this a problem? Can I upgrade my array to 1.0 metadata? Should I?
>
> Hmm. Interesting. Your version of mdadm is insisting on reserving much
> more space between end of content and the v1.0 metadata than when using
> v0.90 metadata.
>
> I'm curious how much. Please show the output of "cat /proc/partitions".
root@bazsalikom:/home/tantos# cat /proc/partitions
major minor #blocks name
8 16 1465138584 sdb
8 17 1465136001 sdb1
8 48 1465138584 sdd
8 49 1465136001 sdd1
8 80 1465138584 sdf
8 81 1465136001 sdf1
8 96 1953513527 sdg
8 97 1953512001 sdg1
8 112 1953514584 sdh
8 113 538145 sdh1
8 114 1465138552 sdh2
8 115 487837854 sdh3
8 64 1953514584 sde
8 65 538145 sde1
8 66 1465138552 sde2
8 67 487837854 sde3
8 32 1953514584 sdc
8 33 538145 sdc1
8 34 1465138552 sdc2
8 35 487837854 sdc3
9 0 487837760 md0
9 1 7325679680 md1
Andras
^ permalink raw reply
* Re: How to recover after md crash during reshape?
From: Phil Turmel @ 2015-10-28 17:38 UTC (permalink / raw)
To: Andras Tantos; +Cc: Linux-RAID
In-Reply-To: <56310197.1010708@tantosonline.com>
On 10/28/2015 01:10 PM, Andras Tantos wrote:
> Phil,
>
>>> To be able to re-assemble the array, I *have* to specify metadata
>>> version 0.9:
>>>
>>> Is this a problem? Can I upgrade my array to 1.0 metadata? Should I?
>>
>> Hmm. Interesting. Your version of mdadm is insisting on reserving much
>> more space between end of content and the v1.0 metadata than when using
>> v0.90 metadata.
>>
>> I'm curious how much. Please show the output of "cat /proc/partitions".
Ok. I think your version of mdadm is trying to put a bitmap on the v1.0
array, which can be suppressed with --bitmap=none. Or just do the
--assemble --update.
Phil
^ permalink raw reply
* Raid array name by mdadm
From: Sheng Yang @ 2015-10-28 18:43 UTC (permalink / raw)
To: linux-raid
Hi guys,
I am trying to find a way to make mdadm create /dev/md/<devicename>
instead of /dev/mdnn only. I found it seems possible from the manual
of mdadm(http://linux.die.net/man/8/mdadm). It said:
If the md device name is in a 'standard' format as described in DEVICE
NAMES, then it will be created, if necessary, with the appropriate
device number based on that name. If the device name is not in one of
these formats, then a unused device number will be allocated. The
device number will be considered unused if there is no active array
for that number, and there is no entry in /dev for that number and
with a non-standard name. Names that are not in 'standard' format are
only allowed in "/dev/md/".
I tried to create a raid using:
# mdadm --create device-123 --verbose --level=mirror --raid-devices=2
--name=device-123 /dev/sda /dev/sdb
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md/device-123 started.
But what I got in the end is only /dev/md127. The directory of
/dev/md/ doesn't exist.
Though the detail showed the correct name
# mdadm --detail --scan
ARRAY /dev/md127 metadata=1.2 name=98abb2346b83:device-123
UUID=6a31df29:adec4bff:e8867610:c0e10723
It's much more convenient if I could use the name of
/dev/md/device-123 instead of /dev/md127. I've tried google for hours
without success.
# mdadm --version
mdadm - v3.3 - 3rd September 2013
If anyone can shed some light on it, I would be appreciate!
Thanks!
--Sheng
^ permalink raw reply
* [PATCH] [mdadm] add crc32c and use it for r5l checksum
From: Song Liu @ 2015-10-28 19:06 UTC (permalink / raw)
To: linux-raid; +Cc: neilb, shli, hch, dan.j.williams, kernel-team, Song Liu
In kernel space, r5l checksum will use crc32c:
http://marc.info/?l=linux-raid&m=144598970529191
mdadm need to change too.
This patch ports a simplified crc32c algorithm from kernel code,
and used in super1.c:write_empty_r5l_meta_block();
Signed-off-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Shaohua Li <shli@fb.com>
---
Makefile | 2 +-
crc32c.c | 104 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
super1.c | 11 +++----
3 files changed, 109 insertions(+), 8 deletions(-)
create mode 100644 crc32c.c
diff --git a/Makefile b/Makefile
index 544e6cb..fde2e63 100644
--- a/Makefile
+++ b/Makefile
@@ -133,7 +133,7 @@ OBJS = mdadm.o config.o policy.o mdstat.o ReadMe.o util.o maps.o lib.o \
mdopen.o super0.o super1.o super-ddf.o super-intel.o bitmap.o \
super-mbr.o super-gpt.o \
restripe.o sysfs.o sha1.o mapfile.o crc32.o sg_io.o msg.o xmalloc.o \
- platform-intel.o probe_roms.o
+ platform-intel.o probe_roms.o crc32c.o
CHECK_OBJS = restripe.o sysfs.o maps.o lib.o xmalloc.o dlink.o
diff --git a/crc32c.c b/crc32c.c
new file mode 100644
index 0000000..156cba1
--- /dev/null
+++ b/crc32c.c
@@ -0,0 +1,104 @@
+/*
+ * Oct 28, 2015 Song Liu simplified the code and port it to mdadm
+ *
+ * Aug 8, 2011 Bob Pearson with help from Joakim Tjernlund and George Spelvin
+ * cleaned up code to current version of sparse and added the slicing-by-8
+ * algorithm to the closely similar existing slicing-by-4 algorithm.
+ *
+ * Oct 15, 2000 Matt Domsch <Matt_Domsch@dell.com>
+ * Nicer crc32 functions/docs submitted by linux@horizon.com. Thanks!
+ * Code was from the public domain, copyright abandoned. Code was
+ * subsequently included in the kernel, thus was re-licensed under the
+ * GNU GPL v2.
+ *
+ * Oct 12, 2000 Matt Domsch <Matt_Domsch@dell.com>
+ * Same crc32 function was used in 5 other places in the kernel.
+ * I made one version, and deleted the others.
+ * There are various incantations of crc32(). Some use a seed of 0 or ~0.
+ * Some xor at the end with ~0. The generic crc32() function takes
+ * seed as an argument, and doesn't xor at the end. Then individual
+ * users can do whatever they need.
+ * drivers/net/smc9194.c uses seed ~0, doesn't xor with ~0.
+ * fs/jffs2 uses seed 0, doesn't xor with ~0.
+ * fs/partitions/efi.c uses seed ~0, xor's with ~0.
+ *
+ * This source code is licensed under the GNU General Public License,
+ * Version 2. See the file COPYING for more details.
+ */
+
+#include <sys/types.h>
+#include <asm/types.h>
+#include <stdlib.h>
+
+/*
+ * There are multiple 16-bit CRC polynomials in common use, but this is
+ * *the* standard CRC-32 polynomial, first popularized by Ethernet.
+ * x^32+x^26+x^23+x^22+x^16+x^12+x^11+x^10+x^8+x^7+x^5+x^4+x^2+x^1+x^0
+ */
+#define CRCPOLY_LE 0xedb88320
+#define CRCPOLY_BE 0x04c11db7
+
+/*
+ * This is the CRC32c polynomial, as outlined by Castagnoli.
+ * x^32+x^28+x^27+x^26+x^25+x^23+x^22+x^20+x^19+x^18+x^14+x^13+x^11+x^10+x^9+
+ * x^8+x^6+x^0
+ */
+#define CRC32C_POLY_LE 0x82F63B78
+
+/**
+ * crc32_le_generic() - Calculate bitwise little-endian Ethernet AUTODIN II
+ * CRC32/CRC32C
+ * @crc: seed value for computation. ~0 for Ethernet, sometimes 0 for other
+ * uses, or the previous crc32/crc32c value if computing incrementally.
+ * @p: pointer to buffer over which CRC32/CRC32C is run
+ * @len: length of buffer @p
+ * @polynomial: CRC32/CRC32c LE polynomial
+ */
+static inline __u32 crc32_le_generic(__u32 crc, unsigned char const *p,
+ size_t len, __u32 polynomial)
+{
+ int i;
+ while (len--) {
+ crc ^= *p++;
+ for (i = 0; i < 8; i++)
+ crc = (crc >> 1) ^ ((crc & 1) ? polynomial : 0);
+ }
+ return crc;
+}
+
+__u32 crc32_le(__u32 crc, unsigned char const *p, size_t len)
+{
+ return crc32_le_generic(crc, p, len, CRCPOLY_LE);
+}
+
+__u32 crc32c_le(__u32 crc, unsigned char const *p, size_t len)
+{
+ return crc32_le_generic(crc, p, len, CRC32C_POLY_LE);
+}
+
+/**
+ * crc32_be_generic() - Calculate bitwise big-endian Ethernet AUTODIN II CRC32
+ * @crc: seed value for computation. ~0 for Ethernet, sometimes 0 for
+ * other uses, or the previous crc32 value if computing incrementally.
+ * @p: pointer to buffer over which CRC32 is run
+ * @len: length of buffer @p
+ * @polynomial: CRC32 BE polynomial
+ */
+static inline __u32 crc32_be_generic(__u32 crc, unsigned char const *p,
+ size_t len, __u32 polynomial)
+{
+ int i;
+ while (len--) {
+ crc ^= *p++ << 24;
+ for (i = 0; i < 8; i++)
+ crc =
+ (crc << 1) ^ ((crc & 0x80000000) ? polynomial :
+ 0);
+ }
+ return crc;
+}
+
+__u32 crc32_be(__u32 crc, unsigned char const *p, size_t len)
+{
+ return crc32_be_generic(crc, p, len, CRCPOLY_BE);
+}
diff --git a/super1.c b/super1.c
index 58e6f9d..1735c2d 100644
--- a/super1.c
+++ b/super1.c
@@ -1625,10 +1625,7 @@ static unsigned long choose_bm_space(unsigned long devsize)
static void free_super1(struct supertype *st);
#define META_BLOCK_SIZE 4096
-unsigned long crc32(
- unsigned long crc,
- const unsigned char *buf,
- unsigned len);
+__u32 crc32c_le(__u32 crc, unsigned char const *p, size_t len);
static int write_empty_r5l_meta_block(struct supertype *st, int fd)
{
@@ -1652,9 +1649,9 @@ static int write_empty_r5l_meta_block(struct supertype *st, int fd)
mb->seq = __cpu_to_le64(random32());
mb->position = __cpu_to_le64(0);
- crc = crc32(0xffffffff, sb->set_uuid, sizeof(sb->set_uuid));
- crc = crc32(crc, (void *)mb, META_BLOCK_SIZE);
- mb->checksum = __cpu_to_le32(crc);
+ crc = crc32c_le(0xffffffff, sb->set_uuid, sizeof(sb->set_uuid));
+ crc = crc32c_le(crc, (void *)mb, META_BLOCK_SIZE);
+ mb->checksum = crc;
if (lseek64(fd, (sb->data_offset) * 512, 0) < 0LL) {
pr_err("cannot seek to offset of the meta block\n");
--
2.4.6
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox