* Re: [PATCH v2 0/2] Introduce the bulk IV mode for improving the crypto engine efficiency
From: Baolin Wang @ 2016-01-04 6:58 UTC (permalink / raw)
To: Milan Broz
Cc: Jens Axboe, Alasdair G Kergon, Mike Snitzer, dm-devel, neilb,
dan.j.williams, martin.petersen, sagig, Kent Overstreet,
keith.busch, tj, Mark Brown, Arnd Bergmann, linux-block,
linux-raid, LKML
In-Reply-To: <56885330.9080801@gmail.com>
Hi Milan,
On 3 January 2016 at 06:46, Milan Broz <gmazyland@gmail.com> wrote:
>
> Sorry for delay, I tried to compile it.
> It doesn't crash now, but it also does not work.
>
> You usage of IV in XTS mode is not correct - it cannot just work this way,
> you have to initialize IV after each block. And just one write not aligned
> to your large XTS block will corrupt it.
>
> Did you tried to _read_ data you write to the device?
>
> See this test :
>
> # create device with your patch
> $ echo "test"|cryptsetup create -s 512 -c aes-xts-bulk tst /dev/sdg
>
> # prepare random test file
> $ dd if=/dev/urandom of=/src.img bs=1M count=16
>
> # now copy the file to the plaintext device and drop caches
> $ dd if=/src.img of=/dev/mapper/tst bs=1M count=16
>
> $ echo 3 > /proc/sys/vm/drop_caches
>
> # and verify that we are (not) reading the same data ...
>
> $ dd if=/dev/mapper/tst of=/dst1.img bs=1M count=16
>
> $ sha256sum /src.img /dst1.img
> 5401119fa9975bbeebac58e0b2598bc87247a29e62417f9f58fe200b531602ad /src.img
> e9bf5efa95031fdb5adf618db141f48ed23f71b12c017b8a0cbe0a694f18b979 /dst1.img
>
> (I think only first page-sized block is correct, because without direct-io
> it writes in page-sized IOs.)
>
>
> ... or just try to mkfs and mount it
> $ mkfs -t ext4 /dev/mapper/tst
>
> mke2fs 1.42.13 (17-May-2015)
> Creating filesystem with 262144 4k blocks and 65536 inodes
> ...
>
> $ mount /dev/mapper/tst /mnt/tst
> mount: wrong fs type, bad option, bad superblock on /dev/mapper/tst,
> missing codepage or helper program, or other error
>
>
> You approach simply does not work. (It will probably work for ECB mode but it is
> unusable in real world.)
>
>
> Anyway, I think that you should optimize driver, not add strange hw-dependent
> crypto modes to dmcrypt. This is not the first crypto accelerator that is just not
> suited for this kind of use.
Very grateful for your feedback. I'm sorry I didn't check much data
correctness, mostly focus on the encryption speed. It looks like there
are something wrong when I follow your test procedure. I will optimize
the driver and need to be known much about XTS mode to check why it
can not work. Thanks.
>
> (If it can process batch of chunks of data each with own IV, then it can work
> with dmcrypt, but I think such optimized code should be inside crypto API,
> not in dmcrypt.)
>
> Milan
--
Baolin.wang
Best Regards
^ permalink raw reply
* Re: RAID 6 "Failed to restore critical section for reshape, sorry." - recovery advice?
From: NeilBrown @ 2016-01-04 2:16 UTC (permalink / raw)
To: George Rapp; +Cc: linux-raid
In-Reply-To: <CAF-KpgZ=HY_HKvj5buFOKseUV0GLeOLR1m3B0EYxrYcD3R5ieA@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 1830 bytes --]
On Sun, Dec 27 2015, George Rapp wrote:
> Please find attached the output of the following command:
>
> # tar -c -v -z --sparse -f dev-md4-mdadm-dump.tar.gz dev-md4-mdadm-dump
>
> Thanks again for your help!
Thanks.
If you apply the following patch to mdadm (
git clone git://neil.brown.name/mdadm
apply patch
make
) and then try to assemble with --update=revert-reshape, it should
assemble as a 5-device array with no reshape happening.
I probably want more safety checks before this goes upstream, but it
is safe enough for you.
NeilBrown
diff --git a/super1.c b/super1.c
index 10e00652c4ee..efc0491fc94d 100644
--- a/super1.c
+++ b/super1.c
@@ -1317,6 +1317,17 @@ static int update_super1(struct supertype *st, struct mdinfo *info,
unsigned long long reshape_sectors;
long reshape_chunk;
rv = 0;
+ /* If the reshape hasn't started, just stop it */
+ if (sb->reshape_position == 0 &&
+ (__le32_to_cpu(sb->delta_disks) > 0 ||
+ (__le32_to_cpu(sb->delta_disks) == 0 &&
+ !(sb->feature_map & __cpu_to_le32(MD_FEATURE_RESHAPE_BACKWARDS))))) {
+ sb->feature_map &= ~__cpu_to_le32(MD_FEATURE_RESHAPE_ACTIVE);
+ sb->raid_disks = __cpu_to_le32(__le32_to_cpu(sb->raid_disks) -
+ __le32_to_cpu(sb->delta_disks));
+ sb->delta_disks = 0;
+ goto done;
+ }
/* reshape_position is a little messy.
* Its value must be a multiple of the larger
* chunk size, and of the "after" data disks.
@@ -1363,6 +1374,7 @@ static int update_super1(struct supertype *st, struct mdinfo *info,
sb->new_offset = __cpu_to_le32(-offset_delta);
sb->data_size = __cpu_to_le64(__le64_to_cpu(sb->data_size) - offset_delta);
}
+ done:;
}
} else if (strcmp(update, "_reshape_progress")==0)
sb->reshape_position = __cpu_to_le64(info->reshape_progress);
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]
^ permalink raw reply related
* Re: [PATCH] md: remove unnecesary md_new_event_inintr
From: NeilBrown @ 2016-01-04 1:36 UTC (permalink / raw)
Cc: linux-raid, Guoqing Jiang
In-Reply-To: <1451270798-25056-1-git-send-email-gqjiang@suse.com>
[-- Attachment #1: Type: text/plain, Size: 1346 bytes --]
On Mon, Dec 28 2015, Guoqing Jiang wrote:
> md_new_event had removed sysfs_notify since 'commit 72a23c211e45
> ("Make sure all changes to md/sync_action are notified.")', so we
> can use md_new_event and delete md_new_event_inintr.
>
> Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
> ---
> drivers/md/md.c | 11 +----------
> 1 file changed, 1 insertion(+), 10 deletions(-)
>
> diff --git a/drivers/md/md.c b/drivers/md/md.c
> index c0c3e6d..43a1404 100644
> --- a/drivers/md/md.c
> +++ b/drivers/md/md.c
> @@ -205,15 +205,6 @@ void md_new_event(struct mddev *mddev)
> }
> EXPORT_SYMBOL_GPL(md_new_event);
>
> -/* Alternate version that can be called from interrupts
> - * when calling sysfs_notify isn't needed.
> - */
> -static void md_new_event_inintr(struct mddev *mddev)
> -{
> - atomic_inc(&md_event_count);
> - wake_up(&md_event_waiters);
> -}
> -
> /*
> * Enables to iterate over all existing md arrays
> * all_mddevs_lock protects this list.
> @@ -7209,7 +7200,7 @@ void md_error(struct mddev *mddev, struct md_rdev *rdev)
> md_wakeup_thread(mddev->thread);
> if (mddev->event_work.func)
> queue_work(md_misc_wq, &mddev->event_work);
> - md_new_event_inintr(mddev);
> + md_new_event(mddev);
> }
> EXPORT_SYMBOL(md_error);
>
> --
> 2.6.2
Applied, thanks.
NeilBrown
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]
^ permalink raw reply
* Re: raid5 async_xor: sleep in atomic
From: NeilBrown @ 2016-01-04 1:33 UTC (permalink / raw)
To: Stanislav Samsonov, Dan Williams; +Cc: linux-raid
In-Reply-To: <CAFskJNg9_gJ3HRMqD4N6FSue2R2W3CSz+1KwiZw1Bp6==dOp5A@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 2376 bytes --]
On Mon, Dec 28 2015, Stanislav Samsonov wrote:
> On 24 December 2015 at 00:46, Dan Williams <dan.j.williams@intel.com> wrote:
>>
>> On Wed, Dec 23, 2015 at 2:39 PM, NeilBrown <neilb@suse.com> wrote:
>> > On Thu, Dec 24 2015, Dan Williams wrote:
>> >>> Changing the GFP_NOIO to GFP_ATOMIC in all the calls to
>> >>> dmaengine_get_unmap_data() in crypto/async_tx/ would probably fix the
>> >>> issue... or make it crash even worse :-)
>> >>>
>> >>> Dan: do you have any wisdom here? The xor is using the percpu data in
>> >>> raid5, so it cannot be sleep, but GFP_NOIO allows sleep.
>> >>> Does the code handle failure to get_unmap_data() safely? It looks like
>> >>> it probably does.
>> >>
>> >> Those GFP_NOIO should move to GFP_NOWAIT. We don't want GFP_ATOMIC
>> >> allocations to consume emergency reserves for a performance
>> >> optimization. Longer term async_tx needs to be merged into md
>> >> directly as we can allocate this unmap data statically per-stripe
>> >> rather than per request. This asyntc_tx re-write has been on the todo
>> >> list for years, but never seems to make it to the top.
>> >
>> > So the following maybe?
>> > If I could get an acked-by from you Dan, and a Tested-by: from you
>> > Slava, I'll submit upstream.
>> >
>> > Thanks,
>> > NeilBrown
>> >
>> > From: NeilBrown <neilb@suse.com>
>> > Date: Thu, 24 Dec 2015 09:35:18 +1100
>> > Subject: [PATCH] async_tx: use GFP_NOWAIT rather than GFP_IO
>> >
>> > These async_XX functions are called from md/raid5 in an atomic
>> > section, between get_cpu() and put_cpu(), so they must not sleep.
>> > So use GFP_NOWAIT rather than GFP_IO.
>> >
>> > Dan Williams writes: Longer term async_tx needs to be merged into md
>> > directly as we can allocate this unmap data statically per-stripe
>> > rather than per request.
>> >
>> > Reported-by: Stanislav Samsonov <slava@annapurnalabs.com>
>> > Signed-off-by: NeilBrown <neilb@suse.com>
>>
>> Acked-by: Dan Williams <dan.j.williams@intel.com>
>
> Tested-by: Slava Samsonov <slava@annapurnalabs.com>
Thanks.
I guess this was problem was introduced by
Commit: 7476bd79fc01 ("async_pq: convert to dmaengine_unmap_data")
in 3.13. Do we think it deserves to go to -stable?
(I just realised that this is really Dan's code more than mine,
so why am I submitting it ??? But we are here now so it may as well go
in through the md tree.)
NeilBrown
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]
^ permalink raw reply
* Intel IMSM RAID 5 won't start
From: Guido D'Arezzo @ 2016-01-03 19:44 UTC (permalink / raw)
To: linux-raid
Hi
After 20 months trouble-free Intel IMSM RAID, I had to do a hard reset
and the array has failed to start. I don’t know if the failed RAID
was the cause of the problems before the reset. The system won’t boot
because everything is on the RAID array. Booting from a live Fedora
USB shows no sign that the discs are broken and I was able to copy 1
GB off each disc with dd. I hope someone can help me to rescue the
array.
It is a 4 x 1 TB disc RAID 5 array. The system was running Archlinux
and I had patched it a day or 2 before for the first time in a few
months, thought it had been rebooted more than once afterwards without
incident.
The Intel oROM says disc 2 is “Offline Member” and 3 is “Failed Disk”.
-----------------------------------------------------------------------
Intel(R) Rapid Storage Technology - Option ROM - 11.6.0.1702
RAID Volumes:
ID Name Level Strip Size Status Bootable
O md0 RAID5(Parity) 128KB 2.6TB Failed No
1 mdl RAID5(Parity) 128KB 94.5GB Failed No
Physical Devices:
ID Device Model Serial # Size Type/Status(Vol ID)
O WDC WD10EZEK-00K WD-ACC1S5684189 931.5GB Member Disk(0,1)
1 SAMSUNG HD103UJ S13PJDAS608384 931.5GB Member Disk(O,1)
2 SAMSUNG HD103SJ SZ46J9GZC04Z67 931.5GB Offline Member
3 SAMSUNG HD103UJ S13PJDAS608386 931.5GB Unknown Disk
4 WDC WD10EZEK-08M WD-ACC3F1681668 931.5GB Non-RAID Disk
-----------------------------------------------------------------------
The 2 RAID volumes were both spread across all 4 discs. This is how
it looks now:
# mdadm -D /dev/md/imsm0
/dev/md/imsm0:
Version : imsm
Raid Level : container
Total Devices : 1
Working Devices : 1
UUID : 76cff3f5:1a3a7a83:49fc86a8:84cf6604
Member Arrays :
Number Major Minor RaidDevice
0 8 48 - /dev/sdd
#
# mdadm -D /dev/md/imsm1
/dev/md/imsm1:
Version : imsm
Raid Level : container
Total Devices : 3
Working Devices : 3
UUID : e8286680:de9642f4:04200a4a:acbdb566
Member Arrays :
Number Major Minor RaidDevice
0 8 16 - /dev/sdb
1 8 32 - /dev/sdc
2 8 0 - /dev/sda
#
# mdadm --detail-platform
Platform : Intel(R) Matrix Storage Manager
Version : 11.6.0.1702
RAID Levels : raid0 raid1 raid10 raid5
Chunk Sizes : 4k 8k 16k 32k 64k 128k
2TB volumes : supported
2TB disks : supported
Max Disks : 6
Max Volumes : 2 per array, 4 per controller
I/O Controller : /sys/devices/pci0000:00/0000:00:1f.2 (SATA)
#
# mdadm --examine /dev/sd[abcd]
/dev/sda:
Magic : Intel Raid ISM Cfg Sig.
Version : 1.3.00
Orig Family : d12e9b21
Family : d12e9b21
Generation : 00695bbd
Attributes : All supported
UUID : e8286680:de9642f4:04200a4a:acbdb566
Checksum : 8f6fe1cb correct
MPB Sectors : 2
Disks : 4
RAID Devices : 2
Disk01 Serial : WD-WCC1S5684189
State : active
Id : 00000000
Usable Size : 1953518862 (931.51 GiB 1000.20 GB)
[md0]:
UUID : d5bf7ab7:2cda417d:f0c6542f:c77d9289
RAID Level : 5
Members : 4
Slots : [_U_U]
Failed disk : 2
This Slot : 1
Array Size : 5662310400 (2700.00 GiB 2899.10 GB)
Per Dev Size : 1887436800 (900.00 GiB 966.37 GB)
Sector Offset : 0
Num Stripes : 7372800
Chunk Size : 128 KiB
Reserved : 0
Migrate State : idle
Map State : failed
Dirty State : clean
[md1]:
UUID : 26671da2:0d23f085:3d12dbbe:f63aad5a
RAID Level : 5
Members : 4
Slots : [__UU]
Failed disk : 0
This Slot : 2
Array Size : 198232064 (94.52 GiB 101.49 GB)
Per Dev Size : 66077952 (31.51 GiB 33.83 GB)
Sector Offset : 1887440896
Num Stripes : 258117
Chunk Size : 128 KiB
Reserved : 0
Migrate State : idle
Map State : failed
Dirty State : clean
Disk00 Serial : PJDWS608386:0:0
State : active
Id : ffffffff
Usable Size : 1953518862 (931.51 GiB 1000.20 GB)
Disk02 Serial : 6J9GZC04267:0:0
State : active failed
Id : ffffffff
Usable Size : 1953518862 (931.51 GiB 1000.20 GB)
Disk03 Serial : S13PJDWS608384
State : active
Id : 00000001
Usable Size : 1953518862 (931.51 GiB 1000.20 GB)
/dev/sdb:
Magic : Intel Raid ISM Cfg Sig.
Version : 1.3.00
Orig Family : d12e9b21
Family : d12e9b21
Generation : 00695bbd
Attributes : All supported
UUID : e8286680:de9642f4:04200a4a:acbdb566
Checksum : 8f6fe1cb correct
MPB Sectors : 2
Disks : 4
RAID Devices : 2
Disk03 Serial : S13PJDWS608384
State : active
Id : 00000001
Usable Size : 1953518862 (931.51 GiB 1000.20 GB)
[md0]:
UUID : d5bf7ab7:2cda417d:f0c6542f:c77d9289
RAID Level : 5
Members : 4
Slots : [_U_U]
Failed disk : 2
This Slot : 3
Array Size : 5662310400 (2700.00 GiB 2899.10 GB)
Per Dev Size : 1887436800 (900.00 GiB 966.37 GB)
Sector Offset : 0
Num Stripes : 7372800
Chunk Size : 128 KiB
Reserved : 0
Migrate State : idle
Map State : failed
Dirty State : clean
[md1]:
UUID : 26671da2:0d23f085:3d12dbbe:f63aad5a
RAID Level : 5
Members : 4
Slots : [__UU]
Failed disk : 0
This Slot : 3
Array Size : 198232064 (94.52 GiB 101.49 GB)
Per Dev Size : 66077952 (31.51 GiB 33.83 GB)
Sector Offset : 1887440896
Num Stripes : 258117
Chunk Size : 128 KiB
Reserved : 0
Migrate State : idle
Map State : failed
Dirty State : clean
Disk00 Serial : PJDWS608386:0:0
State : active
Id : ffffffff
Usable Size : 1953518862 (931.51 GiB 1000.20 GB)
Disk01 Serial : WD-WCC1S5684189
State : active
Id : 00000000
Usable Size : 1953518862 (931.51 GiB 1000.20 GB)
Disk02 Serial : 6J9GZC04267:0:0
State : active failed
Id : ffffffff
Usable Size : 1953518862 (931.51 GiB 1000.20 GB)
/dev/sdc:
Magic : Intel Raid ISM Cfg Sig.
Version : 1.3.00
Orig Family : d12e9b21
Family : d12e9b21
Generation : 00695b88
Attributes : All supported
UUID : e8286680:de9642f4:04200a4a:acbdb566
Checksum : a72daa29 correct
MPB Sectors : 2
Disks : 4
RAID Devices : 2
Disk02 Serial : S246J9GZC04267
State : active
Id : 00000002
Usable Size : 1953518862 (931.51 GiB 1000.20 GB)
[md0]:
UUID : d5bf7ab7:2cda417d:f0c6542f:c77d9289
RAID Level : 5
Members : 4
Slots : [UUUU]
Failed disk : none
This Slot : 2
Array Size : 5662310400 (2700.00 GiB 2899.10 GB)
Per Dev Size : 1887436800 (900.00 GiB 966.37 GB)
Sector Offset : 0
Num Stripes : 7372800
Chunk Size : 128 KiB
Reserved : 0
Migrate State : idle
Map State : normal
Dirty State : dirty
[md1]:
UUID : 26671da2:0d23f085:3d12dbbe:f63aad5a
RAID Level : 5
Members : 4
Slots : [UUUU]
Failed disk : none
This Slot : 0
Array Size : 198232064 (94.52 GiB 101.49 GB)
Per Dev Size : 66077952 (31.51 GiB 33.83 GB)
Sector Offset : 1887440896
Num Stripes : 258117
Chunk Size : 128 KiB
Reserved : 0
Migrate State : idle
Map State : normal
Dirty State : clean
Disk00 Serial : S13PJDWS608386
State : active
Id : 00000003
Usable Size : 1953518862 (931.51 GiB 1000.20 GB)
Disk01 Serial : WD-WCC1S5684189
State : active
Id : 00000000
Usable Size : 1953518862 (931.51 GiB 1000.20 GB)
Disk03 Serial : S13PJDWS608384
State : active
Id : 00000001
Usable Size : 1953518862 (931.51 GiB 1000.20 GB)
/dev/sdd:
Magic : Intel Raid ISM Cfg Sig.
Version : 1.0.00
Orig Family : c7e42747
Family : c7e42747
Generation : 00000000
Attributes : All supported
UUID : 76cff3f5:1a3a7a83:49fc86a8:84cf6604
Checksum : 4f820c2e correct
MPB Sectors : 1
Disks : 1
RAID Devices : 0
Disk00 Serial : S13PJDWS608386
State :
Id : 00000003
Usable Size : 1953518862 (931.51 GiB 1000.20 GB)
#
Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Charity Donation
From: Jeff Skoll @ 2016-01-03 10:21 UTC (permalink / raw)
To: Me
Hi,
My name is Jeffrey Skoll, a philanthropist and the founder of one of the largest private foundations in the world. I believe strongly in ‘giving while living.’ I had one idea that never changed in my mind — that you should use your wealth to help people and I have decided to secretly give USD2.498 Million to a randomly selected individual. On receipt of this email, you should count yourself as the individual. Kindly get back to me at your earliest convenience, so I know your email address is valid.
Visit the web page to know more about me: http://www.theglobeandmail.com/news/national/meet-the-canadian-billionaire-whos-giving-it-all-away/article4209888/ or you can read an article of me on Wikipedia.
Regards,
Jeffrey Skoll.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH v2 0/2] Introduce the bulk IV mode for improving the crypto engine efficiency
From: Milan Broz @ 2016-01-02 22:46 UTC (permalink / raw)
To: Baolin Wang
Cc: Jens Axboe, Alasdair G Kergon, Mike Snitzer, dm-devel, neilb,
dan.j.williams, martin.petersen, sagig, Kent Overstreet,
keith.busch, tj, Mark Brown, Arnd Bergmann, linux-block,
linux-raid, LKML
In-Reply-To: <CAMz4kuL2b5Qad5iD3odNVh9VO6ctu_M=QE=8fbdC_+ea+5wJ5w@mail.gmail.com>
On 12/17/2015 08:37 AM, Baolin Wang wrote:
> Hi Milan,
>
> On 16 December 2015 at 16:08, Milan Broz <gmazyland@gmail.com> wrote:
>> On 12/16/2015 04:18 AM, Baolin Wang wrote:
>>> From the dm-crypt performance report, we found it shows low efficiency
>>> with crypto engine for some mode (like ecb or xts mode). Because in dm
>>> crypt, it will map the IO data buffer with scatterlist, and send the
>>> scatterlist of one bio to the encryption engine, if send more scatterlists
>>> with bigger size at one time, that helps the engine palys best performance,
>>> which means a high encryption speed.
>>>
>>> But now the dm-crypt only map one segment (always one sector) of one bio
>>> with one scatterlist to the crypto engine at one time. which is more
>>> time-consuming and ineffective for the crypto engine. Especially for some
>>> modes which don't need different IV for each sector, we can map the whole
>>> bio with multiple scatterlists to improve the engine performance.
>>>
>>> But this optimization is not support some ciphers and IV modes which should
>>> do sector by sector and need different IV for each sector.
>>>
>>> Change since v1:
>>> - Introduce one different IV mode.
>>> - Change the conditions for bulk mode.
>>
>> I tried the patchset on 32bit Intel VM and kernel immediately OOPsed (just tried aes-ecb)...
>>
>
> I've checked the code and I guess some macros I used with different
> definitions on different arch. Could you please try the new patchset
> with some optimization on your platform? It can work well on my arm
> board. Thanks.
Sorry for delay, I tried to compile it.
It doesn't crash now, but it also does not work.
You usage of IV in XTS mode is not correct - it cannot just work this way,
you have to initialize IV after each block. And just one write not aligned
to your large XTS block will corrupt it.
Did you tried to _read_ data you write to the device?
See this test :
# create device with your patch
$ echo "test"|cryptsetup create -s 512 -c aes-xts-bulk tst /dev/sdg
# prepare random test file
$ dd if=/dev/urandom of=/src.img bs=1M count=16
# now copy the file to the plaintext device and drop caches
$ dd if=/src.img of=/dev/mapper/tst bs=1M count=16
$ echo 3 > /proc/sys/vm/drop_caches
# and verify that we are (not) reading the same data ...
$ dd if=/dev/mapper/tst of=/dst1.img bs=1M count=16
$ sha256sum /src.img /dst1.img
5401119fa9975bbeebac58e0b2598bc87247a29e62417f9f58fe200b531602ad /src.img
e9bf5efa95031fdb5adf618db141f48ed23f71b12c017b8a0cbe0a694f18b979 /dst1.img
(I think only first page-sized block is correct, because without direct-io
it writes in page-sized IOs.)
... or just try to mkfs and mount it
$ mkfs -t ext4 /dev/mapper/tst
mke2fs 1.42.13 (17-May-2015)
Creating filesystem with 262144 4k blocks and 65536 inodes
...
$ mount /dev/mapper/tst /mnt/tst
mount: wrong fs type, bad option, bad superblock on /dev/mapper/tst,
missing codepage or helper program, or other error
You approach simply does not work. (It will probably work for ECB mode but it is
unusable in real world.)
Anyway, I think that you should optimize driver, not add strange hw-dependent
crypto modes to dmcrypt. This is not the first crypto accelerator that is just not
suited for this kind of use.
(If it can process batch of chunks of data each with own IV, then it can work
with dmcrypt, but I think such optimized code should be inside crypto API,
not in dmcrypt.)
Milan
^ permalink raw reply
* Charity Donation
From: Jeff Skoll @ 2016-01-02 21:03 UTC (permalink / raw)
To: Me
Hi,
My name is Jeffrey Skoll, a philanthropist and the founder of one of the largest private foundations in the world. I believe strongly in ‘giving while living.’ I had one idea that never changed in my mind — that you should use your wealth to help people and I have decided to secretly give USD2.498 Million to a randomly selected individual. On receipt of this email, you should count yourself as the individual. Kindly get back to me at your earliest convenience, so I know your email address is valid.
Visit the web page to know more about me: http://www.theglobeandmail.com/news/national/meet-the-canadian-billionaire-whos-giving-it-all-away/article4209888/ or you can read an article of me on Wikipedia.
Regards,
Jeffrey Skoll.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Problems reassembling raid6 with --force option
From: René @ 2016-01-02 18:13 UTC (permalink / raw)
To: linux-raid
Hi,
I have a software raid6 consisting of 6 drives (sd[f-k] at the moment) at md1.
For maintainance I deactivated the array via mdadm --stop /dev/md1
As I wanted to reassamble the array later I got the following:
# mdadm --assemble -v /dev/md1
mdadm: added /dev/sdk1 to /dev/md1 as 0 (possibly out of date)
mdadm: added /dev/sdh1 to /dev/md1 as 2
mdadm: added /dev/sdf1 to /dev/md1 as 3
mdadm: added /dev/sdj1 to /dev/md1 as 4
mdadm: added /dev/sdg1 to /dev/md1 as 5 (possibly out of date)
mdadm: added /dev/sdi1 to /dev/md1 as 1
mdadm: /dev/md1 has been started with 4 drives (out of 6).
The event counts of the drives were only off by 3:
# mdadm --examine /dev/sd[fghijk]1 | egrep 'Events|/dev/sd'
/dev/sdf1:
Events : 17405
/dev/sdg1:
Events : 17402
/dev/sdh1:
Events : 17405
/dev/sdi1:
Events : 17405
/dev/sdj1:
Events : 17405
/dev/sdk1:
Events : 17402
Reading the man-page and searching the internet I thought --force should do the trick. But it didn't:
# mdadm --assemble -v --force /dev/md1 /dev/sd[fghijk]1
mdadm: looking for devices for /dev/md1
mdadm: /dev/sdf1 is identified as a member of /dev/md1, slot 3.
mdadm: /dev/sdg1 is identified as a member of /dev/md1, slot 5.
mdadm: /dev/sdh1 is identified as a member of /dev/md1, slot 2.
mdadm: /dev/sdi1 is identified as a member of /dev/md1, slot 1.
mdadm: /dev/sdj1 is identified as a member of /dev/md1, slot 4.
mdadm: /dev/sdk1 is identified as a member of /dev/md1, slot 0.
mdadm: added /dev/sdk1 to /dev/md1 as 0 (possibly out of date)
mdadm: added /dev/sdh1 to /dev/md1 as 2
mdadm: added /dev/sdf1 to /dev/md1 as 3
mdadm: added /dev/sdj1 to /dev/md1 as 4
mdadm: added /dev/sdg1 to /dev/md1 as 5 (possibly out of date)
mdadm: added /dev/sdi1 to /dev/md1 as 1
mdadm: /dev/md1 has been started with 4 drives (out of 6).
After looking at the code of mdadm I got to this bit around the forced assembly of an array (Assemble.c):
static int force_array(struct mdinfo *content,
struct devs *devices,
int *best, int bestcnt, char *avail,
int most_recent,
struct supertype *st,
struct context *c)
{
int okcnt = 0;
while (!enough(content->array.level, content->array.raid_disks,
content->array.layout, 1,
avail)
||
(content->reshape_active && content->delta_disks > 0 &&
!enough(content->array.level, (content->array.raid_disks
- content->delta_disks),
content->new_layout, 1,
avail)
)) {
...
}
return okcnt;
}
So it only updates the event count, when it doesn't have enough disks to start the array. Because only two of my drives were "out of date" and it had four valid drives the --force did nothing.
Running the assembly with one of the up-to-date drives missing (replaced sdj1 with sdx1 on the command line) worked:
# mdadm --assemble -fv /dev/md1 /dev/sd[fghixk]1
mdadm: looking for devices for /dev/md1
mdadm: /dev/sdf1 is identified as a member of /dev/md1, slot 3.
mdadm: /dev/sdg1 is identified as a member of /dev/md1, slot 5.
mdadm: /dev/sdh1 is identified as a member of /dev/md1, slot 2.
mdadm: /dev/sdi1 is identified as a member of /dev/md1, slot 1.
mdadm: /dev/sdk1 is identified as a member of /dev/md1, slot 0.
mdadm: forcing event count in /dev/sdk1(0) from 17402 upto 17405
mdadm: forcing event count in /dev/sdg1(5) from 17402 upto 17405
mdadm: added /dev/sdi1 to /dev/md1 as 1
mdadm: added /dev/sdh1 to /dev/md1 as 2
mdadm: added /dev/sdf1 to /dev/md1 as 3
mdadm: no uptodate device for slot 8 of /dev/md1
mdadm: added /dev/sdg1 to /dev/md1 as 5
mdadm: added /dev/sdk1 to /dev/md1 as 0
mdadm: /dev/md1 has been started with 5 drives (out of 6).
The intention of this behaviour might be that a rebuild is safer for data integrity when there are enough disks. (Because I shut down the array properly [at least I think so] and the event count was off by that little I chose to trick mdadm.)
Is that assumption on the intention of the code right? If so I think it should be mentioned in the man-page.
Regards,
René
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Charity Donation
From: Jeff Skoll @ 2016-01-02 17:50 UTC (permalink / raw)
To: Me
Hi,
My name is Jeffrey Skoll, a philanthropist and the founder of one of the largest private foundations in the world. I believe strongly in ‘giving while living.’ I had one idea that never changed in my mind — that you should use your wealth to help people and I have decided to secretly give USD2.498 Million to a randomly selected individual. On receipt of this email, you should count yourself as the individual. Kindly get back to me at your earliest convenience, so I know your email address is valid.
Visit the web page to know more about me: http://www.theglobeandmail.com/news/national/meet-the-canadian-billionaire-whos-giving-it-all-away/article4209888/ or you can read an article of me on Wikipedia.
Regards,
Jeffrey Skoll.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Charity Donation
From: Jeff Skoll @ 2016-01-02 17:07 UTC (permalink / raw)
To: Me
Hi,
My name is Jeffrey Skoll, a philanthropist and the founder of one of the largest private foundations in the world. I believe strongly in ‘giving while living.’ I had one idea that never changed in my mind — that you should use your wealth to help people and I have decided to secretly give USD2.498 Million to a randomly selected individual. On receipt of this email, you should count yourself as the individual. Kindly get back to me at your earliest convenience, so I know your email address is valid.
Visit the web page to know more about me: http://www.theglobeandmail.com/news/national/meet-the-canadian-billionaire-whos-giving-it-all-away/article4209888/ or you can read an article of me on Wikipedia.
Regards,
Jeffrey Skoll.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: Create Lock to Eliminate RMW in RAID/456 when writing perfect stripes
From: Doug Dumitru @ 2015-12-30 19:28 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid
In-Reply-To: <87h9j7yn5x.fsf@notabene.neil.brown.name>
A new lock for RAID-4/5/6 to minimize read/modify/write operations.
The Problem: The background thread for raid can wake up
asynchronously and will sometimes wake up and start processing a write
before the writing thread has finished updating the stripe cache
blocks. If the calling thread write was a long write (longer than
chunk size), then the background thread will configure a raid write
operation that is sub-optimal resulting in extra IO operations, slower
performance, and higher wear on Flash storage.
The Easy Fix: When the calling thread has a long write, it "locks"
the stripe number with a semaphore. When the background thread wakes
up and starts working on a stripe, it locks the same lock, and then
immediately releases it. This way the background thread will wait for
the write to fully populate the stripe caches before it start to build
a write request.
The Lock Structure: ... is just a small, binary sized, array of
semaphores. The lock is picked by just "stripe_number %
number_of_semaphores". There will be collisions, but they should be
short and lots of IO is happening for each operation, so a semaphore
is cheap enough.
Memory Usage: With 64 semaphores, this adds 1.5K to the raid control block.
A More Comprehensive Fix: This Easy Fix assumes that updates are
contained within a single BIO request. A better fix is to lock linear
operations that span BIOs by looking back on the queue. I don't have
a lot of experience playing with queues, but this is probably workable
with only a little more complexity, provided that you don't try to do
this across cores and queues. Again, I don't play much in queues, so
I may be totally missing the structures here.
The Really High Performance Fix: If the application is well enough
behaved to write complete, perfect stripes contained in a single BIO
request, then the whole stripe cache logic can be bypassed. This lets
you submit the member disk IO operations directly from the calling
thread. I have this running in a patch in the field and it works
well, but the use case is very limited and something probably breaks
with more "normal" IO patterns. I have hit 11GB/sec with RAID-5 and
8GB/sec with RAID-6 this way with 24 SSDs.
Tweak-ability: All of these changes can be exposed in /sys to allow
sysadmins to tune their system possibly enabling or disabling
features. Most useful for early code that might have broken use
cases. Then again, too many knobs sometimes just increases confusion.
Asking for Feedback: I am happy to write "all of the above" and
submit it and work with the group to get it tested etc. If this
interests you, please comment on how far you think I should go. Also,
if there are any notes on "submission style", how and where to post
patches, which kernel version to patch/develop against, documentation
style, sign-off requirements, etc. please point me at them.
Thanks in advance,
Doug Dumitru
WildFire Storage
^ permalink raw reply
* raid5d hangs when stopping an array during reshape
From: Artur Paszkiewicz @ 2015-12-30 13:45 UTC (permalink / raw)
To: linux-raid
Hi,
I'm seeing a hang when trying to stop a RAID5 array that is undergoing
reshape:
[ 99.629924] md: reshape of RAID array md0
[ 99.631150] md: minimum _guaranteed_ speed: 1000 KB/sec/disk.
[ 99.632737] md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for reshape.
[ 99.635366] md: using 128k window, over a total of 1047552k.
[ 103.819848] md: md0: reshape interrupted.
[ 150.127132] INFO: task md0_raid5:3234 blocked for more than 30 seconds.
[ 150.128717] Not tainted 4.4.0-rc5+ #54
[ 150.129939] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 150.132116] md0_raid5 D ffff88003b1d7ba0 14104 3234 2 0x00000000
[ 150.134081] ffff88003b1d7ba0 ffffffff81e104c0 ffff88003bad0000 ffff88003b1d8000
[ 150.137205] ffff88003d66380c 0000000000000001 ffff88003d663a50 ffff88003d663800
[ 150.139994] ffff88003b1d7bb8 ffffffff81876050 ffff88003d663800 ffff88003b1d7c28
[ 150.142606] Call Trace:
[ 150.143551] [<ffffffff81876050>] schedule+0x30/0x80
[ 150.144883] [<ffffffffa005fc80>] raid5_quiesce+0x200/0x250 [raid456]
[ 150.147964] [<ffffffff810882a0>] ? prepare_to_wait_event+0xf0/0xf0
[ 150.149661] [<ffffffffa0003bca>] mddev_suspend.part.26+0x7a/0x90 [md_mod]
[ 150.151376] [<ffffffffa0003bf7>] mddev_suspend+0x17/0x20 [md_mod]
[ 150.153268] [<ffffffffa0064e29>] check_reshape+0xb9/0x6b0 [raid456]
[ 150.154869] [<ffffffff8107e63f>] ? set_next_entity+0x9f/0x6d0
[ 150.156359] [<ffffffff8107af68>] ? sched_clock_local+0x18/0x80
[ 150.157848] [<ffffffff81081400>] ? pick_next_entity+0xa0/0x150
[ 150.159348] [<ffffffff810830ae>] ? pick_next_task_fair+0x3fe/0x460
[ 150.160887] [<ffffffffa0065471>] raid5_check_reshape+0x51/0xa0 [raid456]
[ 150.162482] [<ffffffffa000ba59>] md_check_recovery+0x2f9/0x480 [md_mod]
[ 150.164074] [<ffffffffa00697b4>] raid5d+0x34/0x650 [raid456]
[ 150.165751] [<ffffffff81876050>] ? schedule+0x30/0x80
[ 150.167508] [<ffffffff818786ef>] ? schedule_timeout+0x1ef/0x270
[ 150.169784] [<ffffffff81875ac3>] ? __schedule+0x313/0x870
[ 150.171194] [<ffffffffa0002e61>] md_thread+0x111/0x130 [md_mod]
[ 150.172671] [<ffffffff810882a0>] ? prepare_to_wait_event+0xf0/0xf0
[ 150.174206] [<ffffffffa0002d50>] ? find_pers+0x70/0x70 [md_mod]
[ 150.175697] [<ffffffff8106c8d4>] kthread+0xc4/0xe0
[ 150.178294] [<ffffffff8106c810>] ? kthread_park+0x50/0x50
[ 150.179745] [<ffffffff818796df>] ret_from_fork+0x3f/0x70
[ 150.181134] [<ffffffff8106c810>] ? kthread_park+0x50/0x50
Two tasks end up blocked:
3866 ? D 0:00 [systemd-udevd]
4051 ? D 0:00 [md0_raid5]
This happens when udev change event is triggered by mdadm -S and it
causes some reads on the array. I think the hang occurs because
raid5_quiesce() is called from the raid5d thread and it blocks waiting
for active_stripes to become 0, which won't happen, since stripes are
released by raid5d. Commit 738a273 ("md/raid5: fix allocation of
'scribble' array.") added mddev_suspend() in resize_chunks(), causing
this problem. Skipping mddev_suspend()/mddev_resume() in resize_chunks()
when running in raid5d context seems to fix it, but I don't think that's
a correct fix...
Regards,
Artur
^ permalink raw reply
* Assalam o Alaikum
From: Zahra Robert @ 2015-12-29 2:03 UTC (permalink / raw)
To: jakonyvtar
Cordial greeting message from Eva Zahra, I am seeking for your help,I will
be very glad if you do assist me to relocate a sum of (US$4 Million
Dollars)into your Bank account in your country for the benefit of both of
us i want to use this money for investment. I will give you more details
as you reply Yours Eva Zahra Robert
^ permalink raw reply
* Charity Donation
From: Jeff Skoll @ 2015-12-28 22:09 UTC (permalink / raw)
To: Me
Hi,
My name is Jeffrey Skoll, a philanthropist and the founder of one of the largest private foundations in the world. I believe strongly in ‘giving while living.’ I had one idea that never changed in my mind — that you should use your wealth to help people and I have decided to secretly give USD2.498 Million to a randomly selected individual. On receipt of this email, you should count yourself as the individual. Kindly get back to me at your earliest convenience, so I know your email address is valid.
Visit the web page to know more about me: http://www.theglobeandmail.com/news/national/meet-the-canadian-billionaire-whos-giving-it-all-away/article4209888/ or you can read an article of me on Wikipedia.
Regards,
Jeffrey Skoll.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: raid5 async_xor: sleep in atomic
From: Stanislav Samsonov @ 2015-12-28 8:43 UTC (permalink / raw)
To: Dan Williams; +Cc: NeilBrown, linux-raid
In-Reply-To: <CAPcyv4g=HBHn3f75gW9JKNdjYEnjxNCYEp3vjJBa_fGm7e+hJw@mail.gmail.com>
On 24 December 2015 at 00:46, Dan Williams <dan.j.williams@intel.com> wrote:
>
> On Wed, Dec 23, 2015 at 2:39 PM, NeilBrown <neilb@suse.com> wrote:
> > On Thu, Dec 24 2015, Dan Williams wrote:
> >>> Changing the GFP_NOIO to GFP_ATOMIC in all the calls to
> >>> dmaengine_get_unmap_data() in crypto/async_tx/ would probably fix the
> >>> issue... or make it crash even worse :-)
> >>>
> >>> Dan: do you have any wisdom here? The xor is using the percpu data in
> >>> raid5, so it cannot be sleep, but GFP_NOIO allows sleep.
> >>> Does the code handle failure to get_unmap_data() safely? It looks like
> >>> it probably does.
> >>
> >> Those GFP_NOIO should move to GFP_NOWAIT. We don't want GFP_ATOMIC
> >> allocations to consume emergency reserves for a performance
> >> optimization. Longer term async_tx needs to be merged into md
> >> directly as we can allocate this unmap data statically per-stripe
> >> rather than per request. This asyntc_tx re-write has been on the todo
> >> list for years, but never seems to make it to the top.
> >
> > So the following maybe?
> > If I could get an acked-by from you Dan, and a Tested-by: from you
> > Slava, I'll submit upstream.
> >
> > Thanks,
> > NeilBrown
> >
> > From: NeilBrown <neilb@suse.com>
> > Date: Thu, 24 Dec 2015 09:35:18 +1100
> > Subject: [PATCH] async_tx: use GFP_NOWAIT rather than GFP_IO
> >
> > These async_XX functions are called from md/raid5 in an atomic
> > section, between get_cpu() and put_cpu(), so they must not sleep.
> > So use GFP_NOWAIT rather than GFP_IO.
> >
> > Dan Williams writes: Longer term async_tx needs to be merged into md
> > directly as we can allocate this unmap data statically per-stripe
> > rather than per request.
> >
> > Reported-by: Stanislav Samsonov <slava@annapurnalabs.com>
> > Signed-off-by: NeilBrown <neilb@suse.com>
>
> Acked-by: Dan Williams <dan.j.williams@intel.com>
Tested-by: Slava Samsonov <slava@annapurnalabs.com>
^ permalink raw reply
* [PATCH] md: remove unnecesary md_new_event_inintr
From: Guoqing Jiang @ 2015-12-28 2:46 UTC (permalink / raw)
To: neilb; +Cc: linux-raid, Guoqing Jiang
md_new_event had removed sysfs_notify since 'commit 72a23c211e45
("Make sure all changes to md/sync_action are notified.")', so we
can use md_new_event and delete md_new_event_inintr.
Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
---
drivers/md/md.c | 11 +----------
1 file changed, 1 insertion(+), 10 deletions(-)
diff --git a/drivers/md/md.c b/drivers/md/md.c
index c0c3e6d..43a1404 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -205,15 +205,6 @@ void md_new_event(struct mddev *mddev)
}
EXPORT_SYMBOL_GPL(md_new_event);
-/* Alternate version that can be called from interrupts
- * when calling sysfs_notify isn't needed.
- */
-static void md_new_event_inintr(struct mddev *mddev)
-{
- atomic_inc(&md_event_count);
- wake_up(&md_event_waiters);
-}
-
/*
* Enables to iterate over all existing md arrays
* all_mddevs_lock protects this list.
@@ -7209,7 +7200,7 @@ void md_error(struct mddev *mddev, struct md_rdev *rdev)
md_wakeup_thread(mddev->thread);
if (mddev->event_work.func)
queue_work(md_misc_wq, &mddev->event_work);
- md_new_event_inintr(mddev);
+ md_new_event(mddev);
}
EXPORT_SYMBOL(md_error);
--
2.6.2
^ permalink raw reply related
* Re: Recreating RAID-5 after SATA controller failure and botched auto-assemble
From: Phil Turmel @ 2015-12-27 23:11 UTC (permalink / raw)
To: Wolfgang Draxinger, linux-raid
In-Reply-To: <20151223105827.55498cb1@narfi.yggdrasil.draxit.de>
On 12/23/2015 04:58 AM, Wolfgang Draxinger wrote:
> Anyway, I'd appreciate if anyone knew how I could read-only assemble
> the disks into an array that resembles the 42e59462... configuration to
> see if I can pull anything useful from it. There are a few things on
> it (hopefully left) that I'd like to extract, if possible.
I think it's gone. You only have two members of the 42e59462 uuid
array, and you need at least three. The others all have 2008 timestamps
so could not have been part of the desired array at any recent date.
Sorry.
You *could* assemble the other array, though, to see what's there:
mdadm -Afv /dev/mdX /dev/sdb1 /dev/sdb4 /dev/sdd
With the failure mode you described, I doubt the series of rebuilds
would have destroyed much.
Phil
^ permalink raw reply
* buone notizie per voi
From: hi @ 2015-12-26 4:44 UTC (permalink / raw)
To: quik_2_kurupt
Salve
buone notizie per voi
dj, tv, biciclette, orologi di marca, macchina fotografica ... la spedizione è gratuita
iphone 6s,330euro,samsung note5,260euro
si te: poazzlo .com
^ permalink raw reply
* Re: clustered MD - beyond RAID1
From: roger zhou @ 2015-12-25 8:47 UTC (permalink / raw)
To: Tejas Rao, Aaron Knister
Cc: NeilBrown, linux-raid-u79uwXL29TY76Z2rM5mHXA, Scott Sinno,
users-iJOQKD3IPJ/s2ve5o0e2fw
In-Reply-To: <5678B693.40907-IGkKxAqZmp0@public.gmane.org>
[-- Attachment #1.1: Type: text/plain, Size: 1179 bytes --]
On 12/22/2015 10:33 AM, Tejas Rao wrote:
> On 12/21/2015 20:50, Aaron Knister wrote:
>
[...]
>>
>> I'm curious now, Redhat doesn't support SW raid failover? I did some
>> googling and found this:
>>
>> https://access.redhat.com/solutions/231643
>>
>> While I can't read the solution I have to figure that they're now
>> supporting that. I might actually explore that for this project.
> https://access.redhat.com/solutions/410203
> This article states that md raid is not supported in RHEL6/7 under any
> circumstances, including active/passive modes.
OCFS2 or GFS2(same for GPFS, as the shared filesystem) over a shared
storage is a typical Cluster configuration for Linux High Availability.
Where, Clustered LVM (cLVM) is supported by both SUSE and Redhat to do
mirroring to protect the data. However, the performance loss is very big
and make people not so happy about this clustered mirror solution. This
is where the motivation for clustered MD solution comes from.
With clustered md, this new solution could provide nearly the same
performance as the native raid1. You may have interest to validate this
from your lab with your configuration;)
Cheers,
Roger
[-- Attachment #1.2: Type: text/html, Size: 2232 bytes --]
[-- Attachment #2: Type: text/plain, Size: 0 bytes --]
^ permalink raw reply
* Re: Create Lock to Eliminate RMW in RAID/456 when writing perfect stripes
From: NeilBrown @ 2015-12-25 7:58 UTC (permalink / raw)
To: doug, linux-raid
In-Reply-To: <CAFx4rwR1gtLZUF6MUx71QTvw3hT5GKhsEMU2xcBpAz5wZWBUCw@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 2740 bytes --]
On Thu, Dec 24 2015, Doug Dumitru wrote:
> The issue:
>
> The background thread in RAID-5 can wake up in the middle of a process
> populating stripe cache entries with a long write. If the long write
> contains a complete stripe, the background thread "should" be able to
> process the require without doing any reads.
>
> Sometimes the background thread is too quick at starting up a write
> and schedules a RMW (Read Modify Write) even though the needed blocks
> will soon be available.
>
> Seeing this happen:
>
> You can see this happen by creating an MD set with a small stripe size
> and then doing DIRECT_IO writes that are exactly aligned on a stripe.
> For example, with 4 disks and 64K stripes, write 192K blocks aligned
> on 192K boundaries. You can do this from C or with 'dd' or 'fio'.
>
> If you have this running, you can then run iostat and you should see
> absolutely no read activity on the disks.
>
> The probability of this happening goes up when there are more disks.
> It may also go up the faster the disks are. My use case is 24 SSDs.
>
> The problem with this:
>
> There are really three issues.
>
> 1) The code does not need to work this way. It is not "broken" but
> just seems wrong.
> 2) There is a performance penalty here.
> 3) There is a Flash wear penalty here.
>
> It is 3) that most interests me.
>
> The fix:
>
> Create a waitq or semaphore based lock so that if a write includes a
> complete stripe, the background thread will wait for the write to
> completely populate the thread.
>
> I would do this with a small array of locks. When a write includes a
> complete stripe, it sets a lock (stripe_number % sizeof_lock_array).
> This lock is released as soon as the write finishes populating the
> stripe cache. The background thread checks this lock before it starts
> a write. If the lock is set, it waits until the stripe cache is
> completely populated which should eliminate the RMW.
>
> If no writes are full stripes, then the lock never gets set, so most
> code runs without any real overhead.
>
> Implementing this:
>
> I am happy to implement this. I have quite a bit of experience with
> lock structures like this. I can also test on x86 and x86_64, but
> will need help with other arch's.
>
> Then again, if this is too much of an "edge case", I will just keep my
> patches in-house.
Hi,
this is certainly something that needs fixing. I can't really say if
your approach would work or not without seeing it and testing it on a
variety of work loads.
Certainly if you do implement something, please post it for other to
test and review. If it makes measurable improvements without causing
significant regressions, it will likely be included upstream.
Thanks,
NeilBrown
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]
^ permalink raw reply
* [PATCH v5 3/3] md: convert to use the generic badblocks code
From: Vishal Verma @ 2015-12-25 2:20 UTC (permalink / raw)
To: linux-nvdimm
Cc: Vishal Verma, linux-block, linux-raid, linux-scsi, Jens Axboe,
NeilBrown, Jeff Moyer, Dan Williams
In-Reply-To: <1451010034-11236-1-git-send-email-vishal.l.verma@intel.com>
Retain badblocks as part of rdev, but use the accessor functions from
include/linux/badblocks for all manipulation.
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
drivers/md/md.c | 516 +++-----------------------------------------------------
drivers/md/md.h | 40 +----
2 files changed, 28 insertions(+), 528 deletions(-)
diff --git a/drivers/md/md.c b/drivers/md/md.c
index dbedc58..51dc9f3 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -34,6 +34,7 @@
#include <linux/kthread.h>
#include <linux/blkdev.h>
+#include <linux/badblocks.h>
#include <linux/sysctl.h>
#include <linux/seq_file.h>
#include <linux/fs.h>
@@ -710,8 +711,7 @@ void md_rdev_clear(struct md_rdev *rdev)
put_page(rdev->bb_page);
rdev->bb_page = NULL;
}
- kfree(rdev->badblocks.page);
- rdev->badblocks.page = NULL;
+ badblocks_free(&rdev->badblocks);
}
EXPORT_SYMBOL_GPL(md_rdev_clear);
@@ -1361,8 +1361,6 @@ static __le32 calc_sb_1_csum(struct mdp_superblock_1 *sb)
return cpu_to_le32(csum);
}
-static int md_set_badblocks(struct badblocks *bb, sector_t s, int sectors,
- int acknowledged);
static int super_1_load(struct md_rdev *rdev, struct md_rdev *refdev, int minor_version)
{
struct mdp_superblock_1 *sb;
@@ -1487,8 +1485,7 @@ static int super_1_load(struct md_rdev *rdev, struct md_rdev *refdev, int minor_
count <<= sb->bblog_shift;
if (bb + 1 == 0)
break;
- if (md_set_badblocks(&rdev->badblocks,
- sector, count, 1) == 0)
+ if (badblocks_set(&rdev->badblocks, sector, count, 1))
return -EINVAL;
}
} else if (sb->bblog_offset != 0)
@@ -2320,7 +2317,7 @@ repeat:
rdev_for_each(rdev, mddev) {
if (rdev->badblocks.changed) {
rdev->badblocks.changed = 0;
- md_ack_all_badblocks(&rdev->badblocks);
+ ack_all_badblocks(&rdev->badblocks);
md_error(mddev, rdev);
}
clear_bit(Blocked, &rdev->flags);
@@ -2446,7 +2443,7 @@ repeat:
clear_bit(Blocked, &rdev->flags);
if (any_badblocks_changed)
- md_ack_all_badblocks(&rdev->badblocks);
+ ack_all_badblocks(&rdev->badblocks);
clear_bit(BlockedBadBlocks, &rdev->flags);
wake_up(&rdev->blocked_wait);
}
@@ -3054,11 +3051,17 @@ static ssize_t recovery_start_store(struct md_rdev *rdev, const char *buf, size_
static struct rdev_sysfs_entry rdev_recovery_start =
__ATTR(recovery_start, S_IRUGO|S_IWUSR, recovery_start_show, recovery_start_store);
-static ssize_t
-badblocks_show(struct badblocks *bb, char *page, int unack);
-static ssize_t
-badblocks_store(struct badblocks *bb, const char *page, size_t len, int unack);
-
+/* sysfs access to bad-blocks list.
+ * We present two files.
+ * 'bad-blocks' lists sector numbers and lengths of ranges that
+ * are recorded as bad. The list is truncated to fit within
+ * the one-page limit of sysfs.
+ * Writing "sector length" to this file adds an acknowledged
+ * bad block list.
+ * 'unacknowledged-bad-blocks' lists bad blocks that have not yet
+ * been acknowledged. Writing to this file adds bad blocks
+ * without acknowledging them. This is largely for testing.
+ */
static ssize_t bb_show(struct md_rdev *rdev, char *page)
{
return badblocks_show(&rdev->badblocks, page, 0);
@@ -3173,14 +3176,7 @@ int md_rdev_init(struct md_rdev *rdev)
* This reserves the space even on arrays where it cannot
* be used - I wonder if that matters
*/
- rdev->badblocks.count = 0;
- rdev->badblocks.shift = -1; /* disabled until explicitly enabled */
- rdev->badblocks.page = kmalloc(PAGE_SIZE, GFP_KERNEL);
- seqlock_init(&rdev->badblocks.lock);
- if (rdev->badblocks.page == NULL)
- return -ENOMEM;
-
- return 0;
+ return badblocks_init(&rdev->badblocks, 0);
}
EXPORT_SYMBOL_GPL(md_rdev_init);
/*
@@ -8486,254 +8482,9 @@ void md_finish_reshape(struct mddev *mddev)
}
EXPORT_SYMBOL(md_finish_reshape);
-/* Bad block management.
- * We can record which blocks on each device are 'bad' and so just
- * fail those blocks, or that stripe, rather than the whole device.
- * Entries in the bad-block table are 64bits wide. This comprises:
- * Length of bad-range, in sectors: 0-511 for lengths 1-512
- * Start of bad-range, sector offset, 54 bits (allows 8 exbibytes)
- * A 'shift' can be set so that larger blocks are tracked and
- * consequently larger devices can be covered.
- * 'Acknowledged' flag - 1 bit. - the most significant bit.
- *
- * Locking of the bad-block table uses a seqlock so md_is_badblock
- * might need to retry if it is very unlucky.
- * We will sometimes want to check for bad blocks in a bi_end_io function,
- * so we use the write_seqlock_irq variant.
- *
- * When looking for a bad block we specify a range and want to
- * know if any block in the range is bad. So we binary-search
- * to the last range that starts at-or-before the given endpoint,
- * (or "before the sector after the target range")
- * then see if it ends after the given start.
- * We return
- * 0 if there are no known bad blocks in the range
- * 1 if there are known bad block which are all acknowledged
- * -1 if there are bad blocks which have not yet been acknowledged in metadata.
- * plus the start/length of the first bad section we overlap.
- */
-int md_is_badblock(struct badblocks *bb, sector_t s, int sectors,
- sector_t *first_bad, int *bad_sectors)
-{
- int hi;
- int lo;
- u64 *p = bb->page;
- int rv;
- sector_t target = s + sectors;
- unsigned seq;
-
- if (bb->shift > 0) {
- /* round the start down, and the end up */
- s >>= bb->shift;
- target += (1<<bb->shift) - 1;
- target >>= bb->shift;
- sectors = target - s;
- }
- /* 'target' is now the first block after the bad range */
-
-retry:
- seq = read_seqbegin(&bb->lock);
- lo = 0;
- rv = 0;
- hi = bb->count;
-
- /* Binary search between lo and hi for 'target'
- * i.e. for the last range that starts before 'target'
- */
- /* INVARIANT: ranges before 'lo' and at-or-after 'hi'
- * are known not to be the last range before target.
- * VARIANT: hi-lo is the number of possible
- * ranges, and decreases until it reaches 1
- */
- while (hi - lo > 1) {
- int mid = (lo + hi) / 2;
- sector_t a = BB_OFFSET(p[mid]);
- if (a < target)
- /* This could still be the one, earlier ranges
- * could not. */
- lo = mid;
- else
- /* This and later ranges are definitely out. */
- hi = mid;
- }
- /* 'lo' might be the last that started before target, but 'hi' isn't */
- if (hi > lo) {
- /* need to check all range that end after 's' to see if
- * any are unacknowledged.
- */
- while (lo >= 0 &&
- BB_OFFSET(p[lo]) + BB_LEN(p[lo]) > s) {
- if (BB_OFFSET(p[lo]) < target) {
- /* starts before the end, and finishes after
- * the start, so they must overlap
- */
- if (rv != -1 && BB_ACK(p[lo]))
- rv = 1;
- else
- rv = -1;
- *first_bad = BB_OFFSET(p[lo]);
- *bad_sectors = BB_LEN(p[lo]);
- }
- lo--;
- }
- }
-
- if (read_seqretry(&bb->lock, seq))
- goto retry;
-
- return rv;
-}
-EXPORT_SYMBOL_GPL(md_is_badblock);
-
-/*
- * Add a range of bad blocks to the table.
- * This might extend the table, or might contract it
- * if two adjacent ranges can be merged.
- * We binary-search to find the 'insertion' point, then
- * decide how best to handle it.
- */
-static int md_set_badblocks(struct badblocks *bb, sector_t s, int sectors,
- int acknowledged)
-{
- u64 *p;
- int lo, hi;
- int rv = 1;
- unsigned long flags;
-
- if (bb->shift < 0)
- /* badblocks are disabled */
- return 0;
-
- if (bb->shift) {
- /* round the start down, and the end up */
- sector_t next = s + sectors;
- s >>= bb->shift;
- next += (1<<bb->shift) - 1;
- next >>= bb->shift;
- sectors = next - s;
- }
-
- write_seqlock_irqsave(&bb->lock, flags);
-
- p = bb->page;
- lo = 0;
- hi = bb->count;
- /* Find the last range that starts at-or-before 's' */
- while (hi - lo > 1) {
- int mid = (lo + hi) / 2;
- sector_t a = BB_OFFSET(p[mid]);
- if (a <= s)
- lo = mid;
- else
- hi = mid;
- }
- if (hi > lo && BB_OFFSET(p[lo]) > s)
- hi = lo;
-
- if (hi > lo) {
- /* we found a range that might merge with the start
- * of our new range
- */
- sector_t a = BB_OFFSET(p[lo]);
- sector_t e = a + BB_LEN(p[lo]);
- int ack = BB_ACK(p[lo]);
- if (e >= s) {
- /* Yes, we can merge with a previous range */
- if (s == a && s + sectors >= e)
- /* new range covers old */
- ack = acknowledged;
- else
- ack = ack && acknowledged;
-
- if (e < s + sectors)
- e = s + sectors;
- if (e - a <= BB_MAX_LEN) {
- p[lo] = BB_MAKE(a, e-a, ack);
- s = e;
- } else {
- /* does not all fit in one range,
- * make p[lo] maximal
- */
- if (BB_LEN(p[lo]) != BB_MAX_LEN)
- p[lo] = BB_MAKE(a, BB_MAX_LEN, ack);
- s = a + BB_MAX_LEN;
- }
- sectors = e - s;
- }
- }
- if (sectors && hi < bb->count) {
- /* 'hi' points to the first range that starts after 's'.
- * Maybe we can merge with the start of that range */
- sector_t a = BB_OFFSET(p[hi]);
- sector_t e = a + BB_LEN(p[hi]);
- int ack = BB_ACK(p[hi]);
- if (a <= s + sectors) {
- /* merging is possible */
- if (e <= s + sectors) {
- /* full overlap */
- e = s + sectors;
- ack = acknowledged;
- } else
- ack = ack && acknowledged;
-
- a = s;
- if (e - a <= BB_MAX_LEN) {
- p[hi] = BB_MAKE(a, e-a, ack);
- s = e;
- } else {
- p[hi] = BB_MAKE(a, BB_MAX_LEN, ack);
- s = a + BB_MAX_LEN;
- }
- sectors = e - s;
- lo = hi;
- hi++;
- }
- }
- if (sectors == 0 && hi < bb->count) {
- /* we might be able to combine lo and hi */
- /* Note: 's' is at the end of 'lo' */
- sector_t a = BB_OFFSET(p[hi]);
- int lolen = BB_LEN(p[lo]);
- int hilen = BB_LEN(p[hi]);
- int newlen = lolen + hilen - (s - a);
- if (s >= a && newlen < BB_MAX_LEN) {
- /* yes, we can combine them */
- int ack = BB_ACK(p[lo]) && BB_ACK(p[hi]);
- p[lo] = BB_MAKE(BB_OFFSET(p[lo]), newlen, ack);
- memmove(p + hi, p + hi + 1,
- (bb->count - hi - 1) * 8);
- bb->count--;
- }
- }
- while (sectors) {
- /* didn't merge (it all).
- * Need to add a range just before 'hi' */
- if (bb->count >= MD_MAX_BADBLOCKS) {
- /* No room for more */
- rv = 0;
- break;
- } else {
- int this_sectors = sectors;
- memmove(p + hi + 1, p + hi,
- (bb->count - hi) * 8);
- bb->count++;
-
- if (this_sectors > BB_MAX_LEN)
- this_sectors = BB_MAX_LEN;
- p[hi] = BB_MAKE(s, this_sectors, acknowledged);
- sectors -= this_sectors;
- s += this_sectors;
- }
- }
-
- bb->changed = 1;
- if (!acknowledged)
- bb->unacked_exist = 1;
- write_sequnlock_irqrestore(&bb->lock, flags);
-
- return rv;
-}
+/* Bad block management */
+/* Returns 1 on success, 0 on failure */
int rdev_set_badblocks(struct md_rdev *rdev, sector_t s, int sectors,
int is_new)
{
@@ -8742,114 +8493,19 @@ int rdev_set_badblocks(struct md_rdev *rdev, sector_t s, int sectors,
s += rdev->new_data_offset;
else
s += rdev->data_offset;
- rv = md_set_badblocks(&rdev->badblocks,
- s, sectors, 0);
- if (rv) {
+ rv = badblocks_set(&rdev->badblocks, s, sectors, 0);
+ if (rv == 0) {
/* Make sure they get written out promptly */
sysfs_notify_dirent_safe(rdev->sysfs_state);
set_bit(MD_CHANGE_CLEAN, &rdev->mddev->flags);
set_bit(MD_CHANGE_PENDING, &rdev->mddev->flags);
md_wakeup_thread(rdev->mddev->thread);
- }
- return rv;
+ return 1;
+ } else
+ return 0;
}
EXPORT_SYMBOL_GPL(rdev_set_badblocks);
-/*
- * Remove a range of bad blocks from the table.
- * This may involve extending the table if we spilt a region,
- * but it must not fail. So if the table becomes full, we just
- * drop the remove request.
- */
-static int md_clear_badblocks(struct badblocks *bb, sector_t s, int sectors)
-{
- u64 *p;
- int lo, hi;
- sector_t target = s + sectors;
- int rv = 0;
-
- if (bb->shift > 0) {
- /* When clearing we round the start up and the end down.
- * This should not matter as the shift should align with
- * the block size and no rounding should ever be needed.
- * However it is better the think a block is bad when it
- * isn't than to think a block is not bad when it is.
- */
- s += (1<<bb->shift) - 1;
- s >>= bb->shift;
- target >>= bb->shift;
- sectors = target - s;
- }
-
- write_seqlock_irq(&bb->lock);
-
- p = bb->page;
- lo = 0;
- hi = bb->count;
- /* Find the last range that starts before 'target' */
- while (hi - lo > 1) {
- int mid = (lo + hi) / 2;
- sector_t a = BB_OFFSET(p[mid]);
- if (a < target)
- lo = mid;
- else
- hi = mid;
- }
- if (hi > lo) {
- /* p[lo] is the last range that could overlap the
- * current range. Earlier ranges could also overlap,
- * but only this one can overlap the end of the range.
- */
- if (BB_OFFSET(p[lo]) + BB_LEN(p[lo]) > target) {
- /* Partial overlap, leave the tail of this range */
- int ack = BB_ACK(p[lo]);
- sector_t a = BB_OFFSET(p[lo]);
- sector_t end = a + BB_LEN(p[lo]);
-
- if (a < s) {
- /* we need to split this range */
- if (bb->count >= MD_MAX_BADBLOCKS) {
- rv = -ENOSPC;
- goto out;
- }
- memmove(p+lo+1, p+lo, (bb->count - lo) * 8);
- bb->count++;
- p[lo] = BB_MAKE(a, s-a, ack);
- lo++;
- }
- p[lo] = BB_MAKE(target, end - target, ack);
- /* there is no longer an overlap */
- hi = lo;
- lo--;
- }
- while (lo >= 0 &&
- BB_OFFSET(p[lo]) + BB_LEN(p[lo]) > s) {
- /* This range does overlap */
- if (BB_OFFSET(p[lo]) < s) {
- /* Keep the early parts of this range. */
- int ack = BB_ACK(p[lo]);
- sector_t start = BB_OFFSET(p[lo]);
- p[lo] = BB_MAKE(start, s - start, ack);
- /* now low doesn't overlap, so.. */
- break;
- }
- lo--;
- }
- /* 'lo' is strictly before, 'hi' is strictly after,
- * anything between needs to be discarded
- */
- if (hi - lo > 1) {
- memmove(p+lo+1, p+hi, (bb->count - hi) * 8);
- bb->count -= (hi - lo - 1);
- }
- }
-
- bb->changed = 1;
-out:
- write_sequnlock_irq(&bb->lock);
- return rv;
-}
-
int rdev_clear_badblocks(struct md_rdev *rdev, sector_t s, int sectors,
int is_new)
{
@@ -8857,133 +8513,11 @@ int rdev_clear_badblocks(struct md_rdev *rdev, sector_t s, int sectors,
s += rdev->new_data_offset;
else
s += rdev->data_offset;
- return md_clear_badblocks(&rdev->badblocks,
+ return badblocks_clear(&rdev->badblocks,
s, sectors);
}
EXPORT_SYMBOL_GPL(rdev_clear_badblocks);
-/*
- * Acknowledge all bad blocks in a list.
- * This only succeeds if ->changed is clear. It is used by
- * in-kernel metadata updates
- */
-void md_ack_all_badblocks(struct badblocks *bb)
-{
- if (bb->page == NULL || bb->changed)
- /* no point even trying */
- return;
- write_seqlock_irq(&bb->lock);
-
- if (bb->changed == 0 && bb->unacked_exist) {
- u64 *p = bb->page;
- int i;
- for (i = 0; i < bb->count ; i++) {
- if (!BB_ACK(p[i])) {
- sector_t start = BB_OFFSET(p[i]);
- int len = BB_LEN(p[i]);
- p[i] = BB_MAKE(start, len, 1);
- }
- }
- bb->unacked_exist = 0;
- }
- write_sequnlock_irq(&bb->lock);
-}
-EXPORT_SYMBOL_GPL(md_ack_all_badblocks);
-
-/* sysfs access to bad-blocks list.
- * We present two files.
- * 'bad-blocks' lists sector numbers and lengths of ranges that
- * are recorded as bad. The list is truncated to fit within
- * the one-page limit of sysfs.
- * Writing "sector length" to this file adds an acknowledged
- * bad block list.
- * 'unacknowledged-bad-blocks' lists bad blocks that have not yet
- * been acknowledged. Writing to this file adds bad blocks
- * without acknowledging them. This is largely for testing.
- */
-
-static ssize_t
-badblocks_show(struct badblocks *bb, char *page, int unack)
-{
- size_t len;
- int i;
- u64 *p = bb->page;
- unsigned seq;
-
- if (bb->shift < 0)
- return 0;
-
-retry:
- seq = read_seqbegin(&bb->lock);
-
- len = 0;
- i = 0;
-
- while (len < PAGE_SIZE && i < bb->count) {
- sector_t s = BB_OFFSET(p[i]);
- unsigned int length = BB_LEN(p[i]);
- int ack = BB_ACK(p[i]);
- i++;
-
- if (unack && ack)
- continue;
-
- len += snprintf(page+len, PAGE_SIZE-len, "%llu %u\n",
- (unsigned long long)s << bb->shift,
- length << bb->shift);
- }
- if (unack && len == 0)
- bb->unacked_exist = 0;
-
- if (read_seqretry(&bb->lock, seq))
- goto retry;
-
- return len;
-}
-
-#define DO_DEBUG 1
-
-static ssize_t
-badblocks_store(struct badblocks *bb, const char *page, size_t len, int unack)
-{
- unsigned long long sector;
- int length;
- char newline;
-#ifdef DO_DEBUG
- /* Allow clearing via sysfs *only* for testing/debugging.
- * Normally only a successful write may clear a badblock
- */
- int clear = 0;
- if (page[0] == '-') {
- clear = 1;
- page++;
- }
-#endif /* DO_DEBUG */
-
- switch (sscanf(page, "%llu %d%c", §or, &length, &newline)) {
- case 3:
- if (newline != '\n')
- return -EINVAL;
- case 2:
- if (length <= 0)
- return -EINVAL;
- break;
- default:
- return -EINVAL;
- }
-
-#ifdef DO_DEBUG
- if (clear) {
- md_clear_badblocks(bb, sector, length);
- return len;
- }
-#endif /* DO_DEBUG */
- if (md_set_badblocks(bb, sector, length, !unack))
- return len;
- else
- return -ENOSPC;
-}
-
static int md_notify_reboot(struct notifier_block *this,
unsigned long code, void *x)
{
diff --git a/drivers/md/md.h b/drivers/md/md.h
index ca0b643..75b9aaa 100644
--- a/drivers/md/md.h
+++ b/drivers/md/md.h
@@ -17,6 +17,7 @@
#include <linux/blkdev.h>
#include <linux/backing-dev.h>
+#include <linux/badblocks.h>
#include <linux/kobject.h>
#include <linux/list.h>
#include <linux/mm.h>
@@ -28,13 +29,6 @@
#define MaxSector (~(sector_t)0)
-/* Bad block numbers are stored sorted in a single page.
- * 64bits is used for each block or extent.
- * 54 bits are sector number, 9 bits are extent size,
- * 1 bit is an 'acknowledged' flag.
- */
-#define MD_MAX_BADBLOCKS (PAGE_SIZE/8)
-
/*
* MD's 'extended' device
*/
@@ -117,22 +111,7 @@ struct md_rdev {
struct kernfs_node *sysfs_state; /* handle for 'state'
* sysfs entry */
- struct badblocks {
- int count; /* count of bad blocks */
- int unacked_exist; /* there probably are unacknowledged
- * bad blocks. This is only cleared
- * when a read discovers none
- */
- int shift; /* shift from sectors to block size
- * a -ve shift means badblocks are
- * disabled.*/
- u64 *page; /* badblock list */
- int changed;
- seqlock_t lock;
-
- sector_t sector;
- sector_t size; /* in sectors */
- } badblocks;
+ struct badblocks badblocks;
};
enum flag_bits {
Faulty, /* device is known to have a fault */
@@ -185,22 +164,11 @@ enum flag_bits {
*/
};
-#define BB_LEN_MASK (0x00000000000001FFULL)
-#define BB_OFFSET_MASK (0x7FFFFFFFFFFFFE00ULL)
-#define BB_ACK_MASK (0x8000000000000000ULL)
-#define BB_MAX_LEN 512
-#define BB_OFFSET(x) (((x) & BB_OFFSET_MASK) >> 9)
-#define BB_LEN(x) (((x) & BB_LEN_MASK) + 1)
-#define BB_ACK(x) (!!((x) & BB_ACK_MASK))
-#define BB_MAKE(a, l, ack) (((a)<<9) | ((l)-1) | ((u64)(!!(ack)) << 63))
-
-extern int md_is_badblock(struct badblocks *bb, sector_t s, int sectors,
- sector_t *first_bad, int *bad_sectors);
static inline int is_badblock(struct md_rdev *rdev, sector_t s, int sectors,
sector_t *first_bad, int *bad_sectors)
{
if (unlikely(rdev->badblocks.count)) {
- int rv = md_is_badblock(&rdev->badblocks, rdev->data_offset + s,
+ int rv = badblocks_check(&rdev->badblocks, rdev->data_offset + s,
sectors,
first_bad, bad_sectors);
if (rv)
@@ -213,8 +181,6 @@ extern int rdev_set_badblocks(struct md_rdev *rdev, sector_t s, int sectors,
int is_new);
extern int rdev_clear_badblocks(struct md_rdev *rdev, sector_t s, int sectors,
int is_new);
-extern void md_ack_all_badblocks(struct badblocks *bb);
-
struct md_cluster_info;
struct mddev {
--
2.5.0
^ permalink raw reply related
* [PATCH v5 2/3] block: Add badblock management for gendisks
From: Vishal Verma @ 2015-12-25 2:20 UTC (permalink / raw)
To: linux-nvdimm
Cc: Vishal Verma, linux-block, linux-raid, linux-scsi, Jens Axboe,
NeilBrown, Jeff Moyer, Dan Williams
In-Reply-To: <1451010034-11236-1-git-send-email-vishal.l.verma@intel.com>
NVDIMM devices, which can behave more like DRAM rather than block
devices, may develop bad cache lines, or 'poison'. A block device
exposed by the pmem driver can then consume poison via a read (or
write), and cause a machine check. On platforms without machine
check recovery features, this would mean a crash.
The block device maintaining a runtime list of all known sectors that
have poison can directly avoid this, and also provide a path forward
to enable proper handling/recovery for DAX faults on such a device.
Use the new badblock management interfaces to add a badblocks list to
gendisks.
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
block/genhd.c | 76 +++++++++++++++++++++++++++++++++++++++++++++++++++
include/linux/genhd.h | 7 +++++
2 files changed, 83 insertions(+)
diff --git a/block/genhd.c b/block/genhd.c
index e5cafa5..81dcf32 100644
--- a/block/genhd.c
+++ b/block/genhd.c
@@ -20,6 +20,7 @@
#include <linux/idr.h>
#include <linux/log2.h>
#include <linux/pm_runtime.h>
+#include <linux/badblocks.h>
#include "blk.h"
@@ -505,6 +506,16 @@ static int exact_lock(dev_t devt, void *data)
return 0;
}
+int disk_alloc_badblocks(struct gendisk *disk)
+{
+ disk->bb = kzalloc(sizeof(*(disk->bb)), GFP_KERNEL);
+ if (!disk->bb)
+ return -ENOMEM;
+
+ return badblocks_init(disk->bb, 1);
+}
+EXPORT_SYMBOL(disk_alloc_badblocks);
+
static void register_disk(struct gendisk *disk)
{
struct device *ddev = disk_to_dev(disk);
@@ -659,6 +670,11 @@ void del_gendisk(struct gendisk *disk)
blk_unregister_queue(disk);
blk_unregister_region(disk_devt(disk), disk->minors);
+ if (disk->bb) {
+ badblocks_free(disk->bb);
+ kfree(disk->bb);
+ }
+
part_stat_set_all(&disk->part0, 0);
disk->part0.stamp = 0;
@@ -672,6 +688,63 @@ void del_gendisk(struct gendisk *disk)
}
EXPORT_SYMBOL(del_gendisk);
+/*
+ * The gendisk usage of badblocks does not track acknowledgements for
+ * badblocks. We always assume they are acknowledged.
+ */
+int disk_check_badblocks(struct gendisk *disk, sector_t s, int sectors,
+ sector_t *first_bad, int *bad_sectors)
+{
+ if (!disk->bb)
+ return 0;
+
+ return badblocks_check(disk->bb, s, sectors, first_bad, bad_sectors);
+}
+EXPORT_SYMBOL(disk_check_badblocks);
+
+int disk_set_badblocks(struct gendisk *disk, sector_t s, int sectors)
+{
+ if (!disk->bb)
+ return 0;
+
+ return badblocks_set(disk->bb, s, sectors, 1);
+}
+EXPORT_SYMBOL(disk_set_badblocks);
+
+int disk_clear_badblocks(struct gendisk *disk, sector_t s, int sectors)
+{
+ if (!disk->bb)
+ return 0;
+
+ return badblocks_clear(disk->bb, s, sectors);
+}
+EXPORT_SYMBOL(disk_clear_badblocks);
+
+/* sysfs access to bad-blocks list. */
+static ssize_t disk_badblocks_show(struct device *dev,
+ struct device_attribute *attr,
+ char *page)
+{
+ struct gendisk *disk = dev_to_disk(dev);
+
+ if (!disk->bb)
+ return 0;
+
+ return badblocks_show(disk->bb, page, 0);
+}
+
+static ssize_t disk_badblocks_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *page, size_t len)
+{
+ struct gendisk *disk = dev_to_disk(dev);
+
+ if (!disk->bb)
+ return 0;
+
+ return badblocks_store(disk->bb, page, len, 0);
+}
+
/**
* get_gendisk - get partitioning information for a given device
* @devt: device to get partitioning information for
@@ -990,6 +1063,8 @@ static DEVICE_ATTR(discard_alignment, S_IRUGO, disk_discard_alignment_show,
static DEVICE_ATTR(capability, S_IRUGO, disk_capability_show, NULL);
static DEVICE_ATTR(stat, S_IRUGO, part_stat_show, NULL);
static DEVICE_ATTR(inflight, S_IRUGO, part_inflight_show, NULL);
+static DEVICE_ATTR(badblocks, S_IRUGO | S_IWUSR, disk_badblocks_show,
+ disk_badblocks_store);
#ifdef CONFIG_FAIL_MAKE_REQUEST
static struct device_attribute dev_attr_fail =
__ATTR(make-it-fail, S_IRUGO|S_IWUSR, part_fail_show, part_fail_store);
@@ -1011,6 +1086,7 @@ static struct attribute *disk_attrs[] = {
&dev_attr_capability.attr,
&dev_attr_stat.attr,
&dev_attr_inflight.attr,
+ &dev_attr_badblocks.attr,
#ifdef CONFIG_FAIL_MAKE_REQUEST
&dev_attr_fail.attr,
#endif
diff --git a/include/linux/genhd.h b/include/linux/genhd.h
index 847cc1d..0bbec68 100644
--- a/include/linux/genhd.h
+++ b/include/linux/genhd.h
@@ -162,6 +162,7 @@ struct disk_part_tbl {
};
struct disk_events;
+struct badblocks;
#if defined(CONFIG_BLK_DEV_INTEGRITY)
@@ -213,6 +214,7 @@ struct gendisk {
struct kobject integrity_kobj;
#endif /* CONFIG_BLK_DEV_INTEGRITY */
int node_id;
+ struct badblocks *bb;
};
static inline struct gendisk *part_to_disk(struct hd_struct *part)
@@ -433,6 +435,11 @@ extern void add_disk(struct gendisk *disk);
extern void del_gendisk(struct gendisk *gp);
extern struct gendisk *get_gendisk(dev_t dev, int *partno);
extern struct block_device *bdget_disk(struct gendisk *disk, int partno);
+int disk_alloc_badblocks(struct gendisk *disk);
+extern int disk_check_badblocks(struct gendisk *disk, sector_t s, int sectors,
+ sector_t *first_bad, int *bad_sectors);
+extern int disk_set_badblocks(struct gendisk *disk, sector_t s, int sectors);
+extern int disk_clear_badblocks(struct gendisk *disk, sector_t s, int sectors);
extern void set_device_ro(struct block_device *bdev, int flag);
extern void set_disk_ro(struct gendisk *disk, int flag);
--
2.5.0
^ permalink raw reply related
* [PATCH v5 1/3] badblocks: Add core badblock management code
From: Vishal Verma @ 2015-12-25 2:20 UTC (permalink / raw)
To: linux-nvdimm
Cc: Vishal Verma, linux-block, linux-raid, linux-scsi, Jens Axboe,
NeilBrown, Jeff Moyer, Dan Williams
In-Reply-To: <1451010034-11236-1-git-send-email-vishal.l.verma@intel.com>
Take the core badblocks implementation from md, and make it generally
available. This follows the same style as kernel implementations of
linked lists, rb-trees etc, where you can have a structure that can be
embedded anywhere, and accessor functions to manipulate the data.
The only changes in this copy of the code are ones to generalize
function/variable names from md-specific ones. Also add init and free
functions.
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
block/Makefile | 2 +-
block/badblocks.c | 561 ++++++++++++++++++++++++++++++++++++++++++++++
include/linux/badblocks.h | 53 +++++
3 files changed, 615 insertions(+), 1 deletion(-)
create mode 100644 block/badblocks.c
create mode 100644 include/linux/badblocks.h
diff --git a/block/Makefile b/block/Makefile
index 00ecc97..db5f622 100644
--- a/block/Makefile
+++ b/block/Makefile
@@ -8,7 +8,7 @@ obj-$(CONFIG_BLOCK) := bio.o elevator.o blk-core.o blk-tag.o blk-sysfs.o \
blk-iopoll.o blk-lib.o blk-mq.o blk-mq-tag.o \
blk-mq-sysfs.o blk-mq-cpu.o blk-mq-cpumap.o ioctl.o \
genhd.o scsi_ioctl.o partition-generic.o ioprio.o \
- partitions/
+ badblocks.o partitions/
obj-$(CONFIG_BOUNCE) += bounce.o
obj-$(CONFIG_BLK_DEV_BSG) += bsg.o
diff --git a/block/badblocks.c b/block/badblocks.c
new file mode 100644
index 0000000..96aeb91
--- /dev/null
+++ b/block/badblocks.c
@@ -0,0 +1,561 @@
+/*
+ * Bad block management
+ *
+ * - Heavily based on MD badblocks code from Neil Brown
+ *
+ * Copyright (c) 2015, Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ */
+
+#include <linux/badblocks.h>
+#include <linux/seqlock.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/stddef.h>
+#include <linux/types.h>
+#include <linux/slab.h>
+
+/**
+ * badblocks_check() - check a given range for bad sectors
+ * @bb: the badblocks structure that holds all badblock information
+ * @s: sector (start) at which to check for badblocks
+ * @sectors: number of sectors to check for badblocks
+ * @first_bad: pointer to store location of the first badblock
+ * @bad_sectors: pointer to store number of badblocks after @first_bad
+ *
+ * We can record which blocks on each device are 'bad' and so just
+ * fail those blocks, or that stripe, rather than the whole device.
+ * Entries in the bad-block table are 64bits wide. This comprises:
+ * Length of bad-range, in sectors: 0-511 for lengths 1-512
+ * Start of bad-range, sector offset, 54 bits (allows 8 exbibytes)
+ * A 'shift' can be set so that larger blocks are tracked and
+ * consequently larger devices can be covered.
+ * 'Acknowledged' flag - 1 bit. - the most significant bit.
+ *
+ * Locking of the bad-block table uses a seqlock so badblocks_check
+ * might need to retry if it is very unlucky.
+ * We will sometimes want to check for bad blocks in a bi_end_io function,
+ * so we use the write_seqlock_irq variant.
+ *
+ * When looking for a bad block we specify a range and want to
+ * know if any block in the range is bad. So we binary-search
+ * to the last range that starts at-or-before the given endpoint,
+ * (or "before the sector after the target range")
+ * then see if it ends after the given start.
+ *
+ * Return:
+ * 0: there are no known bad blocks in the range
+ * 1: there are known bad block which are all acknowledged
+ * -1: there are bad blocks which have not yet been acknowledged in metadata.
+ * plus the start/length of the first bad section we overlap.
+ */
+int badblocks_check(struct badblocks *bb, sector_t s, int sectors,
+ sector_t *first_bad, int *bad_sectors)
+{
+ int hi;
+ int lo;
+ u64 *p = bb->page;
+ int rv;
+ sector_t target = s + sectors;
+ unsigned seq;
+
+ if (bb->shift > 0) {
+ /* round the start down, and the end up */
+ s >>= bb->shift;
+ target += (1<<bb->shift) - 1;
+ target >>= bb->shift;
+ sectors = target - s;
+ }
+ /* 'target' is now the first block after the bad range */
+
+retry:
+ seq = read_seqbegin(&bb->lock);
+ lo = 0;
+ rv = 0;
+ hi = bb->count;
+
+ /* Binary search between lo and hi for 'target'
+ * i.e. for the last range that starts before 'target'
+ */
+ /* INVARIANT: ranges before 'lo' and at-or-after 'hi'
+ * are known not to be the last range before target.
+ * VARIANT: hi-lo is the number of possible
+ * ranges, and decreases until it reaches 1
+ */
+ while (hi - lo > 1) {
+ int mid = (lo + hi) / 2;
+ sector_t a = BB_OFFSET(p[mid]);
+
+ if (a < target)
+ /* This could still be the one, earlier ranges
+ * could not.
+ */
+ lo = mid;
+ else
+ /* This and later ranges are definitely out. */
+ hi = mid;
+ }
+ /* 'lo' might be the last that started before target, but 'hi' isn't */
+ if (hi > lo) {
+ /* need to check all range that end after 's' to see if
+ * any are unacknowledged.
+ */
+ while (lo >= 0 &&
+ BB_OFFSET(p[lo]) + BB_LEN(p[lo]) > s) {
+ if (BB_OFFSET(p[lo]) < target) {
+ /* starts before the end, and finishes after
+ * the start, so they must overlap
+ */
+ if (rv != -1 && BB_ACK(p[lo]))
+ rv = 1;
+ else
+ rv = -1;
+ *first_bad = BB_OFFSET(p[lo]);
+ *bad_sectors = BB_LEN(p[lo]);
+ }
+ lo--;
+ }
+ }
+
+ if (read_seqretry(&bb->lock, seq))
+ goto retry;
+
+ return rv;
+}
+EXPORT_SYMBOL_GPL(badblocks_check);
+
+/**
+ * badblocks_set() - Add a range of bad blocks to the table.
+ * @bb: the badblocks structure that holds all badblock information
+ * @s: first sector to mark as bad
+ * @sectors: number of sectors to mark as bad
+ * @acknowledged: weather to mark the bad sectors as acknowledged
+ *
+ * This might extend the table, or might contract it if two adjacent ranges
+ * can be merged. We binary-search to find the 'insertion' point, then
+ * decide how best to handle it.
+ *
+ * Return:
+ * 0: success
+ * 1: failed to set badblocks (out of space)
+ */
+int badblocks_set(struct badblocks *bb, sector_t s, int sectors,
+ int acknowledged)
+{
+ u64 *p;
+ int lo, hi;
+ int rv = 0;
+ unsigned long flags;
+
+ if (bb->shift < 0)
+ /* badblocks are disabled */
+ return 0;
+
+ if (bb->shift) {
+ /* round the start down, and the end up */
+ sector_t next = s + sectors;
+
+ s >>= bb->shift;
+ next += (1<<bb->shift) - 1;
+ next >>= bb->shift;
+ sectors = next - s;
+ }
+
+ write_seqlock_irqsave(&bb->lock, flags);
+
+ p = bb->page;
+ lo = 0;
+ hi = bb->count;
+ /* Find the last range that starts at-or-before 's' */
+ while (hi - lo > 1) {
+ int mid = (lo + hi) / 2;
+ sector_t a = BB_OFFSET(p[mid]);
+
+ if (a <= s)
+ lo = mid;
+ else
+ hi = mid;
+ }
+ if (hi > lo && BB_OFFSET(p[lo]) > s)
+ hi = lo;
+
+ if (hi > lo) {
+ /* we found a range that might merge with the start
+ * of our new range
+ */
+ sector_t a = BB_OFFSET(p[lo]);
+ sector_t e = a + BB_LEN(p[lo]);
+ int ack = BB_ACK(p[lo]);
+
+ if (e >= s) {
+ /* Yes, we can merge with a previous range */
+ if (s == a && s + sectors >= e)
+ /* new range covers old */
+ ack = acknowledged;
+ else
+ ack = ack && acknowledged;
+
+ if (e < s + sectors)
+ e = s + sectors;
+ if (e - a <= BB_MAX_LEN) {
+ p[lo] = BB_MAKE(a, e-a, ack);
+ s = e;
+ } else {
+ /* does not all fit in one range,
+ * make p[lo] maximal
+ */
+ if (BB_LEN(p[lo]) != BB_MAX_LEN)
+ p[lo] = BB_MAKE(a, BB_MAX_LEN, ack);
+ s = a + BB_MAX_LEN;
+ }
+ sectors = e - s;
+ }
+ }
+ if (sectors && hi < bb->count) {
+ /* 'hi' points to the first range that starts after 's'.
+ * Maybe we can merge with the start of that range
+ */
+ sector_t a = BB_OFFSET(p[hi]);
+ sector_t e = a + BB_LEN(p[hi]);
+ int ack = BB_ACK(p[hi]);
+
+ if (a <= s + sectors) {
+ /* merging is possible */
+ if (e <= s + sectors) {
+ /* full overlap */
+ e = s + sectors;
+ ack = acknowledged;
+ } else
+ ack = ack && acknowledged;
+
+ a = s;
+ if (e - a <= BB_MAX_LEN) {
+ p[hi] = BB_MAKE(a, e-a, ack);
+ s = e;
+ } else {
+ p[hi] = BB_MAKE(a, BB_MAX_LEN, ack);
+ s = a + BB_MAX_LEN;
+ }
+ sectors = e - s;
+ lo = hi;
+ hi++;
+ }
+ }
+ if (sectors == 0 && hi < bb->count) {
+ /* we might be able to combine lo and hi */
+ /* Note: 's' is at the end of 'lo' */
+ sector_t a = BB_OFFSET(p[hi]);
+ int lolen = BB_LEN(p[lo]);
+ int hilen = BB_LEN(p[hi]);
+ int newlen = lolen + hilen - (s - a);
+
+ if (s >= a && newlen < BB_MAX_LEN) {
+ /* yes, we can combine them */
+ int ack = BB_ACK(p[lo]) && BB_ACK(p[hi]);
+
+ p[lo] = BB_MAKE(BB_OFFSET(p[lo]), newlen, ack);
+ memmove(p + hi, p + hi + 1,
+ (bb->count - hi - 1) * 8);
+ bb->count--;
+ }
+ }
+ while (sectors) {
+ /* didn't merge (it all).
+ * Need to add a range just before 'hi'
+ */
+ if (bb->count >= MAX_BADBLOCKS) {
+ /* No room for more */
+ rv = 1;
+ break;
+ } else {
+ int this_sectors = sectors;
+
+ memmove(p + hi + 1, p + hi,
+ (bb->count - hi) * 8);
+ bb->count++;
+
+ if (this_sectors > BB_MAX_LEN)
+ this_sectors = BB_MAX_LEN;
+ p[hi] = BB_MAKE(s, this_sectors, acknowledged);
+ sectors -= this_sectors;
+ s += this_sectors;
+ }
+ }
+
+ bb->changed = 1;
+ if (!acknowledged)
+ bb->unacked_exist = 1;
+ write_sequnlock_irqrestore(&bb->lock, flags);
+
+ return rv;
+}
+EXPORT_SYMBOL_GPL(badblocks_set);
+
+/**
+ * badblocks_clear() - Remove a range of bad blocks to the table.
+ * @bb: the badblocks structure that holds all badblock information
+ * @s: first sector to mark as bad
+ * @sectors: number of sectors to mark as bad
+ *
+ * This may involve extending the table if we spilt a region,
+ * but it must not fail. So if the table becomes full, we just
+ * drop the remove request.
+ *
+ * Return:
+ * 0: success
+ * 1: failed to clear badblocks
+ */
+int badblocks_clear(struct badblocks *bb, sector_t s, int sectors)
+{
+ u64 *p;
+ int lo, hi;
+ sector_t target = s + sectors;
+ int rv = 0;
+
+ if (bb->shift > 0) {
+ /* When clearing we round the start up and the end down.
+ * This should not matter as the shift should align with
+ * the block size and no rounding should ever be needed.
+ * However it is better the think a block is bad when it
+ * isn't than to think a block is not bad when it is.
+ */
+ s += (1<<bb->shift) - 1;
+ s >>= bb->shift;
+ target >>= bb->shift;
+ sectors = target - s;
+ }
+
+ write_seqlock_irq(&bb->lock);
+
+ p = bb->page;
+ lo = 0;
+ hi = bb->count;
+ /* Find the last range that starts before 'target' */
+ while (hi - lo > 1) {
+ int mid = (lo + hi) / 2;
+ sector_t a = BB_OFFSET(p[mid]);
+
+ if (a < target)
+ lo = mid;
+ else
+ hi = mid;
+ }
+ if (hi > lo) {
+ /* p[lo] is the last range that could overlap the
+ * current range. Earlier ranges could also overlap,
+ * but only this one can overlap the end of the range.
+ */
+ if (BB_OFFSET(p[lo]) + BB_LEN(p[lo]) > target) {
+ /* Partial overlap, leave the tail of this range */
+ int ack = BB_ACK(p[lo]);
+ sector_t a = BB_OFFSET(p[lo]);
+ sector_t end = a + BB_LEN(p[lo]);
+
+ if (a < s) {
+ /* we need to split this range */
+ if (bb->count >= MAX_BADBLOCKS) {
+ rv = -ENOSPC;
+ goto out;
+ }
+ memmove(p+lo+1, p+lo, (bb->count - lo) * 8);
+ bb->count++;
+ p[lo] = BB_MAKE(a, s-a, ack);
+ lo++;
+ }
+ p[lo] = BB_MAKE(target, end - target, ack);
+ /* there is no longer an overlap */
+ hi = lo;
+ lo--;
+ }
+ while (lo >= 0 &&
+ BB_OFFSET(p[lo]) + BB_LEN(p[lo]) > s) {
+ /* This range does overlap */
+ if (BB_OFFSET(p[lo]) < s) {
+ /* Keep the early parts of this range. */
+ int ack = BB_ACK(p[lo]);
+ sector_t start = BB_OFFSET(p[lo]);
+
+ p[lo] = BB_MAKE(start, s - start, ack);
+ /* now low doesn't overlap, so.. */
+ break;
+ }
+ lo--;
+ }
+ /* 'lo' is strictly before, 'hi' is strictly after,
+ * anything between needs to be discarded
+ */
+ if (hi - lo > 1) {
+ memmove(p+lo+1, p+hi, (bb->count - hi) * 8);
+ bb->count -= (hi - lo - 1);
+ }
+ }
+
+ bb->changed = 1;
+out:
+ write_sequnlock_irq(&bb->lock);
+ return rv;
+}
+EXPORT_SYMBOL_GPL(badblocks_clear);
+
+/**
+ * ack_all_badblocks() - Acknowledge all bad blocks in a list.
+ * @bb: the badblocks structure that holds all badblock information
+ *
+ * This only succeeds if ->changed is clear. It is used by
+ * in-kernel metadata updates
+ */
+void ack_all_badblocks(struct badblocks *bb)
+{
+ if (bb->page == NULL || bb->changed)
+ /* no point even trying */
+ return;
+ write_seqlock_irq(&bb->lock);
+
+ if (bb->changed == 0 && bb->unacked_exist) {
+ u64 *p = bb->page;
+ int i;
+
+ for (i = 0; i < bb->count ; i++) {
+ if (!BB_ACK(p[i])) {
+ sector_t start = BB_OFFSET(p[i]);
+ int len = BB_LEN(p[i]);
+
+ p[i] = BB_MAKE(start, len, 1);
+ }
+ }
+ bb->unacked_exist = 0;
+ }
+ write_sequnlock_irq(&bb->lock);
+}
+EXPORT_SYMBOL_GPL(ack_all_badblocks);
+
+/**
+ * badblocks_show() - sysfs access to bad-blocks list
+ * @bb: the badblocks structure that holds all badblock information
+ * @page: buffer received from sysfs
+ * @unack: weather to show unacknowledged badblocks
+ *
+ * Return:
+ * Length of returned data
+ */
+ssize_t badblocks_show(struct badblocks *bb, char *page, int unack)
+{
+ size_t len;
+ int i;
+ u64 *p = bb->page;
+ unsigned seq;
+
+ if (bb->shift < 0)
+ return 0;
+
+retry:
+ seq = read_seqbegin(&bb->lock);
+
+ len = 0;
+ i = 0;
+
+ while (len < PAGE_SIZE && i < bb->count) {
+ sector_t s = BB_OFFSET(p[i]);
+ unsigned int length = BB_LEN(p[i]);
+ int ack = BB_ACK(p[i]);
+
+ i++;
+
+ if (unack && ack)
+ continue;
+
+ len += snprintf(page+len, PAGE_SIZE-len, "%llu %u\n",
+ (unsigned long long)s << bb->shift,
+ length << bb->shift);
+ }
+ if (unack && len == 0)
+ bb->unacked_exist = 0;
+
+ if (read_seqretry(&bb->lock, seq))
+ goto retry;
+
+ return len;
+}
+EXPORT_SYMBOL_GPL(badblocks_show);
+
+/**
+ * badblocks_store() - sysfs access to bad-blocks list
+ * @bb: the badblocks structure that holds all badblock information
+ * @page: buffer received from sysfs
+ * @len: length of data received from sysfs
+ * @unack: weather to show unacknowledged badblocks
+ *
+ * Return:
+ * Length of the buffer processed or -ve error.
+ */
+ssize_t badblocks_store(struct badblocks *bb, const char *page, size_t len,
+ int unack)
+{
+ unsigned long long sector;
+ int length;
+ char newline;
+
+ switch (sscanf(page, "%llu %d%c", §or, &length, &newline)) {
+ case 3:
+ if (newline != '\n')
+ return -EINVAL;
+ case 2:
+ if (length <= 0)
+ return -EINVAL;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ if (badblocks_set(bb, sector, length, !unack))
+ return -ENOSPC;
+ else
+ return len;
+}
+EXPORT_SYMBOL_GPL(badblocks_store);
+
+/**
+ * badblocks_init() - initialize the badblocks structure
+ * @bb: the badblocks structure that holds all badblock information
+ * @enable: weather to enable badblocks accounting
+ *
+ * Return:
+ * 0: success
+ * -ve errno: on error
+ */
+int badblocks_init(struct badblocks *bb, int enable)
+{
+ bb->count = 0;
+ if (enable)
+ bb->shift = 0;
+ else
+ bb->shift = -1;
+ bb->page = kzalloc(PAGE_SIZE, GFP_KERNEL);
+ if (bb->page == (u64 *)0) {
+ bb->shift = -1;
+ return -ENOMEM;
+ }
+ seqlock_init(&bb->lock);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(badblocks_init);
+
+/**
+ * badblocks_free() - free the badblocks structure
+ * @bb: the badblocks structure that holds all badblock information
+ */
+void badblocks_free(struct badblocks *bb)
+{
+ kfree(bb->page);
+ bb->page = NULL;
+}
+EXPORT_SYMBOL_GPL(badblocks_free);
diff --git a/include/linux/badblocks.h b/include/linux/badblocks.h
new file mode 100644
index 0000000..9293446
--- /dev/null
+++ b/include/linux/badblocks.h
@@ -0,0 +1,53 @@
+#ifndef _LINUX_BADBLOCKS_H
+#define _LINUX_BADBLOCKS_H
+
+#include <linux/seqlock.h>
+#include <linux/kernel.h>
+#include <linux/stddef.h>
+#include <linux/types.h>
+
+#define BB_LEN_MASK (0x00000000000001FFULL)
+#define BB_OFFSET_MASK (0x7FFFFFFFFFFFFE00ULL)
+#define BB_ACK_MASK (0x8000000000000000ULL)
+#define BB_MAX_LEN 512
+#define BB_OFFSET(x) (((x) & BB_OFFSET_MASK) >> 9)
+#define BB_LEN(x) (((x) & BB_LEN_MASK) + 1)
+#define BB_ACK(x) (!!((x) & BB_ACK_MASK))
+#define BB_MAKE(a, l, ack) (((a)<<9) | ((l)-1) | ((u64)(!!(ack)) << 63))
+
+/* Bad block numbers are stored sorted in a single page.
+ * 64bits is used for each block or extent.
+ * 54 bits are sector number, 9 bits are extent size,
+ * 1 bit is an 'acknowledged' flag.
+ */
+#define MAX_BADBLOCKS (PAGE_SIZE/8)
+
+struct badblocks {
+ int count; /* count of bad blocks */
+ int unacked_exist; /* there probably are unacknowledged
+ * bad blocks. This is only cleared
+ * when a read discovers none
+ */
+ int shift; /* shift from sectors to block size
+ * a -ve shift means badblocks are
+ * disabled.*/
+ u64 *page; /* badblock list */
+ int changed;
+ seqlock_t lock;
+ sector_t sector;
+ sector_t size; /* in sectors */
+};
+
+int badblocks_check(struct badblocks *bb, sector_t s, int sectors,
+ sector_t *first_bad, int *bad_sectors);
+int badblocks_set(struct badblocks *bb, sector_t s, int sectors,
+ int acknowledged);
+int badblocks_clear(struct badblocks *bb, sector_t s, int sectors);
+void ack_all_badblocks(struct badblocks *bb);
+ssize_t badblocks_show(struct badblocks *bb, char *page, int unack);
+ssize_t badblocks_store(struct badblocks *bb, const char *page, size_t len,
+ int unack);
+int badblocks_init(struct badblocks *bb, int enable);
+void badblocks_free(struct badblocks *bb);
+
+#endif
--
2.5.0
^ permalink raw reply related
* [PATCH v5 0/3] Badblock tracking for gendisks
From: Vishal Verma @ 2015-12-25 2:20 UTC (permalink / raw)
To: linux-nvdimm
Cc: Vishal Verma, linux-block, linux-raid, linux-scsi, Jens Axboe,
NeilBrown, Jeff Moyer, Dan Williams
v5:
- Rebase to v4.4-rc6
- Revert back to using kzalloc from __get_free_page based on the discussion at:
http://thread.gmane.org/gmane.linux.kernel/2113292
v4:
- Rebase to v4.4-rc4
v3:
- Add kernel-doc style comments to all exported functions in badblocks.c (James)
- Make return values from badblocks functions consistent with themselves
and the kernel style. Change the polarity of badblocks_set, and update
all callers accordingly (James)
- In gendisk, don't unconditionally allocate badblocks, export the initializer.
This also allows the initializer to be a non-void return type, so that the
badblocks user can act upon failures better (James)
v2:
- In badblocks_free, make 'page' NULL (patch 1)
- Move the core badblocks code to a new .c file (patch 1) (Jens)
- Fix a sizeof usage in disk_alloc_badblocks (patch 2) (Dan)
- Since disk_alloc_badblocks can fail, check disk->bb for NULL in the
genhd wrappers (patch 2) (Jeff)
- Update the md conversion to also ise the badblocks init and free
functions (patch 3)
- Remove the BB_* macros from md.h as they are now in badblocks.h (patch 3)
Patch 1 copies badblock management code into a header of its own,
making it generally available. It follows common libraries of code
such as linked lists, where anyone may embed a core data structure
in another place, and use the provided accessor functions to
manipulate the data.
Patch 2 adds badblock tracking to gendisks (in preparation for use
by NVDIMM devices).
Patch 3 converts md over to use the new badblocks 'library'. I have
done some pretty simple testing on this - created a raid 1 device,
made sure the sysfs entries show up, and can be used to add and view
badblocks. A closer look by the md folks would be nice here.
Vishal Verma (3):
badblocks: Add core badblock management code
block: Add badblock management for gendisks
md: convert to use the generic badblocks code
block/Makefile | 2 +-
block/badblocks.c | 561 ++++++++++++++++++++++++++++++++++++++++++++++
block/genhd.c | 76 +++++++
drivers/md/md.c | 516 +++---------------------------------------
drivers/md/md.h | 40 +---
include/linux/badblocks.h | 53 +++++
include/linux/genhd.h | 7 +
7 files changed, 726 insertions(+), 529 deletions(-)
create mode 100644 block/badblocks.c
create mode 100644 include/linux/badblocks.h
--
2.5.0
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox