* Re: [PATCH/RFC/RFT] md: allow resync to go faster when there is competing IO.
From: Shaohua Li @ 2016-01-26 22:52 UTC (permalink / raw)
To: NeilBrown; +Cc: Chien Lee, linux-raid, owner-linux-raid
In-Reply-To: <87si1k2do8.fsf@notabene.neil.brown.name>
On Wed, Jan 27, 2016 at 09:12:23AM +1100, Neil Brown wrote:
> On Tue, Jan 26 2016, Chien Lee wrote:
>
> > Hello,
> >
> > Recently we find a bug about this patch (commit No. is
> > ac8fa4196d205ac8fff3f8932bddbad4f16e4110 ).
> >
> > We know that this patch committed after Linux kernel 4.1.x is intended
> > to allowing resync to go faster when there is competing IO. However,
> > we find the performance of random read on syncing Raid6 will come up
> > with a huge drop in this case. The following is our testing detail.
> >
> > The OS what we choose in our test is CentOS Linux release 7.1.1503
> > (Core) and the kernel image will be replaced for testing. In our
> > testing result, the 4K random read performance on syncing raid6 in
> > Kernel 4.2.8 is much lower than in Kernel 3.19.8. In order to find out
> > the root cause, we try to rollback this patch in Kernel 4.2.8, and we
> > find the 4K random read performance on syncing Raid6 will be improved
> > and go back to as what it should be in Kernel 3.19.8.
> >
> > Nevertheless, it seems that it will not affect some other read/write
> > patterns. In our testing result, the 1M sequential read/write, 4K
> > random write performance in Kernel 4.2.8 is performed almost the same
> > as in Kernel 3.19.8.
> >
> > It seems that although this patch increases the resync speed, the
> > logic of !is_mddev_idle() cause the sync request wait too short and
> > reduce the chance for raid5d to handle the random read I/O.
>
> This has been raised before.
> Can you please try the patch at the end of
>
> http://permalink.gmane.org/gmane.linux.raid/51002
>
> and let me know if it makes any difference. If it isn't sufficient I
> will explore further.
I'm curious why we don't calculate the wait time. Say the target resync speed
is speed_min. The wait time should be:
(currspeed * SYNC_MARK_STEP - speed_min * SYNC_MARK_STEP) / speed_min
= (currspeed / speed_min - 1) * SYNC_MARK_STEP
if SYNC_MARK_STEP is too big and sync speed has drift, we can make it smaller.
Thanks,
Shaohua
^ permalink raw reply
* Re: [PATCH/RFC/RFT] md: allow resync to go faster when there is competing IO.
From: NeilBrown @ 2016-01-26 22:12 UTC (permalink / raw)
To: Chien Lee, linux-raid, shli, owner-linux-raid
In-Reply-To: <CAByoP04f6YgzhXn+55iEpg0SrVnK3mUxOuEfp-Hki_mppOBieg@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 3998 bytes --]
On Tue, Jan 26 2016, Chien Lee wrote:
> Hello,
>
> Recently we find a bug about this patch (commit No. is
> ac8fa4196d205ac8fff3f8932bddbad4f16e4110 ).
>
> We know that this patch committed after Linux kernel 4.1.x is intended
> to allowing resync to go faster when there is competing IO. However,
> we find the performance of random read on syncing Raid6 will come up
> with a huge drop in this case. The following is our testing detail.
>
> The OS what we choose in our test is CentOS Linux release 7.1.1503
> (Core) and the kernel image will be replaced for testing. In our
> testing result, the 4K random read performance on syncing raid6 in
> Kernel 4.2.8 is much lower than in Kernel 3.19.8. In order to find out
> the root cause, we try to rollback this patch in Kernel 4.2.8, and we
> find the 4K random read performance on syncing Raid6 will be improved
> and go back to as what it should be in Kernel 3.19.8.
>
> Nevertheless, it seems that it will not affect some other read/write
> patterns. In our testing result, the 1M sequential read/write, 4K
> random write performance in Kernel 4.2.8 is performed almost the same
> as in Kernel 3.19.8.
>
> It seems that although this patch increases the resync speed, the
> logic of !is_mddev_idle() cause the sync request wait too short and
> reduce the chance for raid5d to handle the random read I/O.
This has been raised before.
Can you please try the patch at the end of
http://permalink.gmane.org/gmane.linux.raid/51002
and let me know if it makes any difference. If it isn't sufficient I
will explore further.
Thanks,
NeilBrown
>
>
> Following is our test environment and some testing results:
>
>
> OS: CentOS Linux release 7.1.1503 (Core)
>
> CPU: Intel(R) Xeon(R) CPU E3-1245 v3 @ 3.40GHz
>
> Processor number: 8
>
> Memory: 12GB
>
> fio command:
>
> 1. (for numjobs=64):
>
> fio --filename=/dev/md2 --sync=0 --direct=0 --rw=randread --bs=4K
> --runtime=180 --size=50G --name=test-read --ioengine=libaio
> --numjobs=64 --iodepth=1 --group_reporting
>
> 2. (for numjobs=1):
>
> fio --filename=/dev/md2 --sync=0 --direct=0 --rw=randread --bs=4K
> --runtime=180 --size=50G --name=test-read --ioengine=libaio
> --numjobs=1 --iodepth=1 --group_reporting
>
>
>
> Here are test results:
>
>
> Part I. SSD (4 x 240GB Intel SSD create Raid6(syncing))
>
>
> a. 4K Random Read, numjobs=64
>
> Average Throughput Average IOPS
>
> Kernel 3.19.8 715937KB/s 178984
>
> Kernel 4.2.8 489874KB/s 122462
>
> Kernel 4.2.8 Patch Rollback 717377KB/s 179344
>
>
>
> b. 4K Random Read, numjobs=1
>
> Average Throughput Average IOPS
>
> Kernel 3.19.8 32203KB/s 8051
>
> Kernel 4.2.8 2535.7KB/s 633
>
> Kernel 4.2.8 Patch Rollback 31861KB/s 7965
>
>
>
>
> Part II. HDD (4 x 1TB TOSHIBA HDD create Raid6(syncing))
>
>
> a. 4K Random Read, numjobs=64
>
> Average Throughput Average IOPS
>
> Kernel 3.19.8 2976.6KB/s 744
>
> Kernel 4.2.8 2915.8KB/s 728
>
> Kernel 4.2.8 Patch Rollback 2973.3KB/s 743
>
>
>
> b. 4K Random Read, numjobs=1
>
> Average Throughput Average IOPS
>
> Kernel 3.19.8 481844 B/s 117
>
> Kernel 4.2.8 24718 B/s 5
>
> Kernel 4.2.8 Patch Rollback 460090 B/s 112
>
>
>
> Thanks,
>
> --
>
> Chien Lee
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]
^ permalink raw reply
* Re: md127 auto created when use "-B" to build a legacy array without superblocks
From: NeilBrown @ 2016-01-26 22:03 UTC (permalink / raw)
To: yizhan; +Cc: linux-raid
In-Reply-To: <56A72D21.20707@redhat.com>
[-- Attachment #1: Type: text/plain, Size: 4841 bytes --]
On Tue, Jan 26 2016, yizhan wrote:
> On 10/02/2015 03:55 PM, Neil Brown wrote:
>> Yi Zhang <yizhan@redhat.com> writes:
>>
>>> Hi Neil
>>>
>>> When testing 00raid1, found the md127 auto created when use "-B" to build a legacy array without superblocks, is it reasonable?
>> This happens because udev notices a new device has appeared, looks
>> inside it, sees that it could be part of an md array, and so runs
>> "mdadm --incremental" on it.
> Hi Neil
> I tried the mdadm --incremental, but it doesn't work, below is the
> operation.
> # mdadm -D /dev/md0
> /dev/md0:
> Version :
> Creation Time : Tue Jan 26 03:16:17 2016
> Raid Level : raid1
> Array Size : 2097152 (2.00 GiB 2.15 GB)
> Used Dev Size : 2097152 (2.00 GiB 2.15 GB)
> Raid Devices : 2
> Total Devices : 2
>
> State : clean
> Active Devices : 2
> Working Devices : 2
> Failed Devices : 0
> Spare Devices : 0
>
> Number Major Minor RaidDevice State
> 0 7 0 0 active sync /dev/loop0
> 1 7 1 1 active sync /dev/loop1
> # mdadm -D /dev/md127
> /dev/md127:
> Version : 1.2
> Raid Level : raid0
> Total Devices : 1
> Persistence : Superblock is persistent
>
> State : inactive
>
> Name : 0
> UUID : 13ada118:129135ff:686de7b7:4cb71d6d
> Events : 17
>
> Number Major Minor RaidDevice
>
> - 9 0 - /dev/md0
> # mdadm -I /dev/md127
> mdadm: /dev/md127 is not part of an md array.
> # mdadm -I /dev/md0
> mdadm: cannot reopen /dev/md0: Device or resource busy.
>
> After I stopped md127, the md0 can be used again.
If, after stopping md127, you try
mdadm -I /dev/md0
again it will start /dev/md127 again. This is what udev does which
causes md127 to appear in the first place.
NeilBrown
>
> Thanks
> Yi
>> Sometimes we want udev to do that. Sometimes we don't.
>> There is no easy way for udev to know what we want.
>>
>> NeilBrown
>>
>>
>>> pls check below detailed info:
>>>
>>> + mdadm -CR /dev/md0 --level=raid1 -n3 /dev/loop0 /dev/loop1 /dev/loop2
>>> mdadm: /dev/loop0 appears to contain an ext2fs file system
>>> size=58368K mtime=Thu Jan 1 08:00:00 1970
>>> mdadm: Note: this array has metadata at the start and
>>> may not be suitable as a boot device. If you plan to
>>> store '/boot' on this device please ensure that
>>> your boot-loader understands md/v1.x metadata, or use
>>> --metadata=0.90
>>> mdadm: /dev/loop1 appears to contain an ext2fs file system
>>> size=38912K mtime=Thu Jan 1 08:00:00 1970
>>> mdadm: Defaulting to version 1.2 metadata
>>> mdadm: array /dev/md0 started.
>>> + mdadm --wait /dev/md0
>>> + cat /proc/mdstat
>>> Personalities : [raid6] [raid5] [raid4] [raid1]
>>> md0 : active raid1 loop2[2] loop1[1] loop0[0]
>>> 19968 blocks super 1.2 [3/3] [UUU]
>>>
>>> unused devices: <none>
>>> + mdadm -S /dev/md0
>>> mdadm: stopped /dev/md0
>>> + mdadm -B /dev/md0 -l raid1 -n2 /dev/loop0 /dev/loop1
>>> mdadm: array /dev/md0 built and started.
>>> + sleep 2
>>> + cat /proc/mdstat
>>> Personalities : [raid6] [raid5] [raid4] [raid1]
>>> md127 : inactive md0[0](S)
>>> 19968 blocks super 1.2
>>>
>>> md0 : active raid1 loop1[1] loop0[0]
>>> 20000 blocks super non-persistent [2/2] [UU]
>>>
>>> unused devices: <none>
>>> [root@dhcp-12-171 bug]# uname -r
>>> 4.2.0
>>> [root@dhcp-12-171 bug]# mdadm -D /dev/md0
>>> /dev/md0:
>>> Version :
>>> Creation Time : Mon Sep 7 20:21:20 2015
>>> Raid Level : raid1
>>> Array Size : 20000 (19.53 MiB 20.48 MB)
>>> Used Dev Size : 20000 (19.53 MiB 20.48 MB)
>>> Raid Devices : 2
>>> Total Devices : 2
>>>
>>> State : clean
>>> Active Devices : 2
>>> Working Devices : 2
>>> Failed Devices : 0
>>> Spare Devices : 0
>>>
>>> Number Major Minor RaidDevice State
>>> 0 7 0 0 active sync /dev/loop0
>>> 1 7 1 1 active sync /dev/loop1
>>> [root@dhcp-12-171 bug]# mdadm -D /dev/md127
>>> /dev/md127:
>>> Version : 1.2
>>> Raid Level : raid0
>>> Total Devices : 1
>>> Persistence : Superblock is persistent
>>>
>>> State : inactive
>>>
>>> Name : dhcp-12-171.nay.redhat.com:0 (local to host dhcp-12-171.nay.redhat.com)
>>> UUID : 40ace956:a9dd0793:f4984d2b:8431b92b
>>> Events : 17
>>>
>>> Number Major Minor RaidDevice
>>>
>>> - 9 0 - /dev/md0
>>>
>>> Best Regards,
>>> Yi Zhang
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]
^ permalink raw reply
* Problem with array raid10 array resync on 4.4.0 (keeps reyncing each reboot)
From: Eric Valette @ 2016-01-26 14:49 UTC (permalink / raw)
To: linux-raid
Hi,
My raid 10 array (5 disk with one spare) was doing a resync after an
upgrade to 4.4.0 from 4.1.15. The resync progress was steady and at the
end the /proc/mdstat was apparently complete but when rebooting, it
started resycing over and over. I noticed my dmesg was totally filled
with raid10 conf printout message so it was impossible to trace anything
else.
Did a resync test with 3.14.58 (because I knew it had worked for resync
before and was still available as a boot option) and the array was
correctly rebuild.
Runs fine with 4.1.16 now.
Please CC me as I'm not subscribed.
-- eric
^ permalink raw reply
* Re: md127 auto created when use "-B" to build a legacy array without superblocks
From: yizhan @ 2016-01-26 8:24 UTC (permalink / raw)
To: Neil Brown; +Cc: linux-raid
In-Reply-To: <87k2r5iu0t.fsf@notabene.neil.brown.name>
On 10/02/2015 03:55 PM, Neil Brown wrote:
> Yi Zhang <yizhan@redhat.com> writes:
>
>> Hi Neil
>>
>> When testing 00raid1, found the md127 auto created when use "-B" to build a legacy array without superblocks, is it reasonable?
> This happens because udev notices a new device has appeared, looks
> inside it, sees that it could be part of an md array, and so runs
> "mdadm --incremental" on it.
Hi Neil
I tried the mdadm --incremental, but it doesn't work, below is the
operation.
# mdadm -D /dev/md0
/dev/md0:
Version :
Creation Time : Tue Jan 26 03:16:17 2016
Raid Level : raid1
Array Size : 2097152 (2.00 GiB 2.15 GB)
Used Dev Size : 2097152 (2.00 GiB 2.15 GB)
Raid Devices : 2
Total Devices : 2
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Number Major Minor RaidDevice State
0 7 0 0 active sync /dev/loop0
1 7 1 1 active sync /dev/loop1
# mdadm -D /dev/md127
/dev/md127:
Version : 1.2
Raid Level : raid0
Total Devices : 1
Persistence : Superblock is persistent
State : inactive
Name : 0
UUID : 13ada118:129135ff:686de7b7:4cb71d6d
Events : 17
Number Major Minor RaidDevice
- 9 0 - /dev/md0
# mdadm -I /dev/md127
mdadm: /dev/md127 is not part of an md array.
# mdadm -I /dev/md0
mdadm: cannot reopen /dev/md0: Device or resource busy.
After I stopped md127, the md0 can be used again.
Thanks
Yi
> Sometimes we want udev to do that. Sometimes we don't.
> There is no easy way for udev to know what we want.
>
> NeilBrown
>
>
>> pls check below detailed info:
>>
>> + mdadm -CR /dev/md0 --level=raid1 -n3 /dev/loop0 /dev/loop1 /dev/loop2
>> mdadm: /dev/loop0 appears to contain an ext2fs file system
>> size=58368K mtime=Thu Jan 1 08:00:00 1970
>> mdadm: Note: this array has metadata at the start and
>> may not be suitable as a boot device. If you plan to
>> store '/boot' on this device please ensure that
>> your boot-loader understands md/v1.x metadata, or use
>> --metadata=0.90
>> mdadm: /dev/loop1 appears to contain an ext2fs file system
>> size=38912K mtime=Thu Jan 1 08:00:00 1970
>> mdadm: Defaulting to version 1.2 metadata
>> mdadm: array /dev/md0 started.
>> + mdadm --wait /dev/md0
>> + cat /proc/mdstat
>> Personalities : [raid6] [raid5] [raid4] [raid1]
>> md0 : active raid1 loop2[2] loop1[1] loop0[0]
>> 19968 blocks super 1.2 [3/3] [UUU]
>>
>> unused devices: <none>
>> + mdadm -S /dev/md0
>> mdadm: stopped /dev/md0
>> + mdadm -B /dev/md0 -l raid1 -n2 /dev/loop0 /dev/loop1
>> mdadm: array /dev/md0 built and started.
>> + sleep 2
>> + cat /proc/mdstat
>> Personalities : [raid6] [raid5] [raid4] [raid1]
>> md127 : inactive md0[0](S)
>> 19968 blocks super 1.2
>>
>> md0 : active raid1 loop1[1] loop0[0]
>> 20000 blocks super non-persistent [2/2] [UU]
>>
>> unused devices: <none>
>> [root@dhcp-12-171 bug]# uname -r
>> 4.2.0
>> [root@dhcp-12-171 bug]# mdadm -D /dev/md0
>> /dev/md0:
>> Version :
>> Creation Time : Mon Sep 7 20:21:20 2015
>> Raid Level : raid1
>> Array Size : 20000 (19.53 MiB 20.48 MB)
>> Used Dev Size : 20000 (19.53 MiB 20.48 MB)
>> Raid Devices : 2
>> Total Devices : 2
>>
>> State : clean
>> Active Devices : 2
>> Working Devices : 2
>> Failed Devices : 0
>> Spare Devices : 0
>>
>> Number Major Minor RaidDevice State
>> 0 7 0 0 active sync /dev/loop0
>> 1 7 1 1 active sync /dev/loop1
>> [root@dhcp-12-171 bug]# mdadm -D /dev/md127
>> /dev/md127:
>> Version : 1.2
>> Raid Level : raid0
>> Total Devices : 1
>> Persistence : Superblock is persistent
>>
>> State : inactive
>>
>> Name : dhcp-12-171.nay.redhat.com:0 (local to host dhcp-12-171.nay.redhat.com)
>> UUID : 40ace956:a9dd0793:f4984d2b:8431b92b
>> Events : 17
>>
>> Number Major Minor RaidDevice
>>
>> - 9 0 - /dev/md0
>>
>> Best Regards,
>> Yi Zhang
^ permalink raw reply
* Re: [PATCH/RFC/RFT] md: allow resync to go faster when there is competing IO.
From: Chien Lee @ 2016-01-26 2:32 UTC (permalink / raw)
To: linux-raid, neilb, shli, owner-linux-raid
Hello,
Recently we find a bug about this patch (commit No. is
ac8fa4196d205ac8fff3f8932bddbad4f16e4110 ).
We know that this patch committed after Linux kernel 4.1.x is intended
to allowing resync to go faster when there is competing IO. However,
we find the performance of random read on syncing Raid6 will come up
with a huge drop in this case. The following is our testing detail.
The OS what we choose in our test is CentOS Linux release 7.1.1503
(Core) and the kernel image will be replaced for testing. In our
testing result, the 4K random read performance on syncing raid6 in
Kernel 4.2.8 is much lower than in Kernel 3.19.8. In order to find out
the root cause, we try to rollback this patch in Kernel 4.2.8, and we
find the 4K random read performance on syncing Raid6 will be improved
and go back to as what it should be in Kernel 3.19.8.
Nevertheless, it seems that it will not affect some other read/write
patterns. In our testing result, the 1M sequential read/write, 4K
random write performance in Kernel 4.2.8 is performed almost the same
as in Kernel 3.19.8.
It seems that although this patch increases the resync speed, the
logic of !is_mddev_idle() cause the sync request wait too short and
reduce the chance for raid5d to handle the random read I/O.
Following is our test environment and some testing results:
OS: CentOS Linux release 7.1.1503 (Core)
CPU: Intel(R) Xeon(R) CPU E3-1245 v3 @ 3.40GHz
Processor number: 8
Memory: 12GB
fio command:
1. (for numjobs=64):
fio --filename=/dev/md2 --sync=0 --direct=0 --rw=randread --bs=4K
--runtime=180 --size=50G --name=test-read --ioengine=libaio
--numjobs=64 --iodepth=1 --group_reporting
2. (for numjobs=1):
fio --filename=/dev/md2 --sync=0 --direct=0 --rw=randread --bs=4K
--runtime=180 --size=50G --name=test-read --ioengine=libaio
--numjobs=1 --iodepth=1 --group_reporting
Here are test results:
Part I. SSD (4 x 240GB Intel SSD create Raid6(syncing))
a. 4K Random Read, numjobs=64
Average Throughput Average IOPS
Kernel 3.19.8 715937KB/s 178984
Kernel 4.2.8 489874KB/s 122462
Kernel 4.2.8 Patch Rollback 717377KB/s 179344
b. 4K Random Read, numjobs=1
Average Throughput Average IOPS
Kernel 3.19.8 32203KB/s 8051
Kernel 4.2.8 2535.7KB/s 633
Kernel 4.2.8 Patch Rollback 31861KB/s 7965
Part II. HDD (4 x 1TB TOSHIBA HDD create Raid6(syncing))
a. 4K Random Read, numjobs=64
Average Throughput Average IOPS
Kernel 3.19.8 2976.6KB/s 744
Kernel 4.2.8 2915.8KB/s 728
Kernel 4.2.8 Patch Rollback 2973.3KB/s 743
b. 4K Random Read, numjobs=1
Average Throughput Average IOPS
Kernel 3.19.8 481844 B/s 117
Kernel 4.2.8 24718 B/s 5
Kernel 4.2.8 Patch Rollback 460090 B/s 112
Thanks,
--
Chien Lee
^ permalink raw reply
* Re: [PATCH RESEND v2 00/19] Support fuse mounts in user namespaces
From: Seth Forshee @ 2016-01-25 20:36 UTC (permalink / raw)
To: Eric W. Biederman
Cc: linux-bcache, dm-devel, linux-raid, linux-mtd, linux-fsdevel,
fuse-devel, linux-security-module, selinux, Alexander Viro,
Serge Hallyn, Richard Weinberger, Austin S Hemmelgarn,
Miklos Szeredi, linux-kernel
In-Reply-To: <87mvrtqvhp.fsf@x220.int.ebiederm.org>
On Mon, Jan 25, 2016 at 02:01:22PM -0600, Eric W. Biederman wrote:
> Seth Forshee <seth.forshee@canonical.com> writes:
>
> > On Mon, Jan 04, 2016 at 12:03:39PM -0600, Seth Forshee wrote:
> >> These patches implement support for mounting filesystems in user
> >> namespaces using fuse. They are based on the patches in the for-testing
> >> branch of
> >> git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace.git,
> >> but I've rebased them onto 4.4-rc3. I've pushed all of this to:
> >>
> >> git://git.kernel.org/pub/scm/linux/kernel/git/sforshee/linux.git fuse-userns
> >>
> >> The patches are organized into three high-level groups.
> >>
> >> Patches 1-6 are related to security, adding restrictions for
> >> unprivileged mounts and updating the LSMs as needed. Patches 1-2
> >> (checking inode permissions for block device mounts) may not be strictly
> >> necessary for fuseblk mounts since fuse doesn't do any IO on the block
> >> device in the kernel, but it still seems like a good idea to fail the
> >> mount if the user doesn't have the required permissions for the inode
> >> (though this is a bit misleading with fuse since the mounts are done via
> >> a suid-root helper).
> >>
> >> Patches 7-14 update most of the vfs to translate ids correctly and deal
> >> with inodes which may have invalid user/group ids. I've omitted patches
> >> for anything not used by fuse - quota, fs freezing, some helper
> >> functions, etc. - but if these are wanted for the sake of completeness I
> >> can include them.
> >>
> >> Patches 15-18 update fuse to deal with mounts from non-init pid and user
> >> namespaces and enable mounting from user namespaces.
> >>
> >> Changes since v1:
> >> - Drop patch for FIBMAP.
> >> - Use current_in_userns in fuse_allow_current_process.
> >> - Remove checks for uid/gid validity in fuse. Intead, ids from the
> >> backing store which do not map into s_user_ns will result in invalid
> >> ids in the vfs inode. Checks in the vfs will prevent unmappable ids
> >> from being passed in from above.
> >> - Update a couple of commit messages to provide more detail about
> >> changes.
> >
> > Now that the merge window is over, I'm wondering whether it might be
> > possible to get some feedback on these patches this cycle?
>
> Definitely. Apologies for not giving you much feedback earlier.
>
> I had been hoping this was the kind of thing I could just double check
> to be certain you weren't doing anything silly and just apply. After my
> last round of looking at this I realized that for me to be comfortable
> with these patches I will have to give them very close scrutiny, and
> check every detail.
>
> Unfortunatly last cycle I had failed to budget enough time to give these
> patches the close scrutiny they need.
>
> From a high level I am still very much in favor of this approach and
> at least getting as far as safe unprivileged fuse mounts.
>
> I have one or two little things to look at and then I hope to be going
> through your patches one by one in detail.
Great. Thanks Eric.
^ permalink raw reply
* Re: [PATCH RESEND v2 00/19] Support fuse mounts in user namespaces
From: Eric W. Biederman @ 2016-01-25 20:01 UTC (permalink / raw)
To: Seth Forshee
Cc: linux-bcache, dm-devel, linux-raid, linux-mtd, linux-fsdevel,
fuse-devel, linux-security-module, selinux, Alexander Viro,
Serge Hallyn, Richard Weinberger, Austin S Hemmelgarn,
Miklos Szeredi, linux-kernel
In-Reply-To: <20160125194722.GA10638@ubuntu-hedt>
Seth Forshee <seth.forshee@canonical.com> writes:
> On Mon, Jan 04, 2016 at 12:03:39PM -0600, Seth Forshee wrote:
>> These patches implement support for mounting filesystems in user
>> namespaces using fuse. They are based on the patches in the for-testing
>> branch of
>> git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace.git,
>> but I've rebased them onto 4.4-rc3. I've pushed all of this to:
>>
>> git://git.kernel.org/pub/scm/linux/kernel/git/sforshee/linux.git fuse-userns
>>
>> The patches are organized into three high-level groups.
>>
>> Patches 1-6 are related to security, adding restrictions for
>> unprivileged mounts and updating the LSMs as needed. Patches 1-2
>> (checking inode permissions for block device mounts) may not be strictly
>> necessary for fuseblk mounts since fuse doesn't do any IO on the block
>> device in the kernel, but it still seems like a good idea to fail the
>> mount if the user doesn't have the required permissions for the inode
>> (though this is a bit misleading with fuse since the mounts are done via
>> a suid-root helper).
>>
>> Patches 7-14 update most of the vfs to translate ids correctly and deal
>> with inodes which may have invalid user/group ids. I've omitted patches
>> for anything not used by fuse - quota, fs freezing, some helper
>> functions, etc. - but if these are wanted for the sake of completeness I
>> can include them.
>>
>> Patches 15-18 update fuse to deal with mounts from non-init pid and user
>> namespaces and enable mounting from user namespaces.
>>
>> Changes since v1:
>> - Drop patch for FIBMAP.
>> - Use current_in_userns in fuse_allow_current_process.
>> - Remove checks for uid/gid validity in fuse. Intead, ids from the
>> backing store which do not map into s_user_ns will result in invalid
>> ids in the vfs inode. Checks in the vfs will prevent unmappable ids
>> from being passed in from above.
>> - Update a couple of commit messages to provide more detail about
>> changes.
>
> Now that the merge window is over, I'm wondering whether it might be
> possible to get some feedback on these patches this cycle?
Definitely. Apologies for not giving you much feedback earlier.
I had been hoping this was the kind of thing I could just double check
to be certain you weren't doing anything silly and just apply. After my
last round of looking at this I realized that for me to be comfortable
with these patches I will have to give them very close scrutiny, and
check every detail.
Unfortunatly last cycle I had failed to budget enough time to give these
patches the close scrutiny they need.
From a high level I am still very much in favor of this approach and
at least getting as far as safe unprivileged fuse mounts.
I have one or two little things to look at and then I hope to be going
through your patches one by one in detail.
Eric
^ permalink raw reply
* Re: [PATCH RESEND v2 00/19] Support fuse mounts in user namespaces
From: Seth Forshee @ 2016-01-25 19:47 UTC (permalink / raw)
To: Eric W. Biederman, linux-bcache, dm-devel, linux-raid, linux-mtd,
linux-fsdevel, fuse-devel, linux-security-module, selinux
Cc: Alexander Viro, Serge Hallyn, Richard Weinberger,
Austin S Hemmelgarn, Miklos Szeredi, linux-kernel
In-Reply-To: <1451930639-94331-1-git-send-email-seth.forshee@canonical.com>
On Mon, Jan 04, 2016 at 12:03:39PM -0600, Seth Forshee wrote:
> These patches implement support for mounting filesystems in user
> namespaces using fuse. They are based on the patches in the for-testing
> branch of
> git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace.git,
> but I've rebased them onto 4.4-rc3. I've pushed all of this to:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/sforshee/linux.git fuse-userns
>
> The patches are organized into three high-level groups.
>
> Patches 1-6 are related to security, adding restrictions for
> unprivileged mounts and updating the LSMs as needed. Patches 1-2
> (checking inode permissions for block device mounts) may not be strictly
> necessary for fuseblk mounts since fuse doesn't do any IO on the block
> device in the kernel, but it still seems like a good idea to fail the
> mount if the user doesn't have the required permissions for the inode
> (though this is a bit misleading with fuse since the mounts are done via
> a suid-root helper).
>
> Patches 7-14 update most of the vfs to translate ids correctly and deal
> with inodes which may have invalid user/group ids. I've omitted patches
> for anything not used by fuse - quota, fs freezing, some helper
> functions, etc. - but if these are wanted for the sake of completeness I
> can include them.
>
> Patches 15-18 update fuse to deal with mounts from non-init pid and user
> namespaces and enable mounting from user namespaces.
>
> Changes since v1:
> - Drop patch for FIBMAP.
> - Use current_in_userns in fuse_allow_current_process.
> - Remove checks for uid/gid validity in fuse. Intead, ids from the
> backing store which do not map into s_user_ns will result in invalid
> ids in the vfs inode. Checks in the vfs will prevent unmappable ids
> from being passed in from above.
> - Update a couple of commit messages to provide more detail about
> changes.
Now that the merge window is over, I'm wondering whether it might be
possible to get some feedback on these patches this cycle?
Thanks,
Seth
^ permalink raw reply
* Re: Uneven wear on raid1 devices
From: Phil Turmel @ 2016-01-25 18:00 UTC (permalink / raw)
To: Roman Mamedov, Adam Goryachev; +Cc: linux-raid
In-Reply-To: <20160125104027.597041a3@natsu>
On 01/25/2016 12:40 AM, Roman Mamedov wrote:
> On Mon, 25 Jan 2016 16:29:02 +1100
> Adam Goryachev <adam@websitemanagers.com.au> wrote:
>
>> 9 Power_On_Hours -O--CK 098 098 000 - 6435
>
>> 9 Power_On_Hours -O--CK 095 095 000 - 23178
>
> 2nd drive has almost 4x as much power-on time than the first one. My guess
> would be that it accumulated all that write usage back before you put it into
> this RAID1.
Or you are doing "repair" scrubs when you should be doing "check"
scrubs. Any operation that requires resynchronization will read from
the first mirror and write to the others.
> If you want to ensure the RAID1 usage is even, record the SMART data you have
> now, and compare to the readings you will have a month later.
By the way, your devices show scterc disabled. That's bad. You should
definitely use boot scripts or udev rules to enable it.
Has sdd been getting bumped out of the array lately, and resyncing when
you put it back?
Phil
^ permalink raw reply
* Re: broken raid level 5 array caused by user error
From: Mathias Mueller @ 2016-01-25 10:02 UTC (permalink / raw)
To: Phil Turmel; +Cc: Linux raid, linux-raid-owner
In-Reply-To: <56A27F5E.1000602@turmel.org>
Hi Phil,
thanks a lot once more, I would never managed it without your help. It's
very nice, that you spend so much time in supporting everyone and
writing very helpful tools. I learned a lot, technically and also to
have more patience and thinking twice _before_ doing things :)
> Sadly, it's hard to ship beer here in the US without government
> involvement.
That's a very sad fact! I send you something via paypal, so you might
get some german beer from some US store, if you like. Tell me, if you
need consulting ;)
Keep on going and best regards
Mathias
^ permalink raw reply
* Re: Uneven wear on raid1 devices
From: Roman Mamedov @ 2016-01-25 5:40 UTC (permalink / raw)
To: Adam Goryachev; +Cc: linux-raid
In-Reply-To: <56A5B29E.3020708@websitemanagers.com.au>
[-- Attachment #1: Type: text/plain, Size: 577 bytes --]
On Mon, 25 Jan 2016 16:29:02 +1100
Adam Goryachev <adam@websitemanagers.com.au> wrote:
> 9 Power_On_Hours -O--CK 098 098 000 - 6435
> 9 Power_On_Hours -O--CK 095 095 000 - 23178
2nd drive has almost 4x as much power-on time than the first one. My guess
would be that it accumulated all that write usage back before you put it into
this RAID1.
If you want to ensure the RAID1 usage is even, record the SMART data you have
now, and compare to the readings you will have a month later.
--
With respect,
Roman
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]
^ permalink raw reply
* Uneven wear on raid1 devices
From: Adam Goryachev @ 2016-01-25 5:29 UTC (permalink / raw)
To: linux-raid
Hi all,
I was hoping to get some advice on why my two SSD's seem to be wearing
unevenly, when they are both in the same raid1 array.
First, the array information:
# mdadm --misc --detail /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Fri Jun 14 00:42:04 2013
Raid Level : raid1
Array Size : 499975360 (476.81 GiB 511.97 GB)
Used Dev Size : 499975360 (476.81 GiB 511.97 GB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent
Intent Bitmap : Internal
Update Time : Mon Jan 25 16:20:11 2016
State : active
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Name : flail:0 (local to host flail)
UUID : 81b25e43:787b0bc4:f934d006:a726aa89
Events : 1240606
Number Major Minor RaidDevice State
2 8 17 0 active sync /dev/sdb1
1 8 49 1 active sync /dev/sdd1
# smartctl -A /dev/sdb|grep Wear
177 Wear_Leveling_Count 0x0013 093 093 000 Pre-fail
Always - 248
# smartctl -A /dev/sdd|grep Wear
177 Wear_Leveling_Count 0x0013 074 074 000 Pre-fail
Always - 915
Equally, we can see a different amount of data written to each drive:
# smartctl -A /dev/sdb|grep Written
241 Total_LBAs_Written 0x0032 099 099 000 Old_age
Always - 7169916114
# smartctl -A /dev/sdd|grep Written
241 Total_LBAs_Written 0x0032 099 099 000 Old_age
Always - 24013780683
I'm stuck wondering why one drive is getting so many more writes than
the other, and is there some way to "swap" this so that the other drive
will get the heavier load over the next 12 months?
In theory, this suggests that one drive has 74% life remaining, and the
other has 93% (unless I'm reading that incorrectly...), while 74% is
still a lot, I guess I would prefer to prevent it getting too much lower
and needing replacement....
Any suggestions or advice greatly appreciated.
Full output of smartctl -x below for each drive:
# smartctl -x /dev/sdb
smartctl 5.41 2011-06-09 r3365 [i686-linux-3.2.0-4-686-pae] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net
=== START OF INFORMATION SECTION ===
Device Model: Samsung SSD 840 PRO Series
Serial Number: S1AXNSAF702689E
LU WWN Device Id: 5 002538 5a02701b8
Firmware Version: DXM06B0Q
User Capacity: 512,110,190,592 bytes [512 GB]
Sector Size: 512 bytes logical/physical
Device is: Not in smartctl database [for details use: -P showall]
ATA Version is: 8
ATA Standard is: ATA-8-ACS revision 4c
Local Time is: Mon Jan 25 16:28:01 2016 AEDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x00) Offline data collection activity
was never started.
Auto Offline Data Collection: Disabled.
Self-test execution status: ( 0) The previous self-test
routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: (53956) seconds.
Offline data collection
capabilities: (0x53) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
No Offline surface scan supported.
Self-test supported.
No Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 35) minutes.
SCT capabilities: (0x003d) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 1
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAGS VALUE WORST THRESH FAIL RAW_VALUE
5 Reallocated_Sector_Ct PO--CK 100 100 010 - 0
9 Power_On_Hours -O--CK 098 098 000 - 6435
12 Power_Cycle_Count -O--CK 099 099 000 - 1
177 Wear_Leveling_Count PO--C- 093 093 000 - 248
179 Used_Rsvd_Blk_Cnt_Tot PO--C- 100 100 010 - 0
181 Program_Fail_Cnt_Total -O--CK 100 100 010 - 0
182 Erase_Fail_Count_Total -O--CK 100 100 010 - 0
183 Runtime_Bad_Block PO--C- 100 100 010 - 0
187 Reported_Uncorrect -O--CK 100 100 000 - 0
190 Airflow_Temperature_Cel -O--CK 072 060 000 - 28
195 Hardware_ECC_Recovered -O-RC- 200 200 000 - 0
199 UDMA_CRC_Error_Count -OSRCK 100 100 000 - 0
235 Unknown_Attribute -O--C- 100 100 000 - 0
241 Total_LBAs_Written -O--CK 099 099 000 - 7169972542
||||||_ K auto-keep
|||||__ C event count
||||___ R error rate
|||____ S speed/performance
||_____ O updated online
|______ P prefailure warning
General Purpose Log Directory Version 1
SMART Log Directory Version 1 [multi-sector log support]
GP/S Log at address 0x00 has 1 sectors [Log Directory]
GP/S Log at address 0x01 has 1 sectors [Summary SMART error log]
GP/S Log at address 0x02 has 1 sectors [Comprehensive SMART error log]
GP/S Log at address 0x03 has 1 sectors [Ext. Comprehensive SMART
error log]
GP/S Log at address 0x06 has 1 sectors [SMART self-test log]
GP/S Log at address 0x07 has 1 sectors [Extended self-test log]
GP/S Log at address 0x09 has 1 sectors [Selective self-test log]
GP/S Log at address 0x10 has 1 sectors [NCQ Command Error]
GP/S Log at address 0x11 has 1 sectors [SATA Phy Event Counters]
GP/S Log at address 0x30 has 1 sectors [Reserved]
GP/S Log at address 0x80 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x81 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x82 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x83 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x84 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x85 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x86 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x87 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x88 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x89 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x8a has 16 sectors [Host vendor specific log]
GP/S Log at address 0x8b has 16 sectors [Host vendor specific log]
GP/S Log at address 0x8c has 16 sectors [Host vendor specific log]
GP/S Log at address 0x8d has 16 sectors [Host vendor specific log]
GP/S Log at address 0x8e has 16 sectors [Host vendor specific log]
GP/S Log at address 0x8f has 16 sectors [Host vendor specific log]
GP/S Log at address 0x90 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x91 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x92 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x93 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x94 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x95 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x96 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x97 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x98 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x99 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x9a has 16 sectors [Host vendor specific log]
GP/S Log at address 0x9b has 16 sectors [Host vendor specific log]
GP/S Log at address 0x9c has 16 sectors [Host vendor specific log]
GP/S Log at address 0x9d has 16 sectors [Host vendor specific log]
GP/S Log at address 0x9e has 16 sectors [Host vendor specific log]
GP/S Log at address 0x9f has 16 sectors [Host vendor specific log]
GP/S Log at address 0xa0 has 16 sectors [Device vendor specific log]
SMART Extended Comprehensive Error Log Version: 1 (1 sectors)
No Errors Logged
SMART Extended Self-test Log Version: 1 (1 sectors)
No self-tests have been logged. [To run self-tests, use: smartctl -t]
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
255 0 65535 Read_scanning was never started
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
Error unknown SCT Temperature History Format Version (3), should be 2.
SCT Error Recovery Control:
Read: Disabled
Write: Disabled
SATA Phy Event Counters (GP Log 0x11)
ID Size Value Description
0x0001 2 0 Command failed due to ICRC error
0x0002 2 0 R_ERR response for data FIS
0x0003 2 0 R_ERR response for device-to-host data FIS
0x0004 2 0 R_ERR response for host-to-device data FIS
0x0005 2 0 R_ERR response for non-data FIS
0x0006 2 0 R_ERR response for device-to-host non-data FIS
0x0007 2 0 R_ERR response for host-to-device non-data FIS
0x0008 2 0 Device-to-host non-data FIS retries
0x0009 2 11 Transition from drive PhyRdy to drive PhyNRdy
0x000a 2 11 Device-to-host register FISes sent due to a COMRESET
0x000b 2 0 CRC errors within host-to-device FIS
0x000d 2 0 Non-CRC errors within host-to-device FIS
0x000f 2 0 R_ERR response for host-to-device data FIS, CRC
0x0010 2 0 R_ERR response for host-to-device data FIS, non-CRC
0x0012 2 0 R_ERR response for host-to-device non-data FIS, CRC
0x0013 2 0 R_ERR response for host-to-device non-data FIS,
non-CRC
# smartctl -x /dev/sdd
smartctl 5.41 2011-06-09 r3365 [i686-linux-3.2.0-4-686-pae] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net
=== START OF INFORMATION SECTION ===
Device Model: Samsung SSD 840 PRO Series
Serial Number: S12SNEAD204208R
LU WWN Device Id: 5 002538 5501d20d0
Firmware Version: DXM04B0Q
User Capacity: 512,110,190,592 bytes [512 GB]
Sector Size: 512 bytes logical/physical
Device is: Not in smartctl database [for details use: -P showall]
ATA Version is: 8
ATA Standard is: ATA-8-ACS revision 4c
Local Time is: Mon Jan 25 16:26:24 2016 AEDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x00) Offline data collection activity
was never started.
Auto Offline Data Collection: Disabled.
Self-test execution status: ( 0) The previous self-test
routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: (53956) seconds.
Offline data collection
capabilities: (0x53) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
No Offline surface scan supported.
Self-test supported.
No Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 35) minutes.
SCT capabilities: (0x003d) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 1
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAGS VALUE WORST THRESH FAIL RAW_VALUE
5 Reallocated_Sector_Ct PO--CK 100 100 010 - 0
9 Power_On_Hours -O--CK 095 095 000 - 23178
12 Power_Cycle_Count -O--CK 099 099 000 - 8
177 Wear_Leveling_Count PO--C- 074 074 000 - 915
179 Used_Rsvd_Blk_Cnt_Tot PO--C- 100 100 010 - 0
181 Program_Fail_Cnt_Total -O--CK 100 100 010 - 0
182 Erase_Fail_Count_Total -O--CK 100 100 010 - 0
183 Runtime_Bad_Block PO--C- 100 100 010 - 0
187 Reported_Uncorrect -O--CK 100 100 000 - 0
190 Airflow_Temperature_Cel -O--CK 073 058 000 - 27
195 Hardware_ECC_Recovered -O-RC- 200 200 000 - 0
199 UDMA_CRC_Error_Count -OSRCK 100 100 000 - 0
235 Unknown_Attribute -O--C- 099 099 000 - 6
241 Total_LBAs_Written -O--CK 099 099 000 - 24013821507
||||||_ K auto-keep
|||||__ C event count
||||___ R error rate
|||____ S speed/performance
||_____ O updated online
|______ P prefailure warning
General Purpose Log Directory Version 1
SMART Log Directory Version 1 [multi-sector log support]
GP/S Log at address 0x00 has 1 sectors [Log Directory]
GP/S Log at address 0x01 has 1 sectors [Summary SMART error log]
GP/S Log at address 0x02 has 1 sectors [Comprehensive SMART error log]
GP/S Log at address 0x03 has 1 sectors [Ext. Comprehensive SMART
error log]
GP/S Log at address 0x06 has 1 sectors [SMART self-test log]
GP/S Log at address 0x07 has 1 sectors [Extended self-test log]
GP/S Log at address 0x09 has 1 sectors [Selective self-test log]
GP/S Log at address 0x10 has 1 sectors [NCQ Command Error]
GP/S Log at address 0x11 has 1 sectors [SATA Phy Event Counters]
GP/S Log at address 0x30 has 1 sectors [Reserved]
GP/S Log at address 0x80 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x81 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x82 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x83 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x84 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x85 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x86 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x87 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x88 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x89 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x8a has 16 sectors [Host vendor specific log]
GP/S Log at address 0x8b has 16 sectors [Host vendor specific log]
GP/S Log at address 0x8c has 16 sectors [Host vendor specific log]
GP/S Log at address 0x8d has 16 sectors [Host vendor specific log]
GP/S Log at address 0x8e has 16 sectors [Host vendor specific log]
GP/S Log at address 0x8f has 16 sectors [Host vendor specific log]
GP/S Log at address 0x90 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x91 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x92 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x93 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x94 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x95 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x96 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x97 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x98 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x99 has 16 sectors [Host vendor specific log]
GP/S Log at address 0x9a has 16 sectors [Host vendor specific log]
GP/S Log at address 0x9b has 16 sectors [Host vendor specific log]
GP/S Log at address 0x9c has 16 sectors [Host vendor specific log]
GP/S Log at address 0x9d has 16 sectors [Host vendor specific log]
GP/S Log at address 0x9e has 16 sectors [Host vendor specific log]
GP/S Log at address 0x9f has 16 sectors [Host vendor specific log]
GP/S Log at address 0xa0 has 16 sectors [Device vendor specific log]
SMART Extended Comprehensive Error Log Version: 1 (1 sectors)
No Errors Logged
SMART Extended Self-test Log Version: 1 (1 sectors)
No self-tests have been logged. [To run self-tests, use: smartctl -t]
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
255 0 65535 Read_scanning was never started
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
Error unknown SCT Temperature History Format Version (3), should be 2.
SCT Error Recovery Control:
Read: Disabled
Write: Disabled
SATA Phy Event Counters (GP Log 0x11)
ID Size Value Description
0x0001 2 0 Command failed due to ICRC error
0x0002 2 0 R_ERR response for data FIS
0x0003 2 0 R_ERR response for device-to-host data FIS
0x0004 2 0 R_ERR response for host-to-device data FIS
0x0005 2 0 R_ERR response for non-data FIS
0x0006 2 0 R_ERR response for device-to-host non-data FIS
0x0007 2 0 R_ERR response for host-to-device non-data FIS
0x0008 2 0 Device-to-host non-data FIS retries
0x0009 2 11 Transition from drive PhyRdy to drive PhyNRdy
0x000a 2 11 Device-to-host register FISes sent due to a COMRESET
0x000b 2 0 CRC errors within host-to-device FIS
0x000d 2 0 Non-CRC errors within host-to-device FIS
0x000f 2 0 R_ERR response for host-to-device data FIS, CRC
0x0010 2 0 R_ERR response for host-to-device data FIS, non-CRC
0x0012 2 0 R_ERR response for host-to-device non-data FIS, CRC
0x0013 2 0 R_ERR response for host-to-device non-data FIS,
non-CRC
^ permalink raw reply
* Re: [PATCH 2/2] md-cluster: delete useless code
From: Guoqing Jiang @ 2016-01-25 1:40 UTC (permalink / raw)
To: Shaohua Li, linux-raid; +Cc: Goldwyn Rodrigues, NeilBrown
In-Reply-To: <688b654fe7566f182d6f4fdae2c3a3242d32a179.1453509823.git.shli@fb.com>
On 01/23/2016 08:45 AM, Shaohua Li wrote:
> page->index already considers node offset. The node_offset calculation
> in write_sb_page is useless and confusion.
Acked-by: Guoqing Jiang <gqjiang@suse.com>
Thanks,
Guoqing
> Cc: Goldwyn Rodrigues <rgoldwyn@suse.com>
> Cc: Guoqing Jiang <gqjiang@suse.com>
> Cc: NeilBrown <neilb@suse.com>
> Signed-off-by: Shaohua Li <shli@fb.com>
> ---
> drivers/md/bitmap.c | 4 ----
> 1 file changed, 4 deletions(-)
>
> diff --git a/drivers/md/bitmap.c b/drivers/md/bitmap.c
> index 4f22e91..d80cce4 100644
> --- a/drivers/md/bitmap.c
> +++ b/drivers/md/bitmap.c
> @@ -210,10 +210,6 @@ static int write_sb_page(struct bitmap *bitmap, struct page *page, int wait)
> struct block_device *bdev;
> struct mddev *mddev = bitmap->mddev;
> struct bitmap_storage *store = &bitmap->storage;
> - int node_offset = 0;
> -
> - if (mddev_is_clustered(bitmap->mddev))
> - node_offset = bitmap->cluster_slot * store->file_pages;
>
> while ((rdev = next_active_rdev(rdev, mddev)) != NULL) {
> int size = PAGE_SIZE;
^ permalink raw reply
* Re: [PATCH 1/2] md-cluster: fix missing memory free
From: Guoqing Jiang @ 2016-01-25 1:33 UTC (permalink / raw)
To: Shaohua Li, linux-raid; +Cc: Goldwyn Rodrigues, NeilBrown
In-Reply-To: <0bc7fe7dece2c5aa4f8ed7194df27b02e0ae3142.1453509823.git.shli@fb.com>
On 01/23/2016 08:45 AM, Shaohua Li wrote:
> There are several places we allocate dlm_lock_resource, but not free it.
Culd you also add below change? Thanks.
diff --git a/drivers/md/md-cluster.c b/drivers/md/md-cluster.c
index 0ded8e9..6c756e2 100644
--- a/drivers/md/md-cluster.c
+++ b/drivers/md/md-cluster.c
@@ -858,6 +858,7 @@ static int leave(struct mddev *mddev)
lockres_free(cinfo->token_lockres);
lockres_free(cinfo->ack_lockres);
lockres_free(cinfo->no_new_dev_lockres);
+ lockres_free(cinfo->resync_lockres);
lockres_free(cinfo->bitmap_lockres);
unlock_all_bitmaps(mddev);
dlm_release_lockspace(cinfo->lockspace, 2);
Guoqing
>
> Cc: Goldwyn Rodrigues <rgoldwyn@suse.com>
> Cc: Guoqing Jiang <gqjiang@suse.com>
> Cc: NeilBrown <neilb@suse.com>
> Signed-off-by: Shaohua Li <shli@fb.com>
> ---
> drivers/md/md-cluster.c | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/md/md-cluster.c b/drivers/md/md-cluster.c
> index 0ded8e9..662369c 100644
> --- a/drivers/md/md-cluster.c
> +++ b/drivers/md/md-cluster.c
> @@ -293,6 +293,7 @@ static void recover_bitmaps(struct md_thread *thread)
> dlm_unlock:
> dlm_unlock_sync(bm_lockres);
> clear_bit:
> + lockres_free(bm_lockres);
> clear_bit(slot, &cinfo->recovery_map);
> }
> }
> @@ -682,8 +683,10 @@ static int gather_all_resync_info(struct mddev *mddev, int total_slots)
> bm_lockres = lockres_init(mddev, str, NULL, 1);
> if (!bm_lockres)
> return -ENOMEM;
> - if (i == (cinfo->slot_number - 1))
> + if (i == (cinfo->slot_number - 1)) {
> + lockres_free(bm_lockres);
> continue;
> + }
>
> bm_lockres->flags |= DLM_LKF_NOQUEUE;
> ret = dlm_lock_sync(bm_lockres, DLM_LOCK_PW);
^ permalink raw reply related
* Re: md failing mechanism
From: Adam Goryachev @ 2016-01-24 22:13 UTC (permalink / raw)
To: James J, Wols Lists, linux-raid
In-Reply-To: <56A3CE28.9090901@shiftmail.org>
On 24/01/2016 06:02, James J wrote:
> On 23/01/2016 15:09, Wols Lists wrote:
>> On 22/01/16 23:40, James J wrote:
>>> The recommentation of raising the timeout to 120+ is for the opposite
>>> purpose of what you want. It is for the case the sysadmin accepts to
>>> wait a long time because he wants to prevent the kicking of the
>>> drive at
>>> the first read-error (normally drives are kicked for a write error).
>>> This might be wanted in order to a) defer the replacement of the drive,
>>> either to perform the replacement at a more opportune time and/or in a
>>> better manner such as a no-degrade replace operation, or b) because he
>>> does not want to replace the drive at all: maybe he believes that the
>>> error might be spurious and will not happen again and the drive is
>>> still
>>> of acceptable fitness for the purpose, e.g. in a low-cost file server.
>> Except, aiui, even in your scenario! drives are kicked for a *write*
>> error.
>>
>> What happens (should be) is the kernel times out, the raid handles the
>> read error by trying a rewrite, the drive is still hung on the read
>> error so it doesn't respond to the write request, and the drive gets
>> kicked for a write failure.
>
> Oh yes you are correct, so the drive would be kicked after 60secs and
> not after 30secs contrary to what I said.
> So the sequence would be: drive stuck on read --> scsi read failure
> due to timeout at the 30th second --> MD receives failure and attempts
> rewrite --> scsi write failure due to timeout at the 60th second -->
> drive kicked by MD at the 60th second
> I think this is what should have happened, but it didn't happen like
> this anyway so I think there is probably a kernel bug somewhere.
I don't have a lot to add, except that I recall the OP suggested it was
an IDE drive. I wonder if the IDE sub-system and/or hardware operates
differently compared to the sata variants. Possibly the MD layer never
got any timeout or error on the read, and (or maybe it was the write)
and hence it was never kicked from the array.
Regards,
Adam
^ permalink raw reply
* Re: md failing mechanism
From: James J @ 2016-01-23 19:02 UTC (permalink / raw)
To: Wols Lists, linux-raid
In-Reply-To: <56A389A9.1080203@youngman.org.uk>
On 23/01/2016 15:09, Wols Lists wrote:
> On 22/01/16 23:40, James J wrote:
>> The recommentation of raising the timeout to 120+ is for the opposite
>> purpose of what you want. It is for the case the sysadmin accepts to
>> wait a long time because he wants to prevent the kicking of the drive at
>> the first read-error (normally drives are kicked for a write error).
>> This might be wanted in order to a) defer the replacement of the drive,
>> either to perform the replacement at a more opportune time and/or in a
>> better manner such as a no-degrade replace operation, or b) because he
>> does not want to replace the drive at all: maybe he believes that the
>> error might be spurious and will not happen again and the drive is still
>> of acceptable fitness for the purpose, e.g. in a low-cost file server.
> Except, aiui, even in your scenario! drives are kicked for a *write* error.
>
> What happens (should be) is the kernel times out, the raid handles the
> read error by trying a rewrite, the drive is still hung on the read
> error so it doesn't respond to the write request, and the drive gets
> kicked for a write failure.
Oh yes you are correct, so the drive would be kicked after 60secs and
not after 30secs contrary to what I said.
So the sequence would be: drive stuck on read --> scsi read failure due
to timeout at the 30th second --> MD receives failure and attempts
rewrite --> scsi write failure due to timeout at the 60th second -->
drive kicked by MD at the 60th second
I think this is what should have happened, but it didn't happen like
this anyway so I think there is probably a kernel bug somewhere.
^ permalink raw reply
* Re: md failing mechanism
From: Phil Turmel @ 2016-01-23 15:12 UTC (permalink / raw)
To: Dark Penguin, Edward Kuns, linux-raid
In-Reply-To: <56A35703.2020702@yandex.ru>
On 01/23/2016 05:33 AM, Dark Penguin wrote:
>>> I wonder what happens if the only drive in a degraded mirror fails?..
>>
>> Or any drive in a raid5 hits an error while replacing a failed disk.
>>
>> BOOM.
>
> Oh... That sucks. I understand that BOOM would happen on a RAID-F, but
> kicking the last drive doesn't seem like a very rational thing to do -
> you would get a RAID with zero drives, which is not something anybody
> would want in any situation!.. I thought it would get into some kind of
> extreme panic mode so that you could at least save what you can...
It does. It's called manual intervention.
> Actually, it would make sense to avoid "BOOM" as much as possible;
> instead of going into a "BOOM" mode, it should go into some kind of
> "recovery" mode, when the array immediately turns read-only and allows
> you to salvage as much data as you can from the faulty drive. It would
> be especially useful in RAID5; only one faulty sector or even some other
> stupid error without even faulty sectors should not cause total data
> loss!.. I don't think it would be very hard to implement, or contradict
> some important philosophy, but it would save data, jobs and lives. :)
Well, the root causes vary, and the actions needed vary accordingly.
There isn't enough information to proceed without operator assistance.
This list has helped numerous people recover their data. ( But not
always. )
> Well, suppose "BOOM" happened on my RAID1, and both drives failed.
> Naturally, not at the same time, so data differs between the drives. So
> I have two faulty drives; is it possible (with some other recovery
> software?) to just mount each one read-only and copy data from it?.. I
> can live with having to copy as much data as I can from the first one
> and try to copy the rest from the older drive. I mean, I basically have
> the entire filesystem there! I should be able to just mount the
> filesystem on that drive, skipping the RAID header - just fdisk the
> drive, delete the RAID partition and create another one starting a few
> blocks later!.. Would that not work?.. That's actually why I chose
> RAID1; even if everything goes BOOM, I should be able to mount the
> drives separately!..
Yes, you can use dmsetup to gain access to any raid1's data area.
Mirrors using v0.90 and v1.0 metadata can be accessed directly. (v.90
is strongly discouraged, though, as there are unavoidable ambiguities in
that old format that can blow up on you.)
raid10 and parity raids cannot be accessed like this. You need
scrubbing and good drives, quick response to failures, and backups for
the final tiny probability of failure.
Phil
^ permalink raw reply
* Re: Wrong device name after hot-swap
From: Wols Lists @ 2016-01-23 14:45 UTC (permalink / raw)
To: Paul van der Vlis, linux-raid
In-Reply-To: <n7u8gr$1bv$1@ger.gmane.org>
On 22/01/16 21:55, Paul van der Vlis wrote:
> Hello,
>
> I want to put bigger disks into my server. What I want to do is replace
> the first disk, rebuild the raid, replace the second disk, rebuild the
> raid. The machine has two disks, sda and sdb.
Have you only got two slots?
>
> But, when I replace a disk, it gets a new device name. E.g. /dev/sdb
> becomes /dev/sdc. After a reboot it's good again, but I prefer not to
> reboot this machine!
Sounds like you're pulling a drive then replacing it. Can't you replace
it then pull?
That'll mean that when the array is at its most vulnerable, ie it's
rebuilding, you'll have three drives on line not one.
How new is your mdadm? You'll need one that supports the "--replace" syntax.
Cheers,
Wol
^ permalink raw reply
* Re: md failing mechanism
From: Wols Lists @ 2016-01-23 14:09 UTC (permalink / raw)
To: James J, linux-raid
In-Reply-To: <56A2BDF7.7020101@shiftmail.org>
On 22/01/16 23:40, James J wrote:
> The recommentation of raising the timeout to 120+ is for the opposite
> purpose of what you want. It is for the case the sysadmin accepts to
> wait a long time because he wants to prevent the kicking of the drive at
> the first read-error (normally drives are kicked for a write error).
> This might be wanted in order to a) defer the replacement of the drive,
> either to perform the replacement at a more opportune time and/or in a
> better manner such as a no-degrade replace operation, or b) because he
> does not want to replace the drive at all: maybe he believes that the
> error might be spurious and will not happen again and the drive is still
> of acceptable fitness for the purpose, e.g. in a low-cost file server.
Except, aiui, even in your scenario! drives are kicked for a *write* error.
What happens (should be) is the kernel times out, the raid handles the
read error by trying a rewrite, the drive is still hung on the read
error so it doesn't respond to the write request, and the drive gets
kicked for a write failure.
Cheers,
Wol
^ permalink raw reply
* Re: md failing mechanism
From: Dark Penguin @ 2016-01-23 10:33 UTC (permalink / raw)
To: Phil Turmel, Edward Kuns, linux-raid
In-Reply-To: <56A2CA93.8020806@turmel.org>
>> I wonder what happens if the only drive in a degraded mirror fails?..
>
> Or any drive in a raid5 hits an error while replacing a failed disk.
>
> BOOM.
Oh... That sucks. I understand that BOOM would happen on a RAID-F, but
kicking the last drive doesn't seem like a very rational thing to do -
you would get a RAID with zero drives, which is not something anybody
would want in any situation!.. I thought it would get into some kind of
extreme panic mode so that you could at least save what you can...
Actually, it would make sense to avoid "BOOM" as much as possible;
instead of going into a "BOOM" mode, it should go into some kind of
"recovery" mode, when the array immediately turns read-only and allows
you to salvage as much data as you can from the faulty drive. It would
be especially useful in RAID5; only one faulty sector or even some other
stupid error without even faulty sectors should not cause total data
loss!.. I don't think it would be very hard to implement, or contradict
some important philosophy, but it would save data, jobs and lives. :)
Well, suppose "BOOM" happened on my RAID1, and both drives failed.
Naturally, not at the same time, so data differs between the drives. So
I have two faulty drives; is it possible (with some other recovery
software?) to just mount each one read-only and copy data from it?.. I
can live with having to copy as much data as I can from the first one
and try to copy the rest from the older drive. I mean, I basically have
the entire filesystem there! I should be able to just mount the
filesystem on that drive, skipping the RAID header - just fdisk the
drive, delete the RAID partition and create another one starting a few
blocks later!.. Would that not work?.. That's actually why I chose
RAID1; even if everything goes BOOM, I should be able to mount the
drives separately!..
--
darkpenguin
^ permalink raw reply
* Re: Wrong device name after hot-swap
From: Phil Turmel @ 2016-01-23 0:51 UTC (permalink / raw)
To: Paul van der Vlis, linux-raid
In-Reply-To: <n7ud6o$b1f$1@ger.gmane.org>
On 01/22/2016 06:15 PM, Paul van der Vlis wrote:
> Op 22-01-16 om 23:27 schreef Phil Turmel:
>> MD stores signatures in the devices it uses that identify them for later
>> assembly. It does not depend on the device name, though it is recorded
>> in the superblock as a "last connected as" kind of indicator.
>
> So I can do it as described above?
Yes. Though I won't comment on grub part. (I've stopped using it and
am no long current.)
>> In general, you should not rely on device names in your system
>> configuration. UUIDs and filesystem labels were implemented
>> specifically to avoid this problem.
>
> So far I know I cannot configure anywhere a disk UUID or a filesytem
> label to a MD-device.
Modern udev creates /dev/disk/by-id/ and similar symlinks, but its not
important for md.
> It feels wrong when I add /dev/sdc1 to the raid, when the name is
> normally /dev/sdb1. But maybe it's no problem, because the device name
> is not impartant while booting from an MD-device.
It's not a problem. Unless you have specific member device names in
mdadm.conf. (Don't do that.)
Phil
^ permalink raw reply
* [PATCH 2/2] md-cluster: delete useless code
From: Shaohua Li @ 2016-01-23 0:45 UTC (permalink / raw)
To: linux-raid; +Cc: Goldwyn Rodrigues, Guoqing Jiang, NeilBrown
In-Reply-To: <0bc7fe7dece2c5aa4f8ed7194df27b02e0ae3142.1453509823.git.shli@fb.com>
page->index already considers node offset. The node_offset calculation
in write_sb_page is useless and confusion.
Cc: Goldwyn Rodrigues <rgoldwyn@suse.com>
Cc: Guoqing Jiang <gqjiang@suse.com>
Cc: NeilBrown <neilb@suse.com>
Signed-off-by: Shaohua Li <shli@fb.com>
---
drivers/md/bitmap.c | 4 ----
1 file changed, 4 deletions(-)
diff --git a/drivers/md/bitmap.c b/drivers/md/bitmap.c
index 4f22e91..d80cce4 100644
--- a/drivers/md/bitmap.c
+++ b/drivers/md/bitmap.c
@@ -210,10 +210,6 @@ static int write_sb_page(struct bitmap *bitmap, struct page *page, int wait)
struct block_device *bdev;
struct mddev *mddev = bitmap->mddev;
struct bitmap_storage *store = &bitmap->storage;
- int node_offset = 0;
-
- if (mddev_is_clustered(bitmap->mddev))
- node_offset = bitmap->cluster_slot * store->file_pages;
while ((rdev = next_active_rdev(rdev, mddev)) != NULL) {
int size = PAGE_SIZE;
--
2.4.6
^ permalink raw reply related
* [PATCH 1/2] md-cluster: fix missing memory free
From: Shaohua Li @ 2016-01-23 0:45 UTC (permalink / raw)
To: linux-raid; +Cc: Goldwyn Rodrigues, Guoqing Jiang, NeilBrown
There are several places we allocate dlm_lock_resource, but not free it.
Cc: Goldwyn Rodrigues <rgoldwyn@suse.com>
Cc: Guoqing Jiang <gqjiang@suse.com>
Cc: NeilBrown <neilb@suse.com>
Signed-off-by: Shaohua Li <shli@fb.com>
---
drivers/md/md-cluster.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/md/md-cluster.c b/drivers/md/md-cluster.c
index 0ded8e9..662369c 100644
--- a/drivers/md/md-cluster.c
+++ b/drivers/md/md-cluster.c
@@ -293,6 +293,7 @@ static void recover_bitmaps(struct md_thread *thread)
dlm_unlock:
dlm_unlock_sync(bm_lockres);
clear_bit:
+ lockres_free(bm_lockres);
clear_bit(slot, &cinfo->recovery_map);
}
}
@@ -682,8 +683,10 @@ static int gather_all_resync_info(struct mddev *mddev, int total_slots)
bm_lockres = lockres_init(mddev, str, NULL, 1);
if (!bm_lockres)
return -ENOMEM;
- if (i == (cinfo->slot_number - 1))
+ if (i == (cinfo->slot_number - 1)) {
+ lockres_free(bm_lockres);
continue;
+ }
bm_lockres->flags |= DLM_LKF_NOQUEUE;
ret = dlm_lock_sync(bm_lockres, DLM_LOCK_PW);
--
2.4.6
^ permalink raw reply related
* Re: md failing mechanism
From: Phil Turmel @ 2016-01-23 0:44 UTC (permalink / raw)
To: James J, Dark Penguin, linux-raid
In-Reply-To: <56A2BDF7.7020101@shiftmail.org>
On 01/22/2016 06:40 PM, James J wrote:
> On 22/01/2016 22:44, Dark Penguin wrote:
>>
>> As I understand, one way around this problem is to change the kernel
>> timeout to exceed the drive timeout by changing
>> /sys/block/sd?/device/timeout to something larger than the default 30,
>> but I'd have to do that after every reboot, is all that correct?
>>
>
> No, this part needs further investigation and comments from the gurus.
Yes, DP had that correct.
> With a SCSI timeout 30 secs, which is the setting you had at the time of
> the incident AFAIU, what should have happened was that the drive should
> have been kicked out at the 30th second, this is BEFORE it had a chance
> to return a read failure because your desktop drive takes more than
> 30secs to return a read failure. This was what you indeed expected but
> it is not what has happened.
His problem description doesn't perfectly match timeout mismatch. He
probably had a real problem that was exacerbated by his now-discovered
timeout problem. He no longer has the dmesg so further speculation is
moot. If it happens again, we can look closer.
> The recommentation of raising the timeout to 120+ is for the opposite
> purpose of what you want. It is for the case the sysadmin accepts to
> wait a long time because he wants to prevent the kicking of the drive at
> the first read-error (normally drives are kicked for a write error).
> This might be wanted in order to a) defer the replacement of the drive,
> either to perform the replacement at a more opportune time and/or in a
> better manner such as a no-degrade replace operation, or b) because he
> does not want to replace the drive at all: maybe he believes that the
> error might be spurious and will not happen again and the drive is still
> of acceptable fitness for the purpose, e.g. in a low-cost file server.
No. If you have a drive that doesn't support scterc or has it turned
off, you *must* set a timeout longer than the drive's native timeout or
you will have great problems. I suggest you read the references to the
archives I posted.
Keep in mind that in a properly working array UREs are *fixed* when
discovered by overwriting them. This is vital to array robustness, as
many UREs are transient (don't need relocation at all).
Phil
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox