Linux RAID subsystem development
 help / color / mirror / Atom feed
* Re: [PATCH/RFC/RFT] md: allow resync to go faster when there is competing IO.
From: Shaohua Li @ 2016-01-26 22:52 UTC (permalink / raw)
  To: NeilBrown; +Cc: Chien Lee, linux-raid, owner-linux-raid
In-Reply-To: <87si1k2do8.fsf@notabene.neil.brown.name>

On Wed, Jan 27, 2016 at 09:12:23AM +1100, Neil Brown wrote:
> On Tue, Jan 26 2016, Chien Lee wrote:
> 
> > Hello,
> >
> > Recently we find a bug about this patch (commit No. is
> > ac8fa4196d205ac8fff3f8932bddbad4f16e4110 ).
> >
> > We know that this patch committed after Linux kernel 4.1.x is intended
> > to allowing resync to go faster when there is competing IO. However,
> > we find the performance of random read on syncing Raid6 will come up
> > with a huge drop in this case. The following is our testing detail.
> >
> > The OS what we choose in our test is CentOS Linux release 7.1.1503
> > (Core) and the kernel image will be replaced for testing. In our
> > testing result, the 4K random read performance on syncing raid6 in
> > Kernel 4.2.8 is much lower than in Kernel 3.19.8. In order to find out
> > the root cause, we try to rollback this patch in Kernel 4.2.8, and we
> > find the 4K random read performance on syncing Raid6 will be improved
> > and go back to as what it should be in Kernel 3.19.8.
> >
> > Nevertheless, it seems that it will not affect some other read/write
> > patterns. In our testing result, the 1M sequential read/write, 4K
> > random write performance in Kernel 4.2.8 is performed almost the same
> > as in Kernel 3.19.8.
> >
> > It seems that although this patch increases the resync speed, the
> > logic of !is_mddev_idle() cause the sync request wait too short and
> > reduce the chance for raid5d to handle the random read I/O.
> 
> This has been raised before.
> Can you please try the patch at the end of 
> 
>   http://permalink.gmane.org/gmane.linux.raid/51002
> 
> and let me know if it makes any difference.  If it isn't sufficient I
> will explore further.

I'm curious why we don't calculate the wait time. Say the target resync speed
is speed_min. The wait time should be:

(currspeed * SYNC_MARK_STEP - speed_min * SYNC_MARK_STEP) / speed_min
= (currspeed / speed_min - 1) * SYNC_MARK_STEP

if SYNC_MARK_STEP is too big and sync speed has drift, we can make it smaller.

Thanks,
Shaohua

^ permalink raw reply

* Re: [PATCH/RFC/RFT] md: allow resync to go faster when there is competing IO.
From: NeilBrown @ 2016-01-26 22:12 UTC (permalink / raw)
  To: Chien Lee, linux-raid, shli, owner-linux-raid
In-Reply-To: <CAByoP04f6YgzhXn+55iEpg0SrVnK3mUxOuEfp-Hki_mppOBieg@mail.gmail.com>

[-- Attachment #1: Type: text/plain, Size: 3998 bytes --]

On Tue, Jan 26 2016, Chien Lee wrote:

> Hello,
>
> Recently we find a bug about this patch (commit No. is
> ac8fa4196d205ac8fff3f8932bddbad4f16e4110 ).
>
> We know that this patch committed after Linux kernel 4.1.x is intended
> to allowing resync to go faster when there is competing IO. However,
> we find the performance of random read on syncing Raid6 will come up
> with a huge drop in this case. The following is our testing detail.
>
> The OS what we choose in our test is CentOS Linux release 7.1.1503
> (Core) and the kernel image will be replaced for testing. In our
> testing result, the 4K random read performance on syncing raid6 in
> Kernel 4.2.8 is much lower than in Kernel 3.19.8. In order to find out
> the root cause, we try to rollback this patch in Kernel 4.2.8, and we
> find the 4K random read performance on syncing Raid6 will be improved
> and go back to as what it should be in Kernel 3.19.8.
>
> Nevertheless, it seems that it will not affect some other read/write
> patterns. In our testing result, the 1M sequential read/write, 4K
> random write performance in Kernel 4.2.8 is performed almost the same
> as in Kernel 3.19.8.
>
> It seems that although this patch increases the resync speed, the
> logic of !is_mddev_idle() cause the sync request wait too short and
> reduce the chance for raid5d to handle the random read I/O.

This has been raised before.
Can you please try the patch at the end of 

  http://permalink.gmane.org/gmane.linux.raid/51002

and let me know if it makes any difference.  If it isn't sufficient I
will explore further.

Thanks,
NeilBrown


>
>
> Following is our test environment and some testing results:
>
>
> OS: CentOS Linux release 7.1.1503 (Core)
>
> CPU: Intel(R) Xeon(R) CPU E3-1245 v3 @ 3.40GHz
>
> Processor number: 8
>
> Memory: 12GB
>
> fio command:
>
> 1.      (for numjobs=64):
>
> fio --filename=/dev/md2 --sync=0 --direct=0 --rw=randread --bs=4K
> --runtime=180 --size=50G --name=test-read --ioengine=libaio
> --numjobs=64 --iodepth=1 --group_reporting
>
> 2.      (for numjobs=1):
>
> fio --filename=/dev/md2 --sync=0 --direct=0 --rw=randread --bs=4K
> --runtime=180 --size=50G --name=test-read --ioengine=libaio
> --numjobs=1 --iodepth=1 --group_reporting
>
>
>
> Here are test results:
>
>
> Part I. SSD (4 x 240GB Intel SSD create Raid6(syncing))
>
>
> a.      4K Random Read, numjobs=64
>
>                                              Average Throughput    Average IOPS
>
> Kernel 3.19.8                                 715937KB/s              178984
>
> Kernel 4.2.8                                   489874KB/s              122462
>
> Kernel 4.2.8 Patch Rollback            717377KB/s              179344
>
>
>
> b.      4K Random Read, numjobs=1
>
>                                              Average Throughput    Average IOPS
>
> Kernel 3.19.8                                 32203KB/s                8051
>
> Kernel 4.2.8                                  2535.7KB/s                633
>
> Kernel 4.2.8 Patch Rollback            31861KB/s                7965
>
>
>
>
> Part II. HDD (4 x 1TB TOSHIBA HDD create Raid6(syncing))
>
>
> a.      4K Random Read, numjobs=64
>
>                                              Average Throughput    Average IOPS
>
> Kernel 3.19.8                                2976.6KB/s               744
>
> Kernel 4.2.8                                  2915.8KB/s               728
>
> Kernel 4.2.8 Patch Rollback           2973.3KB/s               743
>
>
>
> b.      4K Random Read, numjobs=1
>
>                                              Average Throughput    Average IOPS
>
> Kernel 3.19.8                                481844 B/s                 117
>
> Kernel 4.2.8                                   24718 B/s                   5
>
> Kernel 4.2.8 Patch Rollback           460090 B/s                 112
>
>
>
> Thanks,
>
> -- 
>
> Chien Lee

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

^ permalink raw reply

* Re: md127 auto created when use "-B" to build a legacy array without superblocks
From: NeilBrown @ 2016-01-26 22:03 UTC (permalink / raw)
  To: yizhan; +Cc: linux-raid
In-Reply-To: <56A72D21.20707@redhat.com>

[-- Attachment #1: Type: text/plain, Size: 4841 bytes --]

On Tue, Jan 26 2016, yizhan wrote:

> On 10/02/2015 03:55 PM, Neil Brown wrote:
>> Yi Zhang <yizhan@redhat.com> writes:
>>
>>> Hi Neil
>>>
>>> When testing 00raid1, found the md127 auto created when use "-B" to build a legacy array without superblocks, is it reasonable?
>> This happens because udev notices a new device has appeared, looks
>> inside it, sees that it could be part of an md array, and so runs
>> "mdadm --incremental" on it.
> Hi Neil
> I tried the mdadm --incremental, but it doesn't work, below is the 
> operation.
> # mdadm -D /dev/md0
> /dev/md0:
>          Version :
>    Creation Time : Tue Jan 26 03:16:17 2016
>       Raid Level : raid1
>       Array Size : 2097152 (2.00 GiB 2.15 GB)
>    Used Dev Size : 2097152 (2.00 GiB 2.15 GB)
>     Raid Devices : 2
>    Total Devices : 2
>
>            State : clean
>   Active Devices : 2
> Working Devices : 2
>   Failed Devices : 0
>    Spare Devices : 0
>
>      Number   Major   Minor   RaidDevice State
>         0       7        0        0      active sync   /dev/loop0
>         1       7        1        1      active sync   /dev/loop1
> # mdadm -D /dev/md127
> /dev/md127:
>          Version : 1.2
>       Raid Level : raid0
>    Total Devices : 1
>      Persistence : Superblock is persistent
>
>            State : inactive
>
>             Name : 0
>             UUID : 13ada118:129135ff:686de7b7:4cb71d6d
>           Events : 17
>
>      Number   Major   Minor   RaidDevice
>
>         -       9        0        -        /dev/md0
> # mdadm -I /dev/md127
> mdadm: /dev/md127 is not part of an md array.
> # mdadm -I /dev/md0
> mdadm: cannot reopen /dev/md0: Device or resource busy.
>
> After I stopped md127, the md0 can be used again.

If, after stopping md127, you try

  mdadm -I /dev/md0

again it will start /dev/md127 again.  This is what udev does which
causes md127 to appear in the first place.

NeilBrown


>
> Thanks
> Yi
>> Sometimes we want udev to do that.  Sometimes we don't.
>> There is no easy way for udev to know what we want.
>>
>> NeilBrown
>>
>>
>>> pls check below detailed info:
>>>
>>> + mdadm -CR /dev/md0 --level=raid1 -n3 /dev/loop0 /dev/loop1 /dev/loop2
>>> mdadm: /dev/loop0 appears to contain an ext2fs file system
>>>         size=58368K  mtime=Thu Jan  1 08:00:00 1970
>>> mdadm: Note: this array has metadata at the start and
>>>      may not be suitable as a boot device.  If you plan to
>>>      store '/boot' on this device please ensure that
>>>      your boot-loader understands md/v1.x metadata, or use
>>>      --metadata=0.90
>>> mdadm: /dev/loop1 appears to contain an ext2fs file system
>>>         size=38912K  mtime=Thu Jan  1 08:00:00 1970
>>> mdadm: Defaulting to version 1.2 metadata
>>> mdadm: array /dev/md0 started.
>>> + mdadm --wait /dev/md0
>>> + cat /proc/mdstat
>>> Personalities : [raid6] [raid5] [raid4] [raid1]
>>> md0 : active raid1 loop2[2] loop1[1] loop0[0]
>>>        19968 blocks super 1.2 [3/3] [UUU]
>>>        
>>> unused devices: <none>
>>> + mdadm -S /dev/md0
>>> mdadm: stopped /dev/md0
>>> + mdadm -B /dev/md0 -l raid1 -n2 /dev/loop0 /dev/loop1
>>> mdadm: array /dev/md0 built and started.
>>> + sleep 2
>>> + cat /proc/mdstat
>>> Personalities : [raid6] [raid5] [raid4] [raid1]
>>> md127 : inactive md0[0](S)
>>>        19968 blocks super 1.2
>>>         
>>> md0 : active raid1 loop1[1] loop0[0]
>>>        20000 blocks super non-persistent [2/2] [UU]
>>>        
>>> unused devices: <none>
>>> [root@dhcp-12-171 bug]# uname -r
>>> 4.2.0
>>> [root@dhcp-12-171 bug]# mdadm -D /dev/md0
>>> /dev/md0:
>>>          Version :
>>>    Creation Time : Mon Sep  7 20:21:20 2015
>>>       Raid Level : raid1
>>>       Array Size : 20000 (19.53 MiB 20.48 MB)
>>>    Used Dev Size : 20000 (19.53 MiB 20.48 MB)
>>>     Raid Devices : 2
>>>    Total Devices : 2
>>>
>>>            State : clean
>>>   Active Devices : 2
>>> Working Devices : 2
>>>   Failed Devices : 0
>>>    Spare Devices : 0
>>>
>>>      Number   Major   Minor   RaidDevice State
>>>         0       7        0        0      active sync   /dev/loop0
>>>         1       7        1        1      active sync   /dev/loop1
>>> [root@dhcp-12-171 bug]# mdadm -D /dev/md127
>>> /dev/md127:
>>>          Version : 1.2
>>>       Raid Level : raid0
>>>    Total Devices : 1
>>>      Persistence : Superblock is persistent
>>>
>>>            State : inactive
>>>
>>>             Name : dhcp-12-171.nay.redhat.com:0  (local to host dhcp-12-171.nay.redhat.com)
>>>             UUID : 40ace956:a9dd0793:f4984d2b:8431b92b
>>>           Events : 17
>>>
>>>      Number   Major   Minor   RaidDevice
>>>
>>>         -       9        0        -        /dev/md0
>>>
>>> Best Regards,
>>>    Yi Zhang

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

^ permalink raw reply

* Problem with array raid10 array resync on 4.4.0 (keeps reyncing each reboot)
From: Eric Valette @ 2016-01-26 14:49 UTC (permalink / raw)
  To: linux-raid

Hi,

My raid 10 array (5 disk with one spare) was doing a  resync after an 
upgrade to 4.4.0 from 4.1.15. The resync progress was steady and at the 
end the /proc/mdstat was apparently complete but when rebooting, it 
started resycing over and over. I noticed my dmesg was totally filled 
with raid10 conf printout message so it was impossible to trace anything 
else.

Did a resync test with 3.14.58 (because I knew it had worked for resync 
before and was still available as a boot option)  and the array was 
correctly rebuild.
Runs fine with 4.1.16 now.

Please CC me as I'm not subscribed.

-- eric


^ permalink raw reply

* Re: md127 auto created when use "-B" to build a legacy array without superblocks
From: yizhan @ 2016-01-26  8:24 UTC (permalink / raw)
  To: Neil Brown; +Cc: linux-raid
In-Reply-To: <87k2r5iu0t.fsf@notabene.neil.brown.name>



On 10/02/2015 03:55 PM, Neil Brown wrote:
> Yi Zhang <yizhan@redhat.com> writes:
>
>> Hi Neil
>>
>> When testing 00raid1, found the md127 auto created when use "-B" to build a legacy array without superblocks, is it reasonable?
> This happens because udev notices a new device has appeared, looks
> inside it, sees that it could be part of an md array, and so runs
> "mdadm --incremental" on it.
Hi Neil
I tried the mdadm --incremental, but it doesn't work, below is the 
operation.
# mdadm -D /dev/md0
/dev/md0:
         Version :
   Creation Time : Tue Jan 26 03:16:17 2016
      Raid Level : raid1
      Array Size : 2097152 (2.00 GiB 2.15 GB)
   Used Dev Size : 2097152 (2.00 GiB 2.15 GB)
    Raid Devices : 2
   Total Devices : 2

           State : clean
  Active Devices : 2
Working Devices : 2
  Failed Devices : 0
   Spare Devices : 0

     Number   Major   Minor   RaidDevice State
        0       7        0        0      active sync   /dev/loop0
        1       7        1        1      active sync   /dev/loop1
# mdadm -D /dev/md127
/dev/md127:
         Version : 1.2
      Raid Level : raid0
   Total Devices : 1
     Persistence : Superblock is persistent

           State : inactive

            Name : 0
            UUID : 13ada118:129135ff:686de7b7:4cb71d6d
          Events : 17

     Number   Major   Minor   RaidDevice

        -       9        0        -        /dev/md0
# mdadm -I /dev/md127
mdadm: /dev/md127 is not part of an md array.
# mdadm -I /dev/md0
mdadm: cannot reopen /dev/md0: Device or resource busy.

After I stopped md127, the md0 can be used again.

Thanks
Yi
> Sometimes we want udev to do that.  Sometimes we don't.
> There is no easy way for udev to know what we want.
>
> NeilBrown
>
>
>> pls check below detailed info:
>>
>> + mdadm -CR /dev/md0 --level=raid1 -n3 /dev/loop0 /dev/loop1 /dev/loop2
>> mdadm: /dev/loop0 appears to contain an ext2fs file system
>>         size=58368K  mtime=Thu Jan  1 08:00:00 1970
>> mdadm: Note: this array has metadata at the start and
>>      may not be suitable as a boot device.  If you plan to
>>      store '/boot' on this device please ensure that
>>      your boot-loader understands md/v1.x metadata, or use
>>      --metadata=0.90
>> mdadm: /dev/loop1 appears to contain an ext2fs file system
>>         size=38912K  mtime=Thu Jan  1 08:00:00 1970
>> mdadm: Defaulting to version 1.2 metadata
>> mdadm: array /dev/md0 started.
>> + mdadm --wait /dev/md0
>> + cat /proc/mdstat
>> Personalities : [raid6] [raid5] [raid4] [raid1]
>> md0 : active raid1 loop2[2] loop1[1] loop0[0]
>>        19968 blocks super 1.2 [3/3] [UUU]
>>        
>> unused devices: <none>
>> + mdadm -S /dev/md0
>> mdadm: stopped /dev/md0
>> + mdadm -B /dev/md0 -l raid1 -n2 /dev/loop0 /dev/loop1
>> mdadm: array /dev/md0 built and started.
>> + sleep 2
>> + cat /proc/mdstat
>> Personalities : [raid6] [raid5] [raid4] [raid1]
>> md127 : inactive md0[0](S)
>>        19968 blocks super 1.2
>>         
>> md0 : active raid1 loop1[1] loop0[0]
>>        20000 blocks super non-persistent [2/2] [UU]
>>        
>> unused devices: <none>
>> [root@dhcp-12-171 bug]# uname -r
>> 4.2.0
>> [root@dhcp-12-171 bug]# mdadm -D /dev/md0
>> /dev/md0:
>>          Version :
>>    Creation Time : Mon Sep  7 20:21:20 2015
>>       Raid Level : raid1
>>       Array Size : 20000 (19.53 MiB 20.48 MB)
>>    Used Dev Size : 20000 (19.53 MiB 20.48 MB)
>>     Raid Devices : 2
>>    Total Devices : 2
>>
>>            State : clean
>>   Active Devices : 2
>> Working Devices : 2
>>   Failed Devices : 0
>>    Spare Devices : 0
>>
>>      Number   Major   Minor   RaidDevice State
>>         0       7        0        0      active sync   /dev/loop0
>>         1       7        1        1      active sync   /dev/loop1
>> [root@dhcp-12-171 bug]# mdadm -D /dev/md127
>> /dev/md127:
>>          Version : 1.2
>>       Raid Level : raid0
>>    Total Devices : 1
>>      Persistence : Superblock is persistent
>>
>>            State : inactive
>>
>>             Name : dhcp-12-171.nay.redhat.com:0  (local to host dhcp-12-171.nay.redhat.com)
>>             UUID : 40ace956:a9dd0793:f4984d2b:8431b92b
>>           Events : 17
>>
>>      Number   Major   Minor   RaidDevice
>>
>>         -       9        0        -        /dev/md0
>>
>> Best Regards,
>>    Yi Zhang


^ permalink raw reply

* Re: [PATCH/RFC/RFT] md: allow resync to go faster when there is competing IO.
From: Chien Lee @ 2016-01-26  2:32 UTC (permalink / raw)
  To: linux-raid, neilb, shli, owner-linux-raid

Hello,

Recently we find a bug about this patch (commit No. is
ac8fa4196d205ac8fff3f8932bddbad4f16e4110 ).

We know that this patch committed after Linux kernel 4.1.x is intended
to allowing resync to go faster when there is competing IO. However,
we find the performance of random read on syncing Raid6 will come up
with a huge drop in this case. The following is our testing detail.

The OS what we choose in our test is CentOS Linux release 7.1.1503
(Core) and the kernel image will be replaced for testing. In our
testing result, the 4K random read performance on syncing raid6 in
Kernel 4.2.8 is much lower than in Kernel 3.19.8. In order to find out
the root cause, we try to rollback this patch in Kernel 4.2.8, and we
find the 4K random read performance on syncing Raid6 will be improved
and go back to as what it should be in Kernel 3.19.8.

Nevertheless, it seems that it will not affect some other read/write
patterns. In our testing result, the 1M sequential read/write, 4K
random write performance in Kernel 4.2.8 is performed almost the same
as in Kernel 3.19.8.

It seems that although this patch increases the resync speed, the
logic of !is_mddev_idle() cause the sync request wait too short and
reduce the chance for raid5d to handle the random read I/O.


Following is our test environment and some testing results:


OS: CentOS Linux release 7.1.1503 (Core)

CPU: Intel(R) Xeon(R) CPU E3-1245 v3 @ 3.40GHz

Processor number: 8

Memory: 12GB

fio command:

1.      (for numjobs=64):

fio --filename=/dev/md2 --sync=0 --direct=0 --rw=randread --bs=4K
--runtime=180 --size=50G --name=test-read --ioengine=libaio
--numjobs=64 --iodepth=1 --group_reporting

2.      (for numjobs=1):

fio --filename=/dev/md2 --sync=0 --direct=0 --rw=randread --bs=4K
--runtime=180 --size=50G --name=test-read --ioengine=libaio
--numjobs=1 --iodepth=1 --group_reporting



Here are test results:


Part I. SSD (4 x 240GB Intel SSD create Raid6(syncing))


a.      4K Random Read, numjobs=64

                                             Average Throughput    Average IOPS

Kernel 3.19.8                                 715937KB/s              178984

Kernel 4.2.8                                   489874KB/s              122462

Kernel 4.2.8 Patch Rollback            717377KB/s              179344



b.      4K Random Read, numjobs=1

                                             Average Throughput    Average IOPS

Kernel 3.19.8                                 32203KB/s                8051

Kernel 4.2.8                                  2535.7KB/s                633

Kernel 4.2.8 Patch Rollback            31861KB/s                7965




Part II. HDD (4 x 1TB TOSHIBA HDD create Raid6(syncing))


a.      4K Random Read, numjobs=64

                                             Average Throughput    Average IOPS

Kernel 3.19.8                                2976.6KB/s               744

Kernel 4.2.8                                  2915.8KB/s               728

Kernel 4.2.8 Patch Rollback           2973.3KB/s               743



b.      4K Random Read, numjobs=1

                                             Average Throughput    Average IOPS

Kernel 3.19.8                                481844 B/s                 117

Kernel 4.2.8                                   24718 B/s                   5

Kernel 4.2.8 Patch Rollback           460090 B/s                 112



Thanks,

-- 

Chien Lee

^ permalink raw reply

* Re: [PATCH RESEND v2 00/19] Support fuse mounts in user namespaces
From: Seth Forshee @ 2016-01-25 20:36 UTC (permalink / raw)
  To: Eric W. Biederman
  Cc: linux-bcache, dm-devel, linux-raid, linux-mtd, linux-fsdevel,
	fuse-devel, linux-security-module, selinux, Alexander Viro,
	Serge Hallyn, Richard Weinberger, Austin S Hemmelgarn,
	Miklos Szeredi, linux-kernel
In-Reply-To: <87mvrtqvhp.fsf@x220.int.ebiederm.org>

On Mon, Jan 25, 2016 at 02:01:22PM -0600, Eric W. Biederman wrote:
> Seth Forshee <seth.forshee@canonical.com> writes:
> 
> > On Mon, Jan 04, 2016 at 12:03:39PM -0600, Seth Forshee wrote:
> >> These patches implement support for mounting filesystems in user
> >> namespaces using fuse. They are based on the patches in the for-testing
> >> branch of
> >> git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace.git,
> >> but I've rebased them onto 4.4-rc3. I've pushed all of this to:
> >> 
> >>  git://git.kernel.org/pub/scm/linux/kernel/git/sforshee/linux.git fuse-userns
> >> 
> >> The patches are organized into three high-level groups.
> >> 
> >> Patches 1-6 are related to security, adding restrictions for
> >> unprivileged mounts and updating the LSMs as needed. Patches 1-2
> >> (checking inode permissions for block device mounts) may not be strictly
> >> necessary for fuseblk mounts since fuse doesn't do any IO on the block
> >> device in the kernel, but it still seems like a good idea to fail the
> >> mount if the user doesn't have the required permissions for the inode
> >> (though this is a bit misleading with fuse since the mounts are done via
> >> a suid-root helper).
> >> 
> >> Patches 7-14 update most of the vfs to translate ids correctly and deal
> >> with inodes which may have invalid user/group ids. I've omitted patches
> >> for anything not used by fuse - quota, fs freezing, some helper
> >> functions, etc. - but if these are wanted for the sake of completeness I
> >> can include them.
> >> 
> >> Patches 15-18 update fuse to deal with mounts from non-init pid and user
> >> namespaces and enable mounting from user namespaces.
> >> 
> >> Changes since v1:
> >>  - Drop patch for FIBMAP.
> >>  - Use current_in_userns in fuse_allow_current_process.
> >>  - Remove checks for uid/gid validity in fuse. Intead, ids from the
> >>    backing store which do not map into s_user_ns will result in invalid
> >>    ids in the vfs inode. Checks in the vfs will prevent unmappable ids
> >>    from being passed in from above.
> >>  - Update a couple of commit messages to provide more detail about
> >>    changes.
> >
> > Now that the merge window is over, I'm wondering whether it might be
> > possible to get some feedback on these patches this cycle?
> 
> Definitely.  Apologies for not giving you much feedback earlier.
> 
> I had been hoping this was the kind of thing I could just double check
> to be certain you weren't doing anything silly and just apply.  After my
> last round of looking at this I realized that for me to be comfortable
> with these patches I will have to give them very close scrutiny, and
> check every detail.
> 
> Unfortunatly last cycle I had failed to budget enough time to give these
> patches the close scrutiny they need.
> 
> From a high level I am still very much in favor of this approach and
> at least getting as far as safe unprivileged fuse mounts.
> 
> I have one or two little things to look at and then I hope to be going
> through your patches one by one in detail.

Great. Thanks Eric.

^ permalink raw reply

* Re: [PATCH RESEND v2 00/19] Support fuse mounts in user namespaces
From: Eric W. Biederman @ 2016-01-25 20:01 UTC (permalink / raw)
  To: Seth Forshee
  Cc: linux-bcache, dm-devel, linux-raid, linux-mtd, linux-fsdevel,
	fuse-devel, linux-security-module, selinux, Alexander Viro,
	Serge Hallyn, Richard Weinberger, Austin S Hemmelgarn,
	Miklos Szeredi, linux-kernel
In-Reply-To: <20160125194722.GA10638@ubuntu-hedt>

Seth Forshee <seth.forshee@canonical.com> writes:

> On Mon, Jan 04, 2016 at 12:03:39PM -0600, Seth Forshee wrote:
>> These patches implement support for mounting filesystems in user
>> namespaces using fuse. They are based on the patches in the for-testing
>> branch of
>> git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace.git,
>> but I've rebased them onto 4.4-rc3. I've pushed all of this to:
>> 
>>  git://git.kernel.org/pub/scm/linux/kernel/git/sforshee/linux.git fuse-userns
>> 
>> The patches are organized into three high-level groups.
>> 
>> Patches 1-6 are related to security, adding restrictions for
>> unprivileged mounts and updating the LSMs as needed. Patches 1-2
>> (checking inode permissions for block device mounts) may not be strictly
>> necessary for fuseblk mounts since fuse doesn't do any IO on the block
>> device in the kernel, but it still seems like a good idea to fail the
>> mount if the user doesn't have the required permissions for the inode
>> (though this is a bit misleading with fuse since the mounts are done via
>> a suid-root helper).
>> 
>> Patches 7-14 update most of the vfs to translate ids correctly and deal
>> with inodes which may have invalid user/group ids. I've omitted patches
>> for anything not used by fuse - quota, fs freezing, some helper
>> functions, etc. - but if these are wanted for the sake of completeness I
>> can include them.
>> 
>> Patches 15-18 update fuse to deal with mounts from non-init pid and user
>> namespaces and enable mounting from user namespaces.
>> 
>> Changes since v1:
>>  - Drop patch for FIBMAP.
>>  - Use current_in_userns in fuse_allow_current_process.
>>  - Remove checks for uid/gid validity in fuse. Intead, ids from the
>>    backing store which do not map into s_user_ns will result in invalid
>>    ids in the vfs inode. Checks in the vfs will prevent unmappable ids
>>    from being passed in from above.
>>  - Update a couple of commit messages to provide more detail about
>>    changes.
>
> Now that the merge window is over, I'm wondering whether it might be
> possible to get some feedback on these patches this cycle?

Definitely.  Apologies for not giving you much feedback earlier.

I had been hoping this was the kind of thing I could just double check
to be certain you weren't doing anything silly and just apply.  After my
last round of looking at this I realized that for me to be comfortable
with these patches I will have to give them very close scrutiny, and
check every detail.

Unfortunatly last cycle I had failed to budget enough time to give these
patches the close scrutiny they need.

From a high level I am still very much in favor of this approach and
at least getting as far as safe unprivileged fuse mounts.

I have one or two little things to look at and then I hope to be going
through your patches one by one in detail.

Eric


^ permalink raw reply

* Re: [PATCH RESEND v2 00/19] Support fuse mounts in user namespaces
From: Seth Forshee @ 2016-01-25 19:47 UTC (permalink / raw)
  To: Eric W. Biederman, linux-bcache, dm-devel, linux-raid, linux-mtd,
	linux-fsdevel, fuse-devel, linux-security-module, selinux
  Cc: Alexander Viro, Serge Hallyn, Richard Weinberger,
	Austin S Hemmelgarn, Miklos Szeredi, linux-kernel
In-Reply-To: <1451930639-94331-1-git-send-email-seth.forshee@canonical.com>

On Mon, Jan 04, 2016 at 12:03:39PM -0600, Seth Forshee wrote:
> These patches implement support for mounting filesystems in user
> namespaces using fuse. They are based on the patches in the for-testing
> branch of
> git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace.git,
> but I've rebased them onto 4.4-rc3. I've pushed all of this to:
> 
>  git://git.kernel.org/pub/scm/linux/kernel/git/sforshee/linux.git fuse-userns
> 
> The patches are organized into three high-level groups.
> 
> Patches 1-6 are related to security, adding restrictions for
> unprivileged mounts and updating the LSMs as needed. Patches 1-2
> (checking inode permissions for block device mounts) may not be strictly
> necessary for fuseblk mounts since fuse doesn't do any IO on the block
> device in the kernel, but it still seems like a good idea to fail the
> mount if the user doesn't have the required permissions for the inode
> (though this is a bit misleading with fuse since the mounts are done via
> a suid-root helper).
> 
> Patches 7-14 update most of the vfs to translate ids correctly and deal
> with inodes which may have invalid user/group ids. I've omitted patches
> for anything not used by fuse - quota, fs freezing, some helper
> functions, etc. - but if these are wanted for the sake of completeness I
> can include them.
> 
> Patches 15-18 update fuse to deal with mounts from non-init pid and user
> namespaces and enable mounting from user namespaces.
> 
> Changes since v1:
>  - Drop patch for FIBMAP.
>  - Use current_in_userns in fuse_allow_current_process.
>  - Remove checks for uid/gid validity in fuse. Intead, ids from the
>    backing store which do not map into s_user_ns will result in invalid
>    ids in the vfs inode. Checks in the vfs will prevent unmappable ids
>    from being passed in from above.
>  - Update a couple of commit messages to provide more detail about
>    changes.

Now that the merge window is over, I'm wondering whether it might be
possible to get some feedback on these patches this cycle?

Thanks,
Seth

^ permalink raw reply

* Re: Uneven wear on raid1 devices
From: Phil Turmel @ 2016-01-25 18:00 UTC (permalink / raw)
  To: Roman Mamedov, Adam Goryachev; +Cc: linux-raid
In-Reply-To: <20160125104027.597041a3@natsu>

On 01/25/2016 12:40 AM, Roman Mamedov wrote:
> On Mon, 25 Jan 2016 16:29:02 +1100
> Adam Goryachev <adam@websitemanagers.com.au> wrote:
> 
>>    9 Power_On_Hours          -O--CK   098   098   000    -    6435
> 
>>    9 Power_On_Hours          -O--CK   095   095   000    -    23178
> 
> 2nd drive has almost 4x as much power-on time than the first one. My guess
> would be that it accumulated all that write usage back before you put it into
> this RAID1.

Or you are doing "repair" scrubs when you should be doing "check"
scrubs.  Any operation that requires resynchronization will read from
the first mirror and write to the others.

> If you want to ensure the RAID1 usage is even, record the SMART data you have
> now, and compare to the readings you will have a month later.

By the way, your devices show scterc disabled.  That's bad.  You should
definitely use boot scripts or udev rules to enable it.

Has sdd been getting bumped out of the array lately, and resyncing when
you put it back?

Phil


^ permalink raw reply

* Re: broken raid level 5 array caused by user error
From: Mathias Mueller @ 2016-01-25 10:02 UTC (permalink / raw)
  To: Phil Turmel; +Cc: Linux raid, linux-raid-owner
In-Reply-To: <56A27F5E.1000602@turmel.org>

Hi Phil,

thanks a lot once more, I would never managed it without your help. It's 
very nice, that you spend so much time in supporting everyone and 
writing very helpful tools. I learned a lot, technically and also to 
have more patience and thinking twice _before_ doing things :)

> Sadly, it's hard to ship beer here in the US without government 
> involvement.

That's a very sad fact! I send you something via paypal, so you might 
get some german beer from some US store, if you like. Tell me, if you 
need consulting ;)

Keep on going and best regards

Mathias

^ permalink raw reply

* Re: Uneven wear on raid1 devices
From: Roman Mamedov @ 2016-01-25  5:40 UTC (permalink / raw)
  To: Adam Goryachev; +Cc: linux-raid
In-Reply-To: <56A5B29E.3020708@websitemanagers.com.au>

[-- Attachment #1: Type: text/plain, Size: 577 bytes --]

On Mon, 25 Jan 2016 16:29:02 +1100
Adam Goryachev <adam@websitemanagers.com.au> wrote:

>    9 Power_On_Hours          -O--CK   098   098   000    -    6435

>    9 Power_On_Hours          -O--CK   095   095   000    -    23178

2nd drive has almost 4x as much power-on time than the first one. My guess
would be that it accumulated all that write usage back before you put it into
this RAID1.

If you want to ensure the RAID1 usage is even, record the SMART data you have
now, and compare to the readings you will have a month later.

-- 
With respect,
Roman

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

^ permalink raw reply

* Uneven wear on raid1 devices
From: Adam Goryachev @ 2016-01-25  5:29 UTC (permalink / raw)
  To: linux-raid

Hi all,

I was hoping to get some advice on why my two SSD's seem to be wearing 
unevenly, when they are both in the same raid1 array.

First, the array information:
# mdadm --misc --detail /dev/md0
/dev/md0:
         Version : 1.2
   Creation Time : Fri Jun 14 00:42:04 2013
      Raid Level : raid1
      Array Size : 499975360 (476.81 GiB 511.97 GB)
   Used Dev Size : 499975360 (476.81 GiB 511.97 GB)
    Raid Devices : 2
   Total Devices : 2
     Persistence : Superblock is persistent

   Intent Bitmap : Internal

     Update Time : Mon Jan 25 16:20:11 2016
           State : active
  Active Devices : 2
Working Devices : 2
  Failed Devices : 0
   Spare Devices : 0

            Name : flail:0  (local to host flail)
            UUID : 81b25e43:787b0bc4:f934d006:a726aa89
          Events : 1240606

     Number   Major   Minor   RaidDevice State
        2       8       17        0      active sync   /dev/sdb1
        1       8       49        1      active sync   /dev/sdd1

# smartctl -A /dev/sdb|grep Wear
177 Wear_Leveling_Count     0x0013   093   093   000    Pre-fail 
Always       -       248
# smartctl -A /dev/sdd|grep Wear
177 Wear_Leveling_Count     0x0013   074   074   000    Pre-fail 
Always       -       915

Equally, we can see a different amount of data written to each drive:
# smartctl -A /dev/sdb|grep Written
241 Total_LBAs_Written      0x0032   099   099   000    Old_age 
Always       -       7169916114
# smartctl -A /dev/sdd|grep Written
241 Total_LBAs_Written      0x0032   099   099   000    Old_age 
Always       -       24013780683

I'm stuck wondering why one drive is getting so many more writes than 
the other, and is there some way to "swap" this so that the other drive 
will get the heavier load over the next 12 months?

In theory, this suggests that one drive has 74% life remaining, and the 
other has 93% (unless I'm reading that incorrectly...), while 74% is 
still a lot, I guess I would prefer to prevent it getting too much lower 
and needing replacement....

Any suggestions or advice greatly appreciated.

Full output of smartctl -x below for each drive:

# smartctl -x /dev/sdb
smartctl 5.41 2011-06-09 r3365 [i686-linux-3.2.0-4-686-pae] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF INFORMATION SECTION ===
Device Model:     Samsung SSD 840 PRO Series
Serial Number:    S1AXNSAF702689E
LU WWN Device Id: 5 002538 5a02701b8
Firmware Version: DXM06B0Q
User Capacity:    512,110,190,592 bytes [512 GB]
Sector Size:      512 bytes logical/physical
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   8
ATA Standard is:  ATA-8-ACS revision 4c
Local Time is:    Mon Jan 25 16:28:01 2016 AEDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00)    Offline data collection activity
                     was never started.
                     Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0)    The previous self-test 
routine completed
                     without error or no self-test has ever
                     been run.
Total time to complete Offline
data collection:         (53956) seconds.
Offline data collection
capabilities:              (0x53) SMART execute Offline immediate.
                     Auto Offline data collection on/off support.
                     Suspend Offline collection upon new
                     command.
                     No Offline surface scan supported.
                     Self-test supported.
                     No Conveyance Self-test supported.
                     Selective Self-test supported.
SMART capabilities:            (0x0003)    Saves SMART data before entering
                     power-saving mode.
                     Supports SMART auto save timer.
Error logging capability:        (0x01)    Error logging supported.
                     General Purpose Logging supported.
Short self-test routine
recommended polling time:      (   2) minutes.
Extended self-test routine
recommended polling time:      (  35) minutes.
SCT capabilities:            (0x003d)    SCT Status supported.
                     SCT Error Recovery Control supported.
                     SCT Feature Control supported.
                     SCT Data Table supported.

SMART Attributes Data Structure revision number: 1
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
   5 Reallocated_Sector_Ct   PO--CK   100   100   010    -    0
   9 Power_On_Hours          -O--CK   098   098   000    -    6435
  12 Power_Cycle_Count       -O--CK   099   099   000    -    1
177 Wear_Leveling_Count     PO--C-   093   093   000    -    248
179 Used_Rsvd_Blk_Cnt_Tot   PO--C-   100   100   010    -    0
181 Program_Fail_Cnt_Total  -O--CK   100   100   010    -    0
182 Erase_Fail_Count_Total  -O--CK   100   100   010    -    0
183 Runtime_Bad_Block       PO--C-   100   100   010    -    0
187 Reported_Uncorrect      -O--CK   100   100   000    -    0
190 Airflow_Temperature_Cel -O--CK   072   060   000    -    28
195 Hardware_ECC_Recovered  -O-RC-   200   200   000    -    0
199 UDMA_CRC_Error_Count    -OSRCK   100   100   000    -    0
235 Unknown_Attribute       -O--C-   100   100   000    -    0
241 Total_LBAs_Written      -O--CK   099   099   000    - 7169972542
                             ||||||_ K auto-keep
                             |||||__ C event count
                             ||||___ R error rate
                             |||____ S speed/performance
                             ||_____ O updated online
                             |______ P prefailure warning

General Purpose Log Directory Version 1
SMART           Log Directory Version 1 [multi-sector log support]
GP/S  Log at address 0x00 has    1 sectors [Log Directory]
GP/S  Log at address 0x01 has    1 sectors [Summary SMART error log]
GP/S  Log at address 0x02 has    1 sectors [Comprehensive SMART error log]
GP/S  Log at address 0x03 has    1 sectors [Ext. Comprehensive SMART 
error log]
GP/S  Log at address 0x06 has    1 sectors [SMART self-test log]
GP/S  Log at address 0x07 has    1 sectors [Extended self-test log]
GP/S  Log at address 0x09 has    1 sectors [Selective self-test log]
GP/S  Log at address 0x10 has    1 sectors [NCQ Command Error]
GP/S  Log at address 0x11 has    1 sectors [SATA Phy Event Counters]
GP/S  Log at address 0x30 has    1 sectors [Reserved]
GP/S  Log at address 0x80 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x81 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x82 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x83 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x84 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x85 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x86 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x87 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x88 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x89 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8a has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8b has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8c has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8d has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8e has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8f has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x90 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x91 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x92 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x93 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x94 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x95 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x96 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x97 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x98 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x99 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9a has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9b has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9c has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9d has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9e has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9f has   16 sectors [Host vendor specific log]
GP/S  Log at address 0xa0 has   16 sectors [Device vendor specific log]

SMART Extended Comprehensive Error Log Version: 1 (1 sectors)
No Errors Logged

SMART Extended Self-test Log Version: 1 (1 sectors)
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
  SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
     1        0        0  Not_testing
     2        0        0  Not_testing
     3        0        0  Not_testing
     4        0        0  Not_testing
     5        0        0  Not_testing
   255        0    65535  Read_scanning was never started
Selective self-test flags (0x0):
   After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

Error unknown SCT Temperature History Format Version (3), should be 2.
SCT Error Recovery Control:
            Read: Disabled
           Write: Disabled

SATA Phy Event Counters (GP Log 0x11)
ID      Size     Value  Description
0x0001  2            0  Command failed due to ICRC error
0x0002  2            0  R_ERR response for data FIS
0x0003  2            0  R_ERR response for device-to-host data FIS
0x0004  2            0  R_ERR response for host-to-device data FIS
0x0005  2            0  R_ERR response for non-data FIS
0x0006  2            0  R_ERR response for device-to-host non-data FIS
0x0007  2            0  R_ERR response for host-to-device non-data FIS
0x0008  2            0  Device-to-host non-data FIS retries
0x0009  2           11  Transition from drive PhyRdy to drive PhyNRdy
0x000a  2           11  Device-to-host register FISes sent due to a COMRESET
0x000b  2            0  CRC errors within host-to-device FIS
0x000d  2            0  Non-CRC errors within host-to-device FIS
0x000f  2            0  R_ERR response for host-to-device data FIS, CRC
0x0010  2            0  R_ERR response for host-to-device data FIS, non-CRC
0x0012  2            0  R_ERR response for host-to-device non-data FIS, CRC
0x0013  2            0  R_ERR response for host-to-device non-data FIS, 
non-CRC



# smartctl -x /dev/sdd
smartctl 5.41 2011-06-09 r3365 [i686-linux-3.2.0-4-686-pae] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF INFORMATION SECTION ===
Device Model:     Samsung SSD 840 PRO Series
Serial Number:    S12SNEAD204208R
LU WWN Device Id: 5 002538 5501d20d0
Firmware Version: DXM04B0Q
User Capacity:    512,110,190,592 bytes [512 GB]
Sector Size:      512 bytes logical/physical
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   8
ATA Standard is:  ATA-8-ACS revision 4c
Local Time is:    Mon Jan 25 16:26:24 2016 AEDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00)    Offline data collection activity
                     was never started.
                     Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0)    The previous self-test 
routine completed
                     without error or no self-test has ever
                     been run.
Total time to complete Offline
data collection:         (53956) seconds.
Offline data collection
capabilities:              (0x53) SMART execute Offline immediate.
                     Auto Offline data collection on/off support.
                     Suspend Offline collection upon new
                     command.
                     No Offline surface scan supported.
                     Self-test supported.
                     No Conveyance Self-test supported.
                     Selective Self-test supported.
SMART capabilities:            (0x0003)    Saves SMART data before entering
                     power-saving mode.
                     Supports SMART auto save timer.
Error logging capability:        (0x01)    Error logging supported.
                     General Purpose Logging supported.
Short self-test routine
recommended polling time:      (   2) minutes.
Extended self-test routine
recommended polling time:      (  35) minutes.
SCT capabilities:            (0x003d)    SCT Status supported.
                     SCT Error Recovery Control supported.
                     SCT Feature Control supported.
                     SCT Data Table supported.

SMART Attributes Data Structure revision number: 1
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
   5 Reallocated_Sector_Ct   PO--CK   100   100   010    -    0
   9 Power_On_Hours          -O--CK   095   095   000    -    23178
  12 Power_Cycle_Count       -O--CK   099   099   000    -    8
177 Wear_Leveling_Count     PO--C-   074   074   000    -    915
179 Used_Rsvd_Blk_Cnt_Tot   PO--C-   100   100   010    -    0
181 Program_Fail_Cnt_Total  -O--CK   100   100   010    -    0
182 Erase_Fail_Count_Total  -O--CK   100   100   010    -    0
183 Runtime_Bad_Block       PO--C-   100   100   010    -    0
187 Reported_Uncorrect      -O--CK   100   100   000    -    0
190 Airflow_Temperature_Cel -O--CK   073   058   000    -    27
195 Hardware_ECC_Recovered  -O-RC-   200   200   000    -    0
199 UDMA_CRC_Error_Count    -OSRCK   100   100   000    -    0
235 Unknown_Attribute       -O--C-   099   099   000    -    6
241 Total_LBAs_Written      -O--CK   099   099   000    - 24013821507
                             ||||||_ K auto-keep
                             |||||__ C event count
                             ||||___ R error rate
                             |||____ S speed/performance
                             ||_____ O updated online
                             |______ P prefailure warning

General Purpose Log Directory Version 1
SMART           Log Directory Version 1 [multi-sector log support]
GP/S  Log at address 0x00 has    1 sectors [Log Directory]
GP/S  Log at address 0x01 has    1 sectors [Summary SMART error log]
GP/S  Log at address 0x02 has    1 sectors [Comprehensive SMART error log]
GP/S  Log at address 0x03 has    1 sectors [Ext. Comprehensive SMART 
error log]
GP/S  Log at address 0x06 has    1 sectors [SMART self-test log]
GP/S  Log at address 0x07 has    1 sectors [Extended self-test log]
GP/S  Log at address 0x09 has    1 sectors [Selective self-test log]
GP/S  Log at address 0x10 has    1 sectors [NCQ Command Error]
GP/S  Log at address 0x11 has    1 sectors [SATA Phy Event Counters]
GP/S  Log at address 0x30 has    1 sectors [Reserved]
GP/S  Log at address 0x80 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x81 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x82 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x83 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x84 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x85 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x86 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x87 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x88 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x89 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8a has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8b has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8c has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8d has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8e has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x8f has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x90 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x91 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x92 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x93 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x94 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x95 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x96 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x97 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x98 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x99 has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9a has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9b has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9c has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9d has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9e has   16 sectors [Host vendor specific log]
GP/S  Log at address 0x9f has   16 sectors [Host vendor specific log]
GP/S  Log at address 0xa0 has   16 sectors [Device vendor specific log]

SMART Extended Comprehensive Error Log Version: 1 (1 sectors)
No Errors Logged

SMART Extended Self-test Log Version: 1 (1 sectors)
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
  SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
     1        0        0  Not_testing
     2        0        0  Not_testing
     3        0        0  Not_testing
     4        0        0  Not_testing
     5        0        0  Not_testing
   255        0    65535  Read_scanning was never started
Selective self-test flags (0x0):
   After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

Error unknown SCT Temperature History Format Version (3), should be 2.
SCT Error Recovery Control:
            Read: Disabled
           Write: Disabled

SATA Phy Event Counters (GP Log 0x11)
ID      Size     Value  Description
0x0001  2            0  Command failed due to ICRC error
0x0002  2            0  R_ERR response for data FIS
0x0003  2            0  R_ERR response for device-to-host data FIS
0x0004  2            0  R_ERR response for host-to-device data FIS
0x0005  2            0  R_ERR response for non-data FIS
0x0006  2            0  R_ERR response for device-to-host non-data FIS
0x0007  2            0  R_ERR response for host-to-device non-data FIS
0x0008  2            0  Device-to-host non-data FIS retries
0x0009  2           11  Transition from drive PhyRdy to drive PhyNRdy
0x000a  2           11  Device-to-host register FISes sent due to a COMRESET
0x000b  2            0  CRC errors within host-to-device FIS
0x000d  2            0  Non-CRC errors within host-to-device FIS
0x000f  2            0  R_ERR response for host-to-device data FIS, CRC
0x0010  2            0  R_ERR response for host-to-device data FIS, non-CRC
0x0012  2            0  R_ERR response for host-to-device non-data FIS, CRC
0x0013  2            0  R_ERR response for host-to-device non-data FIS, 
non-CRC





^ permalink raw reply

* Re: [PATCH 2/2] md-cluster: delete useless code
From: Guoqing Jiang @ 2016-01-25  1:40 UTC (permalink / raw)
  To: Shaohua Li, linux-raid; +Cc: Goldwyn Rodrigues, NeilBrown
In-Reply-To: <688b654fe7566f182d6f4fdae2c3a3242d32a179.1453509823.git.shli@fb.com>



On 01/23/2016 08:45 AM, Shaohua Li wrote:
> page->index already considers node offset. The node_offset calculation
> in write_sb_page is useless and confusion.

Acked-by: Guoqing Jiang <gqjiang@suse.com>

Thanks,
Guoqing
> Cc: Goldwyn Rodrigues <rgoldwyn@suse.com>
> Cc: Guoqing Jiang <gqjiang@suse.com>
> Cc: NeilBrown <neilb@suse.com>
> Signed-off-by: Shaohua Li <shli@fb.com>
> ---
>   drivers/md/bitmap.c | 4 ----
>   1 file changed, 4 deletions(-)
>
> diff --git a/drivers/md/bitmap.c b/drivers/md/bitmap.c
> index 4f22e91..d80cce4 100644
> --- a/drivers/md/bitmap.c
> +++ b/drivers/md/bitmap.c
> @@ -210,10 +210,6 @@ static int write_sb_page(struct bitmap *bitmap, struct page *page, int wait)
>   	struct block_device *bdev;
>   	struct mddev *mddev = bitmap->mddev;
>   	struct bitmap_storage *store = &bitmap->storage;
> -	int node_offset = 0;
> -
> -	if (mddev_is_clustered(bitmap->mddev))
> -		node_offset = bitmap->cluster_slot * store->file_pages;
>   
>   	while ((rdev = next_active_rdev(rdev, mddev)) != NULL) {
>   		int size = PAGE_SIZE;


^ permalink raw reply

* Re: [PATCH 1/2] md-cluster: fix missing memory free
From: Guoqing Jiang @ 2016-01-25  1:33 UTC (permalink / raw)
  To: Shaohua Li, linux-raid; +Cc: Goldwyn Rodrigues, NeilBrown
In-Reply-To: <0bc7fe7dece2c5aa4f8ed7194df27b02e0ae3142.1453509823.git.shli@fb.com>



On 01/23/2016 08:45 AM, Shaohua Li wrote:
> There are several places we allocate dlm_lock_resource, but not free it.

Culd you also add below change? Thanks.

diff --git a/drivers/md/md-cluster.c b/drivers/md/md-cluster.c
index 0ded8e9..6c756e2 100644
--- a/drivers/md/md-cluster.c
+++ b/drivers/md/md-cluster.c
@@ -858,6 +858,7 @@ static int leave(struct mddev *mddev)
         lockres_free(cinfo->token_lockres);
         lockres_free(cinfo->ack_lockres);
         lockres_free(cinfo->no_new_dev_lockres);
+       lockres_free(cinfo->resync_lockres);
         lockres_free(cinfo->bitmap_lockres);
         unlock_all_bitmaps(mddev);
         dlm_release_lockspace(cinfo->lockspace, 2);

Guoqing

>
> Cc: Goldwyn Rodrigues <rgoldwyn@suse.com>
> Cc: Guoqing Jiang <gqjiang@suse.com>
> Cc: NeilBrown <neilb@suse.com>
> Signed-off-by: Shaohua Li <shli@fb.com>
> ---
>   drivers/md/md-cluster.c | 5 ++++-
>   1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/md/md-cluster.c b/drivers/md/md-cluster.c
> index 0ded8e9..662369c 100644
> --- a/drivers/md/md-cluster.c
> +++ b/drivers/md/md-cluster.c
> @@ -293,6 +293,7 @@ static void recover_bitmaps(struct md_thread *thread)
>   dlm_unlock:
>   		dlm_unlock_sync(bm_lockres);
>   clear_bit:
> +		lockres_free(bm_lockres);
>   		clear_bit(slot, &cinfo->recovery_map);
>   	}
>   }
> @@ -682,8 +683,10 @@ static int gather_all_resync_info(struct mddev *mddev, int total_slots)
>   		bm_lockres = lockres_init(mddev, str, NULL, 1);
>   		if (!bm_lockres)
>   			return -ENOMEM;
> -		if (i == (cinfo->slot_number - 1))
> +		if (i == (cinfo->slot_number - 1)) {
> +			lockres_free(bm_lockres);
>   			continue;
> +		}
>   
>   		bm_lockres->flags |= DLM_LKF_NOQUEUE;
>   		ret = dlm_lock_sync(bm_lockres, DLM_LOCK_PW);


^ permalink raw reply related

* Re: md failing mechanism
From: Adam Goryachev @ 2016-01-24 22:13 UTC (permalink / raw)
  To: James J, Wols Lists, linux-raid
In-Reply-To: <56A3CE28.9090901@shiftmail.org>



On 24/01/2016 06:02, James J wrote:
> On 23/01/2016 15:09, Wols Lists wrote:
>> On 22/01/16 23:40, James J wrote:
>>> The recommentation of raising the timeout to 120+ is for the opposite
>>> purpose of what you want. It is for the case the sysadmin accepts to
>>> wait a long time because he wants to prevent the kicking of the 
>>> drive at
>>> the first read-error (normally drives are kicked for a write error).
>>> This might be wanted in order to a) defer the replacement of the drive,
>>> either to perform the replacement at a more opportune time and/or in a
>>> better manner such as a no-degrade replace operation, or b) because he
>>> does not want to replace the drive at all: maybe he believes that the
>>> error might be spurious and will not happen again and the drive is 
>>> still
>>> of acceptable fitness for the purpose, e.g. in a low-cost file server.
>> Except, aiui, even in your scenario! drives are kicked for a *write* 
>> error.
>>
>> What happens (should be) is the kernel times out, the raid handles the
>> read error by trying a rewrite, the drive is still hung on the read
>> error so it doesn't respond to the write request, and the drive gets
>> kicked for a write failure.
>
> Oh yes you are correct, so the drive would be kicked after 60secs and 
> not after 30secs contrary to what I said.
> So the sequence would be: drive stuck on read --> scsi read failure 
> due to timeout at the 30th second --> MD receives failure and attempts 
> rewrite --> scsi write failure due to timeout at the 60th second --> 
> drive kicked by MD at the 60th second
> I think this is what should have happened, but it didn't happen like 
> this anyway so I think there is probably a kernel bug somewhere.
I don't have a lot to add, except that I recall the OP suggested it was 
an IDE drive. I wonder if the IDE sub-system and/or hardware operates 
differently compared to the sata variants. Possibly the MD layer never 
got any timeout or error on the read, and (or maybe it was the write) 
and hence it was never kicked from the array.

Regards,
Adam

^ permalink raw reply

* Re: md failing mechanism
From: James J @ 2016-01-23 19:02 UTC (permalink / raw)
  To: Wols Lists, linux-raid
In-Reply-To: <56A389A9.1080203@youngman.org.uk>

On 23/01/2016 15:09, Wols Lists wrote:
> On 22/01/16 23:40, James J wrote:
>> The recommentation of raising the timeout to 120+ is for the opposite
>> purpose of what you want. It is for the case the sysadmin accepts to
>> wait a long time because he wants to prevent the kicking of the drive at
>> the first read-error (normally drives are kicked for a write error).
>> This might be wanted in order to a) defer the replacement of the drive,
>> either to perform the replacement at a more opportune time and/or in a
>> better manner such as a no-degrade replace operation, or b) because he
>> does not want to replace the drive at all: maybe he believes that the
>> error might be spurious and will not happen again and the drive is still
>> of acceptable fitness for the purpose, e.g. in a low-cost file server.
> Except, aiui, even in your scenario! drives are kicked for a *write* error.
>
> What happens (should be) is the kernel times out, the raid handles the
> read error by trying a rewrite, the drive is still hung on the read
> error so it doesn't respond to the write request, and the drive gets
> kicked for a write failure.

Oh yes you are correct, so the drive would be kicked after 60secs and 
not after 30secs contrary to what I said.
So the sequence would be: drive stuck on read --> scsi read failure due 
to timeout at the 30th second --> MD receives failure and attempts 
rewrite --> scsi write failure due to timeout at the 60th second --> 
drive kicked by MD at the 60th second
I think this is what should have happened, but it didn't happen like 
this anyway so I think there is probably a kernel bug somewhere.


^ permalink raw reply

* Re: md failing mechanism
From: Phil Turmel @ 2016-01-23 15:12 UTC (permalink / raw)
  To: Dark Penguin, Edward Kuns, linux-raid
In-Reply-To: <56A35703.2020702@yandex.ru>

On 01/23/2016 05:33 AM, Dark Penguin wrote:
>>> I wonder what happens if the only drive in a degraded mirror fails?..
>>
>> Or any drive in a raid5 hits an error while replacing a failed disk.
>>
>> BOOM.
> 
> Oh... That sucks. I understand that BOOM would happen on a RAID-F, but
> kicking the last drive doesn't seem like a very rational thing to do -
> you would get a RAID with zero drives, which is not something anybody
> would want in any situation!.. I thought it would get into some kind of
> extreme panic mode so that you could at least save what you can...

It does.  It's called manual intervention.

> Actually, it would make sense to avoid "BOOM" as much as possible;
> instead of going into a "BOOM" mode, it should go into some kind of
> "recovery" mode, when the array immediately turns read-only and allows
> you to salvage as much data as you can from the faulty drive. It would
> be especially useful in RAID5; only one faulty sector or even some other
> stupid error without even faulty sectors should not cause total data
> loss!.. I don't think it would be very hard to implement, or contradict
> some important philosophy, but it would save data, jobs and lives. :)

Well, the root causes vary, and the actions needed vary accordingly.
There isn't enough information to proceed without operator assistance.
This list has helped numerous people recover their data.  ( But not
always. )

> Well, suppose "BOOM" happened on my RAID1, and both drives failed.
> Naturally, not at the same time, so data differs between the drives. So
> I have two faulty drives; is it possible (with some other recovery
> software?) to just mount each one read-only and copy data from it?.. I
> can live with having to copy as much data as I can from the first one
> and try to copy the rest from the older drive. I mean, I basically have
> the entire filesystem there! I should be able to just mount the
> filesystem on that drive, skipping the RAID header - just fdisk the
> drive, delete the RAID partition and create another one starting a few
> blocks later!.. Would that not work?.. That's actually why I chose
> RAID1; even if everything goes BOOM, I should be able to mount the
> drives separately!..

Yes, you can use dmsetup to gain access to any raid1's data area.
Mirrors using v0.90 and v1.0 metadata can be accessed directly.  (v.90
is strongly discouraged, though, as there are unavoidable ambiguities in
that old format that can blow up on you.)

raid10 and parity raids cannot be accessed like this.  You need
scrubbing and good drives, quick response to failures, and backups for
the final tiny probability of failure.

Phil

^ permalink raw reply

* Re: Wrong device name after hot-swap
From: Wols Lists @ 2016-01-23 14:45 UTC (permalink / raw)
  To: Paul van der Vlis, linux-raid
In-Reply-To: <n7u8gr$1bv$1@ger.gmane.org>

On 22/01/16 21:55, Paul van der Vlis wrote:
> Hello,
> 
> I want to put bigger disks into my server. What I want to do is replace
> the first disk, rebuild the raid, replace the second disk, rebuild the
> raid. The machine has two disks, sda and sdb.

Have you only got two slots?
> 
> But, when I replace a disk, it gets a new device name. E.g. /dev/sdb
> becomes /dev/sdc. After a reboot it's good again, but I prefer not to
> reboot this machine!

Sounds like you're pulling a drive then replacing it. Can't you replace
it then pull?

That'll mean that when the array is at its most vulnerable, ie it's
rebuilding, you'll have three drives on line not one.

How new is your mdadm? You'll need one that supports the "--replace" syntax.

Cheers,
Wol

^ permalink raw reply

* Re: md failing mechanism
From: Wols Lists @ 2016-01-23 14:09 UTC (permalink / raw)
  To: James J, linux-raid
In-Reply-To: <56A2BDF7.7020101@shiftmail.org>

On 22/01/16 23:40, James J wrote:
> The recommentation of raising the timeout to 120+ is for the opposite
> purpose of what you want. It is for the case the sysadmin accepts to
> wait a long time because he wants to prevent the kicking of the drive at
> the first read-error (normally drives are kicked for a write error).
> This might be wanted in order to a) defer the replacement of the drive,
> either to perform the replacement at a more opportune time and/or in a
> better manner such as a no-degrade replace operation, or b) because he
> does not want to replace the drive at all: maybe he believes that the
> error might be spurious and will not happen again and the drive is still
> of acceptable fitness for the purpose, e.g. in a low-cost file server.

Except, aiui, even in your scenario! drives are kicked for a *write* error.

What happens (should be) is the kernel times out, the raid handles the
read error by trying a rewrite, the drive is still hung on the read
error so it doesn't respond to the write request, and the drive gets
kicked for a write failure.

Cheers,
Wol

^ permalink raw reply

* Re: md failing mechanism
From: Dark Penguin @ 2016-01-23 10:33 UTC (permalink / raw)
  To: Phil Turmel, Edward Kuns, linux-raid
In-Reply-To: <56A2CA93.8020806@turmel.org>

>> I wonder what happens if the only drive in a degraded mirror fails?..
>
> Or any drive in a raid5 hits an error while replacing a failed disk.
>
> BOOM.

Oh... That sucks. I understand that BOOM would happen on a RAID-F, but 
kicking the last drive doesn't seem like a very rational thing to do - 
you would get a RAID with zero drives, which is not something anybody 
would want in any situation!.. I thought it would get into some kind of 
extreme panic mode so that you could at least save what you can...

Actually, it would make sense to avoid "BOOM" as much as possible; 
instead of going into a "BOOM" mode, it should go into some kind of 
"recovery" mode, when the array immediately turns read-only and allows 
you to salvage as much data as you can from the faulty drive. It would 
be especially useful in RAID5; only one faulty sector or even some other 
stupid error without even faulty sectors should not cause total data 
loss!.. I don't think it would be very hard to implement, or contradict 
some important philosophy, but it would save data, jobs and lives. :)

Well, suppose "BOOM" happened on my RAID1, and both drives failed. 
Naturally, not at the same time, so data differs between the drives. So 
I have two faulty drives; is it possible (with some other recovery 
software?) to just mount each one read-only and copy data from it?.. I 
can live with having to copy as much data as I can from the first one 
and try to copy the rest from the older drive. I mean, I basically have 
the entire filesystem there! I should be able to just mount the 
filesystem on that drive, skipping the RAID header - just fdisk the 
drive, delete the RAID partition and create another one starting a few 
blocks later!.. Would that not work?.. That's actually why I chose 
RAID1; even if everything goes BOOM, I should be able to mount the 
drives separately!..


-- 
darkpenguin

^ permalink raw reply

* Re: Wrong device name after hot-swap
From: Phil Turmel @ 2016-01-23  0:51 UTC (permalink / raw)
  To: Paul van der Vlis, linux-raid
In-Reply-To: <n7ud6o$b1f$1@ger.gmane.org>

On 01/22/2016 06:15 PM, Paul van der Vlis wrote:
> Op 22-01-16 om 23:27 schreef Phil Turmel:

>> MD stores signatures in the devices it uses that identify them for later
>> assembly.  It does not depend on the device name, though it is recorded
>> in the superblock as a "last connected as" kind of indicator.
> 
> So I can do it as described above?

Yes.  Though I won't comment on grub part.  (I've stopped using it and
am no long current.)

>> In general, you should not rely on device names in your system
>> configuration.  UUIDs and filesystem labels were implemented
>> specifically to avoid this problem.
> 
> So far I know I cannot configure anywhere a disk UUID or a filesytem
> label to a MD-device.

Modern udev creates /dev/disk/by-id/ and similar symlinks, but its not
important for md.

> It feels wrong when I add /dev/sdc1 to the raid, when the name is
> normally /dev/sdb1. But maybe it's no problem, because the device name
> is not impartant while booting from an MD-device.

It's not a problem.  Unless you have specific member device names in
mdadm.conf.  (Don't do that.)

Phil

^ permalink raw reply

* [PATCH 2/2] md-cluster: delete useless code
From: Shaohua Li @ 2016-01-23  0:45 UTC (permalink / raw)
  To: linux-raid; +Cc: Goldwyn Rodrigues, Guoqing Jiang, NeilBrown
In-Reply-To: <0bc7fe7dece2c5aa4f8ed7194df27b02e0ae3142.1453509823.git.shli@fb.com>

page->index already considers node offset. The node_offset calculation
in write_sb_page is useless and confusion.

Cc: Goldwyn Rodrigues <rgoldwyn@suse.com>
Cc: Guoqing Jiang <gqjiang@suse.com>
Cc: NeilBrown <neilb@suse.com>
Signed-off-by: Shaohua Li <shli@fb.com>
---
 drivers/md/bitmap.c | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/drivers/md/bitmap.c b/drivers/md/bitmap.c
index 4f22e91..d80cce4 100644
--- a/drivers/md/bitmap.c
+++ b/drivers/md/bitmap.c
@@ -210,10 +210,6 @@ static int write_sb_page(struct bitmap *bitmap, struct page *page, int wait)
 	struct block_device *bdev;
 	struct mddev *mddev = bitmap->mddev;
 	struct bitmap_storage *store = &bitmap->storage;
-	int node_offset = 0;
-
-	if (mddev_is_clustered(bitmap->mddev))
-		node_offset = bitmap->cluster_slot * store->file_pages;
 
 	while ((rdev = next_active_rdev(rdev, mddev)) != NULL) {
 		int size = PAGE_SIZE;
-- 
2.4.6


^ permalink raw reply related

* [PATCH 1/2] md-cluster: fix missing memory free
From: Shaohua Li @ 2016-01-23  0:45 UTC (permalink / raw)
  To: linux-raid; +Cc: Goldwyn Rodrigues, Guoqing Jiang, NeilBrown

There are several places we allocate dlm_lock_resource, but not free it.

Cc: Goldwyn Rodrigues <rgoldwyn@suse.com>
Cc: Guoqing Jiang <gqjiang@suse.com>
Cc: NeilBrown <neilb@suse.com>
Signed-off-by: Shaohua Li <shli@fb.com>
---
 drivers/md/md-cluster.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/md/md-cluster.c b/drivers/md/md-cluster.c
index 0ded8e9..662369c 100644
--- a/drivers/md/md-cluster.c
+++ b/drivers/md/md-cluster.c
@@ -293,6 +293,7 @@ static void recover_bitmaps(struct md_thread *thread)
 dlm_unlock:
 		dlm_unlock_sync(bm_lockres);
 clear_bit:
+		lockres_free(bm_lockres);
 		clear_bit(slot, &cinfo->recovery_map);
 	}
 }
@@ -682,8 +683,10 @@ static int gather_all_resync_info(struct mddev *mddev, int total_slots)
 		bm_lockres = lockres_init(mddev, str, NULL, 1);
 		if (!bm_lockres)
 			return -ENOMEM;
-		if (i == (cinfo->slot_number - 1))
+		if (i == (cinfo->slot_number - 1)) {
+			lockres_free(bm_lockres);
 			continue;
+		}
 
 		bm_lockres->flags |= DLM_LKF_NOQUEUE;
 		ret = dlm_lock_sync(bm_lockres, DLM_LOCK_PW);
-- 
2.4.6


^ permalink raw reply related

* Re: md failing mechanism
From: Phil Turmel @ 2016-01-23  0:44 UTC (permalink / raw)
  To: James J, Dark Penguin, linux-raid
In-Reply-To: <56A2BDF7.7020101@shiftmail.org>

On 01/22/2016 06:40 PM, James J wrote:
> On 22/01/2016 22:44, Dark Penguin wrote:
>>
>> As I understand, one way around this problem is to change the kernel
>> timeout to exceed the drive timeout by changing
>> /sys/block/sd?/device/timeout to something larger than the default 30,
>> but I'd have to do that after every reboot, is all that correct?
>>
> 
> No, this part needs further investigation and comments from the gurus.

Yes, DP had that correct.

> With a SCSI timeout 30 secs, which is the setting you had at the time of
> the incident AFAIU, what should have happened was that the drive should
> have been kicked out at the 30th second, this is BEFORE it had a chance
> to return a read failure because your desktop drive takes more than
> 30secs to return a read failure. This was what you indeed expected but
> it is not what has happened.

His problem description doesn't perfectly match timeout mismatch.  He
probably had a real problem that was exacerbated by his now-discovered
timeout problem.  He no longer has the dmesg so further speculation is
moot.  If it happens again, we can look closer.

> The recommentation of raising the timeout to 120+ is for the opposite
> purpose of what you want. It is for the case the sysadmin accepts to
> wait a long time because he wants to prevent the kicking of the drive at
> the first read-error (normally drives are kicked for a write error).
> This might be wanted in order to a) defer the replacement of the drive,
> either to perform the replacement at a more opportune time and/or in a
> better manner such as a no-degrade replace operation, or b) because he
> does not want to replace the drive at all: maybe he believes that the
> error might be spurious and will not happen again and the drive is still
> of acceptable fitness for the purpose, e.g. in a low-cost file server.

No.  If you have a drive that doesn't support scterc or has it turned
off, you *must* set a timeout longer than the drive's native timeout or
you will have great problems.  I suggest you read the references to the
archives I posted.

Keep in mind that in a properly working array UREs are *fixed* when
discovered by overwriting them.  This is vital to array robustness, as
many UREs are transient (don't need relocation at all).

Phil


^ permalink raw reply


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox