* [PATCH 0/8] mdadm static checker fixes
From: Jes.Sorensen @ 2016-03-08 17:30 UTC (permalink / raw)
To: linux-raid; +Cc: neilb, gqjiang, pawel.baldysiak
From: Jes Sorensen <Jes.Sorensen@redhat.com>
Hi,
I have been running mdadm through Coverity and fixed a number of
issues that were raised in the scan. A couple of them were non issues
related to conditions where we know for sure the kernel will not
return strings longer than a given size, but there were also a number
of potential memory leaks and buffer overflows.
These patches are sitting in my pending queue. If you are on the CC
list, would you mind having a look at the portions touching code you
previously pushed wrote.
Please hollor if you notice I did anything wrong, otherwise I'll push
this set into git within the next couple of days.
Cheers,
Jes
Jes Sorensen (8):
Manage: Manage_add(): Fix memory leak
load_sys(): Add a buffer size argument
Grow: Grow_continue_command() remove dead code
Grow: Grow_addbitmap(): Add check to quiet down static code checkers
{platform,super}-intel: Fix two resource leaks
bitmap: Fix resource leak in bitmap_file_open()
Manage: Manage_subdevs() fix file descriptor leak
super1: Fix potential buffer overflows when copying cluster_name
Detail.c | 2 +-
Grow.c | 11 ++++++++---
Manage.c | 8 ++++++--
bitmap.c | 1 +
mdadm.h | 2 +-
platform-intel.c | 7 ++++++-
super-intel.c | 12 ++++++++----
super1.c | 18 ++++++++++++------
sysfs.c | 47 ++++++++++++++++++++++++-----------------------
9 files changed, 67 insertions(+), 41 deletions(-)
--
2.5.0
^ permalink raw reply
* Re: [PATCH 2/2] Remove dead code about LKF_CONVERT flag
From: Guoqing Jiang @ 2016-03-08 1:28 UTC (permalink / raw)
To: Jes Sorensen, Guoqing Jiang; +Cc: linux-raid, rgoldwyn, neilb
In-Reply-To: <wrfjio0yj9fq.fsf@redhat.com>
On 03/08/2016 04:54 AM, Jes Sorensen wrote:
> Guoqing Jiang <gqjiang@suse.com> writes:
>> Since flags is only set as LKF_NOQUEUE, the code
>> with LKF_CONVERT flag should be delete.
>>
>> Reported-by: Jes Sorensen <Jes.Sorensen@redhat.com>
>> Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
>> ---
>> mdadm.h | 1 -
>> util.c | 8 --------
>> 2 files changed, 9 deletions(-)
> Applied!
>
> For multi-patch series, would you mind adding a cover letter when you
> post them? Then I can just respond to that if all patches are applied.
Sorry for inconvenience, I will do it in next time for similar case.
Thanks,
Guoqing
^ permalink raw reply
* Re: Import/recover RAID6 created with SAS2108?
From: Andreas Klauer @ 2016-03-07 21:50 UTC (permalink / raw)
To: Dan Russell; +Cc: linux-raid
In-Reply-To: <2832CD22-EADC-4BC6-A250-FE8919A92E0A@aol.com>
On Sun, Mar 06, 2016 at 04:37:42PM -0500, Dan Russell wrote:
> can md read a RAID6 created by the 2108, or would I be wasting my time?
It might be possible.
I don't use HW-RAID anywhere but if I was forced to, this would be
one of the first things to determine, how to read it with software
in case the card fails.
> I can paste or gist the output of madam -E (304 lines) if it would help.
I don't know if it would help, but paste everything you have anyhow.
Is the data encrypted? If not - search for known file type headers
(like a large jpeg or tar.gz, something larger than disks * chunksize),
then look at the data on the other disks at the same offsets
(grab a few megs of each disks), then try to deduce the layout
and structure from that.
Basically use known data to reverse-engineer the layout if you
currently know nothing about disk orders, chunk sizes, etc.
Regards
Andreas Klauer
^ permalink raw reply
* Re: [PATCH 2/2] Remove dead code about LKF_CONVERT flag
From: Jes Sorensen @ 2016-03-07 20:54 UTC (permalink / raw)
To: Guoqing Jiang; +Cc: linux-raid, rgoldwyn, neilb
In-Reply-To: <1457343063-14884-2-git-send-email-gqjiang@suse.com>
Guoqing Jiang <gqjiang@suse.com> writes:
> Since flags is only set as LKF_NOQUEUE, the code
> with LKF_CONVERT flag should be delete.
>
> Reported-by: Jes Sorensen <Jes.Sorensen@redhat.com>
> Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
> ---
> mdadm.h | 1 -
> util.c | 8 --------
> 2 files changed, 9 deletions(-)
Applied!
For multi-patch series, would you mind adding a cover letter when you
post them? Then I can just respond to that if all patches are applied.
Thanks,
Jes
^ permalink raw reply
* Re: [PATCH 1/2] Fix wrong bitmap output for cluster raid
From: Jes Sorensen @ 2016-03-07 20:53 UTC (permalink / raw)
To: Guoqing Jiang; +Cc: linux-raid, rgoldwyn, neilb
In-Reply-To: <1457343063-14884-1-git-send-email-gqjiang@suse.com>
Guoqing Jiang <gqjiang@suse.com> writes:
> For cluster raid, we need to displays bitmap related
> contents from different bitmaps which are based on node
> num. So bitmap_file_open and locate_bitmap are changed a
> little bit for the purpose.
>
> Reported-by: Jes Sorensen <Jes.Sorensen@redhat.com>
> Fixes: b98043a2f8 ("Show all bitmaps while examining bitmap")
> Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
> ---
> bitmap.c | 18 +++++++++---------
> mdadm.h | 2 +-
> super0.c | 2 +-
> super1.c | 12 ++++++------
> 4 files changed, 17 insertions(+), 17 deletions(-)
Applied!
Thanks,
Jes
>
> diff --git a/bitmap.c b/bitmap.c
> index dab674b..5ad7401 100644
> --- a/bitmap.c
> +++ b/bitmap.c
> @@ -194,7 +194,7 @@ out:
> return info;
> }
>
> -int bitmap_file_open(char *filename, struct supertype **stp)
> +int bitmap_file_open(char *filename, struct supertype **stp, int node_num)
> {
> int fd;
> struct stat stb;
> @@ -222,7 +222,7 @@ int bitmap_file_open(char *filename, struct supertype **stp)
> st->ss->name);
> return -1;
> } else {
> - if (st->ss->locate_bitmap(st, fd)) {
> + if (st->ss->locate_bitmap(st, fd, node_num)) {
> pr_err("%s doesn't have bitmap\n", filename);
> fd = -1;
> }
> @@ -267,7 +267,7 @@ int ExamineBitmap(char *filename, int brief, struct supertype *st)
> int fd, i;
> __u32 uuid32[4];
>
> - fd = bitmap_file_open(filename, &st);
> + fd = bitmap_file_open(filename, &st, 0);
> if (fd < 0)
> return rv;
>
> @@ -348,11 +348,11 @@ int ExamineBitmap(char *filename, int brief, struct supertype *st)
> printf(" Cluster nodes : %d\n", sb->nodes);
> printf(" Cluster name : %-64s\n", sb->cluster_name);
> for (i = 0; i < (int)sb->nodes; i++) {
> - if (i) {
> - free(info);
> - info = bitmap_fd_read(fd, brief);
> - sb = &info->sb;
> - }
> + st = NULL;
> + free(info);
> + fd = bitmap_file_open(filename, &st, i);
> + info = bitmap_fd_read(fd, brief);
> + sb = &info->sb;
> if (sb->magic != BITMAP_MAGIC)
> pr_err("invalid bitmap magic 0x%x, the bitmap file appears to be corrupted\n", sb->magic);
>
> @@ -367,7 +367,7 @@ int ExamineBitmap(char *filename, int brief, struct supertype *st)
> printf(" Bitmap : %llu bits (chunks), %llu dirty (%2.1f%%)\n",
> info->total_bits, info->dirty_bits,
> 100.0 * info->dirty_bits / (info->total_bits?:1));
> -
> + close(fd);
> }
> }
>
> diff --git a/mdadm.h b/mdadm.h
> index 72888e2..355c455 100755
> --- a/mdadm.h
> +++ b/mdadm.h
> @@ -904,7 +904,7 @@ extern struct superswitch {
> /* Seek 'fd' to start of write-intent-bitmap. Must be an
> * md-native format bitmap
> */
> - int (*locate_bitmap)(struct supertype *st, int fd);
> + int (*locate_bitmap)(struct supertype *st, int fd, int node_num);
> /* if add_internal_bitmap succeeded for existing array, this
> * writes it out.
> */
> diff --git a/super0.c b/super0.c
> index 59a6a03..6fc1ac4 100644
> --- a/super0.c
> +++ b/super0.c
> @@ -1156,7 +1156,7 @@ static int add_internal_bitmap0(struct supertype *st, int *chunkp,
> return 1;
> }
>
> -static int locate_bitmap0(struct supertype *st, int fd)
> +static int locate_bitmap0(struct supertype *st, int fd, int node_num)
> {
> unsigned long long dsize;
> unsigned long long offset;
> diff --git a/super1.c b/super1.c
> index 9b877e5..f8a35ac 100644
> --- a/super1.c
> +++ b/super1.c
> @@ -1548,7 +1548,7 @@ static int add_to_super1(struct supertype *st, mdu_disk_info_t *dk,
> }
> #endif
>
> -static int locate_bitmap1(struct supertype *st, int fd);
> +static int locate_bitmap1(struct supertype *st, int fd, int node_num);
>
> static int store_super1(struct supertype *st, int fd)
> {
> @@ -1622,7 +1622,7 @@ static int store_super1(struct supertype *st, int fd)
> struct bitmap_super_s *bm = (struct bitmap_super_s*)
> (((char*)sb)+MAX_SB_SIZE);
> if (__le32_to_cpu(bm->magic) == BITMAP_MAGIC) {
> - locate_bitmap1(st, fd);
> + locate_bitmap1(st, fd, 0);
> if (awrite(&afd, bm, sizeof(*bm)) != sizeof(*bm))
> return 5;
> }
> @@ -2062,7 +2062,7 @@ static int load_super1(struct supertype *st, int fd, char *devname)
> * valid. If it doesn't clear the bit. An --assemble --force
> * should get that written out.
> */
> - locate_bitmap1(st, fd);
> + locate_bitmap1(st, fd, 0);
> if (aread(&afd, bsb, 512) != 512)
> goto no_bitmap;
>
> @@ -2334,7 +2334,7 @@ add_internal_bitmap1(struct supertype *st,
> return 1;
> }
>
> -static int locate_bitmap1(struct supertype *st, int fd)
> +static int locate_bitmap1(struct supertype *st, int fd, int node_num)
> {
> unsigned long long offset;
> struct mdp_superblock_1 *sb;
> @@ -2353,7 +2353,7 @@ static int locate_bitmap1(struct supertype *st, int fd)
> else
> ret = -1;
> offset = __le64_to_cpu(sb->super_offset);
> - offset += (int32_t) __le32_to_cpu(sb->bitmap_offset);
> + offset += (int32_t) __le32_to_cpu(sb->bitmap_offset) * (node_num + 1);
> if (mustfree)
> free(sb);
> lseek64(fd, offset<<9, 0);
> @@ -2408,7 +2408,7 @@ static int write_bitmap1(struct supertype *st, int fd, enum bitmap_update update
>
> init_afd(&afd, fd);
>
> - locate_bitmap1(st, fd);
> + locate_bitmap1(st, fd, 0);
>
> if (posix_memalign(&buf, 4096, 4096))
> return -ENOMEM;
^ permalink raw reply
* Re: [PATCH] md/bitmap: remove redundant check
From: Shaohua Li @ 2016-03-07 17:26 UTC (permalink / raw)
To: Eric Engestrom; +Cc: linux-raid, linux-kernel
In-Reply-To: <1457352065-28994-1-git-send-email-eric.engestrom@imgtec.com>
On Mon, Mar 07, 2016 at 12:01:05PM +0000, Eric Engestrom wrote:
> daemon_sleep is an unsigned, so testing if it's 0 or less than 1 does
> the same thing.
>
> Signed-off-by: Eric Engestrom <eric.engestrom@imgtec.com>
> ---
> drivers/md/bitmap.c | 3 +--
> 1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/drivers/md/bitmap.c b/drivers/md/bitmap.c
> index d80cce4..bbe7b64 100644
> --- a/drivers/md/bitmap.c
> +++ b/drivers/md/bitmap.c
> @@ -510,8 +510,7 @@ static int bitmap_new_disk_sb(struct bitmap *bitmap)
> sb->chunksize = cpu_to_le32(chunksize);
>
> daemon_sleep = bitmap->mddev->bitmap_info.daemon_sleep;
> - if (!daemon_sleep ||
> - (daemon_sleep < 1) || (daemon_sleep > MAX_SCHEDULE_TIMEOUT)) {
> + if (!daemon_sleep || (daemon_sleep > MAX_SCHEDULE_TIMEOUT)) {
> printk(KERN_INFO "Choosing daemon_sleep default (5 sec)\n");
> daemon_sleep = 5 * HZ;
> }
Applied, thanks!
^ permalink raw reply
* Re: use after free bug in b98043a2f8e7bb5b1918e2e02778f822f9dd4d3a
From: Guoqing Jiang @ 2016-03-07 16:33 UTC (permalink / raw)
To: Jes Sorensen; +Cc: linux-raid, Goldwyn Rodrigues, NeilBrown
In-Reply-To: <wrfjy49yym97.fsf@redhat.com>
On 03/05/2016 05:19 AM, Jes Sorensen wrote:
> Hi,
>
> I was looking at ExamineBitmap() and noticed that your patch below,
> seems to introduce a use after free bug.
>
> commit b98043a2f8e7bb5b1918e2e02778f822f9dd4d3a
> Author: Guoqing Jiang <gqjiang@suse.com>
> Date: Wed Jun 10 13:42:07 2015 +0800
>
> Show all bitmaps while examining bitmap
>
> This adds capability of exmining bitmaps corresponding to all
> nodes/slots on the device.
>
> Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
> Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
> Signed-off-by: NeilBrown <neilb@suse.de>
>
> Line 284 does a close(fd), but further down the code your patch
> introduces:
>
> + } else {
> + printf(" Cluster nodes : %d\n", sb->nodes);
> + printf(" Cluster name : %64s\n", sb->cluster_name);
> + for (i = 0; i < (int)sb->nodes; i++) {
> + if (i) {
> + free(info);
> + info = bitmap_fd_read(fd, brief);
> + sb = &info->sb;
> + }
>
> It's not totally obvious to me here what the intention is for clustered
> md here. Are you intending to use the same fd as was used at the top of
> ExamineBitmap() or is it meant to open a new fd based on each clustered
> entry?
>
> In either case, the code as it is right now is certainly not going to
> work :(
Yes, it is wrong, thanks for the catch, I will handle it too.
Best Regards,
Guoqing
^ permalink raw reply
* Re: Caching raid with SSD.
From: John Stoffel @ 2016-03-07 15:17 UTC (permalink / raw)
To: Ram Ramesh; +Cc: John Stoffel, Linux Raid
In-Reply-To: <56DD0774.3070601@gmail.com>
>>>>> "Ram" == Ram Ramesh <rramesh2400@gmail.com> writes:
Ram> On 03/06/2016 07:58 PM, John Stoffel wrote:
Ram> Any one here actually use SSD caches for RAID arrays? Can you
Ram> share your experience and let me know your choice of the type of
Ram> cache methods your tired/used and why you think one is better or
Ram> worse than other? If it is possible, please provide raid
Ram> type/size and ssd size used.
>>
>> I'm using a pair of 4Tb drives mirrored, and a pair of 512gb SSDs,
>> also mirrored, along with lvmcache to setup my caching across a couple
>> of volumes.
>>
>> I honestly haven't seen huge improvements, but I also haven't had the
>> time to do any serious testing either. Whcih I should do. I've been
>> sorta thinking that using the Phoronix testing stuff would be the way
>> to go.
>>
>> My SSDs and 4Tb drives are all on an LSI 8-port SATA controller, PCI-E
>> 4x I think. It's an MPT SAS-2 controller. I did this so that my boot
>> drives are some partitions on the SSDs, and then I use two more
>> mirrored partitions for the cache.
>>
>> And this is an NFS server for my home directories, etc.
>>
>> I didn't use bcache because you can't remove a cache device without
>> rebooting, or at least bringing a device offline and back online,
>> which doesn't fit my desires to be able to dynamically add/remove
>> caches, esp for the testing I've never bothered to do.
>>
>> John
Ram> I do not have lvm, but already have a live (regular) file
Ram> system. While I can accept downtime, I cannot accept formatting
Ram> drives/disks. I simply do not have the extra space to copy back
Ram> and forth. That is why I thought of dmcache. I ran a fio
Ram> experiment on my ssd (old curial M4) and I am getting 6K (random)
Ram> IOPs whereas my raid gives me about 1.5K. I really do not see
Ram> much point unless my new SSD puts out some decent numbers stand
Ram> alone.
I'm not sure fio is the right test here, but it all depends on what
you do, which is the curse of performance testing!
I kinda like kernel compiles, and doing lots of image viewing
(building of thumbnails, etc) to see how things will speed up in my
common usecase.
Ram> Thanks for sharing the details of your setup.
Good luck with your testing, I'd love to see your results when you've
decided which way to go.
^ permalink raw reply
* Re: [PATCH RESEND v2 11/18] fs: Ensure the mounter of a filesystem is privileged towards its inodes
From: Seth Forshee @ 2016-03-07 13:32 UTC (permalink / raw)
To: Eric W. Biederman
Cc: Alexander Viro, Serge Hallyn, Richard Weinberger,
Austin S Hemmelgarn, Miklos Szeredi, linux-kernel, linux-bcache,
dm-devel, linux-raid, linux-mtd, linux-fsdevel, fuse-devel,
linux-security-module, selinux
In-Reply-To: <8737s32rbe.fsf@x220.int.ebiederm.org>
On Sun, Mar 06, 2016 at 04:07:49PM -0600, Eric W. Biederman wrote:
> Seth Forshee <seth.forshee@canonical.com> writes:
>
> > On Fri, Mar 04, 2016 at 04:43:06PM -0600, Eric W. Biederman wrote:
> >> Seth Forshee <seth.forshee@canonical.com> writes:
> >>
> >> > On Mon, Jan 04, 2016 at 12:03:50PM -0600, Seth Forshee wrote:
> >> >> The mounter of a filesystem should be privileged towards the
> >> >> inodes of that filesystem. Extend the checks in
> >> >> inode_owner_or_capable() and capable_wrt_inode_uidgid() to
> >> >> permit access by users priviliged in the user namespace of the
> >> >> inode's superblock.
> >> >
> >> > Eric - I've discovered a problem related to this patch. The patches
> >> > you've already applied to your testing branch make it so that s_user_ns
> >> > can be an unprivileged user for proc and kernfs-based mounts. In some
> >> > cases DAC is the only thing protecting files in these mounts (ignoring
> >> > MAC), and with this patch an unprivileged user could bypass DAC.
> >> >
> >> > There's a simple solution - always set s_user_ns to &init_user_ns for
> >> > those filesystems. I think this is the right thing to do, since the
> >> > backing store behind these filesystems are really kernel objects. But
> >> > this would break the assumption behind your patch "userns: Simpilify
> >> > MNT_NODEV handling" and cause a regression in mounting behavior.
> >> >
> >> > I've come up with several possible solutions for this conflict.
> >> >
> >> > 1. Drop this patch and keep on setting s_user_ns to unprivilged users.
> >> > This would be unfortunate because I think this patch does make sense
> >> > for most filesystems.
> >> > 2. Restrict this patch so that a user privileged towards s_user_ns is
> >> > only privileged towards the super blocks inodes if s_user_ns has a
> >> > mapping for both i_uid and i_gid. This is better than (1) but still
> >> > not ideal in my mind.
> >> > 3. Drop your patch and maintain the current MNT_NODEV behavior.
> >> > 4. Add a new s_iflags flag to indicate a super block is from an
> >> > unprivileged mount, and use this in your patch instead of s_user_ns.
> >> >
> >> > Any preference, or any other ideas?
> >>
> >> In general this is only an issue if uids and gids on the filesystem
> >> do not map into the user namespace.
> >
> > Yes, both capable_wrt_inode_uidgid and inode_owner_or_capable will
> > return true for a privileged user in the current namespace if the ids
> > map into that namespace.
> >
> >> Therefore the general fix is to limit the logic of checking for
> >> capabilities in s_user_ns if we are dealing with INVALID_UID and
> >> INVALID_GID. For proc and kernfs that should never be the case
> >> so the problem becomes a non-issue.
> >>
> >> Further I would look at limiting that relaxation to just
> >> inode_change_ok. So that we can easily wrap that check per filesystem
> >> and deny the relaxation for proc and kernfs. proc and kernfs already
> >> have wrappers for .setattr so denying changes when !uid_vaid and
> >> !gid_valid would be a trivial addition, and ensure calamity does
> >> not ensure.
> >>
> >> Furthmore by limiting any additional to inode_change_ok we keep
> >> the work of the additional tests off of the fast paths.
> >
> > So then the inode would need to be chowned before a privileged user in a
> > non-init namespace would be capable towards it. That seems workable. It
> > looks like INVALID_UID and INVALID_GID do map into init_user_ns (which
> > seems a bit odd) so real root remains capable towards those indoes.
> >
> > That seems okay to me then.
>
> If I was not clear I was suggesting that we allow a sufficiently
> privileged user in the filesysteme's s_user_ns to allow chowning files
> with INVALID_UID and INVALID_GID.
Right, I got that.
> The global root user would always be able to do that because unless
> capabilities are dropped it is sufficiently privileged in ever user
> namespace.
Sure. I was just commenting on one result - that ns-root has to chown
the file before being privileged wrt that file but global root does not,
on account of the fact that the invalid ids are mapped in init_user_ns.
Seth
^ permalink raw reply
* Re: dead code question
From: Guoqing Jiang @ 2016-03-07 13:18 UTC (permalink / raw)
To: Jes Sorensen; +Cc: linux-raid, NeilBrown, Goldwyn Rodrigues
In-Reply-To: <wrfjpovaylve.fsf@redhat.com>
On 03/05/2016 05:27 AM, Jes Sorensen wrote:
> Hi,
>
> Another question regarding the cluster code. This one was introduced
> with:
>
> commit d15a1f72bd92bc4724ee94b2ae8132633ffeb72b
> Author: Guoqing Jiang <gqjiang@suse.com>
> Date: Mon Oct 19 16:03:19 2015 +0800
>
> Safeguard against writing to an active device of another node
>
> Modifying an exiting device's superblock or creating a new superblock
> on an existing device needs to be checked because the device could be
> in use by another node in another array. So, we check this by taking
> all superblock locks in userspace so that we don't step onto an active
> device used by another node and safeguard against accidental edits.
> After the edit is complete, we release all locks and the lockspace so
> that it can be used by the kernel space.
>
> Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
> Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
> Signed-off-by: NeilBrown <neilb@suse.com>
>
> [snip]
>
> +/* Create the lockspace, take bitmapXXX locks on all the bitmaps. */
> +int cluster_get_dlmlock(struct supertype *st, int *lockid)
> +{
> + int ret = -1;
> + char str[64];
> + int flags = LKF_NOQUEUE;
> +
> + dlm_lock_res = xmalloc(sizeof(struct dlm_lock_resource));
> + dlm_lock_res->ls = dlm_hooks->create_lockspace(st->cluster_name, O_RDWR);
> + if (!dlm_lock_res->ls) {
> + pr_err("%s failed to create lockspace\n", st->cluster_name);
> + goto out;
> + }
> +
> + /* Conversions need the lockid in the LKSB */
> + if (flags & LKF_CONVERT)
> + dlm_lock_res->lksb.sb_lkid = *lockid;
>
> You set flags = LKF_NOQUEUE but later check (flags & LKF_CONVERT), which
> cannot be true given that:
>
> mdadm.h:#define LKF_NOQUEUE 0x00000001
> mdadm.h:#define LKF_CONVERT 0x00000004
>
> Is this a bug, or do you have pending patches for this?
Oops, I should remove the later check and will send the fix.
Thanks,
Guoqing
^ permalink raw reply
* [PATCH] md/bitmap: remove redundant check
From: Eric Engestrom @ 2016-03-07 12:01 UTC (permalink / raw)
To: Shaohua Li, linux-raid, linux-kernel; +Cc: Eric Engestrom
daemon_sleep is an unsigned, so testing if it's 0 or less than 1 does
the same thing.
Signed-off-by: Eric Engestrom <eric.engestrom@imgtec.com>
---
drivers/md/bitmap.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/drivers/md/bitmap.c b/drivers/md/bitmap.c
index d80cce4..bbe7b64 100644
--- a/drivers/md/bitmap.c
+++ b/drivers/md/bitmap.c
@@ -510,8 +510,7 @@ static int bitmap_new_disk_sb(struct bitmap *bitmap)
sb->chunksize = cpu_to_le32(chunksize);
daemon_sleep = bitmap->mddev->bitmap_info.daemon_sleep;
- if (!daemon_sleep ||
- (daemon_sleep < 1) || (daemon_sleep > MAX_SCHEDULE_TIMEOUT)) {
+ if (!daemon_sleep || (daemon_sleep > MAX_SCHEDULE_TIMEOUT)) {
printk(KERN_INFO "Choosing daemon_sleep default (5 sec)\n");
daemon_sleep = 5 * HZ;
}
--
2.7.1
^ permalink raw reply related
* [PATCH 2/2] Remove dead code about LKF_CONVERT flag
From: Guoqing Jiang @ 2016-03-07 9:31 UTC (permalink / raw)
To: Jes.Sorensen; +Cc: linux-raid, rgoldwyn, neilb, Guoqing Jiang
In-Reply-To: <1457343063-14884-1-git-send-email-gqjiang@suse.com>
Since flags is only set as LKF_NOQUEUE, the code
with LKF_CONVERT flag should be delete.
Reported-by: Jes Sorensen <Jes.Sorensen@redhat.com>
Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
---
mdadm.h | 1 -
util.c | 8 --------
2 files changed, 9 deletions(-)
diff --git a/mdadm.h b/mdadm.h
index 355c455..97aac52 100755
--- a/mdadm.h
+++ b/mdadm.h
@@ -64,7 +64,6 @@ typedef uint64_t cmap_handle_t;
#include <errno.h>
#else
#define LKF_NOQUEUE 0x00000001
-#define LKF_CONVERT 0x00000004
#define LKM_PWMODE 4
#define EUNLOCK 0x10002
diff --git a/util.c b/util.c
index 96a806d..09c2f6f 100644
--- a/util.c
+++ b/util.c
@@ -147,13 +147,7 @@ int cluster_get_dlmlock(int *lockid)
return -ENOMEM;
}
- /* Conversions need the lockid in the LKSB */
- if (flags & LKF_CONVERT)
- dlm_lock_res->lksb.sb_lkid = *lockid;
-
snprintf(str, 64, "bitmap%s", cluster_name);
- /* if flags with LKF_CONVERT causes below return ENOENT which means
- * "No such file or directory" */
ret = dlm_hooks->ls_lock(dlm_lock_res->ls, LKM_PWMODE, &dlm_lock_res->lksb,
flags, str, strlen(str), 0, dlm_ast,
dlm_lock_res, NULL, NULL);
@@ -177,8 +171,6 @@ int cluster_release_dlmlock(int lockid)
if (!cluster_name)
return -1;
- /* if flags with LKF_CONVERT causes below return EINVAL which means
- * "Invalid argument" */
ret = dlm_hooks->ls_unlock(dlm_lock_res->ls, lockid, 0,
&dlm_lock_res->lksb, dlm_lock_res);
if (ret) {
--
2.6.2
^ permalink raw reply related
* [PATCH 1/2] Fix wrong bitmap output for cluster raid
From: Guoqing Jiang @ 2016-03-07 9:31 UTC (permalink / raw)
To: Jes.Sorensen; +Cc: linux-raid, rgoldwyn, neilb, Guoqing Jiang
For cluster raid, we need to displays bitmap related
contents from different bitmaps which are based on node
num. So bitmap_file_open and locate_bitmap are changed a
little bit for the purpose.
Reported-by: Jes Sorensen <Jes.Sorensen@redhat.com>
Fixes: b98043a2f8 ("Show all bitmaps while examining bitmap")
Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
---
bitmap.c | 18 +++++++++---------
mdadm.h | 2 +-
super0.c | 2 +-
super1.c | 12 ++++++------
4 files changed, 17 insertions(+), 17 deletions(-)
diff --git a/bitmap.c b/bitmap.c
index dab674b..5ad7401 100644
--- a/bitmap.c
+++ b/bitmap.c
@@ -194,7 +194,7 @@ out:
return info;
}
-int bitmap_file_open(char *filename, struct supertype **stp)
+int bitmap_file_open(char *filename, struct supertype **stp, int node_num)
{
int fd;
struct stat stb;
@@ -222,7 +222,7 @@ int bitmap_file_open(char *filename, struct supertype **stp)
st->ss->name);
return -1;
} else {
- if (st->ss->locate_bitmap(st, fd)) {
+ if (st->ss->locate_bitmap(st, fd, node_num)) {
pr_err("%s doesn't have bitmap\n", filename);
fd = -1;
}
@@ -267,7 +267,7 @@ int ExamineBitmap(char *filename, int brief, struct supertype *st)
int fd, i;
__u32 uuid32[4];
- fd = bitmap_file_open(filename, &st);
+ fd = bitmap_file_open(filename, &st, 0);
if (fd < 0)
return rv;
@@ -348,11 +348,11 @@ int ExamineBitmap(char *filename, int brief, struct supertype *st)
printf(" Cluster nodes : %d\n", sb->nodes);
printf(" Cluster name : %-64s\n", sb->cluster_name);
for (i = 0; i < (int)sb->nodes; i++) {
- if (i) {
- free(info);
- info = bitmap_fd_read(fd, brief);
- sb = &info->sb;
- }
+ st = NULL;
+ free(info);
+ fd = bitmap_file_open(filename, &st, i);
+ info = bitmap_fd_read(fd, brief);
+ sb = &info->sb;
if (sb->magic != BITMAP_MAGIC)
pr_err("invalid bitmap magic 0x%x, the bitmap file appears to be corrupted\n", sb->magic);
@@ -367,7 +367,7 @@ int ExamineBitmap(char *filename, int brief, struct supertype *st)
printf(" Bitmap : %llu bits (chunks), %llu dirty (%2.1f%%)\n",
info->total_bits, info->dirty_bits,
100.0 * info->dirty_bits / (info->total_bits?:1));
-
+ close(fd);
}
}
diff --git a/mdadm.h b/mdadm.h
index 72888e2..355c455 100755
--- a/mdadm.h
+++ b/mdadm.h
@@ -904,7 +904,7 @@ extern struct superswitch {
/* Seek 'fd' to start of write-intent-bitmap. Must be an
* md-native format bitmap
*/
- int (*locate_bitmap)(struct supertype *st, int fd);
+ int (*locate_bitmap)(struct supertype *st, int fd, int node_num);
/* if add_internal_bitmap succeeded for existing array, this
* writes it out.
*/
diff --git a/super0.c b/super0.c
index 59a6a03..6fc1ac4 100644
--- a/super0.c
+++ b/super0.c
@@ -1156,7 +1156,7 @@ static int add_internal_bitmap0(struct supertype *st, int *chunkp,
return 1;
}
-static int locate_bitmap0(struct supertype *st, int fd)
+static int locate_bitmap0(struct supertype *st, int fd, int node_num)
{
unsigned long long dsize;
unsigned long long offset;
diff --git a/super1.c b/super1.c
index 9b877e5..f8a35ac 100644
--- a/super1.c
+++ b/super1.c
@@ -1548,7 +1548,7 @@ static int add_to_super1(struct supertype *st, mdu_disk_info_t *dk,
}
#endif
-static int locate_bitmap1(struct supertype *st, int fd);
+static int locate_bitmap1(struct supertype *st, int fd, int node_num);
static int store_super1(struct supertype *st, int fd)
{
@@ -1622,7 +1622,7 @@ static int store_super1(struct supertype *st, int fd)
struct bitmap_super_s *bm = (struct bitmap_super_s*)
(((char*)sb)+MAX_SB_SIZE);
if (__le32_to_cpu(bm->magic) == BITMAP_MAGIC) {
- locate_bitmap1(st, fd);
+ locate_bitmap1(st, fd, 0);
if (awrite(&afd, bm, sizeof(*bm)) != sizeof(*bm))
return 5;
}
@@ -2062,7 +2062,7 @@ static int load_super1(struct supertype *st, int fd, char *devname)
* valid. If it doesn't clear the bit. An --assemble --force
* should get that written out.
*/
- locate_bitmap1(st, fd);
+ locate_bitmap1(st, fd, 0);
if (aread(&afd, bsb, 512) != 512)
goto no_bitmap;
@@ -2334,7 +2334,7 @@ add_internal_bitmap1(struct supertype *st,
return 1;
}
-static int locate_bitmap1(struct supertype *st, int fd)
+static int locate_bitmap1(struct supertype *st, int fd, int node_num)
{
unsigned long long offset;
struct mdp_superblock_1 *sb;
@@ -2353,7 +2353,7 @@ static int locate_bitmap1(struct supertype *st, int fd)
else
ret = -1;
offset = __le64_to_cpu(sb->super_offset);
- offset += (int32_t) __le32_to_cpu(sb->bitmap_offset);
+ offset += (int32_t) __le32_to_cpu(sb->bitmap_offset) * (node_num + 1);
if (mustfree)
free(sb);
lseek64(fd, offset<<9, 0);
@@ -2408,7 +2408,7 @@ static int write_bitmap1(struct supertype *st, int fd, enum bitmap_update update
init_afd(&afd, fd);
- locate_bitmap1(st, fd);
+ locate_bitmap1(st, fd, 0);
if (posix_memalign(&buf, 4096, 4096))
return -ENOMEM;
--
2.6.2
^ permalink raw reply related
* Kernel BUG on RAID1 resize with external bitmap.
From: Dr. Greg Wettstein @ 2016-03-07 9:14 UTC (permalink / raw)
To: linux-raid, shli
Good morning, I hope the week is starting out well for everyone.
We had a production storage server generate a kernel BUG and kill an
mdadm process which was executing a size extension of a RAID1 array
with an external persistent bitmap. The kernel trace for the event is
included just before my .sig below.
Since the BUG killed the mdadm process there was nothing left to walk
out the active locks. This left active the spinlock which protects
the variable holding the Hamming weight of the persistent bitmap. In
addition the MD reconfiguration mutex lock on the device itself was
left active.
There were around 25 RAID1 arrays active on this server and the
incident technically took out I/O only to the RAID1 array which was
being resized. This was secondary to the stuck lock on the Hamming
weight variable blocking the read/write path for that device.
Unfortunately, the stuck reconfiguration lock on the device itself
ended up blocking any references to /proc/mdstat.
Since any type of logical volume management ends up opening the
supporting physical volumes, any attempt to manage the logical volume
system resulted in processes hung in 'D' state. So we had to
remediate the problem by scheduling an outage for the server which has
expected uptimes of a year or more.
Given the reliability requirements for this storage server we
simulated the error condition in a virtual machine environment to see
if we could somehow work around the problem. We were able to
demonstrate the ability to forcibly evict the constituent block
devices but the presence of the 'dead' device in the mddev list was
just too much of an insurmountable obstacle to triage.
The kernel in question is a member of the 3.10.x longterm maintenance
series so we certainly appreciate the reluctance of anyone to look at
this report. As I noted, we measure these server uptimes in multiples
of years, that is simply the reality of production systems of these
types.
The codepaths involved have seen little development activity so if
this isn't a random hardware/memory corruption issue the problem is
still lurking. If nothing else we wanted to get this issue documented
in public in case anyone else searches for something similar.
Any thoughts or reflections are always appreciated.
Best wishes for a productive week to everyone.
Greg
---------------------------------------------------------------------------
Mar 1 01:15:35 fc-iacc1-prox1-s kernel: ------------[ cut here ]------------
Mar 1 01:15:35 fc-iacc1-prox1-s kernel: kernel BUG at drivers/md/bitmap.c:274!
Mar 1 01:15:35 fc-iacc1-prox1-s kernel: invalid opcode: 0000 [#1] SMP
Mar 1 01:15:35 fc-iacc1-prox1-s kernel: CPU: 4 PID: 20377 Comm: mdadm Not tainted 3.10.79 #1
Mar 1 01:15:35 fc-iacc1-prox1-s kernel: Hardware name: Intel Corporation S5520UR/S5520UR, BIOS S5500.86B.01.00.0050.050620101605 05/06/2010
Mar 1 01:15:35 fc-iacc1-prox1-s kernel: task: ffff8803678534e0 ti: ffff880362d3a000 task.ti: ffff880362d3a000
Mar 1 01:15:35 fc-iacc1-prox1-s kernel: RIP: 0010:[<ffffffff8129be69>] [<ffffffff8129be69>] write_page+0x20d/0x2f3
Mar 1 01:15:35 fc-iacc1-prox1-s kernel: RSP: 0018:ffff880362d3ba78 EFLAGS: 00010246
Mar 1 01:15:35 fc-iacc1-prox1-s kernel: RAX: 0200000000000000 RBX: ffff880367988000 RCX: 00000000ffffffff
Mar 1 01:15:35 fc-iacc1-prox1-s kernel: RDX: 0000000000000000 RSI: ffffea000797d500 RDI: ffff880367988000
Mar 1 01:15:35 fc-iacc1-prox1-s kernel: RBP: 0000000000000000 R08: 0000000000000ee0 R09: ffff880367988000
Mar 1 01:15:35 fc-iacc1-prox1-s kernel: R10: ffffffff8129b1f4 R11: 0000000000010b20 R12: ffff880367988000
Mar 1 01:15:35 fc-iacc1-prox1-s kernel: R13: ffffea000797d500 R14: 000000000000ef90 R15: 00000001dd1f8000
Mar 1 01:15:35 fc-iacc1-prox1-s kernel: FS: 0000000000000000(0000) GS:ffff8801e9d00000(0063) knlGS:00000000f763d6b0
Mar 1 01:15:35 fc-iacc1-prox1-s kernel: CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b
Mar 1 01:15:35 fc-iacc1-prox1-s kernel: CR2: 000000000805b9d2 CR3: 00000001e63f3000 CR4: 00000000000007e0
Mar 1 01:15:35 fc-iacc1-prox1-s kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Mar 1 01:15:35 fc-iacc1-prox1-s kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Mar 1 01:15:35 fc-iacc1-prox1-s kernel: Stack:
Mar 1 01:15:35 fc-iacc1-prox1-s kernel: 000000000000000f 0000000080080008 ffffea00079fc158 ffffea00079fc150
Mar 1 01:15:35 fc-iacc1-prox1-s kernel: ffff880367988000 ffff8803685b7c98 0000000000000000 ffff88036fff9c00
Mar 1 01:15:35 fc-iacc1-prox1-s kernel: ffff880367988000 ffff880362d3bbf8 0000000000008010 ffff880367988000
Mar 1 01:15:35 fc-iacc1-prox1-s kernel: Call Trace:
Mar 1 01:15:36 fc-iacc1-prox1-s kernel: [<ffffffff8129c48a>] ? bitmap_unplug+0x7a/0x124
Mar 1 01:15:36 fc-iacc1-prox1-s kernel: [<ffffffff8129b1ff>] ? bitmap_get_counter+0x7c/0x139
Mar 1 01:15:36 fc-iacc1-prox1-s kernel: [<ffffffff8129ca59>] ? bitmap_resize+0x525/0x551
Mar 1 01:15:36 fc-iacc1-prox1-s kernel: [<ffffffff81073259>] ? get_page_from_freelist+0x59b/0x682
Mar 1 01:15:36 fc-iacc1-prox1-s kernel: [<ffffffff8126cb95>] ? raid1_resize+0x48/0xaf
Mar 1 01:15:36 fc-iacc1-prox1-s kernel: [<ffffffff8128ed0b>] ? update_size+0x6c/0x86
Mar 1 01:15:36 fc-iacc1-prox1-s kernel: [<ffffffff81299a0a>] ? md_ioctl+0xac8/0x1775
Mar 1 01:15:36 fc-iacc1-prox1-s kernel: [<ffffffff8106d7f4>] ? filemap_fault+0x5f/0x335
Mar 1 01:15:36 fc-iacc1-prox1-s kernel: [<ffffffff8112877c>] ? compat_blkdev_ioctl+0x4ec/0x1368
Mar 1 01:15:36 fc-iacc1-prox1-s kernel: [<ffffffff81085c9e>] ? handle_mm_fault+0x18e/0x19e
Mar 1 01:15:36 fc-iacc1-prox1-s kernel: [<ffffffff810201fa>] ? do_page_fault+0x3bf/0x40c
Mar 1 01:15:36 fc-iacc1-prox1-s kernel: [<ffffffff810d7257>] ? compat_sys_ioctl+0x1a7/0xf18
Mar 1 01:15:36 fc-iacc1-prox1-s kernel: [<ffffffff810a558b>] ? vfs_fstat+0x35/0x51
Mar 1 01:15:36 fc-iacc1-prox1-s kernel: [<ffffffff81024a68>] ? sys32_fstat64+0x20/0x29
Mar 1 01:15:36 fc-iacc1-prox1-s kernel: [<ffffffff813832df>] ? sysenter_dispatch+0x7/0x1e
Mar 1 01:15:36 fc-iacc1-prox1-s kernel: Code: 84 ec 00 00 00 48 89 ef e8 8d 5a ff ff e9 df 00 00 00 49 8d 44 24 78 f0 41 80 4c 24 78 04 e9 ce 00 00 00 48 8b 06 f6 c4 08 75 04 <0f> 0b eb fe 48 8b 5e 30 48 8d af a0 00 00 00 eb 1d f0 ff 45 00
Mar 1 01:15:36 fc-iacc1-prox1-s kernel: RIP [<ffffffff8129be69>] write_page+0x20d/0x2f3
Mar 1 01:15:36 fc-iacc1-prox1-s kernel: RSP <ffff880362d3ba78>
Mar 1 01:15:36 fc-iacc1-prox1-s kernel: ---[ end trace e97bde0b0a8c45a8 ]---
---------------------------------------------------------------------------
As always,
Dr. G.W. Wettstein, Ph.D. Enjellic Systems Development, LLC.
4206 N. 19th Ave. Specializing in information infra-structure
Fargo, ND 58102 development.
PH: 701-281-1686
FAX: 701-281-3949 EMAIL: greg@enjellic.com
------------------------------------------------------------------------------
"We can't solve today's problems by using the same thinking we used in
creating them."
-- Einstein
--
^ permalink raw reply
* Re: When do you replace old hard drives in a raid6?
From: Carsten Aulbert @ 2016-03-07 6:59 UTC (permalink / raw)
To: Phil Turmel, Ram Ramesh, Linux Raid
In-Reply-To: <56DCCB55.1060906@turmel.org>
[-- Attachment #1: Type: text/plain, Size: 1050 bytes --]
Hi
On 03/07/2016 01:29 AM, Phil Turmel wrote:
> I replace drives when their relocation count hits double digits. In my
> limited sample, that's typically after 40,000 hours.
It really depends a lot on the drive type and manufacturer (for example
see the various reports by Backblaze).
We run quite a number of "desktop" style drives and many have seen 60k
and more power on hours, but then our data collection is a bit biased as
we replace disks once they do not complete a "long" smartcheck (smartctl
-t long).
I've attached a sorted list of current data, columns are mostly:
manufacturer
model number
reallocated sectors (ID 5 of smartctl -a)
power on hours (also according to smartctl -a)
So, not sure one could infer much from these lines due to the inherent bias.
Personally, I would monitor the number of reallocated and pending
blocks, run short tests often and long tests about once a week. This
should give you at least some hint, if a drive may go down. And if it
does unexpectedly, you should be covered by RAID6.
Cheers
Carsten
[-- Attachment #2: disk-realloc-poweron.gz --]
[-- Type: application/gzip, Size: 9866 bytes --]
^ permalink raw reply
* Re: When do you replace old hard drives in a raid6?
From: Phil Turmel @ 2016-03-07 5:18 UTC (permalink / raw)
To: Ram Ramesh, Linux Raid
In-Reply-To: <56DCD0E9.8040506@gmail.com>
On 03/06/2016 07:52 PM, Ram Ramesh wrote:
> On 03/06/2016 06:29 PM, Phil Turmel wrote:
>> On 03/05/2016 03:49 PM, Ram Ramesh wrote:
>>> I am curious if people actually replace hard drives periodically because
>>> they are old or out of warranty. My 5 device raid6 has several older
>>> drives (3/5 are 3+ years old and out of warranty) They seem fine with
>>> SMART and raid scrubs. However, it makes me wonder when they will die.
>>> What is the best policy in such situations? More importantly, do people
>>> wait for disks to die and then replace or have some ad hoc schedule of
>>> replacing (like every 6mo replace oldest) to keep things safe?
>> I replace drives when their relocation count hits double digits. In my
>> limited sample, that's typically after 40,000 hours.
>>
>> Phil
>
> Thanks for the data point. 40K hours means roughly 4.5 years with 24/7.
> That is very good. You use enterprise drives? Mine are desktop (and may
> be one HGST NAS)
I moved from desktop drives to NAS drives about 4 years ago. So the
40k+ hours were on desktop drives. (A couple started dying in the mid
30,000's, but I suspect I overheated those two.) The oldest NAS drives
I have now are approaching 40k, and are all still @ zero relocations.
WD Reds, fwiw.
> My SMART is perfect except for power on hours. I am going to take it
> easy for now as I have a spare (not part of a RAID) just in case
> something bad happens.
Yes, sounds reasonable.
Phil
^ permalink raw reply
* Re: Caching raid with SSD.
From: Ram Ramesh @ 2016-03-07 4:45 UTC (permalink / raw)
To: John Stoffel; +Cc: Linux Raid
In-Reply-To: <22236.57403.411891.23182@quad.stoffel.home>
On 03/06/2016 07:58 PM, John Stoffel wrote:
> Ram> Any one here actually use SSD caches for RAID arrays? Can you
> Ram> share your experience and let me know your choice of the type of
> Ram> cache methods your tired/used and why you think one is better or
> Ram> worse than other? If it is possible, please provide raid
> Ram> type/size and ssd size used.
>
> I'm using a pair of 4Tb drives mirrored, and a pair of 512gb SSDs,
> also mirrored, along with lvmcache to setup my caching across a couple
> of volumes.
>
> I honestly haven't seen huge improvements, but I also haven't had the
> time to do any serious testing either. Whcih I should do. I've been
> sorta thinking that using the Phoronix testing stuff would be the way
> to go.
>
> My SSDs and 4Tb drives are all on an LSI 8-port SATA controller, PCI-E
> 4x I think. It's an MPT SAS-2 controller. I did this so that my boot
> drives are some partitions on the SSDs, and then I use two more
> mirrored partitions for the cache.
>
> And this is an NFS server for my home directories, etc.
>
> I didn't use bcache because you can't remove a cache device without
> rebooting, or at least bringing a device offline and back online,
> which doesn't fit my desires to be able to dynamically add/remove
> caches, esp for the testing I've never bothered to do.
>
> John
I do not have lvm, but already have a live (regular) file system. While
I can accept downtime, I cannot accept formatting drives/disks. I simply
do not have the extra space to copy back and forth. That is why I
thought of dmcache. I ran a fio experiment on my ssd (old curial M4) and
I am getting 6K (random) IOPs whereas my raid gives me about 1.5K. I
really do not see much point unless my new SSD puts out some decent
numbers stand alone.
Thanks for sharing the details of your setup.
Ramesh
^ permalink raw reply
* Re: When do you replace old hard drives in a raid6?
From: Ram Ramesh @ 2016-03-07 4:40 UTC (permalink / raw)
To: Weedy; +Cc: Phil Turmel, Linux Raid
In-Reply-To: <CAFE24U3in=jKonie7i_tG-kP=8xHQufPaJwHdJAR0vYfAzLBXg@mail.gmail.com>
On 03/06/2016 08:31 PM, Weedy wrote:
> On Sun, Mar 6, 2016 at 7:52 PM, Ram Ramesh <rramesh2400@gmail.com> wrote:
>> On 03/06/2016 06:29 PM, Phil Turmel wrote:
>>> On 03/05/2016 03:49 PM, Ram Ramesh wrote:
>>>> I am curious if people actually replace hard drives periodically because
>>>> they are old or out of warranty. My 5 device raid6 has several older
>>>> drives (3/5 are 3+ years old and out of warranty) They seem fine with
>>>> SMART and raid scrubs. However, it makes me wonder when they will die.
>>>> What is the best policy in such situations? More importantly, do people
>>>> wait for disks to die and then replace or have some ad hoc schedule of
>>>> replacing (like every 6mo replace oldest) to keep things safe?
>>> I replace drives when their relocation count hits double digits. In my
>>> limited sample, that's typically after 40,000 hours.
>>>
>>> Phil
>>
>> Thanks for the data point. 40K hours means roughly 4.5 years with 24/7. That
>> is very good. You use enterprise drives?
> They don't have to be, cheap crap can last. My case slots the drives
> in vertically with large silicone dampers, I feel like this helps.
>
> Model Family: Seagate Barracuda 7200.10
> Device Model: ST3320620AS
> 5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail
> Always - 0
> 9 Power_On_Hours 0x0032 040 040 000 Old_age
> Always - 52576
>
> Model Family: Seagate Barracuda 7200.10
> Device Model: ST3320620AS
> 5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail
> Always - 0
> 9 Power_On_Hours 0x0032 048 048 000 Old_age
> Always - 46196
>
> Model Family: Western Digital Caviar Black
> Device Model: WDC WD1001FALS-00E8B0
> 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail
> Always - 0
> 9 Power_On_Hours 0x0032 039 039 000 Old_age
> Always - 44551
>
>
> Model Family: SAMSUNG SpinPoint F1 DT
> Device Model: SAMSUNG HD103UJ
> 5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail
> Always - 10
> 9 Power_On_Hours 0x0032 087 087 000 Old_age
> Always - 67735
>
>
> Model Family: Western Digital Caviar Black
> Device Model: WDC WD1001FALS-00E8B0
> 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail
> Always - 0
> 9 Power_On_Hours 0x0032 040 040 000 Old_age
> Always - 44427
>
> Model Family: SAMSUNG SpinPoint F1 DT
> Device Model: SAMSUNG HD103UJ
> 5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail
> Always - 6
> 9 Power_On_Hours 0x0032 093 093 000 Old_age
> Always - 36570
>
>
> I would say the biggest thing is how often you get a reallocated
> sector. The Samsungs seem to get 1-3 a year, they will probably keep
> doing that until they die. Past experience with seagate tells me I'm
> going to get 10 in one day and the drive will die in a week. The WD
> will probably throw a few at a time and I'll dump them when they get
> to 10-15 sectors.
I hear you. I have not had any fails myself in my 20+ years. I have
gotten rid of them because they became too small relative what market
offered. Until recently, I had a working 8G (yes, 8G!) IBM IDE drive
from my first computer (20+ years ago) It was a keepsake. They do not
make them this tough these days. I am almost sure that a new seagate
desktop 6TB is not going to last this long.
All said and done, I think I feel I have less to worry. Thanks for
helping me to see that.
I run long selftest once a month and will watch the RAID scrubs and
SMART values. I should be ok until I see my first reallocated sector
hit. After that I will buy my replacements.
Ramesh
^ permalink raw reply
* Re: When do you replace old hard drives in a raid6?
From: Weedy @ 2016-03-07 2:31 UTC (permalink / raw)
To: Ram Ramesh; +Cc: Phil Turmel, Linux Raid
In-Reply-To: <56DCD0E9.8040506@gmail.com>
On Sun, Mar 6, 2016 at 7:52 PM, Ram Ramesh <rramesh2400@gmail.com> wrote:
> On 03/06/2016 06:29 PM, Phil Turmel wrote:
>>
>> On 03/05/2016 03:49 PM, Ram Ramesh wrote:
>>>
>>> I am curious if people actually replace hard drives periodically because
>>> they are old or out of warranty. My 5 device raid6 has several older
>>> drives (3/5 are 3+ years old and out of warranty) They seem fine with
>>> SMART and raid scrubs. However, it makes me wonder when they will die.
>>> What is the best policy in such situations? More importantly, do people
>>> wait for disks to die and then replace or have some ad hoc schedule of
>>> replacing (like every 6mo replace oldest) to keep things safe?
>>
>> I replace drives when their relocation count hits double digits. In my
>> limited sample, that's typically after 40,000 hours.
>>
>> Phil
>
>
> Thanks for the data point. 40K hours means roughly 4.5 years with 24/7. That
> is very good. You use enterprise drives?
They don't have to be, cheap crap can last. My case slots the drives
in vertically with large silicone dampers, I feel like this helps.
Model Family: Seagate Barracuda 7200.10
Device Model: ST3320620AS
5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail
Always - 0
9 Power_On_Hours 0x0032 040 040 000 Old_age
Always - 52576
Model Family: Seagate Barracuda 7200.10
Device Model: ST3320620AS
5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail
Always - 0
9 Power_On_Hours 0x0032 048 048 000 Old_age
Always - 46196
Model Family: Western Digital Caviar Black
Device Model: WDC WD1001FALS-00E8B0
5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail
Always - 0
9 Power_On_Hours 0x0032 039 039 000 Old_age
Always - 44551
Model Family: SAMSUNG SpinPoint F1 DT
Device Model: SAMSUNG HD103UJ
5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail
Always - 10
9 Power_On_Hours 0x0032 087 087 000 Old_age
Always - 67735
Model Family: Western Digital Caviar Black
Device Model: WDC WD1001FALS-00E8B0
5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail
Always - 0
9 Power_On_Hours 0x0032 040 040 000 Old_age
Always - 44427
Model Family: SAMSUNG SpinPoint F1 DT
Device Model: SAMSUNG HD103UJ
5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail
Always - 6
9 Power_On_Hours 0x0032 093 093 000 Old_age
Always - 36570
I would say the biggest thing is how often you get a reallocated
sector. The Samsungs seem to get 1-3 a year, they will probably keep
doing that until they die. Past experience with seagate tells me I'm
going to get 10 in one day and the drive will die in a week. The WD
will probably throw a few at a time and I'll dump them when they get
to 10-15 sectors.
^ permalink raw reply
* Re: Caching raid with SSD.
From: John Stoffel @ 2016-03-07 1:58 UTC (permalink / raw)
To: Ram Ramesh; +Cc: Linux Raid
In-Reply-To: <56DB4A5C.7020901@gmail.com>
Ram> Any one here actually use SSD caches for RAID arrays? Can you
Ram> share your experience and let me know your choice of the type of
Ram> cache methods your tired/used and why you think one is better or
Ram> worse than other? If it is possible, please provide raid
Ram> type/size and ssd size used.
I'm using a pair of 4Tb drives mirrored, and a pair of 512gb SSDs,
also mirrored, along with lvmcache to setup my caching across a couple
of volumes.
I honestly haven't seen huge improvements, but I also haven't had the
time to do any serious testing either. Whcih I should do. I've been
sorta thinking that using the Phoronix testing stuff would be the way
to go.
My SSDs and 4Tb drives are all on an LSI 8-port SATA controller, PCI-E
4x I think. It's an MPT SAS-2 controller. I did this so that my boot
drives are some partitions on the SSDs, and then I use two more
mirrored partitions for the cache.
And this is an NFS server for my home directories, etc.
I didn't use bcache because you can't remove a cache device without
rebooting, or at least bringing a device offline and back online,
which doesn't fit my desires to be able to dynamically add/remove
caches, esp for the testing I've never bothered to do.
John
^ permalink raw reply
* Re: When do you replace old hard drives in a raid6?
From: Ram Ramesh @ 2016-03-07 0:52 UTC (permalink / raw)
To: Phil Turmel, Linux Raid
In-Reply-To: <56DCCB55.1060906@turmel.org>
On 03/06/2016 06:29 PM, Phil Turmel wrote:
> On 03/05/2016 03:49 PM, Ram Ramesh wrote:
>> I am curious if people actually replace hard drives periodically because
>> they are old or out of warranty. My 5 device raid6 has several older
>> drives (3/5 are 3+ years old and out of warranty) They seem fine with
>> SMART and raid scrubs. However, it makes me wonder when they will die.
>> What is the best policy in such situations? More importantly, do people
>> wait for disks to die and then replace or have some ad hoc schedule of
>> replacing (like every 6mo replace oldest) to keep things safe?
> I replace drives when their relocation count hits double digits. In my
> limited sample, that's typically after 40,000 hours.
>
> Phil
Thanks for the data point. 40K hours means roughly 4.5 years with 24/7.
That is very good. You use enterprise drives? Mine are desktop (and may
be one HGST NAS)
My SMART is perfect except for power on hours. I am going to take it
easy for now as I have a spare (not part of a RAID) just in case
something bad happens.
Ramesh
^ permalink raw reply
* Re: When do you replace old hard drives in a raid6?
From: Phil Turmel @ 2016-03-07 0:29 UTC (permalink / raw)
To: Ram Ramesh, Linux Raid
In-Reply-To: <56DB465E.9010703@gmail.com>
On 03/05/2016 03:49 PM, Ram Ramesh wrote:
> I am curious if people actually replace hard drives periodically because
> they are old or out of warranty. My 5 device raid6 has several older
> drives (3/5 are 3+ years old and out of warranty) They seem fine with
> SMART and raid scrubs. However, it makes me wonder when they will die.
> What is the best policy in such situations? More importantly, do people
> wait for disks to die and then replace or have some ad hoc schedule of
> replacing (like every 6mo replace oldest) to keep things safe?
I replace drives when their relocation count hits double digits. In my
limited sample, that's typically after 40,000 hours.
Phil
^ permalink raw reply
* Re: [PATCH] raid1: include bio_end_io_list in nr_queued to prevent freeze_array hang
From: Shaohua Li @ 2016-03-06 23:33 UTC (permalink / raw)
To: Nate Dailey; +Cc: linux-raid
In-Reply-To: <1456760638-23936-1-git-send-email-nate.dailey@stratus.com>
On Mon, Feb 29, 2016 at 10:43:58AM -0500, Nate Dailey wrote:
> If raid1d is handling a mix of read and write errors, handle_read_error's
> call to freeze_array can get stuck.
>
> This can happen because, though the bio_end_io_list is initially drained,
> writes can be added to it via handle_write_finished as the retry_list
> is processed. These writes contribute to nr_pending but are not included
> in nr_queued.
>
> If a later entry on the retry_list triggers a call to handle_read_error,
> freeze array hangs waiting for nr_pending == nr_queued+extra. The writes
> on the bio_end_io_list aren't included in nr_queued so the condition will
> never be satisfied.
>
> To prevent the hang, include bio_end_io_list writes in nr_queued.
>
> There's probably a better way to handle decrementing nr_queued, but this
> seemed like the safest way to avoid breaking surrounding code.
>
> I'm happy to supply the script I used to repro this hang.
Looks good. Could you please also fix raid10?
Thanks,
Shaohua
^ permalink raw reply
* Re: [PATCH RESEND v2 11/18] fs: Ensure the mounter of a filesystem is privileged towards its inodes
From: Eric W. Biederman @ 2016-03-06 22:07 UTC (permalink / raw)
To: Seth Forshee
Cc: Alexander Viro, Serge Hallyn, Richard Weinberger,
Austin S Hemmelgarn, Miklos Szeredi, linux-kernel, linux-bcache,
dm-devel, linux-raid, linux-mtd, linux-fsdevel, fuse-devel,
linux-security-module, selinux
In-Reply-To: <20160306154832.GA13631@ubuntu-xps13>
Seth Forshee <seth.forshee@canonical.com> writes:
> On Fri, Mar 04, 2016 at 04:43:06PM -0600, Eric W. Biederman wrote:
>> Seth Forshee <seth.forshee@canonical.com> writes:
>>
>> > On Mon, Jan 04, 2016 at 12:03:50PM -0600, Seth Forshee wrote:
>> >> The mounter of a filesystem should be privileged towards the
>> >> inodes of that filesystem. Extend the checks in
>> >> inode_owner_or_capable() and capable_wrt_inode_uidgid() to
>> >> permit access by users priviliged in the user namespace of the
>> >> inode's superblock.
>> >
>> > Eric - I've discovered a problem related to this patch. The patches
>> > you've already applied to your testing branch make it so that s_user_ns
>> > can be an unprivileged user for proc and kernfs-based mounts. In some
>> > cases DAC is the only thing protecting files in these mounts (ignoring
>> > MAC), and with this patch an unprivileged user could bypass DAC.
>> >
>> > There's a simple solution - always set s_user_ns to &init_user_ns for
>> > those filesystems. I think this is the right thing to do, since the
>> > backing store behind these filesystems are really kernel objects. But
>> > this would break the assumption behind your patch "userns: Simpilify
>> > MNT_NODEV handling" and cause a regression in mounting behavior.
>> >
>> > I've come up with several possible solutions for this conflict.
>> >
>> > 1. Drop this patch and keep on setting s_user_ns to unprivilged users.
>> > This would be unfortunate because I think this patch does make sense
>> > for most filesystems.
>> > 2. Restrict this patch so that a user privileged towards s_user_ns is
>> > only privileged towards the super blocks inodes if s_user_ns has a
>> > mapping for both i_uid and i_gid. This is better than (1) but still
>> > not ideal in my mind.
>> > 3. Drop your patch and maintain the current MNT_NODEV behavior.
>> > 4. Add a new s_iflags flag to indicate a super block is from an
>> > unprivileged mount, and use this in your patch instead of s_user_ns.
>> >
>> > Any preference, or any other ideas?
>>
>> In general this is only an issue if uids and gids on the filesystem
>> do not map into the user namespace.
>
> Yes, both capable_wrt_inode_uidgid and inode_owner_or_capable will
> return true for a privileged user in the current namespace if the ids
> map into that namespace.
>
>> Therefore the general fix is to limit the logic of checking for
>> capabilities in s_user_ns if we are dealing with INVALID_UID and
>> INVALID_GID. For proc and kernfs that should never be the case
>> so the problem becomes a non-issue.
>>
>> Further I would look at limiting that relaxation to just
>> inode_change_ok. So that we can easily wrap that check per filesystem
>> and deny the relaxation for proc and kernfs. proc and kernfs already
>> have wrappers for .setattr so denying changes when !uid_vaid and
>> !gid_valid would be a trivial addition, and ensure calamity does
>> not ensure.
>>
>> Furthmore by limiting any additional to inode_change_ok we keep
>> the work of the additional tests off of the fast paths.
>
> So then the inode would need to be chowned before a privileged user in a
> non-init namespace would be capable towards it. That seems workable. It
> looks like INVALID_UID and INVALID_GID do map into init_user_ns (which
> seems a bit odd) so real root remains capable towards those indoes.
>
> That seems okay to me then.
If I was not clear I was suggesting that we allow a sufficiently
privileged user in the filesysteme's s_user_ns to allow chowning files
with INVALID_UID and INVALID_GID.
The global root user would always be able to do that because unless
capabilities are dropped it is sufficiently privileged in ever user
namespace.
Eric
^ permalink raw reply
* Import/recover RAID6 created with SAS2108?
From: Dan Russell @ 2016-03-06 21:37 UTC (permalink / raw)
To: linux-raid
I have a RAID6 array which was created on a Supermicro card built around the LSI 2108. Two drives failed, I replaced one and started a rebuild, then two more failed, making my array go offline.
It looks like the failure was due to a cabling or backplane problem, and I suspect the data on the drives is good. However, the Supermicro card is unable to read the foreign configs (it thinks there are 3 foreign configs, none of which are importable into a working array).
I would like to attempt recovery per the Wiki’s instructions (overlay files, force assembly, etc); can md read a RAID6 created by the 2108, or would I be wasting my time?
I have moved the drives from the hardware RAID card to a non-RAID SAS card and can see all drives. However, mdadm -E on these drives seems to show the RAID controller’s field of vision (how many defined arrays, total number of physical disks in the system), not any insight into this drive’s role as a member of an array. I can paste or gist the output of madam -E (304 lines) if it would help.
Thanks in advance-
Dan
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox