* [PATCH] md/raid5: Compare apples to apples (or sectors to sectors)
From: Jes.Sorensen @ 2016-02-16 21:44 UTC (permalink / raw)
To: shli; +Cc: linux-raid
From: Jes Sorensen <Jes.Sorensen@redhat.com>
'max_discard_sectors' is in sectors, while 'stripe' is in bytes.
This fixes the problem where DISCARD would get disabled on some larger
RAID5 configurations (6 or more drives in my testing), while it worked
as expected with smaller configurations.
Fixes: 620125f2bf8 ("MD: raid5 trim support")
Signed-off-by: Jes Sorensen <Jes.Sorensen@redhat.com>
---
This bug has been present since RAID5 DISCARD was introduced, so it applies
to stable as well.
drivers/md/raid5.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index b4f02c9..7f770b0 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -7014,8 +7014,8 @@ static int raid5_run(struct mddev *mddev)
}
if (discard_supported &&
- mddev->queue->limits.max_discard_sectors >= stripe &&
- mddev->queue->limits.discard_granularity >= stripe)
+ mddev->queue->limits.max_discard_sectors >= (stripe >> 9) &&
+ mddev->queue->limits.discard_granularity >= stripe)
queue_flag_set_unlocked(QUEUE_FLAG_DISCARD,
mddev->queue);
else
--
1.8.3.1
^ permalink raw reply related
* Re: [PATCH 2/2] Manage: Inform udev about device removal when stopping
From: Jes Sorensen @ 2016-02-16 22:02 UTC (permalink / raw)
To: NeilBrown
Cc: Hannes Reinecke, Sebastian Parschauer, linux-raid, Shaohua Li,
Brassow Jonathan, Artur Paszkiewicz, systemd-devel
In-Reply-To: <8737sstmbh.fsf@notabene.neil.brown.name>
NeilBrown <neilb@suse.com> writes:
> On Wed, Feb 17 2016, Jes Sorensen wrote:
>
>> Hannes Reinecke <hare@suse.de> writes:
>>> On 02/16/2016 07:03 PM, Sebastian Parschauer wrote:
>>>> The worst thing that can happen is that the kernel sends the change
>>>> event after the remove event. Then it is the current situation again.
>>>> From my tests mdadm does enough other stuff in between. Udev is able to
>>>> handle receiving two remove events from my testing. Multiple mdadm
>>>> instances can't run in parallel any ways. So userspace around it needs
>>>> some serialization for it any ways. So also stopping an MD device and
>>>> assembling a new one with the same minor number shouldn't race.
>>>>
>>>> I still prefer this solution here. But if you decide to drop the udev
>>>> event sending in mdadm, then I'm also fine with that.
>>>>
>>> I strongly prefer removing the udev event generation altogether.
>>> As this appears to be a carry-over from older kernels, it looks to me
>>> as being an incomplete conversion:
>>> At one point udev introduced 'ONLINE' and 'OFFLINE' events, which were
>>> supposed to be used for this kind of scenario.
>>> (ONLINE being a companion to 'ADD', and 'OFFLINE' being the companion
>>> to 'DELETE'). However, later the 'ONLINE' got modified to 'CHANGE',
>>> and the 'OFFLINE' got dropped completely.
>>> Or that was the plan.
>>> So it looks as if the conversion to 'CHANGE' got applied to the
>>> 'OFFLINE' event, too.
>>> Hence I strongly recommend to drop it completely, and let the kernel
>>> or the MD module decide if and when a uevent should be send.
>>
>> I am totally fine with this, however we should make mdadm fail if run
>> against a pre-2.6.28 kernel then.
>>
>> Cheers,
>> Jes
>
> I would suggest protecting the
>
> if (fd >= 0)
> ioctl(fd, BLKRRPART, 0);
> if (mdi)
> sysfs_uevent(mdi, "change");
>
> code with
>
> if (get_linux_version() < 2006028)
>
> That should be completely safe - 2.6.28 and later do this (if needed).
Seems a better fix to me. I much prefer the duplicated events.
Sebastian, does this patch resolve the problem for you? If nobody
hollors, I will push this into mdadm.
Cheers,
Jes
commit 7856fa44b8f0bc217a6bbcb5f7c51b2f03717655
Author: Jes Sorensen <Jes.Sorensen@redhat.com>
Date: Tue Feb 16 16:58:36 2016 -0500
Manage.c: Only issue change events for kernels older then 2.6.28
2.6.28+ kernels handle this themselves and issuing the event here can
cause a race.
Reported-by: Sebastian Parschauer <sebastian.riemer@profitbricks.com>
Suggested-by: NeilBrown <neilb@suse.com>
Signed-off-by: Jes Sorensen <Jes.Sorensen@redhat.com>
diff --git a/Manage.c b/Manage.c
index 7e1b94b..eae96e1 100644
--- a/Manage.c
+++ b/Manage.c
@@ -493,14 +493,17 @@ done:
rv = 1;
goto out;
}
- /* prior to 2.6.28, KOBJ_CHANGE was not sent when an md array
- * was stopped, so We'll do it here just to be sure. Drop any
- * partitions as well...
- */
- if (fd >= 0)
- ioctl(fd, BLKRRPART, 0);
- if (mdi)
- sysfs_uevent(mdi, "change");
+
+ if (get_linux_version() < 2006028) {
+ /* prior to 2.6.28, KOBJ_CHANGE was not sent when an md array
+ * was stopped, so We'll do it here just to be sure. Drop any
+ * partitions as well...
+ */
+ if (fd >= 0)
+ ioctl(fd, BLKRRPART, 0);
+ if (mdi)
+ sysfs_uevent(mdi, "change");
+ }
if (devnm[0] && use_udev()) {
struct map_ent *mp = map_by_devnm(&map, devnm);
^ permalink raw reply related
* Re: [PATCH] md/raid5: Compare apples to apples (or sectors to sectors)
From: Shaohua Li @ 2016-02-16 22:06 UTC (permalink / raw)
To: Jes.Sorensen; +Cc: linux-raid
In-Reply-To: <1455659064-15940-1-git-send-email-Jes.Sorensen@redhat.com>
On Tue, Feb 16, 2016 at 04:44:24PM -0500, Jes.Sorensen@redhat.com wrote:
> From: Jes Sorensen <Jes.Sorensen@redhat.com>
>
> 'max_discard_sectors' is in sectors, while 'stripe' is in bytes.
>
> This fixes the problem where DISCARD would get disabled on some larger
> RAID5 configurations (6 or more drives in my testing), while it worked
> as expected with smaller configurations.
>
> Fixes: 620125f2bf8 ("MD: raid5 trim support")
> Signed-off-by: Jes Sorensen <Jes.Sorensen@redhat.com>
Applied, thanks!
^ permalink raw reply
* Salve
From: hi @ 2016-02-17 4:53 UTC (permalink / raw)
To: eman
Ciao
tv, dj, bici, orologio, chitarra ... spedizione è € 0
iphone6s, 368 €
si te: elecdigitalvip. com
Se non hai ricevuto la tua email. Si prega di rispondere a noi, non saremo mai inviare e-mail a voi
^ permalink raw reply
* Re: [PATCH 2/2] Manage: Inform udev about device removal when stopping
From: Hannes Reinecke @ 2016-02-17 7:03 UTC (permalink / raw)
To: NeilBrown, Jes Sorensen
Cc: Sebastian Parschauer, linux-raid, Shaohua Li, Brassow Jonathan,
Artur Paszkiewicz, systemd-devel
In-Reply-To: <8737sstmbh.fsf@notabene.neil.brown.name>
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 02/16/2016 09:46 PM, NeilBrown wrote:
> On Wed, Feb 17 2016, Jes Sorensen wrote:
>
>> Hannes Reinecke <hare@suse.de> writes:
>>> On 02/16/2016 07:03 PM, Sebastian Parschauer wrote:
>>>> On 16.02.2016 18:41, Jes Sorensen wrote:
>>>>> Sebastian Parschauer
>>>>> <sebastian.riemer@profitbricks.com> writes:
>>>>>> When stopping an MD device, then its device node
>>>>>> /dev/mdX may still exist afterwards or it is
>>>>>> recreated by udev. The next open() call can lead to
>>>>>> creation of an inoperable MD device. The reason for
>>>>>> this is that a change event (KOBJ_CHANGE) is
>>>>>> announced to udev. So announce a removal event
>>>>>> (KOBJ_REMOVE) to udev instead.
>>>>>>
>>>>>> This also overrides the change event sent by the
>>>>>> kernel.
>>>>>>
>>>>>> Signed-off-by: Sebastian Parschauer
>>>>>> <sebastian.riemer@profitbricks.com> --- Manage.c |
>>>>>> 6 +++--- 1 file changed, 3 insertions(+), 3
>>>>>> deletions(-)
>>>>>>
>>>>>> diff --git a/Manage.c b/Manage.c index
>>>>>> 7e1b94b..bc89764 100644 --- a/Manage.c +++
>>>>>> b/Manage.c @@ -494,13 +494,13 @@ done: goto out; } /*
>>>>>> prior to 2.6.28, KOBJ_CHANGE was not sent when an md
>>>>>> array - * was stopped, so We'll do it here just to
>>>>>> be sure. Drop any - * partitions as well... + *
>>>>>> was stopped, it should be KOBJ_REMOVE instead, so we
>>>>>> set the + * remove event here just to be sure. Drop
>>>>>> any partitions as well... */ if (fd >= 0) ioctl(fd,
>>>>>> BLKRRPART, 0); if (mdi) - sysfs_uevent(mdi,
>>>>>> "change"); + sysfs_uevent(mdi, "remove");
>>>>>
>>>>> I am a little concerned about this change. You assume
>>>>> the kernel and mdadm will be updated in sync, which is
>>>>> unlikely to happen. I believe you need to match the
>>>>> kernel version and send the corresponding event
>>>>> currectly for this to work correctly?
>>>>
>>>> The worst thing that can happen is that the kernel sends
>>>> the change event after the remove event. Then it is the
>>>> current situation again. From my tests mdadm does enough
>>>> other stuff in between. Udev is able to handle receiving
>>>> two remove events from my testing. Multiple mdadm
>>>> instances can't run in parallel any ways. So userspace
>>>> around it needs some serialization for it any ways. So
>>>> also stopping an MD device and assembling a new one with
>>>> the same minor number shouldn't race.
>>>>
>>>> I still prefer this solution here. But if you decide to
>>>> drop the udev event sending in mdadm, then I'm also fine
>>>> with that.
>>>>
>>> I strongly prefer removing the udev event generation
>>> altogether. As this appears to be a carry-over from older
>>> kernels, it looks to me as being an incomplete conversion:
>>> At one point udev introduced 'ONLINE' and 'OFFLINE' events,
>>> which were supposed to be used for this kind of scenario.
>>> (ONLINE being a companion to 'ADD', and 'OFFLINE' being the
>>> companion to 'DELETE'). However, later the 'ONLINE' got
>>> modified to 'CHANGE', and the 'OFFLINE' got dropped
>>> completely. Or that was the plan. So it looks as if the
>>> conversion to 'CHANGE' got applied to the 'OFFLINE' event,
>>> too. Hence I strongly recommend to drop it completely, and
>>> let the kernel or the MD module decide if and when a uevent
>>> should be send.
>>
>> I am totally fine with this, however we should make mdadm
>> fail if run against a pre-2.6.28 kernel then.
>>
>> Cheers, Jes
>
> I would suggest protecting the
>
> if (fd >= 0) ioctl(fd, BLKRRPART, 0); if (mdi)
> sysfs_uevent(mdi, "change");
>
> code with
>
> if (get_linux_version() < 2006028)
>
> That should be completely safe - 2.6.28 and later do this (if
> needed).
>
+1.
Yes, this is the best solution.
Cheers,
Hannes
- --
Dr. Hannes Reinecke Teamlead Storage & Networking
hare@suse.de +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iQIcBAEBAgAGBQJWxBs6AAoJEGz4yi9OyKjP8a0P/RqXuKmFFOcBXnST1f1ZWq24
gExfzeo8VAQicWi/CBFu+lumePOiypzKfP0NfHw4PDGPYuLQQq6OXRmiQCqhWz/p
EBRZ9a8NyUIjUpra2j66IiMGwh1KGYl9AeIZmvGTNVkNcOySKdJxLWkFnI6wLwvU
YmUez04UFxEleynt5c00ZYvioYnVchVDNEc4/8yTG6jAUg4+6Q7tTw8vvR3wko+K
vmDbUyUz+q8R5tyTjCKB/KWgMPnMUv+wYoZx+jWtpRyUO6a76U9T5if+ZKk4EUkb
NBHy76L6/YxvOhJqAuX9dMiPDADgHVzD5mnzTSzt9HF/ArXBXtEMcaMxhLPYpLTU
lY7FQqzQ5sGQKbo0nm3EHrLjIP1bWe3BKaniyFQG39wlzdhhFGCzJzHnl1KwSnu6
gy+/AuQSibvxUQhD/ZO4+AJjMq1sXLwRlwwPFr/pI8wrcIqFIUuZG0JjY9NY2UQ2
+povgSj4UnXpRS7BKjvmN/VyUIbnXzf8cpB8w2qwxI5nSwKgjhSNz+o3NeCDQqpw
u2E0MIciPDKXb2GnfPA2+Depm8VfcL9uaRNbHmnV9shIlRsQLB9/IzzVA5Cf5T3f
GA9pHLKEM2LHAWqPmUVIghzyTTj5CXsZrH2GdJVyNTc1bymDBKmQbbiO+IVIHg45
XnJ7L15O6ZTXEW1WMNYT
=okXv
-----END PGP SIGNATURE-----
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH 2/2] Manage: Inform udev about device removal when stopping
From: Sebastian Parschauer @ 2016-02-17 10:31 UTC (permalink / raw)
To: Jes Sorensen, NeilBrown
Cc: Hannes Reinecke, linux-raid, Shaohua Li, Brassow Jonathan,
Artur Paszkiewicz, systemd-devel
In-Reply-To: <wrfjmvr09uvi.fsf@redhat.com>
[-- Attachment #1: Type: text/plain, Size: 4516 bytes --]
On 16.02.2016 23:02, Jes Sorensen wrote:
> NeilBrown <neilb@suse.com> writes:
>> On Wed, Feb 17 2016, Jes Sorensen wrote:
>>
>>> Hannes Reinecke <hare@suse.de> writes:
>>>> On 02/16/2016 07:03 PM, Sebastian Parschauer wrote:
>>>>> The worst thing that can happen is that the kernel sends the change
>>>>> event after the remove event. Then it is the current situation again.
>>>>> From my tests mdadm does enough other stuff in between. Udev is able to
>>>>> handle receiving two remove events from my testing. Multiple mdadm
>>>>> instances can't run in parallel any ways. So userspace around it needs
>>>>> some serialization for it any ways. So also stopping an MD device and
>>>>> assembling a new one with the same minor number shouldn't race.
>>>>>
>>>>> I still prefer this solution here. But if you decide to drop the udev
>>>>> event sending in mdadm, then I'm also fine with that.
>>>>>
>>>> I strongly prefer removing the udev event generation altogether.
>>>> As this appears to be a carry-over from older kernels, it looks to me
>>>> as being an incomplete conversion:
>>>> At one point udev introduced 'ONLINE' and 'OFFLINE' events, which were
>>>> supposed to be used for this kind of scenario.
>>>> (ONLINE being a companion to 'ADD', and 'OFFLINE' being the companion
>>>> to 'DELETE'). However, later the 'ONLINE' got modified to 'CHANGE',
>>>> and the 'OFFLINE' got dropped completely.
>>>> Or that was the plan.
>>>> So it looks as if the conversion to 'CHANGE' got applied to the
>>>> 'OFFLINE' event, too.
>>>> Hence I strongly recommend to drop it completely, and let the kernel
>>>> or the MD module decide if and when a uevent should be send.
>>>
>>> I am totally fine with this, however we should make mdadm fail if run
>>> against a pre-2.6.28 kernel then.
>>>
>>> Cheers,
>>> Jes
>>
>> I would suggest protecting the
>>
>> if (fd >= 0)
>> ioctl(fd, BLKRRPART, 0);
>> if (mdi)
>> sysfs_uevent(mdi, "change");
>>
>> code with
>>
>> if (get_linux_version() < 2006028)
>>
>> That should be completely safe - 2.6.28 and later do this (if needed).
>
> Seems a better fix to me. I much prefer the duplicated events.
>
> Sebastian, does this patch resolve the problem for you? If nobody
> hollors, I will push this into mdadm.
I've tested this on top of mdadm-3.4 and with vanilla kernel 4.5.0-rc3.
The device nodes are still there. I guess this is because of the change
event from the kernel. This confirms my former tests.
Run 1, 2: one of two device nodes gone
Run 3: both device nodes gone
Run 4: both device nodes are there (again)
It only works if the kernel sends the remove event instead of the change
event as well. So I'm only fine with this if the kernel change gets
accepted as well.
I create two MD devices (md0 and md1) on top of LVM. Then I check if the
creation worked, stop the devices and check the devices again. For a
clean state afterwards, I clean up everything and check devices again in
my script. I have 5s sleep between each action to get final states.
I've attached my test script.
> commit 7856fa44b8f0bc217a6bbcb5f7c51b2f03717655
> Author: Jes Sorensen <Jes.Sorensen@redhat.com>
> Date: Tue Feb 16 16:58:36 2016 -0500
>
> Manage.c: Only issue change events for kernels older then 2.6.28
Shouldn't the prefix be "Manage:" or "Manage/stop:"? Here is a typo.
Should be "than" instead of "then" here.
>
> 2.6.28+ kernels handle this themselves and issuing the event here can
> cause a race.
>
> Reported-by: Sebastian Parschauer <sebastian.riemer@profitbricks.com>
> Suggested-by: NeilBrown <neilb@suse.com>
> Signed-off-by: Jes Sorensen <Jes.Sorensen@redhat.com>
>
> diff --git a/Manage.c b/Manage.c
> index 7e1b94b..eae96e1 100644
> --- a/Manage.c
> +++ b/Manage.c
> @@ -493,14 +493,17 @@ done:
> rv = 1;
> goto out;
> }
> - /* prior to 2.6.28, KOBJ_CHANGE was not sent when an md array
> - * was stopped, so We'll do it here just to be sure. Drop any
> - * partitions as well...
> - */
> - if (fd >= 0)
> - ioctl(fd, BLKRRPART, 0);
> - if (mdi)
> - sysfs_uevent(mdi, "change");
> +
> + if (get_linux_version() < 2006028) {
> + /* prior to 2.6.28, KOBJ_CHANGE was not sent when an md array
> + * was stopped, so We'll do it here just to be sure. Drop any
> + * partitions as well...
> + */
> + if (fd >= 0)
> + ioctl(fd, BLKRRPART, 0);
> + if (mdi)
> + sysfs_uevent(mdi, "change");
> + }
>
> if (devnm[0] && use_udev()) {
> struct map_ent *mp = map_by_devnm(&map, devnm);
>
[-- Attachment #2: test_mdadm_stop.sh.txt --]
[-- Type: text/plain, Size: 836 bytes --]
#!/bin/bash
NUM_VOL=2
remove_devices() {
for ((i=0;i<${NUM_VOL};i++)); do
rm -fv /dev/md${i}
done
}
check_devices() {
ls -l /dev/disk/by-id/md*
ls -d /dev/md*
ls -d /sys/block/md*
cat /proc/mdstat
}
stop_devices() {
for ((i=0;i<${NUM_VOL};i++)); do
mdadm --stop /dev/md${i} 2>&1
done
}
create_devices() {
for ((i=0;i<${NUM_VOL};i++)); do
mdadm --zero-superblock /dev/dm-${i}
mdadm -C /dev/md${i} -l 1 -n 2 -e 1.2 \
--run /dev/dm-${i} missing
done
}
####### MAIN #######
create_devices
sleep 5
echo "==== CREATE CHECK ===="
check_devices
sleep 5
stop_devices
sleep 5
echo "==== CHECK 1 ===="
check_devices
stop_devices
remove_devices
sleep 5
echo "==== CHECK 2 ===="
check_devices
^ permalink raw reply
* Re: [PATCH 1/2] md: Inform udev about device removal when stopping
From: Sebastian Parschauer @ 2016-02-17 11:24 UTC (permalink / raw)
To: NeilBrown, Shaohua Li, linux-raid
Cc: Jes Sorensen, Brassow Jonathan, Artur Paszkiewicz, systemd-devel
In-Reply-To: <8760xotmi0.fsf@notabene.neil.brown.name>
On 16.02.2016 21:43, NeilBrown wrote:
> On Wed, Feb 17 2016, Shaohua Li wrote:
>
>> On Tue, Feb 16, 2016 at 03:44:36PM +0100, Sebastian Parschauer wrote:
>>> When stopping an MD device, then its device node /dev/mdX may still
>>> exist afterwards or it is recreated by udev. The next open() call
>>> can lead to creation of an inoperable MD device. The reason for
>>> this is that a change event (KOBJ_CHANGE) is announced to udev.
>>> So announce a removal event (KOBJ_REMOVE) to udev instead.
>>>
>>> A change is likely also required in mdadm because of the support
>>> for kernels prior to 2.6.28.
>>
>> I didn't follow why we need the change. Shouldn't the KOBJ_REMOVE event be sent
>> automatically when gendisk is deleted?
>> mddev_put()->mddev_delayed_delete()->md_free()->del_gendisk().
>>
>> Thanks,
>> Shaohua
>
> For a bit of context: this KOBJ_CHANGE event was added in Oct 2008
>
> Commit: 934d9c23b4c7 ("md: destroy partitions and notify udev when md array is stopped.")
>
> At the time, md devices weren't getting removed at all.
> Now they are (I figured out the locking), though they can still come
> back.
>
> There are still two stages. The array is stopped, and then the block
> device is destroyed. It is theoretically possible to stop the array
> without destroying the block device, though I don't think that happens
> in practice.
>
> So this KOBJ_CHANGE is, I think, technically correct (change from
> "active" to "inactive") but probably isn't needed any more - not to the
> extent it was at the time.
>
> There are some annoying races with caused by udev responding (belatedly)
> to events by running programs that open s/dev/mdXX and so automatically
> re-creates the md device.
> The real problem here is not the event or the delays in udev. It is the
> fact that opening /dev/mdXX transparently creates a device.
>
> The only way (I know of) to really avoid these races is to use named
> arrays.
> Put
> CREATE names=yes
>
> in mdadm.conf. Then md arrays will be created by writing a name to a
> magic file in /sys. The arrays have a minor number >=512 and are not
> auto-re-created if the device node is re-opened before udev unlinks it.
>
> So: the patch might be safe, and might solve a particular problem, but
> it is really just a bandaid. The best fix is "CREATE named=yes" (and
> use named like "md_home", not "md4".
Older mdadm versions like 3.2.6 have really bad scaling issues as they
search the whole /dev directory with map_dev() for the correct device
and we've hit further issues with the symlinks in /dev/md/. This is why
we've decided to go for the /dev/mdX devices directly as then also the
minor number is clear.
I remember custom commits:
* dev_open: add parameter 'do_map_dev'
* mdopen: don't do 'map_dev' in 'create_mddev' if devname is /dev/mdX
I did a further test: If mdadm and the kernel don't send any uevent when
stopping, then it also works. Might be the best solution.
Cheers,
Sebastian
^ permalink raw reply
* Re: [PATCH 2/2] Manage: Inform udev about device removal when stopping
From: Jes Sorensen @ 2016-02-17 13:06 UTC (permalink / raw)
To: Hannes Reinecke
Cc: NeilBrown, Sebastian Parschauer, linux-raid, Shaohua Li,
Brassow Jonathan, Artur Paszkiewicz, systemd-devel
In-Reply-To: <56C41B3A.90902@suse.de>
Hannes Reinecke <hare@suse.de> writes:
> On 02/16/2016 09:46 PM, NeilBrown wrote:
>> On Wed, Feb 17 2016, Jes Sorensen wrote:
>>>
>>> I am totally fine with this, however we should make mdadm
>>> fail if run against a pre-2.6.28 kernel then.
>>>
>>> Cheers, Jes
>>
>> I would suggest protecting the
>>
>> if (fd >= 0) ioctl(fd, BLKRRPART, 0); if (mdi)
>> sysfs_uevent(mdi, "change");
>>
>> code with
>>
>> if (get_linux_version() < 2006028)
>>
>> That should be completely safe - 2.6.28 and later do this (if
>> needed).
>>
> +1.
>
> Yes, this is the best solution.
Sebastian indicates it only works if the kernel patch he submitted is
applied too - should we tweak the mdadm version check to match the next
upstream kernel, or stick with it as is here?
Jes
^ permalink raw reply
* Re: [PATCH 2/2] Manage: Inform udev about device removal when stopping
From: Sebastian Parschauer @ 2016-02-17 13:16 UTC (permalink / raw)
To: Jes Sorensen, Hannes Reinecke
Cc: NeilBrown, linux-raid, Shaohua Li, Brassow Jonathan,
Artur Paszkiewicz, systemd-devel
In-Reply-To: <wrfj1t8ba3l7.fsf@redhat.com>
On 17.02.2016 14:06, Jes Sorensen wrote:
> Hannes Reinecke <hare@suse.de> writes:
>> On 02/16/2016 09:46 PM, NeilBrown wrote:
>>> On Wed, Feb 17 2016, Jes Sorensen wrote:
>>>>
>>>> I am totally fine with this, however we should make mdadm
>>>> fail if run against a pre-2.6.28 kernel then.
>>>>
>>>> Cheers, Jes
>>>
>>> I would suggest protecting the
>>>
>>> if (fd >= 0) ioctl(fd, BLKRRPART, 0); if (mdi)
>>> sysfs_uevent(mdi, "change");
>>>
>>> code with
>>>
>>> if (get_linux_version() < 2006028)
>>>
>>> That should be completely safe - 2.6.28 and later do this (if
>>> needed).
>>>
>> +1.
>>
>> Yes, this is the best solution.
>
> Sebastian indicates it only works if the kernel patch he submitted is
> applied too - should we tweak the mdadm version check to match the next
> upstream kernel, or stick with it as is here?
Sorry, it also works if dropping the sending of the change event in the
kernel as well. This seems to be the preferred solution so far. So for
kernels still sending the change event, the problem is not fixed this
way. But your mdadm commit also doesn't make it worse.
Cheers,
Sebastian
^ permalink raw reply
* [PATCH] md: Drop sending a change uevent when stopping
From: Sebastian Parschauer @ 2016-02-17 16:25 UTC (permalink / raw)
To: linux-raid
Cc: Sebastian Parschauer, Shaohua Li, Jes Sorensen, Brassow Jonathan,
Artur Paszkiewicz, NeilBrown, Hannes Reinecke, systemd-devel
When stopping an MD device, then its device node /dev/mdX may still
exist afterwards or it is recreated by udev. The next open() call
can lead to creation of an inoperable MD device. The reason for
this is that a change event (KOBJ_CHANGE) is sent to udev which
races against the remove event (KOBJ_REMOVE) from md_free().
So drop sending the change event.
A change is likely also required in mdadm as many versions send the
change event to udev as well.
Signed-off-by: Sebastian Parschauer <sebastian.riemer@profitbricks.com>
---
drivers/md/md.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/drivers/md/md.c b/drivers/md/md.c
index e55e6cf..464627b 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -5671,7 +5671,6 @@ static int do_md_stop(struct mddev *mddev, int mode,
export_array(mddev);
md_clean(mddev);
- kobject_uevent(&disk_to_dev(mddev->gendisk)->kobj, KOBJ_CHANGE);
if (mddev->hold_active == UNTIL_STOP)
mddev->hold_active = 0;
}
--
1.7.9.5
^ permalink raw reply related
* Re: [PATCH 2/2] Manage: Inform udev about device removal when stopping
From: Jes Sorensen @ 2016-02-17 17:33 UTC (permalink / raw)
To: Sebastian Parschauer
Cc: Hannes Reinecke, NeilBrown, linux-raid, Shaohua Li,
Brassow Jonathan, Artur Paszkiewicz, systemd-devel
In-Reply-To: <56C472C5.4010602@profitbricks.com>
Sebastian Parschauer <sebastian.riemer@profitbricks.com> writes:
> On 17.02.2016 14:06, Jes Sorensen wrote:
>> Hannes Reinecke <hare@suse.de> writes:
>>> On 02/16/2016 09:46 PM, NeilBrown wrote:
>>>> On Wed, Feb 17 2016, Jes Sorensen wrote:
>>>>>
>>>>> I am totally fine with this, however we should make mdadm
>>>>> fail if run against a pre-2.6.28 kernel then.
>>>>>
>>>>> Cheers, Jes
>>>>
>>>> I would suggest protecting the
>>>>
>>>> if (fd >= 0) ioctl(fd, BLKRRPART, 0); if (mdi)
>>>> sysfs_uevent(mdi, "change");
>>>>
>>>> code with
>>>>
>>>> if (get_linux_version() < 2006028)
>>>>
>>>> That should be completely safe - 2.6.28 and later do this (if
>>>> needed).
>>>>
>>> +1.
>>>
>>> Yes, this is the best solution.
>>
>> Sebastian indicates it only works if the kernel patch he submitted is
>> applied too - should we tweak the mdadm version check to match the next
>> upstream kernel, or stick with it as is here?
>
> Sorry, it also works if dropping the sending of the change event in the
> kernel as well. This seems to be the preferred solution so far. So for
> kernels still sending the change event, the problem is not fixed this
> way. But your mdadm commit also doesn't make it worse.
Since there is pretty broad agreement on this approach, I have pushed
the fix out for mdadm.
Jes
^ permalink raw reply
* Re: [PATCH] md: Drop sending a change uevent when stopping
From: Shaohua Li @ 2016-02-17 18:19 UTC (permalink / raw)
To: Sebastian Parschauer
Cc: linux-raid, Jes Sorensen, Brassow Jonathan, Artur Paszkiewicz,
NeilBrown, Hannes Reinecke, systemd-devel
In-Reply-To: <1455726300-20340-1-git-send-email-sebastian.riemer@profitbricks.com>
On Wed, Feb 17, 2016 at 05:25:00PM +0100, Sebastian Parschauer wrote:
> When stopping an MD device, then its device node /dev/mdX may still
> exist afterwards or it is recreated by udev. The next open() call
> can lead to creation of an inoperable MD device. The reason for
> this is that a change event (KOBJ_CHANGE) is sent to udev which
> races against the remove event (KOBJ_REMOVE) from md_free().
> So drop sending the change event.
>
> A change is likely also required in mdadm as many versions send the
> change event to udev as well.
Makes sense, it's unlikely we need the CHANGE event. Applied.
Thanks,
Shaohua
^ permalink raw reply
* Re: [PATCH] md: Drop sending a change uevent when stopping
From: NeilBrown @ 2016-02-17 21:29 UTC (permalink / raw)
To: Shaohua Li, Sebastian Parschauer
Cc: linux-raid, Jes Sorensen, Brassow Jonathan, Artur Paszkiewicz,
Hannes Reinecke, systemd-devel
In-Reply-To: <20160217181943.GA28011@kernel.org>
[-- Attachment #1: Type: text/plain, Size: 1155 bytes --]
On Thu, Feb 18 2016, Shaohua Li wrote:
> On Wed, Feb 17, 2016 at 05:25:00PM +0100, Sebastian Parschauer wrote:
>> When stopping an MD device, then its device node /dev/mdX may still
>> exist afterwards or it is recreated by udev. The next open() call
>> can lead to creation of an inoperable MD device. The reason for
>> this is that a change event (KOBJ_CHANGE) is sent to udev which
>> races against the remove event (KOBJ_REMOVE) from md_free().
>> So drop sending the change event.
>>
>> A change is likely also required in mdadm as many versions send the
>> change event to udev as well.
>
> Makes sense, it's unlikely we need the CHANGE event. Applied.
>
> Thanks,
> Shaohua
It would be worth checking, but I think that with this change, you can
write
"inactive" to /sys/block/mdXXX/md/array_state
and the array will become inactive, but no uevent will be generated,
which isn't good.
Maybe send the uevent that was just removed from the 'inactive' case of
array_state_store() instead.
(But I still think this is just a bandaid and doesn't provide any
guarantees that there will be no races with udev)
NeilBrown
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]
^ permalink raw reply
* Re: [PATCH] md: Drop sending a change uevent when stopping
From: Shaohua Li @ 2016-02-17 22:14 UTC (permalink / raw)
To: NeilBrown
Cc: Sebastian Parschauer, linux-raid, Jes Sorensen, Brassow Jonathan,
Artur Paszkiewicz, Hannes Reinecke, systemd-devel
In-Reply-To: <871t8brpon.fsf@notabene.neil.brown.name>
On Thu, Feb 18, 2016 at 08:29:28AM +1100, Neil Brown wrote:
> On Thu, Feb 18 2016, Shaohua Li wrote:
>
> > On Wed, Feb 17, 2016 at 05:25:00PM +0100, Sebastian Parschauer wrote:
> >> When stopping an MD device, then its device node /dev/mdX may still
> >> exist afterwards or it is recreated by udev. The next open() call
> >> can lead to creation of an inoperable MD device. The reason for
> >> this is that a change event (KOBJ_CHANGE) is sent to udev which
> >> races against the remove event (KOBJ_REMOVE) from md_free().
> >> So drop sending the change event.
> >>
> >> A change is likely also required in mdadm as many versions send the
> >> change event to udev as well.
> >
> > Makes sense, it's unlikely we need the CHANGE event. Applied.
> >
> > Thanks,
> > Shaohua
>
> It would be worth checking, but I think that with this change, you can
> write
> "inactive" to /sys/block/mdXXX/md/array_state
> and the array will become inactive, but no uevent will be generated,
> which isn't good.
> Maybe send the uevent that was just removed from the 'inactive' case of
> array_state_store() instead.
with 'inactive', the mode == 2, do_md_stop() doesn't send the event either, so
the behavior isn't changed.
> (But I still think this is just a bandaid and doesn't provide any
> guarantees that there will be no races with udev)
that's correct. I'd expect races in other CHNAGE/REMOVE cases are very rare.
Thanks,
Shaohua
^ permalink raw reply
* mdadm regression tests, slight progress
From: Bruce Dubbs @ 2016-02-17 22:48 UTC (permalink / raw)
To: linux-raid
I'm still trying to figure out why some of the mdadm regression tests
fail. I started looking at tests/00raid1:
# ./test --tests=00raid1
It is nice that ./test is a bash script so it is easy to do some debugging.
The test script looks like:
# create a simple mirror
# test version0, version1, and no super
# test resync and recovery.
mdadm -CR $md0 -l 1 -n2 $dev0 $dev1
check resync
check raid1
testdev $md0 1 $mdsize1a 64
mdadm -S $md0
# now with version-0.90 superblock, spare
mdadm -CR $md0 -e0.90 --level=raid1 -n3 -x2 $dev0 missing missing $dev1 $dev2
check recovery
check raid1
testdev $md0 1 $mdsize0 64
mdadm -S $md0
# now with no superblock
mdadm -B $md0 -l mirror -n2 $dev0 $dev1
check resync
check raid1
testdev $md0 1 $size 1
#### Point 1
mdadm -S $md0
# again, but with no resync
mdadm -B $md0 -l 1 --assume-clean -n2 $dev0 $dev1
check raid1
check nosync
testdev $md0 1 $size 1
mdadm -S $md0
exit 0
What I did was add a couple of 'cat /proc/mdstat' commands where the
script was failing. At Point 1 above, I got:
++ cat /proc/mdstat
Personalities : [raid1] [raid10] [raid6] [raid5] [raid4] [multipath]
md127 : inactive md0[0](S)
19904 blocks
md0 : active raid1 loop1[1] loop0[0]
20000 blocks super non-persistent [2/2] [UU]
[==>..................] resync = 12.5% (2688/20000) finish=0.1min
speed=2688K/sec
The line with resync looked suspicious so at that point I added a 'sleep
10' command. The test now passes.
The full log is at
http://anduin.linuxfromscratch.org/~bdubbs/files/mdadm-test1.log
Does anyone know what is going on to need this sleep in order for this
test to pass?
-- Bruce Dubbs
^ permalink raw reply
* Re: [PATCH 1/2] md: Inform udev about device removal when stopping
From: NeilBrown @ 2016-02-17 22:57 UTC (permalink / raw)
To: Sebastian Parschauer, Shaohua Li, linux-raid
Cc: Jes Sorensen, Brassow Jonathan, Artur Paszkiewicz, systemd-devel
In-Reply-To: <56C45886.7020200@profitbricks.com>
[-- Attachment #1: Type: text/plain, Size: 3908 bytes --]
On Wed, Feb 17 2016, Sebastian Parschauer wrote:
> On 16.02.2016 21:43, NeilBrown wrote:
>> On Wed, Feb 17 2016, Shaohua Li wrote:
>>
>>> On Tue, Feb 16, 2016 at 03:44:36PM +0100, Sebastian Parschauer wrote:
>>>> When stopping an MD device, then its device node /dev/mdX may still
>>>> exist afterwards or it is recreated by udev. The next open() call
>>>> can lead to creation of an inoperable MD device. The reason for
>>>> this is that a change event (KOBJ_CHANGE) is announced to udev.
>>>> So announce a removal event (KOBJ_REMOVE) to udev instead.
>>>>
>>>> A change is likely also required in mdadm because of the support
>>>> for kernels prior to 2.6.28.
>>>
>>> I didn't follow why we need the change. Shouldn't the KOBJ_REMOVE event be sent
>>> automatically when gendisk is deleted?
>>> mddev_put()->mddev_delayed_delete()->md_free()->del_gendisk().
>>>
>>> Thanks,
>>> Shaohua
>>
>> For a bit of context: this KOBJ_CHANGE event was added in Oct 2008
>>
>> Commit: 934d9c23b4c7 ("md: destroy partitions and notify udev when md array is stopped.")
>>
>> At the time, md devices weren't getting removed at all.
>> Now they are (I figured out the locking), though they can still come
>> back.
>>
>> There are still two stages. The array is stopped, and then the block
>> device is destroyed. It is theoretically possible to stop the array
>> without destroying the block device, though I don't think that happens
>> in practice.
>>
>> So this KOBJ_CHANGE is, I think, technically correct (change from
>> "active" to "inactive") but probably isn't needed any more - not to the
>> extent it was at the time.
>>
>> There are some annoying races with caused by udev responding (belatedly)
>> to events by running programs that open s/dev/mdXX and so automatically
>> re-creates the md device.
>> The real problem here is not the event or the delays in udev. It is the
>> fact that opening /dev/mdXX transparently creates a device.
>>
>> The only way (I know of) to really avoid these races is to use named
>> arrays.
>> Put
>> CREATE names=yes
>>
>> in mdadm.conf. Then md arrays will be created by writing a name to a
>> magic file in /sys. The arrays have a minor number >=512 and are not
>> auto-re-created if the device node is re-opened before udev unlinks it.
>>
>> So: the patch might be safe, and might solve a particular problem, but
>> it is really just a bandaid. The best fix is "CREATE named=yes" (and
>> use named like "md_home", not "md4".
>
> Older mdadm versions like 3.2.6 have really bad scaling issues as they
> search the whole /dev directory with map_dev() for the correct device
> and we've hit further issues with the symlinks in /dev/md/. This is why
> we've decided to go for the /dev/mdX devices directly as then also the
> minor number is clear.
Why would anyone care about the minor number?
with 'name=yes', the entries in /dev are e.g. "md_foo" - no symlinks in
/dev (the exact same symlinked are in /dev/md).
If there are scaling issues, we should try to fix them. Please report
details.
>
> I remember custom commits:
> * dev_open: add parameter 'do_map_dev'
> * mdopen: don't do 'map_dev' in 'create_mddev' if devname is /dev/mdX
Have you posted these? Please do.
>
> I did a further test: If mdadm and the kernel don't send any uevent when
> stopping, then it also works. Might be the best solution.
I'm glad it works for your test cases, but that doesn't necessarily
means it is correct or sufficient.
I'm not exactly against removing the uevent, but I wouldn't be surprised
if that ends up causing a regression for someone who does things
differently to you.
And as I have said, I think there are other situation, maybe less
common, where udev can get bogged down and end up handling change events after
the array has been destroyed.
NeilBrown
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]
^ permalink raw reply
* [PATCH] super-intel: ensure suspended region is removed when reshape completes.
From: NeilBrown @ 2016-02-18 4:53 UTC (permalink / raw)
To: Jes Sorensen; +Cc: linux-raid, Ken Moffat, Artur Paszkiewicz
[-- Attachment #1: Type: text/plain, Size: 1857 bytes --]
A recent commit removed a call to abort_reshape() when IMSM reshape
completed. An unanticipated result of this is that the suspended
region is not cleared as it should be.
So after a reshape, a region of the array will cause all IO to block.
Re-instate the required updates to suspend_{lo,hi} coped from
abort_reshape().
This is caught (sometimes) by the test suite.
Also fix a couple of typos found while exploring the code.
Reported-by: Ken Moffat <zarniwhoop@ntlworld.com>
Cc: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Fixes: 2139b03c2080 ("imsm: don't call abort_reshape() in imsm_manage_reshape()")
Signed-off-by: NeilBrown <neilb@suse.com>
---
super-intel.c | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/super-intel.c b/super-intel.c
index 90b7b6dee5d0..80b48d0fdd47 100644
--- a/super-intel.c
+++ b/super-intel.c
@@ -10465,7 +10465,7 @@ int check_degradation_change(struct mdinfo *info,
* Function: imsm_manage_reshape
* Description: Function finds array under reshape and it manages reshape
* process. It creates stripes backups (if required) and sets
- * checheckpoits.
+ * checkpoints.
* Parameters:
* afd : Backup handle (nattive) - not used
* sra : general array info
@@ -10595,7 +10595,7 @@ static int imsm_manage_reshape(
start = current_position * 512;
- /* allign reading start to old geometry */
+ /* align reading start to old geometry */
start_buf_shift = start % old_data_stripe_length;
start_src = start - start_buf_shift;
@@ -10700,6 +10700,9 @@ static int imsm_manage_reshape(
ret_val = 1;
abort:
free(buf);
+ sysfs_set_num(sra, NULL, "suspend_lo", 0x7FFFFFFFFFFFFFFFULL);
+ sysfs_set_num(sra, NULL, "suspend_hi", 0);
+ sysfs_set_num(sra, NULL, "suspend_lo", 0);
return ret_val;
}
--
2.7.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]
^ permalink raw reply related
* Re: [PATCH] md: Drop sending a change uevent when stopping
From: Hannes Reinecke @ 2016-02-18 6:45 UTC (permalink / raw)
To: NeilBrown, Shaohua Li, Sebastian Parschauer
Cc: linux-raid, Jes Sorensen, Brassow Jonathan, Artur Paszkiewicz,
systemd-devel
In-Reply-To: <871t8brpon.fsf@notabene.neil.brown.name>
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 02/17/2016 10:29 PM, NeilBrown wrote:
> On Thu, Feb 18 2016, Shaohua Li wrote:
>
>> On Wed, Feb 17, 2016 at 05:25:00PM +0100, Sebastian
>> Parschauer wrote:
>>> When stopping an MD device, then its device node /dev/mdX
>>> may still exist afterwards or it is recreated by udev. The
>>> next open() call can lead to creation of an inoperable MD
>>> device. The reason for this is that a change event
>>> (KOBJ_CHANGE) is sent to udev which races against the
>>> remove event (KOBJ_REMOVE) from md_free(). So drop sending
>>> the change event.
>>>
>>> A change is likely also required in mdadm as many versions
>>> send the change event to udev as well.
>>
>> Makes sense, it's unlikely we need the CHANGE event.
>> Applied.
>>
>> Thanks, Shaohua
>
> It would be worth checking, but I think that with this change,
> you can write "inactive" to /sys/block/mdXXX/md/array_state and
> the array will become inactive, but no uevent will be
> generated, which isn't good. Maybe send the uevent that was
> just removed from the 'inactive' case of array_state_store()
> instead.
>
> (But I still think this is just a bandaid and doesn't provide
> any guarantees that there will be no races with udev)
>
Thing is, _none_ of the other subsystems will ever send a uevent
when it becomes inactive.
(Would be pretty pointless, too, as what exactly is one supposed
to do here?)
The current usage has it that CHANGE events are only ever sent if
a device becomes active.
Cheers,
Hannes
- --
Dr. Hannes Reinecke Teamlead Storage & Networking
hare@suse.de +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iQIcBAEBAgAGBQJWxWiUAAoJEGz4yi9OyKjPUhUP+gJXhNTCYTbLNzR7LYcPQplY
rqcALLhZIDt8inveiSaPVXs5F1VCQsT87qS6JtD3EBSU64eWVq0+xowxKStoyjPl
/MaBFQs7yxJCdf5Enx0/hKPN3MYuQT2nf5EiB461mlnfxLZKUEgwKbDK6+6HqToI
x0rtUFv4JpVDd9HFY3PNqZjGtQTMrbXMVxsBtefIiYPeyaPpYU9Zo0qM+17CmJIr
J3JdXOjHluengKcdi1O6GDvshUiysWP/ukG/q7If4JxpomKS5ljOn5MHcCgBl/CL
UwjUuSmZ9e5ZKyIIxU2oMAFRYqLCGX5Fw5Q90YG7UOZQ3ODbYPJfR7d61OSjomYt
j0bME+QXkdoxOkwlG7EwSU8fG6dv4H55RxrrFcu4ZBl6TRo3jpCOCrq+kY1XapF4
NGeY0j3vBQ4ZziMNodelg+KZaBrCSFGu+cn7uqEsrOJ+N4e7gJv28trPiPzE4Hiz
07buvtTNEYQuqUNvR9MiuDCubnHy0imaA+3fb0orXZOllhYFAMfUVxpOsJvO/ySg
XVSrpcm7BnU/4i6sNXJUpZdTrmWPi8kvZU9avlZFwL0CrVtdhjkoGOu20Ou7H53g
nR3wXQWEybd8yK62lqb3g7one/JnEeVupb69aSiaJnAt7kLZE2Dcg5QVFPAXOIDU
eqYQYG60/85zdeLcqzlM
=e02n
-----END PGP SIGNATURE-----
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* More information on linux raid partition problem
From: d c @ 2016-02-18 16:12 UTC (permalink / raw)
To: linux-raid
For anyone who can help me here is some more information on my linux
raid partition problem.
As stated before, I only have two disks from a three disk raid 5.
When I try:
mdadm --assemble /dev/md0 /dev/hdd2 /dev/hde2
I get the following error:
mdadm: cannot open device /dev/hdd2: No such file or directory
mdadm: /dev/hdd2 has no superblock - assembly aborted
When I do a mdadm examine, I get a different number of events:
mdadm --examine /dev/sdd2 | egrep Event
Events : 60541
mdadm --examine /dev/sde2 | egrep Event
Events : 60544
Also mdadm examine gives different States:
mdadm --examine /dev/sdd2 | egrep State
State : active
mdadm --examine /dev/sde2 | egrep State
State : clean
But mdadm shows that their Magic and UUID numbers are the same.
Can anyone give suggestions on how I can repair this?
Here are the full mdadm --examine outputs:
mdadm --examine /dev/sdd2
/dev/sdd2:
Magic : a92b4efc
Version : 0.90.00
UUID : 1787919b:a8648f43:ae108b15:09b5fb69
Creation Time : Fri Jan 7 17:54:08 2005
Raid Level : raid5
Used Dev Size : 48194816 (45.96 GiB 49.35 GB)
Array Size : 96389632 (91.92 GiB 98.70 GB)
Raid Devices : 3
Total Devices : 2
Preferred Minor : 126
Update Time : Tue May 1 15:55:58 2012
State : active
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Checksum : b53ef3cf - correct
Events : 60541
Layout : left-symmetric
Chunk Size : 128K
Number Major Minor RaidDevice State
this 2 8 66 2 active sync /dev/sde2
0 0 0 0 0 removed
1 1 8 50 1 active sync /dev/sdd2
2 2 8 66 2 active sync /dev/sde2
mdadm --examine /dev/sde2
/dev/sde2:
Magic : a92b4efc
Version : 0.90.00
UUID : 1787919b:a8648f43:ae108b15:09b5fb69
Creation Time : Fri Jan 7 17:54:08 2005
Raid Level : raid5
Used Dev Size : 48194816 (45.96 GiB 49.35 GB)
Array Size : 96389632 (91.92 GiB 98.70 GB)
Raid Devices : 3
Total Devices : 2
Preferred Minor : 126
Update Time : Tue May 1 15:56:20 2012
State : clean
Active Devices : 1
Working Devices : 1
Failed Devices : 1
Spare Devices : 0
Checksum : b53fe060 - correct
Events : 60544
Layout : left-symmetric
Chunk Size : 128K
Number Major Minor RaidDevice State
this 1 8 50 1 active sync /dev/sdd2
0 0 0 0 0 removed
1 1 8 50 1 active sync /dev/sdd2
2 2 0 0 2 faulty removed
^ permalink raw reply
* Re: More information on linux raid partition problem
From: d c @ 2016-02-18 17:17 UTC (permalink / raw)
To: Admin@DH; +Cc: linux-raid
In-Reply-To: <56C5F6A4.4040205@digitallyhosted.com>
Thank you for the reply.
Yes that was just a typo when I rerun the command for the e-mail.
Originally I typed it correct (and that does not work either)
mdadm --assemble /dev/md0 /dev/sdd2 /dev/sde2
mdadm: cannot open device /dev/sdd2: Device or resource busy
mdadm: /dev/sdd2 has no superblock - assembly aborted
On Thu, Feb 18, 2016 at 5:51 PM, Admin@DH <admin@digitallyhosted.com> wrote:
> You typed:
>
> mdadm --assemble /dev/md0 /dev/hdd2 /dev/hde2
>
> It should be:
>
> mdadm --assemble /dev/md0 /dev/sdd2 /dev/sde2
>
> Was that just a typo?
>
>
> On 18/02/2016 16:12, d c wrote:
>
> For anyone who can help me here is some more information on my linux
> raid partition problem.
>
> As stated before, I only have two disks from a three disk raid 5.
>
> When I try:
>
> mdadm --assemble /dev/md0 /dev/hdd2 /dev/hde2
>
> I get the following error:
>
> mdadm: cannot open device /dev/hdd2: No such file or directory
> mdadm: /dev/hdd2 has no superblock - assembly aborted
>
> When I do a mdadm examine, I get a different number of events:
>
> mdadm --examine /dev/sdd2 | egrep Event
> Events : 60541
> mdadm --examine /dev/sde2 | egrep Event
> Events : 60544
>
> Also mdadm examine gives different States:
>
> mdadm --examine /dev/sdd2 | egrep State
> State : active
> mdadm --examine /dev/sde2 | egrep State
> State : clean
>
> But mdadm shows that their Magic and UUID numbers are the same.
>
> Can anyone give suggestions on how I can repair this?
>
> Here are the full mdadm --examine outputs:
>
> mdadm --examine /dev/sdd2
> /dev/sdd2:
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 1787919b:a8648f43:ae108b15:09b5fb69
> Creation Time : Fri Jan 7 17:54:08 2005
> Raid Level : raid5
> Used Dev Size : 48194816 (45.96 GiB 49.35 GB)
> Array Size : 96389632 (91.92 GiB 98.70 GB)
> Raid Devices : 3
> Total Devices : 2
> Preferred Minor : 126
>
> Update Time : Tue May 1 15:55:58 2012
> State : active
> Active Devices : 2
> Working Devices : 2
> Failed Devices : 0
> Spare Devices : 0
> Checksum : b53ef3cf - correct
> Events : 60541
>
> Layout : left-symmetric
> Chunk Size : 128K
>
> Number Major Minor RaidDevice State
> this 2 8 66 2 active sync /dev/sde2
>
> 0 0 0 0 0 removed
> 1 1 8 50 1 active sync /dev/sdd2
> 2 2 8 66 2 active sync /dev/sde2
>
> mdadm --examine /dev/sde2
> /dev/sde2:
> Magic : a92b4efc
> Version : 0.90.00
> UUID : 1787919b:a8648f43:ae108b15:09b5fb69
> Creation Time : Fri Jan 7 17:54:08 2005
> Raid Level : raid5
> Used Dev Size : 48194816 (45.96 GiB 49.35 GB)
> Array Size : 96389632 (91.92 GiB 98.70 GB)
> Raid Devices : 3
> Total Devices : 2
> Preferred Minor : 126
>
> Update Time : Tue May 1 15:56:20 2012
> State : clean
> Active Devices : 1
> Working Devices : 1
> Failed Devices : 1
> Spare Devices : 0
> Checksum : b53fe060 - correct
> Events : 60544
>
> Layout : left-symmetric
> Chunk Size : 128K
>
> Number Major Minor RaidDevice State
> this 1 8 50 1 active sync /dev/sdd2
>
> 0 0 0 0 0 removed
> 1 1 8 50 1 active sync /dev/sdd2
> 2 2 0 0 2 faulty removed
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message tomajordomo@vger.kernel.org
> More majordomo info athttp://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply
* Re: More information on linux raid partition problem
From: Benjamin ESTRABAUD @ 2016-02-18 17:26 UTC (permalink / raw)
To: d c, Admin@DH; +Cc: linux-raid
In-Reply-To: <CAM1AHSwBhjcaQ5dOfu7nfYQGtwQ0RGzGuFdo6ALB44aArGPLSQ@mail.gmail.com>
sdd2 must be opened by something already. Maybe a RAID is already
assembled with that particular device? What does "cat /proc/mdstat" outputs?
Otherwise "lsof|grep sdd" might help to find out what has a open handle
on that drive.
Regards,
Ben.
On 18/02/16 17:17, d c wrote:
> Thank you for the reply.
>
> Yes that was just a typo when I rerun the command for the e-mail.
>
> Originally I typed it correct (and that does not work either)
>
> mdadm --assemble /dev/md0 /dev/sdd2 /dev/sde2
> mdadm: cannot open device /dev/sdd2: Device or resource busy
> mdadm: /dev/sdd2 has no superblock - assembly aborted
>
> On Thu, Feb 18, 2016 at 5:51 PM, Admin@DH <admin@digitallyhosted.com> wrote:
>> You typed:
>>
>> mdadm --assemble /dev/md0 /dev/hdd2 /dev/hde2
>>
>> It should be:
>>
>> mdadm --assemble /dev/md0 /dev/sdd2 /dev/sde2
>>
>> Was that just a typo?
>>
>>
>> On 18/02/2016 16:12, d c wrote:
>>
>> For anyone who can help me here is some more information on my linux
>> raid partition problem.
>>
>> As stated before, I only have two disks from a three disk raid 5.
>>
>> When I try:
>>
>> mdadm --assemble /dev/md0 /dev/hdd2 /dev/hde2
>>
>> I get the following error:
>>
>> mdadm: cannot open device /dev/hdd2: No such file or directory
>> mdadm: /dev/hdd2 has no superblock - assembly aborted
>>
>> When I do a mdadm examine, I get a different number of events:
>>
>> mdadm --examine /dev/sdd2 | egrep Event
>> Events : 60541
>> mdadm --examine /dev/sde2 | egrep Event
>> Events : 60544
>>
>> Also mdadm examine gives different States:
>>
>> mdadm --examine /dev/sdd2 | egrep State
>> State : active
>> mdadm --examine /dev/sde2 | egrep State
>> State : clean
>>
>> But mdadm shows that their Magic and UUID numbers are the same.
>>
>> Can anyone give suggestions on how I can repair this?
>>
>> Here are the full mdadm --examine outputs:
>>
>> mdadm --examine /dev/sdd2
>> /dev/sdd2:
>> Magic : a92b4efc
>> Version : 0.90.00
>> UUID : 1787919b:a8648f43:ae108b15:09b5fb69
>> Creation Time : Fri Jan 7 17:54:08 2005
>> Raid Level : raid5
>> Used Dev Size : 48194816 (45.96 GiB 49.35 GB)
>> Array Size : 96389632 (91.92 GiB 98.70 GB)
>> Raid Devices : 3
>> Total Devices : 2
>> Preferred Minor : 126
>>
>> Update Time : Tue May 1 15:55:58 2012
>> State : active
>> Active Devices : 2
>> Working Devices : 2
>> Failed Devices : 0
>> Spare Devices : 0
>> Checksum : b53ef3cf - correct
>> Events : 60541
>>
>> Layout : left-symmetric
>> Chunk Size : 128K
>>
>> Number Major Minor RaidDevice State
>> this 2 8 66 2 active sync /dev/sde2
>>
>> 0 0 0 0 0 removed
>> 1 1 8 50 1 active sync /dev/sdd2
>> 2 2 8 66 2 active sync /dev/sde2
>>
>> mdadm --examine /dev/sde2
>> /dev/sde2:
>> Magic : a92b4efc
>> Version : 0.90.00
>> UUID : 1787919b:a8648f43:ae108b15:09b5fb69
>> Creation Time : Fri Jan 7 17:54:08 2005
>> Raid Level : raid5
>> Used Dev Size : 48194816 (45.96 GiB 49.35 GB)
>> Array Size : 96389632 (91.92 GiB 98.70 GB)
>> Raid Devices : 3
>> Total Devices : 2
>> Preferred Minor : 126
>>
>> Update Time : Tue May 1 15:56:20 2012
>> State : clean
>> Active Devices : 1
>> Working Devices : 1
>> Failed Devices : 1
>> Spare Devices : 0
>> Checksum : b53fe060 - correct
>> Events : 60544
>>
>> Layout : left-symmetric
>> Chunk Size : 128K
>>
>> Number Major Minor RaidDevice State
>> this 1 8 50 1 active sync /dev/sdd2
>>
>> 0 0 0 0 0 removed
>> 1 1 8 50 1 active sync /dev/sdd2
>> 2 2 0 0 2 faulty removed
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message tomajordomo@vger.kernel.org
>> More majordomo info athttp://vger.kernel.org/majordomo-info.html
>>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply
* Re: More information on linux raid partition problem
From: d c @ 2016-02-18 18:03 UTC (permalink / raw)
To: Benjamin ESTRABAUD; +Cc: Admin@DH, linux-raid
In-Reply-To: <56C5FED9.9080001@mpstor.com>
Ben,
You are correct, sdd2 was tied up. This must have been caused by my
experimenting on mounting md0.
[root@server ~]# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : inactive sdd2[2](S)
48194880 blocks
. . .
I rebooted the system and now both sdd2 and sde2 show "clean" with a
mdadm --examine.
But I still can not assemble the raid:
mdadm --assemble /dev/md0 /dev/sdd2 /dev/sde2
mdadm: cannot open device /dev/sde2: Device or resource busy
mdadm: /dev/sde2 has no superblock - assembly aborted
I hope you have more ideas.
donald
On Thu, Feb 18, 2016 at 6:26 PM, Benjamin ESTRABAUD <be@mpstor.com> wrote:
> sdd2 must be opened by something already. Maybe a RAID is already assembled
> with that particular device? What does "cat /proc/mdstat" outputs?
>
> Otherwise "lsof|grep sdd" might help to find out what has a open handle on
> that drive.
>
> Regards,
> Ben.
>
>
> On 18/02/16 17:17, d c wrote:
>>
>> Thank you for the reply.
>>
>> Yes that was just a typo when I rerun the command for the e-mail.
>>
>> Originally I typed it correct (and that does not work either)
>>
>> mdadm --assemble /dev/md0 /dev/sdd2 /dev/sde2
>> mdadm: cannot open device /dev/sdd2: Device or resource busy
>> mdadm: /dev/sdd2 has no superblock - assembly aborted
>>
>> On Thu, Feb 18, 2016 at 5:51 PM, Admin@DH <admin@digitallyhosted.com>
>> wrote:
>>>
>>> You typed:
>>>
>>> mdadm --assemble /dev/md0 /dev/hdd2 /dev/hde2
>>>
>>> It should be:
>>>
>>> mdadm --assemble /dev/md0 /dev/sdd2 /dev/sde2
>>>
>>> Was that just a typo?
>>>
>>>
>>> On 18/02/2016 16:12, d c wrote:
>>>
>>> For anyone who can help me here is some more information on my linux
>>> raid partition problem.
>>>
>>> As stated before, I only have two disks from a three disk raid 5.
>>>
>>> When I try:
>>>
>>> mdadm --assemble /dev/md0 /dev/hdd2 /dev/hde2
>>>
>>> I get the following error:
>>>
>>> mdadm: cannot open device /dev/hdd2: No such file or directory
>>> mdadm: /dev/hdd2 has no superblock - assembly aborted
>>>
>>> When I do a mdadm examine, I get a different number of events:
>>>
>>> mdadm --examine /dev/sdd2 | egrep Event
>>> Events : 60541
>>> mdadm --examine /dev/sde2 | egrep Event
>>> Events : 60544
>>>
>>> Also mdadm examine gives different States:
>>>
>>> mdadm --examine /dev/sdd2 | egrep State
>>> State : active
>>> mdadm --examine /dev/sde2 | egrep State
>>> State : clean
>>>
>>> But mdadm shows that their Magic and UUID numbers are the same.
>>>
>>> Can anyone give suggestions on how I can repair this?
>>>
>>> Here are the full mdadm --examine outputs:
>>>
>>> mdadm --examine /dev/sdd2
>>> /dev/sdd2:
>>> Magic : a92b4efc
>>> Version : 0.90.00
>>> UUID : 1787919b:a8648f43:ae108b15:09b5fb69
>>> Creation Time : Fri Jan 7 17:54:08 2005
>>> Raid Level : raid5
>>> Used Dev Size : 48194816 (45.96 GiB 49.35 GB)
>>> Array Size : 96389632 (91.92 GiB 98.70 GB)
>>> Raid Devices : 3
>>> Total Devices : 2
>>> Preferred Minor : 126
>>>
>>> Update Time : Tue May 1 15:55:58 2012
>>> State : active
>>> Active Devices : 2
>>> Working Devices : 2
>>> Failed Devices : 0
>>> Spare Devices : 0
>>> Checksum : b53ef3cf - correct
>>> Events : 60541
>>>
>>> Layout : left-symmetric
>>> Chunk Size : 128K
>>>
>>> Number Major Minor RaidDevice State
>>> this 2 8 66 2 active sync /dev/sde2
>>>
>>> 0 0 0 0 0 removed
>>> 1 1 8 50 1 active sync /dev/sdd2
>>> 2 2 8 66 2 active sync /dev/sde2
>>>
>>> mdadm --examine /dev/sde2
>>> /dev/sde2:
>>> Magic : a92b4efc
>>> Version : 0.90.00
>>> UUID : 1787919b:a8648f43:ae108b15:09b5fb69
>>> Creation Time : Fri Jan 7 17:54:08 2005
>>> Raid Level : raid5
>>> Used Dev Size : 48194816 (45.96 GiB 49.35 GB)
>>> Array Size : 96389632 (91.92 GiB 98.70 GB)
>>> Raid Devices : 3
>>> Total Devices : 2
>>> Preferred Minor : 126
>>>
>>> Update Time : Tue May 1 15:56:20 2012
>>> State : clean
>>> Active Devices : 1
>>> Working Devices : 1
>>> Failed Devices : 1
>>> Spare Devices : 0
>>> Checksum : b53fe060 - correct
>>> Events : 60544
>>>
>>> Layout : left-symmetric
>>> Chunk Size : 128K
>>>
>>> Number Major Minor RaidDevice State
>>> this 1 8 50 1 active sync /dev/sdd2
>>>
>>> 0 0 0 0 0 removed
>>> 1 1 8 50 1 active sync /dev/sdd2
>>> 2 2 0 0 2 faulty removed
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>>> the body of a message tomajordomo@vger.kernel.org
>>> More majordomo info athttp://vger.kernel.org/majordomo-info.html
>>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>
^ permalink raw reply
* Re: [PATCH] super-intel: ensure suspended region is removed when reshape completes.
From: Jes Sorensen @ 2016-02-18 19:03 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid, Ken Moffat, Artur Paszkiewicz
In-Reply-To: <871t8ar54j.fsf@notabene.neil.brown.name>
NeilBrown <neilb@suse.com> writes:
> A recent commit removed a call to abort_reshape() when IMSM reshape
> completed. An unanticipated result of this is that the suspended
> region is not cleared as it should be.
> So after a reshape, a region of the array will cause all IO to block.
>
> Re-instate the required updates to suspend_{lo,hi} coped from
> abort_reshape().
>
> This is caught (sometimes) by the test suite.
>
> Also fix a couple of typos found while exploring the code.
>
> Reported-by: Ken Moffat <zarniwhoop@ntlworld.com>
> Cc: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
> Fixes: 2139b03c2080 ("imsm: don't call abort_reshape() in imsm_manage_reshape()")
> Signed-off-by: NeilBrown <neilb@suse.com>
> ---
> super-intel.c | 7 +++++--
> 1 file changed, 5 insertions(+), 2 deletions(-)
>
> diff --git a/super-intel.c b/super-intel.c
> index 90b7b6dee5d0..80b48d0fdd47 100644
> --- a/super-intel.c
> +++ b/super-intel.c
> @@ -10465,7 +10465,7 @@ int check_degradation_change(struct mdinfo *info,
> * Function: imsm_manage_reshape
> * Description: Function finds array under reshape and it manages reshape
> * process. It creates stripes backups (if required) and sets
> - * checheckpoits.
> + * checkpoints.
> * Parameters:
> * afd : Backup handle (nattive) - not used
> * sra : general array info
> @@ -10595,7 +10595,7 @@ static int imsm_manage_reshape(
>
> start = current_position * 512;
>
> - /* allign reading start to old geometry */
> + /* align reading start to old geometry */
> start_buf_shift = start % old_data_stripe_length;
> start_src = start - start_buf_shift;
>
> @@ -10700,6 +10700,9 @@ static int imsm_manage_reshape(
> ret_val = 1;
> abort:
> free(buf);
> + sysfs_set_num(sra, NULL, "suspend_lo", 0x7FFFFFFFFFFFFFFFULL);
> + sysfs_set_num(sra, NULL, "suspend_hi", 0);
> + sysfs_set_num(sra, NULL, "suspend_lo", 0);
>
> return ret_val;
> }
This does indeed match the behavior of abort_reshape(), however looking
through git history, I cannot find any explanation as to why the code
sets suspend_lo twice.
Any chance you can enlighten me why this is necessary?
Thanks,
Jes
^ permalink raw reply
* Re: mdadm regression tests, slight progress
From: Jes Sorensen @ 2016-02-18 19:20 UTC (permalink / raw)
To: Bruce Dubbs; +Cc: linux-raid
In-Reply-To: <56C4F8A4.2080909@gmail.com>
Bruce Dubbs <bruce.dubbs@gmail.com> writes:
> I'm still trying to figure out why some of the mdadm regression tests
> fail. I started looking at tests/00raid1:
>
> # ./test --tests=00raid1
>
> It is nice that ./test is a bash script so it is easy to do some debugging.
>
> The test script looks like:
>
> # create a simple mirror
> # test version0, version1, and no super
> # test resync and recovery.
>
> mdadm -CR $md0 -l 1 -n2 $dev0 $dev1
> check resync
> check raid1
> testdev $md0 1 $mdsize1a 64
> mdadm -S $md0
>
> # now with version-0.90 superblock, spare
> mdadm -CR $md0 -e0.90 --level=raid1 -n3 -x2 $dev0 missing missing $dev1 $dev2
> check recovery
> check raid1
> testdev $md0 1 $mdsize0 64
> mdadm -S $md0
>
> # now with no superblock
> mdadm -B $md0 -l mirror -n2 $dev0 $dev1
> check resync
> check raid1
> testdev $md0 1 $size 1
> #### Point 1
> mdadm -S $md0
>
> # again, but with no resync
> mdadm -B $md0 -l 1 --assume-clean -n2 $dev0 $dev1
> check raid1
> check nosync
> testdev $md0 1 $size 1
> mdadm -S $md0
> exit 0
>
> What I did was add a couple of 'cat /proc/mdstat' commands where the
> script was failing. At Point 1 above, I got:
>
> ++ cat /proc/mdstat
> Personalities : [raid1] [raid10] [raid6] [raid5] [raid4] [multipath]
> md127 : inactive md0[0](S)
> 19904 blocks
>
> md0 : active raid1 loop1[1] loop0[0]
> 20000 blocks super non-persistent [2/2] [UU]
> [==>..................] resync = 12.5% (2688/20000)
> finish=0.1min speed=2688K/sec
>
>
> The line with resync looked suspicious so at that point I added a
> 'sleep 10' command. The test now passes.
>
> The full log is at
> http://anduin.linuxfromscratch.org/~bdubbs/files/mdadm-test1.log
>
> Does anyone know what is going on to need this sleep in order for this
> test to pass?
It sounds to like the thing hangs if you try to stop the array before
the resync completes. That could be a race in the kernel code.
Jes
^ permalink raw reply
* Re: swiotlb buffer is full
From: Wolfgang Denk @ 2016-02-18 19:44 UTC (permalink / raw)
To: Shaohua Li; +Cc: linux-raid
In-Reply-To: <20160216201347.GB13119@kernel.org>
Dear Shaohua Li,
In message <20160216201347.GB13119@kernel.org> you wrote:
>
> > I think it is interesting that always the same RAID array gets
> > kicked, and always the same disk. I cannot see any hardware
> > problems, and a preventive replacement of the disk drive did not fix
> > the problem.
>
> this doesn't like a md problem.
I tend to agree, but so far I have not found any other test case that
would trigger this problem.
> Probably a dma address leak in the driver. To verify this, you can
> do some IO against the raw disk (sdf/sdg) and check if you see the
> 'swiotlb buffer is full' issue.
At least sequentially reading the drive does not appear to have any
effect; I've completely read it several times with no errors.
> Did you really need iommu, eg if iommu=off works?
This is a good idea; I will enable this setting next time the server
crashed (probably next Sunday night). but then, is iommu=off not
supposed to cause a performance degradation?
Best regards,
Wolfgang Denk
--
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de
Some people march to the beat of a different drummer. And some people
tango!
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox