[linux-lvm] bug in CLING allocator on lv extend

[linux-lvm] bug in CLING allocator on lv extend

[Jens Wilke]

[-- Attachment #1: Type: text/plain, Size: 764 bytes --]

Hi,

I did some extensive tests of the new CLING allocator algorithm which
should allow proper mirror extension.

Testing with CVS HEAD dated 1st december, this is the error I get on 
lvextend in a specific test configuration:
lvextend: metadata/pv_map.c:197: consume_pv_area: Assertion `to_go <= pva->count' failed.

The problem arises in case of an unbalanced amount of free space on the data disks,
mimages twisted (mimage_1 on PV0), and a hole (or at least two areas, I suppose)
that needs to be allocated.

The attached extendtest.sh is the generic code that leads to the error situation 
(set environment varibales disk[0-2] and vg respectively).

The attached extendtest.vg is the meta data backup before the lvextend
with 3 PVs of 128MB each.

Best,

Jens

[-- Attachment #2: extendtest.sh --]
[-- Type: application/x-shellscript, Size: 1224 bytes --]

[-- Attachment #3: extendtest.vg --]
[-- Type: text/plain, Size: 3829 bytes --]

# Generated by LVM2: Wed Dec  6 11:41:23 2006

contents = "Text Format Volume Group"
version = 1

description = "vgcfgbackup -f extend.vg dmtestvg"

creation_host = "t6345027"	# Linux t6345027 2.6.9-prep #8 SMP Tue Jun 20 15:36:09 CEST 2006 s390x
creation_time = 1165401683	# Wed Dec  6 11:41:23 2006

dmtestvg {
	id = "y4I28Z-YHvd-ZffY-f7Zw-515t-wB5U-dS5Hv2"
	seqno = 12
	status = ["RESIZEABLE", "READ", "WRITE"]
	extent_size = 8192		# 4 Megabytes
	max_lv = 0
	max_pv = 0

	physical_volumes {

		pv0 {
			id = "pdqOdr-j5sJ-RQeX-wfkd-DiJW-MeHt-Di4Izx"
			device = "/dev/dm-0"	# Hint only

			status = ["ALLOCATABLE"]
			dev_size = 262144	# 128 Megabytes
			pe_start = 384
			pe_count = 31	# 124 Megabytes
		}

		pv1 {
			id = "k7EULd-xRpN-tFtV-gAW9-4zZc-njst-RLwKxk"
			device = "/dev/dm-1"	# Hint only

			status = ["ALLOCATABLE"]
			dev_size = 262144	# 128 Megabytes
			pe_start = 384
			pe_count = 31	# 124 Megabytes
		}

		pv2 {
			id = "jNqEdU-ZWHf-kW7D-XFGG-ti9s-g0VQ-9hbdiL"
			device = "/dev/dm-2"	# Hint only

			status = ["ALLOCATABLE"]
			dev_size = 262144	# 128 Megabytes
			pe_start = 384
			pe_count = 31	# 124 Megabytes
		}
	}

	logical_volumes {

		lx1 {
			id = "ITQ4ue-FCs4-MCmm-RSYX-Vnjh-TbtQ-Myy3uo"
			status = ["READ", "WRITE", "VISIBLE"]
			segment_count = 1

			segment1 {
				start_extent = 0
				extent_count = 1	# 4 Megabytes

				type = "striped"
				stripe_count = 1	# linear

				stripes = [
					"pv0", 0
				]
			}
		}

		lx2 {
			id = "6Ola3N-lOJt-G6Dw-ItpA-T8Cx-wNyn-G8x0IM"
			status = ["READ", "WRITE", "VISIBLE"]
			segment_count = 1

			segment1 {
				start_extent = 0
				extent_count = 1	# 4 Megabytes

				type = "striped"
				stripe_count = 1	# linear

				stripes = [
					"pv1", 0
				]
			}
		}

		m1 {
			id = "FE1p7Y-Criz-mPWF-hJMJ-eY19-wjTM-uPocc0"
			status = ["READ", "WRITE", "VISIBLE"]
			segment_count = 1

			segment1 {
				start_extent = 0
				extent_count = 9	# 36 Megabytes

				type = "mirror"
				mirror_count = 2
				mirror_log = "m1_mlog"
				region_size = 1024

				mirrors = [
					"m1_mimage_0", 0,
					"m1_mimage_1", 0
				]
			}
		}

		lx3 {
			id = "BpzPY5-E87r-VaeG-2FXV-uLId-P94j-5S17iu"
			status = ["READ", "WRITE", "VISIBLE"]
			segment_count = 1

			segment1 {
				start_extent = 0
				extent_count = 3	# 12 Megabytes

				type = "striped"
				stripe_count = 1	# linear

				stripes = [
					"pv2", 9
				]
			}
		}

		l2 {
			id = "1XrfVI-QBn6-CqOF-fxPN-ja8C-rgor-nkrMLI"
			status = ["READ", "WRITE", "VISIBLE"]
			segment_count = 1

			segment1 {
				start_extent = 0
				extent_count = 1	# 4 Megabytes

				type = "striped"
				stripe_count = 1	# linear

				stripes = [
					"pv2", 13
				]
			}
		}

		l4 {
			id = "R8CvQp-EDQx-Ui25-EGZM-i3Tf-zfSL-5Iv6AD"
			status = ["READ", "WRITE", "VISIBLE"]
			segment_count = 1

			segment1 {
				start_extent = 0
				extent_count = 1	# 4 Megabytes

				type = "striped"
				stripe_count = 1	# linear

				stripes = [
					"pv0", 11
				]
			}
		}

		m1_mlog {
			id = "q5XnM3-VtHZ-EoB9-VT52-2Zct-5oCM-UeWwSg"
			status = ["READ", "WRITE"]
			segment_count = 1

			segment1 {
				start_extent = 0
				extent_count = 1	# 4 Megabytes

				type = "striped"
				stripe_count = 1	# linear

				stripes = [
					"pv1", 1
				]
			}
		}

		m1_mimage_0 {
			id = "NZHdXZ-mzK3-eDj8-nLTn-auSY-tou0-xuaicM"
			status = ["READ", "WRITE"]
			segment_count = 1

			segment1 {
				start_extent = 0
				extent_count = 9	# 36 Megabytes

				type = "striped"
				stripe_count = 1	# linear

				stripes = [
					"pv2", 0
				]
			}
		}

		m1_mimage_1 {
			id = "lJphqJ-NebL-V9LD-0SVe-n4ud-sisi-rkE1vO"
			status = ["READ", "WRITE"]
			segment_count = 1

			segment1 {
				start_extent = 0
				extent_count = 9	# 36 Megabytes

				type = "striped"
				stripe_count = 1	# linear

				stripes = [
					"pv0", 1
				]
			}
		}
	}
}

[linux-lvm] Re: bug in CLING allocator on lv extend

[Jun'ichi Nomura]

[-- Attachment #1: Type: text/plain, Size: 1538 bytes --]

Hi Jens,

Thanks for testing and detailed bug report.

Attached patch will fix the problem.
I have tested it with your test case and it works well.
Please check on your environment, too.

The cause of the problem was that _alloc_parallel_area()
in lib/metadata/lv_manip.c assumed either:
  - areas in "areas[]" are sorted in size
or
  - all areas in areas[] are large enough to fulfill the
    allocation request at once
So it's been ok just to check the last item of areas[]
for adjusting the allocation size.

Under the "cling" policy, this assumption is no longer true.

BTW, the bug was already fixed in the allocation patch set
posted to lvm-devel on Oct. 13, which is not yet accepted.

Jens Wilke wrote:
> I did some extensive tests of the new CLING allocator algorithm which
> should allow proper mirror extension.
> 
> Testing with CVS HEAD dated 1st december, this is the error I get on 
> lvextend in a specific test configuration:
> lvextend: metadata/pv_map.c:197: consume_pv_area: Assertion `to_go <= pva->count' failed.
> 
> The problem arises in case of an unbalanced amount of free space on the data disks,
> mimages twisted (mimage_1 on PV0), and a hole (or at least two areas, I suppose)
> that needs to be allocated.
> 
> The attached extendtest.sh is the generic code that leads to the error situation 
> (set environment varibales disk[0-2] and vg respectively).
> 
> The attached extendtest.vg is the meta data backup before the lvextend
> with 3 PVs of 128MB each.

-- 
Jun'ichi Nomura, NEC Corporation of America

[-- Attachment #2: fix-alloc_parallel_area-for-unsorted-areas.patch --]
[-- Type: text/x-patch, Size: 1241 bytes --]

_alloc_parallel_area() assumes areas[] being sorted or every items
of areas[] being larger than or equal to area_len.
It's no longer true under "cling" allocation policy.
Because, areas[] are indexed based on the layout of previous LV segment
and some of the areas can be smaller than area_len.

Patch applicable to LVM2 2.02.16.

 lib/metadata/lv_manip.c |   10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

--- LVM2.orig/lib/metadata/lv_manip.c	30 Oct 2006 16:10:55 -0000	1.112
+++ LVM2/lib/metadata/lv_manip.c	6 Dec 2006 18:06:28 -0000
@@ -628,17 +628,17 @@ static int _alloc_parallel_area(struct a
 				struct pv_area **areas,
 				uint32_t *ix, struct pv_area *log_area)
 {
-	uint32_t area_len, smallest, remaining;
+	uint32_t area_len, remaining;
 	uint32_t s;
 	struct alloced_area *aa;
 
 	remaining = needed - *ix;
 	area_len = remaining / ah->area_multiple;
 
-	smallest = areas[ah->area_count - 1]->count;
-
-	if (area_len > smallest)
-		area_len = smallest;
+	/* adjust area_len to the smallest area */
+	for (s = 0; s < ah->area_count; s++)
+		if (area_len > areas[s]->count)
+			area_len = areas[s]->count;
 
 	if (!(aa = dm_pool_alloc(ah->mem, sizeof(*aa) *
 			      (ah->area_count + (log_area ? 1 : 0))))) {