From mboxrd@z Thu Jan 1 00:00:00 1970 From: Heinz Mauelshagen Subject: Re: RHEL6.6 dm-cache and dm-era Date: Thu, 25 Sep 2014 16:57:49 +0200 Message-ID: <54242D6D.4090004@redhat.com> References: <54164DC0.3020907@gmail.com> <541A977D.7010108@gmail.com> <6FC8490A-55CA-454E-83FE-061A33FDE8DE@redhat.com> <5423B1E0.4000400@gmail.com> <5423EC4B.5070904@redhat.com> Reply-To: device-mapper development Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============8949088376365014164==" Return-path: In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com To: device-mapper development List-Id: dm-devel.ids This is a multi-part message in MIME format. --===============8949088376365014164== Content-Type: multipart/alternative; boundary="------------050902090104060506080404" This is a multi-part message in MIME format. --------------050902090104060506080404 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Romu, "dmsetup create MyEra --table "0 20971520 /dev/mapper/mpathbp1 /dev/mapper/mpathap1 8" should read dmsetup create MyEra --table "0 20971520 era /dev/mapper/mpathbp1 /dev/mapper/mpathap1 8" Heinz On 09/25/2014 04:02 PM, Romu wrote: > 2014-09-25 18:19 GMT+08:00 Heinz Mauelshagen >: > > On 09/25/2014 08:10 AM, Romu Hu wrote: >> On 2014/9/24 12:10, Brassow Jonathan wrote: >>> On Sep 22, 2014, at 7:35 AM, Romu wrote: >>> >>>> I tried the following command to set up my era target but the >>>> command immediately panics the system and the system reboots. >>>> >>>> # dmsetup create MyEra --table "0 41941903 era >>>> /dev/mapper/VG-CacheDataLV_cmeta /dev/mapper/VG-OriginLV 4096" >>>> >>>> The metadata dev and the origin dev are all part of a LVM cache >>>> LV. VG-CacheDataLV_cmeta is the cache metadata LV on the >>>> smaller and faster device, VG-OriginLV is the origin LV on the >>>> faster and slower device, 41941903 is the total sector number >>>> of the device of OriginLV (the LV takes 100% space of the >>>> device), 4096 is the block size of OriginLV, I have run >>>> 'mkfs.ext4 /dev/mapper/VG-OriginLV' before running the dmsetup >>>> command. >>>> >>>> Below is the message in /var/log/messages after running the >>>> dmsetup comnmand: >>>> >>>> kernel: device-mapper: era: sb_check failed: magic 1623043: >>>> wanted 2126579579 >>>> kernel: device-mapper: block manager: superblock validator >>>> check failed for block 0 >>>> kernel: device-mapper: era: couldn't read_lock superblock >>>> >>>> >>>> Any idea? >>>> >>> >>> Sorry, I haven't used dm-era yet. However, it does appear that >>> you are perhaps specifying the wrong devices when creating the >>> era device? Looks like you might be allowing the era target and >>> the cache target to use the same metadata area at the same time >>> - causing them to corrupt each other's metadata area? >>> >>> brassow >> >> I think the dmsetup command to set up an era target should be >> something like this: >> >> # dmsetup create MyEra --table "0 sector_number era metadeta_dev >> origin_dev block_size" >> >> My questions: >> >> 1) How to calculate sector_number? According to >> https://www.kernel.org/doc/Documentation/device-mapper/era.txt, I >> guess it should be (4 * nr_blocks) bytes + buffers, but what is >> nr_blocks and what is buffers? > > No calculation: it's the size of your era target in sectors. > Typically "blockdev --getsz origin_dev" > >> 2) Any documentation for dmsetup tables? > > Look for examples in the kernel source trees > Documentaion/device.mapper. > > table line syntax is: "start_sector length_in_sectors > " > >> 3) Both my metadata_dev and origin_dev are 10G partitions, is >> this all right? > > Metadata device size is plenty but that depends on how many eras > with how many updates you want to have. > >> 4) My origin_dev is a ext4 filesystem with a block size of 4096, >> so the block_size in the command line should also be 4096? > > You may use 4096 = _8_ sectors, you used 4096 sectors = 2MB below. > It's the granularity the era target housekeeps blocks for. > >> >> I tried the following command: >> >> # dmsetup create MyEra --table "0 160000 era /dev/mapper/mpathap1 >> /dev/mapper/mpathbp1 4096" > > May not be same physical device for data and metadata! > If it is, thta'd explain your oops. > > # dmsetup create MyEra --table "0 $(blockdev --getsz > /dev/mapper/mpathbp) era whatever_disctinct_metadata_device > /dev/mapper/mpathbp1 8" > > Would use 8 sector block size (which is tiny!) with disctinct > metadata and data devices (presumabyl your ext4 sits on > /dev/mapper/mpathbp1) > > Heinz > > > Heinz, thank you for your help! > > My origin dev is 10G so total sector number is 20971520. > /dev/mapper/mpathbp1 (/dev/dm-6) is the metadata dev, > /dev/mapper/mpathap1 (/dev/dm-7) is the origin dev, they are on > different LUNs. > > I tried the following commands: > > # dmsetup create MyEra --table "0 20971520 /dev/mapper/mpathbp1 > /dev/mapper/mpathap1 8" > # Target type name /dev/mapper/mpathbp1 is too long. > # Command failed > > # dmsetup create MyEra --table "0 20971520 /dev/dm-6 /dev/dm-7 8" > # device-mapper: reload ioctl on MyEra failed: Invalid argument > # Command failed > > And when executing the second command I see the following in > /var/log/messages: > > Sep 25 06:37:48 localhost kernel: device-mapper: table: 253:8: > /dev/dm-6: unknown target type > Sep 25 06:37:48 localhost kernel: device-mapper: ioctl: error adding > target to table > Sep 25 06:37:48 localhost multipathd: dm-8: remove map (uevent) > Sep 25 06:37:48 localhost multipathd: dm-8: devmap not registered, > can't remove > Sep 25 06:37:48 localhost multipathd: dm-8: remove map (uevent) > Sep 25 06:37:48 localhost multipathd: dm-8: devmap not registered, > can't remove > > Then I run 'mkfs.ext4 /dev/dm-6' and tried the second command again > but got the same result. > > Any idea? > > Thanks > Romu > > > -- > dm-devel mailing list > dm-devel@redhat.com > https://www.redhat.com/mailman/listinfo/dm-devel --------------050902090104060506080404 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by mx4-phx2.redhat.com id s8PEvrqY021183 Romu,

"dmsetup create MyEra --table "0 20971520 /dev/mapper/mpathbp1 /dev/mapper/mpathap1 8"

should read

dmsetup create MyEra --table "0 20971520 era /dev/mapper/mpathbp1 /dev/mapper/mpathap1 8"


Heinz

On 09/25/2014 04:02 PM, Romu wrote:
2014-09-25 18:19 GMT+08:00 Heinz Mauelshagen <heinz= m@redhat.com>:
= On 09/25/2014 08:10 AM, Romu Hu wrote:
On 2014/9/24 12:10, Brassow Jonathan wrote:
On Sep 22, 2014, at 7:35 AM= , Romu wrote:

I tried the following command to set up my era target but the command immediately panics the system and the system reboots.

# dmsetup create MyEra --table "0 41941903=A0era /dev/mapper/VG-CacheDataLV_cmeta /dev/mapper/VG-OriginLV 4096"

The metadata dev and the origin dev are all part of a LVM cache LV.=A0 VG-CacheDataLV_cmeta is the=A0cache metadat= a LV on the smaller and faster device, VG-OriginLV is the origin LV on the faster and slower device, 41941903 is the total sector number of the device of OriginLV (the LV takes 100% space of the device), 4096 is the block size of OriginLV, I have run 'mkfs.ext4 /dev/mapper/VG-OriginLV' before running the dmsetup command.

Below is the message in /var/log/messages after running the dmsetup comnmand:

kernel: device-mapper: era: sb_check failed: magic 1623043: wanted = 2126579579
kernel: device-mapper: block manager: superblock validator check failed for block 0
kernel: device-mapper: era: couldn't read_lock superblock


Any idea?


Sorry, I haven't used dm-era yet.=A0 However, it does appear that you are perhaps specifying the wrong devices when creating the era device?=A0 Looks like you might be allowing the era target and the cache target to use the same metadata area at the same time - causing them to corrupt each other's metadata area?

=A0brassow

I think the dmsetup command to set up an era target should be something like this:

# dmsetup create MyEra --table "0 sector_number era metadeta_dev origin_dev block_size"

My questions:

1) How to calculate sector_number?=A0 According to https://www.kernel.org/doc/Docume= ntation/device-mapper/era.txt, I guess it should be (4 * nr_blocks) bytes + buffers, but what is nr_blocks and what is buffers?

No calculation: it's the size of your era target in sectors. Typically "blockdev --getsz origin_dev"

2) Any documentation for dmsetup tables?

 Look for examples in the kernel source trees Documentaion/device.mapper.

table line syntax is: "start_sector length_in_sectors <target> <target_arguments{0,N}>"

3) Both my metadata_dev and origin_dev are 10G partitions, is this all right?

 Metadata device size is plenty but that depends on how many eras with how many updates you want to have.<= span class=3D"">

4) My origin_dev is a ext4 filesystem with a block size of 4096, so the block_size in the command line should also be 4096?

You may use 4096 =3D _8_ sectors, you used 4096 sectors =3D 2MB below.
It's the granularity the era target housekeeps blocks for.


I tried the following command:

# dmsetup create MyEra --table "0 160000 era /dev/mapper/mpathap1 /dev/mapper/mpathbp1 4096"

 May not be=A0 same physical device for data and metadata!
If it is, thta'd explain your oops.

# dmsetup create MyEra --table "0 $(blockdev --getsz /dev/mapper/mpathbp) era whatever_disctinct_metadata_device /dev/mapper/mpathbp1 8"

Would use 8 sector block size (which is tiny!) with disctinct metadata and data devices (presumabyl your ext4 sits on /dev/mapper/mpathbp1)

Heinz

Heinz, thank you for your help!

My origin dev is 10G so total sector number is=A020971520. =A0/dev/mapper/mpathbp1 (/dev/dm-6) is the metadata dev, /dev/mapper/mpathap1 (/dev/dm-7) is the origin dev, they are on different LUNs.

I tried the following commands:

# dmsetup create MyEra --table "0 20971520 /dev/mapper/mpathbp1 /dev/mapper/mpathap1 8"
# Target type name /dev/mapper/mpathbp1 is too long.
# Command failed

# dmsetup create MyEra --table "0 20971520 /dev/dm-6 /dev/dm-7 8"
# device-mapper: reload ioctl on MyEra failed: Invalid argument
# Command failed

And when executing the second command I see the following in /var/log/messages:

Sep 25 06:37:48 localhost kernel: device-mapper: table: 253:8: /dev/dm-6: unknown target type
Sep 25 06:37:48 localhost kernel: device-mapper: ioctl: error adding target to table
Sep 25 06:37:48 localhost multipathd: dm-8: remove map (uevent)
Sep 25 06:37:48 localhost multipathd: dm-8: devmap not registered, can't remove
Sep 25 06:37:48 localhost multipathd: dm-8: remove map (uevent)
Sep 25 06:37:48 localhost multipathd: dm-8: devmap not registered, can't remove

Then I run 'mkfs.ext4 /dev/dm-6' and tried the second command again but got the same result.

Any idea?

Thanks
Romu


--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel<=
/pre>

--------------050902090104060506080404-- --===============8949088376365014164== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Disposition: inline Content-Transfer-Encoding: 7bit --===============8949088376365014164==--