* RHEL6.6 dm-cache and dm-era @ 2014-09-15 2:24 Romu Hu 2014-09-15 20:25 ` Brassow Jonathan 0 siblings, 1 reply; 13+ messages in thread From: Romu Hu @ 2014-09-15 2:24 UTC (permalink / raw) To: dm-devel Hi, According to https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6-Beta/html-single/6.6_Release_Notes/index.html#bh-storage, dm-cache and dm-era have been added into RHEL6.6 as technology preview, but I couldn't find any other documentation about using dm-cache and dm-era. I found the following utilities in /usr/sbin/era_check /usr/sbin/era_dump /usr/sbin/era_invalidate /usr/sbin/cache_check /usr/sbin/cache_dump /usr/sbin/cache_repair /usr/sbin/cache_restore I guess these utilities are related to dm-era and dm-cache, respectively. There are only manpages for the cache utilities. After reading the manpages I still have no clue how to use these technologies. I want to do a sanity check of these technologies on my SAN storage. Is it possible to try dm-cache and dm-era without a SSD? e.g. using ram to imitate a fast block device.. Any idea? Thanks Romu ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: RHEL6.6 dm-cache and dm-era 2014-09-15 2:24 RHEL6.6 dm-cache and dm-era Romu Hu @ 2014-09-15 20:25 ` Brassow Jonathan 2014-09-18 8:27 ` Romu Hu 0 siblings, 1 reply; 13+ messages in thread From: Brassow Jonathan @ 2014-09-15 20:25 UTC (permalink / raw) To: device-mapper development On Sep 14, 2014, at 9:24 PM, Romu Hu wrote: > Hi, > > According to https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6-Beta/html-single/6.6_Release_Notes/index.html#bh-storage, dm-cache and dm-era have been added into RHEL6.6 as technology preview, but I couldn't find any other documentation about using dm-cache and dm-era. I found the following utilities in > > /usr/sbin/era_check > /usr/sbin/era_dump > /usr/sbin/era_invalidate > > /usr/sbin/cache_check > /usr/sbin/cache_dump > /usr/sbin/cache_repair > /usr/sbin/cache_restore > > I guess these utilities are related to dm-era and dm-cache, respectively. There are only manpages for the cache utilities. After reading the manpages I still have no clue how to use these technologies. I want to do a sanity check of these technologies on my SAN storage. Is it possible to try dm-cache and dm-era without a SSD? e.g. using ram to imitate a fast block device.. > > Any idea? You can use cache to teir any block device over another. Layering faster, smaller drives over slower, larger ones is the point; but for testing non-performance things you can still do it. For the cache target, it would be a more inclusive test to go through LVM. The lvmcache.7 man page would help out with that. There is no dm-era support in LVM ATM. brassow ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: RHEL6.6 dm-cache and dm-era 2014-09-15 20:25 ` Brassow Jonathan @ 2014-09-18 8:27 ` Romu Hu 2014-09-22 12:35 ` Romu 0 siblings, 1 reply; 13+ messages in thread From: Romu Hu @ 2014-09-18 8:27 UTC (permalink / raw) To: device-mapper development On 2014/9/16 4:25, Brassow Jonathan wrote: > On Sep 14, 2014, at 9:24 PM, Romu Hu wrote: > >> Hi, >> >> According to https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6-Beta/html-single/6.6_Release_Notes/index.html#bh-storage, dm-cache and dm-era have been added into RHEL6.6 as technology preview, but I couldn't find any other documentation about using dm-cache and dm-era. I found the following utilities in >> >> /usr/sbin/era_check >> /usr/sbin/era_dump >> /usr/sbin/era_invalidate >> >> /usr/sbin/cache_check >> /usr/sbin/cache_dump >> /usr/sbin/cache_repair >> /usr/sbin/cache_restore >> >> I guess these utilities are related to dm-era and dm-cache, respectively. There are only manpages for the cache utilities. After reading the manpages I still have no clue how to use these technologies. I want to do a sanity check of these technologies on my SAN storage. Is it possible to try dm-cache and dm-era without a SSD? e.g. using ram to imitate a fast block device.. >> >> Any idea? > You can use cache to teir any block device over another. Layering faster, smaller drives over slower, larger ones is the point; but for testing non-performance things you can still do it. For the cache target, it would be a more inclusive test to go through LVM. The lvmcache.7 man page would help out with that. There is no dm-era support in LVM ATM. > > brassow Thanks, I have successfully set up LVM caching with dm-cache, works like a charm. However, I still haven't any useful documentation about how to set dm-era. Any idea? Thanks Romu ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: RHEL6.6 dm-cache and dm-era 2014-09-18 8:27 ` Romu Hu @ 2014-09-22 12:35 ` Romu 2014-09-24 4:10 ` Brassow Jonathan 0 siblings, 1 reply; 13+ messages in thread From: Romu @ 2014-09-22 12:35 UTC (permalink / raw) To: device-mapper development [-- Attachment #1.1: Type: text/plain, Size: 2718 bytes --] I tried the following command to set up my era target but the command immediately panics the system and the system reboots. # dmsetup create MyEra --table "0 41941903 era /dev/mapper/VG-CacheDataLV_cmeta /dev/mapper/VG-OriginLV 4096" The metadata dev and the origin dev are all part of a LVM cache LV. VG-CacheDataLV_cmeta is the cache metadata LV on the smaller and faster device, VG-OriginLV is the origin LV on the faster and slower device, 41941903 is the total sector number of the device of OriginLV (the LV takes 100% space of the device), 4096 is the block size of OriginLV, I have run 'mkfs.ext4 /dev/mapper/VG-OriginLV' before running the dmsetup command. Below is the message in /var/log/messages after running the dmsetup comnmand: kernel: device-mapper: era: sb_check failed: magic 1623043: wanted 2126579579 kernel: device-mapper: block manager: superblock validator check failed for block 0 kernel: device-mapper: era: couldn't read_lock superblock Any idea? Thanks Romu 2014-09-18 16:27 GMT+08:00 Romu Hu <huruomu@gmail.com>: > On 2014/9/16 4:25, Brassow Jonathan wrote: > >> On Sep 14, 2014, at 9:24 PM, Romu Hu wrote: >> >> Hi, >>> >>> According to https://access.redhat.com/documentation/en-US/Red_Hat_ >>> Enterprise_Linux/6-Beta/html-single/6.6_Release_Notes/ >>> index.html#bh-storage, dm-cache and dm-era have been added into RHEL6.6 >>> as technology preview, but I couldn't find any other documentation about >>> using dm-cache and dm-era. I found the following utilities in >>> >>> /usr/sbin/era_check >>> /usr/sbin/era_dump >>> /usr/sbin/era_invalidate >>> >>> /usr/sbin/cache_check >>> /usr/sbin/cache_dump >>> /usr/sbin/cache_repair >>> /usr/sbin/cache_restore >>> >>> I guess these utilities are related to dm-era and dm-cache, >>> respectively. There are only manpages for the cache utilities. After >>> reading the manpages I still have no clue how to use these technologies. I >>> want to do a sanity check of these technologies on my SAN storage. Is it >>> possible to try dm-cache and dm-era without a SSD? e.g. using ram to >>> imitate a fast block device.. >>> >>> Any idea? >>> >> You can use cache to teir any block device over another. Layering >> faster, smaller drives over slower, larger ones is the point; but for >> testing non-performance things you can still do it. For the cache target, >> it would be a more inclusive test to go through LVM. The lvmcache.7 man >> page would help out with that. There is no dm-era support in LVM ATM. >> >> brassow >> > > Thanks, I have successfully set up LVM caching with dm-cache, works like a > charm. However, I still haven't any useful documentation about how to set > dm-era. Any idea? > > Thanks > Romu > [-- Attachment #1.2: Type: text/html, Size: 3786 bytes --] [-- Attachment #2: Type: text/plain, Size: 0 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: RHEL6.6 dm-cache and dm-era 2014-09-22 12:35 ` Romu @ 2014-09-24 4:10 ` Brassow Jonathan 2014-09-25 6:10 ` Romu Hu 0 siblings, 1 reply; 13+ messages in thread From: Brassow Jonathan @ 2014-09-24 4:10 UTC (permalink / raw) To: device-mapper development [-- Attachment #1.1: Type: text/plain, Size: 1396 bytes --] On Sep 22, 2014, at 7:35 AM, Romu wrote: > I tried the following command to set up my era target but the command immediately panics the system and the system reboots. > > # dmsetup create MyEra --table "0 41941903 era /dev/mapper/VG-CacheDataLV_cmeta /dev/mapper/VG-OriginLV 4096" > > The metadata dev and the origin dev are all part of a LVM cache LV. VG-CacheDataLV_cmeta is the cache metadata LV on the smaller and faster device, VG-OriginLV is the origin LV on the faster and slower device, 41941903 is the total sector number of the device of OriginLV (the LV takes 100% space of the device), 4096 is the block size of OriginLV, I have run 'mkfs.ext4 /dev/mapper/VG-OriginLV' before running the dmsetup command. > > Below is the message in /var/log/messages after running the dmsetup comnmand: > > kernel: device-mapper: era: sb_check failed: magic 1623043: wanted 2126579579 > kernel: device-mapper: block manager: superblock validator check failed for block 0 > kernel: device-mapper: era: couldn't read_lock superblock > > > Any idea? > Sorry, I haven't used dm-era yet. However, it does appear that you are perhaps specifying the wrong devices when creating the era device? Looks like you might be allowing the era target and the cache target to use the same metadata area at the same time - causing them to corrupt each other's metadata area? brassow [-- Attachment #1.2: Type: text/html, Size: 2398 bytes --] [-- Attachment #2: Type: text/plain, Size: 0 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: RHEL6.6 dm-cache and dm-era 2014-09-24 4:10 ` Brassow Jonathan @ 2014-09-25 6:10 ` Romu Hu 2014-09-25 10:19 ` Heinz Mauelshagen 0 siblings, 1 reply; 13+ messages in thread From: Romu Hu @ 2014-09-25 6:10 UTC (permalink / raw) To: dm-devel [-- Attachment #1.1: Type: text/plain, Size: 3082 bytes --] On 2014/9/24 12:10, Brassow Jonathan wrote: > On Sep 22, 2014, at 7:35 AM, Romu wrote: > >> I tried the following command to set up my era target but the command >> immediately panics the system and the system reboots. >> >> # dmsetup create MyEra --table "0 41941903 era >> /dev/mapper/VG-CacheDataLV_cmeta /dev/mapper/VG-OriginLV 4096" >> >> The metadata dev and the origin dev are all part of a LVM cache LV. >> VG-CacheDataLV_cmeta is the cache metadata LV on the smaller and >> faster device, VG-OriginLV is the origin LV on the faster and slower >> device, 41941903 is the total sector number of the device of OriginLV >> (the LV takes 100% space of the device), 4096 is the block size of >> OriginLV, I have run 'mkfs.ext4 /dev/mapper/VG-OriginLV' before >> running the dmsetup command. >> >> Below is the message in /var/log/messages after running the dmsetup >> comnmand: >> >> kernel: device-mapper: era: sb_check failed: magic 1623043: wanted >> 2126579579 >> kernel: device-mapper: block manager: superblock validator check >> failed for block 0 >> kernel: device-mapper: era: couldn't read_lock superblock >> >> >> Any idea? >> > > Sorry, I haven't used dm-era yet. However, it does appear that you > are perhaps specifying the wrong devices when creating the era device? > Looks like you might be allowing the era target and the cache target > to use the same metadata area at the same time - causing them to > corrupt each other's metadata area? > > brassow I think the dmsetup command to set up an era target should be something like this: # dmsetup create MyEra --table "0 sector_number era metadeta_dev origin_dev block_size" My questions: 1) How to calculate sector_number? According to https://www.kernel.org/doc/Documentation/device-mapper/era.txt, I guess it should be (4 * nr_blocks) bytes + buffers, but what is nr_blocks and what is buffers? 2) Any documentation for dmsetup tables? 3) Both my metadata_dev and origin_dev are 10G partitions, is this all right? 4) My origin_dev is a ext4 filesystem with a block size of 4096, so the block_size in the command line should also be 4096? I tried the following command: # dmsetup create MyEra --table "0 160000 era /dev/mapper/mpathap1 /dev/mapper/mpathbp1 4096" It immediately panics the kernel (2.6.32-494.el6.i686) with the following info in /var/log/messages: Sep 24 22:44:22 localhost kernel: device-mapper: era: sb_check failed: magic 0: wanted 2126579579 Sep 24 22:44:22 localhost kernel: device-mapper: block manager: superblock validator check failed for block 0 Sep 24 22:44:22 localhost kernel: device-mapper: era: couldn't read_lock superblock Sep 24 22:44:22 localhost kernel: BUG: unable to handle kernel NULL pointer dereference at 00000008 Sep 24 22:44:22 localhost kernel: IP: [<f7e9a3c7>] era_destroy+0x7/0x60 [dm_era] Sep 24 22:44:22 localhost kernel: *pdpt = 0000000001945001 *pde = 00000003ff97f067 Sep 24 22:44:22 localhost kernel: Oops: 0000 [#1] SMP Sep 24 22:44:22 localhost kernel: last sysfs file: /sys/module/dm_mod/initstate Thanks Romu [-- Attachment #1.2: Type: text/html, Size: 5939 bytes --] [-- Attachment #2: Type: text/plain, Size: 0 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: RHEL6.6 dm-cache and dm-era 2014-09-25 6:10 ` Romu Hu @ 2014-09-25 10:19 ` Heinz Mauelshagen 2014-09-25 14:02 ` Romu 0 siblings, 1 reply; 13+ messages in thread From: Heinz Mauelshagen @ 2014-09-25 10:19 UTC (permalink / raw) To: device-mapper development [-- Attachment #1.1: Type: text/plain, Size: 4236 bytes --] On 09/25/2014 08:10 AM, Romu Hu wrote: > On 2014/9/24 12:10, Brassow Jonathan wrote: >> On Sep 22, 2014, at 7:35 AM, Romu wrote: >> >>> I tried the following command to set up my era target but the >>> command immediately panics the system and the system reboots. >>> >>> # dmsetup create MyEra --table "0 41941903 era >>> /dev/mapper/VG-CacheDataLV_cmeta /dev/mapper/VG-OriginLV 4096" >>> >>> The metadata dev and the origin dev are all part of a LVM cache LV. >>> VG-CacheDataLV_cmeta is the cache metadata LV on the smaller and >>> faster device, VG-OriginLV is the origin LV on the faster and slower >>> device, 41941903 is the total sector number of the device of >>> OriginLV (the LV takes 100% space of the device), 4096 is the block >>> size of OriginLV, I have run 'mkfs.ext4 /dev/mapper/VG-OriginLV' >>> before running the dmsetup command. >>> >>> Below is the message in /var/log/messages after running the dmsetup >>> comnmand: >>> >>> kernel: device-mapper: era: sb_check failed: magic 1623043: wanted >>> 2126579579 >>> kernel: device-mapper: block manager: superblock validator check >>> failed for block 0 >>> kernel: device-mapper: era: couldn't read_lock superblock >>> >>> >>> Any idea? >>> >> >> Sorry, I haven't used dm-era yet. However, it does appear that you >> are perhaps specifying the wrong devices when creating the era >> device? Looks like you might be allowing the era target and the >> cache target to use the same metadata area at the same time - causing >> them to corrupt each other's metadata area? >> >> brassow > > I think the dmsetup command to set up an era target should be > something like this: > > # dmsetup create MyEra --table "0 sector_number era metadeta_dev > origin_dev block_size" > > My questions: > > 1) How to calculate sector_number? According to > https://www.kernel.org/doc/Documentation/device-mapper/era.txt, I > guess it should be (4 * nr_blocks) bytes + buffers, but what is > nr_blocks and what is buffers? No calculation: it's the size of your era target in sectors. Typically "blockdev --getsz origin_dev" > 2) Any documentation for dmsetup tables? Look for examples in the kernel source trees Documentaion/device.mapper. table line syntax is: "start_sector length_in_sectors <target> <target_arguments{0,N}>" > 3) Both my metadata_dev and origin_dev are 10G partitions, is this all > right? Metadata device size is plenty but that depends on how many eras with how many updates you want to have. > 4) My origin_dev is a ext4 filesystem with a block size of 4096, so > the block_size in the command line should also be 4096? You may use 4096 = _8_ sectors, you used 4096 sectors = 2MB below. It's the granularity the era target housekeeps blocks for. > > I tried the following command: > > # dmsetup create MyEra --table "0 160000 era /dev/mapper/mpathap1 > /dev/mapper/mpathbp1 4096" May not be same physical device for data and metadata! If it is, thta'd explain your oops. # dmsetup create MyEra --table "0 $(blockdev --getsz /dev/mapper/mpathbp) era whatever_disctinct_metadata_device /dev/mapper/mpathbp1 8" Would use 8 sector block size (which is tiny!) with disctinct metadata and data devices (presumabyl your ext4 sits on /dev/mapper/mpathbp1) Heinz > > It immediately panics the kernel (2.6.32-494.el6.i686) with the > following info in /var/log/messages: > > Sep 24 22:44:22 localhost kernel: device-mapper: era: sb_check failed: > magic 0: wanted 2126579579 > Sep 24 22:44:22 localhost kernel: device-mapper: block manager: > superblock validator check failed for block 0 > Sep 24 22:44:22 localhost kernel: device-mapper: era: couldn't > read_lock superblock > Sep 24 22:44:22 localhost kernel: BUG: unable to handle kernel NULL > pointer dereference at 00000008 > Sep 24 22:44:22 localhost kernel: IP: [<f7e9a3c7>] > era_destroy+0x7/0x60 [dm_era] > Sep 24 22:44:22 localhost kernel: *pdpt = 0000000001945001 *pde = > 00000003ff97f067 > Sep 24 22:44:22 localhost kernel: Oops: 0000 [#1] SMP > Sep 24 22:44:22 localhost kernel: last sysfs file: > /sys/module/dm_mod/initstate > > Thanks > Romu > > > -- > dm-devel mailing list > dm-devel@redhat.com > https://www.redhat.com/mailman/listinfo/dm-devel [-- Attachment #1.2: Type: text/html, Size: 8463 bytes --] [-- Attachment #2: Type: text/plain, Size: 0 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: RHEL6.6 dm-cache and dm-era 2014-09-25 10:19 ` Heinz Mauelshagen @ 2014-09-25 14:02 ` Romu 2014-09-25 14:57 ` Heinz Mauelshagen 0 siblings, 1 reply; 13+ messages in thread From: Romu @ 2014-09-25 14:02 UTC (permalink / raw) To: device-mapper development [-- Attachment #1.1: Type: text/plain, Size: 4554 bytes --] 2014-09-25 18:19 GMT+08:00 Heinz Mauelshagen <heinzm@redhat.com>: > On 09/25/2014 08:10 AM, Romu Hu wrote: > > On 2014/9/24 12:10, Brassow Jonathan wrote: > > On Sep 22, 2014, at 7:35 AM, Romu wrote: > > I tried the following command to set up my era target but the command > immediately panics the system and the system reboots. > > # dmsetup create MyEra --table "0 41941903 era > /dev/mapper/VG-CacheDataLV_cmeta /dev/mapper/VG-OriginLV 4096" > > The metadata dev and the origin dev are all part of a LVM cache LV. > VG-CacheDataLV_cmeta is the cache metadata LV on the smaller and faster > device, VG-OriginLV is the origin LV on the faster and slower device, > 41941903 is the total sector number of the device of OriginLV (the LV takes > 100% space of the device), 4096 is the block size of OriginLV, I have run > 'mkfs.ext4 /dev/mapper/VG-OriginLV' before running the dmsetup command. > > Below is the message in /var/log/messages after running the dmsetup > comnmand: > > kernel: device-mapper: era: sb_check failed: magic 1623043: wanted > 2126579579 > kernel: device-mapper: block manager: superblock validator check failed > for block 0 > kernel: device-mapper: era: couldn't read_lock superblock > > > Any idea? > > > Sorry, I haven't used dm-era yet. However, it does appear that you are > perhaps specifying the wrong devices when creating the era device? Looks > like you might be allowing the era target and the cache target to use the > same metadata area at the same time - causing them to corrupt each other's > metadata area? > > brassow > > > I think the dmsetup command to set up an era target should be something > like this: > > # dmsetup create MyEra --table "0 sector_number era metadeta_dev > origin_dev block_size" > > My questions: > > 1) How to calculate sector_number? According to > https://www.kernel.org/doc/Documentation/device-mapper/era.txt, I guess > it should be (4 * nr_blocks) bytes + buffers, but what is nr_blocks and > what is buffers? > > > No calculation: it's the size of your era target in sectors. Typically > "blockdev --getsz origin_dev" > > 2) Any documentation for dmsetup tables? > > > Look for examples in the kernel source trees Documentaion/device.mapper. > > table line syntax is: "start_sector length_in_sectors <target> > <target_arguments{0,N}>" > > 3) Both my metadata_dev and origin_dev are 10G partitions, is this all > right? > > > Metadata device size is plenty but that depends on how many eras with how > many updates you want to have. > > 4) My origin_dev is a ext4 filesystem with a block size of 4096, so the > block_size in the command line should also be 4096? > > > You may use 4096 = _8_ sectors, you used 4096 sectors = 2MB below. > It's the granularity the era target housekeeps blocks for. > > > I tried the following command: > > # dmsetup create MyEra --table "0 160000 era /dev/mapper/mpathap1 > /dev/mapper/mpathbp1 4096" > > > May not be same physical device for data and metadata! > If it is, thta'd explain your oops. > > # dmsetup create MyEra --table "0 $(blockdev --getsz /dev/mapper/mpathbp) > era whatever_disctinct_metadata_device /dev/mapper/mpathbp1 8" > > Would use 8 sector block size (which is tiny!) with disctinct metadata and > data devices (presumabyl your ext4 sits on /dev/mapper/mpathbp1) > > Heinz > Heinz, thank you for your help! My origin dev is 10G so total sector number is 20971520. /dev/mapper/mpathbp1 (/dev/dm-6) is the metadata dev, /dev/mapper/mpathap1 (/dev/dm-7) is the origin dev, they are on different LUNs. I tried the following commands: # dmsetup create MyEra --table "0 20971520 /dev/mapper/mpathbp1 /dev/mapper/mpathap1 8" # Target type name /dev/mapper/mpathbp1 is too long. # Command failed # dmsetup create MyEra --table "0 20971520 /dev/dm-6 /dev/dm-7 8" # device-mapper: reload ioctl on MyEra failed: Invalid argument # Command failed And when executing the second command I see the following in /var/log/messages: Sep 25 06:37:48 localhost kernel: device-mapper: table: 253:8: /dev/dm-6: unknown target type Sep 25 06:37:48 localhost kernel: device-mapper: ioctl: error adding target to table Sep 25 06:37:48 localhost multipathd: dm-8: remove map (uevent) Sep 25 06:37:48 localhost multipathd: dm-8: devmap not registered, can't remove Sep 25 06:37:48 localhost multipathd: dm-8: remove map (uevent) Sep 25 06:37:48 localhost multipathd: dm-8: devmap not registered, can't remove Then I run 'mkfs.ext4 /dev/dm-6' and tried the second command again but got the same result. Any idea? Thanks Romu [-- Attachment #1.2: Type: text/html, Size: 8079 bytes --] [-- Attachment #2: Type: text/plain, Size: 0 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: RHEL6.6 dm-cache and dm-era 2014-09-25 14:02 ` Romu @ 2014-09-25 14:57 ` Heinz Mauelshagen 2014-09-26 3:16 ` Romu Hu 0 siblings, 1 reply; 13+ messages in thread From: Heinz Mauelshagen @ 2014-09-25 14:57 UTC (permalink / raw) To: device-mapper development [-- Attachment #1.1: Type: text/plain, Size: 5403 bytes --] Romu, "dmsetup create MyEra --table "0 20971520 /dev/mapper/mpathbp1 /dev/mapper/mpathap1 8" should read dmsetup create MyEra --table "0 20971520 era /dev/mapper/mpathbp1 /dev/mapper/mpathap1 8" Heinz On 09/25/2014 04:02 PM, Romu wrote: > 2014-09-25 18:19 GMT+08:00 Heinz Mauelshagen <heinzm@redhat.com > <mailto:heinzm@redhat.com>>: > > On 09/25/2014 08:10 AM, Romu Hu wrote: >> On 2014/9/24 12:10, Brassow Jonathan wrote: >>> On Sep 22, 2014, at 7:35 AM, Romu wrote: >>> >>>> I tried the following command to set up my era target but the >>>> command immediately panics the system and the system reboots. >>>> >>>> # dmsetup create MyEra --table "0 41941903 era >>>> /dev/mapper/VG-CacheDataLV_cmeta /dev/mapper/VG-OriginLV 4096" >>>> >>>> The metadata dev and the origin dev are all part of a LVM cache >>>> LV. VG-CacheDataLV_cmeta is the cache metadata LV on the >>>> smaller and faster device, VG-OriginLV is the origin LV on the >>>> faster and slower device, 41941903 is the total sector number >>>> of the device of OriginLV (the LV takes 100% space of the >>>> device), 4096 is the block size of OriginLV, I have run >>>> 'mkfs.ext4 /dev/mapper/VG-OriginLV' before running the dmsetup >>>> command. >>>> >>>> Below is the message in /var/log/messages after running the >>>> dmsetup comnmand: >>>> >>>> kernel: device-mapper: era: sb_check failed: magic 1623043: >>>> wanted 2126579579 <tel:2126579579> >>>> kernel: device-mapper: block manager: superblock validator >>>> check failed for block 0 >>>> kernel: device-mapper: era: couldn't read_lock superblock >>>> >>>> >>>> Any idea? >>>> >>> >>> Sorry, I haven't used dm-era yet. However, it does appear that >>> you are perhaps specifying the wrong devices when creating the >>> era device? Looks like you might be allowing the era target and >>> the cache target to use the same metadata area at the same time >>> - causing them to corrupt each other's metadata area? >>> >>> brassow >> >> I think the dmsetup command to set up an era target should be >> something like this: >> >> # dmsetup create MyEra --table "0 sector_number era metadeta_dev >> origin_dev block_size" >> >> My questions: >> >> 1) How to calculate sector_number? According to >> https://www.kernel.org/doc/Documentation/device-mapper/era.txt, I >> guess it should be (4 * nr_blocks) bytes + buffers, but what is >> nr_blocks and what is buffers? > > No calculation: it's the size of your era target in sectors. > Typically "blockdev --getsz origin_dev" > >> 2) Any documentation for dmsetup tables? > > Look for examples in the kernel source trees > Documentaion/device.mapper. > > table line syntax is: "start_sector length_in_sectors <target> > <target_arguments{0,N}>" > >> 3) Both my metadata_dev and origin_dev are 10G partitions, is >> this all right? > > Metadata device size is plenty but that depends on how many eras > with how many updates you want to have. > >> 4) My origin_dev is a ext4 filesystem with a block size of 4096, >> so the block_size in the command line should also be 4096? > > You may use 4096 = _8_ sectors, you used 4096 sectors = 2MB below. > It's the granularity the era target housekeeps blocks for. > >> >> I tried the following command: >> >> # dmsetup create MyEra --table "0 160000 era /dev/mapper/mpathap1 >> /dev/mapper/mpathbp1 4096" > > May not be same physical device for data and metadata! > If it is, thta'd explain your oops. > > # dmsetup create MyEra --table "0 $(blockdev --getsz > /dev/mapper/mpathbp) era whatever_disctinct_metadata_device > /dev/mapper/mpathbp1 8" > > Would use 8 sector block size (which is tiny!) with disctinct > metadata and data devices (presumabyl your ext4 sits on > /dev/mapper/mpathbp1) > > Heinz > > > Heinz, thank you for your help! > > My origin dev is 10G so total sector number is 20971520. > /dev/mapper/mpathbp1 (/dev/dm-6) is the metadata dev, > /dev/mapper/mpathap1 (/dev/dm-7) is the origin dev, they are on > different LUNs. > > I tried the following commands: > > # dmsetup create MyEra --table "0 20971520 /dev/mapper/mpathbp1 > /dev/mapper/mpathap1 8" > # Target type name /dev/mapper/mpathbp1 is too long. > # Command failed > > # dmsetup create MyEra --table "0 20971520 /dev/dm-6 /dev/dm-7 8" > # device-mapper: reload ioctl on MyEra failed: Invalid argument > # Command failed > > And when executing the second command I see the following in > /var/log/messages: > > Sep 25 06:37:48 localhost kernel: device-mapper: table: 253:8: > /dev/dm-6: unknown target type > Sep 25 06:37:48 localhost kernel: device-mapper: ioctl: error adding > target to table > Sep 25 06:37:48 localhost multipathd: dm-8: remove map (uevent) > Sep 25 06:37:48 localhost multipathd: dm-8: devmap not registered, > can't remove > Sep 25 06:37:48 localhost multipathd: dm-8: remove map (uevent) > Sep 25 06:37:48 localhost multipathd: dm-8: devmap not registered, > can't remove > > Then I run 'mkfs.ext4 /dev/dm-6' and tried the second command again > but got the same result. > > Any idea? > > Thanks > Romu > > > -- > dm-devel mailing list > dm-devel@redhat.com > https://www.redhat.com/mailman/listinfo/dm-devel [-- Attachment #1.2: Type: text/html, Size: 12384 bytes --] [-- Attachment #2: Type: text/plain, Size: 0 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: RHEL6.6 dm-cache and dm-era 2014-09-25 14:57 ` Heinz Mauelshagen @ 2014-09-26 3:16 ` Romu Hu 2014-09-26 10:33 ` Heinz Mauelshagen 0 siblings, 1 reply; 13+ messages in thread From: Romu Hu @ 2014-09-26 3:16 UTC (permalink / raw) To: device-mapper development [-- Attachment #1.1: Type: text/plain, Size: 6827 bytes --] On 2014/9/25 22:57, Heinz Mauelshagen wrote: > Romu, > > "dmsetup create MyEra --table "0 20971520 /dev/mapper/mpathbp1 > /dev/mapper/mpathap1 8" > > should read > > dmsetup create MyEra --table "0 20971520 era /dev/mapper/mpathbp1 > /dev/mapper/mpathap1 8" Sorry my bad. I ran 'dmsetup create MyEra --table "0 20971520 era /dev/mapper/mpathbp1 /dev/mapper/mpathap1 8"' but kernel (2.6.32-494.el6.i686) oopsed: Sep 25 18:34:11 localhost kernel: device-mapper: era: sb_check failed: magic 0: wanted 2126579579 Sep 25 18:34:11 localhost kernel: device-mapper: block manager: superblock validator check failed for block 0 Sep 25 18:34:11 localhost kernel: device-mapper: era: couldn't read_lock superblock Sep 25 18:34:11 localhost kernel: BUG: unable to handle kernel NULL pointer dereference at 00000008 Sep 25 18:34:11 localhost kernel: IP: [<f7e2b3c7>] era_destroy+0x7/0x60 [dm_era]Sep 25 18:34:11 localhost kernel: *pdpt = 0000000032457001 *pde = 00000003fb5fa067 Sep 25 18:34:11 localhost kernel: Oops: 0000 [#1] SMP Sep 25 18:34:11 localhost kernel: last sysfs file: /sys/module/dm_persistent_data/initstate Perhaps this should be cherry-picked? https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=989f26f5ad308f40a95f280bf9cd75e558d4f18d The log says sb_check failed, anything wrong with the dmsetup command parameters? Thanks Romu > On 09/25/2014 04:02 PM, Romu wrote: >> 2014-09-25 18:19 GMT+08:00 Heinz Mauelshagen <heinzm@redhat.com >> <mailto:heinzm@redhat.com>>: >> >> On 09/25/2014 08:10 AM, Romu Hu wrote: >>> On 2014/9/24 12:10, Brassow Jonathan wrote: >>>> On Sep 22, 2014, at 7:35 AM, Romu wrote: >>>> >>>>> I tried the following command to set up my era target but the >>>>> command immediately panics the system and the system reboots. >>>>> >>>>> # dmsetup create MyEra --table "0 41941903 era >>>>> /dev/mapper/VG-CacheDataLV_cmeta /dev/mapper/VG-OriginLV 4096" >>>>> >>>>> The metadata dev and the origin dev are all part of a LVM >>>>> cache LV. VG-CacheDataLV_cmeta is the cache metadata LV on the >>>>> smaller and faster device, VG-OriginLV is the origin LV on the >>>>> faster and slower device, 41941903 is the total sector number >>>>> of the device of OriginLV (the LV takes 100% space of the >>>>> device), 4096 is the block size of OriginLV, I have run >>>>> 'mkfs.ext4 /dev/mapper/VG-OriginLV' before running the dmsetup >>>>> command. >>>>> >>>>> Below is the message in /var/log/messages after running the >>>>> dmsetup comnmand: >>>>> >>>>> kernel: device-mapper: era: sb_check failed: magic 1623043: >>>>> wanted 2126579579 <tel:2126579579> >>>>> kernel: device-mapper: block manager: superblock validator >>>>> check failed for block 0 >>>>> kernel: device-mapper: era: couldn't read_lock superblock >>>>> >>>>> >>>>> Any idea? >>>>> >>>> >>>> Sorry, I haven't used dm-era yet. However, it does appear that >>>> you are perhaps specifying the wrong devices when creating the >>>> era device? Looks like you might be allowing the era target >>>> and the cache target to use the same metadata area at the same >>>> time - causing them to corrupt each other's metadata area? >>>> >>>> brassow >>> >>> I think the dmsetup command to set up an era target should be >>> something like this: >>> >>> # dmsetup create MyEra --table "0 sector_number era metadeta_dev >>> origin_dev block_size" >>> >>> My questions: >>> >>> 1) How to calculate sector_number? According to >>> https://www.kernel.org/doc/Documentation/device-mapper/era.txt, >>> I guess it should be (4 * nr_blocks) bytes + buffers, but what >>> is nr_blocks and what is buffers? >> >> No calculation: it's the size of your era target in sectors. >> Typically "blockdev --getsz origin_dev" >> >>> 2) Any documentation for dmsetup tables? >> >> Look for examples in the kernel source trees >> Documentaion/device.mapper. >> >> table line syntax is: "start_sector length_in_sectors <target> >> <target_arguments{0,N}>" >> >>> 3) Both my metadata_dev and origin_dev are 10G partitions, is >>> this all right? >> >> Metadata device size is plenty but that depends on how many eras >> with how many updates you want to have. >> >>> 4) My origin_dev is a ext4 filesystem with a block size of 4096, >>> so the block_size in the command line should also be 4096? >> >> You may use 4096 = _8_ sectors, you used 4096 sectors = 2MB below. >> It's the granularity the era target housekeeps blocks for. >> >>> >>> I tried the following command: >>> >>> # dmsetup create MyEra --table "0 160000 era >>> /dev/mapper/mpathap1 /dev/mapper/mpathbp1 4096" >> >> May not be same physical device for data and metadata! >> If it is, thta'd explain your oops. >> >> # dmsetup create MyEra --table "0 $(blockdev --getsz >> /dev/mapper/mpathbp) era whatever_disctinct_metadata_device >> /dev/mapper/mpathbp1 8" >> >> Would use 8 sector block size (which is tiny!) with disctinct >> metadata and data devices (presumabyl your ext4 sits on >> /dev/mapper/mpathbp1) >> >> Heinz >> >> >> Heinz, thank you for your help! >> >> My origin dev is 10G so total sector number is 20971520. >> /dev/mapper/mpathbp1 (/dev/dm-6) is the metadata dev, >> /dev/mapper/mpathap1 (/dev/dm-7) is the origin dev, they are on >> different LUNs. >> >> I tried the following commands: >> >> # dmsetup create MyEra --table "0 20971520 /dev/mapper/mpathbp1 >> /dev/mapper/mpathap1 8" >> # Target type name /dev/mapper/mpathbp1 is too long. >> # Command failed >> >> # dmsetup create MyEra --table "0 20971520 /dev/dm-6 /dev/dm-7 8" >> # device-mapper: reload ioctl on MyEra failed: Invalid argument >> # Command failed >> >> And when executing the second command I see the following in >> /var/log/messages: >> >> Sep 25 06:37:48 localhost kernel: device-mapper: table: 253:8: >> /dev/dm-6: unknown target type >> Sep 25 06:37:48 localhost kernel: device-mapper: ioctl: error adding >> target to table >> Sep 25 06:37:48 localhost multipathd: dm-8: remove map (uevent) >> Sep 25 06:37:48 localhost multipathd: dm-8: devmap not registered, >> can't remove >> Sep 25 06:37:48 localhost multipathd: dm-8: remove map (uevent) >> Sep 25 06:37:48 localhost multipathd: dm-8: devmap not registered, >> can't remove >> >> Then I run 'mkfs.ext4 /dev/dm-6' and tried the second command again >> but got the same result. >> >> Any idea? >> >> Thanks >> Romu >> >> >> -- >> dm-devel mailing list >> dm-devel@redhat.com >> https://www.redhat.com/mailman/listinfo/dm-devel > > > > -- > dm-devel mailing list > dm-devel@redhat.com > https://www.redhat.com/mailman/listinfo/dm-devel [-- Attachment #1.2: Type: text/html, Size: 15280 bytes --] [-- Attachment #2: Type: text/plain, Size: 0 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: RHEL6.6 dm-cache and dm-era 2014-09-26 3:16 ` Romu Hu @ 2014-09-26 10:33 ` Heinz Mauelshagen 2014-09-29 2:31 ` Romu Hu 0 siblings, 1 reply; 13+ messages in thread From: Heinz Mauelshagen @ 2014-09-26 10:33 UTC (permalink / raw) To: device-mapper development [-- Attachment #1.1: Type: text/plain, Size: 7788 bytes --] Romu, did you initialize the metadata device before trying to create the era target? Zeroes in the first KBs will do. Yes, check for NULL in era_destroy is needed to handle the error path you hit properly. Heinz On 09/26/2014 05:16 AM, Romu Hu wrote: > On 2014/9/25 22:57, Heinz Mauelshagen wrote: >> Romu, >> >> "dmsetup create MyEra --table "0 20971520 /dev/mapper/mpathbp1 >> /dev/mapper/mpathap1 8" >> >> should read >> >> dmsetup create MyEra --table "0 20971520 era /dev/mapper/mpathbp1 >> /dev/mapper/mpathap1 8" > > Sorry my bad. > > I ran 'dmsetup create MyEra --table "0 20971520 era > /dev/mapper/mpathbp1 /dev/mapper/mpathap1 8"' but kernel > (2.6.32-494.el6.i686) oopsed: > > Sep 25 18:34:11 localhost kernel: device-mapper: era: sb_check failed: > magic 0: wanted 2126579579 > Sep 25 18:34:11 localhost kernel: device-mapper: block manager: > superblock validator check failed for block 0 > Sep 25 18:34:11 localhost kernel: device-mapper: era: couldn't > read_lock superblock > Sep 25 18:34:11 localhost kernel: BUG: unable to handle kernel NULL > pointer dereference at 00000008 > Sep 25 18:34:11 localhost kernel: IP: [<f7e2b3c7>] > era_destroy+0x7/0x60 [dm_era]Sep 25 18:34:11 localhost kernel: *pdpt = > 0000000032457001 *pde = 00000003fb5fa067 > Sep 25 18:34:11 localhost kernel: Oops: 0000 [#1] SMP > Sep 25 18:34:11 localhost kernel: last sysfs file: > /sys/module/dm_persistent_data/initstate > > Perhaps this should be cherry-picked? > https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=989f26f5ad308f40a95f280bf9cd75e558d4f18d > > The log says sb_check failed, anything wrong with the dmsetup command > parameters? > > Thanks > Romu > >> On 09/25/2014 04:02 PM, Romu wrote: >>> 2014-09-25 18:19 GMT+08:00 Heinz Mauelshagen <heinzm@redhat.com >>> <mailto:heinzm@redhat.com>>: >>> >>> On 09/25/2014 08:10 AM, Romu Hu wrote: >>>> On 2014/9/24 12:10, Brassow Jonathan wrote: >>>>> On Sep 22, 2014, at 7:35 AM, Romu wrote: >>>>> >>>>>> I tried the following command to set up my era target but the >>>>>> command immediately panics the system and the system reboots. >>>>>> >>>>>> # dmsetup create MyEra --table "0 41941903 era >>>>>> /dev/mapper/VG-CacheDataLV_cmeta /dev/mapper/VG-OriginLV 4096" >>>>>> >>>>>> The metadata dev and the origin dev are all part of a LVM >>>>>> cache LV. VG-CacheDataLV_cmeta is the cache metadata LV on >>>>>> the smaller and faster device, VG-OriginLV is the origin LV >>>>>> on the faster and slower device, 41941903 is the total sector >>>>>> number of the device of OriginLV (the LV takes 100% space of >>>>>> the device), 4096 is the block size of OriginLV, I have run >>>>>> 'mkfs.ext4 /dev/mapper/VG-OriginLV' before running the >>>>>> dmsetup command. >>>>>> >>>>>> Below is the message in /var/log/messages after running the >>>>>> dmsetup comnmand: >>>>>> >>>>>> kernel: device-mapper: era: sb_check failed: magic 1623043: >>>>>> wanted 2126579579 <tel:2126579579> >>>>>> kernel: device-mapper: block manager: superblock validator >>>>>> check failed for block 0 >>>>>> kernel: device-mapper: era: couldn't read_lock superblock >>>>>> >>>>>> >>>>>> Any idea? >>>>>> >>>>> >>>>> Sorry, I haven't used dm-era yet. However, it does appear that >>>>> you are perhaps specifying the wrong devices when creating the >>>>> era device? Looks like you might be allowing the era target >>>>> and the cache target to use the same metadata area at the same >>>>> time - causing them to corrupt each other's metadata area? >>>>> >>>>> brassow >>>> >>>> I think the dmsetup command to set up an era target should be >>>> something like this: >>>> >>>> # dmsetup create MyEra --table "0 sector_number era >>>> metadeta_dev origin_dev block_size" >>>> >>>> My questions: >>>> >>>> 1) How to calculate sector_number? According to >>>> https://www.kernel.org/doc/Documentation/device-mapper/era.txt, >>>> I guess it should be (4 * nr_blocks) bytes + buffers, but what >>>> is nr_blocks and what is buffers? >>> >>> No calculation: it's the size of your era target in sectors. >>> Typically "blockdev --getsz origin_dev" >>> >>>> 2) Any documentation for dmsetup tables? >>> >>> Look for examples in the kernel source trees >>> Documentaion/device.mapper. >>> >>> table line syntax is: "start_sector length_in_sectors <target> >>> <target_arguments{0,N}>" >>> >>>> 3) Both my metadata_dev and origin_dev are 10G partitions, is >>>> this all right? >>> >>> Metadata device size is plenty but that depends on how many eras >>> with how many updates you want to have. >>> >>>> 4) My origin_dev is a ext4 filesystem with a block size of >>>> 4096, so the block_size in the command line should also be 4096? >>> >>> You may use 4096 = _8_ sectors, you used 4096 sectors = 2MB below. >>> It's the granularity the era target housekeeps blocks for. >>> >>>> >>>> I tried the following command: >>>> >>>> # dmsetup create MyEra --table "0 160000 era >>>> /dev/mapper/mpathap1 /dev/mapper/mpathbp1 4096" >>> >>> May not be same physical device for data and metadata! >>> If it is, thta'd explain your oops. >>> >>> # dmsetup create MyEra --table "0 $(blockdev --getsz >>> /dev/mapper/mpathbp) era whatever_disctinct_metadata_device >>> /dev/mapper/mpathbp1 8" >>> >>> Would use 8 sector block size (which is tiny!) with disctinct >>> metadata and data devices (presumabyl your ext4 sits on >>> /dev/mapper/mpathbp1) >>> >>> Heinz >>> >>> >>> Heinz, thank you for your help! >>> >>> My origin dev is 10G so total sector number is 20971520. >>> /dev/mapper/mpathbp1 (/dev/dm-6) is the metadata dev, >>> /dev/mapper/mpathap1 (/dev/dm-7) is the origin dev, they are on >>> different LUNs. >>> >>> I tried the following commands: >>> >>> # dmsetup create MyEra --table "0 20971520 /dev/mapper/mpathbp1 >>> /dev/mapper/mpathap1 8" >>> # Target type name /dev/mapper/mpathbp1 is too long. >>> # Command failed >>> >>> # dmsetup create MyEra --table "0 20971520 /dev/dm-6 /dev/dm-7 8" >>> # device-mapper: reload ioctl on MyEra failed: Invalid argument >>> # Command failed >>> >>> And when executing the second command I see the following in >>> /var/log/messages: >>> >>> Sep 25 06:37:48 localhost kernel: device-mapper: table: 253:8: >>> /dev/dm-6: unknown target type >>> Sep 25 06:37:48 localhost kernel: device-mapper: ioctl: error adding >>> target to table >>> Sep 25 06:37:48 localhost multipathd: dm-8: remove map (uevent) >>> Sep 25 06:37:48 localhost multipathd: dm-8: devmap not registered, >>> can't remove >>> Sep 25 06:37:48 localhost multipathd: dm-8: remove map (uevent) >>> Sep 25 06:37:48 localhost multipathd: dm-8: devmap not registered, >>> can't remove >>> >>> Then I run 'mkfs.ext4 /dev/dm-6' and tried the second command again >>> but got the same result. >>> >>> Any idea? >>> >>> Thanks >>> Romu >>> >>> >>> -- >>> dm-devel mailing list >>> dm-devel@redhat.com >>> https://www.redhat.com/mailman/listinfo/dm-devel >> >> >> >> -- >> dm-devel mailing list >> dm-devel@redhat.com >> https://www.redhat.com/mailman/listinfo/dm-devel > > > > -- > dm-devel mailing list > dm-devel@redhat.com > https://www.redhat.com/mailman/listinfo/dm-devel -- =============================================================== Heinz Mauelshagen +49 2626 141200 Consulting Development Engineer FAX +49 2626 924446 Red Hat GmbH Am Sonnenhang 11 56242 Marienrachdorf Germany heinzm@redhat.com =============================================================== [-- Attachment #1.2: Type: text/html, Size: 17586 bytes --] [-- Attachment #2: Type: text/plain, Size: 0 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: RHEL6.6 dm-cache and dm-era 2014-09-26 10:33 ` Heinz Mauelshagen @ 2014-09-29 2:31 ` Romu Hu 2014-09-29 10:12 ` Heinz Mauelshagen 0 siblings, 1 reply; 13+ messages in thread From: Romu Hu @ 2014-09-29 2:31 UTC (permalink / raw) To: dm-devel [-- Attachment #1.1: Type: text/plain, Size: 8309 bytes --] On 2014/9/26 18:33, Heinz Mauelshagen wrote: > > Romu, > > did you initialize the metadata device before trying to create the era > target? > Zeroes in the first KBs will do. Thanks Heinz, it solved the problem. Is it the proper way to initialize the metadata device of an era target or just a workaround? Thanks Romu > Yes, check for NULL in era_destroy is needed to handle the error path > you hit properly. > > Heinz > > On 09/26/2014 05:16 AM, Romu Hu wrote: >> On 2014/9/25 22:57, Heinz Mauelshagen wrote: >>> Romu, >>> >>> "dmsetup create MyEra --table "0 20971520 /dev/mapper/mpathbp1 >>> /dev/mapper/mpathap1 8" >>> >>> should read >>> >>> dmsetup create MyEra --table "0 20971520 era /dev/mapper/mpathbp1 >>> /dev/mapper/mpathap1 8" >> >> Sorry my bad. >> >> I ran 'dmsetup create MyEra --table "0 20971520 era >> /dev/mapper/mpathbp1 /dev/mapper/mpathap1 8"' but kernel >> (2.6.32-494.el6.i686) oopsed: >> >> Sep 25 18:34:11 localhost kernel: device-mapper: era: sb_check >> failed: magic 0: wanted 2126579579 >> Sep 25 18:34:11 localhost kernel: device-mapper: block manager: >> superblock validator check failed for block 0 >> Sep 25 18:34:11 localhost kernel: device-mapper: era: couldn't >> read_lock superblock >> Sep 25 18:34:11 localhost kernel: BUG: unable to handle kernel NULL >> pointer dereference at 00000008 >> Sep 25 18:34:11 localhost kernel: IP: [<f7e2b3c7>] >> era_destroy+0x7/0x60 [dm_era]Sep 25 18:34:11 localhost kernel: *pdpt >> = 0000000032457001 *pde = 00000003fb5fa067 >> Sep 25 18:34:11 localhost kernel: Oops: 0000 [#1] SMP >> Sep 25 18:34:11 localhost kernel: last sysfs file: >> /sys/module/dm_persistent_data/initstate >> >> Perhaps this should be cherry-picked? >> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=989f26f5ad308f40a95f280bf9cd75e558d4f18d >> >> The log says sb_check failed, anything wrong with the dmsetup command >> parameters? >> >> Thanks >> Romu >> >>> On 09/25/2014 04:02 PM, Romu wrote: >>>> 2014-09-25 18:19 GMT+08:00 Heinz Mauelshagen <heinzm@redhat.com >>>> <mailto:heinzm@redhat.com>>: >>>> >>>> On 09/25/2014 08:10 AM, Romu Hu wrote: >>>>> On 2014/9/24 12:10, Brassow Jonathan wrote: >>>>>> On Sep 22, 2014, at 7:35 AM, Romu wrote: >>>>>> >>>>>>> I tried the following command to set up my era target but >>>>>>> the command immediately panics the system and the system >>>>>>> reboots. >>>>>>> >>>>>>> # dmsetup create MyEra --table "0 41941903 era >>>>>>> /dev/mapper/VG-CacheDataLV_cmeta /dev/mapper/VG-OriginLV 4096" >>>>>>> >>>>>>> The metadata dev and the origin dev are all part of a LVM >>>>>>> cache LV. VG-CacheDataLV_cmeta is the cache metadata LV on >>>>>>> the smaller and faster device, VG-OriginLV is the origin LV >>>>>>> on the faster and slower device, 41941903 is the total >>>>>>> sector number of the device of OriginLV (the LV takes 100% >>>>>>> space of the device), 4096 is the block size of OriginLV, I >>>>>>> have run 'mkfs.ext4 /dev/mapper/VG-OriginLV' before running >>>>>>> the dmsetup command. >>>>>>> >>>>>>> Below is the message in /var/log/messages after running the >>>>>>> dmsetup comnmand: >>>>>>> >>>>>>> kernel: device-mapper: era: sb_check failed: magic 1623043: >>>>>>> wanted 2126579579 <tel:2126579579> >>>>>>> kernel: device-mapper: block manager: superblock validator >>>>>>> check failed for block 0 >>>>>>> kernel: device-mapper: era: couldn't read_lock superblock >>>>>>> >>>>>>> >>>>>>> Any idea? >>>>>>> >>>>>> >>>>>> Sorry, I haven't used dm-era yet. However, it does appear >>>>>> that you are perhaps specifying the wrong devices when >>>>>> creating the era device? Looks like you might be allowing >>>>>> the era target and the cache target to use the same metadata >>>>>> area at the same time - causing them to corrupt each other's >>>>>> metadata area? >>>>>> >>>>>> brassow >>>>> >>>>> I think the dmsetup command to set up an era target should be >>>>> something like this: >>>>> >>>>> # dmsetup create MyEra --table "0 sector_number era >>>>> metadeta_dev origin_dev block_size" >>>>> >>>>> My questions: >>>>> >>>>> 1) How to calculate sector_number? According to >>>>> https://www.kernel.org/doc/Documentation/device-mapper/era.txt, I >>>>> guess it should be (4 * nr_blocks) bytes + buffers, but what >>>>> is nr_blocks and what is buffers? >>>> >>>> No calculation: it's the size of your era target in sectors. >>>> Typically "blockdev --getsz origin_dev" >>>> >>>>> 2) Any documentation for dmsetup tables? >>>> >>>> Look for examples in the kernel source trees >>>> Documentaion/device.mapper. >>>> >>>> table line syntax is: "start_sector length_in_sectors <target> >>>> <target_arguments{0,N}>" >>>> >>>>> 3) Both my metadata_dev and origin_dev are 10G partitions, is >>>>> this all right? >>>> >>>> Metadata device size is plenty but that depends on how many >>>> eras with how many updates you want to have. >>>> >>>>> 4) My origin_dev is a ext4 filesystem with a block size of >>>>> 4096, so the block_size in the command line should also be 4096? >>>> >>>> You may use 4096 = _8_ sectors, you used 4096 sectors = 2MB below. >>>> It's the granularity the era target housekeeps blocks for. >>>> >>>>> >>>>> I tried the following command: >>>>> >>>>> # dmsetup create MyEra --table "0 160000 era >>>>> /dev/mapper/mpathap1 /dev/mapper/mpathbp1 4096" >>>> >>>> May not be same physical device for data and metadata! >>>> If it is, thta'd explain your oops. >>>> >>>> # dmsetup create MyEra --table "0 $(blockdev --getsz >>>> /dev/mapper/mpathbp) era whatever_disctinct_metadata_device >>>> /dev/mapper/mpathbp1 8" >>>> >>>> Would use 8 sector block size (which is tiny!) with disctinct >>>> metadata and data devices (presumabyl your ext4 sits on >>>> /dev/mapper/mpathbp1) >>>> >>>> Heinz >>>> >>>> >>>> Heinz, thank you for your help! >>>> >>>> My origin dev is 10G so total sector number is 20971520. >>>> /dev/mapper/mpathbp1 (/dev/dm-6) is the metadata dev, >>>> /dev/mapper/mpathap1 (/dev/dm-7) is the origin dev, they are on >>>> different LUNs. >>>> >>>> I tried the following commands: >>>> >>>> # dmsetup create MyEra --table "0 20971520 /dev/mapper/mpathbp1 >>>> /dev/mapper/mpathap1 8" >>>> # Target type name /dev/mapper/mpathbp1 is too long. >>>> # Command failed >>>> >>>> # dmsetup create MyEra --table "0 20971520 /dev/dm-6 /dev/dm-7 8" >>>> # device-mapper: reload ioctl on MyEra failed: Invalid argument >>>> # Command failed >>>> >>>> And when executing the second command I see the following in >>>> /var/log/messages: >>>> >>>> Sep 25 06:37:48 localhost kernel: device-mapper: table: 253:8: >>>> /dev/dm-6: unknown target type >>>> Sep 25 06:37:48 localhost kernel: device-mapper: ioctl: error >>>> adding target to table >>>> Sep 25 06:37:48 localhost multipathd: dm-8: remove map (uevent) >>>> Sep 25 06:37:48 localhost multipathd: dm-8: devmap not registered, >>>> can't remove >>>> Sep 25 06:37:48 localhost multipathd: dm-8: remove map (uevent) >>>> Sep 25 06:37:48 localhost multipathd: dm-8: devmap not registered, >>>> can't remove >>>> >>>> Then I run 'mkfs.ext4 /dev/dm-6' and tried the second command again >>>> but got the same result. >>>> >>>> Any idea? >>>> >>>> Thanks >>>> Romu >>>> >>>> >>>> -- >>>> dm-devel mailing list >>>> dm-devel@redhat.com >>>> https://www.redhat.com/mailman/listinfo/dm-devel >>> >>> >>> >>> -- >>> dm-devel mailing list >>> dm-devel@redhat.com >>> https://www.redhat.com/mailman/listinfo/dm-devel >> >> >> >> -- >> dm-devel mailing list >> dm-devel@redhat.com >> https://www.redhat.com/mailman/listinfo/dm-devel > > -- > =============================================================== > Heinz Mauelshagen +49 2626 141200 > Consulting Development Engineer FAX +49 2626 924446 > Red Hat GmbH > Am Sonnenhang 11 > 56242 Marienrachdorf > Germanyheinzm@redhat.com > =============================================================== > > > -- > dm-devel mailing list > dm-devel@redhat.com > https://www.redhat.com/mailman/listinfo/dm-devel [-- Attachment #1.2: Type: text/html, Size: 19319 bytes --] [-- Attachment #2: Type: text/plain, Size: 0 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: RHEL6.6 dm-cache and dm-era 2014-09-29 2:31 ` Romu Hu @ 2014-09-29 10:12 ` Heinz Mauelshagen 0 siblings, 0 replies; 13+ messages in thread From: Heinz Mauelshagen @ 2014-09-29 10:12 UTC (permalink / raw) To: device-mapper development [-- Attachment #1.1: Type: text/plain, Size: 9020 bytes --] On 09/29/2014 04:31 AM, Romu Hu wrote: > On 2014/9/26 18:33, Heinz Mauelshagen wrote: >> >> Romu, >> >> did you initialize the metadata device before trying to create the >> era target? >> Zeroes in the first KBs will do. > > Thanks Heinz, it solved the problem. Is it the proper way to > initialize the metadata device of an era target or just a workaround? It is the proper way to address it when using dmsetup. MInd you dmsetup is a low-level tool which is very handy for testing/evatuation like in your use case. Ultimately dm-era will be supported by lvm and eventually other higher level tools, which will be in charge of doing metadata device initialization. Heinz > > Thanks > Romu > >> Yes, check for NULL in era_destroy is needed to handle the error >> path you hit properly. >> >> Heinz >> >> On 09/26/2014 05:16 AM, Romu Hu wrote: >>> On 2014/9/25 22:57, Heinz Mauelshagen wrote: >>>> Romu, >>>> >>>> "dmsetup create MyEra --table "0 20971520 /dev/mapper/mpathbp1 >>>> /dev/mapper/mpathap1 8" >>>> >>>> should read >>>> >>>> dmsetup create MyEra --table "0 20971520 era /dev/mapper/mpathbp1 >>>> /dev/mapper/mpathap1 8" >>> >>> Sorry my bad. >>> >>> I ran 'dmsetup create MyEra --table "0 20971520 era >>> /dev/mapper/mpathbp1 /dev/mapper/mpathap1 8"' but kernel >>> (2.6.32-494.el6.i686) oopsed: >>> >>> Sep 25 18:34:11 localhost kernel: device-mapper: era: sb_check >>> failed: magic 0: wanted 2126579579 >>> Sep 25 18:34:11 localhost kernel: device-mapper: block manager: >>> superblock validator check failed for block 0 >>> Sep 25 18:34:11 localhost kernel: device-mapper: era: couldn't >>> read_lock superblock >>> Sep 25 18:34:11 localhost kernel: BUG: unable to handle kernel NULL >>> pointer dereference at 00000008 >>> Sep 25 18:34:11 localhost kernel: IP: [<f7e2b3c7>] >>> era_destroy+0x7/0x60 [dm_era]Sep 25 18:34:11 localhost kernel: *pdpt >>> = 0000000032457001 *pde = 00000003fb5fa067 >>> Sep 25 18:34:11 localhost kernel: Oops: 0000 [#1] SMP >>> Sep 25 18:34:11 localhost kernel: last sysfs file: >>> /sys/module/dm_persistent_data/initstate >>> >>> Perhaps this should be cherry-picked? >>> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=989f26f5ad308f40a95f280bf9cd75e558d4f18d >>> >>> The log says sb_check failed, anything wrong with the dmsetup >>> command parameters? >>> >>> Thanks >>> Romu >>> >>>> On 09/25/2014 04:02 PM, Romu wrote: >>>>> 2014-09-25 18:19 GMT+08:00 Heinz Mauelshagen <heinzm@redhat.com >>>>> <mailto:heinzm@redhat.com>>: >>>>> >>>>> On 09/25/2014 08:10 AM, Romu Hu wrote: >>>>>> On 2014/9/24 12:10, Brassow Jonathan wrote: >>>>>>> On Sep 22, 2014, at 7:35 AM, Romu wrote: >>>>>>> >>>>>>>> I tried the following command to set up my era target but >>>>>>>> the command immediately panics the system and the system >>>>>>>> reboots. >>>>>>>> >>>>>>>> # dmsetup create MyEra --table "0 41941903 era >>>>>>>> /dev/mapper/VG-CacheDataLV_cmeta /dev/mapper/VG-OriginLV 4096" >>>>>>>> >>>>>>>> The metadata dev and the origin dev are all part of a LVM >>>>>>>> cache LV. VG-CacheDataLV_cmeta is the cache metadata LV on >>>>>>>> the smaller and faster device, VG-OriginLV is the origin LV >>>>>>>> on the faster and slower device, 41941903 is the total >>>>>>>> sector number of the device of OriginLV (the LV takes 100% >>>>>>>> space of the device), 4096 is the block size of OriginLV, I >>>>>>>> have run 'mkfs.ext4 /dev/mapper/VG-OriginLV' before running >>>>>>>> the dmsetup command. >>>>>>>> >>>>>>>> Below is the message in /var/log/messages after running the >>>>>>>> dmsetup comnmand: >>>>>>>> >>>>>>>> kernel: device-mapper: era: sb_check failed: magic 1623043: >>>>>>>> wanted 2126579579 <tel:2126579579> >>>>>>>> kernel: device-mapper: block manager: superblock validator >>>>>>>> check failed for block 0 >>>>>>>> kernel: device-mapper: era: couldn't read_lock superblock >>>>>>>> >>>>>>>> >>>>>>>> Any idea? >>>>>>>> >>>>>>> >>>>>>> Sorry, I haven't used dm-era yet. However, it does appear >>>>>>> that you are perhaps specifying the wrong devices when >>>>>>> creating the era device? Looks like you might be allowing >>>>>>> the era target and the cache target to use the same metadata >>>>>>> area at the same time - causing them to corrupt each other's >>>>>>> metadata area? >>>>>>> >>>>>>> brassow >>>>>> >>>>>> I think the dmsetup command to set up an era target should be >>>>>> something like this: >>>>>> >>>>>> # dmsetup create MyEra --table "0 sector_number era >>>>>> metadeta_dev origin_dev block_size" >>>>>> >>>>>> My questions: >>>>>> >>>>>> 1) How to calculate sector_number? According to >>>>>> https://www.kernel.org/doc/Documentation/device-mapper/era.txt, >>>>>> I guess it should be (4 * nr_blocks) bytes + buffers, but >>>>>> what is nr_blocks and what is buffers? >>>>> >>>>> No calculation: it's the size of your era target in sectors. >>>>> Typically "blockdev --getsz origin_dev" >>>>> >>>>>> 2) Any documentation for dmsetup tables? >>>>> >>>>> Look for examples in the kernel source trees >>>>> Documentaion/device.mapper. >>>>> >>>>> table line syntax is: "start_sector length_in_sectors <target> >>>>> <target_arguments{0,N}>" >>>>> >>>>>> 3) Both my metadata_dev and origin_dev are 10G partitions, is >>>>>> this all right? >>>>> >>>>> Metadata device size is plenty but that depends on how many >>>>> eras with how many updates you want to have. >>>>> >>>>>> 4) My origin_dev is a ext4 filesystem with a block size of >>>>>> 4096, so the block_size in the command line should also be 4096? >>>>> >>>>> You may use 4096 = _8_ sectors, you used 4096 sectors = 2MB below. >>>>> It's the granularity the era target housekeeps blocks for. >>>>> >>>>>> >>>>>> I tried the following command: >>>>>> >>>>>> # dmsetup create MyEra --table "0 160000 era >>>>>> /dev/mapper/mpathap1 /dev/mapper/mpathbp1 4096" >>>>> >>>>> May not be same physical device for data and metadata! >>>>> If it is, thta'd explain your oops. >>>>> >>>>> # dmsetup create MyEra --table "0 $(blockdev --getsz >>>>> /dev/mapper/mpathbp) era whatever_disctinct_metadata_device >>>>> /dev/mapper/mpathbp1 8" >>>>> >>>>> Would use 8 sector block size (which is tiny!) with disctinct >>>>> metadata and data devices (presumabyl your ext4 sits on >>>>> /dev/mapper/mpathbp1) >>>>> >>>>> Heinz >>>>> >>>>> >>>>> Heinz, thank you for your help! >>>>> >>>>> My origin dev is 10G so total sector number is 20971520. >>>>> /dev/mapper/mpathbp1 (/dev/dm-6) is the metadata dev, >>>>> /dev/mapper/mpathap1 (/dev/dm-7) is the origin dev, they are on >>>>> different LUNs. >>>>> >>>>> I tried the following commands: >>>>> >>>>> # dmsetup create MyEra --table "0 20971520 /dev/mapper/mpathbp1 >>>>> /dev/mapper/mpathap1 8" >>>>> # Target type name /dev/mapper/mpathbp1 is too long. >>>>> # Command failed >>>>> >>>>> # dmsetup create MyEra --table "0 20971520 /dev/dm-6 /dev/dm-7 8" >>>>> # device-mapper: reload ioctl on MyEra failed: Invalid argument >>>>> # Command failed >>>>> >>>>> And when executing the second command I see the following in >>>>> /var/log/messages: >>>>> >>>>> Sep 25 06:37:48 localhost kernel: device-mapper: table: 253:8: >>>>> /dev/dm-6: unknown target type >>>>> Sep 25 06:37:48 localhost kernel: device-mapper: ioctl: error >>>>> adding target to table >>>>> Sep 25 06:37:48 localhost multipathd: dm-8: remove map (uevent) >>>>> Sep 25 06:37:48 localhost multipathd: dm-8: devmap not registered, >>>>> can't remove >>>>> Sep 25 06:37:48 localhost multipathd: dm-8: remove map (uevent) >>>>> Sep 25 06:37:48 localhost multipathd: dm-8: devmap not registered, >>>>> can't remove >>>>> >>>>> Then I run 'mkfs.ext4 /dev/dm-6' and tried the second command >>>>> again but got the same result. >>>>> >>>>> Any idea? >>>>> >>>>> Thanks >>>>> Romu >>>>> >>>>> >>>>> -- >>>>> dm-devel mailing list >>>>> dm-devel@redhat.com >>>>> https://www.redhat.com/mailman/listinfo/dm-devel >>>> >>>> >>>> >>>> -- >>>> dm-devel mailing list >>>> dm-devel@redhat.com >>>> https://www.redhat.com/mailman/listinfo/dm-devel >>> >>> >>> >>> -- >>> dm-devel mailing list >>> dm-devel@redhat.com >>> https://www.redhat.com/mailman/listinfo/dm-devel >> >> -- >> =============================================================== >> Heinz Mauelshagen +49 2626 141200 >> Consulting Development Engineer FAX +49 2626 924446 >> Red Hat GmbH >> Am Sonnenhang 11 >> 56242 Marienrachdorf >> Germanyheinzm@redhat.com >> =============================================================== >> >> >> -- >> dm-devel mailing list >> dm-devel@redhat.com >> https://www.redhat.com/mailman/listinfo/dm-devel > > > > -- > dm-devel mailing list > dm-devel@redhat.com > https://www.redhat.com/mailman/listinfo/dm-devel [-- Attachment #1.2: Type: text/html, Size: 21335 bytes --] [-- Attachment #2: Type: text/plain, Size: 0 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2014-09-29 10:12 UTC | newest] Thread overview: 13+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2014-09-15 2:24 RHEL6.6 dm-cache and dm-era Romu Hu 2014-09-15 20:25 ` Brassow Jonathan 2014-09-18 8:27 ` Romu Hu 2014-09-22 12:35 ` Romu 2014-09-24 4:10 ` Brassow Jonathan 2014-09-25 6:10 ` Romu Hu 2014-09-25 10:19 ` Heinz Mauelshagen 2014-09-25 14:02 ` Romu 2014-09-25 14:57 ` Heinz Mauelshagen 2014-09-26 3:16 ` Romu Hu 2014-09-26 10:33 ` Heinz Mauelshagen 2014-09-29 2:31 ` Romu Hu 2014-09-29 10:12 ` Heinz Mauelshagen
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.