From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx2.redhat.com (mx2.redhat.com [10.255.15.25]) by int-mx2.corp.redhat.com (8.13.1/8.13.1) with ESMTP id l17EHEQF029305 for ; Wed, 7 Feb 2007 09:17:14 -0500 Received: from souja.net (souja.net [64.22.224.60]) by mx2.redhat.com (8.13.1/8.13.1) with ESMTP id l17EHBXC016449 for ; Wed, 7 Feb 2007 09:17:12 -0500 Mime-Version: 1.0 (Apple Message framework v752.3) Content-Type: multipart/alternative; boundary=Apple-Mail-4--722945206 Message-Id: From: Jayson Vantuyl Date: Wed, 7 Feb 2007 08:16:23 -0600 Subject: [linux-lvm] Strange LVM Error With AoE Disks Reply-To: LVM general discussion and development List-Id: LVM general discussion and development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , List-Id: To: Linux LVM List Cc: Coraid Support --Apple-Mail-4--722945206 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Hello, We have been using Coraid's ATA-Over-Ethernet shelves for a while with much success. Recently, we added a second shelf (numbered 1) to our first shelf (numbered 0). CLVM has been running on the old shelf perfectly fine. As soon as I added the second shelf, attempting to lvcreate a new lv utilizing the new disks generated roughly the following errors: Error locking on node ey00-02: Internal lvm error, check syslog Error locking on node ey00-05: Internal lvm error, check syslog Error locking on node ey00-01: Internal lvm error, check syslog Error locking on node ey00-00: Internal lvm error, check syslog Error locking on node ey00-04: Internal lvm error, check syslog Error locking on node ey00-03: Internal lvm error, check syslog Failed to activate new LV. All of the nodes show the following errors in syslog: Feb 7 06:09:36 ey00-00 lvm[4869]: Couldn't find all physical volumes for volume group ey00-data. Feb 7 06:09:37 ey00-00 lvm[4869]: Couldn't find device with uuid '0Cot9Z-BHjK-2Nkw-eEdy-fbFF-Wh1q-qhRaut'. Feb 7 06:09:37 ey00-00 lvm[4869]: Couldn't find all physical volumes for volume group ey00-data. Feb 7 06:09:37 ey00-00 lvm[4869]: Couldn't find device with uuid '0Cot9Z-BHjK-2Nkw-eEdy-fbFF-Wh1q-qhRaut'. Feb 7 06:09:37 ey00-00 lvm[4869]: Couldn't find all physical volumes for volume group ey00-data. Feb 7 06:09:37 ey00-00 lvm[4869]: Couldn't find device with uuid '0Cot9Z-BHjK-2Nkw-eEdy-fbFF-Wh1q-qhRaut'. Feb 7 06:09:37 ey00-00 lvm[4869]: Couldn't find all physical volumes for volume group ey00-data. Feb 7 06:09:37 ey00-00 lvm[4869]: Couldn't find device with uuid '0Cot9Z-BHjK-2Nkw-eEdy-fbFF-Wh1q-qhRaut'. Feb 7 06:09:37 ey00-00 lvm[4869]: Couldn't find all physical volumes for volume group ey00-data. Feb 7 06:09:37 ey00-00 lvm[4869]: Volume group for uuid not found: WWbD8SXOsAJzDYRCFQiciQho84Rl99nVF7QbO0ArRxnH4cZeKgzG0Nx4gbEhgALU Inspecting the lvm.conf shows that these devices are the new ones that were added. Even more bizarre, pvscan finds them just fine on all nodes. The only thing I can note about these devices that is particularly different is that they appear to be using minor numbers above 256. Note this ls output: brw-rw---- 1 root disk 152, 288 Feb 7 04:43 /dev/etherd/e1.2 brw-rw---- 1 root disk 152, 289 Feb 7 06:10 /dev/etherd/e1.2p1 brw-rw---- 1 root disk 152, 304 Feb 7 04:44 /dev/etherd/e1.3 brw-rw---- 1 root disk 152, 305 Feb 7 06:10 /dev/etherd/e1.3p1 Is there a known problem with LVM or CLVM related to large device minor numbers? -- Jayson Vantuyl Systems Architect Engine Yard jvantuyl@engineyard.com --Apple-Mail-4--722945206 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1 Hello,

We have been using Coraid's = ATA-Over-Ethernet shelves for a while with much success.

Recently, we added a second = shelf (numbered 1) to our first shelf (numbered 0).=A0 CLVM has been = running on the old shelf perfectly fine.

As soon as I added the = second shelf, attempting to lvcreate a new lv utilizing the new disks = generated roughly the following errors:

=A0 Error locking on node = ey00-02: Internal lvm error, check syslog
=A0 Error locking on = node ey00-05: Internal lvm error, check syslog
=A0 Error = locking on node ey00-01: Internal lvm error, check syslog
=A0 = Error locking on node ey00-00: Internal lvm error, check = syslog
=A0 Error locking on node ey00-04: Internal lvm error, = check syslog
=A0 Error locking on node ey00-03: Internal lvm = error, check syslog
=A0 Failed to activate new = LV.

All of = the nodes show the following errors in syslog:

Feb=A0 7 06:09:36 ey00-00 = lvm[4869]: Couldn't find all physical volumes for volume group = ey00-data.
Feb=A0 7 06:09:37 ey00-00 lvm[4869]: Couldn't find = device with uuid = '0Cot9Z-BHjK-2Nkw-eEdy-fbFF-Wh1q-qhRaut'.
Feb=A0 7 06:09:37 = ey00-00 lvm[4869]: Couldn't find all physical volumes for volume group = ey00-data.
Feb=A0 7 06:09:37 ey00-00 lvm[4869]: Couldn't find = device with uuid = '0Cot9Z-BHjK-2Nkw-eEdy-fbFF-Wh1q-qhRaut'.
Feb=A0 7 06:09:37 = ey00-00 lvm[4869]: Couldn't find all physical volumes for volume group = ey00-data.
Feb=A0 7 06:09:37 ey00-00 lvm[4869]: Couldn't find = device with uuid = '0Cot9Z-BHjK-2Nkw-eEdy-fbFF-Wh1q-qhRaut'.
Feb=A0 7 06:09:37 = ey00-00 lvm[4869]: Couldn't find all physical volumes for volume group = ey00-data.
Feb=A0 7 06:09:37 ey00-00 lvm[4869]: Couldn't find = device with uuid = '0Cot9Z-BHjK-2Nkw-eEdy-fbFF-Wh1q-qhRaut'.
Feb=A0 7 06:09:37 = ey00-00 lvm[4869]: Couldn't find all physical volumes for volume group = ey00-data.
Feb=A0 7 06:09:37 ey00-00 lvm[4869]: Volume group = for uuid not found: = WWbD8SXOsAJzDYRCFQiciQho84Rl99nVF7QbO0ArRxnH4cZeKgzG0Nx4gbEhgALU

Inspecting the = lvm.conf shows that these devices are the new ones that were = added.

Even = more bizarre, pvscan finds them just fine on all nodes.

The only thing I can note = about these devices that is particularly different is that they appear = to be using minor numbers above 256.=A0 Note this ls = output:

brw-rw---- 1 root disk 152, = 288 Feb=A0 7 04:43 /dev/etherd/e1.2
brw-rw---- 1 root disk = 152, 289 Feb=A0 7 06:10 /dev/etherd/e1.2p1
brw-rw---- 1 root = disk 152, 304 Feb=A0 7 04:44 /dev/etherd/e1.3
brw-rw---- 1 = root disk 152, 305 Feb=A0 7 06:10 /dev/etherd/e1.3p1

Is there a known problem = with LVM or CLVM related to large device minor numbers?

Systems = Architect

=

= --Apple-Mail-4--722945206--