* NFSoRDMA not working with KVM when cache disabled
@ 2012-08-31 17:05 ` Andrew Holway
2012-08-31 17:32 ` Andrew Holway
` (3 more replies)
0 siblings, 4 replies; 10+ messages in thread
From: Andrew Holway @ 2012-08-31 17:05 UTC (permalink / raw)
To: kvm
Hi,
I am trying to host KVM machines on an NFSoRDMA mount.
This works:
-drive file=/mnt/vm001.img,if=none,id=drive-virtio-disk0,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0
This Doesn't!
-drive file=/mnt/vm001.img,if=none,id=drive-virtio-disk0,format=raw,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=2
Any ideas why this could be?
Thanks,
Andrew
dmesg:
vda:
vda: vda1
vda: vda1
vda: vda1 vda2
vda: vda1 vda2
vda: vda1 vda2 vda3
vda: vda1 vda2 vda3
vda: vda1 vda2 vda3 vda4 < >
vda: vda1 vda2 vda3 vda4 < >
vda: vda1 vda2 vda3 vda4 < vda5 >
vda: vda1 vda2 vda3 vda4 < vda5 >
vda: vda1 vda2 vda3 vda4 < vda5 vda6 >
vda: vda1 vda2 vda3 vda4 < vda5 vda6 >
Adding 15998968k swap on /dev/vda5. Priority:-1 extents:1 across:15998968k
EXT3-fs (vda1): error: can't find ext3 filesystem on dev vda1.
/var/log/messages
Aug 31 18:53:46 (none) node-installer: Mounting disks.
Aug 31 18:53:46 (none) node-installer: Updating device status: mounting disks
Aug 31 18:53:46 (none) node-installer: Detecting device '/dev/sda': not found
Aug 31 18:53:46 (none) node-installer: Detecting device '/dev/hda': not found
Aug 31 18:53:46 (none) node-installer: Detecting device '/dev/vda': found
Aug 31 18:53:46 (none) node-installer: swapon /dev/vda5
Aug 31 18:53:46 (none) node-installer: mkdir -p /localdisk/
Aug 31 18:53:46 (none) kernel: Adding 15998968k swap on /dev/vda5. Priority:-1 extents:1 across:15998968k
Aug 31 18:53:46 (none) node-installer: Mounting /dev/vda1 on /localdisk/
Aug 31 18:53:46 (none) node-installer: mount -t ext3 -o defaults,noatime,nodiratime /dev/vda1 /localdisk/
Aug 31 18:53:46 (none) node-installer: mount: wrong fs type, bad option, bad superblock on /dev/vda1,
Aug 31 18:53:46 (none) node-installer: missing codepage or helper program, or other error
Aug 31 18:53:46 (none) node-installer: In some cases useful info is found in syslog - try
Aug 31 18:53:46 (none) node-installer: dmesg | tail or so
Aug 31 18:53:46 (none) node-installer:
Aug 31 18:53:46 (none) node-installer: Command failed.
Aug 31 18:53:46 (none) node-installer: Running: "mount -t ext3 -o defaults,noatime,nodiratime /dev/vda1 /localdisk/" failed:
Aug 31 18:53:46 (none) node-installer: Non zero exit code: 32
Aug 31 18:53:46 (none) kernel: EXT3-fs (vda1): error: can't find ext3 filesystem on dev vda1.
Aug 31 18:53:46 (none) node-installer: Failed to mount disks. (Exit code 12, signal 0)
Aug 31 18:53:46 (none) node-installer: There was a fatal problem. This node can not be installed until the problem is corrected.
Aug 31 18:53:46 (none) node-installer: The error was: failed to mount disks
Aug 31 18:53:46 (none) node-installer: Updating device status: failed to mount disks
Aug 31 18:53:28 (none) node-installer: Creating primary partition /dev/vda3.
Aug 31 18:53:28 (none) node-installer: parted -s -- /dev/vda mkpart primary ext3 22530 24578
Aug 31 18:53:28 (none) kernel: vda: vda1 vda2 vda3
Aug 31 18:53:28 (none) node-installer: dd if=/dev/zero of=/dev/vda3 bs=1k count=4
Aug 31 18:53:28 (none) node-installer: 4+0 records in
Aug 31 18:53:28 (none) node-installer: 4+0 records out
Aug 31 18:53:28 (none) node-installer: 4096 bytes (4.1 kB) copied, 0.000842261 s, 4.9 MB/s
Aug 31 18:53:28 (none) node-installer: Running partprobe on device: /dev/vda
Aug 31 18:53:28 (none) kernel: vda: vda1 vda2 vda3
Aug 31 18:53:45 (none) node-installer: Creating ext3 filesystem on /dev/vda3
Aug 31 18:53:45 (none) node-installer: mke2fs -j /dev/vda3
Aug 31 18:53:45 (none) node-installer: mke2fs 1.41.12 (17-May-2010)
Aug 31 18:53:45 (none) node-installer: Filesystem label=
Aug 31 18:53:45 (none) node-installer: OS type: Linux
Aug 31 18:53:45 (none) node-installer: Block size=4096 (log=2)
Aug 31 18:53:45 (none) node-installer: Fragment size=4096 (log=2)
Aug 31 18:53:45 (none) node-installer: Stride=0 blocks, Stripe width=0 blocks
Aug 31 18:53:45 (none) node-installer: 125184 inodes, 499968 blocks
Aug 31 18:53:45 (none) node-installer: 24998 blocks (5.00%) reserved for the super user
Aug 31 18:53:45 (none) node-installer: First data block=0
Aug 31 18:53:45 (none) node-installer: Maximum filesystem blocks=515899392
Aug 31 18:53:45 (none) node-installer: 16 block groups
Aug 31 18:53:45 (none) node-installer: 32768 blocks per group, 32768 fragments per group
Aug 31 18:53:45 (none) node-installer: 7824 inodes per group
Aug 31 18:53:45 (none) node-installer: Superblock backups stored on blocks:
Aug 31 18:53:45 (none) node-installer: #01132768, 98304, 163840, 229376, 294912
Aug 31 18:53:45 (none) node-installer:
Aug 31 18:53:45 (none) node-installer: Writing inode tables: 0/16#010#010#010#010#010 1/16#010#010#010#010#010 2/16#010#010#010#010#010 3/16#010#010#010#010#010 4/16#010#010#010#010#010 5/16#010#010#010#010#010 6/16#010#010#010#010#010 7/16#010#010#010#010#010 8/16#010#010#010#010#010 9/16#010#010#010#010#01010/16#010#010#010#010#01011/16#010#010#010#010#01012/16#010#010#010#010#01013/16#010#010#010#010#01014/16#010#010#010#010#01015/16#010#010#010#010#010done
Aug 31 18:53:45 (none) node-installer: Creating journal (8192 blocks): done
Aug 31 18:53:45 (none) node-installer: Writing superblocks and filesystem accounting information: done
Aug 31 18:53:45 (none) node-installer:
Aug 31 18:53:45 (none) node-installer: This filesystem will be automatically checked every 24 mounts or
Aug 31 18:53:45 (none) node-installer: 180 days, whichever comes first. Use tune2fs -c or -i to override.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: NFSoRDMA not working with KVM when cache disabled
2012-08-31 17:05 ` NFSoRDMA not working with KVM when cache disabled Andrew Holway
@ 2012-08-31 17:32 ` Andrew Holway
[not found] ` <rt-3.8.7-5952-1346432760-1448.2699-4-0@BrightComputing>
` (2 subsequent siblings)
3 siblings, 0 replies; 10+ messages in thread
From: Andrew Holway @ 2012-08-31 17:32 UTC (permalink / raw)
To: kvm
A bit more....
[<installer> root@vm001 ~]# dumpe2fs /dev/vda1
dumpe2fs 1.41.12 (17-May-2010)
dumpe2fs: Bad magic number in super-block while trying to open /dev/vda1
Couldn't find valid filesystem superblock.
[<installer> root@vm001 ~]# fdisk -ul
Disk /dev/vda: 48.3 GB, 48318382080 bytes
255 heads, 63 sectors/track, 5874 cylinders, total 94371840 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00098bc1
Device Boot Start End Blocks Id System
/dev/vda1 1 40000000 20000000 83 Linux
/dev/vda2 40001536 44001279 1999872 83 Linux
/dev/vda3 44003328 48003071 1999872 83 Linux
/dev/vda4 48005120 94371839 23183360 f W95 Ext'd (LBA)
/dev/vda5 48007168 80005119 15998976 82 Linux swap / Solaris
/dev/vda6 80007168 94371839 7182336 83 LinuxOn Aug 31, 2012, at 7:05 PM, Andrew Holway wrote:
> Hi,
>
> I am trying to host KVM machines on an NFSoRDMA mount.
>
> This works:
>
> -drive file=/mnt/vm001.img,if=none,id=drive-virtio-disk0,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0
>
> This Doesn't!
>
> -drive file=/mnt/vm001.img,if=none,id=drive-virtio-disk0,format=raw,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=2
>
> Any ideas why this could be?
>
> Thanks,
>
> Andrew
>
>
>
> dmesg:
>
> vda:
> vda: vda1
> vda: vda1
> vda: vda1 vda2
> vda: vda1 vda2
> vda: vda1 vda2 vda3
> vda: vda1 vda2 vda3
> vda: vda1 vda2 vda3 vda4 < >
> vda: vda1 vda2 vda3 vda4 < >
> vda: vda1 vda2 vda3 vda4 < vda5 >
> vda: vda1 vda2 vda3 vda4 < vda5 >
> vda: vda1 vda2 vda3 vda4 < vda5 vda6 >
> vda: vda1 vda2 vda3 vda4 < vda5 vda6 >
> Adding 15998968k swap on /dev/vda5. Priority:-1 extents:1 across:15998968k
> EXT3-fs (vda1): error: can't find ext3 filesystem on dev vda1.
>
> /var/log/messages
>
> Aug 31 18:53:46 (none) node-installer: Mounting disks.
> Aug 31 18:53:46 (none) node-installer: Updating device status: mounting disks
> Aug 31 18:53:46 (none) node-installer: Detecting device '/dev/sda': not found
> Aug 31 18:53:46 (none) node-installer: Detecting device '/dev/hda': not found
> Aug 31 18:53:46 (none) node-installer: Detecting device '/dev/vda': found
> Aug 31 18:53:46 (none) node-installer: swapon /dev/vda5
> Aug 31 18:53:46 (none) node-installer: mkdir -p /localdisk/
> Aug 31 18:53:46 (none) kernel: Adding 15998968k swap on /dev/vda5. Priority:-1 extents:1 across:15998968k
> Aug 31 18:53:46 (none) node-installer: Mounting /dev/vda1 on /localdisk/
> Aug 31 18:53:46 (none) node-installer: mount -t ext3 -o defaults,noatime,nodiratime /dev/vda1 /localdisk/
> Aug 31 18:53:46 (none) node-installer: mount: wrong fs type, bad option, bad superblock on /dev/vda1,
> Aug 31 18:53:46 (none) node-installer: missing codepage or helper program, or other error
> Aug 31 18:53:46 (none) node-installer: In some cases useful info is found in syslog - try
> Aug 31 18:53:46 (none) node-installer: dmesg | tail or so
> Aug 31 18:53:46 (none) node-installer:
> Aug 31 18:53:46 (none) node-installer: Command failed.
> Aug 31 18:53:46 (none) node-installer: Running: "mount -t ext3 -o defaults,noatime,nodiratime /dev/vda1 /localdisk/" failed:
> Aug 31 18:53:46 (none) node-installer: Non zero exit code: 32
> Aug 31 18:53:46 (none) kernel: EXT3-fs (vda1): error: can't find ext3 filesystem on dev vda1.
> Aug 31 18:53:46 (none) node-installer: Failed to mount disks. (Exit code 12, signal 0)
> Aug 31 18:53:46 (none) node-installer: There was a fatal problem. This node can not be installed until the problem is corrected.
> Aug 31 18:53:46 (none) node-installer: The error was: failed to mount disks
> Aug 31 18:53:46 (none) node-installer: Updating device status: failed to mount disks
>
> Aug 31 18:53:28 (none) node-installer: Creating primary partition /dev/vda3.
> Aug 31 18:53:28 (none) node-installer: parted -s -- /dev/vda mkpart primary ext3 22530 24578
> Aug 31 18:53:28 (none) kernel: vda: vda1 vda2 vda3
> Aug 31 18:53:28 (none) node-installer: dd if=/dev/zero of=/dev/vda3 bs=1k count=4
> Aug 31 18:53:28 (none) node-installer: 4+0 records in
> Aug 31 18:53:28 (none) node-installer: 4+0 records out
> Aug 31 18:53:28 (none) node-installer: 4096 bytes (4.1 kB) copied, 0.000842261 s, 4.9 MB/s
> Aug 31 18:53:28 (none) node-installer: Running partprobe on device: /dev/vda
> Aug 31 18:53:28 (none) kernel: vda: vda1 vda2 vda3
>
> Aug 31 18:53:45 (none) node-installer: Creating ext3 filesystem on /dev/vda3
> Aug 31 18:53:45 (none) node-installer: mke2fs -j /dev/vda3
> Aug 31 18:53:45 (none) node-installer: mke2fs 1.41.12 (17-May-2010)
> Aug 31 18:53:45 (none) node-installer: Filesystem label=
> Aug 31 18:53:45 (none) node-installer: OS type: Linux
> Aug 31 18:53:45 (none) node-installer: Block size=4096 (log=2)
> Aug 31 18:53:45 (none) node-installer: Fragment size=4096 (log=2)
> Aug 31 18:53:45 (none) node-installer: Stride=0 blocks, Stripe width=0 blocks
> Aug 31 18:53:45 (none) node-installer: 125184 inodes, 499968 blocks
> Aug 31 18:53:45 (none) node-installer: 24998 blocks (5.00%) reserved for the super user
> Aug 31 18:53:45 (none) node-installer: First data block=0
> Aug 31 18:53:45 (none) node-installer: Maximum filesystem blocks=515899392
> Aug 31 18:53:45 (none) node-installer: 16 block groups
> Aug 31 18:53:45 (none) node-installer: 32768 blocks per group, 32768 fragments per group
> Aug 31 18:53:45 (none) node-installer: 7824 inodes per group
> Aug 31 18:53:45 (none) node-installer: Superblock backups stored on blocks:
> Aug 31 18:53:45 (none) node-installer: #01132768, 98304, 163840, 229376, 294912
> Aug 31 18:53:45 (none) node-installer:
> Aug 31 18:53:45 (none) node-installer: Writing inode tables: 0/16#010#010#010#010#010 1/16#010#010#010#010#010 2/16#010#010#010#010#010 3/16#010#010#010#010#010 4/16#010#010#010#010#010 5/16#010#010#010#010#010 6/16#010#010#010#010#010 7/16#010#010#010#010#010 8/16#010#010#010#010#010 9/16#010#010#010#010#01010/16#010#010#010#010#01011/16#010#010#010#010#01012/16#010#010#010#010#01013/16#010#010#010#010#01014/16#010#010#010#010#01015/16#010#010#010#010#010done
> Aug 31 18:53:45 (none) node-installer: Creating journal (8192 blocks): done
> Aug 31 18:53:45 (none) node-installer: Writing superblocks and filesystem accounting information: done
> Aug 31 18:53:45 (none) node-installer:
> Aug 31 18:53:45 (none) node-installer: This filesystem will be automatically checked every 24 mounts or
> Aug 31 18:53:45 (none) node-installer: 180 days, whichever comes first. Use tune2fs -c or -i to override.
>
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Bright Cluster Manager Support #2699] NFSoRDMA not working with KVM when cache disabled
[not found] ` <CALiqupEThihQAj1mf43f2pNztXaFoxojYkTcrMbFkc1G=+vEAA@mail.gmail.com>
@ 2012-08-31 21:20 ` Martijn de Vries
[not found] ` <056E76E7-5AAB-4622-A245-62725340D504@syseleven.de>
0 siblings, 1 reply; 10+ messages in thread
From: Martijn de Vries @ 2012-08-31 21:20 UTC (permalink / raw)
To: a.holway; +Cc: kvm
Hi Andrew,
That's pretty strange. I am not a KVM expert, so I don't know what happens
under the hood when you disable cache. Have you tried doing a FULL install?
Best regards,
Martijn
On Fri, Aug 31, 2012 at 7:06 PM, Andrew Holway
<support@brightcomputing.com>wrote:
>
> Fri Aug 31 19:06:00 2012: Request 2699 was acted upon.
> Transaction: Ticket created by a.holway@syseleven.de
> Queue: Bright
> Subject: NFSoRDMA not working with KVM when cache disabled
> Owner: Nobody
> Requestors: a.holway@syseleven.de
> Status: new
> Ticket <URL:
> http://support.brightcomputing.com/rt/Ticket/Display.html?id=2699 >
>
>
> Hi,
>
> I am trying to host KVM machines on an NFSoRDMA mount.
>
> This works:
>
> -drive file=/mnt/vm001.img,if=none,id=drive-virtio-disk0,format=raw
> -device
> virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0
>
> This Doesn't!
>
> -drive
> file=/mnt/vm001.img,if=none,id=drive-virtio-disk0,format=raw,cache=none
> -device
> virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=2
>
> Any ideas why this could be?
>
> Thanks,
>
> Andrew
>
>
>
> dmesg:
>
> vda:
> vda: vda1
> vda: vda1
> vda: vda1 vda2
> vda: vda1 vda2
> vda: vda1 vda2 vda3
> vda: vda1 vda2 vda3
> vda: vda1 vda2 vda3 vda4 < >
> vda: vda1 vda2 vda3 vda4 < >
> vda: vda1 vda2 vda3 vda4 < vda5 >
> vda: vda1 vda2 vda3 vda4 < vda5 >
> vda: vda1 vda2 vda3 vda4 < vda5 vda6 >
> vda: vda1 vda2 vda3 vda4 < vda5 vda6 >
> Adding 15998968k swap on /dev/vda5. Priority:-1 extents:1 across:15998968k
> EXT3-fs (vda1): error: can't find ext3 filesystem on dev vda1.
>
> /var/log/messages
>
> Aug 31 18:53:46 (none) node-installer: Mounting disks.
> Aug 31 18:53:46 (none) node-installer: Updating device status: mounting
> disks
> Aug 31 18:53:46 (none) node-installer: Detecting device '/dev/sda': not
> found
> Aug 31 18:53:46 (none) node-installer: Detecting device '/dev/hda': not
> found
> Aug 31 18:53:46 (none) node-installer: Detecting device '/dev/vda': found
> Aug 31 18:53:46 (none) node-installer: swapon /dev/vda5
> Aug 31 18:53:46 (none) node-installer: mkdir -p /localdisk/
> Aug 31 18:53:46 (none) kernel: Adding 15998968k swap on /dev/vda5.
> Priority:-1 extents:1 across:15998968k
> Aug 31 18:53:46 (none) node-installer: Mounting /dev/vda1 on /localdisk/
> Aug 31 18:53:46 (none) node-installer: mount -t ext3 -o
> defaults,noatime,nodiratime /dev/vda1 /localdisk/
> Aug 31 18:53:46 (none) node-installer: mount: wrong fs type, bad option,
> bad superblock on /dev/vda1,
> Aug 31 18:53:46 (none) node-installer: missing codepage or helper
> program, or other error
> Aug 31 18:53:46 (none) node-installer: In some cases useful info is
> found in syslog - try
> Aug 31 18:53:46 (none) node-installer: dmesg | tail or so
> Aug 31 18:53:46 (none) node-installer:
> Aug 31 18:53:46 (none) node-installer: Command failed.
> Aug 31 18:53:46 (none) node-installer: Running: "mount -t ext3 -o
> defaults,noatime,nodiratime /dev/vda1 /localdisk/" failed:
> Aug 31 18:53:46 (none) node-installer: Non zero exit code: 32
> Aug 31 18:53:46 (none) kernel: EXT3-fs (vda1): error: can't find ext3
> filesystem on dev vda1.
> Aug 31 18:53:46 (none) node-installer: Failed to mount disks. (Exit code
> 12, signal 0)
> Aug 31 18:53:46 (none) node-installer: There was a fatal problem. This
> node can not be installed until the problem is corrected.
> Aug 31 18:53:46 (none) node-installer: The error was: failed to mount disks
> Aug 31 18:53:46 (none) node-installer: Updating device status: failed to
> mount disks
>
> Aug 31 18:53:28 (none) node-installer: Creating primary partition
> /dev/vda3.
> Aug 31 18:53:28 (none) node-installer: parted -s -- /dev/vda mkpart
> primary ext3 22530 24578
> Aug 31 18:53:28 (none) kernel: vda: vda1 vda2 vda3
> Aug 31 18:53:28 (none) node-installer: dd if=/dev/zero of=/dev/vda3 bs=1k
> count=4
> Aug 31 18:53:28 (none) node-installer: 4+0 records in
> Aug 31 18:53:28 (none) node-installer: 4+0 records out
> Aug 31 18:53:28 (none) node-installer: 4096 bytes (4.1 kB) copied,
> 0.000842261 s, 4.9 MB/s
> Aug 31 18:53:28 (none) node-installer: Running partprobe on device:
> /dev/vda
> Aug 31 18:53:28 (none) kernel: vda: vda1 vda2 vda3
>
> Aug 31 18:53:45 (none) node-installer: Creating ext3 filesystem on
> /dev/vda3
> Aug 31 18:53:45 (none) node-installer: mke2fs -j /dev/vda3
> Aug 31 18:53:45 (none) node-installer: mke2fs 1.41.12 (17-May-2010)
> Aug 31 18:53:45 (none) node-installer: Filesystem label=
> Aug 31 18:53:45 (none) node-installer: OS type: Linux
> Aug 31 18:53:45 (none) node-installer: Block size=4096 (log=2)
> Aug 31 18:53:45 (none) node-installer: Fragment size=4096 (log=2)
> Aug 31 18:53:45 (none) node-installer: Stride=0 blocks, Stripe width=0
> blocks
> Aug 31 18:53:45 (none) node-installer: 125184 inodes, 499968 blocks
> Aug 31 18:53:45 (none) node-installer: 24998 blocks (5.00%) reserved for
> the super user
> Aug 31 18:53:45 (none) node-installer: First data block=0
> Aug 31 18:53:45 (none) node-installer: Maximum filesystem blocks=515899392
> Aug 31 18:53:45 (none) node-installer: 16 block groups
> Aug 31 18:53:45 (none) node-installer: 32768 blocks per group, 32768
> fragments per group
> Aug 31 18:53:45 (none) node-installer: 7824 inodes per group
> Aug 31 18:53:45 (none) node-installer: Superblock backups stored on blocks:
> Aug 31 18:53:45 (none) node-installer: #01132768, 98304, 163840, 229376,
> 294912
> Aug 31 18:53:45 (none) node-installer:
> Aug 31 18:53:45 (none) node-installer: Writing inode tables:
> 0/16#010#010#010#010#010 1/16#010#010#010#010#010 2/16#010#010#010#010#010
> 3/16#010#010#010#010#010 4/16#010#010#010#010#010 5/16#010#010#010#010#010
> 6/16#010#010#010#010#010 7/16#010#010#010#010#010 8/16#010#010#010#010#010
> 9/16#010#010#010#010#01010/16#010#010#010#010#01011/16#010#010#010#010#01012/16#010#010#010#010#01013/16#010#010#010#010#01014/16#010#010#010#010#01015/16#010#010#010#010#010done
> Aug 31 18:53:45 (none) node-installer: Creating journal (8192 blocks): done
> Aug 31 18:53:45 (none) node-installer: Writing superblocks and filesystem
> accounting information: done
> Aug 31 18:53:45 (none) node-installer:
> Aug 31 18:53:45 (none) node-installer: This filesystem will be
> automatically checked every 24 mounts or
> Aug 31 18:53:45 (none) node-installer: 180 days, whichever comes first.
> Use tune2fs -c or -i to override.
>
>
>
>
>
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Bright Cluster Manager Support #2699] NFSoRDMA not working with KVM when cache disabled
[not found] ` <056E76E7-5AAB-4622-A245-62725340D504@syseleven.de>
@ 2012-09-01 11:51 ` Andrew Holway
[not found] ` <rt-3.8.7-5947-1346500280-1759.2699-16-0@BrightComputing>
1 sibling, 0 replies; 10+ messages in thread
From: Andrew Holway @ 2012-09-01 11:51 UTC (permalink / raw)
Cc: kvm
Hi,
That is FULL install (I think)
I create a new virtual machine each time I test it.
Ta,
Andrew
On Aug 31, 2012, at 11:20 PM, Martijn de Vries wrote:
> Hi Andrew,
>
> That's pretty strange. I am not a KVM expert, so I don't know what happens
> under the hood when you disable cache. Have you tried doing a FULL install?
>
> Best regards,
>
> Martijn
>
> On Fri, Aug 31, 2012 at 7:06 PM, Andrew Holway
> <support@brightcomputing.com>wrote:
>
>>
>> Fri Aug 31 19:06:00 2012: Request 2699 was acted upon.
>> Transaction: Ticket created by a.holway@syseleven.de
>> Queue: Bright
>> Subject: NFSoRDMA not working with KVM when cache disabled
>> Owner: Nobody
>> Requestors: a.holway@syseleven.de
>> Status: new
>> Ticket <URL:
>> http://support.brightcomputing.com/rt/Ticket/Display.html?id=2699 >
>>
>>
>> Hi,
>>
>> I am trying to host KVM machines on an NFSoRDMA mount.
>>
>> This works:
>>
>> -drive file=/mnt/vm001.img,if=none,id=drive-virtio-disk0,format=raw
>> -device
>> virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0
>>
>> This Doesn't!
>>
>> -drive
>> file=/mnt/vm001.img,if=none,id=drive-virtio-disk0,format=raw,cache=none
>> -device
>> virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=2
>>
>> Any ideas why this could be?
>>
>> Thanks,
>>
>> Andrew
>>
>>
>>
>> dmesg:
>>
>> vda:
>> vda: vda1
>> vda: vda1
>> vda: vda1 vda2
>> vda: vda1 vda2
>> vda: vda1 vda2 vda3
>> vda: vda1 vda2 vda3
>> vda: vda1 vda2 vda3 vda4 < >
>> vda: vda1 vda2 vda3 vda4 < >
>> vda: vda1 vda2 vda3 vda4 < vda5 >
>> vda: vda1 vda2 vda3 vda4 < vda5 >
>> vda: vda1 vda2 vda3 vda4 < vda5 vda6 >
>> vda: vda1 vda2 vda3 vda4 < vda5 vda6 >
>> Adding 15998968k swap on /dev/vda5. Priority:-1 extents:1 across:15998968k
>> EXT3-fs (vda1): error: can't find ext3 filesystem on dev vda1.
>>
>> /var/log/messages
>>
>> Aug 31 18:53:46 (none) node-installer: Mounting disks.
>> Aug 31 18:53:46 (none) node-installer: Updating device status: mounting
>> disks
>> Aug 31 18:53:46 (none) node-installer: Detecting device '/dev/sda': not
>> found
>> Aug 31 18:53:46 (none) node-installer: Detecting device '/dev/hda': not
>> found
>> Aug 31 18:53:46 (none) node-installer: Detecting device '/dev/vda': found
>> Aug 31 18:53:46 (none) node-installer: swapon /dev/vda5
>> Aug 31 18:53:46 (none) node-installer: mkdir -p /localdisk/
>> Aug 31 18:53:46 (none) kernel: Adding 15998968k swap on /dev/vda5.
>> Priority:-1 extents:1 across:15998968k
>> Aug 31 18:53:46 (none) node-installer: Mounting /dev/vda1 on /localdisk/
>> Aug 31 18:53:46 (none) node-installer: mount -t ext3 -o
>> defaults,noatime,nodiratime /dev/vda1 /localdisk/
>> Aug 31 18:53:46 (none) node-installer: mount: wrong fs type, bad option,
>> bad superblock on /dev/vda1,
>> Aug 31 18:53:46 (none) node-installer: missing codepage or helper
>> program, or other error
>> Aug 31 18:53:46 (none) node-installer: In some cases useful info is
>> found in syslog - try
>> Aug 31 18:53:46 (none) node-installer: dmesg | tail or so
>> Aug 31 18:53:46 (none) node-installer:
>> Aug 31 18:53:46 (none) node-installer: Command failed.
>> Aug 31 18:53:46 (none) node-installer: Running: "mount -t ext3 -o
>> defaults,noatime,nodiratime /dev/vda1 /localdisk/" failed:
>> Aug 31 18:53:46 (none) node-installer: Non zero exit code: 32
>> Aug 31 18:53:46 (none) kernel: EXT3-fs (vda1): error: can't find ext3
>> filesystem on dev vda1.
>> Aug 31 18:53:46 (none) node-installer: Failed to mount disks. (Exit code
>> 12, signal 0)
>> Aug 31 18:53:46 (none) node-installer: There was a fatal problem. This
>> node can not be installed until the problem is corrected.
>> Aug 31 18:53:46 (none) node-installer: The error was: failed to mount disks
>> Aug 31 18:53:46 (none) node-installer: Updating device status: failed to
>> mount disks
>>
>> Aug 31 18:53:28 (none) node-installer: Creating primary partition
>> /dev/vda3.
>> Aug 31 18:53:28 (none) node-installer: parted -s -- /dev/vda mkpart
>> primary ext3 22530 24578
>> Aug 31 18:53:28 (none) kernel: vda: vda1 vda2 vda3
>> Aug 31 18:53:28 (none) node-installer: dd if=/dev/zero of=/dev/vda3 bs=1k
>> count=4
>> Aug 31 18:53:28 (none) node-installer: 4+0 records in
>> Aug 31 18:53:28 (none) node-installer: 4+0 records out
>> Aug 31 18:53:28 (none) node-installer: 4096 bytes (4.1 kB) copied,
>> 0.000842261 s, 4.9 MB/s
>> Aug 31 18:53:28 (none) node-installer: Running partprobe on device:
>> /dev/vda
>> Aug 31 18:53:28 (none) kernel: vda: vda1 vda2 vda3
>>
>> Aug 31 18:53:45 (none) node-installer: Creating ext3 filesystem on
>> /dev/vda3
>> Aug 31 18:53:45 (none) node-installer: mke2fs -j /dev/vda3
>> Aug 31 18:53:45 (none) node-installer: mke2fs 1.41.12 (17-May-2010)
>> Aug 31 18:53:45 (none) node-installer: Filesystem label=
>> Aug 31 18:53:45 (none) node-installer: OS type: Linux
>> Aug 31 18:53:45 (none) node-installer: Block size=4096 (log=2)
>> Aug 31 18:53:45 (none) node-installer: Fragment size=4096 (log=2)
>> Aug 31 18:53:45 (none) node-installer: Stride=0 blocks, Stripe width=0
>> blocks
>> Aug 31 18:53:45 (none) node-installer: 125184 inodes, 499968 blocks
>> Aug 31 18:53:45 (none) node-installer: 24998 blocks (5.00%) reserved for
>> the super user
>> Aug 31 18:53:45 (none) node-installer: First data block=0
>> Aug 31 18:53:45 (none) node-installer: Maximum filesystem blocks=515899392
>> Aug 31 18:53:45 (none) node-installer: 16 block groups
>> Aug 31 18:53:45 (none) node-installer: 32768 blocks per group, 32768
>> fragments per group
>> Aug 31 18:53:45 (none) node-installer: 7824 inodes per group
>> Aug 31 18:53:45 (none) node-installer: Superblock backups stored on blocks:
>> Aug 31 18:53:45 (none) node-installer: #01132768, 98304, 163840, 229376,
>> 294912
>> Aug 31 18:53:45 (none) node-installer:
>> Aug 31 18:53:45 (none) node-installer: Writing inode tables:
>> 0/16#010#010#010#010#010 1/16#010#010#010#010#010 2/16#010#010#010#010#010
>> 3/16#010#010#010#010#010 4/16#010#010#010#010#010 5/16#010#010#010#010#010
>> 6/16#010#010#010#010#010 7/16#010#010#010#010#010 8/16#010#010#010#010#010
>> 9/16#010#010#010#010#01010/16#010#010#010#010#01011/16#010#010#010#010#01012/16#010#010#010#010#01013/16#010#010#010#010#01014/16#010#010#010#010#01015/16#010#010#010#010#010done
>> Aug 31 18:53:45 (none) node-installer: Creating journal (8192 blocks): done
>> Aug 31 18:53:45 (none) node-installer: Writing superblocks and filesystem
>> accounting information: done
>> Aug 31 18:53:45 (none) node-installer:
>> Aug 31 18:53:45 (none) node-installer: This filesystem will be
>> automatically checked every 24 mounts or
>> Aug 31 18:53:45 (none) node-installer: 180 days, whichever comes first.
>> Use tune2fs -c or -i to override.
>>
>>
>>
>>
>>
>>
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Bright Cluster Manager Support #2699] NFSoRDMA not working with KVM when cache disabled
[not found] ` <CALiqupEgJ2=2Mbg1trvQVEbgUProJj87dPzJOOG6XR2GwWZf5Q@mail.gmail.com>
@ 2012-09-02 20:46 ` Martijn de Vries
[not found] ` <rt-3.8.7-19587-1346618782-561.2699-6-0@BrightComputing>
1 sibling, 0 replies; 10+ messages in thread
From: Martijn de Vries @ 2012-09-02 20:46 UTC (permalink / raw)
To: a.holway; +Cc: kvm
Hi Andrew,
In that case I don't know. I would suggest not to disable the cache.
Best regards,
Martijn
On Sat, Sep 1, 2012 at 1:51 PM, Andrew Holway
<support@brightcomputing.com>wrote:
>
> <URL: http://support.brightcomputing.com/rt/Ticket/Display.html?id=2699 >
>
> Hi,
>
> That is FULL install (I think)
>
> I create a new virtual machine each time I test it.
>
> Ta,
>
> Andrew
>
>
> On Aug 31, 2012, at 11:20 PM, Martijn de Vries wrote:
>
> > Hi Andrew,
> >
> > That's pretty strange. I am not a KVM expert, so I don't know what
> happens
> > under the hood when you disable cache. Have you tried doing a FULL
> install?
> >
> > Best regards,
> >
> > Martijn
> >
> > On Fri, Aug 31, 2012 at 7:06 PM, Andrew Holway
> > <support@brightcomputing.com>wrote:
> >
> >>
> >> Fri Aug 31 19:06:00 2012: Request 2699 was acted upon.
> >> Transaction: Ticket created by a.holway@syseleven.de
> >> Queue: Bright
> >> Subject: NFSoRDMA not working with KVM when cache disabled
> >> Owner: Nobody
> >> Requestors: a.holway@syseleven.de
> >> Status: new
> >> Ticket <URL:
> >> http://support.brightcomputing.com/rt/Ticket/Display.html?id=2699 >
> >>
> >>
> >> Hi,
> >>
> >> I am trying to host KVM machines on an NFSoRDMA mount.
> >>
> >> This works:
> >>
> >> -drive file=/mnt/vm001.img,if=none,id=drive-virtio-disk0,format=raw
> >> -device
> >>
> virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0
> >>
> >> This Doesn't!
> >>
> >> -drive
> >> file=/mnt/vm001.img,if=none,id=drive-virtio-disk0,format=raw,cache=none
> >> -device
> >>
> virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=2
> >>
> >> Any ideas why this could be?
> >>
> >> Thanks,
> >>
> >> Andrew
> >>
> >>
> >>
> >> dmesg:
> >>
> >> vda:
> >> vda: vda1
> >> vda: vda1
> >> vda: vda1 vda2
> >> vda: vda1 vda2
> >> vda: vda1 vda2 vda3
> >> vda: vda1 vda2 vda3
> >> vda: vda1 vda2 vda3 vda4 < >
> >> vda: vda1 vda2 vda3 vda4 < >
> >> vda: vda1 vda2 vda3 vda4 < vda5 >
> >> vda: vda1 vda2 vda3 vda4 < vda5 >
> >> vda: vda1 vda2 vda3 vda4 < vda5 vda6 >
> >> vda: vda1 vda2 vda3 vda4 < vda5 vda6 >
> >> Adding 15998968k swap on /dev/vda5. Priority:-1 extents:1
> across:15998968k
> >> EXT3-fs (vda1): error: can't find ext3 filesystem on dev vda1.
> >>
> >> /var/log/messages
> >>
> >> Aug 31 18:53:46 (none) node-installer: Mounting disks.
> >> Aug 31 18:53:46 (none) node-installer: Updating device status: mounting
> >> disks
> >> Aug 31 18:53:46 (none) node-installer: Detecting device '/dev/sda': not
> >> found
> >> Aug 31 18:53:46 (none) node-installer: Detecting device '/dev/hda': not
> >> found
> >> Aug 31 18:53:46 (none) node-installer: Detecting device '/dev/vda':
> found
> >> Aug 31 18:53:46 (none) node-installer: swapon /dev/vda5
> >> Aug 31 18:53:46 (none) node-installer: mkdir -p /localdisk/
> >> Aug 31 18:53:46 (none) kernel: Adding 15998968k swap on /dev/vda5.
> >> Priority:-1 extents:1 across:15998968k
> >> Aug 31 18:53:46 (none) node-installer: Mounting /dev/vda1 on /localdisk/
> >> Aug 31 18:53:46 (none) node-installer: mount -t ext3 -o
> >> defaults,noatime,nodiratime /dev/vda1 /localdisk/
> >> Aug 31 18:53:46 (none) node-installer: mount: wrong fs type, bad option,
> >> bad superblock on /dev/vda1,
> >> Aug 31 18:53:46 (none) node-installer: missing codepage or helper
> >> program, or other error
> >> Aug 31 18:53:46 (none) node-installer: In some cases useful info
> is
> >> found in syslog - try
> >> Aug 31 18:53:46 (none) node-installer: dmesg | tail or so
> >> Aug 31 18:53:46 (none) node-installer:
> >> Aug 31 18:53:46 (none) node-installer: Command failed.
> >> Aug 31 18:53:46 (none) node-installer: Running: "mount -t ext3 -o
> >> defaults,noatime,nodiratime /dev/vda1 /localdisk/" failed:
> >> Aug 31 18:53:46 (none) node-installer: Non zero exit code: 32
> >> Aug 31 18:53:46 (none) kernel: EXT3-fs (vda1): error: can't find ext3
> >> filesystem on dev vda1.
> >> Aug 31 18:53:46 (none) node-installer: Failed to mount disks. (Exit code
> >> 12, signal 0)
> >> Aug 31 18:53:46 (none) node-installer: There was a fatal problem. This
> >> node can not be installed until the problem is corrected.
> >> Aug 31 18:53:46 (none) node-installer: The error was: failed to mount
> disks
> >> Aug 31 18:53:46 (none) node-installer: Updating device status: failed to
> >> mount disks
> >>
> >> Aug 31 18:53:28 (none) node-installer: Creating primary partition
> >> /dev/vda3.
> >> Aug 31 18:53:28 (none) node-installer: parted -s -- /dev/vda mkpart
> >> primary ext3 22530 24578
> >> Aug 31 18:53:28 (none) kernel: vda: vda1 vda2 vda3
> >> Aug 31 18:53:28 (none) node-installer: dd if=/dev/zero of=/dev/vda3
> bs=1k
> >> count=4
> >> Aug 31 18:53:28 (none) node-installer: 4+0 records in
> >> Aug 31 18:53:28 (none) node-installer: 4+0 records out
> >> Aug 31 18:53:28 (none) node-installer: 4096 bytes (4.1 kB) copied,
> >> 0.000842261 s, 4.9 MB/s
> >> Aug 31 18:53:28 (none) node-installer: Running partprobe on device:
> >> /dev/vda
> >> Aug 31 18:53:28 (none) kernel: vda: vda1 vda2 vda3
> >>
> >> Aug 31 18:53:45 (none) node-installer: Creating ext3 filesystem on
> >> /dev/vda3
> >> Aug 31 18:53:45 (none) node-installer: mke2fs -j /dev/vda3
> >> Aug 31 18:53:45 (none) node-installer: mke2fs 1.41.12 (17-May-2010)
> >> Aug 31 18:53:45 (none) node-installer: Filesystem label=
> >> Aug 31 18:53:45 (none) node-installer: OS type: Linux
> >> Aug 31 18:53:45 (none) node-installer: Block size=4096 (log=2)
> >> Aug 31 18:53:45 (none) node-installer: Fragment size=4096 (log=2)
> >> Aug 31 18:53:45 (none) node-installer: Stride=0 blocks, Stripe width=0
> >> blocks
> >> Aug 31 18:53:45 (none) node-installer: 125184 inodes, 499968 blocks
> >> Aug 31 18:53:45 (none) node-installer: 24998 blocks (5.00%) reserved for
> >> the super user
> >> Aug 31 18:53:45 (none) node-installer: First data block=0
> >> Aug 31 18:53:45 (none) node-installer: Maximum filesystem
> blocks=515899392
> >> Aug 31 18:53:45 (none) node-installer: 16 block groups
> >> Aug 31 18:53:45 (none) node-installer: 32768 blocks per group, 32768
> >> fragments per group
> >> Aug 31 18:53:45 (none) node-installer: 7824 inodes per group
> >> Aug 31 18:53:45 (none) node-installer: Superblock backups stored on
> blocks:
> >> Aug 31 18:53:45 (none) node-installer: #01132768, 98304, 163840, 229376,
> >> 294912
> >> Aug 31 18:53:45 (none) node-installer:
> >> Aug 31 18:53:45 (none) node-installer: Writing inode tables:
> >> 0/16#010#010#010#010#010 1/16#010#010#010#010#010
> 2/16#010#010#010#010#010
> >> 3/16#010#010#010#010#010 4/16#010#010#010#010#010
> 5/16#010#010#010#010#010
> >> 6/16#010#010#010#010#010 7/16#010#010#010#010#010
> 8/16#010#010#010#010#010
> >>
> 9/16#010#010#010#010#01010/16#010#010#010#010#01011/16#010#010#010#010#01012/16#010#010#010#010#01013/16#010#010#010#010#01014/16#010#010#010#010#01015/16#010#010#010#010#010done
> >> Aug 31 18:53:45 (none) node-installer: Creating journal (8192 blocks):
> done
> >> Aug 31 18:53:45 (none) node-installer: Writing superblocks and
> filesystem
> >> accounting information: done
> >> Aug 31 18:53:45 (none) node-installer:
> >> Aug 31 18:53:45 (none) node-installer: This filesystem will be
> >> automatically checked every 24 mounts or
> >> Aug 31 18:53:45 (none) node-installer: 180 days, whichever comes first.
> >> Use tune2fs -c or -i to override.
> >>
> >>
> >>
> >>
> >>
> >>
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe kvm" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
>
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: NFSoRDMA not working with KVM when cache disabled
2012-08-31 17:05 ` NFSoRDMA not working with KVM when cache disabled Andrew Holway
2012-08-31 17:32 ` Andrew Holway
[not found] ` <rt-3.8.7-5952-1346432760-1448.2699-4-0@BrightComputing>
@ 2012-09-03 9:12 ` Stefan Hajnoczi
2012-09-03 9:58 ` Avi Kivity
3 siblings, 0 replies; 10+ messages in thread
From: Stefan Hajnoczi @ 2012-09-03 9:12 UTC (permalink / raw)
To: Andrew Holway; +Cc: kvm
On Fri, Aug 31, 2012 at 6:05 PM, Andrew Holway <a.holway@syseleven.de> wrote:
> I am trying to host KVM machines on an NFSoRDMA mount.
>
> This works:
>
> -drive file=/mnt/vm001.img,if=none,id=drive-virtio-disk0,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0
>
> This Doesn't!
>
> -drive file=/mnt/vm001.img,if=none,id=drive-virtio-disk0,format=raw,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=2
>
> Any ideas why this could be?
-drive cache=none results in QEMU opening the image file with O_DIRECT.
It's not clear to me what the first error is. The node-installer
output you have posted does not show a series of steps to reproduce
the problem. Can you try running isolated tests with dd or
verify-data (http://people.redhat.com/sct/src/verify-data/)?
Have you tried running a different guest OS, say a Fedora or Debian Live CD?
Stefan
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: NFSoRDMA not working with KVM when cache disabled
2012-08-31 17:05 ` NFSoRDMA not working with KVM when cache disabled Andrew Holway
` (2 preceding siblings ...)
2012-09-03 9:12 ` Stefan Hajnoczi
@ 2012-09-03 9:58 ` Avi Kivity
2012-09-03 11:57 ` Andrew Holway
3 siblings, 1 reply; 10+ messages in thread
From: Avi Kivity @ 2012-09-03 9:58 UTC (permalink / raw)
To: Andrew Holway; +Cc: kvm
On 08/31/2012 08:05 PM, Andrew Holway wrote:
> Hi,
>
> I am trying to host KVM machines on an NFSoRDMA mount.
>
> This works:
>
> -drive file=/mnt/vm001.img,if=none,id=drive-virtio-disk0,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0
>
> This Doesn't!
>
> -drive file=/mnt/vm001.img,if=none,id=drive-virtio-disk0,format=raw,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=2
>
> Any ideas why this could be?
Please try
dd bs=512 if=/mnt/vm001.img of=/mnt/x1
dd bs=512 if=/mnt/vm001.img of=/mnt/x2
dd bs=4096 if=/mnt/vm001.img of=/mnt/y1 iflag=direct oflag=direct
dd bs=4096 if=/mnt/vm001.img of=/mnt/y2 iflag=direct oflag=direct
and report which (if any) of the output files (x1, x2, y1, y2) are
corrupted, by comparing them against the original. This will tell us
whether O_DIRECT is broken, or 512 byte block size, or neither.
--
error compiling committee.c: too many arguments to function
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: NFSoRDMA not working with KVM when cache disabled
2012-09-03 9:58 ` Avi Kivity
@ 2012-09-03 11:57 ` Andrew Holway
2012-09-03 12:08 ` Avi Kivity
0 siblings, 1 reply; 10+ messages in thread
From: Andrew Holway @ 2012-09-03 11:57 UTC (permalink / raw)
To: Avi Kivity; +Cc: kvm
>
> and report which (if any) of the output files (x1, x2, y1, y2) are
> corrupted, by comparing them against the original. This will tell us
> whether O_DIRECT is broken, or 512 byte block size, or neither.
>
Looks like you were directly on the money there. 512, 1K and 2K O_DIRECT looks broken.
[root@node001 mnt]# for f in 512 1024 2048 4096 8192 16384 32768 65536 131072; do dd bs="$f" if=CentOS-6.3-x86_64-netinstall.iso of=hello && md5sum hello && rm -f hello; done409600+0 records in
409600+0 records out
209715200 bytes (210 MB) copied, 0.774718 s, 271 MB/s
690138908de516b6e5d7d180d085c3f3 hello
204800+0 records in
204800+0 records out
209715200 bytes (210 MB) copied, 0.609578 s, 344 MB/s
690138908de516b6e5d7d180d085c3f3 hello
102400+0 records in
102400+0 records out
209715200 bytes (210 MB) copied, 0.490421 s, 428 MB/s
690138908de516b6e5d7d180d085c3f3 hello
51200+0 records in
51200+0 records out
209715200 bytes (210 MB) copied, 0.440156 s, 476 MB/s
690138908de516b6e5d7d180d085c3f3 hello
25600+0 records in
25600+0 records out
209715200 bytes (210 MB) copied, 0.4229 s, 496 MB/s
690138908de516b6e5d7d180d085c3f3 hello
12800+0 records in
12800+0 records out
209715200 bytes (210 MB) copied, 0.40914 s, 513 MB/s
690138908de516b6e5d7d180d085c3f3 hello
6400+0 records in
6400+0 records out
209715200 bytes (210 MB) copied, 0.427247 s, 491 MB/s
690138908de516b6e5d7d180d085c3f3 hello
3200+0 records in
3200+0 records out
209715200 bytes (210 MB) copied, 0.411776 s, 509 MB/s
690138908de516b6e5d7d180d085c3f3 hello
1600+0 records in
1600+0 records out
209715200 bytes (210 MB) copied, 0.417098 s, 503 MB/s
690138908de516b6e5d7d180d085c3f3 hello
[root@node001 mnt]# for f in 512 1024 2048 4096 8192 16384 32768 65536 131072; do dd bs="$f" if=CentOS-6.3-x86_64-netinstall.iso of=hello iflag=direct oflag=direct && md5sum hello && rm -f hello; done
409600+0 records in
409600+0 records out
209715200 bytes (210 MB) copied, 62.3649 s, 3.4 MB/s
aadd0ffe3c9dfa35d8354e99ecac9276 hello
204800+0 records in
204800+0 records out
209715200 bytes (210 MB) copied, 41.3876 s, 5.1 MB/s
336f6da78f93dab591edc18da81f002e hello
102400+0 records in
102400+0 records out
209715200 bytes (210 MB) copied, 21.1712 s, 9.9 MB/s
f4cefe0a05c9b47ba68effdb17dc95d6 hello
51200+0 records in
51200+0 records out
209715200 bytes (210 MB) copied, 10.9631 s, 19.1 MB/s
690138908de516b6e5d7d180d085c3f3 hello
25600+0 records in
25600+0 records out
209715200 bytes (210 MB) copied, 5.4136 s, 38.7 MB/s
690138908de516b6e5d7d180d085c3f3 hello
12800+0 records in
12800+0 records out
209715200 bytes (210 MB) copied, 3.1448 s, 66.7 MB/s
690138908de516b6e5d7d180d085c3f3 hello
6400+0 records in
6400+0 records out
209715200 bytes (210 MB) copied, 1.77304 s, 118 MB/s
690138908de516b6e5d7d180d085c3f3 hello
3200+0 records in
3200+0 records out
209715200 bytes (210 MB) copied, 1.4331 s, 146 MB/s
690138908de516b6e5d7d180d085c3f3 hello
1600+0 records in
1600+0 records out
209715200 bytes (210 MB) copied, 0.922167 s, 227 MB/s
690138908de516b6e5d7d180d085c3f3 hello
>
> --
> error compiling committee.c: too many arguments to function
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: NFSoRDMA not working with KVM when cache disabled
2012-09-03 11:57 ` Andrew Holway
@ 2012-09-03 12:08 ` Avi Kivity
0 siblings, 0 replies; 10+ messages in thread
From: Avi Kivity @ 2012-09-03 12:08 UTC (permalink / raw)
To: Andrew Holway; +Cc: kvm
On 09/03/2012 02:57 PM, Andrew Holway wrote:
>>
>> and report which (if any) of the output files (x1, x2, y1, y2) are
>> corrupted, by comparing them against the original. This will tell us
>> whether O_DIRECT is broken, or 512 byte block size, or neither.
>>
>
>
> Looks like you were directly on the money there. 512, 1K and 2K O_DIRECT looks broken.
It was hardly unexpected. Please report the problem to the nfs/rdma
list (and keep kvm@ copied, nfs/rdma is fairly relevant to kvm).
You can work around it by exposing a 4k logical block size to the guest
(logical_block_size=4k). Note not all guests support this.
--
error compiling committee.c: too many arguments to function
^ permalink raw reply [flat|nested] 10+ messages in thread
* [Bright Cluster Manager Support #2699] NFSoRDMA not working with KVM when cache disabled
[not found] ` <rt-3.8.7-19587-1346618782-561.2699-6-0@BrightComputing>
@ 2012-09-03 13:21 ` Pawan Jaitly
0 siblings, 0 replies; 10+ messages in thread
From: Pawan Jaitly @ 2012-09-03 13:21 UTC (permalink / raw)
To: a.holway; +Cc: kvm
It looks like this issue has been dealt with, so I'm closing its ticket.
* If you would like to reopen the ticket, then please respond to this mail with
the same subject header.
* If you would like to open a new ticket, then please send a mail with a new
subject header.
In either case, the mail must go to the support@brightcomputing.com address.
regards
Pawan
On Sun Sep 02 22:46:22 2012, martijn wrote:
> Hi Andrew,
>
>
> In that case I don't know. I would suggest not to disable the cache.
>
> Best regards,
>
> Martijn
>
> On Sat, Sep 1, 2012 at 1:51 PM, Andrew Holway
> <support@brightcomputing.com>wrote:
>
> >
> > <URL:
> http://support.brightcomputing.com/rt/Ticket/Display.html?id=2699 >
> >
> > Hi,
> >
> > That is FULL install (I think)
> >
> > I create a new virtual machine each time I test it.
> >
> > Ta,
> >
> > Andrew
> >
> >
> > On Aug 31, 2012, at 11:20 PM, Martijn de Vries wrote:
> >
> > > Hi Andrew,
> > >
> > > That's pretty strange. I am not a KVM expert, so I don't know what
> > happens
> > > under the hood when you disable cache. Have you tried doing a FULL
> > install?
> > >
> > > Best regards,
> > >
> > > Martijn
> > >
> > > On Fri, Aug 31, 2012 at 7:06 PM, Andrew Holway
> > > <support@brightcomputing.com>wrote:
> > >
> > >>
> > >> Fri Aug 31 19:06:00 2012: Request 2699 was acted upon.
> > >> Transaction: Ticket created by a.holway@syseleven.de
> > >> Queue: Bright
> > >> Subject: NFSoRDMA not working with KVM when cache disabled
> > >> Owner: Nobody
> > >> Requestors: a.holway@syseleven.de
> > >> Status: new
> > >> Ticket <URL:
> > >> http://support.brightcomputing.com/rt/Ticket/Display.html?id=2699
> >
> > >>
> > >>
> > >> Hi,
> > >>
> > >> I am trying to host KVM machines on an NFSoRDMA mount.
> > >>
> > >> This works:
> > >>
> > >> -drive file=/mnt/vm001.img,if=none,id=drive-virtio-
> disk0,format=raw
> > >> -device
> > >>
> > virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-
> disk0,id=virtio-disk0
> > >>
> > >> This Doesn't!
> > >>
> > >> -drive
> > >> file=/mnt/vm001.img,if=none,id=drive-virtio-
> disk0,format=raw,cache=none
> > >> -device
> > >>
> > virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-
> disk0,id=virtio-disk0,bootindex=2
> > >>
> > >> Any ideas why this could be?
> > >>
> > >> Thanks,
> > >>
> > >> Andrew
> > >>
> > >>
> > >>
> > >> dmesg:
> > >>
> > >> vda:
> > >> vda: vda1
> > >> vda: vda1
> > >> vda: vda1 vda2
> > >> vda: vda1 vda2
> > >> vda: vda1 vda2 vda3
> > >> vda: vda1 vda2 vda3
> > >> vda: vda1 vda2 vda3 vda4 < >
> > >> vda: vda1 vda2 vda3 vda4 < >
> > >> vda: vda1 vda2 vda3 vda4 < vda5 >
> > >> vda: vda1 vda2 vda3 vda4 < vda5 >
> > >> vda: vda1 vda2 vda3 vda4 < vda5 vda6 >
> > >> vda: vda1 vda2 vda3 vda4 < vda5 vda6 >
> > >> Adding 15998968k swap on /dev/vda5. Priority:-1 extents:1
> > across:15998968k
> > >> EXT3-fs (vda1): error: can't find ext3 filesystem on dev vda1.
> > >>
> > >> /var/log/messages
> > >>
> > >> Aug 31 18:53:46 (none) node-installer: Mounting disks.
> > >> Aug 31 18:53:46 (none) node-installer: Updating device status:
> mounting
> > >> disks
> > >> Aug 31 18:53:46 (none) node-installer: Detecting device
> '/dev/sda': not
> > >> found
> > >> Aug 31 18:53:46 (none) node-installer: Detecting device
> '/dev/hda': not
> > >> found
> > >> Aug 31 18:53:46 (none) node-installer: Detecting device
> '/dev/vda':
> > found
> > >> Aug 31 18:53:46 (none) node-installer: swapon /dev/vda5
> > >> Aug 31 18:53:46 (none) node-installer: mkdir -p /localdisk/
> > >> Aug 31 18:53:46 (none) kernel: Adding 15998968k swap on
> /dev/vda5.
> > >> Priority:-1 extents:1 across:15998968k
> > >> Aug 31 18:53:46 (none) node-installer: Mounting /dev/vda1 on
> /localdisk/
> > >> Aug 31 18:53:46 (none) node-installer: mount -t ext3 -o
> > >> defaults,noatime,nodiratime /dev/vda1 /localdisk/
> > >> Aug 31 18:53:46 (none) node-installer: mount: wrong fs type, bad
> option,
> > >> bad superblock on /dev/vda1,
> > >> Aug 31 18:53:46 (none) node-installer: missing codepage or
> helper
> > >> program, or other error
> > >> Aug 31 18:53:46 (none) node-installer: In some cases
> useful info
> > is
> > >> found in syslog - try
> > >> Aug 31 18:53:46 (none) node-installer: dmesg | tail or so
> > >> Aug 31 18:53:46 (none) node-installer:
> > >> Aug 31 18:53:46 (none) node-installer: Command failed.
> > >> Aug 31 18:53:46 (none) node-installer: Running: "mount -t ext3 -o
> > >> defaults,noatime,nodiratime /dev/vda1 /localdisk/" failed:
> > >> Aug 31 18:53:46 (none) node-installer: Non zero exit code: 32
> > >> Aug 31 18:53:46 (none) kernel: EXT3-fs (vda1): error: can't find
> ext3
> > >> filesystem on dev vda1.
> > >> Aug 31 18:53:46 (none) node-installer: Failed to mount disks.
> (Exit code
> > >> 12, signal 0)
> > >> Aug 31 18:53:46 (none) node-installer: There was a fatal problem.
> This
> > >> node can not be installed until the problem is corrected.
> > >> Aug 31 18:53:46 (none) node-installer: The error was: failed to
> mount
> > disks
> > >> Aug 31 18:53:46 (none) node-installer: Updating device status:
> failed to
> > >> mount disks
> > >>
> > >> Aug 31 18:53:28 (none) node-installer: Creating primary partition
> > >> /dev/vda3.
> > >> Aug 31 18:53:28 (none) node-installer: parted -s -- /dev/vda
> mkpart
> > >> primary ext3 22530 24578
> > >> Aug 31 18:53:28 (none) kernel: vda: vda1 vda2 vda3
> > >> Aug 31 18:53:28 (none) node-installer: dd if=/dev/zero
> of=/dev/vda3
> > bs=1k
> > >> count=4
> > >> Aug 31 18:53:28 (none) node-installer: 4+0 records in
> > >> Aug 31 18:53:28 (none) node-installer: 4+0 records out
> > >> Aug 31 18:53:28 (none) node-installer: 4096 bytes (4.1 kB)
> copied,
> > >> 0.000842261 s, 4.9 MB/s
> > >> Aug 31 18:53:28 (none) node-installer: Running partprobe on
> device:
> > >> /dev/vda
> > >> Aug 31 18:53:28 (none) kernel: vda: vda1 vda2 vda3
> > >>
> > >> Aug 31 18:53:45 (none) node-installer: Creating ext3 filesystem
> on
> > >> /dev/vda3
> > >> Aug 31 18:53:45 (none) node-installer: mke2fs -j /dev/vda3
> > >> Aug 31 18:53:45 (none) node-installer: mke2fs 1.41.12 (17-May-
> 2010)
> > >> Aug 31 18:53:45 (none) node-installer: Filesystem label=
> > >> Aug 31 18:53:45 (none) node-installer: OS type: Linux
> > >> Aug 31 18:53:45 (none) node-installer: Block size=4096 (log=2)
> > >> Aug 31 18:53:45 (none) node-installer: Fragment size=4096 (log=2)
> > >> Aug 31 18:53:45 (none) node-installer: Stride=0 blocks, Stripe
> width=0
> > >> blocks
> > >> Aug 31 18:53:45 (none) node-installer: 125184 inodes, 499968
> blocks
> > >> Aug 31 18:53:45 (none) node-installer: 24998 blocks (5.00%)
> reserved for
> > >> the super user
> > >> Aug 31 18:53:45 (none) node-installer: First data block=0
> > >> Aug 31 18:53:45 (none) node-installer: Maximum filesystem
> > blocks=515899392
> > >> Aug 31 18:53:45 (none) node-installer: 16 block groups
> > >> Aug 31 18:53:45 (none) node-installer: 32768 blocks per group,
> 32768
> > >> fragments per group
> > >> Aug 31 18:53:45 (none) node-installer: 7824 inodes per group
> > >> Aug 31 18:53:45 (none) node-installer: Superblock backups stored
> on
> > blocks:
> > >> Aug 31 18:53:45 (none) node-installer: #01132768, 98304, 163840,
> 229376,
> > >> 294912
> > >> Aug 31 18:53:45 (none) node-installer:
> > >> Aug 31 18:53:45 (none) node-installer: Writing inode tables:
> > >> 0/16#010#010#010#010#010 1/16#010#010#010#010#010
> > 2/16#010#010#010#010#010
> > >> 3/16#010#010#010#010#010 4/16#010#010#010#010#010
> > 5/16#010#010#010#010#010
> > >> 6/16#010#010#010#010#010 7/16#010#010#010#010#010
> > 8/16#010#010#010#010#010
> > >>
> >
>
9/16#010#010#010#010#01010/16#010#010#010#010#01011/16#010#010#010#010#01012/16#010#010#010#010#01013/16#010#010#010#010#01014/16#010#010#010#010#01015/16#010#010#010#010#010done
> > >> Aug 31 18:53:45 (none) node-installer: Creating journal (8192
> blocks):
> > done
> > >> Aug 31 18:53:45 (none) node-installer: Writing superblocks and
> > filesystem
> > >> accounting information: done
> > >> Aug 31 18:53:45 (none) node-installer:
> > >> Aug 31 18:53:45 (none) node-installer: This filesystem will be
> > >> automatically checked every 24 mounts or
> > >> Aug 31 18:53:45 (none) node-installer: 180 days, whichever comes
> first.
> > >> Use tune2fs -c or -i to override.
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe kvm" in
> > > the body of a message to majordomo@vger.kernel.org
> > > More majordomo info at http://vger.kernel.org/majordomo-info.html
> >
> >
> >
> >
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2012-09-03 13:21 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <RT-Ticket-2699@BrightComputing>
2012-08-31 17:05 ` NFSoRDMA not working with KVM when cache disabled Andrew Holway
2012-08-31 17:32 ` Andrew Holway
[not found] ` <rt-3.8.7-5952-1346432760-1448.2699-4-0@BrightComputing>
[not found] ` <CALiqupEThihQAj1mf43f2pNztXaFoxojYkTcrMbFkc1G=+vEAA@mail.gmail.com>
2012-08-31 21:20 ` [Bright Cluster Manager Support #2699] " Martijn de Vries
[not found] ` <056E76E7-5AAB-4622-A245-62725340D504@syseleven.de>
2012-09-01 11:51 ` Andrew Holway
[not found] ` <rt-3.8.7-5947-1346500280-1759.2699-16-0@BrightComputing>
[not found] ` <CALiqupEgJ2=2Mbg1trvQVEbgUProJj87dPzJOOG6XR2GwWZf5Q@mail.gmail.com>
2012-09-02 20:46 ` Martijn de Vries
[not found] ` <rt-3.8.7-19587-1346618782-561.2699-6-0@BrightComputing>
2012-09-03 13:21 ` Pawan Jaitly
2012-09-03 9:12 ` Stefan Hajnoczi
2012-09-03 9:58 ` Avi Kivity
2012-09-03 11:57 ` Andrew Holway
2012-09-03 12:08 ` Avi Kivity
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).