[kvm-ppc-devel] [PATCH] kvm(ppc)-userspace: initialize virtio-block

[kvm-ppc-devel] [PATCH] kvm(ppc)-userspace: initialize virtio-block

[ehrhardt]

From: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>

virtio-block support is atm initialized in qemu/hw/pc.c which is only called
for the "standart pc" platform.
In our case we need to call the virtio-block initialization from our platform
init. It might be worth  moving this initialization into a funvtion in 
virtio-block.c and only call that initializer from any platform init that
supports virtio - any suggestions/preferences if we should do it now or when
another architecture needs that - Anthony ?

This patch adds that initialization for the kvmppc bamboo platform.

Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
---

[diffstat]
 ppc440_bamboo.c |   13 +++++++++++++
 1 files changed, 13 insertions(+)

diff --git a/qemu/hw/ppc440_bamboo.c b/qemu/hw/ppc440_bamboo.c
--- a/qemu/hw/ppc440_bamboo.c
+++ b/qemu/hw/ppc440_bamboo.c
@@ -13,6 +13,7 @@
 #include "net.h"
 #include "hw.h"
 #include "pci.h"
+#include "sysemu.h"
 #include "ppc440.h"
 #include "qemu-kvm.h"
 #include "device_tree.h"
@@ -185,6 +186,18 @@ void bamboo_init(ram_addr_t ram_size, in
 		pci_nic_init(pci->bus, nd, -1);
 	}
 
+	/* Add virtio block devices */
+	if (pci) {
+		int index;
+		int unit_id = 0;
+
+		while ((index = drive_get_index(IF_VIRTIO, 0, unit_id)) != -1) {
+			virtio_blk_init(pci->bus, 0x1AF4, 0x1001,
+				drives_table[index].bdrv);
+			unit_id++;
+		}
+	}
+
 	printf("%s: DONE\n", __func__);
 }
 

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
kvm-ppc-devel mailing list
kvm-ppc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-ppc-devel

Re: [kvm-ppc-devel] [PATCH] kvm(ppc)-userspace: initialize

[Christian Ehrhardt]

ehrhardt@linux.vnet.ibm.com wrote:
> From: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
> 
> virtio-block support is atm initialized in qemu/hw/pc.c which is only called
> for the "standart pc" platform.
> In our case we need to call the virtio-block initialization from our platform
> init. It might be worth  moving this initialization into a funvtion in 
> virtio-block.c and only call that initializer from any platform init that
> supports virtio - any suggestions/preferences if we should do it now or when
> another architecture needs that - Anthony ?
> 
> This patch adds that initialization for the kvmppc bamboo platform.
> 
> Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
> ---
> 
> [diffstat]
>  ppc440_bamboo.c |   13 +++++++++++++
>  1 files changed, 13 insertions(+)

[...]

FYI some output using virtio-block (still buggy on kvmppc)

Initialization when booting guest (VIO = virtio drvier in guest, U-VIO = virtio emulation in host kvm userspace)

PCI: Enabling device 0000:00:01.0 (0000 -> 0001)
PCI: Enabling device 0000:00:02.0 (0000 -> 0001)
U-VIO virtio_blk_update_config - vdev 0x10d0b008
VIO virtblk_probe - vdev 0xc8925200
U-VIO virtio_blk_get_features - vdev 0x10d0b008
U-VIO virtio_blk_get_features - vdev 0x10d0b008
U-VIO virtio_blk_get_features - vdev 0x10d0b008
blk_queue_max_hw_segments: set to minimum 1
 vda:<3>VIO do_virtblk_request - request_queue 0xc8936000
VIO do_req - request_queue 0xc8936000 request 0xc8937320 virtio_blk 0xc880c000 (8secs@0x0 - <NULL> Maj -929934324)
U-VIO virtio_blk_handle_output - vdev 0x10d0b008 vq 0x10d0b20c
VIO blk_done - vq 0xc88bcc00
VIO do_virtblk_request - request_queue 0xc8936000



We have no dynamic /dev so anyone who wants to test that will need to create /dev/vda for his needs.
This is easy to check using /proc/partitions.
Well I think that block size is broken here and using fdisk to check the partitions on that (unformatted) disk killed the guest kernel.
This needs more debugging, but to enable it for our platform is the first step - comments are welcome.

bash-3.00# cat /proc/partitions
major minor  #blocks  name
[...]
 254     0 22517998136852480 vda		<- ?broken?

The related device nod looks like this:
brw-r--r--  1 root root 254, 0 Mar 28  2008 /dev/vda



Below I added the log of the guest crash FYI:
bash-3.00# fdisk -l /dev/vda
VIO do_virtblk_request - request_queue 0xc8936000
VIO do_req - request_queue 0xc8936000 request 0xc89373e8 virtio_blk 0xc880c000 (8secs@0x0 - <NULL> Maj -929934324)
VIO do_req - request_queue 0xc8936000 request 0xc89374b0 virtio_blk 0xc880c000 (8secs@0x0 - <NULL> Maj -929934324)
VIO do_req - request_queue 0xc8936000 request 0xc8937578 virtio_blk 0xc880c000 (8secs@0x0 - <NULL> Maj -929934324)
VIO do_req - request_queue 0xc8936000 request 0xc8937320 virtio_blk 0xc880c000 (8secs@0x0 - <NULL> Maj -929934324)
U-VIO virtio_blk_handle_output - vdev 0x10d0b008 vq 0x10d0b20c
VIO blk_done - vq 0xc88bcc00
VIO do_virtblk_request - request_queue 0xc8936000
VIO do_virtblk_request - request_queue 0xc8936000
BUG: recent printk recursion!
Bamboo
Modules linked in:
NIP: c010bcec LR: c010d170 CTR: ffffffff
REGS: c8999ae0 TRAP: 0300   Not tainted  (2.6.25-rc3-dirty)
MSR: 00021002 <ME>  CR: 84004044  XER: 20000000
DEAR: fffffff8, ESR: 00000000
TASK = c89f6000[990] 'fdisk' THREAD: c8998000
GPR00: ffffffff c8999b90 c89f6000 fffffff8 ffffffff c02092e9 00000000 00000000
GPR08: c02092e8 fffffff8 ffffffff 00000008 24004028 1002c7b0 00000000 100c8510
GPR16: 100c0000 00000000 00000000 c8402208 c0250000 00000400 c899abc8 c0250000
GPR24: 00021002 00000000 c026b954 fffffff8 ffffffff c8999cc0 c026bd54 c026b9b3
NIP [c010bcec] strnlen+0x10/0x48
LR [c010d170] vsnprintf+0x558/0x83c
Call Trace:
[c8999b90] [c010cfec] vsnprintf+0x3d4/0x83c (unreliable)
[c8999bc0] [c010d46c] vscnprintf+0x18/0x38
[c8999bd0] [c0024c04] vprintk+0x70/0x3a8
[c8999c50] [c0024f8c] printk+0x50/0x60
[c8999cd0] [c013a3c4] do_virtblk_request+0x9c/0x2a8
[c8999d00] [c00f9048] __generic_unplug_device+0x3c/0x50
[c8999d10] [c00f90f0] generic_unplug_device+0x14/0x28
[c8999d20] [c00f8474] blk_unplug+0x20/0x30
[c8999d30] [c009a678] block_sync_page+0x54/0x70
[c8999d40] [c004e58c] sync_page+0x50/0x74
[c8999d50] [c004e5c0] sync_page_killable+0x10/0x4c
[c8999d60] [c01be7d0] __wait_on_bit_lock+0x9c/0xf8
[c8999d90] [c004ded8] __lock_page_killable+0x50/0x60
[c8999dc0] [c0050c6c] generic_file_aio_read+0x3bc/0x5bc
[c8999e30] [c0075330] do_sync_read+0xc0/0x134
[c8999ef0] [c0075c30] vfs_read+0xcc/0x16c
[c8999f10] [c007643c] sys_read+0x4c/0x90
[c8999f40] [c000d0cc] ret_from_syscall+0x0/0x3c
Instruction dump:
71200020 4082ffe4 38000000 98030001 881f0000 2f800000 409eff7c 4bffff9c
7c802379 7c691b78 7c0903a6 4182002c <88030000> 2f800000 419e0020 38630001
---[ end trace 74c6a5de04fea905 ]---


-- 

Grüsse / regards, 
Christian Ehrhardt
IBM Linux Technology Center, Open Virtualization

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
kvm-ppc-devel mailing list
kvm-ppc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-ppc-devel

Re: [kvm-ppc-devel] [PATCH] kvm(ppc)-userspace:

[Hollis Blanchard]

On Tue, 2008-04-01 at 14:08 +0200, Christian Ehrhardt wrote:
> FYI some output using virtio-block (still buggy on kvmppc)
> 
> Initialization when booting guest (VIO = virtio drvier in guest, U-VIO = virtio emulation in host kvm userspace)
> 
> PCI: Enabling device 0000:00:01.0 (0000 -> 0001)
> PCI: Enabling device 0000:00:02.0 (0000 -> 0001)
> U-VIO virtio_blk_update_config - vdev 0x10d0b008
> VIO virtblk_probe - vdev 0xc8925200
> U-VIO virtio_blk_get_features - vdev 0x10d0b008
> U-VIO virtio_blk_get_features - vdev 0x10d0b008
> U-VIO virtio_blk_get_features - vdev 0x10d0b008
> blk_queue_max_hw_segments: set to minimum 1
>  vda:<3>VIO do_virtblk_request - request_queue 0xc8936000
> VIO do_req - request_queue 0xc8936000 request 0xc8937320 virtio_blk 0xc880c000 (8secs@0x0 - <NULL> Maj -929934324)
> U-VIO virtio_blk_handle_output - vdev 0x10d0b008 vq 0x10d0b20c
> VIO blk_done - vq 0xc88bcc00
> VIO do_virtblk_request - request_queue 0xc8936000
> 
> 
> 
> We have no dynamic /dev so anyone who wants to test that will need to create /dev/vda for his needs.
> This is easy to check using /proc/partitions.
> Well I think that block size is broken here and using fdisk to check the partitions on that (unformatted) disk killed the guest kernel.
> This needs more debugging, but to enable it for our platform is the first step - comments are welcome.
> 
> bash-3.00# cat /proc/partitions
> major minor  #blocks  name
> [...]
>  254     0 22517998136852480 vda		<- ?broken?

My guess is this is run-of-the-mill endianness mismatch.
22517998136852480 = 0x00500000_00000000, which 64-bit byteswapped would
be 0x5000, and that's probably a reasonable number of 512-byte blocks.
Is your disk image 10MB?

Why would we have a problem, since both guest and host are big-endian?
Because virtio is a PCI device, and PCI MMIO are LE, so
__virtio_config_val() in the guest is (correctly) using le64_to_cpu().

Why didn't we have problems with virtio-net? Because virtio-net doesn't
seem to have anything interesting in PCI config space. virtio-blk's
config space contains the capacity and a few other pieces of
information.

The fix needs to be in qemu, and given the lack of qemu endianness
infrastructure, I'm afraid it will be a hack. See
http://svn.savannah.nongnu.org/viewvc/trunk/hw/e1000.c?root=qemu&r1@46&r2@45&pathrev@46 for reference. We all know that TARGET_WORDS_BIGENDIAN is totally wrong, but unfortunately it also seems to be the only (accidentally) working solution in qemu without major IO system rework. :(

-- 
Hollis Blanchard
IBM Linux Technology Center


-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
kvm-ppc-devel mailing list
kvm-ppc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-ppc-devel

Re: [kvm-ppc-devel] [PATCH] kvm(ppc)-userspace: initialize

[Anthony Liguori]

Hollis Blanchard wrote:
> On Tue, 2008-04-01 at 14:08 +0200, Christian Ehrhardt wrote:
>   
>> FYI some output using virtio-block (still buggy on kvmppc)
>>
>> Initialization when booting guest (VIO = virtio drvier in guest, U-VIO = virtio emulation in host kvm userspace)
>>
>> PCI: Enabling device 0000:00:01.0 (0000 -> 0001)
>> PCI: Enabling device 0000:00:02.0 (0000 -> 0001)
>> U-VIO virtio_blk_update_config - vdev 0x10d0b008
>> VIO virtblk_probe - vdev 0xc8925200
>> U-VIO virtio_blk_get_features - vdev 0x10d0b008
>> U-VIO virtio_blk_get_features - vdev 0x10d0b008
>> U-VIO virtio_blk_get_features - vdev 0x10d0b008
>> blk_queue_max_hw_segments: set to minimum 1
>>  vda:<3>VIO do_virtblk_request - request_queue 0xc8936000
>> VIO do_req - request_queue 0xc8936000 request 0xc8937320 virtio_blk 0xc880c000 (8secs@0x0 - <NULL> Maj -929934324)
>> U-VIO virtio_blk_handle_output - vdev 0x10d0b008 vq 0x10d0b20c
>> VIO blk_done - vq 0xc88bcc00
>> VIO do_virtblk_request - request_queue 0xc8936000
>>
>>
>>
>> We have no dynamic /dev so anyone who wants to test that will need to create /dev/vda for his needs.
>> This is easy to check using /proc/partitions.
>> Well I think that block size is broken here and using fdisk to check the partitions on that (unformatted) disk killed the guest kernel.
>> This needs more debugging, but to enable it for our platform is the first step - comments are welcome.
>>
>> bash-3.00# cat /proc/partitions
>> major minor  #blocks  name
>> [...]
>>  254     0 22517998136852480 vda		<- ?broken?
>>     
>
> My guess is this is run-of-the-mill endianness mismatch.
> 22517998136852480 = 0x00500000_00000000, which 64-bit byteswapped would
> be 0x5000, and that's probably a reasonable number of 512-byte blocks.
> Is your disk image 10MB?
>
> Why would we have a problem, since both guest and host are big-endian?
> Because virtio is a PCI device, and PCI MMIO are LE, so
> __virtio_config_val() in the guest is (correctly) using le64_to_cpu().
>
> Why didn't we have problems with virtio-net? Because virtio-net doesn't
> seem to have anything interesting in PCI config space. virtio-blk's
> config space contains the capacity and a few other pieces of
> information.
>
> The fix needs to be in qemu, and given the lack of qemu endianness
> infrastructure, I'm afraid it will be a hack. See
> http://svn.savannah.nongnu.org/viewvc/trunk/hw/e1000.c?root=qemu&r1@46&r2@45&pathrev@46 for reference. We all know that TARGET_WORDS_BIGENDIAN is totally wrong, but unfortunately it also seems to be the only (accidentally) working solution in qemu without major IO system rework. :(
>
>   

It's actually not so bad since the virtio config space is already read 
one byte at a time.  The following should help.

diff --git a/qemu/hw/virtio-blk.c b/qemu/hw/virtio-blk.c
index 0f55d2a..492bd7f 100644
--- a/qemu/hw/virtio-blk.c
+++ b/qemu/hw/virtio-blk.c
@@ -134,8 +134,8 @@ static void virtio_blk_update_config(VirtIODevice 
*vdev, uin
     int64_t capacity;
 
     bdrv_get_geometry(s->bs, &capacity);
-    blkcfg.capacity = capacity;
-    blkcfg.seg_max = 128 - 2;
+    blkcfg.capacity = cpu_to_le64(capacity);
+    blkcfg.seg_max = cpu_to_le32(128 - 2);
     memcpy(config, &blkcfg, sizeof(blkcfg));
 }
 



-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
kvm-ppc-devel mailing list
kvm-ppc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-ppc-devel

Re: [kvm-ppc-devel] booting from virtio-blk

[Hollis Blanchard]

[-- Attachment #1: Type: text/plain, Size: 2579 bytes --]

On Tue, 2008-04-01 at 09:46 -0500, Anthony Liguori wrote:
> Hollis Blanchard wrote:
> > On Tue, 2008-04-01 at 14:08 +0200, Christian Ehrhardt wrote:
> >> bash-3.00# cat /proc/partitions
> >> major minor  #blocks  name
> >> [...]
> >>  254     0 22517998136852480 vda		<- ?broken?
> >>     
> >
> > My guess is this is run-of-the-mill endianness mismatch.
> > 22517998136852480 = 0x00500000_00000000, which 64-bit byteswapped would
> > be 0x5000, and that's probably a reasonable number of 512-byte blocks.
> > Is your disk image 10MB?
> >
> > Why would we have a problem, since both guest and host are big-endian?
> > Because virtio is a PCI device, and PCI MMIO are LE, so
> > __virtio_config_val() in the guest is (correctly) using le64_to_cpu().
> >
> > Why didn't we have problems with virtio-net? Because virtio-net doesn't
> > seem to have anything interesting in PCI config space. virtio-blk's
> > config space contains the capacity and a few other pieces of
> > information.
> >
> > The fix needs to be in qemu, and given the lack of qemu endianness
> > infrastructure, I'm afraid it will be a hack. See
> > http://svn.savannah.nongnu.org/viewvc/trunk/hw/e1000.c?root=qemu&r1=4046&r2=4045&pathrev=4046 for reference. We all know that TARGET_WORDS_BIGENDIAN is totally wrong, but unfortunately it also seems to be the only (accidentally) working solution in qemu without major IO system rework. :(
> 
> It's actually not so bad since the virtio config space is already read 
> one byte at a time.  The following should help.
> 
> diff --git a/qemu/hw/virtio-blk.c b/qemu/hw/virtio-blk.c
> index 0f55d2a..492bd7f 100644
> --- a/qemu/hw/virtio-blk.c
> +++ b/qemu/hw/virtio-blk.c
> @@ -134,8 +134,8 @@ static void virtio_blk_update_config(VirtIODevice 
> *vdev, uin
>      int64_t capacity;
> 
>      bdrv_get_geometry(s->bs, &capacity);
> -    blkcfg.capacity = capacity;
> -    blkcfg.seg_max = 128 - 2;
> +    blkcfg.capacity = cpu_to_le64(capacity);
> +    blkcfg.seg_max = cpu_to_le32(128 - 2);
>      memcpy(config, &blkcfg, sizeof(blkcfg));
>  }

Thanks Anthony, you've saved me a lot of debug time! Rusty, doing 64-bit
PCI config space accesses with ioread8() definitely violates the
principle of least surprises, and would have taken me a long time to
track down. :(

Attached is a boot log of a PowerPC guest booting from virtio-blk root.

"ramdisk_image" is the standard ~4MB image provided with DENX Embedded
Linux Development Kit. Booting is also *way* faster than NFS root (a few
seconds to get to a shell :) .

-- 
Hollis Blanchard
IBM Linux Technology Center

[-- Attachment #2: virtio-blk.log --]
[-- Type: text/x-log, Size: 5196 bytes --]

bash-3.00# ./qemu-system-ppcemb -M bamboo -nographic -kernel ../../uImage.bamboo -L ../pc-bios/ -append "root=/dev/vda rw debug" -net nic,model=virtio -net tap -drive file=/images/ramdisk_image,if=virtio,boot=on
bamboo_init: START
Ram size passed is: 144 MB
Calling function ppc440_init
setup mmio
setup universal controller
trying to setup sdram controller
sdram_unmap_bcr: Unmap RAM area 0000000000000000 00400000
sdram_unmap_bcr: Unmap RAM area 0000000000000000 00400000
sdram_set_bcr: Map RAM area 0000000000000000 08000000
sdram_set_bcr: Map RAM area 0000000000000000 01000000
Initializing first serial port
ppc405_serial_init: offset 0000000000000300
Done calling ppc440_init
bamboo_init: load kernel
kernel is at guest address: 0x0
bamboo_init: load device tree file
device tree address is at guest address: 0x2b2100
bamboo_init: loading kvm registers
bamboo_init: DONE
Using Bamboo machine description
Linux version 2.6.25-rc3-hg1858cec8eb87-dirty (hollisb@basalt) (gcc version 3.4.2) #152 Tue Apr 1 10:52:01 CDT 2008
Found legacy serial port 0 for /plb/opb/serial@ef600300
  mem=ef600300, taddr=ef600300, irq=0, clk=11059200, speed=115200
Found legacy serial port 1 for /plb/opb/serial@ef600400
  mem=ef600400, taddr=ef600400, irq=0, clk=11059200, speed=0
console [udbg0] enabled
Entering add_active_range(0, 0, 36864) 0 entries of 256 used
setup_arch: bootmem
arch: exit
Top of RAM: 0x9000000, Total RAM: 0x9000000
Memory hole size: 0MB
Zone PFN ranges:
  DMA             0 ->    36864
  Normal      36864 ->    36864
Movable zone start PFN for each node
early_node_map[1] active PFN ranges
    0:        0 ->    36864
On node 0 totalpages: 36864
  DMA zone: 288 pages used for memmap
  DMA zone: 0 pages reserved
  DMA zone: 36576 pages, LIFO batch:7
  Normal zone: 0 pages used for memmap
  Movable zone: 0 pages used for memmap
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 36576
Kernel command line: root=/dev/vda rw debug
irq: Allocated host of type 2 @0xc03f3880
UIC0 (32 IRQ sources) at DCR 0xc0
irq: Default host set to @0xc03f3880
PID hash table entries: 1024 (order: 10, 4096 bytes)
time_init: decrementer frequency = 666.666660 MHz
time_init: processor frequency   = 666.666660 MHz
clocksource: timebase mult[600000] shift[22] registered
clockevent: decrementer mult[aaaa] shift[16] cpu[0]
Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
Memory: 143060k/147456k available (2632k kernel code, 4252k reserved, 100k data, 125k bss, 132k init)
SLUB: Genslabs=10, HWalign=32, Order=0-1, MinObjects=4, CPUs=1, Nodes=1
Calibrating delay loop... 2490.36 BogoMIPS (lpj=4980736)
Mount-cache hash table entries: 512
net_namespace: 156 bytes
NET: Registered protocol family 16

PCI host bridge /plb/pci@ec000000 (primary) ranges:
 MEM 0x00000000a0000000..0x00000000bfffffff -> 0x00000000a0000000
  IO 0x00000000e8000000..0x00000000e800ffff -> 0x0000000000000000
4xx PCI DMA offset set to 0x00000000
PCI: Probing PCI hardware
PCI: Hiding 4xx host bridge resources 0000:00:00.0
irq: irq_create_mapping(0xc03f3880, 0x1c)
irq: -> using host @c03f3880
irq: -> obtained virq 28
irq: irq_create_mapping(0xc03f3880, 0x1b)
irq: -> using host @c03f3880
irq: -> obtained virq 27
Time: timebase clocksource has been installed.
NET: Registered protocol family 2
IP route cache hash table entries: 2048 (order: 1, 8192 bytes)
TCP established hash table entries: 8192 (order: 4, 65536 bytes)
TCP bind hash table entries: 8192 (order: 3, 32768 bytes)
TCP: Hash tables configured (established 8192 bind 8192)
TCP reno registered
irq: irq_create_mapping(0xc03f3880, 0x0)
irq: -> using host @c03f3880
irq: -> obtained virq 16
irq: irq_create_mapping(0xc03f3880, 0x1)
irq: -> using host @c03f3880
irq: -> obtained virq 17
io scheduler noop registered
io scheduler anticipatory registered (default)
io scheduler deadline registered
io scheduler cfq registered
Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing enabled
serial8250.0: ttyS0 at MMIO 0xef600300 (irq = 16) is a 16450
console handover: boot [udbg0] -> real [ttyS0]
irq: irq_create_mapping(0xc03f3880, 0x0)
irq: -> using host @c03f3880
irq: -> existing mapping on virq 16
ef600300.serial: ttyS0 at MMIO 0xef600300 (irq = 16) is a 16450
irq: irq_create_mapping(0xc03f3880, 0x1)
irq: -> using host @c03f3880
irq: -> existing mapping on virq 17
brd: module loaded
Intel(R) PRO/1000 Network Driver - version 7.3.20-k2
Copyright (c) 1999-2006 Intel Corporation.
pcnet32.c:v1.34 14.Aug.2007 tsbogend@alpha.franken.de
tun: Universal TUN/TAP device driver, 1.6
tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
PCI: Enabling device 0000:00:01.0 (0000 -> 0001)
 vda: unknown partition table
PCI: Enabling device 0000:00:02.0 (0000 -> 0001)
TCP cubic registered
NET: Registered protocol family 1
NET: Registered protocol family 17
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
VFS: Mounted root (ext2 filesystem).
Freeing unused kernel memory: 132k init
root:~> ### Application running ...

root:~> ls
bin      etc      home     linuxrc  sbin     usr
dev      ftp      lib      proc     tmp      var
root:~>

[-- Attachment #3: Type: text/plain, Size: 278 bytes --]

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace

[-- Attachment #4: Type: text/plain, Size: 170 bytes --]

_______________________________________________
kvm-ppc-devel mailing list
kvm-ppc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-ppc-devel

Re: [kvm-ppc-devel] booting from virtio-blk

[Anthony Liguori]

Hollis Blanchard wrote:
> On Tue, 2008-04-01 at 09:46 -0500, Anthony Liguori wrote:
>   
> Thanks Anthony, you've saved me a lot of debug time! Rusty, doing 64-bit
> PCI config space accesses with ioread8() definitely violates the
> principle of least surprises, and would have taken me a long time to
> track down. :(
>
> Attached is a boot log of a PowerPC guest booting from virtio-blk root.
>
> "ramdisk_image" is the standard ~4MB image provided with DENX Embedded
> Linux Development Kit. Booting is also *way* faster than NFS root (a few
> seconds to get to a shell :) .
>   

That suggests you have vmexit latency issues.  A 4MB disk is pretty much 
entirely cachable in memory so you probably end up with only a handful 
of requests to get the full disk into memory.  Conversely, when using 
NFS, every single filesystem operation requests in multiple packets 
being delivered/received.  To complicate matters further, NFS means you 
won't be doing any dentry caching so every single filesystem access will 
result in requests as opposed to just the first access.

What sort of ping latency do you get with virtio-net?

Regards,

Anthony Liguori

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
kvm-ppc-devel mailing list
kvm-ppc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-ppc-devel

Re: [kvm-ppc-devel] booting from virtio-blk

[Anthony Liguori]

Hollis Blanchard wrote:
>
> Thanks Anthony, you've saved me a lot of debug time! Rusty, doing 64-bit
> PCI config space accesses with ioread8() definitely violates the
> principle of least surprises, and would have taken me a long time to
> track down. :(
>   

It's the unfortunate side-effect of using PCI config space without 
passing it's semantics through to the virtio devices.  Right now, you do 
a config_get which is basically a memcpy.  If we didn't do accesses with 
ioread8(), you could potentially have a caller than did a config_get() 
of size 4 that didn't intend on having endian conversion applied.

The other option would have been to provide config_get() and 
config_get8/16/32/64() the later performing endian conversion.

Regards,

Anthony Liguori

> Attached is a boot log of a PowerPC guest booting from virtio-blk root.
>
> "ramdisk_image" is the standard ~4MB image provided with DENX Embedded
> Linux Development Kit. Booting is also *way* faster than NFS root (a few
> seconds to get to a shell :) .
>
>   


-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
kvm-ppc-devel mailing list
kvm-ppc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-ppc-devel

Re: [kvm-ppc-devel] booting from virtio-blk

[Benjamin Herrenschmidt]

On Tue, 2008-04-01 at 12:09 -0500, Anthony Liguori wrote:

> It's the unfortunate side-effect of using PCI config space without 
> passing it's semantics through to the virtio devices.  Right now, you do 
> a config_get which is basically a memcpy.  If we didn't do accesses with 
> ioread8(), you could potentially have a caller than did a config_get() 
> of size 4 that didn't intend on having endian conversion applied.
> 
> The other option would have been to provide config_get() and 
> config_get8/16/32/64() the later performing endian conversion.

Config space should be 8/16/32. Is that ever bridged to real PCI config
space anyway ? Or only virtio ? And it should be endian swapped at the
low level, either by your HV calls or by the low level kernel. Always.
That's how PCI config space is supposed to work.

Ben.


-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
kvm-ppc-devel mailing list
kvm-ppc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-ppc-devel

Re: [kvm-ppc-devel] booting from virtio-blk

[Anthony Liguori]

Benjamin Herrenschmidt wrote:
> On Tue, 2008-04-01 at 12:09 -0500, Anthony Liguori wrote:
>
>   
>> It's the unfortunate side-effect of using PCI config space without 
>> passing it's semantics through to the virtio devices.  Right now, you do 
>> a config_get which is basically a memcpy.  If we didn't do accesses with 
>> ioread8(), you could potentially have a caller than did a config_get() 
>> of size 4 that didn't intend on having endian conversion applied.
>>
>> The other option would have been to provide config_get() and 
>> config_get8/16/32/64() the later performing endian conversion.
>>     
>
> Config space should be 8/16/32. Is that ever bridged to real PCI config
> space anyway ? Or only virtio ? And it should be endian swapped at the
> low level, either by your HV calls or by the low level kernel. Always.
> That's how PCI config space is supposed to work.
>   

I guess the point is, is that virtio config space is an abstraction with 
the implementation that is based on PCI converting all accesses to a 
series of 8-bit accesses.  The virtio config space happens to be little 
endian just like the PCI config space.

Regards,

Anthony Liguori

> Ben.
>
>   


-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
kvm-ppc-devel mailing list
kvm-ppc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-ppc-devel

Re: [kvm-ppc-devel] booting from virtio-blk

[Benjamin Herrenschmidt]

> > Config space should be 8/16/32. Is that ever bridged to real PCI config
> > space anyway ? Or only virtio ? And it should be endian swapped at the
> > low level, either by your HV calls or by the low level kernel. Always.
> > That's how PCI config space is supposed to work.
> >   
> 
> I guess the point is, is that virtio config space is an abstraction with 
> the implementation that is based on PCI converting all accesses to a 
> series of 8-bit accesses.  The virtio config space happens to be little 
> endian just like the PCI config space.

But PCI does -not- convert all accesses into a serie of 8 bit
accesses :-)

Ben.



-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
kvm-ppc-devel mailing list
kvm-ppc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-ppc-devel

Re: [kvm-ppc-devel] booting from virtio-blk

[Hollis Blanchard]

On Tue, 2008-04-01 at 16:03 -0500, Anthony Liguori wrote:
> Benjamin Herrenschmidt wrote:
> > On Tue, 2008-04-01 at 12:09 -0500, Anthony Liguori wrote:
> >
> >   
> >> It's the unfortunate side-effect of using PCI config space without 
> >> passing it's semantics through to the virtio devices.  Right now, you do 
> >> a config_get which is basically a memcpy.  If we didn't do accesses with 
> >> ioread8(), you could potentially have a caller than did a config_get() 
> >> of size 4 that didn't intend on having endian conversion applied.
> >>
> >> The other option would have been to provide config_get() and 
> >> config_get8/16/32/64() the later performing endian conversion.
> >>     
> >
> > Config space should be 8/16/32. Is that ever bridged to real PCI config
> > space anyway ? Or only virtio ? And it should be endian swapped at the
> > low level, either by your HV calls or by the low level kernel. Always.
> > That's how PCI config space is supposed to work.

Virtio accesses will not be bridged to real PCI space.

> I guess the point is, is that virtio config space is an abstraction with 
> the implementation that is based on PCI converting all accesses to a 
> series of 8-bit accesses.  The virtio config space happens to be little 
> endian just like the PCI config space.

The point is that a virtio device appears as a PCI device. Like all
other PCI devices, it has config space. Unlike all other PCI devices,
its config space is accessed with 1-byte reads.

-- 
Hollis Blanchard
IBM Linux Technology Center


-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
kvm-ppc-devel mailing list
kvm-ppc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-ppc-devel

Re: [kvm-ppc-devel] booting from virtio-blk

[Benjamin Herrenschmidt]

On Tue, 2008-04-01 at 16:18 -0500, Hollis Blanchard wrote:
> The point is that a virtio device appears as a PCI device. Like all
> other PCI devices, it has config space. Unlike all other PCI devices,
> its config space is accessed with 1-byte reads.

Which is weirdo ... it you guys make it look like PCI, then -really-
make it look like PCI :-)

Ben.



-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
kvm-ppc-devel mailing list
kvm-ppc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-ppc-devel

Re: [kvm-ppc-devel] [PATCH] kvm(ppc)-userspace: initialize

[Anthony Liguori]

ehrhardt@linux.vnet.ibm.com wrote:
> From: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
>
> virtio-block support is atm initialized in qemu/hw/pc.c which is only called
> for the "standart pc" platform.
> In our case we need to call the virtio-block initialization from our platform
> init. It might be worth  moving this initialization into a funvtion in 
> virtio-block.c and only call that initializer from any platform init that
> supports virtio - any suggestions/preferences if we should do it now or when
> another architecture needs that - Anthony ?
>   

I think it's fine the way it is.  It's only a few lines and it's the way 
that QEMU does things right now.

Regards,

Anthony Liguori

> This patch adds that initialization for the kvmppc bamboo platform.
>
> Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
> ---
>
> [diffstat]
>  ppc440_bamboo.c |   13 +++++++++++++
>  1 files changed, 13 insertions(+)
>
> diff --git a/qemu/hw/ppc440_bamboo.c b/qemu/hw/ppc440_bamboo.c
> --- a/qemu/hw/ppc440_bamboo.c
> +++ b/qemu/hw/ppc440_bamboo.c
> @@ -13,6 +13,7 @@
>  #include "net.h"
>  #include "hw.h"
>  #include "pci.h"
> +#include "sysemu.h"
>  #include "ppc440.h"
>  #include "qemu-kvm.h"
>  #include "device_tree.h"
> @@ -185,6 +186,18 @@ void bamboo_init(ram_addr_t ram_size, in
>  		pci_nic_init(pci->bus, nd, -1);
>  	}
>
> +	/* Add virtio block devices */
> +	if (pci) {
> +		int index;
> +		int unit_id = 0;
> +
> +		while ((index = drive_get_index(IF_VIRTIO, 0, unit_id)) != -1) {
> +			virtio_blk_init(pci->bus, 0x1AF4, 0x1001,
> +				drives_table[index].bdrv);
> +			unit_id++;
> +		}
> +	}
> +
>  	printf("%s: DONE\n", __func__);
>  }
>
>   


-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
kvm-ppc-devel mailing list
kvm-ppc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-ppc-devel