From mboxrd@z Thu Jan 1 00:00:00 1970 From: Hollis Blanchard Subject: Re: booting from virtio-blk Date: Tue, 01 Apr 2008 11:13:52 -0500 Message-ID: <1207066432.6214.29.camel@basalt> References: <1207051275240-git-send-email-ehrhardt@linux.vnet.ibm.com> <47F225D2.9020409@linux.vnet.ibm.com> <1207060430.6214.12.camel@basalt> <47F24ABA.3020404@us.ibm.com> Reply-To: Hollis Blanchard Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-qcArvBbQMRAH8S3XyHA8" Cc: kvm-ppc-devel@lists.sourceforge.net, Christian Ehrhardt , Rusty Russell , kvm-devel To: Anthony Liguori Return-path: In-Reply-To: <47F24ABA.3020404@us.ibm.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: kvm-devel-bounces@lists.sourceforge.net Errors-To: kvm-devel-bounces@lists.sourceforge.net List-Id: kvm.vger.kernel.org --=-qcArvBbQMRAH8S3XyHA8 Content-Type: text/plain Content-Transfer-Encoding: 7bit On Tue, 2008-04-01 at 09:46 -0500, Anthony Liguori wrote: > Hollis Blanchard wrote: > > On Tue, 2008-04-01 at 14:08 +0200, Christian Ehrhardt wrote: > >> bash-3.00# cat /proc/partitions > >> major minor #blocks name > >> [...] > >> 254 0 22517998136852480 vda <- ?broken? > >> > > > > My guess is this is run-of-the-mill endianness mismatch. > > 22517998136852480 = 0x00500000_00000000, which 64-bit byteswapped would > > be 0x5000, and that's probably a reasonable number of 512-byte blocks. > > Is your disk image 10MB? > > > > Why would we have a problem, since both guest and host are big-endian? > > Because virtio is a PCI device, and PCI MMIO are LE, so > > __virtio_config_val() in the guest is (correctly) using le64_to_cpu(). > > > > Why didn't we have problems with virtio-net? Because virtio-net doesn't > > seem to have anything interesting in PCI config space. virtio-blk's > > config space contains the capacity and a few other pieces of > > information. > > > > The fix needs to be in qemu, and given the lack of qemu endianness > > infrastructure, I'm afraid it will be a hack. See > > http://svn.savannah.nongnu.org/viewvc/trunk/hw/e1000.c?root=qemu&r1=4046&r2=4045&pathrev=4046 for reference. We all know that TARGET_WORDS_BIGENDIAN is totally wrong, but unfortunately it also seems to be the only (accidentally) working solution in qemu without major IO system rework. :( > > It's actually not so bad since the virtio config space is already read > one byte at a time. The following should help. > > diff --git a/qemu/hw/virtio-blk.c b/qemu/hw/virtio-blk.c > index 0f55d2a..492bd7f 100644 > --- a/qemu/hw/virtio-blk.c > +++ b/qemu/hw/virtio-blk.c > @@ -134,8 +134,8 @@ static void virtio_blk_update_config(VirtIODevice > *vdev, uin > int64_t capacity; > > bdrv_get_geometry(s->bs, &capacity); > - blkcfg.capacity = capacity; > - blkcfg.seg_max = 128 - 2; > + blkcfg.capacity = cpu_to_le64(capacity); > + blkcfg.seg_max = cpu_to_le32(128 - 2); > memcpy(config, &blkcfg, sizeof(blkcfg)); > } Thanks Anthony, you've saved me a lot of debug time! Rusty, doing 64-bit PCI config space accesses with ioread8() definitely violates the principle of least surprises, and would have taken me a long time to track down. :( Attached is a boot log of a PowerPC guest booting from virtio-blk root. "ramdisk_image" is the standard ~4MB image provided with DENX Embedded Linux Development Kit. Booting is also *way* faster than NFS root (a few seconds to get to a shell :) . -- Hollis Blanchard IBM Linux Technology Center --=-qcArvBbQMRAH8S3XyHA8 Content-Disposition: attachment; filename=virtio-blk.log Content-Type: text/x-log; name=virtio-blk.log; charset=UTF-8 Content-Transfer-Encoding: 7bit bash-3.00# ./qemu-system-ppcemb -M bamboo -nographic -kernel ../../uImage.bamboo -L ../pc-bios/ -append "root=/dev/vda rw debug" -net nic,model=virtio -net tap -drive file=/images/ramdisk_image,if=virtio,boot=on bamboo_init: START Ram size passed is: 144 MB Calling function ppc440_init setup mmio setup universal controller trying to setup sdram controller sdram_unmap_bcr: Unmap RAM area 0000000000000000 00400000 sdram_unmap_bcr: Unmap RAM area 0000000000000000 00400000 sdram_set_bcr: Map RAM area 0000000000000000 08000000 sdram_set_bcr: Map RAM area 0000000000000000 01000000 Initializing first serial port ppc405_serial_init: offset 0000000000000300 Done calling ppc440_init bamboo_init: load kernel kernel is at guest address: 0x0 bamboo_init: load device tree file device tree address is at guest address: 0x2b2100 bamboo_init: loading kvm registers bamboo_init: DONE Using Bamboo machine description Linux version 2.6.25-rc3-hg1858cec8eb87-dirty (hollisb@basalt) (gcc version 3.4.2) #152 Tue Apr 1 10:52:01 CDT 2008 Found legacy serial port 0 for /plb/opb/serial@ef600300 mem=ef600300, taddr=ef600300, irq=0, clk=11059200, speed=115200 Found legacy serial port 1 for /plb/opb/serial@ef600400 mem=ef600400, taddr=ef600400, irq=0, clk=11059200, speed=0 console [udbg0] enabled Entering add_active_range(0, 0, 36864) 0 entries of 256 used setup_arch: bootmem arch: exit Top of RAM: 0x9000000, Total RAM: 0x9000000 Memory hole size: 0MB Zone PFN ranges: DMA 0 -> 36864 Normal 36864 -> 36864 Movable zone start PFN for each node early_node_map[1] active PFN ranges 0: 0 -> 36864 On node 0 totalpages: 36864 DMA zone: 288 pages used for memmap DMA zone: 0 pages reserved DMA zone: 36576 pages, LIFO batch:7 Normal zone: 0 pages used for memmap Movable zone: 0 pages used for memmap Built 1 zonelists in Zone order, mobility grouping on. Total pages: 36576 Kernel command line: root=/dev/vda rw debug irq: Allocated host of type 2 @0xc03f3880 UIC0 (32 IRQ sources) at DCR 0xc0 irq: Default host set to @0xc03f3880 PID hash table entries: 1024 (order: 10, 4096 bytes) time_init: decrementer frequency = 666.666660 MHz time_init: processor frequency = 666.666660 MHz clocksource: timebase mult[600000] shift[22] registered clockevent: decrementer mult[aaaa] shift[16] cpu[0] Dentry cache hash table entries: 32768 (order: 5, 131072 bytes) Inode-cache hash table entries: 16384 (order: 4, 65536 bytes) Memory: 143060k/147456k available (2632k kernel code, 4252k reserved, 100k data, 125k bss, 132k init) SLUB: Genslabs=10, HWalign=32, Order=0-1, MinObjects=4, CPUs=1, Nodes=1 Calibrating delay loop... 2490.36 BogoMIPS (lpj=4980736) Mount-cache hash table entries: 512 net_namespace: 156 bytes NET: Registered protocol family 16 PCI host bridge /plb/pci@ec000000 (primary) ranges: MEM 0x00000000a0000000..0x00000000bfffffff -> 0x00000000a0000000 IO 0x00000000e8000000..0x00000000e800ffff -> 0x0000000000000000 4xx PCI DMA offset set to 0x00000000 PCI: Probing PCI hardware PCI: Hiding 4xx host bridge resources 0000:00:00.0 irq: irq_create_mapping(0xc03f3880, 0x1c) irq: -> using host @c03f3880 irq: -> obtained virq 28 irq: irq_create_mapping(0xc03f3880, 0x1b) irq: -> using host @c03f3880 irq: -> obtained virq 27 Time: timebase clocksource has been installed. NET: Registered protocol family 2 IP route cache hash table entries: 2048 (order: 1, 8192 bytes) TCP established hash table entries: 8192 (order: 4, 65536 bytes) TCP bind hash table entries: 8192 (order: 3, 32768 bytes) TCP: Hash tables configured (established 8192 bind 8192) TCP reno registered irq: irq_create_mapping(0xc03f3880, 0x0) irq: -> using host @c03f3880 irq: -> obtained virq 16 irq: irq_create_mapping(0xc03f3880, 0x1) irq: -> using host @c03f3880 irq: -> obtained virq 17 io scheduler noop registered io scheduler anticipatory registered (default) io scheduler deadline registered io scheduler cfq registered Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing enabled serial8250.0: ttyS0 at MMIO 0xef600300 (irq = 16) is a 16450 console handover: boot [udbg0] -> real [ttyS0] irq: irq_create_mapping(0xc03f3880, 0x0) irq: -> using host @c03f3880 irq: -> existing mapping on virq 16 ef600300.serial: ttyS0 at MMIO 0xef600300 (irq = 16) is a 16450 irq: irq_create_mapping(0xc03f3880, 0x1) irq: -> using host @c03f3880 irq: -> existing mapping on virq 17 brd: module loaded Intel(R) PRO/1000 Network Driver - version 7.3.20-k2 Copyright (c) 1999-2006 Intel Corporation. pcnet32.c:v1.34 14.Aug.2007 tsbogend@alpha.franken.de tun: Universal TUN/TAP device driver, 1.6 tun: (C) 1999-2004 Max Krasnyansky PCI: Enabling device 0000:00:01.0 (0000 -> 0001) vda: unknown partition table PCI: Enabling device 0000:00:02.0 (0000 -> 0001) TCP cubic registered NET: Registered protocol family 1 NET: Registered protocol family 17 RPC: Registered udp transport module. RPC: Registered tcp transport module. VFS: Mounted root (ext2 filesystem). Freeing unused kernel memory: 132k init root:~> ### Application running ... root:~> ls bin etc home linuxrc sbin usr dev ftp lib proc tmp var root:~> --=-qcArvBbQMRAH8S3XyHA8 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace --=-qcArvBbQMRAH8S3XyHA8 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel --=-qcArvBbQMRAH8S3XyHA8--