Re: virtio-blk-pci chokes on some PCI slots

[Marcelo Tosatti <mtosatti@redhat.com>]

On Tue, Jun 16, 2009 at 06:47:45PM +0200, Markus Armbruster wrote:
> I'm working on make PCI slots configurable in QEMU.  While testing the
> feature for virtio-blk-pci, I ran into a task hang with Fedora kernel
> 2.6.27.5-117.fc10.  Marcelo tested it with a newer kernel, and will
> follow up with details.
> 
> Instead of the full QEMU patch I'm working on, the appended little hack
> suffices to reproduce the problem.
> 
>     $ QEMU_VIRTIO_BLK=9 qemu foo.qcow2 -drive file=bar.img,if=virtio -serial stdio
> [...]
>     Loading virtio_pci module
>     ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 11
>     virtio-pci 0000:00:04.0: PCI INT A -> Link[LNKD] -> GSI 11 (level, low) -> IRQ 11
>     virtio-pci 0000:00:09.0: can't derive routing for PCI INT A
>     virtio-pci 0000:00:09.0: PCI INT A: no GSI - using IRQ 11
>     Loading virtio_blk module
>      vda:
> [tick-tock-tick-tock...]
>     <3>INFO: task modprobe:491 blocked for more than 120 seconds.
>     "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
>     modprobe      D c04303f1     0   491      1
>            c6baeae8 00000082 c6baea80 c04303f1 c6bae000 c087667c c0879c00 c0879c00 
>            c0879c00 c6b659b0 c6b65c24 c1108c00 00000000 c1108c00 83a43d86 00000002 
>            c781aaa8 c6baead0 c04418d0 c6b65c24 00004d88 c781aaa8 00004d88 c6baeb3c 
>     Call Trace:
>      [<c04303f1>] ? irq_exit+0x5f/0x83
>      [<c04418d0>] ? getnstimeofday+0x3c/0xc9
>      [<c043fcae>] ? ktime_get_ts+0x4a/0x4e
>      [<c06a6110>] io_schedule+0x52/0x8a
>      [<c046da7d>] sync_page+0x46/0x4c
>      [<c06a64c9>] __wait_on_bit_lock+0x34/0x5e
>      [<c046da37>] ? sync_page+0x0/0x4c
>      [<c046d9f3>] __lock_page+0x78/0x81
>      [<c043cd89>] ? wake_bit_function+0x0/0x43
>      [<c046da28>] lock_page+0x2c/0x2f
>      [<c046dff4>] read_cache_page_async+0xa4/0xfb
>      [<c04b1cec>] ? blkdev_readpage+0x0/0x11
>      [<c046e057>] read_cache_page+0xc/0x3c
>      [<c04c8ec1>] read_dev_sector+0x34/0x6a
>      [<c04cab5a>] ? efi_partition+0x0/0x676
>      [<c04ca945>] read_lba+0x69/0xc6
>      [<c04cabde>] ? efi_partition+0x84/0x676
>      [<c04cab5a>] ? efi_partition+0x0/0x676
>      [<c04cabfd>] efi_partition+0xa3/0x676
>      [<c042c71a>] ? vprintk+0x2c8/0x2f3
>      [<c04a13ab>] ? iget5_locked+0x33/0x114
>      [<c051cf05>] ? snprintf+0x15/0x17
>      [<c04cab5a>] ? efi_partition+0x0/0x676
>      [<c04c94db>] rescan_partitions+0x106/0x242
>      [<c041d8ba>] ? need_resched+0x18/0x22
>      [<c06a6192>] ? _cond_resched+0x8/0x32
>      [<c04b1a4a>] do_open+0x1ff/0x290
>      [<c04b1b9d>] __blkdev_get+0x71/0x7c
>      [<c04b1bb5>] blkdev_get+0xd/0xf
>      [<c04c8fe7>] register_disk+0xf0/0x141
>      [<c05111cf>] add_disk+0x34/0x89
>      [<c05101fc>] ? exact_match+0x0/0xb
>      [<c0510e13>] ? exact_lock+0x0/0x11
>      [<c88223b2>] virtblk_probe+0x275/0x2a8 [virtio_blk]
>      [<c062236b>] virtio_dev_probe+0x89/0xb3
>      [<c05937d0>] driver_probe_device+0xa0/0x136
>      [<c05938a0>] __driver_attach+0x3a/0x59
>      [<c05931f6>] bus_for_each_dev+0x3b/0x63
>      [<c0593675>] driver_attach+0x14/0x16
>      [<c0593866>] ? __driver_attach+0x0/0x59
>      [<c0592c6a>] bus_add_driver+0x9d/0x1ba
>      [<c0593a27>] driver_register+0x81/0xe1
>      [<c05112ec>] ? register_blkdev+0xc8/0xdc
>      [<c062245a>] register_virtio_driver+0x1f/0x21
>      [<c8826022>] init+0x22/0x24 [virtio_blk]
>      [<c0401125>] _stext+0x3d/0x115
>      [<c8826000>] ? init+0x0/0x24 [virtio_blk]
>      [<c044bf7d>] sys_init_module+0x87/0x178
>      [<c0403c76>] syscall_call+0x7/0xb
>      =======================
> 
> Some slots work, e.g. 5, some don't, e.g. 6.
> 
> What's going on here?
> 
> 
> diff --git a/hw/pc.c b/hw/pc.c
> index 143b697..15c44ac 100644
> --- a/hw/pc.c
> +++ b/hw/pc.c
> @@ -1146,7 +1146,8 @@ static void pc_init1(ram_addr_t ram_size,
>          int unit_id = 0;
>  
>          while ((index = drive_get_index(IF_VIRTIO, 0, unit_id)) != -1) {
> -            pci_create_simple(pci_bus, -1, "virtio-blk-pci");
> +            char *devfn = getenv("QEMU_VIRTIO_BLK");
> +            pci_create_simple(pci_bus, devfn ? atoi(devfn) * 8: -1, "virtio-blk-pci");
>              unit_id++;
>          }
>      }

no softlockup with 2.6.29-06638-g17db0a0, but not very friendly either
(note the backing device has no partitions in my case, perhaps that
makes a difference).

virtio-pci 0000:00:04.0: PCI INT A -> Link[LNKD] -> GSI 10 (level, low)
-> IRQ 10
virtio-pci 0000:00:09.0: can't derive routing for PCI INT A
virtio-pci 0000:00:09.0: PCI INT A: no GSI - using IRQ 11

Starting udev:  Wait timeout. Will continue in the background.[FAILED]

[root@guest ~]# dmesg | grep vda
 vda:<6>EXT3 FS on dm-0, internal journal

[root@guest ~]# mount /dev/vda /mnt

Hangs there forever.