Re: virtio-blk-pci chokes on some PCI slots

[Markus Armbruster <armbru@redhat.com>]

Marcelo Tosatti <mtosatti@redhat.com> writes:

> On Tue, Jun 16, 2009 at 06:47:45PM +0200, Markus Armbruster wrote:
>> I'm working on make PCI slots configurable in QEMU.  While testing the
>> feature for virtio-blk-pci, I ran into a task hang with Fedora kernel
>> 2.6.27.5-117.fc10.  Marcelo tested it with a newer kernel, and will
>> follow up with details.
>> 
>> Instead of the full QEMU patch I'm working on, the appended little hack
>> suffices to reproduce the problem.
>> 
>>     $ QEMU_VIRTIO_BLK=9 qemu foo.qcow2 -drive file=bar.img,if=virtio -serial stdio
>> [...]
>>     Loading virtio_pci module
>>     ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 11
>>     virtio-pci 0000:00:04.0: PCI INT A -> Link[LNKD] -> GSI 11 (level, low) -> IRQ 11
>>     virtio-pci 0000:00:09.0: can't derive routing for PCI INT A
>>     virtio-pci 0000:00:09.0: PCI INT A: no GSI - using IRQ 11
>>     Loading virtio_blk module
>>      vda:
>> [tick-tock-tick-tock...]
>>     <3>INFO: task modprobe:491 blocked for more than 120 seconds.
>>     "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
>>     modprobe      D c04303f1     0   491      1
>>            c6baeae8 00000082 c6baea80 c04303f1 c6bae000 c087667c c0879c00 c0879c00 
>>            c0879c00 c6b659b0 c6b65c24 c1108c00 00000000 c1108c00 83a43d86 00000002 
>>            c781aaa8 c6baead0 c04418d0 c6b65c24 00004d88 c781aaa8 00004d88 c6baeb3c 
>>     Call Trace:
>>      [<c04303f1>] ? irq_exit+0x5f/0x83
>>      [<c04418d0>] ? getnstimeofday+0x3c/0xc9
>>      [<c043fcae>] ? ktime_get_ts+0x4a/0x4e
>>      [<c06a6110>] io_schedule+0x52/0x8a
>>      [<c046da7d>] sync_page+0x46/0x4c
>>      [<c06a64c9>] __wait_on_bit_lock+0x34/0x5e
>>      [<c046da37>] ? sync_page+0x0/0x4c
>>      [<c046d9f3>] __lock_page+0x78/0x81
>>      [<c043cd89>] ? wake_bit_function+0x0/0x43
>>      [<c046da28>] lock_page+0x2c/0x2f
>>      [<c046dff4>] read_cache_page_async+0xa4/0xfb
>>      [<c04b1cec>] ? blkdev_readpage+0x0/0x11
>>      [<c046e057>] read_cache_page+0xc/0x3c
>>      [<c04c8ec1>] read_dev_sector+0x34/0x6a
>>      [<c04cab5a>] ? efi_partition+0x0/0x676
>>      [<c04ca945>] read_lba+0x69/0xc6
>>      [<c04cabde>] ? efi_partition+0x84/0x676
>>      [<c04cab5a>] ? efi_partition+0x0/0x676
>>      [<c04cabfd>] efi_partition+0xa3/0x676
>>      [<c042c71a>] ? vprintk+0x2c8/0x2f3
>>      [<c04a13ab>] ? iget5_locked+0x33/0x114
>>      [<c051cf05>] ? snprintf+0x15/0x17
>>      [<c04cab5a>] ? efi_partition+0x0/0x676
>>      [<c04c94db>] rescan_partitions+0x106/0x242
>>      [<c041d8ba>] ? need_resched+0x18/0x22
>>      [<c06a6192>] ? _cond_resched+0x8/0x32
>>      [<c04b1a4a>] do_open+0x1ff/0x290
>>      [<c04b1b9d>] __blkdev_get+0x71/0x7c
>>      [<c04b1bb5>] blkdev_get+0xd/0xf
>>      [<c04c8fe7>] register_disk+0xf0/0x141
>>      [<c05111cf>] add_disk+0x34/0x89
>>      [<c05101fc>] ? exact_match+0x0/0xb
>>      [<c0510e13>] ? exact_lock+0x0/0x11
>>      [<c88223b2>] virtblk_probe+0x275/0x2a8 [virtio_blk]
>>      [<c062236b>] virtio_dev_probe+0x89/0xb3
>>      [<c05937d0>] driver_probe_device+0xa0/0x136
>>      [<c05938a0>] __driver_attach+0x3a/0x59
>>      [<c05931f6>] bus_for_each_dev+0x3b/0x63
>>      [<c0593675>] driver_attach+0x14/0x16
>>      [<c0593866>] ? __driver_attach+0x0/0x59
>>      [<c0592c6a>] bus_add_driver+0x9d/0x1ba
>>      [<c0593a27>] driver_register+0x81/0xe1
>>      [<c05112ec>] ? register_blkdev+0xc8/0xdc
>>      [<c062245a>] register_virtio_driver+0x1f/0x21
>>      [<c8826022>] init+0x22/0x24 [virtio_blk]
>>      [<c0401125>] _stext+0x3d/0x115
>>      [<c8826000>] ? init+0x0/0x24 [virtio_blk]
>>      [<c044bf7d>] sys_init_module+0x87/0x178
>>      [<c0403c76>] syscall_call+0x7/0xb
>>      =======================
>> 
>> Some slots work, e.g. 5, some don't, e.g. 6.
>> 
>> What's going on here?
>> 
>> 
>> diff --git a/hw/pc.c b/hw/pc.c
>> index 143b697..15c44ac 100644
>> --- a/hw/pc.c
>> +++ b/hw/pc.c
>> @@ -1146,7 +1146,8 @@ static void pc_init1(ram_addr_t ram_size,
>>          int unit_id = 0;
>>  
>>          while ((index = drive_get_index(IF_VIRTIO, 0, unit_id)) != -1) {
>> -            pci_create_simple(pci_bus, -1, "virtio-blk-pci");
>> +            char *devfn = getenv("QEMU_VIRTIO_BLK");
>> +            pci_create_simple(pci_bus, devfn ? atoi(devfn) * 8: -1, "virtio-blk-pci");
>>              unit_id++;
>>          }
>>      }
>
> no softlockup with 2.6.29-06638-g17db0a0, but not very friendly either
> (note the backing device has no partitions in my case, perhaps that
> makes a difference).
>
> virtio-pci 0000:00:04.0: PCI INT A -> Link[LNKD] -> GSI 10 (level, low)
> -> IRQ 10
> virtio-pci 0000:00:09.0: can't derive routing for PCI INT A
> virtio-pci 0000:00:09.0: PCI INT A: no GSI - using IRQ 11
>
> Starting udev:  Wait timeout. Will continue in the background.[FAILED]
>
> [root@guest ~]# dmesg | grep vda
>  vda:<6>EXT3 FS on dm-0, internal journal
>
> [root@guest ~]# mount /dev/vda /mnt
>
> Hangs there forever.

Even better: I can reproduce it with an unpatched upstream QEMU
(qemu.git commit cfde4bd9) like this:

    $ qemu foo.qcow2 -net nic -net nic -net nic -drive file=bar.img,if=virtio

virtio-blk-pci ends up in slot 6.