FW: Notification from Qemu to Guest

FW: Notification from Qemu to Guest

[Kumar, Venkat]

Hi Avi - Probably you can answer this question?

Thx,

Venkat


-----Original Message-----
From: kvm-owner@vger.kernel.org [mailto:kvm-owner@vger.kernel.org] On Behalf Of Kumar, Venkat
Sent: Tuesday, April 28, 2009 4:16 PM
To: kvm@vger.kernel.org
Subject: Notification from Qemu to Guest

I have emulated a PCI device on Qemu and hooked my sample/simple driver to that virtio device on the guest.

I am testing the notification from Guest-Qemu and vice-versa.

I am able to notify from Guest to Qemu but Qemu-Guest notification is not happening.

As a part of "Kick" routine in my guest driver I could see the notification happening from Guest-Qemu and In the Qemu process as a part of handle output for the emulated device I am simply doing  "virtio_notify(vdev, vq)" but I don't see my callback getting called which is already registered as a part of "find_vq" in guest driver's probe.

BTW, the emulated device is allocated with "GSI 11" where as for other emulated devices like "virtio-blk" is associated with GSI 10 which I found in "dmesg's". Is this a reason why interrupt is not delivered from Qemu-Guest?

Any clues?

Thx,

Venkat

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: FW: Notification from Qemu to Guest

[Avi Kivity]

Kumar, Venkat wrote:
> Hi Avi - Probably you can answer this question?
>
>   

I'm no virtio expert; copying someone who is.

>
> I have emulated a PCI device on Qemu and hooked my sample/simple driver to that virtio device on the guest.
>
> I am testing the notification from Guest-Qemu and vice-versa.
>
> I am able to notify from Guest to Qemu but Qemu-Guest notification is not happening.
>
> As a part of "Kick" routine in my guest driver I could see the notification happening from Guest-Qemu and In the Qemu process as a part of handle output for the emulated device I am simply doing  "virtio_notify(vdev, vq)" but I don't see my callback getting called which is already registered as a part of "find_vq" in guest driver's probe.
>   

You need to enable notifications, not sure how exactly.


> BTW, the emulated device is allocated with "GSI 11" where as for other emulated devices like "virtio-blk" is associated with GSI 10 which I found in "dmesg's". Is this a reason why interrupt is not delivered from Qemu-Guest?
>   

Interrupts for PCI devices are assigned based on the slots where they 
sit. Both GSI 10 and GSI 11 are PCI link interrupts.


-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: FW: Notification from Qemu to Guest

[Anthony Liguori]

Avi Kivity wrote:
>> I have emulated a PCI device on Qemu and hooked my sample/simple 
>> driver to that virtio device on the guest.

This is independent of the existing virtio PCI device?

>> As a part of "Kick" routine in my guest driver I could see the 
>> notification happening from Guest-Qemu and In the Qemu process as a 
>> part of handle output for the emulated device I am simply doing  
>> "virtio_notify(vdev, vq)" but I don't see my callback getting called 
>> which is already registered as a part of "find_vq" in guest driver's 
>> probe.
>>   
>
> You need to enable notifications, not sure how exactly.

By default, if you zeroed the memory for the ring, notifications are 
enabled.  You have to set a bit to disable notifications.  It sounds 
like you aren't properly injecting the IRQ which is hard to assess 
without more detail about what the particular device you've added to QEMU.

Are you reusing the existing virtio PCI infrastructure in QEMU?

>> BTW, the emulated device is allocated with "GSI 11" where as for 
>> other emulated devices like "virtio-blk" is associated with GSI 10 
>> which I found in "dmesg's". Is this a reason why interrupt is not 
>> delivered from Qemu-Guest?
>>   
>
> Interrupts for PCI devices are assigned based on the slots where they 
> sit. Both GSI 10 and GSI 11 are PCI link interrupts.

virtio-pci always uses LNK A.  How it gets mapped to GSI depends on the 
slot as Avi mentioned.

Regards,

Anthony Liguori

RE: FW: Notification from Qemu to Guest

[Kumar, Venkat]

Hi Anthony - My questions and Comments are in line.

Thx,

Venkat

-----Original Message-----
From: Anthony Liguori [mailto:anthony@codemonkey.ws] 
Sent: Tuesday, April 28, 2009 8:41 PM
To: Avi Kivity
Cc: Kumar, Venkat; kvm@vger.kernel.org
Subject: Re: FW: Notification from Qemu to Guest

Avi Kivity wrote:
>> I have emulated a PCI device on Qemu and hooked my sample/simple 
>> driver to that virtio device on the guest.

This is independent of the existing virtio PCI device?

==> No it is the same as the existing virtio PCI device. I am reusing the virtio-blk model. In qemu/hw/pc.c, I am spawning an emulated PCI device like this "virtio_sample_init(pci_bus,drives_table[25].bdrv);". I haven't explored the "blockdrivestate" parameter (second parameter) and hardcoded it to "25" because I don't want to associate the device with an image file on the disk rather I just want use the device for communication. I am manually filling the 25th indexed blockdriverstate in this way.
                BlockDriverState *bdrv_temp;
                int idx_tmp;

                bdrv_temp =  bdrv_new("SAMPLE Disk Device");
                drives_table[25].bdrv = bdrv_temp;
                drives_table[25].type = type;
                drives_table[25].bus = bus_id;
                drives_table[25].unit = unit_id;
                drives_table[25].onerror = onerror;
                nb_drives++;
in qemu/vl.c

>> As a part of "Kick" routine in my guest driver I could see the 
>> notification happening from Guest-Qemu and In the Qemu process as a 
>> part of handle output for the emulated device I am simply doing  
>> "virtio_notify(vdev, vq)" but I don't see my callback getting called 
>> which is already registered as a part of "find_vq" in guest driver's 
>> probe.
>>   
>
> You need to enable notifications, not sure how exactly.

By default, if you zeroed the memory for the ring, notifications are 
enabled.  You have to set a bit to disable notifications.  It sounds 
like you aren't properly injecting the IRQ which is hard to assess 
without more detail about what the particular device you've added to QEMU.

Are you reusing the existing virtio PCI infrastructure in QEMU?

==> I am using the existing vp_find_vq to allocate virtio queues and rings so I assume this function zeroes the ring memory.
As a part of probe in Guest virtio sample driver, I am calling find_vq and kick immediately to test notifications.
============================================================================
static int virtsample_probe(struct virtio_device *vdev)
{

        struct virtio_sample *vsample;
        int err;

        printk("Virtio SAMPLE probe is called \n");

        if (index_to_minor(index) >= 1 << MINORBITS)
                return -ENOSPC;

        vdev->priv = vsample = kmalloc(sizeof(*vsample), GFP_KERNEL);

        if (!vsample) {
                err = -ENOMEM;
                goto out;
        }

                INIT_LIST_HEAD(&vsample->reqs);
        spin_lock_init(&vsample->lock);
        vsample->vdev = vdev;

        /* We expect one virtqueue, for output. */
        vsample->vq = vdev->config->find_vq(vdev, 0, sample_done);
        if (IS_ERR(vsample->vq)) {
                err = PTR_ERR(vsample->vq);
                goto out_free_vsample;
        }


        vsample->vq->vq_ops->kick(vsample->vq);
        return 0;

out_free_vsample:
        kfree(vsample);
        printk("Failed in output_free_sample\n");
out:
        return err;
}
============================================================================
And as a part of handle output for kick in the qemu side I am simply calling "virtio_notify"
static void virtio_sample_handle_output(VirtIODevice *vdev, VirtQueue *vq)
{
            printf("Function = %s, Line = %d\n",__FUNCTION__,__LINE__);
            virtio_notify(vdev, vq);
}

Kick is working fine as I am landing in Qemu when making that call, However virtio_notify is not resulting in my callback invocation registered as part find_vq.
Do you see any missing parts here?

>> BTW, the emulated device is allocated with "GSI 11" where as for 
>> other emulated devices like "virtio-blk" is associated with GSI 10 
>> which I found in "dmesg's". Is this a reason why interrupt is not 
>> delivered from Qemu-Guest?
>>   
>
> Interrupts for PCI devices are assigned based on the slots where they 
> sit. Both GSI 10 and GSI 11 are PCI link interrupts.

virtio-pci always uses LNK A.  How it gets mapped to GSI depends on the 
slot as Avi mentioned.

Regards,

Anthony Liguori

Re: FW: Notification from Qemu to Guest

[Pantelis Koukousoulas]

> ============================================================================
> And as a part of handle output for kick in the qemu side I am simply calling "virtio_notify"
> static void virtio_sample_handle_output(VirtIODevice *vdev, VirtQueue *vq)
> {
>            printf("Function = %s, Line = %d\n",__FUNCTION__,__LINE__);
>            virtio_notify(vdev, vq);
> }
>
> Kick is working fine as I am landing in Qemu when making that call, However virtio_notify is not resulting in my callback invocation registered as part find_vq.
> Do you see any missing parts here?
>

I was playing a bit today with my own virtio driver (I have locally
applied my PCI ID patch) and I bumped into your problem
for a few minutes. I think what you are missing is that your queue is
empty and since you probably haven't negotiated the
VIRTIO_F_NOTIFY_ON_EMPTY feature, you are falling victim of this code in qemu:

hw/virtio.c:669

void virtio_notify(VirtIODevice *vdev, VirtQueue *vq)
{
    /* Always notify when queue is empty (when feature acknowledge) */
    if ((vring_avail_flags(vq) & VRING_AVAIL_F_NO_INTERRUPT) &&
        (!(vdev->features & (1 << VIRTIO_F_NOTIFY_ON_EMPTY)) ||
         (vq->inuse || vring_avail_idx(vq) != vq->last_avail_idx)))
        return;
        ^^^^^^^^
        This is why you are not getting your interrupt


    vdev->isr |= 0x01;
    virtio_update_irq(vdev);
}


HTH,
Pantelis