From mboxrd@z Thu Jan  1 00:00:00 1970
Received: from eggs.gnu.org ([208.118.235.92]:49056)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mst@redhat.com>) id 1TitfG-0001Hc-4H
	for qemu-devel@nongnu.org; Wed, 12 Dec 2012 16:16:31 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <mst@redhat.com>) id 1Titf8-0003JM-LT
	for qemu-devel@nongnu.org; Wed, 12 Dec 2012 16:16:26 -0500
Received: from mx1.redhat.com ([209.132.183.28]:47410)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <mst@redhat.com>) id 1Titf8-0003JG-E2
	for qemu-devel@nongnu.org; Wed, 12 Dec 2012 16:16:18 -0500
Date: Wed, 12 Dec 2012 23:19:26 +0200
From: "Michael S. Tsirkin" <mst@redhat.com>
Message-ID: <20121212211926.GA23087@redhat.com>
References: <20121212144758.GF15555@redhat.com> <50C89C47.7040108@redhat.com>
	<20121212152508.GB16750@redhat.com> <50C8A855.4050607@redhat.com>
	<20121212163713.GD17446@redhat.com> <50C8B627.606@redhat.com>
	<20121212171423.GB18597@redhat.com> <50C8C143.2010503@redhat.com>
	<20121212192336.GA20201@redhat.com> <50C8F075.4000206@redhat.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <50C8F075.4000206@redhat.com>
Subject: Re: [Qemu-devel] [PATCHv2] virtio: verify that all outstanding
	buffers are flushed
List-Id: <qemu-devel.nongnu.org>
List-Unsubscribe: <https://lists.nongnu.org/mailman/options/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <https://lists.nongnu.org/mailman/listinfo/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=subscribe>
To: Paolo Bonzini <pbonzini@redhat.com>
Cc: Stefan Hajnoczi <stefanha@gmail.com>, Anthony Liguori <aliguori@us.ibm.com>, Rusty Russell <rusty@rustcorp.com.au>, qemu-devel@nongnu.org, Stefan Hajnoczi <stefanha@redhat.com>

On Wed, Dec 12, 2012 at 10:00:37PM +0100, Paolo Bonzini wrote:
> Il 12/12/2012 20:23, Michael S. Tsirkin ha scritto:
> >>> Saving inuse counter is useless. We need to know which requests
> >>> are outstanding if we want to retry them on remote.
> >>
> >> And that's what virtio-blk and virtio-scsi have been doing for years.
> > 
> > I don't see it - all I see in save is virtio_save.
> 
> static void virtio_blk_save(QEMUFile *f, void *opaque)
> {
>     VirtIOBlock *s = opaque;
>     VirtIOBlockReq *req = s->rq;
> 
>     virtio_save(&s->vdev, f);
>     
>     while (req) {
>         qemu_put_sbyte(f, 1);
>         qemu_put_buffer(f, (unsigned char*)&req->elem, sizeof(req->elem));

Ow. Does it really save VirtQueueElement?

typedef struct VirtQueueElement
{        
    unsigned int index;
    unsigned int out_num;
    unsigned int in_num;
    hwaddr in_addr[VIRTQUEUE_MAX_SIZE];
    hwaddr out_addr[VIRTQUEUE_MAX_SIZE];
    struct iovec in_sg[VIRTQUEUE_MAX_SIZE];
    struct iovec out_sg[VIRTQUEUE_MAX_SIZE];
} VirtQueueElement; 

Complete with pointers into qemu memory and all?
That's got to hurt.

All we really need is the index.

>         req = req->next;
>     }
>     qemu_put_sbyte(f, 0);
> }
> 
> 
> virtio-scsi does it in virtio_scsi_save_request.
> 
> > You need to retry A1 on remote. How do you do that? There's
> > no way to find out it has not been completed
> > from the ring itself.
> 
> virtio_blk_dma_restart_bh and scsi_dma_restart_bh do it.
> 
> Paolo

Okay, so the only bug is inuse getting negative right?
So all we need to do is fix up the inuse value
after restoring the outstanding requests - basically
count the restored buffers and set inuse accordingly.

-- 
MST