virtio-blk.c handling of i/o which is not a 512 multiple

virtio-blk.c handling of i/o which is not a 512 multiple

[Conor Murphy]

Hi,

I'm trying to write a virtio-blk driver for Solaris. I've gotten it to the point
where Solaris can see the device and create a ZFS file system on it.

However when I try and create a UFS filesystem on the device, the VM crashed
with the error
*** glibc detected *** /usr/bin/qemu-kvm: double free or corruption (!prev):
0x00007f2d38000a00 ***

I can reproduce the problem with a simple dd, i.e.
dd if=/dev/zero of=/dev/rdsk/c2d10p0 bs=5000 count=1

My driver will create a virtio-blk request with two elements in the sg list, one
for the first 4096 byes and the other for the remaining 904.

>From stepping through with gdb, virtio_blk_handle_write will sets n_sectors to 9
(5000 / 512). Later on the code, n_sectors is used the calculate the size of the
buffer required but 9 * 512 is too small and so when the request is process it
ends up writing past the end of the buffer and I guest this triggers the glibc
error.

Is there a requirement for virtio-blk guest drivers that all i/o requests are
sized in multiples of 512 bytes?

Re: virtio-blk.c handling of i/o which is not a 512 multiple

[Alexander Graf]

On 30.03.2011, at 10:15, Conor Murphy wrote:

> Hi,
> 
> I'm trying to write a virtio-blk driver for Solaris. I've gotten it to the point
> where Solaris can see the device and create a ZFS file system on it.
> 
> However when I try and create a UFS filesystem on the device, the VM crashed
> with the error
> *** glibc detected *** /usr/bin/qemu-kvm: double free or corruption (!prev):
> 0x00007f2d38000a00 ***

Ouch.

> 
> I can reproduce the problem with a simple dd, i.e.
> dd if=/dev/zero of=/dev/rdsk/c2d10p0 bs=5000 count=1
> 
> My driver will create a virtio-blk request with two elements in the sg list, one
> for the first 4096 byes and the other for the remaining 904.
> 
> From stepping through with gdb, virtio_blk_handle_write will sets n_sectors to 9
> (5000 / 512). Later on the code, n_sectors is used the calculate the size of the
> buffer required but 9 * 512 is too small and so when the request is process it
> ends up writing past the end of the buffer and I guest this triggers the glibc
> error.
> 
> Is there a requirement for virtio-blk guest drivers that all i/o requests are
> sized in multiples of 512 bytes?

There should be some documentation on virtio-blk, but I can't seem to find it atm. Either way, you're posting this on the wrong mailing list. The correct ones are:

  Qemu: QEMU-devel Developers <qemu-devel@nongnu.org>
  Virtio: virtualization@lists.linux-foundation.org


Please repost it there and hope for someone to reply. I'd personally find it very odd to see that any HBA would accept data that's not aligned to sectors, but let's get some comments from the inventors of the protocol :)


Alex

Re: virtio-blk.c handling of i/o which is not a 512 multiple

[Avi Kivity]

On 03/30/2011 10:41 AM, Alexander Graf wrote:
> On 30.03.2011, at 10:15, Conor Murphy wrote:
>
> >  Hi,
> >
> >  I'm trying to write a virtio-blk driver for Solaris. I've gotten it to the point
> >  where Solaris can see the device and create a ZFS file system on it.
> >
> >  However when I try and create a UFS filesystem on the device, the VM crashed
> >  with the error
> >  *** glibc detected *** /usr/bin/qemu-kvm: double free or corruption (!prev):
> >  0x00007f2d38000a00 ***
>
> Ouch.
>
> >
> >  I can reproduce the problem with a simple dd, i.e.
> >  dd if=/dev/zero of=/dev/rdsk/c2d10p0 bs=5000 count=1
> >
> >  My driver will create a virtio-blk request with two elements in the sg list, one
> >  for the first 4096 byes and the other for the remaining 904.
> >
> >   From stepping through with gdb, virtio_blk_handle_write will sets n_sectors to 9
> >  (5000 / 512). Later on the code, n_sectors is used the calculate the size of the
> >  buffer required but 9 * 512 is too small and so when the request is process it
> >  ends up writing past the end of the buffer and I guest this triggers the glibc
> >  error.
> >
> >  Is there a requirement for virtio-blk guest drivers that all i/o requests are
> >  sized in multiples of 512 bytes?
>
> There should be some documentation on virtio-blk, but I can't seem to find it atm. Either way, you're posting this on the wrong mailing list. The correct ones are:
>
>    Qemu: QEMU-devel Developers<qemu-devel@nongnu.org>
>    Virtio: virtualization@lists.linux-foundation.org
>
>
> Please repost it there and hope for someone to reply. I'd personally find it very odd to see that any HBA would accept data that's not aligned to sectors, but let's get some comments from the inventors of the protocol :)
>

The data should be aligned; but of course a misalignment shouldn't kill 
qemu, only fail the request.

-- 
error compiling committee.c: too many arguments to function

Re: virtio-blk.c handling of i/o which is not a 512 multiple

[Stefan Hajnoczi]

On Wed, Mar 30, 2011 at 9:15 AM, Conor Murphy
<conor_murphy_virt@hotmail.com> wrote:
> I'm trying to write a virtio-blk driver for Solaris. I've gotten it to the point
> where Solaris can see the device and create a ZFS file system on it.
>
> However when I try and create a UFS filesystem on the device, the VM crashed
> with the error
> *** glibc detected *** /usr/bin/qemu-kvm: double free or corruption (!prev):
> 0x00007f2d38000a00 ***

This is a bug in QEMU.  A guest must not be able to trigger a crash.

> I can reproduce the problem with a simple dd, i.e.
> dd if=/dev/zero of=/dev/rdsk/c2d10p0 bs=5000 count=1

I think this a raw character device, which is why you're even able to
perform non-blocksize accesses?  Have you looked at how other drivers
(like the Xen pv blkfront) handle this?

> My driver will create a virtio-blk request with two elements in the sg list, one
> for the first 4096 byes and the other for the remaining 904.
>
> From stepping through with gdb, virtio_blk_handle_write will sets n_sectors to 9
> (5000 / 512). Later on the code, n_sectors is used the calculate the size of the
> buffer required but 9 * 512 is too small and so when the request is process it
> ends up writing past the end of the buffer and I guest this triggers the glibc
> error.

We need to validate that (qiov->size % BDRV_SECTOR_SIZE) == 0 and
reject invalid requests.

> Is there a requirement for virtio-blk guest drivers that all i/o requests are
> sized in multiples of 512 bytes?

There is no strict requirement according to the virtio specification,
but maybe there should be:

http://ozlabs.org/~rusty/virtio-spec/virtio-spec-0.8.9.pdf

Stefan