[Qemu-devel] Summary of CD, DVD passthrough tests with -drive if=scsi

[Qemu-devel] Summary of CD, DVD passthrough tests with -drive if=scsi

[Thomas Schmitt]

Hi,

since i had to give up the plan to test BD burning, i now post the
first summary of passthrough testing on base of  if=scsi .

Next i will try to follow Paolo Bonzini's proposal to use if=virtio.

-------------------------------------------------------------------------

List of successes and failures with various CD and DVD use cases
on an internal SATA drive at host Debian 6.0.2
  Linux ... 2.6.32-5-amd64 #1 SMP Tue Jun 14 09:42:28 UTC 2011 x86_64
with guest Debian 6.0.3
  Linux ... 2.6.32-5-686 #1 SMP Mon Oct 3 04:15:24 UTC 2011 i686


qemu source is a git clone obtained on November 2 by
  git clone git://git.qemu.org/qemu.git

File VERSION has date Nov 2 13:41 (+0100)
  $ cat VERSION
  0.15.90
  $ git describe
  fatal: No annotated tags can describe 'e072ea2fd8fdceef64159b9596d3c15ce01bea91'.
  However, there were unannotated tags: try --tags.
  $ git describe --tags
  v1.0-rc0


qemu start:

  .../x86_64-softmmu/qemu-system-x86_64 \
     -L .../pc-bios \
     -enable-kvm \
     -nographic \
     -m 512 \ 
     -net nic,model=ne2k_pci \
     -net user,hostfwd=tcp::5557-:22 \
     -hda /dvdbuffer/i386-install.qemu \
     -drive file=/dev/sg2,if=scsi \
     -cdrom /dvdbuffer/pseudo_drive

-------------------------------------------------------------------------

Successes:

  All media types

    Tray loading and ejecting.

  CD-RW (and most probably CD-R)

    Writing a session with a single track with write type TAO.
    As first session and as add-on session.

  DVD+RW

    Writing if SET STREAMING is omitted.

  DVD-RW (and most probably DVD-R)

    Write type Incremental (aka Packet) if SET STREAMING is omitted.

    Blanking.

    Writing to thoroughly formatted DVD-RW if SET STREAMING is omitted.

  DVD+R

    Writing sessions without using RESERVE TRACK, if SET STREAMING is omitted.

    Closing of medium.


-------------------------------------------------------------------------

Failures:

I count and classify them by these poetic category names:

  1  HOST CRASHER   kernel Oops and crash of the host system
  2  DRIVE FREEZER  makes the drive unusable on the host until host reboot
  4  GUEST KILLER   makes guest freeze or qemu abort
  1  MEDIUM KILLER  makes medium unusable (at least for current libburn)
  2  SHOWSTOPPER    prevents usage of drive or medium
  2  ANNOYING       severely hampers usage of medium
  1  UGLY           deprives user of less important information


  USB attached drive

    * HOST CRASHER
    * SHOWSTOPPER
    An USB attached drive does not perform any SCSI command,
    qemu ioctl() gets "No such device",
    host gets kernel Oops when guest is shut down,
    host shell sessions are frozen until hardware reset button.

  CD-RW (and most probably CD-R)

    * DRIVE FREEZER
    * GUEST KILLER
    Writing a first session to blank media with write type SAO.
    The guest freezes. The drive gets unusable by the host system
    until host reboot.
      SEND CUE SHEET
      5d 00 00 00 00 00 00 00 20 00 
      To drive: 32b
      41 00 00 01 00 00 00 00 41 01 00 10 00 00 00 00 41 01 01 10
      00 00 02 00 41 aa 01 01 00 00 35 30 
    

  DVD in general

    * DRIVE FREEZER
    * GUEST KILLER
    Guest freezes, qemu goes to 100% CPU, after
      SET STREAMING
      b6 00 00 00 00 00 00 00 00 00 1c 00 
      To drive: 28b
      00 00 00 00 00 00 00 00 00 23 04 88 10 00 00 00 00 00 03 e8
      10 00 00 00 00 00 03 e8 
    The drive is unusable by the host afterwards

    * ANNOYING
    Guest SG_IO timeout with READ DISC STRUCTURE formats
    0x04 , 0x11 , 0x0e , 0x10.
      READ DISC STRUCTURE
      ad 00 00 00 00 00 00 04 00 04 00 00 
      READ DISC STRUCTURE
      ad 00 00 00 00 00 00 11 00 04 00 00
      READ DISC STRUCTURE
      ad 00 00 00 00 00 00 0e 00 04 00 00 
      READ DISC STRUCTURE
      ad 00 00 00 00 00 00 10 00 04 00 00 

  DVD+RW

    * SHOWSTOPPER
    Failure to write to unformatted or partly formatted DVD+RW.
      FORMAT UNIT
      04 11 00 00 00 00 
      To drive: 12b
      00 02 00 08 ff ff ff ff 98 00 00 00
      +++ sense data = F0 00 0B 00 00 00 00 0A 00 00 00 00 00 00 00 00 00 00
      +++ key=B  asc=00h  ascq=00h   (   488 ms)

  DVD-RW

    * GUEST KILLER
    Guest freezes with DVD-RW write type DAO. qemu is on 100 % CPU.
    Most probably after RESERVE TRACK, although this command is not shown
    by the SCSI log. But the command before it is shown and the medium
    has inconsistent knowledge of the desired track size.

    * ANNOYING
    Failure to format sequential DVD-RW, and to write to partially
    formatted DVD-RW.

    * UGLY
    Failure to deliver progress of Blanking DVD-RW via REQUEST SENSE.
      REQUEST SENSE
      03 00 00 00 12 00 
      From drive: 18b
      f0 00 00 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 
           0 ms
    TEST UNIT READY delivers the desired information.

  DVD+R

    * GUEST KILLER
    * MEDIUM KILLER
    Guest freezes, medium has unwritten open reserved track after
      RESERVE TRACK
      53 00 00 00 00 00 00 0f 30 00

-------------------------------------------------------------------------

Proposals:

  Mode page 2A of emulated ATAPI DVD-ROM should have page length 0x14
  like SCSI CD-ROM, rather than 0x12. (Just add two 0-bytes to its end
  and adkust the size dependent fields.)

  MODE SENSE(6) should be thrown out of ATAPI DVD-ROM emulation.
  Its implementation is wrong. It is not specified in the ATAPI list
  of MMC-2. MMC-5 prescribes to use MODE SENSE(10).

  Let passthrough operate on /dev/sr too.
  Using /dev/sg as superuser makes the hard disk prone to be used
  as qemu -drive by mistake of the user.

-------------------------------------------------------------------------

Have a nice day :)

Thomas

Re: [Qemu-devel] Summary of CD, DVD passthrough tests with -drive if=scsi

[Markus Armbruster]

"Thomas Schmitt" <scdbackup@gmx.net> writes:

> Hi,
>
> since i had to give up the plan to test BD burning, i now post the
> first summary of passthrough testing on base of  if=scsi .

[Awesome stuff snipped...]

Thanks a lot!  We could use more tests like that.

Has anyone thought of keeping your tests around for regression testing
purposes?

Re: [Qemu-devel] Summary of CD, DVD passthrough tests with -drive if=scsi

[Thomas Schmitt]

Hi,

Markus Armbruster wrote:
> Has anyone thought of keeping your tests around for regression testing
> purposes?

The full tour lasts a whole workday and needs human attention.

I still did not come to test the newest proposals of Paolo Bonzini
and Zhi Yong Wu, because the last BD-R test was finished just five
minutes ago. (I'm not working full-time on it but i have to 
attend it every half hour or so.)
Now i do a last verification read run, to make sure that all went well.

It comes to me that i forgot to test DVD-RAM.
Thanks heaven that HD DVD-R never made it to the real-life market.

--------------------------------------------------------------------

For unattended tests i would propose to develop some dry exercises,
which perform SCSI commands of all three payload direction classes:
from_drive, to_drive, no_payload.

The test machine would need a real burner drive with e.g. a blank CD-RW
loaded, so that e.g. mode pages can be sent and read. Depending on
the individual drive, it might even work with an empty drive.
Sending a mode page 5 for write preparation is supposed to be harmless.
One could announce to the drive a CD SAO run, read it back, announce
a TAO run, read it back, and then just close the file descriptor to
the drive.
One could try to read from empty medium or drive and watch out
for the sense code that should come as protest.

This would not detect failures like of if=scsi with FORMAT UNIT or
SET STREAMING. But those would need real burn runs to show up.


For the next round of burn tests i will make a shorter cross section
through the media zoo, with re-usable media only.
(The BD-R contains a backup of about all interesting files of the host
 system and of the guest. It is now closed. 7738 MiB payload, 15 GiB wasted.)

I will record the options of all xorriso test runs.
But before this can be reproduced by other testers i will have to
work on libburn, so that one does not need an udev hack to get the
/dev/vda drive accepted as CD device.

The next release will hopefully happen in a few weeks and be able to
work in the guest system out of the box. GNU/Linux, FreeBSD, Solaris
are supported on real systems, currently. (Can FreeBSD and Solaris
use virtio drives ?)

I am open to proposals of other systems which can make use of the
qemu device setup, that works for Linux guests.
Mainly i would need info about possible drive addresses and how to
perform SCSI transactions from userspace. And help with installing
the guest system, of course.

--------------------------------------------------------------------

Have a nice day :)

Thomas

Re: [Qemu-devel] Summary of CD, DVD passthrough tests with -drive if=scsi

[Paolo Bonzini]

On 11/07/2011 03:31 PM, Thomas Schmitt wrote:
>
> The next release will hopefully happen in a few weeks and be able to
> work in the guest system out of the box. GNU/Linux, FreeBSD, Solaris
> are supported on real systems, currently. (Can FreeBSD and Solaris
> use virtio drives ?)

Perhaps, but probably not with SG_IO so that's a "no" for your use case.

Paolo

Re: [Qemu-devel] Summary of CD, DVD passthrough tests with -drive if=scsi

[Thomas Schmitt]

Hi,

Paolo Bonzini wrote:
> Yes, docs/qdev-device-use.txt helps.  "qemu -device virtio-blk,?" too.

I read it and ran '-device ?'. But both left me clueless.
The text i understood mainly as refering to "old ways" and "new way"
of defining devices. I stalled when i found no flesh for "HOST-OPTS"
and "DEV-OPTS". man ./qemu.1 lists options of -devices, but it did
not lead me to "virtio-blk" as driver name. 

Do i get it right that e.g.
  virtio-blk-pci.scsi=on/off
means there is an option
  -device virtio-blk,scsi=on

But how would i come from there to your proposal ?
  -device virtio-blk,drive=scsicd,logical_block_size=2048,physical_block_size=2048

Not nagging, just pointing out the problems of a noob.
To my luck, you are a very friendly tutor.
Else i would have failed early.


i wrote:
> > Can FreeBSD and Solaris use virtio drives ?

Paolo Bonzini wrote:
> Perhaps, but probably not with SG_IO so that's a "no" for your use case.

You mean SG_IO on the host system ?
On the guest, libburn would use CAM resp. uscsi.

I tried to learn about the overall concept of virtio but mostly
find prescriptions how to use it.
How much similarity between host and guest is needed ?


Actually i came to qemu because on the long term i want to see
GNU xorriso burn a DVD on GNU/Hurd. Just for fulfilling my duty
as GNU maintainer. (Not that anybody else would find this to
be a reasonable idea ... :))
But i understand that GNU/Hurd will not offer the guest support
for virtio. So i will have to resort to -cdrom for development
and an old PC for burn testing, if ever an RPC for userspace
SCSI transactions gets approved.
(Inside gnumach, there sit transaction calls for ATAPI and SCSI.
 But there is no way to use them directly from Hurd.)


Have a nice day :)

Thomas

Re: [Qemu-devel] Summary of CD, DVD passthrough tests with -drive if=scsi

[Paolo Bonzini]

On 11/07/2011 06:02 PM, Thomas Schmitt wrote:
>
> But how would i come from there to your proposal ?
>    -device virtio-blk,drive=scsicd,logical_block_size=2048,physical_block_size=2048

First of all, virtio-blk is an alias for virtio-blk-pci.  I should not 
have used it as it's likely confusing, but anyway "qemu-kvm -device 
virtio-blk-pci,?" tells you all the options that a virtio-blk(-pci) 
device can have.

The idea then is that when you want to express extra options, you cannot 
use simply "-drive if=virtio", and qdev-device-use.txt then hints about 
how to convert -drive if=virtio" to "-drive if=none" + "-device virtio-blk".

Paolo

Re: [Qemu-devel] Summary of CD, DVD passthrough tests with -drive if=scsi

[Paolo Bonzini]

On 11/06/2011 12:26 PM, Thomas Schmitt wrote:
>      * DRIVE FREEZER
>      * GUEST KILLER
>        SEND CUE SHEET
>
>      * DRIVE FREEZER
>      * GUEST KILLER
>        SET STREAMING
>
>      * ANNOYING
>        READ DISC STRUCTURE
>
>      * SHOWSTOPPER
>        FORMAT UNIT

I fixed all of these; only these remain:

>
>      * UGLY
>      Failure to deliver progress of Blanking DVD-RW via REQUEST SENSE.
>        REQUEST SENSE
>        03 00 00 00 12 00
>        From drive: 18b
>        f0 00 00 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00
>             0 ms
>      TEST UNIT READY delivers the desired information.
>
>    DVD+R
>
>      * GUEST KILLER
>      * MEDIUM KILLER
>      Guest freezes, medium has unwritten open reserved track after
>        RESERVE TRACK
>        53 00 00 00 00 00 00 0f 30 00
>
>    Mode page 2A of emulated ATAPI DVD-ROM should have page length 0x14
>    like SCSI CD-ROM, rather than 0x12. (Just add two 0-bytes to its end
>    and adkust the size dependent fields.)
>
>    MODE SENSE(6) should be thrown out of ATAPI DVD-ROM emulation.
>    Its implementation is wrong. It is not specified in the ATAPI list
>    of MMC-2. MMC-5 prescribes to use MODE SENSE(10).

The bugs were simply that QEMU did not compute the length of the command 
payload correctly.

Paolo

Re: [Qemu-devel] Summary of CD, DVD passthrough tests with -drive if=scsi

[Paolo Bonzini]

On 11/09/2011 06:47 PM, Paolo Bonzini wrote:
>>
>>    DVD+R
>>
>>      * GUEST KILLER
>>      * MEDIUM KILLER
>>      Guest freezes, medium has unwritten open reserved track after
>>        RESERVE TRACK
>>        53 00 00 00 00 00 00 0f 30 00
>>
>>    Mode page 2A of emulated ATAPI DVD-ROM should have page length 0x14
>>    like SCSI CD-ROM, rather than 0x12. (Just add two 0-bytes to its end
>>    and adkust the size dependent fields.)
>>
>>    MODE SENSE(6) should be thrown out of ATAPI DVD-ROM emulation.
>>    Its implementation is wrong. It is not specified in the ATAPI list
>>    of MMC-2. MMC-5 prescribes to use MODE SENSE(10).

These done too, will send patches tomorrow.

Paolo

Re: [Qemu-devel] Summary of CD, DVD passthrough tests with -drive if=scsi

[Thomas Schmitt]

Hi,

i reported nastily bahaving SCSI commands SEND CUE SHEET, SET STREAMING,
FORMAT UNIT, READ DISC STRUCTURE, REQUEST SENSE , RESERVE TRACK.
This was with
  -drive file=/dev/sg2,if=scsi

Paolo Bonzini wrote:
> I fixed all of these; [...]
> The bugs were simply that QEMU did not compute the length of the command
> payload correctly.
> [...] will send patches tomorrow.

Plus instructions for me, please, when and how to update my git clone. :)

I will then test by my new xorriso-on-qemu wiki

  http://libburnia-project.org/wiki/QemuXorriso

The development tarball of GNU xorriso is now supposed to work on a
GNU/Linux guest with /dev/vda.
The wiki describes guest preparations for Debian GNU/Linux 6.
With other distros, one might have to do different things to get a /dev/sr1
and rw-permissions to /dev/vda. 

Well, -drive if=scsi will produce a /dev/sr automatically.
(Note to myself: disable udev hack before if=scsi test.)


Have a nice day :)

Thomas