[Qemu-devel] [PATCH] Megasas HBA emulation and SCSI update v.2

[Qemu-devel] [PATCH] Megasas HBA emulation and SCSI update v.2

[Hannes Reinecke]

The is the second version of the megasas patchset; changes are:
v1 -> v2:
- Include suggestions (and acked-by) from Christoph Hellwig
- Updated patch description as requested by Gerd Hoffman
- Use generic iov_size() implementation
- Drop old patches
- Make SCSI HBA configurable

This patchset can be found at

git://repo.or.cz/qemu/megasas.git
branch megasas.v2

The original description was:
Due to popular demand here is now the LSI Megaraid HBA emulation.
And an update to the SCSI stack to bring it more in line with
modern standards.

For the SCSI update there are three key points:
- Improved sense code handling: the SCSI standard specifies several
  sense codes to be returned on common errors; we should be following
  that.
- Reworked SCSI layer: Instead of indirectly addressing a SCSI
  request by an abstract 'tag' we're now using the request directly,
  thus saving us a lookup on every invocation.
- Add iovec passing: I've added an interface to pass an iovec
  to a SCSI request, so that we don't need to use bounce buffers
  for data transfer. The old interface is left in place.

And, of course, the megasas driver itself.

A quick 'dd' comparison against virtio on a tmpfs backed sparse file shows:

# dd if=/dev/sdb of=/dev/null bs=16M count=1000 iflag=direct
1000+0 records in
1000+0 records out
16777216000 bytes (17 GB) copied, 9.40123 s, 1.8 GB/s
# dd if=/dev/vda of=/dev/null bs=16M count=1000 iflag=direct
1000+0 records in
1000+0 records out
16777216000 bytes (17 GB) copied, 7.44681 s, 2.3 GB/s

So we're having about 80% virtio speed.
Unoptimized :-)

Hannes Reinecke (15):
  scsi: Increase the number of possible devices
  scsi: Return SAM status codes
  scsi: INQUIRY VPD fixes
  scsi: Move sense handling into the driver
  scsi-disk: Remove duplicate cdb parsing
  scsi: Update sense code handling
  lsi53c895a: Rename 'sense' to 'status'
  scsi-disk: Allocate iovec dynamically
  scsi: Use 'SCSIRequest' directly
  scsi-disk: add data direction checking
  Remove 'bus' argument from SCSI command completion callbacks
  scsi: Implement 'get_sense' callback
  scsi: Implement alloc_req_iov callback
  megasas: LSI Megaraid SAS emulation
  Make SCSI HBA configurable

 Makefile.objs                        |    5 +-
 blockdev.h                           |    2 +-
 default-configs/i386-softmmu.mak     |    2 +
 default-configs/mips-softmmu.mak     |    2 +-
 default-configs/mips64-softmmu.mak   |    2 +-
 default-configs/mips64el-softmmu.mak |    2 +-
 default-configs/mipsel-softmmu.mak   |    2 +-
 default-configs/ppc-softmmu.mak      |    2 +
 default-configs/ppc64-softmmu.mak    |    2 +
 default-configs/ppcemb-softmmu.mak   |    2 +
 default-configs/sparc-softmmu.mak    |    2 +-
 default-configs/sparc64-softmmu.mak  |    2 +
 default-configs/x86_64-softmmu.mak   |    2 +
 hw/esp.c                             |   23 +-
 hw/lsi53c895a.c                      |   47 +-
 hw/megasas.c                         | 1826 ++++++++++++++++++++++++++++++++++
 hw/mfi.h                             | 1197 ++++++++++++++++++++++
 hw/pci_ids.h                         |    2 +
 hw/scsi-bus.c                        |  120 ++-
 hw/scsi-defs.h                       |   20 +-
 hw/scsi-disk.c                       |  446 ++++-----
 hw/scsi-generic.c                    |  238 +++--
 hw/scsi.h                            |   64 +-
 hw/usb-msd.c                         |   26 +-
 24 files changed, 3603 insertions(+), 435 deletions(-)
 create mode 100644 hw/megasas.c
 create mode 100644 hw/mfi.h

Re: [Qemu-devel] [PATCH] Megasas HBA emulation and SCSI update v.2

[Hannes Reinecke]

On 11/22/2010 11:15 AM, Hannes Reinecke wrote:
> 
> The is the second version of the megasas patchset; changes are:
> v1 -> v2:
> - Include suggestions (and acked-by) from Christoph Hellwig
> - Updated patch description as requested by Gerd Hoffman
> - Use generic iov_size() implementation
> - Drop old patches
> - Make SCSI HBA configurable
> 
> This patchset can be found at
> 
> git://repo.or.cz/qemu/megasas.git
> branch megasas.v2
> 
Hey, sorry, just found out I've omitted the series numbers
in the patchset. But reposting the entire patchset just
to get the numbers back is a bit of a waste.
So please check with the above git tree or
use this as a series file:
> 
> Hannes Reinecke (15):
>   scsi: Increase the number of possible devices
>   scsi: Return SAM status codes
>   scsi: INQUIRY VPD fixes
>   scsi: Move sense handling into the driver
>   scsi-disk: Remove duplicate cdb parsing
>   scsi: Update sense code handling
>   lsi53c895a: Rename 'sense' to 'status'
>   scsi-disk: Allocate iovec dynamically
>   scsi: Use 'SCSIRequest' directly
>   scsi-disk: add data direction checking
>   Remove 'bus' argument from SCSI command completion callbacks
>   scsi: Implement 'get_sense' callback
>   scsi: Implement alloc_req_iov callback
>   megasas: LSI Megaraid SAS emulation
>   Make SCSI HBA configurable
> 

Sorry about this.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		      zSeries & Storage
hare@suse.de			      +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Markus Rex, HRB 16746 (AG Nürnberg)

Re: [Qemu-devel] [PATCH] Megasas HBA emulation and SCSI update v.2

[Nicholas A. Bellinger]

On Mon, 2010-11-22 at 11:34 +0100, Hannes Reinecke wrote:
> On 11/22/2010 11:15 AM, Hannes Reinecke wrote:
> > 
> > The is the second version of the megasas patchset; changes are:
> > v1 -> v2:
> > - Include suggestions (and acked-by) from Christoph Hellwig
> > - Updated patch description as requested by Gerd Hoffman
> > - Use generic iov_size() implementation
> > - Drop old patches
> > - Make SCSI HBA configurable
> > 
> > This patchset can be found at
> > 
> > git://repo.or.cz/qemu/megasas.git
> > branch megasas.v2
> > 
> Hey, sorry, just found out I've omitted the series numbers
> in the patchset. But reposting the entire patchset just
> to get the numbers back is a bit of a waste.
> So please check with the above git tree or
> use this as a series file:
> > 
> > Hannes Reinecke (15):
> >   scsi: Increase the number of possible devices
> >   scsi: Return SAM status codes
> >   scsi: INQUIRY VPD fixes
> >   scsi: Move sense handling into the driver
> >   scsi-disk: Remove duplicate cdb parsing
> >   scsi: Update sense code handling
> >   lsi53c895a: Rename 'sense' to 'status'
> >   scsi-disk: Allocate iovec dynamically
> >   scsi: Use 'SCSIRequest' directly
> >   scsi-disk: add data direction checking
> >   Remove 'bus' argument from SCSI command completion callbacks
> >   scsi: Implement 'get_sense' callback
> >   scsi: Implement alloc_req_iov callback
> >   megasas: LSI Megaraid SAS emulation
> >   Make SCSI HBA configurable
> > 
> 

Hey Hannes,

Just a heads up, I noticed that the latest v2 megasas w/ scsi-generic ->
TCM_loop appears to be broken on a Windows7 (Build 7600) guest, which
hangs during boot -> LUN scan with the following:

truelife:/usr/src/qemu-kvm.git# ./x86_64-softmmu/qemu-system-x86_64 -m 512 -boot c ~/windows7.img \
		-drive if=none,id=mydisk1,file=/dev/sg4 -device megasas,id=raid
		-device scsi-generic,bus=raid.0,scsi-id=1,drive=mydisk1

megasas: Using 80 sges, 1000 cmds, raid mode
megasas: Reset
megasas: Mapping MMIO region 0 at f2040000
megasas: Mapping IO region 2 at 0000c200
megasas: Mapping QUEUE region 3 at f2080000
megasas: Mapping MMIO region 0 at f2040000
megasas: Mapping IO region 2 at 0000c200
megasas: Mapping QUEUE region 3 at f2080000
megasas: Mapping MMIO region 0 at f2040000
megasas: Mapping IO region 2 at 0000c200
megasas: Mapping QUEUE region 3 at f2080000
megasas: readl mmio 0xb0
megasas: writel mmio 20: 7
megasas: Reset
megasas: readl mmio 0x20
megasas: writel mmio 40: 1ff9c041
megasas: Received frame addr 1ff9c000 count 32
megasas: MFI cmd 0 context 0 count 32
megasas: Return new frame 0 cmd 0x7f7711654330
megasas: Enqueue frame 0 count 32 context 0 tail 0 busy 1
megasas: MFI init firmware: xfer len 0 pa 0
megasas: MFI init firmware: queue at f000ff53f000e2c3 len -268370093 head f000ff53f000ff53 tail f000ff53f000ff53
megasas: Complete frame context 0


Oddly enough, after trying a number of older megasas versions with this
particular Windows 7 guest w/ scsi-generic -> TCM_Loop, the last version
that I can find which is still working is this branch from May, on what
is effectively v1.0 megasas code.

http://git.kernel.org/?p=virt/kvm/nab/qemu-kvm.git;a=shortlog;h=refs/heads/windows7-megasas-working

Anyways, I have not had a chance to track this down myself and wanted
to see if you had any ideas..

--nab

Re: [Qemu-devel] [PATCH] Megasas HBA emulation and SCSI update v.2

[Hannes Reinecke]

Hey Nic,

On 11/24/2010 10:41 AM, Nicholas A. Bellinger wrote:
> On Mon, 2010-11-22 at 11:34 +0100, Hannes Reinecke wrote:
[ .. ]
> 
> Hey Hannes,
> 
> Just a heads up, I noticed that the latest v2 megasas w/ scsi-generic ->
> TCM_loop appears to be broken on a Windows7 (Build 7600) guest, which
> hangs during boot -> LUN scan with the following:
> 
> truelife:/usr/src/qemu-kvm.git# ./x86_64-softmmu/qemu-system-x86_64 -m 512 -boot c ~/windows7.img \
> 		-drive if=none,id=mydisk1,file=/dev/sg4 -device megasas,id=raid
> 		-device scsi-generic,bus=raid.0,scsi-id=1,drive=mydisk1
> 
> megasas: Using 80 sges, 1000 cmds, raid mode
> megasas: Reset
> megasas: Mapping MMIO region 0 at f2040000
> megasas: Mapping IO region 2 at 0000c200
> megasas: Mapping QUEUE region 3 at f2080000
> megasas: Mapping MMIO region 0 at f2040000
> megasas: Mapping IO region 2 at 0000c200
> megasas: Mapping QUEUE region 3 at f2080000
> megasas: Mapping MMIO region 0 at f2040000
> megasas: Mapping IO region 2 at 0000c200
> megasas: Mapping QUEUE region 3 at f2080000
> megasas: readl mmio 0xb0
> megasas: writel mmio 20: 7
> megasas: Reset
> megasas: readl mmio 0x20
> megasas: writel mmio 40: 1ff9c041
> megasas: Received frame addr 1ff9c000 count 32
> megasas: MFI cmd 0 context 0 count 32
> megasas: Return new frame 0 cmd 0x7f7711654330
> megasas: Enqueue frame 0 count 32 context 0 tail 0 busy 1
> megasas: MFI init firmware: xfer len 0 pa 0
> megasas: MFI init firmware: queue at f000ff53f000e2c3 len -268370093 head f000ff53f000ff53 tail f000ff53f000ff53
> megasas: Complete frame context 0
> 
Okay, it looks as if I've fixed it up.
Win7 32bit works now with my megasas.v3 tree.
Curiously, Win7 64bit fails; it crashes at relative address
28F4, wherever that's supposed to be.
And, of course, Windows Vista with newest driver from LSI fails, too
:-(.
Guess I need to do some more debugging here.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		      zSeries & Storage
hare@suse.de			      +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Markus Rex, HRB 16746 (AG Nürnberg)

Re: [Qemu-devel] [PATCH] Megasas HBA emulation and SCSI update v.2

[Hannes Reinecke]

On 12/01/2010 03:18 PM, Hannes Reinecke wrote:
> Hey Nic,
> 
> On 11/24/2010 10:41 AM, Nicholas A. Bellinger wrote:
>> On Mon, 2010-11-22 at 11:34 +0100, Hannes Reinecke wrote:
> [ .. ]
>>
>> Hey Hannes,
>>
>> Just a heads up, I noticed that the latest v2 megasas w/ scsi-generic ->
>> TCM_loop appears to be broken on a Windows7 (Build 7600) guest, which
>> hangs during boot -> LUN scan with the following:
>>
>> truelife:/usr/src/qemu-kvm.git# ./x86_64-softmmu/qemu-system-x86_64 -m 512 -boot c ~/windows7.img \
>> 		-drive if=none,id=mydisk1,file=/dev/sg4 -device megasas,id=raid
>> 		-device scsi-generic,bus=raid.0,scsi-id=1,drive=mydisk1
>>
>> megasas: Using 80 sges, 1000 cmds, raid mode
>> megasas: Reset
>> megasas: Mapping MMIO region 0 at f2040000
>> megasas: Mapping IO region 2 at 0000c200
>> megasas: Mapping QUEUE region 3 at f2080000
>> megasas: Mapping MMIO region 0 at f2040000
>> megasas: Mapping IO region 2 at 0000c200
>> megasas: Mapping QUEUE region 3 at f2080000
>> megasas: Mapping MMIO region 0 at f2040000
>> megasas: Mapping IO region 2 at 0000c200
>> megasas: Mapping QUEUE region 3 at f2080000
>> megasas: readl mmio 0xb0
>> megasas: writel mmio 20: 7
>> megasas: Reset
>> megasas: readl mmio 0x20
>> megasas: writel mmio 40: 1ff9c041
>> megasas: Received frame addr 1ff9c000 count 32
>> megasas: MFI cmd 0 context 0 count 32
>> megasas: Return new frame 0 cmd 0x7f7711654330
>> megasas: Enqueue frame 0 count 32 context 0 tail 0 busy 1
>> megasas: MFI init firmware: xfer len 0 pa 0
>> megasas: MFI init firmware: queue at f000ff53f000e2c3 len -268370093 head f000ff53f000ff53 tail f000ff53f000ff53
>> megasas: Complete frame context 0
>>
> Okay, it looks as if I've fixed it up.
> Win7 32bit works now with my megasas.v3 tree.
> Curiously, Win7 64bit fails; it crashes at relative address
> 28F4, wherever that's supposed to be.
> And, of course, Windows Vista with newest driver from LSI fails, too
> :-(.
> Guess I need to do some more debugging here.
> 
Hmpf. Using a new vista x86 image (build 6002) with SP2 preloaded
megasas works, too.
Dodgy build I had, apparently.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		      zSeries & Storage
hare@suse.de			      +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Markus Rex, HRB 16746 (AG Nürnberg)

Re: [Qemu-devel] [PATCH] Megasas HBA emulation and SCSI update v.2

[Nicholas A. Bellinger]

On Wed, 2010-12-01 at 16:46 +0100, Hannes Reinecke wrote:
> On 12/01/2010 03:18 PM, Hannes Reinecke wrote:
> > Hey Nic,
> > 
> > On 11/24/2010 10:41 AM, Nicholas A. Bellinger wrote:
> >> On Mon, 2010-11-22 at 11:34 +0100, Hannes Reinecke wrote:
> > [ .. ]
> >>
> >> Hey Hannes,
> >>
> >> Just a heads up, I noticed that the latest v2 megasas w/ scsi-generic ->
> >> TCM_loop appears to be broken on a Windows7 (Build 7600) guest, which
> >> hangs during boot -> LUN scan with the following:
> >>
> >> truelife:/usr/src/qemu-kvm.git# ./x86_64-softmmu/qemu-system-x86_64 -m 512 -boot c ~/windows7.img \
> >> 		-drive if=none,id=mydisk1,file=/dev/sg4 -device megasas,id=raid
> >> 		-device scsi-generic,bus=raid.0,scsi-id=1,drive=mydisk1
> >>
> >> megasas: Using 80 sges, 1000 cmds, raid mode
> >> megasas: Reset
> >> megasas: Mapping MMIO region 0 at f2040000
> >> megasas: Mapping IO region 2 at 0000c200
> >> megasas: Mapping QUEUE region 3 at f2080000
> >> megasas: Mapping MMIO region 0 at f2040000
> >> megasas: Mapping IO region 2 at 0000c200
> >> megasas: Mapping QUEUE region 3 at f2080000
> >> megasas: Mapping MMIO region 0 at f2040000
> >> megasas: Mapping IO region 2 at 0000c200
> >> megasas: Mapping QUEUE region 3 at f2080000
> >> megasas: readl mmio 0xb0
> >> megasas: writel mmio 20: 7
> >> megasas: Reset
> >> megasas: readl mmio 0x20
> >> megasas: writel mmio 40: 1ff9c041
> >> megasas: Received frame addr 1ff9c000 count 32
> >> megasas: MFI cmd 0 context 0 count 32
> >> megasas: Return new frame 0 cmd 0x7f7711654330
> >> megasas: Enqueue frame 0 count 32 context 0 tail 0 busy 1
> >> megasas: MFI init firmware: xfer len 0 pa 0
> >> megasas: MFI init firmware: queue at f000ff53f000e2c3 len -268370093 head f000ff53f000ff53 tail f000ff53f000ff53
> >> megasas: Complete frame context 0
> >>
> > Okay, it looks as if I've fixed it up.
> > Win7 32bit works now with my megasas.v3 tree.
> > Curiously, Win7 64bit fails; it crashes at relative address
> > 28F4, wherever that's supposed to be.
> > And, of course, Windows Vista with newest driver from LSI fails, too
> > :-(.
> > Guess I need to do some more debugging here.
> > 
> Hmpf. Using a new vista x86 image (build 6002) with SP2 preloaded
> megasas works, too.
> Dodgy build I had, apparently.
> 

Thanks for the update..  After testing the lastest megasas.v3 HEAD
at commit:

* megasas.v3 978e61e megasas: Fixup PD query return value

it appears that the same Win7 64-bit Build 7600 that is functioning with
v0.12.5 windows7-megasas-working will now BSOD the guest.  After further checking
it appears that this is not megasas HBA specific, and is due to your tree being
slightly more out of date than mine.  ;)

But the good news is that WinXP SP2 is now working via scsi-generic ->
TCM_Loop in megasas.v3, and even w/o the original sync ioctl patch we
required  in v0.12.5 megasas code.  Very excellent work Hannes!

So, I will be merging the latest changes from megasas.v3 -> megasas-upstream-v1
shortly and retesting with 64-bit Build 7600.

Best,

--nab

Re: [Qemu-devel] [PATCH] Megasas HBA emulation and SCSI update v.2

[Hannes Reinecke]

On 12/02/2010 01:14 AM, Nicholas A. Bellinger wrote:
> On Wed, 2010-12-01 at 16:46 +0100, Hannes Reinecke wrote:
>> On 12/01/2010 03:18 PM, Hannes Reinecke wrote:
>>> Hey Nic,
>>>
>>> On 11/24/2010 10:41 AM, Nicholas A. Bellinger wrote:
>>>> On Mon, 2010-11-22 at 11:34 +0100, Hannes Reinecke wrote:
>>> [ .. ]
>>>>
>>>> Hey Hannes,
>>>>
>>>> Just a heads up, I noticed that the latest v2 megasas w/ scsi-generic ->
>>>> TCM_loop appears to be broken on a Windows7 (Build 7600) guest, which
>>>> hangs during boot -> LUN scan with the following:
>>>>
>>>> truelife:/usr/src/qemu-kvm.git# ./x86_64-softmmu/qemu-system-x86_64 -m 512 -boot c ~/windows7.img \
>>>> 		-drive if=none,id=mydisk1,file=/dev/sg4 -device megasas,id=raid
>>>> 		-device scsi-generic,bus=raid.0,scsi-id=1,drive=mydisk1
>>>>
>>>> megasas: Using 80 sges, 1000 cmds, raid mode
>>>> megasas: Reset
>>>> megasas: Mapping MMIO region 0 at f2040000
>>>> megasas: Mapping IO region 2 at 0000c200
>>>> megasas: Mapping QUEUE region 3 at f2080000
>>>> megasas: Mapping MMIO region 0 at f2040000
>>>> megasas: Mapping IO region 2 at 0000c200
>>>> megasas: Mapping QUEUE region 3 at f2080000
>>>> megasas: Mapping MMIO region 0 at f2040000
>>>> megasas: Mapping IO region 2 at 0000c200
>>>> megasas: Mapping QUEUE region 3 at f2080000
>>>> megasas: readl mmio 0xb0
>>>> megasas: writel mmio 20: 7
>>>> megasas: Reset
>>>> megasas: readl mmio 0x20
>>>> megasas: writel mmio 40: 1ff9c041
>>>> megasas: Received frame addr 1ff9c000 count 32
>>>> megasas: MFI cmd 0 context 0 count 32
>>>> megasas: Return new frame 0 cmd 0x7f7711654330
>>>> megasas: Enqueue frame 0 count 32 context 0 tail 0 busy 1
>>>> megasas: MFI init firmware: xfer len 0 pa 0
>>>> megasas: MFI init firmware: queue at f000ff53f000e2c3 len -268370093 head f000ff53f000ff53 tail f000ff53f000ff53
>>>> megasas: Complete frame context 0
>>>>
>>> Okay, it looks as if I've fixed it up.
>>> Win7 32bit works now with my megasas.v3 tree.
>>> Curiously, Win7 64bit fails; it crashes at relative address
>>> 28F4, wherever that's supposed to be.
>>> And, of course, Windows Vista with newest driver from LSI fails, too
>>> :-(.
>>> Guess I need to do some more debugging here.
>>>
>> Hmpf. Using a new vista x86 image (build 6002) with SP2 preloaded
>> megasas works, too.
>> Dodgy build I had, apparently.
>>
> 
> Thanks for the update..  After testing the lastest megasas.v3 HEAD
> at commit:
> 
> * megasas.v3 978e61e megasas: Fixup PD query return value
> 
> it appears that the same Win7 64-bit Build 7600 that is functioning with
> v0.12.5 windows7-megasas-working will now BSOD the guest.  After further checking
> it appears that this is not megasas HBA specific, and is due to your tree being
> slightly more out of date than mine.  ;)
> 
Yes, this is totally weird. AFAICS the MMIO register data is
_exactly_ identical for both, the old working one and the new
implementation. Yet Win7 is behaving differently in both cases.
So it must be indeed the qemu base which is doing odd things here.

But that's a good hint, I'll be updating my tree and see how far
I'll progress.

> But the good news is that WinXP SP2 is now working via scsi-generic ->
> TCM_Loop in megasas.v3, and even w/o the original sync ioctl patch we
> required  in v0.12.5 megasas code.  Very excellent work Hannes!
> 
> So, I will be merging the latest changes from megasas.v3 -> megasas-upstream-v1
> shortly and retesting with 64-bit Build 7600.
> 
Cool. Thanks. I'll be rebasing my patches, too. I guess it's time
for megasas.v4.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		      zSeries & Storage
hare@suse.de			      +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Markus Rex, HRB 16746 (AG Nürnberg)

Re: [Qemu-devel] [PATCH] Megasas HBA emulation and SCSI update v.2

[Nicholas A. Bellinger]

On Thu, 2010-12-02 at 08:51 +0100, Hannes Reinecke wrote:
> On 12/02/2010 01:14 AM, Nicholas A. Bellinger wrote:
> > On Wed, 2010-12-01 at 16:46 +0100, Hannes Reinecke wrote:
> >> On 12/01/2010 03:18 PM, Hannes Reinecke wrote:

<SNIP>

> >> Hmpf. Using a new vista x86 image (build 6002) with SP2 preloaded
> >> megasas works, too.
> >> Dodgy build I had, apparently.
> >>
> > 
> > Thanks for the update..  After testing the lastest megasas.v3 HEAD
> > at commit:
> > 
> > * megasas.v3 978e61e megasas: Fixup PD query return value
> > 
> > it appears that the same Win7 64-bit Build 7600 that is functioning with
> > v0.12.5 windows7-megasas-working will now BSOD the guest.  After further checking
> > it appears that this is not megasas HBA specific, and is due to your tree being
> > slightly more out of date than mine.  ;)
> > 
> Yes, this is totally weird. AFAICS the MMIO register data is
> _exactly_ identical for both, the old working one and the new
> implementation. Yet Win7 is behaving differently in both cases.
> So it must be indeed the qemu base which is doing odd things here.
> 

Ok, after spending the better part of the evening trying to identify
differences between the two resv, I am inclined to agree with you here.

After merging megasas.v3 into megasas-upstream-v1 and pushing into
qemu-kvm.git, I did finally run into a semi meaningful BSOD with the
64-bit guest here:

http://linux-iscsi.org/builds/megasas-emulation-logs/win7-64bit-megasas-BSOD-12022010-1.png

which is happening after the initial run of DCMDs complete successfully,
and for the first 16-byte INQUIRY frame into megasas_handle_scsi()..  

Here is a snippet from the log:

<SNIP past initial DCMDs completed successfully>

megasas: Enqueue frame 1 count 0 context 3e6 tail 0 busy 1
megasas: frame 1: MFI DCMD opcode 1040500
megasas: DCMD controller event wait
megasas: MFI DCMD wrote 0 bytes
megasas: Complete frame context 3e6

<Last DMCD before first MFI_CMD_PD_SCSI_IO frame:

megasas: writel mmio 0xa0: ffffffff
megasas: Update reply queue head 0 busy 0
megasas: writel mmio 0x34: 7ffffffb
megasas: writel mmio 0x40: 1fd76381
megasas: Received frame addr 1fd76380 count 0
megasas: MFI cmd 4 context 0 count 0
megasas: Return new frame 2 cmd 0x7fb7cebd53e0
megasas: Enqueue frame 2 count 0 context 0 tail 0 busy 1
megasas: PD SCSI physical dev 0 lun 0 sdev 0x139b9f0 xfer 16
megasas: 16 bytes of data available for reading
megasas: PD SCSI req 0x13bb1c0 cmd 0x7fb7cebd53e0 lun 0x139b9f0 command completed, arg 16
megasas: PD SCSI req 0x13bb1c0 cmd 0x7fb7cebd53e0 lun 0x139b9f0 read finished, len 16
megasas: PD SCSI req 0x13bb1c0 cmd 0x7fb7cebd53e0 lun 0x139b9f0 command completed, arg 0
megasas: PD SCSI req 0x13bb1c0 cmd 0x7fb7cebd53e0 lun 0x139b9f0 finished with status 0 len 16
megasas: Complete frame context 0 tail 0 busy 0 doorbell 0
megasas: readl mmio 0x30: 80000001
megasas: writel mmio 0xa0: 80000001
megasas: Update reply queue head 1 busy 0

....  and at this point a BSOD is triggered in the 64-bit Win7 guest with
DRIVER_IRQL_NOT_LESS_OR_EQUAL Stop Code = 0xD1, which seems from a quick
google search to indicate a problem wrt to paging and DMA transfers.

So, after that I started to compare both versions w/ all megasas
debugging enabled, and had another look at the code I did notice a few
subtle differences however between megasas.v3 in
megasas_mmio_writel:MFI_IQP* that you changed recently.  So your
recent change here revets back to v0.12.5 logic, but the frame_count
assignment is still different:

       case MFI_IQP:
            /* Received MFI frame address */
            frame_addr = (val & ~0xF);
            /* Add possible 64 bit offset */
            frame_addr |= (uint64_t)s->frame_hi;
            s->frame_hi = 0;
            frame_count = (val & 0xF) >> 1;
	    <SNIP>

and v0.12.5 windows7-megasas-working:

    case MFI_IQP:
        /* Received MFI frames; up to 8 contiguous frames */
        frame_addr = (val & ~0xF);
        /* Add possible 64 bit offset */
        frame_addr |= (uint64_t)s->frame_hi;
        s->frame_hi = 0;
        frame_count = (val >> 1) & 0x7;
	<SNIP>

Unfortuately this does not appear to make a difference when changing
megasas.v3 follow the existing windows7-megasas-working code, and the
frame_addr assignment recently changed back in megasas.v3 now matches
v0.12.5 code.

Both logs are attached for reference, and aside from the frame_count,
the only other thing that I am noticing is that the struct megasas_cmd_t
%p pointers in the working v0.12.5 are showing low memory addresses,
for example: 

megasas: writel mmio 40: 1f15c381
megasas: Received frame addr 1f15c380 count 0
megasas: MFI cmd 4 context 0 count 0
megasas: Return new frame 2 cmd 0xf077a8
megasas: Enqueue frame context 0 tail 0 busy 1
megasas: PD SCSI dev 0 lun 0 sdev 0xf1e5a0 xfer 16
megasas: PD SCSI req 0xf38120 cmd 0xf077a8 lun 0xf1e5a0 finished with status 0 len 16
megasas: Complete frame context 0 tail 0 busy 0 doorbell 0

and the latest code is showing the same pointers for *cmd as:

megasas: writel mmio 0x40: 1fd76381
megasas: Received frame addr 1fd76380 count 0
megasas: MFI cmd 4 context 0 count 0
megasas: Return new frame 2 cmd 0x7fb7cebd53e0
megasas: Enqueue frame 2 count 0 context 0 tail 0 busy 1
megasas: PD SCSI physical dev 0 lun 0 sdev 0x139b9f0 xfer 16
megasas: 16 bytes of data available for reading
megasas: PD SCSI req 0x13bb1c0 cmd 0x7fb7cebd53e0 lun 0x139b9f0 command completed, arg 16
megasas: PD SCSI req 0x13bb1c0 cmd 0x7fb7cebd53e0 lun 0x139b9f0 read finished, len 16
megasas: PD SCSI req 0x13bb1c0 cmd 0x7fb7cebd53e0 lun 0x139b9f0 command completed, arg 0
megasas: PD SCSI req 0x13bb1c0 cmd 0x7fb7cebd53e0 lun 0x139b9f0 finished with status 0 len 16
megasas: Complete frame context 0 tail 0 busy 0 doorbell 0

I am not sure if this is related, but this seems like it could be something
worth investigating.

Also forceed fw_sge=8 and fw_cmds=1000 in megasas_scsi_init() to follow the
defaults with the working v0.12.5, the again, the MMIO writes and reads
up until the first 16-byte INQUIRY do appear to be identical AFAICT.

Here is the full log for the megasas.v3 -> megasas-upstream-v1 code:

http://linux-iscsi.org/builds/megasas-emulation-logs/win764-bit-megasas-v3.txt

and the working v0.12.5 boot:

http://linux-iscsi.org/builds/megasas-emulation-logs/win764-bit-megasas-v1.txt


> But that's a good hint, I'll be updating my tree and see how far
> I'll progress.
> 
> > But the good news is that WinXP SP2 is now working via scsi-generic ->
> > TCM_Loop in megasas.v3, and even w/o the original sync ioctl patch we
> > required  in v0.12.5 megasas code.  Very excellent work Hannes!
> > 
> > So, I will be merging the latest changes from megasas.v3 -> megasas-upstream-v1
> > shortly and retesting with 64-bit Build 7600.
> > 
> Cool. Thanks. I'll be rebasing my patches, too. I guess it's time
> for megasas.v4.
> 

Sounds good, and please let me know if you have any other ideas or would
like me to test something else.

Thanks Hannes!

--nab