From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=46966 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1OMIJJ-0008FK-7R for qemu-devel@nongnu.org; Wed, 09 Jun 2010 06:15:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1OMIJE-0007Sx-MP for qemu-devel@nongnu.org; Wed, 09 Jun 2010 06:15:01 -0400 Received: from smtp128.sbc.mail.sp1.yahoo.com ([69.147.65.187]:30428) by eggs.gnu.org with smtp (Exim 4.69) (envelope-from ) id 1OMIJE-0007RS-Fa for qemu-devel@nongnu.org; Wed, 09 Jun 2010 06:14:56 -0400 From: "Nicholas A. Bellinger" In-Reply-To: <20100608141506.DC0A52AC5B@ochil.suse.de> References: <20100608141506.DC0A52AC5B@ochil.suse.de> Content-Type: text/plain Date: Wed, 09 Jun 2010 03:14:48 -0700 Message-Id: <1276078488.18353.28.camel@haakon2.linux-iscsi.org> Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: [Qemu-devel] Re: [PATCH] megasas: Update to version 1.01 List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Hannes Reinecke Cc: qemu-devel On Tue, 2010-06-08 at 16:15 +0200, Hannes Reinecke wrote: > This patch updates the megasas HBA emulation to version 1.01. > It fixes the following issues: > > - Remove hand-crafted inquiry command > - Remove bounce-buffer for direct commands > - Implements qdev properties to set 'max_sge', 'max_cmds'. > - Implement JBOD mode > - Improve direct command handling > - Minor cleanups > > Signed-off-by: Hannes Reinecke > Hi Hannes, I applied your changes and everything looks good with the exception of the new MEGASAS_DEFAULT_SGE=80 setting.. > diff --git a/hw/megasas.c b/hw/megasas.c > index 250c3fb..19569a8 100644 > --- a/hw/megasas.c > +++ b/hw/megasas.c > @@ -40,38 +40,17 @@ do { fprintf(stderr, "megasas: error: " fmt , ## __VA_ARGS__);} while (0) > #endif > > /* Static definitions */ > -#define MEGASAS_MAX_FRAMES 1000 > -#define MEGASAS_MAX_SGE 8 > +#define MEGASAS_VERSION "1.01" > +#define MEGASAS_MAX_FRAMES 2048 /* Firmware limit at 65535 */ > +#define MEGASAS_DEFAULT_FRAMES 1000 /* Windows requires this */ > +#define MEGASAS_MAX_SGE 255 /* Firmware limit */ > +#define MEGASAS_DEFAULT_SGE 80 Ok, I have been running some LTP disktest raw bandwith benchmarks with a 256K blocksize with megasas -> TCM_Loop -> TCM/RAMDISK_DR LUNs into a v2.6.26 x86_64 Linux guest (4 VCPUs and 2048 memory) and I noticed something interesting.. With the new MEGASAS_DEFAULT_SGE 80 setting for fw_sge, read/write tests have dropped from the original ~1050 MB/sec to roughly ~400 MB/sec. Passing in the new qdev option using the old default of max_sge=8 the speed jumps back up to the range that where previously observed w/o this patch. Going a bit further, using max_sge=16 jumps up bandwith up to ~1600 MB/sec, and max_sge=24 takes it up to ~2200 MB/sec..! Using max_sge=32 then sharply drops back to ~800 MB/sec, and increasing to larger values brings bandwith down lower and lower.. Taking a look at the megaraid_sas LLD in the KVM guest, the struct scsi_host is being registered with sg_tablesize=28 which appears to be where the sharp dropoff for max_sge > 28 begins to occur. I see that MFI_DCMD_CTRL_GET_INFO is returning the configured fw_sge to the guest, but AFAICT megaraid_sas does not adjust itself to use the larger value reported by GET_INFO. So that said, I think we want to use MEGASAS_DEFAULT_SGE 28 to match what the Linux driver is using. I have not checked what the equivlient sg_tablesize for the MSFT LLD is doing, but it appears we need to error on the conserative side here. What do you think..? Best, --nab