Re: [RFC] SCSI target for IBM Power5 LPAR

Re: [RFC] SCSI target for IBM Power5 LPAR

[Anton Blanchard]

Hi Dave,

> This device driver provides the SCSI target side of the "virtual
> SCSI" on IBM Power5 systems.  The initiator side has been in mainline
> for a while now (drivers/scsi/ibmvscsi/ibmvscsi.c.)  Targets already
> exist for AIX and OS/400.

Good stuff. Got a couple of small suggestions.

+/* Allocate a buffer with a dma_address.  Don't use dma_alloc_coherent
+ * since that uses GFP_ATOMIC internally and we can tollerate a delay
+ */
+static void *alloc_coherent_buffer(struct server_adapter *adapter, size_t size,
+				   dma_addr_t *dma_handle)
+{
+	void *buffer = kmalloc(size, GFP_KERNEL);
+
+	if (buffer) {
+		*dma_handle = dma_map_single(adapter->dev, buffer, size,
+					     DMA_BIDIRECTIONAL);
+
+		if (dma_mapping_error(*dma_handle)) {
+			kfree(buffer);
+			buffer = NULL;
+		}
+	}
+
+	return buffer;
+}

This should be fixed in mainline, on ppc64 we no longer build the dma_*
ops on top of the pci_* ops. This means we actually look at the flags :)

+	adapter->max_sectors = MAX_SECTORS;

Does this mean we are limited to 128kB transfers? Would it be OK to
bump the default?

Anton

Re: [RFC] SCSI target for IBM Power5 LPAR

[Christoph Hellwig]

On Tue, Sep 06, 2005 at 04:28:01PM -0500, Dave C Boutcher wrote:
> This device driver provides the SCSI target side of the "virtual
> SCSI" on IBM Power5 systems.  The initiator side has been in mainline
> for a while now (drivers/scsi/ibmvscsi/ibmvscsi.c.)  Targets already
> exist for AIX and OS/400.

Please try to integrate that with the generic scsi target framework at
http://developer.berlios.de/projects/stgt/.

Re: [RFC] SCSI target for IBM Power5 LPAR

[Dave C Boutcher]

On Wed, Sep 07, 2005 at 12:49:32PM +0200, Christoph Hellwig wrote:
> On Tue, Sep 06, 2005 at 04:28:01PM -0500, Dave C Boutcher wrote:
> > This device driver provides the SCSI target side of the "virtual
> > SCSI" on IBM Power5 systems.  The initiator side has been in mainline
> > for a while now (drivers/scsi/ibmvscsi/ibmvscsi.c.)  Targets already
> > exist for AIX and OS/400.
> 
> Please try to integrate that with the generic scsi target framework at
> http://developer.berlios.de/projects/stgt/.

There hasn't been a lot of forward progress on stgt in over a year, and
there were some issues (lack of scatterlist support, synchronous and
serial command execution) that were an issue when last I looked.

Vlad, can you comment on the state of stgt and whether you see it
being ready for mainline any time soon?

-- 
Dave Boutcher

Re: [RFC] SCSI target for IBM Power5 LPAR

[FUJITA Tomonori]

From: Dave C Boutcher <sleddog@us.ibm.com>
Subject: Re: [RFC] SCSI target for IBM Power5 LPAR
Date: Wed, 7 Sep 2005 07:45:04 -0500

> On Wed, Sep 07, 2005 at 12:49:32PM +0200, Christoph Hellwig wrote:
> > On Tue, Sep 06, 2005 at 04:28:01PM -0500, Dave C Boutcher wrote:
> > > This device driver provides the SCSI target side of the "virtual
> > > SCSI" on IBM Power5 systems.  The initiator side has been in mainline
> > > for a while now (drivers/scsi/ibmvscsi/ibmvscsi.c.)  Targets already
> > > exist for AIX and OS/400.
> > 
> > Please try to integrate that with the generic scsi target framework at
> > http://developer.berlios.de/projects/stgt/.
> 
> There hasn't been a lot of forward progress on stgt in over a year, and
> there were some issues (lack of scatterlist support, synchronous and
> serial command execution) that were an issue when last I looked.

The generic scsi target framework (stgt) is not SCST (that Vlad has
maintained). It is the project that Mike Christie and I started last
month. We discussed it with Christoph and decided that it would be
better to start from scratch because of the design differences.

Re: [RFC] SCSI target for IBM Power5 LPAR

[Santiago Leon]

Hi Anton,

> +	adapter->max_sectors = MAX_SECTORS;
> 
> Does this mean we are limited to 128kB transfers? Would it be OK to
> bump the default?

We use MAX_SECTORS (which is actually 127.5kB) because that's the 
max_sectors of the loopback device (we have a lot of users that like the 
flexibility of using loopback with the ibmvscsis driver)... It can be 
bumped up without any problems because there is code that splits 
requests if they are larger than the target's max_sectors...

What would you recommend? 256kB?

-- 
Santiago A. Leon
Power Linux Development
IBM Linux Technology Center

Re: [RFC] SCSI target for IBM Power5 LPAR

[Mike Christie]

Dave C Boutcher wrote:
> On Wed, Sep 07, 2005 at 12:49:32PM +0200, Christoph Hellwig wrote:
> 
>>On Tue, Sep 06, 2005 at 04:28:01PM -0500, Dave C Boutcher wrote:
>>
>>>This device driver provides the SCSI target side of the "virtual
>>>SCSI" on IBM Power5 systems.  The initiator side has been in mainline
>>>for a while now (drivers/scsi/ibmvscsi/ibmvscsi.c.)  Targets already
>>>exist for AIX and OS/400.
>>
>>Please try to integrate that with the generic scsi target framework at
>>http://developer.berlios.de/projects/stgt/.
> 
> 
> There hasn't been a lot of forward progress on stgt in over a year, and
> there were some issues (lack of scatterlist support, synchronous and
> serial command execution) that were an issue when last I looked.
> 

stgt is not scst. It is new. And yes we will have proper support for 
sccatterlists. We need something similar to what sg and st need so I 
plan on using the block layer functions that are implemented for those 
upper layer drivers for stgt. If you want to help out and implement that 
then that would be good. We are also very early in our design so all 
comments are welcome.

Re: [RFC] SCSI target for IBM Power5 LPAR

[Mike Christie]

FUJITA Tomonori wrote:
> month. We discussed it with Christoph and decided that it would be
> better to start from scratch because of the design differences.

Some of the things we are trying to improve upon are things that are 
better supported in 2.6. Some differences:

- We will support controller hotplug.
- We allow any type of device (dm, scsi, ide, LVM@ etc) as storage. And 
we do not want hook into SCSI-ml's upper layer drivers and deal with 
that refcounting if we can help it so we push a lot of code to userspace 
and only do reads and writes in the kernel.
- We also hope to support any block layer target.
- As mentioned before, scatterlists by using the block layer's support.

There may be more that I am forgetting, but originally we started out by 
trying to clean up the SCST code for 2.6 and make it resemble SCSI-ml's 
hotplug model. As we did this it looked like some code could live in 
userspace and it would end up being a rewrite becuase there was so much 
to do so we started a new project. We hope to work with Vlad and the 
SCST developers.

Re: [RFC] SCSI target for IBM Power5 LPAR

[Vladislav Bolkhovitin]

Dave C Boutcher wrote:
> On Wed, Sep 07, 2005 at 12:49:32PM +0200, Christoph Hellwig wrote:
> 
>>On Tue, Sep 06, 2005 at 04:28:01PM -0500, Dave C Boutcher wrote:
>>
>>>This device driver provides the SCSI target side of the "virtual
>>>SCSI" on IBM Power5 systems.  The initiator side has been in mainline
>>>for a while now (drivers/scsi/ibmvscsi/ibmvscsi.c.)  Targets already
>>>exist for AIX and OS/400.
>>
>>Please try to integrate that with the generic scsi target framework at
>>http://developer.berlios.de/projects/stgt/.
> 
> 
> There hasn't been a lot of forward progress on stgt in over a year, and
> there were some issues (lack of scatterlist support, synchronous and
> serial command execution) that were an issue when last I looked.
> 
> Vlad, can you comment on the state of stgt and whether you see it
> being ready for mainline any time soon?

Sorry, I can see on stgt page only mail lists archive and not from start 
(from Aug 22). Mike, can I see stgt code and some design description, 
please? You can send it directly on my e-mail address, if necessary.

Vlad

Re: [RFC] SCSI target for IBM Power5 LPAR

[Mike Christie]

Vladislav Bolkhovitin wrote:
> Dave C Boutcher wrote:
> 
>> On Wed, Sep 07, 2005 at 12:49:32PM +0200, Christoph Hellwig wrote:
>>
>>> On Tue, Sep 06, 2005 at 04:28:01PM -0500, Dave C Boutcher wrote:
>>>
>>>> This device driver provides the SCSI target side of the "virtual
>>>> SCSI" on IBM Power5 systems.  The initiator side has been in mainline
>>>> for a while now (drivers/scsi/ibmvscsi/ibmvscsi.c.)  Targets already
>>>> exist for AIX and OS/400.
>>>
>>>
>>> Please try to integrate that with the generic scsi target framework at
>>> http://developer.berlios.de/projects/stgt/.
>>
>>
>>
>> There hasn't been a lot of forward progress on stgt in over a year, and
>> there were some issues (lack of scatterlist support, synchronous and
>> serial command execution) that were an issue when last I looked.
>>
>> Vlad, can you comment on the state of stgt and whether you see it
>> being ready for mainline any time soon?
> 
> 
> Sorry, I can see on stgt page only mail lists archive and not from start 
> (from Aug 22). Mike, can I see stgt code and some design description, 
> please? You can send it directly on my e-mail address, if necessary.


goto the svn page for the code
http://developer.berlios.de/svn/?group_id=4492

As for design desc, I do not have anything. It is the evolving source :) 
We are slowly merging leasons we learned from open-iscsi, your SCST 
code, the available software and HW targets, and the SCSI ULD's 
scatterlist code which needs redoing so it is a bit of a mess.

Re: [RFC] SCSI target for IBM Power5 LPAR/SCST 0.9.3-pre1 published

[Vladislav Bolkhovitin]

Mike Christie wrote:
> Vladislav Bolkhovitin wrote:
>> Sorry, I can see on stgt page only mail lists archive and not from 
>> start (from Aug 22). Mike, can I see stgt code and some design 
>> description, please? You can send it directly on my e-mail address, if 
>> necessary.
> 
> goto the svn page for the code
> http://developer.berlios.de/svn/?group_id=4492
> 
> As for design desc, I do not have anything. It is the evolving source :) 
> We are slowly merging leasons we learned from open-iscsi, your SCST 
> code, the available software and HW targets, and the SCSI ULD's 
> scatterlist code which needs redoing so it is a bit of a mess.

OK, thanks, will try tomorrow.

I put SCST 0.9.3-pre1 on its page 
(http://sourceforge.net/projects/scst/). This is not the latest, but 
this is the one, which working. At the end of this week I'll try to put 
there the latest one as well. Hope, you will learn some more lessons 
from it :).

Any comments are welcome.

Vlad

Re: [RFC] SCSI target for IBM Power5 LPAR

[Anton Blanchard]

Hi,

> We use MAX_SECTORS (which is actually 127.5kB) because that's the 
> max_sectors of the loopback device (we have a lot of users that like the 
> flexibility of using loopback with the ibmvscsis driver)... It can be 
> bumped up without any problems because there is code that splits 
> requests if they are larger than the target's max_sectors...
> 
> What would you recommend? 256kB?

~256kB sounds reasonable to me and I notice that other VIO server is
setting it to 256kB :) 

Anton