Re: [RFC PATCH 0/4] scsi: use xarray for devices and targets

[Douglas Gilbert <dgilbert@interlog.com>]

On 2020-05-27 12:36 p.m., Bart Van Assche wrote:
> On 2020-05-27 07:13, Hannes Reinecke wrote:
>> Hi all,
>>
>> based on the ideas from Doug Gilbert here's now my take on using
>> xarrays for devices and targets.
>> It revolves around two ideas:
>> - 'channel' and 'id' are never ever used to the full 32 bit range;
>>    'channels' are well below 10, and no driver is using more than
>>    16 bits for the id. So we can reduce the type of 'channel' and
>>    'id' to 16 bits, and use the 32 bit value 'channel << 16 | id'
>>    as the index into the target xarray.
>> - Most SCSI LUNs are below 256 (to ensure compability with older
>>    systems). So there we can use the LUN number as the index into
>>    the xarray; for larger LUN numbers we'll allocate a separate
>>    index.
>>
>> With these change we can implement an efficient lookup mechanism,
>> devolving into direct lookup for most cases.
>> And iteration should be as efficient as the current, list-based,
>> approach.
>>
>> This is compile-tested only, to give you an impression of the
>> overall idea and to get the discussion rolling.
> 
> Hi Hannes,
> 
> My understanding of the xarray concept is that it provides two
> advantages over using linked lists:
> - Faster lookups.
> - Requires less memory.

Bart,
You might add these:
   - sane deletion semantics
   - inbuilt locking
   - inherently safer iterations, especially if marks are used

Matthew can probably add more.

> Will we benefit from any of these advantages in the SCSI code? Hadn't
> James Bottomley already brought up that lookup by (channel, target, lun)
> only happens from some LLDs and from the procfs code?

The way that the SCSI object tree hangs together with doubly linked
lists may have been a coherent design 20 years ago when JB wrote it,
but it has been white-anted big time since then.

The current state of that code is hard to defend. I have between 10 and 20
more examples of patently stupid things the current code does. See my
exchange with JB concerning the starget iterator over its sdevs that decided
to check on _every_ sdev in that host. That is done in several places.

The redundant sdevs in a shost collection is probably the most glaring
current design flaw.

> Are there any use cases where the number of SCSI devices is large enough
> to benefit from the memory reduction?

I don't believe that overall memory usage is a problem. Fitting the sdev_s
of hot devices in a smaller number of cache lines would help. That is
where Ming Lei was looking that kicked off this exercise that has
morphed into using xarray.

Doug Gilbert