Re: [PATCH 25/26] block: Reduce zone write plugging memory usage

Damien Le Moal
2024-02-12T01:09:09Z

On 2/11/24 12:40, Bart Van Assche wrote:
> On 2/9/24 16:06, Damien Le Moal wrote:
>> On 2/10/24 04:36, Bart Van Assche wrote:
>>> written zones is typically less than 10. Hence, tracking the partially written
>>
>> That is far from guaranteed, especially with devices that have no active zone
>> limits like SMR drives.
>
> Interesting. The zoned devices I'm working with try to keep data in memory
> for all zones that are neither empty nor full and hence impose an upper limit
> on the number of open zones.
>
>> But in any case, what exactly is your idea here ? Can you actually suggest
>> something ? Are you suggesting that a sparse array of zone plugs be used, with
>> an rb-tree or an xarray ? If that is what you are thinking, I can already tell
>> you that this is the first thing I tried to do. Early versions of this work used
>> a sparse xarray of zone plugs. But the problem with such approach is that it is
>> a lot more complicated and there is a need for a single lock to manage that
>> structure (which is really not good for performance).
>
> Hmm ... since the xarray data structure supports RCU I think that locking the
> entire xarray is only required if the zone condition changes from empty into
> not empty or from neither empty nor full into full?

I will try to revisit this. But again, that could be an incremental change on
top of this series...

> For the use cases I'm interested in a hash table implementation that supports
> RCU-lookups probably will work better than an xarray. I think that the hash
> table implementation in <linux/hashtable.h> supports RCU for lookups, insertion
> and removal.

It does, but the API for it is not the easiest, and I do not see how that could
be faster than an xarray, especially as the number of zones grows with high
capacity devices (read: potentially more collisions which will slow zone plug
lookups).

--
Damien Le Moal
Western Digital Research