Re: md-linear accidental(?) removal, removed significant(?) use case?

[Roman Mamedov <rm@romanrm.net>]

On Wed, 1 Jan 2025 20:14:12 -0800
Allen Toureg <thetanix@gmail.com> wrote:

> Preamble: I have extensive arrays of drives and due to their irregular
> sizes, I was using md-linear as a convenient way to manually
> concatenate arrays of underlying MD (raid5/etc) to manually deal with
> redundancy.
> 
> I have probably a few thousand TB in total raw space, and hundreds of
> TB of actual data and files attached to singular systems.
> 
> In a recent OS update, I discovered my larger volumes no longer mount
> because md-linear has been removed entirely from the kernel as of 6.8.
> 
> I am trying to do rationale archaeology. I sent a note to Mr. Neil
> Brown who was responsible for the earliest change I found related to
> this and he suggested I email regressions and linux-raid along with
> the current maintainers about it.
> 
> What I've been able to find:
> 
> In 2604b703b6b3db80e3c75ce472a54dfd0b7bf9f4 (2006) Neil Brown marked a
> MODULE_ALIAS entry for md-personality-1 as deprecated but it appears
> the reason was because the form of the personality was changed (not
> that the underlying md-linear itself was deprecated.)
> 
> d9d166c2a9d5d01af34396793950aa695883eed4 (2006) reinforced this change
> via a diff algorithm that overzealously included that line in a diff
> chunk but which makes annotating prior to it a more manual process.
> 
> 608f52e30aae7dc8da836e5b7b112d50a2d00e43 (2021) marked md-linear as
> deprecated in Kconfig, using the rationale that md-linear was
> deprecated in MODULE_ALIAS—but again which doesn't explain why the
> *module* was deprecated and appears to me at least to accidentally
> misconstrue the original reason for the deprecation comment.
> 
> 849d18e27be9a1253f2318cb4549cc857219d991 (2023) eliminated md-linear
> entirely, again mostly self-referencing a deprecation notice which was
> there in actuality for basically multiple decades and seems to have
> referenced something else entirely.
> 
> I was hoping you could help me understand why this module was removed?
> I have found others who also are running into this. Functionality they
> relied on has disappeared, as per the existence of the following:
> 
> https://github.com/dougvj/md-linear
> 
> https://gitlab.archlinux.org/archlinux/packaging/packages/linux/-/issues/34
> https://bbs.archlinux.org/viewtopic.php?id=294005
> (etc)
> 
> So, it looks like there are many of us who were still using mdadm to
> manage sub-device concatenation, again in my case for 100s of TB of
> admittedly casual data storage, and I can't currently find what the
> actual actual rationale was for removing it. :(
> 
> For utility's sake, I would like to suggest that linear volumes lessen
> problems like substriping. I do not think for many of us that
> shuffling around a few hundred TB is easy to do at any rate. Currently
> I'm manually re-compiling a fairly heavily-modified md-linear as a
> user-built module and it seems to work okay. I am definitely not the
> only one doing this.
> 
> Please consider resurrecting md-linear. :-)

I fully support keeping md-linear for users with existing deployments.

Wanted to only ask out of curiosity, did you try using md-raid0 for the same
scenario?

It can use different sized devices in RAID0. In case of two disks it will
stripe between them over the matching portion of the sizes, and then the tail
of the larger device will be accessed in a linear fashion. Not sure it can
handle 3 or more in this manner, will there be multiple steps of the striping,
each time with a smaller number of the remaining larger devices (but would not
be surprised if yes).

Given that a loss of one device in md-linear likely means complete data loss
anyway (relying on picking up pieces with data recovery tools is not a good
plan), seems like using md-raid0 here instead would have no downsides but
likely improve performance by a lot.

Aside from all that, the "industry" way to do the task of md-linear currently
would be a large LVM LV over a set of multiple PVs in a volume group.

-- 
With respect,
Roman