Re: [LSF/MM/BPF TOPIC] Lustre filesystem upstreaming

["Theodore Ts'o" <tytso@mit.edu>]

On Thu, Jan 30, 2025 at 04:18:29PM +0000, Day, Timothy wrote:
> 
> Lustre has a lot of usage and development outside of DDN/Whamcloud
> [1][2].  HPE, AWS, SuSe, Azure, etc. And at least at AWS, we use
> Lustre on fairly up-to-date kernels [3][4]. And I think this is
> becoming more common - although I don't have solid data on that.

I agree that I am seeing more use/interest of Lustre in various Cloud
deployments, and to the extent that Cloud clients tend to use newer
Linux kernels (e.g., commonly, the the LTS from the year before) that
certainly does make them use kernels newer than a typical RHEL kernel.

It's probably inherent in the nature of cluster file systems that they
won't be of interest for home users who aren't going to be paying the
cost of a dozen or so Cloud VM's being up on a more-or-less continuous
basis.  However, the reality is that more likely than not, developers
who are most likely to be using the latest upstream kernel, or maybe
even Linux-next, are not going to be using cloud VM's.

> And if you have dedicated hardware - setting up a small filesystem over
> TCP/IP isn't much harder than an NFS server IMHO. Just a mkfs and
> mount per storage target. With a single MDS and OSS, you only need two
> disks. So I think we have everything we need to enable upstream
> users/devs to use Lustre without too much hassle. I think it's mostly a
> matter of documentation and scripting.

Hmm... would it be possible to set up a simple toy Lustre file system
using a single system running in qemu --- i.e., using something like a
kvm-xfstests[1] test appliance?  TCP/IP over loopback might be
interesting, if it's posssible to run the Lustre MDS, OSS, and client
on the same kernel.  This would make repro testing a whole lot easier,
if all someone had to do was run the command "kvm-xfstests -c lustre smoke".

[1] https://github.com/tytso/xfstests-bld/blob/master/Documentation/kvm-quickstart.md

							- Ted