[lustre-devel] HSM phase 2 HLD / Roadmap?

[LEIBOVICI Thomas <thomas.leibovici@cea.fr>]

On 06/16/15 18:15, Dilger, Andreas wrote:
> On 2015/06/16, 9:28 AM, "Patrick Farrell" <paf@cray.com> wrote:
>
>> Andreas,
>>
>> Could you expound on "Better integration with DNE"?  I'm not immediately
>> clear on what that might mean.
> I was thinking that the current DNE support is still using a single RBH
> database fed by multiple ChangeLogs (one from each MDT), which doesn't
> scale very well.  That said, I guess it would be possible to have a single
> parallel MySQL database that scales horizontally, so the scaling would be
> independent of the number of MDTs.
>
> Maybe this one should be dropped from the list, or just be considered part
> of the "optimize RBH database".
>
> Cheers, Andreas
AFAIK, there are other aspects of DNE that HSM wouldn't support:
HSM does not currently support striped directories (DNE phase 2)
in particular the volatile file used for "restore" operations wouldn't 
be properly located on the right MDT.

Thomas

>> Thanks,
>> - Patrick
>> On 06/16/2015 10:21 AM, Dilger, Andreas wrote:
>>> I think there are a few features that would be of interest for HSM
>>> improvement.
>>>
>>> * optimization of RobinHood database/POSIX copytool
>>> * better integration with DNE
>>> * partial file restore using composite files
>>>
>>> (http://wiki.opensfs.org/images/e/e4/LayoutEnhancementDesign_HighLevelDes
>>> ig
>>> n.pdf)
>>>      * change HSM to use LOV EA layout instead of dedicated xattr
>>>      * allowing a larger HSM archive ID (e.g. 128-bit UUID) so it can be
>>>        used as the primary archive identifier instead of Lustre FID
>>> * archive/restore of directory trees from the MDT as tarballs
>>>      * allow release/restore of space from the MDT
>>>      * would be needed for handling Data-on-MDT files
>>>
>>> Note that we don't have any plans to work on this, just listing some
>>> ideas
>>> percolating in my brain.
>>>
>>>
>>> Cheers, Andreas
>> _______________________________________________
>> lustre-devel mailing list
>> lustre-devel at lists.lustre.org
>> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
>>
>
> Cheers, Andreas