From mboxrd@z Thu Jan 1 00:00:00 1970 From: LEIBOVICI Thomas Date: Wed, 17 Jun 2015 10:08:28 +0200 Subject: [lustre-devel] HSM phase 2 HLD / Roadmap? In-Reply-To: References: <55803C06.4080003@cea.fr> <558040B5.2050609@cray.com> Message-ID: <55812AFC.1050405@cea.fr> List-Id: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: lustre-devel@lists.lustre.org On 06/16/15 18:15, Dilger, Andreas wrote: > On 2015/06/16, 9:28 AM, "Patrick Farrell" wrote: > >> Andreas, >> >> Could you expound on "Better integration with DNE"? I'm not immediately >> clear on what that might mean. > I was thinking that the current DNE support is still using a single RBH > database fed by multiple ChangeLogs (one from each MDT), which doesn't > scale very well. That said, I guess it would be possible to have a single > parallel MySQL database that scales horizontally, so the scaling would be > independent of the number of MDTs. > > Maybe this one should be dropped from the list, or just be considered part > of the "optimize RBH database". > > Cheers, Andreas AFAIK, there are other aspects of DNE that HSM wouldn't support: HSM does not currently support striped directories (DNE phase 2) in particular the volatile file used for "restore" operations wouldn't be properly located on the right MDT. Thomas >> Thanks, >> - Patrick >> On 06/16/2015 10:21 AM, Dilger, Andreas wrote: >>> I think there are a few features that would be of interest for HSM >>> improvement. >>> >>> * optimization of RobinHood database/POSIX copytool >>> * better integration with DNE >>> * partial file restore using composite files >>> >>> (http://wiki.opensfs.org/images/e/e4/LayoutEnhancementDesign_HighLevelDes >>> ig >>> n.pdf) >>> * change HSM to use LOV EA layout instead of dedicated xattr >>> * allowing a larger HSM archive ID (e.g. 128-bit UUID) so it can be >>> used as the primary archive identifier instead of Lustre FID >>> * archive/restore of directory trees from the MDT as tarballs >>> * allow release/restore of space from the MDT >>> * would be needed for handling Data-on-MDT files >>> >>> Note that we don't have any plans to work on this, just listing some >>> ideas >>> percolating in my brain. >>> >>> >>> Cheers, Andreas >> _______________________________________________ >> lustre-devel mailing list >> lustre-devel at lists.lustre.org >> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org >> > > Cheers, Andreas