Linux and DMAPI

Linux and DMAPI

[Diego Woitasen]

Hi,
 I'm looking for software that supports DMAPI on Linux. I haven't
found anything on the web. I need to implement HSM for a mail server,
to store old messages in a secondary storage.

 Are there something implemented? Open source or propietary one? I
prefer open source obviously...

Regards,
 Diego

-- 
Diego Woitasen

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Linux and DMAPI

[Stan Hoeppner]

Diego Woitasen put forth on 9/18/2010 7:08 PM:
> Hi,
>  I'm looking for software that supports DMAPI on Linux. I haven't
> found anything on the web. I need to implement HSM for a mail server,
> to store old messages in a secondary storage.
> 
>  Are there something implemented? Open source or propietary one? I
> prefer open source obviously...

This may not be exactly what you're looking for, but Dovecot natively
supports two level HSM with the dbox/mdbox mail storage formats.

http://wiki2.dovecot.org/MailboxFormat/dbox

Bottom of the page, "Alternate storage".  See also:
http://wiki2.dovecot.org/Tools/Doveadm/Altmove

So basically you'd cron a script for some duration (daily, weekly,
monthly, etc) containing the above command to move old mails to your
alternate near line storage.

If it's a really busy server you may want to go with the individual user
route with one HSM script per user.  This would allow staggered
scheduling of the cron jobs so as to lessen the load on the server
compared to one cron job which does all users at once.

-- 
Stan

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Linux and DMAPI

[Christoph Hellwig]

On Sat, Sep 18, 2010 at 09:08:16PM -0300, Diego Woitasen wrote:
> Hi,
>  I'm looking for software that supports DMAPI on Linux. I haven't
> found anything on the web. I need to implement HSM for a mail server,
> to store old messages in a secondary storage.
> 
>  Are there something implemented? Open source or propietary one? I
> prefer open source obviously...

The XFS on disk format supports the required metadata for DMAPI, but the
implementation for it never made it into the mainline Linux kernel.  For
years SGI provided a tree to add the support, but for the last few years
the support bitrotted heavily.  Alex promised to publush a tree with
updated DMAPI support in the near future.

You'll still need additional userspace support the to actually driver
the migration.  The xfstests suite has a minimal implementation for QA
purposes as free software, but I don't think that's what your looking
for.  The main user of the XFS DMAPI support is SGI's DMF product,
and at least a few years ago IBM's Tivoli Storage Manager also supported
DMAPI.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Linux and DMAPI

[Björn JACKE]

On 2010-09-18 at 21:08 -0300 Diego Woitasen sent off:
> Hi,
>  I'm looking for software that supports DMAPI on Linux. I haven't
> found anything on the web. I need to implement HSM for a mail server,
> to store old messages in a secondary storage.
> 
>  Are there something implemented? Open source or propietary one? I
> prefer open source obviously...

unmaintained since years but might be interesting for you:
http://sourceforge.net/projects/openhsm/

The only Linux distri I know that contains XFS with DMAPI support is
SLES 8/9/10 (not sure about 11).

Björn
-- 
SerNet GmbH, Bahnhofsallee 1b, 37081 Göttingen
phone: +49-551-370000-0, fax: +49-551-370000-9
AG Göttingen, HRB 2816, GF: Dr. Johannes Loxen

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Linux and DMAPI

[Timothy Sesow]

In the latest tarball at http://dvdvault.sourceforge.net is a
routine called "dmapid" that has a user-space implementation
of a DMAPI service routine, plus a copy of some example source
code from SGI.  Plus there is a "purgefile" program to poke holes
in files in the "migration" directory.  The most recent upload was last 
month.

The tricky parts of session management and event handling are in
the dmapid code.

Its been run against a kernel build from SGI's XFS baseline
(most recently on 2.6.24).  You will either have to go back to
a prior kernel build to get DMAPI included, or run a patch against
a current version of the LINUX kernel.


On 11/03/2010 03:44 PM, Björn JACKE wrote:
> On 2010-09-18 at 21:08 -0300 Diego Woitasen sent off:
>> Hi,
>>   I'm looking for software that supports DMAPI on Linux. I haven't
>> found anything on the web. I need to implement HSM for a mail server,
>> to store old messages in a secondary storage.
>>
>>   Are there something implemented? Open source or propietary one? I
>> prefer open source obviously...
> unmaintained since years but might be interesting for you:
> http://sourceforge.net/projects/openhsm/
>
> The only Linux distri I know that contains XFS with DMAPI support is
> SLES 8/9/10 (not sure about 11).
>
> Björn

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Linux and DMAPI

[Alex Elder]

On Sun, 2010-10-03 at 02:18 -0400, Christoph Hellwig wrote:
> On Sat, Sep 18, 2010 at 09:08:16PM -0300, Diego Woitasen wrote:
> > Hi,
> >  I'm looking for software that supports DMAPI on Linux. I haven't
> > found anything on the web. I need to implement HSM for a mail server,
> > to store old messages in a secondary storage.
> > 
> >  Are there something implemented? Open source or propietary one? I
> > prefer open source obviously...
> 
> The XFS on disk format supports the required metadata for DMAPI, but the
> implementation for it never made it into the mainline Linux kernel.  For
> years SGI provided a tree to add the support, but for the last few years
> the support bitrotted heavily.  Alex promised to publush a tree with
> updated DMAPI support in the near future.

(Catching up on old mail, sorry I didn't respond earlier.)
I will keep my promise...  I have it ported to the
2.6.32-stable and 2.6.33-stable branches at this point.
I was going to try to get up-to-date with the newest
kernels first, but am now setting my sights on getting
just what I have so far published.  I will send an
announcement when it's available.
 
> You'll still need additional userspace support the to actually driver
> the migration.  The xfstests suite has a minimal implementation for QA
> purposes as free software, but I don't think that's what your looking
> for.  The main user of the XFS DMAPI support is SGI's DMF product,
> and at least a few years ago IBM's Tivoli Storage Manager also supported
> DMAPI.

As Christoph said, SGI offers DMF (Data Migration Facility),
but it is a proprietary product.

					-Alex


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs