Re: Linux tape drivers

Re: Linux tape drivers

[Andrew Morton]

(cc's added, with permission)

On Tue, 3 Apr 2007 15:08:37 +0200
Kern Sibbald <kern@sibbald.com> wrote:

> Hello,
> 
> I am the project manager for Bacula, an Open Source network backup program 
> that runs on all popular OSes.  After your presentation at FOSDEM in Febrary, 
> we briefly talked about Linux tape driver problems I am encountering, and you 
> offered to put me in touch with the appropriate kernel developers.
> 
> I would much appreciate any help in this.  Since the problems concern all tape 
> drivers, I provide a very brief outline of what my would like to discuss.  
> First, I must mention that the Linux SCSI driver works perfectly fine with 
> Bacula, it is simply a question of possible improvements, under item 2 below.
> 
> Issues for discussion:
> 
> 1. Bugs:
>    a. Other than the OSST driver, apparently no IDE/SATA tape driver works
>        with Bacula. I don't have such a drive (working on it), but from user
>        reports, it appears to me that there are problems of permitting 
>        variable length blocks, and more serious, when writing to the end of
>        the tape, either the logical end of tape indicator is ignored, or when
>        it is encountered, all further I/O is prohibited -- including a WEOF. 
>        This makes reliable writing of multiple reel tapes impossible.
> 
>        By the way, these IDE/SATA drives work with Bacula using the same
>        source code cross-compiled with GNU C++ on Linux, then run on Windows
>        machines, so it is most likely a driver issue rather than anything in
>        Bacula or the hardware.
> 
> 2. Usability of the current tape driver API (not bugs)
>   a. With the new O_NONBLOCK flag introduced in kernel 2.5.x, opening
>       a tape drive and finding out if a volume is mounted is much more 
>       complicated.  It is really inconvenient and required a lot more code
>       in prior kernels.  This should be an item for discussion.
>   b. There is no simple way to determine if a tape is in a drive -- it is 
>       at least 20 or 30 lines of C code to do it right.
>   c. There is a real lack of information about any error condition
>       (read/write). One can probably get it via direct SCSI commands, but
>       why not through an ioctl.
>   d. The same as c. above, but for all kinds of error information that 
>        can be important -- particularly soft errors.  An ioctl() that returns
>        additional information could be very useful.  Currently we get it by
>        using external SCSI programs such as mtx tapeinfo for TapeAlerts,
>        and smartclt for errors.
>    e. Apparently only root can do the following (IMO, this is a bug or 
>        programming oversight): 
> 
>          struct mtop mt;
>          mt.mt_op = MTSETDRVBUFFER;
>          mt.mt_count = MT_ST_CLEARBOOLEANS;
>          mt.mt_count |= MT_ST_FAST_MTEOM;
>          ...
>          ioctl(fd, MIOCTOP, &mt);
> 
>       Typically Bacula runs tape daemon as non-root.  Is there a good reason 
>        why program that has write permission on the device cannot do this
>        ioctl() and is it possible to change this?
> 
> There are probably a few other items that I have forgotten.  I consider #1 
> rather important (at least for Bacula).  Item #2 is something more long term 
> and some of my requests may have compatibility issues to consider.

I don't think that any particular individual maintains the tape code as a
whole, so progress will probably be slow, I'm afraid.

Kai maintains the scsi tape driver and Willem looks after OSST.  Perhaps they
can comment on some of the issues which you identify?

Thanks.

Re: Linux tape drivers

[Kern Sibbald]

On Tuesday 03 April 2007 22:39, Andrew Morton wrote:
> 
> (cc's added, with permission)
> 
> On Tue, 3 Apr 2007 15:08:37 +0200
> Kern Sibbald <kern@sibbald.com> wrote:
> 
> > Hello,
> > 
> > I am the project manager for Bacula, an Open Source network backup program 
> > that runs on all popular OSes.  After your presentation at FOSDEM in 
Febrary, 
> > we briefly talked about Linux tape driver problems I am encountering, and 
you 
> > offered to put me in touch with the appropriate kernel developers.
> > 
> > I would much appreciate any help in this.  Since the problems concern all 
tape 
> > drivers, I provide a very brief outline of what my would like to discuss.  
> > First, I must mention that the Linux SCSI driver works perfectly fine with 
> > Bacula, it is simply a question of possible improvements, under item 2 
below.
> > 
> > Issues for discussion:
> > 
> > 1. Bugs:
> >    a. Other than the OSST driver, apparently no IDE/SATA tape driver works
> >        with Bacula. I don't have such a drive (working on it), but from 
user
> >        reports, it appears to me that there are problems of permitting 
> >        variable length blocks, and more serious, when writing to the end 
of
> >        the tape, either the logical end of tape indicator is ignored, or 
when
> >        it is encountered, all further I/O is prohibited -- including a 
WEOF. 
> >        This makes reliable writing of multiple reel tapes impossible.
> > 
> >        By the way, these IDE/SATA drives work with Bacula using the same
> >        source code cross-compiled with GNU C++ on Linux, then run on 
Windows
> >        machines, so it is most likely a driver issue rather than anything 
in
> >        Bacula or the hardware.
> > 
> > 2. Usability of the current tape driver API (not bugs)
> >   a. With the new O_NONBLOCK flag introduced in kernel 2.5.x, opening
> >       a tape drive and finding out if a volume is mounted is much more 
> >       complicated.  It is really inconvenient and required a lot more code
> >       in prior kernels.  This should be an item for discussion.
> >   b. There is no simple way to determine if a tape is in a drive -- it is 
> >       at least 20 or 30 lines of C code to do it right.
> >   c. There is a real lack of information about any error condition
> >       (read/write). One can probably get it via direct SCSI commands, but
> >       why not through an ioctl.
> >   d. The same as c. above, but for all kinds of error information that 
> >        can be important -- particularly soft errors.  An ioctl() that 
returns
> >        additional information could be very useful.  Currently we get it 
by
> >        using external SCSI programs such as mtx tapeinfo for TapeAlerts,
> >        and smartclt for errors.
> >    e. Apparently only root can do the following (IMO, this is a bug or 
> >        programming oversight): 
> > 
> >          struct mtop mt;
> >          mt.mt_op = MTSETDRVBUFFER;
> >          mt.mt_count = MT_ST_CLEARBOOLEANS;
> >          mt.mt_count |= MT_ST_FAST_MTEOM;
> >          ...
> >          ioctl(fd, MIOCTOP, &mt);
> > 
> >       Typically Bacula runs tape daemon as non-root.  Is there a good 
reason 
> >        why program that has write permission on the device cannot do this
> >        ioctl() and is it possible to change this?
> > 
> > There are probably a few other items that I have forgotten.  I consider #1 
> > rather important (at least for Bacula).  Item #2 is something more long 
term 
> > and some of my requests may have compatibility issues to consider.
> 
> I don't think that any particular individual maintains the tape code as a
> whole, so progress will probably be slow, I'm afraid.

Thanks for the warning.  I'm familiar with such processes so don't expect it 
to go quickly.

> 
> Kai maintains the scsi tape driver and Willem looks after OSST.  Perhaps 
they
> can comment on some of the issues which you identify?

Thanks for pointing me to the correct places.  I will be interested to hear 
their thoughts ...

I have already had the pleasure to work with Willem, and thanks to his 
efforts, his OSST driver does work with Bacula.  I hope that he or some 
others would like to fix the other "broken" non-scsi drivers.  I'll get  the 
names of the drivers that do not work and report back.

Andrew, do you want to be copied on future email exchanges?

Does anyone object if I copy a Bacula developer who does have a 
non-scsi tape drive that does not work on Linux, but does work 
on Windows?

Re: Linux tape drivers

[Andrew Morton]

On Tue, 3 Apr 2007 23:30:15 +0200
Kern Sibbald <kern@sibbald.com> wrote:

> > 
> > Kai maintains the scsi tape driver and Willem looks after OSST.  Perhaps 
> they
> > can comment on some of the issues which you identify?
> 
> Thanks for pointing me to the correct places.  I will be interested to hear 
> their thoughts ...
> 
> I have already had the pleasure to work with Willem, and thanks to his 
> efforts, his OSST driver does work with Bacula.  I hope that he or some 
> others would like to fix the other "broken" non-scsi drivers.  I'll get  the 
> names of the drivers that do not work and report back.

OK.  The non-scsi drivers are more problematic.  If we're meaning
ide-tape.c here then I don't know of any developers who are really working
on that.  I guess it's nominally Bartlomiej Zolnierkiewicz, but I suspect
he's flat out looking after regular old disk dives.

> Andrew, do you want to be copied on future email exchanges?

Yes please.

> Does anyone object if I copy a Bacula developer who does have a 
> non-scsi tape drive that does not work on Linux, but does work 
> on Windows?

If it's purely a driver problem then it's presumably not relevant to the
scsi developers.  Raising a report at bugzilla.kernel.org would probably be
appropriate - I will then route it to the appropriate developer (if any)
and we'll at least know that we have an issue to be addressed.

Re: Linux tape drivers

[James Bottomley]

On Tue, 2007-04-03 at 14:57 -0700, Andrew Morton wrote:
> If it's purely a driver problem then it's presumably not relevant to the
> scsi developers.  Raising a report at bugzilla.kernel.org would probably be
> appropriate - I will then route it to the appropriate developer (if any)
> and we'll at least know that we have an issue to be addressed.

Actually, that does depend on the kernel.  If it's a SATAPI device, then
it will be using the st driver.  Likewise, if it's one of the new PATA
things moved under drivers/ata, it should also be using st ... it's only
the legacy PATA devices that use ide-tape now.

Do we know what the kernel and device (and IDE controller) are?

James

Re: Linux tape drivers

[Willem Riede]

On 4/3/07, Andrew Morton <akpm@linux-foundation.org> wrote:
> On Tue, 3 Apr 2007 23:30:15 +0200
> Kern Sibbald <kern@sibbald.com> wrote:
>
> > > Kai maintains the scsi tape driver and Willem looks after OSST.  Perhaps
> > > they can comment on some of the issues which you identify?
> >
> > Thanks for pointing me to the correct places.  I will be interested to hear
> > their thoughts ...
> >
> > I have already had the pleasure to work with Willem, and thanks to his
> > efforts, his OSST driver does work with Bacula.  I hope that he or some
> > others would like to fix the other "broken" non-scsi drivers.  I'll get  the
> > names of the drivers that do not work and report back.
>
> OK.  The non-scsi drivers are more problematic.  If we're meaning
> ide-tape.c here then I don't know of any developers who are really working
> on that.  I guess it's nominally Bartlomiej Zolnierkiewicz, but I suspect
> he's flat out looking after regular old disk dives.
>
> > Andrew, do you want to be copied on future email exchanges?
>
> Yes please.
>
> > Does anyone object if I copy a Bacula developer who does have a
> > non-scsi tape drive that does not work on Linux, but does work
> > on Windows?
>
> If it's purely a driver problem then it's presumably not relevant to the
> scsi developers.  Raising a report at bugzilla.kernel.org would probably be
> appropriate - I will then route it to the appropriate developer (if any)
> and we'll at least know that we have an issue to be addressed.

Osst supports both scsi and ide models of the Onstream dirves it is
designed for.
It uses ide-scsi for the ide model. In the past, I did look at
ide-tape to see if it could be made to work properly for the ide
Onstream tape, but that would have taken so much work, I decided to
limit myself to ide-scsi.

For other ide tape drives, the same could be done, using Kai's st
driver (of which osst is a clone). Ide-tape could be deprecated, left
to use for drives/applications it happens to work for. Given that the
new sata world is using st also, this concentrates effort where it has
maximum leverage.

If there are issues using Bacula on ide drives with ide-scsi and st,
we should debug those. If my previous experience getting osst to work
properly with Bacula can be of any use, I'll be happy to help where I
can.

Regards, Willem Riede.

Re: Linux tape drivers

[Kern Sibbald]

On Wednesday 04 April 2007 05:00, Willem Riede wrote:

> > Kern Sibbald <kern@sibbald.com> wrote:
...

Hello Willem,

> 
> Osst supports both scsi and ide models of the Onstream dirves it is
> designed for.
> It uses ide-scsi for the ide model. In the past, I did look at
> ide-tape to see if it could be made to work properly for the ide
> Onstream tape, but that would have taken so much work, I decided to
> limit myself to ide-scsi.
> 
> For other ide tape drives, the same could be done, using Kai's st
> driver (of which osst is a clone). Ide-tape could be deprecated, left
> to use for drives/applications it happens to work for. Given that the
> new sata world is using st also, this concentrates effort where it has
> maximum leverage.

 I'm all in favor of deprecating ide-tape if ide-scsi is a viable alternative, 
and if that is the only one you kernel guys would like to support. Let's 
concentrate on the drivers that will be the most important over the next few 
years (SATA?)

> 
> If there are issues using Bacula on ide drives with ide-scsi and st,
> we should debug those. If my previous experience getting osst to work
> properly with Bacula can be of any use, I'll be happy to help where I
> can.

Your previous experience could be particularly helpful. Thanks for the offer.

I'm in the process of taking a mini-Bacula survey of the problems. Here are 
the results for the moment:

===
Sebastiaan: He returned his Quantum DLT-V4 SATA drive, which did not work with 
Bacula, in exchange for a SCSI drive.

Bob: He has a SDX-260V ATAPI AIT-E and reports the following:
1. Sony SDX-260V
2. ATAPI
3. Tried both ide-scsi and ide-tape
4. ide-scsi simply locked up my machine after ANY attempt to write data
to the tape.  I tried various combinations of block sizes and such.
ide-tape would wirte to the tape and could read it but failed the
positioning tests under btape.  Specifically it would read any block
from the first file on the tape.  When a block from the second file was
requested, the machine would lock up.
5. Gentoo Linux 2.6.20 - gcc 4.1.2 - glibc 2.5 - paludis 0.22.2 - bacula
2.0.3

Robert: Reports:
A year ago I used the IDE drive on Linux.  If I remember
correctly the problem only occurred when using DMA, PIO worked but was very,
very slow.  The symptoms changed depending on whether I used the
SCSI driver stack or the ide-tape driver; but neither worked properly.
===

Please tell me the best way to proceed. I don't have the hardware, but 
understand Bacula, the users have the hardware, but don't understand Bacula's 
needs  (Robert knows Bacula in detail) -- not the best of situations.  I will 
buy the hardware if required, but I'd like to limit ($$$) to what is going to 
be mostly used in the future.


On the other general issues of the tape API (read/write/open/ioctl), I briefly 
sketched my problems/concerns.  If anyone is interested in any of those 
points, I can be a lot more specific.

Re: Linux tape drivers

[Andrew Morton]

On Wed, 4 Apr 2007 16:26:14 +0200 Kern Sibbald <kern@sibbald.com> wrote:

>  I'm all in favor of deprecating ide-tape if ide-scsi is a viable alternative, 

ow.  ide-scsi is in very bad shape and nobody is maintaining it or fixing
bugs in it or anything.  The only reason we retain ide-scsi at all is, err,
because some tape drives need it.

If we have no alternative to using ide-scsi, and if the tape controllers in
question are not some terribly obscure things which nobody would seriously
use then we have a bit of a problem.  I don't know of anyone who we can
turn to to get anything done in ide-scsi.

Re: Linux tape drivers

[Kern Sibbald]

On Wednesday 04 April 2007 20:55, Andrew Morton wrote:
> On Wed, 4 Apr 2007 16:26:14 +0200 Kern Sibbald <kern@sibbald.com> wrote:
> 
> >  I'm all in favor of deprecating ide-tape if ide-scsi is a viable 
alternative, 
> 
> ow.  ide-scsi is in very bad shape and nobody is maintaining it or fixing
> bugs in it or anything.  The only reason we retain ide-scsi at all is, err,
> because some tape drives need it.
> 
> If we have no alternative to using ide-scsi, and if the tape controllers in
> question are not some terribly obscure things which nobody would seriously
> use then we have a bit of a problem.  I don't know of anyone who we can
> turn to to get anything done in ide-scsi.
> 
>

That's quite clear, thanks.  I'll update our documentation to say that 
ide-scsi doesn't work with Bacula, is not actively supported and may be 
removed from the kernel.

What non-SCSI drivers do you feel are or should be well supported?
libata as Kai indicates?

Once I am sure, I'll point Bacula users to those drivers.  

Re: Linux tape drivers

[Andrew Morton]

On Wed, 4 Apr 2007 21:21:08 +0200
Kern Sibbald <kern@sibbald.com> wrote:

> On Wednesday 04 April 2007 20:55, Andrew Morton wrote:
> > On Wed, 4 Apr 2007 16:26:14 +0200 Kern Sibbald <kern@sibbald.com> wrote:
> > 
> > >  I'm all in favor of deprecating ide-tape if ide-scsi is a viable 
> alternative, 
> > 
> > ow.  ide-scsi is in very bad shape and nobody is maintaining it or fixing
> > bugs in it or anything.  The only reason we retain ide-scsi at all is, err,
> > because some tape drives need it.
> > 
> > If we have no alternative to using ide-scsi, and if the tape controllers in
> > question are not some terribly obscure things which nobody would seriously
> > use then we have a bit of a problem.  I don't know of anyone who we can
> > turn to to get anything done in ide-scsi.
> > 
> >
> 
> That's quite clear, thanks.  I'll update our documentation to say that 
> ide-scsi doesn't work with Bacula, is not actively supported and may be 
> removed from the kernel.

That seems like a good idea, I'm afraid.

> What non-SCSI drivers do you feel are or should be well supported?
> libata as Kai indicates?

I don't know this stuff well enough to be able to answer that usefully,
sorry.

> Once I am sure, I'll point Bacula users to those drivers.  

Re: Linux tape drivers

[Willem Riede]

On 4/4/07, Andrew Morton <akpm@linux-foundation.org> wrote:
> On Wed, 4 Apr 2007 16:26:14 +0200 Kern Sibbald <kern@sibbald.com> wrote:
>
> >  I'm all in favor of deprecating ide-tape if ide-scsi is a viable alternative,
>
> ow.  ide-scsi is in very bad shape and nobody is maintaining it or fixing
> bugs in it or anything.  The only reason we retain ide-scsi at all is, err,
> because some tape drives need it.
>
> If we have no alternative to using ide-scsi, and if the tape controllers in
> question are not some terribly obscure things which nobody would seriously
> use then we have a bit of a problem.  I don't know of anyone who we can
> turn to to get anything done in ide-scsi.

Andrew,

Kai suggests that libata should be able to take over from ide-scsi.
Do you concur? That may be a better way forward than ide-scsi.
I know, because I had a hard enough time with it when I needed
to fix it so it worked for osst, at least.

Regards, Willem Riede.

Re: Linux tape drivers

[Andrew Morton]

On Wed, 4 Apr 2007 12:22:35 -0700
"Willem Riede" <wrlk@riede.org> wrote:

> On 4/4/07, Andrew Morton <akpm@linux-foundation.org> wrote:
> > On Wed, 4 Apr 2007 16:26:14 +0200 Kern Sibbald <kern@sibbald.com> wrote:
> >
> > >  I'm all in favor of deprecating ide-tape if ide-scsi is a viable alternative,
> >
> > ow.  ide-scsi is in very bad shape and nobody is maintaining it or fixing
> > bugs in it or anything.  The only reason we retain ide-scsi at all is, err,
> > because some tape drives need it.
> >
> > If we have no alternative to using ide-scsi, and if the tape controllers in
> > question are not some terribly obscure things which nobody would seriously
> > use then we have a bit of a problem.  I don't know of anyone who we can
> > turn to to get anything done in ide-scsi.
> 
> Andrew,
> 
> Kai suggests that libata should be able to take over from ide-scsi.
> Do you concur?

That would be very nice if we can make it happen.  I don't know how close
we are to that, but the fine folks on linux-ide might be able tell us.

> That may be a better way forward than ide-scsi.
> I know, because I had a hard enough time with it when I needed
> to fix it so it worked for osst, at least.
> 
> Regards, Willem Riede.

Re: Linux tape drivers

[Bartlomiej Zolnierkiewicz]

On Wednesday 04 April 2007, Andrew Morton wrote:
> On Wed, 4 Apr 2007 12:22:35 -0700
> "Willem Riede" <wrlk@riede.org> wrote:
> 
> > On 4/4/07, Andrew Morton <akpm@linux-foundation.org> wrote:
> > > On Wed, 4 Apr 2007 16:26:14 +0200 Kern Sibbald <kern@sibbald.com> wrote:
> > >
> > > >  I'm all in favor of deprecating ide-tape if ide-scsi is a viable alternative,
> > >
> > > ow.  ide-scsi is in very bad shape and nobody is maintaining it or fixing
> > > bugs in it or anything.  The only reason we retain ide-scsi at all is, err,

I haven't heard about any major bugs besides well known module unload problem.

What have I missed?

> > > because some tape drives need it.
> > >
> > > If we have no alternative to using ide-scsi, and if the tape controllers in
> > > question are not some terribly obscure things which nobody would seriously
> > > use then we have a bit of a problem.  I don't know of anyone who we can
> > > turn to to get anything done in ide-scsi.
> > 
> > Andrew,
> > 
> > Kai suggests that libata should be able to take over from ide-scsi.
> > Do you concur?
> 
> That would be very nice if we can make it happen.  I don't know how close
> we are to that, but the fine folks on linux-ide might be able tell us.
> 
> > That may be a better way forward than ide-scsi.
> > I know, because I had a hard enough time with it when I needed
> > to fix it so it worked for osst, at least.
> > 
> > Regards, Willem Riede.

Re: Linux tape drivers

[Andrew Morton]

On Wed, 4 Apr 2007 23:31:06 +0200
Bartlomiej Zolnierkiewicz <bzolnier@gmail.com> wrote:

> On Wednesday 04 April 2007, Andrew Morton wrote:
> > On Wed, 4 Apr 2007 12:22:35 -0700
> > "Willem Riede" <wrlk@riede.org> wrote:
> > 
> > > On 4/4/07, Andrew Morton <akpm@linux-foundation.org> wrote:
> > > > On Wed, 4 Apr 2007 16:26:14 +0200 Kern Sibbald <kern@sibbald.com> wrote:
> > > >
> > > > >  I'm all in favor of deprecating ide-tape if ide-scsi is a viable alternative,
> > > >
> > > > ow.  ide-scsi is in very bad shape and nobody is maintaining it or fixing
> > > > bugs in it or anything.  The only reason we retain ide-scsi at all is, err,
> 
> I haven't heard about any major bugs besides well known module unload problem.
> 
> What have I missed?

oop, caught making unsubstantiatable assertions.

I've seen an ongoing dribble of doesn't-work and it-crashes reports and I
simply have not made any record of them, because it's ide-scsi and I don't
expect there's anything we can do about them :(

Often these reports are from people who are writing CDROMs and the usual
response is "use cdrecord dev=/dev/hdc".

I think Alan and Jens might have some thoughts on the ide-scsi status?

Re: Linux tape drivers

[Kai Makisara]

On Tue, 3 Apr 2007, Andrew Morton wrote:

> 
> (cc's added, with permission)
> 
> On Tue, 3 Apr 2007 15:08:37 +0200
> Kern Sibbald <kern@sibbald.com> wrote:
> 
> > Hello,
> > 
> > I am the project manager for Bacula, an Open Source network backup program 
> > that runs on all popular OSes.  After your presentation at FOSDEM in Febrary, 
> > we briefly talked about Linux tape driver problems I am encountering, and you 
> > offered to put me in touch with the appropriate kernel developers.
> > 
> > I would much appreciate any help in this.  Since the problems concern all tape 
> > drivers, I provide a very brief outline of what my would like to discuss.  
> > First, I must mention that the Linux SCSI driver works perfectly fine with 
> > Bacula, it is simply a question of possible improvements, under item 2 below.
> > 
> > Issues for discussion:
> > 
> > 1. Bugs:
> >    a. Other than the OSST driver, apparently no IDE/SATA tape driver works
> >        with Bacula. I don't have such a drive (working on it), but from user
> >        reports, it appears to me that there are problems of permitting 
> >        variable length blocks, and more serious, when writing to the end of
> >        the tape, either the logical end of tape indicator is ignored, or when
> >        it is encountered, all further I/O is prohibited -- including a WEOF. 
> >        This makes reliable writing of multiple reel tapes impossible.
> > 
> >        By the way, these IDE/SATA drives work with Bacula using the same
> >        source code cross-compiled with GNU C++ on Linux, then run on Windows
> >        machines, so it is most likely a driver issue rather than anything in
> >        Bacula or the hardware.
> > 

Others have already answered this and I agree with their view. All of the 
tape drives seem to use the SSC command set or something close to that. 
One high-level driver should be enough to implement the user semantics.
 
Libata should be able to drive the SATA/IDE drives using and the drives 
are visible as SCSI devices in Linux. In future there should be no real 
need for ide-scsi. Probably very few people have tried libata with tapes 
and there may be some problems to fix. Someone should test this with 
real devices and report the problems back to libata maintainers.

> > 2. Usability of the current tape driver API (not bugs)
> >   a. With the new O_NONBLOCK flag introduced in kernel 2.5.x, opening
> >       a tape drive and finding out if a volume is mounted is much more 
> >       complicated.  It is really inconvenient and required a lot more code
> >       in prior kernels.  This should be an item for discussion.

The reasons for the change were:
1. To be compatible with the Unix standards, and
2. To be compatible with other Unix tape driver semantics.

Because of these reasons the changes should probably not be reversed but 
there may be something to improve in the implementation. Suggestions?

> >   b. There is no simple way to determine if a tape is in a drive -- it is 
> >       at least 20 or 30 lines of C code to do it right.

Why not use GMT_ONLINE() with MTIOCGET? The definition from the st man 
page is:

GMT_ONLINE(x): The last open() found the drive with a tape in place and 
ready for operation.

If it does not work correctly, it can be fixed. (Of course, if you want to 
see if a tape is in a drive but not loaded, it is more difficult.)

> >   c. There is a real lack of information about any error condition
> >       (read/write). One can probably get it via direct SCSI commands, but
> >       why not through an ioctl.

This is true. The problem is what and how to tell the user so that the 
solution is satisfactory for a reasonable number of years. The solutions I 
have thought about have not passed this test. One alternative is to pass 
the latest sense data to the user but thinking about the different error 
conditions, this is a problematic choice. Here also suggestions from and 
discussion with real users is needed.

> >   d. The same as c. above, but for all kinds of error information that 
> >        can be important -- particularly soft errors.  An ioctl() that returns
> >        additional information could be very useful.  Currently we get it by
> >        using external SCSI programs such as mtx tapeinfo for TapeAlerts,
> >        and smartclt for errors.

Looking at what the current drives provide in log and what the kernel can 
log, I currently think that this information is best obtained with 
external tools, like you do. The number of things that can be asked from 
the drives is very large and defining a kernel interface is difficult. The 
set also is evolving. (User's suggestions are also here welcome :-)

One way to proceed might be to make a user space library that presents an 
API to users and implements it using SG_IO. If the API seems to stabilize 
and be suitable for kernel implementation, it could be later implemented 
in the kernel.

> >    e. Apparently only root can do the following (IMO, this is a bug or 
> >        programming oversight): 
> > 
> >          struct mtop mt;
> >          mt.mt_op = MTSETDRVBUFFER;
> >          mt.mt_count = MT_ST_CLEARBOOLEANS;
> >          mt.mt_count |= MT_ST_FAST_MTEOM;
> >          ...
> >          ioctl(fd, MIOCTOP, &mt);
> > 
> >       Typically Bacula runs tape daemon as non-root.  Is there a good reason 
> >        why program that has write permission on the device cannot do this
> >        ioctl() and is it possible to change this?
> > 
This is not a bug or an oversight, it is a design decision. The idea is 
that the system provides the same view of the tape to all users. This view 
is defined by the system manager (probably with startup scripts). If each 
user can change the behaviour, the next user never knows what to expect. 

I understand your view. The problem is that the same Linux is used in 
different types of systems. The design was aimed enabling controlled use 
of tapes in large systems where some users know very little about tapes. 
Your use is a small user base system where it can be assumed that the 
users know quite well what they are doing (either really know or use a 
limited set of tools that "know").

Strictly speaking MTSETDRVBUFFER does not need root privileges. It needs 
CAP_SYS_ADMIN. (Now I see that the error message is misleading and must be 
fixed.) Does using some other capability make your use easier but, at same 
time, limit the access to this command to a selected group of users?

> > There are probably a few other items that I have forgotten.  I consider #1 
> > rather important (at least for Bacula).  Item #2 is something more long term 
> > and some of my requests may have compatibility issues to consider.
> 
> I don't think that any particular individual maintains the tape code as a
> whole, so progress will probably be slow, I'm afraid.
> 
> Kai maintains the scsi tape driver and Willem looks after OSST.  Perhaps they
> can comment on some of the issues which you identify?
> 
> Thanks.
> 

-- 
Kai

Re: Linux tape drivers

[Douglas Gilbert]

Kai Makisara wrote:
> On Tue, 3 Apr 2007, Andrew Morton wrote:
> 
>> (cc's added, with permission)
>>
>> On Tue, 3 Apr 2007 15:08:37 +0200
>> Kern Sibbald <kern@sibbald.com> wrote:
>>
>>> Hello,
>>>
>>> I am the project manager for Bacula, an Open Source network backup program 
>>> that runs on all popular OSes.  After your presentation at FOSDEM in Febrary, 
>>> we briefly talked about Linux tape driver problems I am encountering, and you 
>>> offered to put me in touch with the appropriate kernel developers.
>>>
>>> I would much appreciate any help in this.  Since the problems concern all tape 
>>> drivers, I provide a very brief outline of what my would like to discuss.  
>>> First, I must mention that the Linux SCSI driver works perfectly fine with 
>>> Bacula, it is simply a question of possible improvements, under item 2 below.
>>>
>>> Issues for discussion:
>>>
>>> 1. Bugs:
>>>    a. Other than the OSST driver, apparently no IDE/SATA tape driver works
>>>        with Bacula. I don't have such a drive (working on it), but from user
>>>        reports, it appears to me that there are problems of permitting 
>>>        variable length blocks, and more serious, when writing to the end of
>>>        the tape, either the logical end of tape indicator is ignored, or when
>>>        it is encountered, all further I/O is prohibited -- including a WEOF. 
>>>        This makes reliable writing of multiple reel tapes impossible.
>>>
>>>        By the way, these IDE/SATA drives work with Bacula using the same
>>>        source code cross-compiled with GNU C++ on Linux, then run on Windows
>>>        machines, so it is most likely a driver issue rather than anything in
>>>        Bacula or the hardware.
>>>
> 
> Others have already answered this and I agree with their view. All of the 
> tape drives seem to use the SSC command set or something close to that. 
> One high-level driver should be enough to implement the user semantics.
>  
> Libata should be able to drive the SATA/IDE drives using and the drives 
> are visible as SCSI devices in Linux. In future there should be no real 
> need for ide-scsi. Probably very few people have tried libata with tapes 
> and there may be some problems to fix. Someone should test this with 
> real devices and report the problems back to libata maintainers.
> 
>>> 2. Usability of the current tape driver API (not bugs)
>>>   a. With the new O_NONBLOCK flag introduced in kernel 2.5.x, opening
>>>       a tape drive and finding out if a volume is mounted is much more 
>>>       complicated.  It is really inconvenient and required a lot more code
>>>       in prior kernels.  This should be an item for discussion.
> 
> The reasons for the change were:
> 1. To be compatible with the Unix standards, and
> 2. To be compatible with other Unix tape driver semantics.
> 
> Because of these reasons the changes should probably not be reversed but 
> there may be something to improve in the implementation. Suggestions?

Kai,
Perhaps an ignore_nonblock sysfs attribute or driver option
could be added for the old semantics.
As I have found in the past, programs the scan for devices
by opening device nodes don't play well with drivers
that hang on open.

>>>   b. There is no simple way to determine if a tape is in a drive -- it is 
>>>       at least 20 or 30 lines of C code to do it right.
> 
> Why not use GMT_ONLINE() with MTIOCGET? The definition from the st man 
> page is:
> 
> GMT_ONLINE(x): The last open() found the drive with a tape in place and 
> ready for operation.
> 
> If it does not work correctly, it can be fixed. (Of course, if you want to 
> see if a tape is in a drive but not loaded, it is more difficult.)

Sound like a TEST UNIT READY is all that is needed.
They could call out to a utility like sg_turs or
sdparm and check the exit status. They could also
build with sg3_utils-libs and call
sg_ll_test_unit_ready(). [All sg3_utils code is C++
friendly.]

Doug Gilbert

Re: Linux tape drivers

[Kern Sibbald]

On Wednesday 04 April 2007 20:46, Kai Makisara wrote:
...
> > > 
> > > Issues for discussion:
> > > 
> > > 1. Bugs:
> > >    a. Other than the OSST driver, apparently no IDE/SATA tape driver 
works
> > >        with Bacula. I don't have such a drive (working on it), but from 
user
> > >        reports, it appears to me that there are problems of permitting 
> > >        variable length blocks, and more serious, when writing to the end 
of
> > >        the tape, either the logical end of tape indicator is ignored, or 
when
> > >        it is encountered, all further I/O is prohibited -- including a 
WEOF. 
> > >        This makes reliable writing of multiple reel tapes impossible.
> > > 
> > >        By the way, these IDE/SATA drives work with Bacula using the same
> > >        source code cross-compiled with GNU C++ on Linux, then run on 
Windows
> > >        machines, so it is most likely a driver issue rather than 
anything in
> > >        Bacula or the hardware.
> > > 
> 
> Others have already answered this and I agree with their view. All of the 
> tape drives seem to use the SSC command set or something close to that. 
> One high-level driver should be enough to implement the user semantics.
>  
> Libata should be able to drive the SATA/IDE drives using and the drives 
> are visible as SCSI devices in Linux. In future there should be no real 
> need for ide-scsi. Probably very few people have tried libata with tapes 
> and there may be some problems to fix. Someone should test this with 
> real devices and report the problems back to libata maintainers.

Thanks, I'll look into using libdata and ask our users to try it. 

> 
> > > 2. Usability of the current tape driver API (not bugs)
> > >   a. With the new O_NONBLOCK flag introduced in kernel 2.5.x, opening
> > >       a tape drive and finding out if a volume is mounted is much more 
> > >       complicated.  It is really inconvenient and required a lot more 
code
> > >       in prior kernels.  This should be an item for discussion.
> 
> The reasons for the change were:
> 1. To be compatible with the Unix standards, and
> 2. To be compatible with other Unix tape driver semantics.
> 
> Because of these reasons the changes should probably not be reversed but 
> there may be something to improve in the implementation. Suggestions?

I was aware of the reasons for the change, and see the limitations in what can 
be done.  I will spend a little time making my thoughts/suggestions clear and 
get back to you.

> 
> > >   b. There is no simple way to determine if a tape is in a drive -- it 
is 
> > >       at least 20 or 30 lines of C code to do it right.
> 
> Why not use GMT_ONLINE() with MTIOCGET? The definition from the st man 
> page is:
> 
> GMT_ONLINE(x): The last open() found the drive with a tape in place and 
> ready for operation.
> 
> If it does not work correctly, it can be fixed. (Of course, if you want to 
> see if a tape is in a drive but not loaded, it is more difficult.)

I will look at it, but have not tried it because it is a Linux specific 
solution, which I try to avoid.  I'll respond more on this when responding 
correctly to item a above.

> 
> > >   c. There is a real lack of information about any error condition
> > >       (read/write). One can probably get it via direct SCSI commands, 
but
> > >       why not through an ioctl.
> 
> This is true. The problem is what and how to tell the user so that the 
> solution is satisfactory for a reasonable number of years. The solutions I 
> have thought about have not passed this test. One alternative is to pass 
> the latest sense data to the user but thinking about the different error 
> conditions, this is a problematic choice. Here also suggestions from and 
> discussion with real users is needed.

Well, you have in front of you a "real" user.  I was thinking about a new 
ioctl() that would hopefully be simple, generic, and extensible enough that 
it might be picked up by all Unixes. This is one case were even if it is a 
Linux only solution, I will be really happy. Basically, when I get a -1 
return from a read/write and ERRNO=EIO, I would like to be able to call an 
ioctl() and get back some basic info about soft errors, hard errors, ...  If 
you don't tell me that a new ioctl() is out of the question, I'll make a 
proposal -- it will take me some time to put it together.

> 
> > >   d. The same as c. above, but for all kinds of error information that 
> > >        can be important -- particularly soft errors.  An ioctl() that 
returns
> > >        additional information could be very useful.  Currently we get it 
by
> > >        using external SCSI programs such as mtx tapeinfo for TapeAlerts,
> > >        and smartclt for errors.
> 
> Looking at what the current drives provide in log and what the kernel can 
> log, I currently think that this information is best obtained with 
> external tools, like you do. The number of things that can be asked from 
> the drives is very large and defining a kernel interface is difficult. The 
> set also is evolving. (User's suggestions are also here welcome :-)

Yes, the information is there, and I agree with all your points, but getting 
the info into Bacula or any other backup program and displayed for the user 
is somewhat kludgy.

> 
> One way to proceed might be to make a user space library that presents an 
> API to users and implements it using SG_IO. If the API seems to stabilize 
> and be suitable for kernel implementation, it could be later implemented 
> in the kernel.

Yes, this is an excellent idea.

> 
> > >    e. Apparently only root can do the following (IMO, this is a bug or 
> > >        programming oversight): 
> > > 
> > >          struct mtop mt;
> > >          mt.mt_op = MTSETDRVBUFFER;
> > >          mt.mt_count = MT_ST_CLEARBOOLEANS;
> > >          mt.mt_count |= MT_ST_FAST_MTEOM;
> > >          ...
> > >          ioctl(fd, MIOCTOP, &mt);
> > > 
> > >       Typically Bacula runs tape daemon as non-root.  Is there a good 
reason 
> > >        why program that has write permission on the device cannot do 
this
> > >        ioctl() and is it possible to change this?
> > > 
> This is not a bug or an oversight, it is a design decision. The idea is 
> that the system provides the same view of the tape to all users. This view 
> is defined by the system manager (probably with startup scripts). If each 
> user can change the behaviour, the next user never knows what to expect. 

Well, I agree that all users should have the same view.  However, when Bacula 
has the drive open, it has it open exclusively, so no other users can view 
the drive. When Bacula opens the drive, it should be able to ensure that the 
drive, agrees with Bacula's concept of the drive.  When Bacula closes the 
drive, I have no problem if the OS resets values to the common view.  Today, 
if Bacula is running as bacula:disk rather than root:root, it cannot make the 
above changes, even if the disk group has write access to the drive.

> 
> I understand your view. The problem is that the same Linux is used in 
> different types of systems. The design was aimed enabling controlled use 
> of tapes in large systems where some users know very little about tapes. 
> Your use is a small user base system where it can be assumed that the 
> users know quite well what they are doing (either really know or use a 
> limited set of tools that "know").

I agree with the above, and hope we can accomplish both it and what I am 
asking for.

> 
> Strictly speaking MTSETDRVBUFFER does not need root privileges. It needs 
> CAP_SYS_ADMIN. (Now I see that the error message is misleading and must be 
> fixed.) Does using some other capability make your use easier but, at same 
> time, limit the access to this command to a selected group of users?

Being relatively ignorant of the kernel, I wasn't aware of CAP_SYS_ADMIN.  I 
now have a rough idea of what it is, I am unsure if another capability would 
resolve the problem, because I'm not familar with how they are set.  

It seems to me that anyone who is the "owner" or in the "group" and has write 
permission on the device should be able to do MTSETDRVBUFFER.  There is no 
need for any "other" who has write permission to have such access. I think 
this would allow the sys admin to restrict it appropriately.


Thanks for your responses.  I see I have some work to do organizing my 
thoughts :-)

Kern

Re: Linux tape drivers

[Kai Makisara]

On Wed, 4 Apr 2007, Kern Sibbald wrote:

> On Wednesday 04 April 2007 20:46, Kai Makisara wrote:
...
> > > >   c. There is a real lack of information about any error condition
> > > >       (read/write). One can probably get it via direct SCSI commands, 
> but
> > > >       why not through an ioctl.
> > 
> > This is true. The problem is what and how to tell the user so that the 
> > solution is satisfactory for a reasonable number of years. The solutions I 
> > have thought about have not passed this test. One alternative is to pass 
> > the latest sense data to the user but thinking about the different error 
> > conditions, this is a problematic choice. Here also suggestions from and 
> > discussion with real users is needed.
> 
> Well, you have in front of you a "real" user.

Yes, you are one of the people I meant with "real users".

>                                               I was thinking about a new 
> ioctl() that would hopefully be simple, generic, and extensible enough that 
> it might be picked up by all Unixes. This is one case were even if it is a 
> Linux only solution, I will be really happy. Basically, when I get a -1 
> return from a read/write and ERRNO=EIO, I would like to be able to call an 
> ioctl() and get back some basic info about soft errors, hard errors, ...  If 
> you don't tell me that a new ioctl() is out of the question, I'll make a 
> proposal -- it will take me some time to put it together.
> 
The general attitude seems to be against new ioctls but I don't want to 
rule out that. The mt interface already uses ioctls and they solve the 
serialisation problem nicely in this case.

Whatever the method of delivery will be, the most difficult question is 
the contents and format. I don't know any good examples from other 
systems, so it has to be Linux-specific (and hopefully adopted by others 
as you say). I will wait with interest for your suggestion.

...
> > > >    e. Apparently only root can do the following (IMO, this is a bug or 
> > > >        programming oversight): 
> > > > 
> > > >          struct mtop mt;
> > > >          mt.mt_op = MTSETDRVBUFFER;
> > > >          mt.mt_count = MT_ST_CLEARBOOLEANS;
> > > >          mt.mt_count |= MT_ST_FAST_MTEOM;
> > > >          ...
> > > >          ioctl(fd, MIOCTOP, &mt);
> > > > 
> > > >       Typically Bacula runs tape daemon as non-root.  Is there a good 
> reason 
> > > >        why program that has write permission on the device cannot do 
> this
> > > >        ioctl() and is it possible to change this?
> > > > 
> > This is not a bug or an oversight, it is a design decision. The idea is 
> > that the system provides the same view of the tape to all users. This view 
> > is defined by the system manager (probably with startup scripts). If each 
> > user can change the behaviour, the next user never knows what to expect. 
> 
> Well, I agree that all users should have the same view.  However, when Bacula 
> has the drive open, it has it open exclusively, so no other users can view 
> the drive. When Bacula opens the drive, it should be able to ensure that the 
> drive, agrees with Bacula's concept of the drive.  When Bacula closes the 
> drive, I have no problem if the OS resets values to the common view.  Today, 
> if Bacula is running as bacula:disk rather than root:root, it cannot make the 
> above changes, even if the disk group has write access to the drive.

OK. I have not thought about this properly but one possible solution might
be something like this:
- separate the current and permanent behaviour flags; the current flags 
  define the behaviour
- the permanent flags are copied into the current flags when the device is
  opened (or at some other well-defined point)
- the booleans set with MTSETDRVBUFFER change always the current flags; if
  the user has CAP_SYS_ADMIN, also the permanent flags are changed

This would enable anyone to change the behaviour for his/her own purposes 
but there would still be a stable system-defined behaviour. There already 
are attributes that have a current and default value like block size.

I can see several problems in the details of this suggestion but I will 
think about those. One is that if someone having CAP_SYS_ADMIN wants just 
a temporary change, it is not possible. A solution would to have a new 
op code for temporary changes but this would require explicit changes to 
programs. Comments are welcome.

...
> > Strictly speaking MTSETDRVBUFFER does not need root privileges. It needs 
> > CAP_SYS_ADMIN. (Now I see that the error message is misleading and must be 
> > fixed.) Does using some other capability make your use easier but, at same 
> > time, limit the access to this command to a selected group of users?
> 
> Being relatively ignorant of the kernel, I wasn't aware of CAP_SYS_ADMIN.  I 
> now have a rough idea of what it is, I am unsure if another capability would 
> resolve the problem, because I'm not familar with how they are set.  
> 
Yes, setting the capabilities may be a problem. I don't know how well the 
different distributions are able to handle these finer-grained rights. 
A software package has to support ideally all distributions and the common 
denominator may be to be root for CAP_SYS_ADMIN ;-)

> It seems to me that anyone who is the "owner" or in the "group" and has write 
> permission on the device should be able to do MTSETDRVBUFFER.  There is no 
> need for any "other" who has write permission to have such access. I think 
> this would allow the sys admin to restrict it appropriately.
> 
This would be one solution but it sounds complicated. The primary test for 
a file is just if there is write access or not. It does not matter if the 
access right comes as user, group, or other.

> 
> Thanks for your responses.  I see I have some work to do organizing my 
> thoughts :-)

Thanks for your input.

-- 
Kai

Re: Linux tape drivers

[Kern Sibbald]

On Wednesday 04 April 2007 23:54, Kai Makisara wrote:
> On Wed, 4 Apr 2007, Kern Sibbald wrote:
> 
> > On Wednesday 04 April 2007 20:46, Kai Makisara wrote:
> ...
> > > > >   c. There is a real lack of information about any error condition
> > > > >       (read/write). One can probably get it via direct SCSI 
commands, 
> > but
> > > > >       why not through an ioctl.
> > > 
> > > This is true. The problem is what and how to tell the user so that the 
> > > solution is satisfactory for a reasonable number of years. The solutions 
I 
> > > have thought about have not passed this test. One alternative is to pass 
> > > the latest sense data to the user but thinking about the different error 
> > > conditions, this is a problematic choice. Here also suggestions from and 
> > > discussion with real users is needed.
> > 
> > Well, you have in front of you a "real" user.
> 
> Yes, you are one of the people I meant with "real users".

OK, I'll address this issue in my "second" description that I mention below.

> 
> >                                               I was thinking about a new 
> > ioctl() that would hopefully be simple, generic, and extensible enough 
that 
> > it might be picked up by all Unixes. This is one case were even if it is a 
> > Linux only solution, I will be really happy. Basically, when I get a -1 
> > return from a read/write and ERRNO=EIO, I would like to be able to call an 
> > ioctl() and get back some basic info about soft errors, hard errors, ...  
If 
> > you don't tell me that a new ioctl() is out of the question, I'll make a 
> > proposal -- it will take me some time to put it together.
> > 
> The general attitude seems to be against new ioctls but I don't want to 
> rule out that. The mt interface already uses ioctls and they solve the 
> serialisation problem nicely in this case.
> 
> Whatever the method of delivery will be, the most difficult question is 
> the contents and format. I don't know any good examples from other 
> systems, so it has to be Linux-specific (and hopefully adopted by others 
> as you say). I will wait with interest for your suggestion.

OK, I don't think that I have the knowledge of the kernel to be able to make a 
specific proposal, but I will do the following (unless someone suggests 
otherwise).  

First explain to everyone the pains that were caused with the O_NONBLOCK 
change to opening tape drives.  I don't imagine we will go back to the old 
behavior, but I hope something similar won't happen again, and will propose 
how a slight change in philosophy could have created less chaos.  This I can 
do in the next few days.

Second, I can describe what would be a reasonably complete tape interface from 
the applications programmers point of view (i.e. Bacula), and how I think we 
might get there given your input above -- my basic premise is that there does 
not exist any real Unix tape API, but rather a whole slew of different ioctls 
on different systems, making applications programming difficult.  I think we 
need a real Unix tape API, but not necessarily a lot more ioctls.  This will 
take at least a week (I think).

> 
> ...
> > > > >    e. Apparently only root can do the following (IMO, this is a bug 
or 
> > > > >        programming oversight): 
> > > > > 
> > > > >          struct mtop mt;
> > > > >          mt.mt_op = MTSETDRVBUFFER;
> > > > >          mt.mt_count = MT_ST_CLEARBOOLEANS;
> > > > >          mt.mt_count |= MT_ST_FAST_MTEOM;
> > > > >          ...
> > > > >          ioctl(fd, MIOCTOP, &mt);
> > > > > 
> > > > >       Typically Bacula runs tape daemon as non-root.  Is there a 
good 
> > reason 
> > > > >        why program that has write permission on the device cannot do 
> > this
> > > > >        ioctl() and is it possible to change this?
> > > > > 
> > > This is not a bug or an oversight, it is a design decision. The idea is 
> > > that the system provides the same view of the tape to all users. This 
view 
> > > is defined by the system manager (probably with startup scripts). If 
each 
> > > user can change the behaviour, the next user never knows what to expect. 
> > 
> > Well, I agree that all users should have the same view.  However, when 
Bacula 
> > has the drive open, it has it open exclusively, so no other users can view 
> > the drive. When Bacula opens the drive, it should be able to ensure that 
the 
> > drive, agrees with Bacula's concept of the drive.  When Bacula closes the 
> > drive, I have no problem if the OS resets values to the common view.  
Today, 
> > if Bacula is running as bacula:disk rather than root:root, it cannot make 
the 
> > above changes, even if the disk group has write access to the drive.
> 
> OK. I have not thought about this properly but one possible solution might
> be something like this:
> - separate the current and permanent behaviour flags; the current flags 
>   define the behaviour
> - the permanent flags are copied into the current flags when the device is
>   opened (or at some other well-defined point)
> - the booleans set with MTSETDRVBUFFER change always the current flags; if
>   the user has CAP_SYS_ADMIN, also the permanent flags are changed
> 
> This would enable anyone to change the behaviour for his/her own purposes 
> but there would still be a stable system-defined behaviour. There already 
> are attributes that have a current and default value like block size.
> 
> I can see several problems in the details of this suggestion but I will 
> think about those. One is that if someone having CAP_SYS_ADMIN wants just 
> a temporary change, it is not possible. A solution would to have a new 
> op code for temporary changes but this would require explicit changes to 
> programs. Comments are welcome.

The above would solve the problem for me.  

Today, when root (or someone having CAP_SYS_ADMIN) does a MTSETDRVBUFFER, 
doesn't it alter the permanent flags? I think we should look at how the other 
flags are set under various permissions and try to get the MTSETDRVBUFFER to 
follow the same rules.  

The basic problem I see with MTSETDRVBUFFER is that it requires CAP_SYS_ADMIN, 
while the other ones (e.g. MTSETBLK) do not (i.e. an inconsistency).  If they 
all work the same and permit changes without CAP_SYS_ADMIN, then the rules 
will be much easier for everyone to understand.

> 
> ...
> > > Strictly speaking MTSETDRVBUFFER does not need root privileges. It needs 
> > > CAP_SYS_ADMIN. (Now I see that the error message is misleading and must 
be 
> > > fixed.) Does using some other capability make your use easier but, at 
same 
> > > time, limit the access to this command to a selected group of users?
> > 
> > Being relatively ignorant of the kernel, I wasn't aware of CAP_SYS_ADMIN.  
I 
> > now have a rough idea of what it is, I am unsure if another capability 
would 
> > resolve the problem, because I'm not familar with how they are set.  
> > 
> Yes, setting the capabilities may be a problem. I don't know how well the 
> different distributions are able to handle these finer-grained rights. 
> A software package has to support ideally all distributions and the common 
> denominator may be to be root for CAP_SYS_ADMIN ;-)

To the best of my knowledge Linux is the only system that requires root; this 
is true at least for the values that Bacula sets.  I've only been adding such 
OS dependent functions for a year or so to attempt to reduce support problems 
(i.e. making Bacula "self-configure" tape drives to a limited extent). 
Requiring root permission to configure the tape drive is for Bacula a 
security problem and thus probably not the best solution.

> 
> > It seems to me that anyone who is the "owner" or in the "group" and has 
write 
> > permission on the device should be able to do MTSETDRVBUFFER.  There is no 
> > need for any "other" who has write permission to have such access. I think 
> > this would allow the sys admin to restrict it appropriately.
> > 
> This would be one solution but it sounds complicated. The primary test for 
> a file is just if there is write access or not. It does not matter if the 
> access right comes as user, group, or other.

Yes, I can see that would be complicated, and probably not possible, but from 
an applications (my) stand point, it seems reasonable.

> 
> > 
> > Thanks for your responses.  I see I have some work to do organizing my 
> > thoughts :-)
> 
> Thanks for your input.
> 
> -- 
> Kai
>