[OT] Re: New thread RDSL, post-2.6.20 kernels and amanda (tar) miss-fires

[Willy Tarreau <w@1wt.eu>]

On Wed, Mar 14, 2007 at 11:12:48PM -0400, Gene Heskett wrote:
> On Wednesday 14 March 2007, Ray Lee wrote:
> >On 3/13/07, Gene Heskett <gene.heskett@gmail.com> wrote:
> >> On Tuesday 13 March 2007, Gene Heskett wrote:
> >> >On Tuesday 13 March 2007, Gene Heskett wrote:
> >> >>Greetings;
> >> >>Someone suggested a fresh thread for this.
> >> >>
> >> >>I now have my scripts more or less under control, and I can report
> >> >> that kernel-2.6.20.1 with no other patches does not exhibit the
> >> >> undesirable behaviour where tar thinks its all new, even when told
> >> >> to do a level 2 on a directory tree that hasn't been touched in
> >> >> months to update anything.
> >> >>
> >> >>Next up, 2.6.20.2, plain and with the latest RDSL-0.30 patch.
> >> >
> >> >And amanda/tar worked normally for 2.6.20.2 plain.
> >> >
> >> >Next up, 2.6.21-rc1 if it will build here.
> >>
> >> It built, it booted, and its busted big time.  First, with an amdump
> >> running in the background, the machine is so close to unusable that I
> >> considered rebooting, but I needed the data to show the problem.  I am
> >> losing the keyboard and mouse for a minute or more at a time but the
> >> keystrokes seem to be being registered so it eventually catches up.
> >>
> >> Disk i/o seems to be the killer according to gkrellm.
> >>
> >> But to give one an idea of the fits this is giving tar, I'll snip a
> >> line or 2 from an amstatus report here:
> >> coyote:/GenesAmandaHelper-0.6 1 planner: [dumps way too big, 138200
> >> KB, must skip incremental dumps]
> >>
> >> Huh?  138.2GB?  A 'du -h .' in that dir says 766megs.
> >>
> >> coyote:/root                  1     4426m wait for dumping
> >> du -h says 5.0GB so that's ballpark, but its also a level 1, so maybe
> >> 20 megs is actually new since 15:57 this afternoon local.  kmails
> >> final maildir is in that dir.
> >>
> >> This goes on for much of the amstatus report, very few of the reported
> >> sizes are close to sane.
> >>
> >> Now, can someone suggest a patch I can revert that might fix this? 
> >> The total number of patches between 2.6.20 and 2.6.21-rc1 will have me
> >> building kernels to bisect this till the middle of June at this rate.
> >
> >In a previous email, you said you were using ext3. If that's the case,
> >there doesn't appear to be much going on in terms of patches between
> >2.6.20 and 2.6.21-rc1. The only one that even comes close to looking
> >like it might have an effect would only come in to play if you have a
> >filesystem that has ACL information, but is mounted by a kernel that
> >doesn't have ACL support.
> >
> >I have to echo wli here, I'm afraid, and recommend at least a *few*
> >bisections to help narrow down the list of suspect patches.
> >
> >There are tutorials out there for git users. I use the mercurial
> >repository, as I find the mercurial interface and workflow a lot more
> >intuitive, but it has the same capability.
> >
> >Even 2-5 bisections will greatly help others hunt the bug down.
> >
> >Ray
> 
> Probably.  But I've now put a week into this, and from some other clues 
> I've collected, I'm beginning to think tar has a tummy ache. After all, 
> and ls -lc reports totally sane mtimes.  So why is tar going bonkers 
> under kernels 2.6.21-rc*, with or without Cons patches?
> 
> I've also spent a day now looking for a valid place to put a bugzilla 
> entry against tar, but googles search results are sending me to 
> gcc.gnu.org and this is NOT the correct bugzilla for a tar problem.
> 
> Its no secret that with all the churn in tar over the last 5 years, worse 
> churn than the kernel IMO in going from 2.0 to 2.6, that I'm not a fan of 
> yet another _new_ version of tar, when what we just need is _one_ that 
> works.  It is not capable of executing the recovery command listed in the 
> first block of every amdump file it (amdump) ever built right now, and 
> I've played the equ of the 10,000 monkeys writing Shakespear for several 
> hours trying.  Damned frustrating is what it is.
> 
> The error it reports seems to indicate that it cannot write through the 
> pipes involved.  But with tar's error reporting, who the hell knows for 
> sure.
> 
> Here is an example
> [root@coyote data]# dd if=00010.coyote._lib.1 bs=32k count=1
> AMANDA: FILE 20070314104344 coyote /lib  lev 1 comp .gz program /bin/tar
> To restore, position tape at start of file and run:
>  dd if=<tape> bs=32k skip=1 |  /bin/gzip -dc |  /bin/tar -f - ...
> 
> And the elipsis is an error if not removed.  Then one is supposed to be 
> able to redirect tars output with the usual >/tmp/test/ syntax
> 
> So:
> [root@coyote data]# dd if=00010.coyote._lib.1 bs=32k 
> skip=1 |  /bin/gzip -dc |  /bin/tar -f - >/tmp/test/
> -bash: /tmp/test/: Is a directory
> 
> which is the return from any variation in how the redirect is done.
> 
> So what is it that am I doing wrong in the above command line?, so I can 
> add it to my helper scripts to be published eventually on zmanda.org.

with "/bin/tar -f - >/tmp/test/", you ask bash to open the file "/tmp/test/"
for write, then start tar and pass this file as its stdout. Obviously this
is wrong. I think that what you're trying to do is send extracted files to
/tmp/test, which is what '-C' is for. Also, you need to specify a command
for tar. You didn't. I bet if you do the following, it will work :

[root@coyote data]# dd if=00010.coyote._lib.1 bs=32k skip=1 |
    /bin/gzip -dc |  /bin/tar -C /tmp/test/ -xf -

Now, Gene, this is becoming totally off-topic right here.

Regards,
Willy