[linux-lvm] LVM onFly features

[linux-lvm] LVM onFly features

[Mag Gam]

any news about hot(no unmount, no fsck, no fs resize) ext2/ext3
filesystem increase?

Re: [linux-lvm] LVM onFly features

[Marc-Jano Knopp]

On Sat, 10 Dec 2005 at 14:38 (-0500), Mag Gam wrote:
> any news about hot(no unmount, no fsck, no fs resize) ext2/ext3
> filesystem increase?

And when will online-*shrinking* of ext3 appear?


Best regards

  Marc-Jano

Re: [linux-lvm] LVM onFly features

[Michael Loftis]

--On December 10, 2005 8:48:27 PM +0100 Marc-Jano Knopp 
<pub_ml_lvm@marc-jano.de> wrote:

> On Sat, 10 Dec 2005 at 14:38 (-0500), Mag Gam wrote:
>> any news about hot(no unmount, no fsck, no fs resize) ext2/ext3
>> filesystem increase?
>
> And when will online-*shrinking* of ext3 appear?

These are not LVM features.  These are filesystem features.  Better to ask 
on lkml, or the maintainers directly.

Re: [linux-lvm] LVM onFly features

[Marc-Jano Knopp]

On Sat, 10 Dec 2005 at 13:03 (-0700), Michael Loftis wrote:
> --On December 10, 2005 8:48:27 PM +0100 Marc-Jano Knopp <pub_ml_lvm@marc-jano.de> wrote:
> >On Sat, 10 Dec 2005 at 14:38 (-0500), Mag Gam wrote:
> >>any news about hot(no unmount, no fsck, no fs resize) ext2/ext3
> >>filesystem increase?
> >
> >And when will online-*shrinking* of ext3 appear?
> 
> These are not LVM features.  These are filesystem features.  Better to ask 
> on lkml, or the maintainers directly.

D'oh! Sorry for that, I was misguided by the previous post!


Best regards

  Marc-"GPL-ZFS, anyone? ;-)"-Jano

Re: [linux-lvm] LVM onFly features

[Michael Loftis]

--On December 10, 2005 9:06:46 PM +0100 Marc-Jano Knopp 
<pub_ml_lvm@marc-jano.de> wrote:

> On Sat, 10 Dec 2005 at 13:03 (-0700), Michael Loftis wrote:
>> --On December 10, 2005 8:48:27 PM +0100 Marc-Jano Knopp
>> <pub_ml_lvm@marc-jano.de> wrote:
>> > On Sat, 10 Dec 2005 at 14:38 (-0500), Mag Gam wrote:
>> >> any news about hot(no unmount, no fsck, no fs resize) ext2/ext3
>> >> filesystem increase?
>> >
>> > And when will online-*shrinking* of ext3 appear?
>>
>> These are not LVM features.  These are filesystem features.  Better to
>> ask  on lkml, or the maintainers directly.
>
> D'oh! Sorry for that, I was misguided by the previous post!

Not a problem, a lot of people don't see the delineation right off. 
Filesystems like VXVFS (Veritas) in their commercial implementations are 
really LVM+Filesystem in one.  Not sure what the linux port is looking like 
these days though.

ReiserFS has hot expansion capabilities, but no (yet?) hot shrinking 
capabilities.  One of the reasons it has these features and ext2/3 does not 
is because ext2/3 are very old filesystems designed on a different 
mentality of a static filesystem.  On-line expansion of ext2 based 
filesystems is an extremely complicated venture, it might honestly even be 
impossible.

ReiserFS has the advantage here because it doesn't necessarily pre-write 
out a lot of filesystem meta-information (superblocks, inodes, bitmaps, 
etc), instead these structures are entirely dynamic to begin with, so 
enlarging them at runtime is trivial, requires a very short lock and a 
change to a few numbers.  Ext2 resizing requires actually rewriting a lot 
of filesystem metadata.

Re: [linux-lvm] LVM onFly features

[Marc-Jano Knopp]

On Sat, 10 Dec 2005 at 13:14 (-0700), Michael Loftis wrote:
> --On December 10, 2005 9:06:46 PM +0100 Marc-Jano Knopp <pub_ml_lvm@marc-jano.de> wrote:
> >On Sat, 10 Dec 2005 at 13:03 (-0700), Michael Loftis wrote:
> >>--On December 10, 2005 8:48:27 PM +0100 Marc-Jano Knopp
> >><pub_ml_lvm@marc-jano.de> wrote:
> >>> On Sat, 10 Dec 2005 at 14:38 (-0500), Mag Gam wrote:

[Resizing of file systems]

[...]
> ReiserFS has hot expansion capabilities, but no (yet?) hot shrinking 
> capabilities.  One of the reasons it has these features and ext2/3 does not 
> is because ext2/3 are very old filesystems designed on a different 
> mentality of a static filesystem.  On-line expansion of ext2 based 
> filesystems is an extremely complicated venture, it might honestly even be 
> impossible.
> 
> ReiserFS has the advantage here because it doesn't necessarily pre-write 
> out a lot of filesystem meta-information (superblocks, inodes, bitmaps, 
> etc), instead these structures are entirely dynamic to begin with, so 
> enlarging them at runtime is trivial, requires a very short lock and a 
> change to a few numbers.  Ext2 resizing requires actually rewriting a lot 
> of filesystem metadata.

Thanks for the detailled explanation!

Yes, I would *love* to use a totally new file system with a new,
dynamic, good design, but - just as many others - had my experiences
with ReiserFS and it will take a *lot* of time for ReiserFS to restore
confidence. So for now, I'll probably stay with ext3, with which I had
no problems so far.

JFS/XFS should also both be capable of growing, XFS of online-growth,
IIRC.


Gru�

  Marc-Jano

Re: [linux-lvm] LVM onFly features

[Michael Loftis]

--On December 10, 2005 9:22:32 PM +0100 Marc-Jano Knopp 
<pub_ml_lvm@marc-jano.de> wrote:


> Thanks for the detailled explanation!

I try not to say something without actual experience and technical details 
to back it up. :)

>
> Yes, I would *love* to use a totally new file system with a new,
> dynamic, good design, but - just as many others - had my experiences
> with ReiserFS and it will take a *lot* of time for ReiserFS to restore
> confidence. So for now, I'll probably stay with ext3, with which I had
> no problems so far.
>
> JFS/XFS should also both be capable of growing, XFS of online-growth,
> IIRC.

XFS has terrible unpredictable performance in production.  Also it has very 
bad behavior when recovering from crashes, often times it's tools totally 
fail to clean the filesystem.  It also needs larger kernel stacks because 
of some of the really deep call trees, so when you use it with LVM or MD it 
can oops unless you use the larger kernel stacks.  We also have had 
problems with the quota system but the details on that have faded.

I've had far better reliability and performance out of ReiserFS in 
production (late 2.4 series... 2.4.20+, currently 2.4.25, with some patches 
on most of our larger systems) than XFS.  I've not had any close experience 
with JFS.

XFS is also very biased towards streaming rather than random I/O 
performance in our experience.

XFS may be a proven filesystem, but it has not yet been proven in Linux' 
implementation.  That said, all of the filesystems have their own quirks 
and shortcomings.  We had a corruption problem with our CX200 that caused 
our ReiserFS to lose most of it's tails.  Really it was the CX200 (EMC 
Clariion) fault, but it felt (And still does) at the time that ReiserFS 
could've or atleast should've been able to save more of the tail data that 
it lost.  It didn't lose any files, just a the tails.

>
>
> Gruß
>
>   Marc-Jano
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>



--
"Genius might be described as a supreme capacity for getting its possessors
into trouble of all kinds."
-- Samuel Butler

Re: [linux-lvm] LVM onFly features

[Marc-Jano Knopp]

On Sat, 10 Dec 2005 at 15:10 (-0700), Michael Loftis wrote:
> --On December 10, 2005 9:22:32 PM +0100 Marc-Jano Knopp <pub_ml_lvm@marc-jano.de> wrote:
> 
> >Thanks for the detailled explanation!
> 
> I try not to say something without actual experience and technical details 
> to back it up. :)

If only everyone would do that ... (that said, I should better shut up
from now on :-}


> > So for now, I'll probably stay with ext3, with which I had
> > no problems so far.
[...]
> XFS has terrible unpredictable performance in production.  Also it has very 
> bad behavior when recovering from crashes, often times it's tools totally 
> fail to clean the filesystem.

Okay, so I'm even more biased to using ext3. :-}


[...]
> I've had far better reliability and performance out of ReiserFS in 
> production (late 2.4 series... 2.4.20+, currently 2.4.25, with some patches 
> on most of our larger systems) than XFS.

Hmm ... i guess for 2.6.x, experiences can totally differ.


[...]
> XFS may be a proven filesystem, but it has not yet been proven in Linux' 
> implementation.  That said, all of the filesystems have their own quirks 
> and shortcomings.  We had a corruption problem with our CX200 that caused 
> our ReiserFS to lose most of it's tails.  Really it was the CX200 (EMC 
> Clariion) fault, but it felt (And still does) at the time that ReiserFS 
> could've or atleast should've been able to save more of the tail data that 
> it lost.  It didn't lose any files, just a the tails.

I thought the tails would be used to save the file blocks with less than
$BLOCKSIZE? So, if ReiserFS lost the tails, it would be a very lucky
coincidence, if none of the files were damaged. Or am I misguided again?
:-}


Best regards

  Marc-Jano

Re: [linux-lvm] LVM onFly features

[Michael Loftis]

--On December 10, 2005 11:31:40 PM +0100 Marc-Jano Knopp 
<pub_ml_lvm@marc-jano.de> wrote:

> I thought the tails would be used to save the file blocks with less than
> $BLOCKSIZE? So, if ReiserFS lost the tails, it would be a very lucky
> coincidence, if none of the files were damaged. Or am I misguided again?
> :-}

Any tail can be packed if it doesn't match the block size.  In our case the 
corruption evidenced itself as files with the first part being ok, and the 
last part being all NULLs (0x0) up to the 'normal' file size.

Re: [linux-lvm] LVM onFly features

[Michael Loftis]

--On December 10, 2005 3:44:00 PM -0700 Michael Loftis <mloftis@wgops.com> 
wrote:

>
>
> --On December 10, 2005 11:31:40 PM +0100 Marc-Jano Knopp
> <pub_ml_lvm@marc-jano.de> wrote:
>
>> I thought the tails would be used to save the file blocks with less than
>> $BLOCKSIZE? So, if ReiserFS lost the tails, it would be a very lucky
>> coincidence, if none of the files were damaged. Or am I misguided again?
>> :-}
>
> Any tail can be packed if it doesn't match the block size.  In our case
> the corruption evidenced itself as files with the first part being ok,
> and the last part being all NULLs (0x0) up to the 'normal' file size.


I really should say *mostly* because yes, files that were entirely shorter 
than the blocksize were essentially gone (full of NULLs).

Our mail server makes the best use of reiserfs because of the huge number 
of files and the amount of files that end up packed.

Re: [linux-lvm] LVM onFly features

[Marc-Jano Knopp]

On Sat, 10 Dec 2005 at 15:51 (-0700), Michael Loftis wrote:
> --On December 10, 2005 3:44:00 PM -0700 Michael Loftis <mloftis@wgops.com> 
> wrote:
> >--On December 10, 2005 11:31:40 PM +0100 Marc-Jano Knopp <pub_ml_lvm@marc-jano.de> wrote:
[...]
> >Any tail can be packed if it doesn't match the block size.  In our case
> >the corruption evidenced itself as files with the first part being ok,
> >and the last part being all NULLs (0x0) up to the 'normal' file size.
> 
> I really should say *mostly* because yes, files that were entirely shorter 
> than the blocksize were essentially gone (full of NULLs).

I knew it. :-)


> Our mail server makes the best use of reiserfs because of the huge number 
> of files and the amount of files that end up packed.

Same effect with news spool here (until ReiserFS lost files and I
returned remorsefully to ext3 :-).


Best regards

  Marc-Jano

Re: [linux-lvm] LVM onFly features

[Mag Gam]

thanks everyone for their kind replies.

So, will LVM and FS (ext2/3 in particular )ever be considered as 1?
This would be nice for ease of devt.




On 12/10/05, Marc-Jano Knopp <pub_ml_lvm@marc-jano.de> wrote:
> On Sat, 10 Dec 2005 at 15:51 (-0700), Michael Loftis wrote:
> > --On December 10, 2005 3:44:00 PM -0700 Michael Loftis <mloftis@wgops.com>
> > wrote:
> > >--On December 10, 2005 11:31:40 PM +0100 Marc-Jano Knopp <pub_ml_lvm@marc-jano.de> wrote:
> [...]
> > >Any tail can be packed if it doesn't match the block size.  In our case
> > >the corruption evidenced itself as files with the first part being ok,
> > >and the last part being all NULLs (0x0) up to the 'normal' file size.
> >
> > I really should say *mostly* because yes, files that were entirely shorter
> > than the blocksize were essentially gone (full of NULLs).
>
> I knew it. :-)
>
>
> > Our mail server makes the best use of reiserfs because of the huge number
> > of files and the amount of files that end up packed.
>
> Same effect with news spool here (until ReiserFS lost files and I
> returned remorsefully to ext3 :-).
>
>
> Best regards
>
>   Marc-Jano
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>

Re: [linux-lvm] LVM onFly features

[Michael Loftis]

--On December 10, 2005 10:38:46 PM -0500 Mag Gam <magawake@gmail.com> wrote:

> thanks everyone for their kind replies.
>
> So, will LVM and FS (ext2/3 in particular )ever be considered as 1?
> This would be nice for ease of devt.

I sure hope not.  They're different beasts in my mind, not to mention there 
are things other than filesystems that can go on block devices (oracle 
databases for one, though that's becoming depricated).

Re: [linux-lvm] LVM onFly features

[Fredrik Tolf]

On Sat, 2005-12-10 at 22:38 -0500, Mag Gam wrote:
> thanks everyone for their kind replies.
> 
> So, will LVM and FS (ext2/3 in particular )ever be considered as 1?
> This would be nice for ease of devt.

If they would be considered as one, it wouldn't be LVM anymore. LVM is a
pure block device layer, and as such will always have to FS layered
above.

However, stuff like Solaris ZFS does make sense, too, if you ask me.
Having a filesystem managing several block devices allows for pretty
neat stuff, such as per-file replication or striping, which would be
really sweet.

That said, such a thing would, of course, be completely unrelated to
LVM.

Fredrik Tolf

Re: [linux-lvm] LVM onFly features

[Mag Gam]

I know AIX LVM and FS act usually as one. Thats where I was going at....

But, thats everyone for their replies!

On 12/11/05, Fredrik Tolf <fredrik@dolda2000.com> wrote:
> On Sat, 2005-12-10 at 22:38 -0500, Mag Gam wrote:
> > thanks everyone for their kind replies.
> >
> > So, will LVM and FS (ext2/3 in particular )ever be considered as 1?
> > This would be nice for ease of devt.
>
> If they would be considered as one, it wouldn't be LVM anymore. LVM is a
> pure block device layer, and as such will always have to FS layered
> above.
>
> However, stuff like Solaris ZFS does make sense, too, if you ask me.
> Having a filesystem managing several block devices allows for pretty
> neat stuff, such as per-file replication or striping, which would be
> really sweet.
>
> That said, such a thing would, of course, be completely unrelated to
> LVM.
>
> Fredrik Tolf
>
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>

Re: [linux-lvm] LVM onFly features

[David Johnston]

On Sun, 2005-12-11 at 10:32 -0500, Mag Gam wrote:
> I know AIX LVM and FS act usually as one. Thats where I was going at....
> 
> But, thats everyone for their replies!

Not really.  The AIX configuration tools simplify things for you to the
point that you might think that they are acting as one, but they work
exactly the same way under AIX and Linux.  Look at the config tool's
logs, and you will see that to expand a filesystem, it first calls lvm
to expand the logical volume, then calls the jfs utility to expand the
filesystem.  There are internal differences in the two implementations,
but the division between logical block devices and filesystems is there
under both AIX and Linux.

-David

Re: [linux-lvm] LVM onFly features

[Mag Gam]

David:

You are right. On AIX, it first runs, extendlv, and then a chfs...





On 12/15/05, David Johnston <david@littlebald.com> wrote:
> On Sun, 2005-12-11 at 10:32 -0500, Mag Gam wrote:
> > I know AIX LVM and FS act usually as one. Thats where I was going at....
> >
> > But, thats everyone for their replies!
>
> Not really.  The AIX configuration tools simplify things for you to the
> point that you might think that they are acting as one, but they work
> exactly the same way under AIX and Linux.  Look at the config tool's
> logs, and you will see that to expand a filesystem, it first calls lvm
> to expand the logical volume, then calls the jfs utility to expand the
> filesystem.  There are internal differences in the two implementations,
> but the division between logical block devices and filesystems is there
> under both AIX and Linux.
>
> -David
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>

Re: [linux-lvm] LVM onFly features

[Nathan Scott]

On Sat, Dec 10, 2005 at 03:10:05PM -0700, Michael Loftis wrote:
> 
> >Thanks for the detailled explanation!
> 
> I try not to say something without actual experience and technical details 
> to back it up. :)

Hmm, you seem to be doing a pretty good job of that here...

> >Yes, I would *love* to use a totally new file system with a new,
> >dynamic, good design, but - just as many others - had my experiences
> >with ReiserFS and it will take a *lot* of time for ReiserFS to restore
> >confidence. So for now, I'll probably stay with ext3, with which I had
> >no problems so far.
> >
> >JFS/XFS should also both be capable of growing, XFS of online-growth,
> >IIRC.
> 
> XFS has terrible unpredictable performance in production.  Also it has very 

What on earth does that mean?  Whatever it means, it doesn't
sound right - can you back that up with some data please?

> bad behavior when recovering from crashes,

Details?  Are you talking about this post of yours:
http://oss.sgi.com/archives/linux-xfs/2003-06/msg00032.html

There have been several fixes in this area since that post.

> often times it's tools totally fail to clean the filesystem.

In what way?  Did you open a bug report?

> It also needs larger kernel stacks because 
> of some of the really deep call trees,

Those have been long since fixed as far as we are aware.  Do you
have an actual example where things can fail?

> so when you use it with LVM or MD it 
> can oops unless you use the larger kernel stacks.

Anything can oops in combination with enough stacked device drivers
(although there has been block layer work to resolve this recently,
so you should try again with a current kernel...).  If you have an
actual example of this still happening, please open a bug or at least
let the XFS developers know of your test case.  Thanks.

> We also have had 
> problems with the quota system but the details on that have faded.

Seems like details of all the problems you described have faded.
Your mail seems to me like a bit of a troll ... I guess you had a
problem or two a couple of years ago (from searching the lists)
and are still sore.  Can you point me to mailing list reports of
the problems you're refering to here or bug reports you've opened
for these issues?  I'll let you know if any of them are still
relevent.

cheers.

-- 
Nathan

Re: [linux-lvm] LVM onFly features

[Michael Loftis]

--On December 12, 2005 9:15:39 AM +1100 Nathan Scott <nathans@sgi.com> 
wrote:


>> XFS has terrible unpredictable performance in production.  Also it has
>> very
>
> What on earth does that mean?  Whatever it means, it doesn't
> sound right - can you back that up with some data please?

The worst problems we had we're likely most strongly related to running out 
of journal transaction space.  When XFS was under high transaction load 
sometimes it would just hang everything syncing meta-data.  From what I 
understand this has supposedly been dealt with, but we were still having 
these issues when we decommissioned the last XFS based server a year ago. 
Another datapoint is the fact we primarily served via NFS, which XFS 
(atleast at the time) still didn't behave great with, I never did see any 
good answers on that as I recall.

>
>> bad behavior when recovering from crashes,
>
> Details?  Are you talking about this post of yours:
> http://oss.sgi.com/archives/linux-xfs/2003-06/msg00032.html

That particular behavior happened a lot.  And it wasn't annoying that it 
happened, so much so that it happened after the system claimed it was 
clean.  Further, yes, that hardware has been fully checked out.  There's 
nothing wrong with the hardware.  I wish there was, that'd make me feel 
better honestly.  The only thing I can reason is bugs in the XFS 
fsck/repair tools, or *maybe* an interaction with XFS and the DAC960 
controller, or NFS.  The fact that XFS has weird interactions with NFS at 
all bugs me, but I don't understand the code involved well enough.  There 
might be a decent reason.

>
> There have been several fixes in this area since that post.
>
>> often times it's tools totally fail to clean the filesystem.
>
> In what way?  Did you open a bug report?
>
>> It also needs larger kernel stacks because
>> of some of the really deep call trees,
>
> Those have been long since fixed as far as we are aware.  Do you
> have an actual example where things can fail?

We pulled it out of production and replaced XFS with Reiser.  At the time 
Reiser was far more mature on Linux.  XFS Linux implementation (in 
combination with other work in the block layer as you mention later) may 
have matured to atleast a similar (possibly moreso) point now.  I've just 
personally lost more data due to XFS than Reiser.  I've also had problems 
with ext3 in the (now distant) past while it was teething still.


>> so when you use it with LVM or MD it
>> can oops unless you use the larger kernel stacks.
>
> Anything can oops in combination with enough stacked device drivers
> (although there has been block layer work to resolve this recently,
> so you should try again with a current kernel...).  If you have an
> actual example of this still happening, please open a bug or at least
> let the XFS developers know of your test case.  Thanks.

That was actually part of the problem.  There was no time, and no hardware, 
to try to reproduce the problem in the lab.  This isn't an XFS problem 
specifically, this is an Open Source problem really....If you encounter a 
bug, and you're unlucky enough to be a bit of an edge case, you better be 
prepared to pony up with hardware and mantime to diagnose and reproduce it 
or it might not get fixed.  Again though, this is common to the whole open 
source community, and not XFS, Linux, LVM, or any other project specific.

Having said that, if you can reproduce it, and get good details, the open 
source community has a far better track record of *really* fixing and 
addressing bugs than any commercial software.

>
>> We also have had
>> problems with the quota system but the details on that have faded.
>
> Seems like details of all the problems you described have faded.
> Your mail seems to me like a bit of a troll ... I guess you had a
> problem or two a couple of years ago (from searching the lists)
> and are still sore.  Can you point me to mailing list reports of
> the problems you're refering to here or bug reports you've opened
> for these issues?  I'll let you know if any of them are still
> relevent.

No, we had dozens actually.  The only ones that were really crippling were 
when XFS would suddenly unmount in the middle of the business day for no 
apparent reason.  Without details bug reports are ignored, and we couldn't 
really provide details or filesystem dumps because there was too much data, 
and we had to get it back online.  We just moved as fast as we could away 
from XFS.  It wasn't just a one day thing, or a week, there was a trail of 
crashes with XFS at the time.  Sometimes the machine was so locked up from 
XFS pulling the rug out that the console was wedged up pretty badly too.

I wanted to provide the information as a data point from the other side as 
it were not get into a pissing match with the XFS developers and community. 
XFS is still young, as is ReiserFS.  and while Reiser is a completely new 
FS and XFS has roots in IRIX and other implementations, their age is 
similar since XFS' Linux implementation is around the same age.  If the 
state has change in the last 6-12 months then so much the better.  The 
facts are that XFS during operation had many problems, and as we pulled it 
out still had many unresolved problems as we replaced it with ReiserFS. 
And Reiser has been flawless except for one problem already mentioned on 
Linux-LVM very clearly caused by an external SAN/RAID problem which EMC has 
corrected (completely as an aside -- anyone running a CX series REALLY 
needs to be on the latest code rev, you might never run into the bug, and 
i'm still not sure exactly which one we hit, there were atleast two that 
could have caused the data corruption, but if you do, it can be ugly).


The best guess that I have as to why we had such a bad time with XFS was 
the XFS+NFS interaction and possibly an old (unknown to me -- this is just 
a guess) bug that may have created some minor underlying corruption that 
the repair tools couldn't fully fix or diagnose may have caused our 
continual (seemingly random) problems.  I don't believe in really random 
problems, atleast not in computers anyway.

>
> cheers.
>
> --
> Nathan
>



--
"Genius might be described as a supreme capacity for getting its possessors
into trouble of all kinds."
-- Samuel Butler

Re: [linux-lvm] LVM onFly features

[Nathan Scott]

On Sun, Dec 11, 2005 at 06:14:39PM -0700, Michael Loftis wrote:
> --On December 12, 2005 9:15:39 AM +1100 Nathan Scott <nathans@sgi.com> 
> The worst problems we had we're likely most strongly related to running out 
> of journal transaction space.  When XFS was under high transaction load 

Can you define "high load" for your scenario?

> sometimes it would just hang everything syncing meta-data.  From what I 

There is no situation in which XFS will "hang everything".  A process
that is modifying the filesystem may be paused briefly waiting for space
to become available in the log, and that involves flushing the in-core
log buffers.  But only processes that need log space will be paused
waiting for that (relatively small) write to complete.  This is also not
a behaviour peculiar to XFS, and with suitable tuning in terms of mkfs/
mount/sysctl parameters, it can be completely controlled.

> understand this has supposedly been dealt with, but we were still having 
> these issues when we decommissioned the last XFS based server a year ago. 

I'd like some more information describing your workload there if
you could provide it.  Thanks.

> Another datapoint is the fact we primarily served via NFS, which XFS 
> (atleast at the time) still didn't behave great with, I never did see any 
> good answers on that as I recall.

Indeed.  Early 2.6 kernels did have XFS/NFS interaction problems,
with NFS using generation number zero as "magic", and XFS using
that as a valid gen number.  That was fixed a long time ago.

> controller, or NFS.  The fact that XFS has weird interactions with NFS at 
> all bugs me, but I don't understand the code involved well enough.  There 
> might be a decent reason.

No, there's no reason, and XFS does not have "wierd interactions"
with NFS.

> >> It also needs larger kernel stacks because
> >> of some of the really deep call trees,
> >
> > Those have been long since fixed as far as we are aware.  Do you
> > have an actual example where things can fail?
> 
> We pulled it out of production and replaced XFS with Reiser.  At the time 
> Reiser was far more mature on Linux.  XFS Linux implementation (in 

Not because of 4K stacks though surely?  That kernel option wasn't around
then I think, and the reiserfs folks have also had a bunch of work to do
in that area too.

> > Seems like details of all the problems you described have faded.
> > Your mail seems to me like a bit of a troll ... I guess you had a
> > problem or two a couple of years ago (from searching the lists)
> > and are still sore.  Can you point me to mailing list reports of
> > the problems you're refering to here or bug reports you've opened
> > for these issues?  I'll let you know if any of them are still
> > relevent.
> 
> No, we had dozens actually.  The only ones that were really crippling were 
> when XFS would suddenly unmount in the middle of the business day for no 
> apparent reason.  Without details bug reports are ignored, and we couldn't 

The NFS issue had the unfortunate side effect of causing filesystem
corruption and hence forced filesystem shutdowns would result.  There
were also bugs on that error handling path, so probably you hit two
independent XFS bugs on a pretty old kernel version.

> I wanted to provide the information as a data point from the other side as 
> it were not get into a pissing match with the XFS developers and community. 

You were claiming long-resolved issues that existed in an XFS version
from an early 2.6 kernel as still relevent.  That is quite misleading,
and doesn't provide useful information to anyone.

cheers.

-- 
Nathan

Re: [linux-lvm] LVM onFly features

[Graham Wood]

On Sat, Dec 10, 2005 at 01:14:54PM -0700, Michael Loftis wrote:
> Not a problem, a lot of people don't see the delineation right off. 
> Filesystems like VXVFS (Veritas) in their commercial implementations are 
> really LVM+Filesystem in one.  Not sure what the linux port is looking like 
> these days though.
Actually veritas have multiple products, and the FS is separate to
the LVM (E.g. there is file system and volume manager - which are
different products but can be purchased together as part of the
'foundation suite').

> ReiserFS has hot expansion capabilities, but no (yet?) hot shrinking 
> capabilities.  One of the reasons it has these features and ext2/3 does not 
> is because ext2/3 are very old filesystems designed on a different 
> mentality of a static filesystem.  On-line expansion of ext2 based 
> filesystems is an extremely complicated venture, it might honestly even be 
> impossible.
Um.  I've got a couple of RHEL4 boxes at work, and redhat ship
'ext2online' with the platform - that has grown my ext3 filesystems
multiple times.  One of the volumes I've got has been grown 5 times
since the machine was last rebooted - and seems to be fine.


(apologies for the offtopic discussion, but want to make sure that
people leave with accurate answers *grin*)

Re: [linux-lvm] LVM onFly features

[Stephen C. Tweedie]

Hi,

On Sat, 2005-12-10 at 13:14 -0700, Michael Loftis wrote:

> ReiserFS has hot expansion capabilities, but no (yet?) hot shrinking 
> capabilities.  One of the reasons it has these features and ext2/3 does not 
> is because ext2/3 are very old filesystems designed on a different 
> mentality of a static filesystem.  On-line expansion of ext2 based 
> filesystems is an extremely complicated venture, it might honestly even be 
> impossible.

I guess that the code that went into fs/ext3/resize.c last year must
have been in my imagination, then. :-)  And the 

        # lvextend -L+20g /dev/ext/backup
        # ext2online /disk/backup
        
that I did 2 days ago to add another 20G to the mounted backup volume
must have been a dream... (I've used these same commands while the
backup was actively in progress in the past, too.)

Seriously, it's really no big deal to grow ext2/3 filesystems, with one
exception --- there's a single data structure, the group descriptor
table, that we're saddled with for backwards compatibility purposes
which needs to grow in-place when we add new block groups to the fs. 

Andreas Dilger did work a while ago to add pre-allocation for that
space; mkfs with "-O resize_inode" and a new hidden inode is created
with space reserved for the group descriptor table to grow into.  After
that, online resize has no trouble with ext2 on-disk format issues.

> ReiserFS has the advantage here because it doesn't necessarily pre-write 
> out a lot of filesystem meta-information (superblocks, inodes, bitmaps, 
> etc)

ext2/3 writes that metadata out in discrete block groups, so adding new
block groups to grow a fs is really very simple.  

> Ext2 resizing requires actually rewriting a lot 
> of filesystem metadata.

No; it only requires adding new metadata.  Even that troublesome group
descriptor table only needs new entries added, not existing ones
modified.  And once all the new metadata is written, a single write to
the superblock's number-of-block-groups field enables all the new space
atomically.

--Stephen