re: [linux-lvm] EVMS

re: [linux-lvm] EVMS

[Greg Freemyer]

I know that the latest xfs release (March) does not support EVMS with RAID 5 sets.

There may be other problems with the pairing as well.

Both the EVMS and XFS teams are working to get this supported in the next release.  (That looks to be a couple of months away.)

 >>  Hi all,

 >>  I stumbled across something from IBM called EVMS today and was 
 >>  wondering if anyone had used it, and if so, what they thought of it?
 >>  How does it differ from LVM?

 >>  Here's the URL:

 >>      http://oss.software.ibm.com/developerworks/opensource/evms

 >>  It appears to be open source and is on Sourceforge as well.

 >>  Thanks for any comments,
 >>  -- 

 >>  Seeya,
 >>  Paul
 >>  --
 >>      It may look like I'm just sitting here doing nothing,
 >>  but I'm really actively waiting for all my problems to go away.

 >>       If you're not having fun, you're not doing it right!



 >>  _______________________________________________
 >>  linux-lvm mailing list
 >>  linux-lvm@sistina.com
 >>  http://lists.sistina.com/mailman/listinfo/linux-lvm
 >>  read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html




Greg Freemyer
Internet Engineer
Deployment and Integration Specialist
Compaq ASE - Tru64 v4, v5
Compaq Master ASE - SAN Architect
The Norcross Group
www.NorcrossGroup.com

Re: [linux-lvm] EVMS

[Paul Lussier]

In a message dated: Tue, 27 Aug 2002 16:34:49 EDT
Greg Freemyer said:

>I know that the latest xfs release (March) does not support EVMS with RAID 5 =
>sets.
>
>There may be other problems with the pairing as well.
>
>Both the EVMS and XFS teams are working to get this supported in the next =
>release.  (That looks to be a couple of months away.)

Well, currently I'm not using XFS, I'm mucking around with ext3 and 
ReiserFS.  Though it's good to know XFS isn't an option right now
(I did like XFS back when I was playing with it, and will probably go 
back that way sometime :)

Thanks!
-- 

Seeya,
Paul
--
	It may look like I'm just sitting here doing nothing,
   but I'm really actively waiting for all my problems to go away.

	 If you're not having fun, you're not doing it right!

re: [linux-lvm] Re: LVM writes on raw disk of ATA RAID Mirror

[Greg Freemyer]

 >>  Now y'all can tell me that the Promise ATA RAID cards suck and I
 >>  shouldn't use them, and I should get a hardware RAID card, and so on
 >>  (as I saw happened to the person who described this issue in October
 >>  2002; see
 >>  http://lists.sistina.com/pipermail/linux-lvm/2002-October/012508.html
 >>  and
 >>  http://lists.sistina.com/pipermail/linux-lvm/2002-October/012516.html).

 >>  And I'll happily agree with you, but for two small facts:
 >>  - the hardware RAID cards cost more than twice what the drives cost, and
 >>  they're 120GB IDE drives with large caches; let alone SCSI hardware
 >>  RAID (which also increases the cost of the disks a lot too);

Just thought I would point out that a 2-channel 3ware card is about the price 
of a single 80GB drive.  (i.e. $130 US)

I love the 3-ware cards and they are not cost prohibitive in my opinion.  My 
only problem is I wish the failure/rebuild process was more automated than it currently is.

Obviously, LVM and MD should still work together regardless.

Greg
-- 
Greg Freemyer

Re: [linux-lvm] Re: LVM writes on raw disk of ATA RAID Mirror

[Ewen McNeill]

In message <20030429220328.XULC259.imf39bis.bellsouth.net@tiger2>, Greg Freemyer
 writes:
> >>  Now y'all can tell me that the Promise ATA RAID cards suck and I
> >>  shouldn't use them, and I should get a hardware RAID card, [....]
> >>  And I'll happily agree with you, but for two small facts:
> >>  - the hardware RAID cards cost more than twice what the drives cost, and
> >>  they're 120GB IDE drives with large caches; let alone SCSI hardware
> >>  RAID (which also increases the cost of the disks a lot too);
>
>Just thought I would point out that a 2-channel 3ware card is about the price
>of a single 80GB drive.  (i.e. $130 US)

Around here (New Zealand) I can buy a 120GB IDE disk, with a 3 year
warrenty, and a 8MB cache, for that sort of price (NZ$280 -> US$140 ish).

Still, that's a lot closer to the price of the Promise cards (I'd
previously only seen the 3-ware cards with prices around the US$250 ish
sort of mark).   Is that for the 3Ware 7000-2 card, which appears to be
a 2-drive (single IDE channel?), card as best I can tell?  Or for the
3Ware 7500-4LP card which appears to be a 4-drive (2 IDE channel?) card?
(The Promise TX2000, and most of the Promise on-board ata-raid chipsets,
are 2 IDE channels, 4 drives max -- although of course for best
performance you really only want one drive per IDE channel.)

Now all I need to find is a source of 3ware IDE cards in New Zealand;
I haven't found any retail sources so far, and the one distributor I
found mentioned doesn't even have a website in New Zealand (it redirects
to a flash-only overseas website).

Thanks for the comments.

>Obviously, LVM and MD should still work together regardless.

FWIW, my problem is with LVM and ata-raid (as implemented by various
Promise cards).  I haven't been able to determine whether LVM and MD
(Linux software RAID) will work together or not -- although based on
what I found I'd be somewhat surprised if they do work together
correctly, because it seems to be an accident rather than good design
(the same vgscan rescan issues would seem to affect MD as affect
software RAID, unless some lucky coincidence prevents the raw IDE
channels being recognised).

Perhaps making LVM and ataraid work together properly is "too hard"
(although personally I think it can be done with merely an ugly kludge
-- viz, scan the ataraid and md devices first).

But IMHO either LVM and ataraid should work properly, or it should be
really really obvious that they won't work together (eg, fails to
initialise on ataraid devices, reports errors if you try, etc).  The
current situation of mentioning ataraid support, and then silently failing 
is very dangerous.

Ewen

Re: [linux-lvm] Re: LVM writes on raw disk of ATA RAID Mirror

[Goetz Bock]

On Wed, Apr 30 '03 at 10:42, Ewen McNeill wrote:
> [ ... ]  Is that for the 3Ware 7000-2 card, which appears to be
> a 2-drive (single IDE channel?), [ ... ]
most likely for the 7000-2 card, what can only do raid0/1 and threfor
lakes the expensive raid5 engine. still i use it happily

> >Obviously, LVM and MD should still work together regardless.
> FWIW, my problem is with LVM and ata-raid [ ... ]
> Perhaps making LVM and ataraid work together properly is "too hard"
> [ ... ]
> But IMHO either LVM and ataraid should work properly, or it should be
> really really obvious that they won't work together
during my last/first/only time i had to deal with ataraid, and IIRC,
the promise IDE driver had a compile time flag where it would not touch
initialised raid arrays.
aktivating this flag would simply remove the offending disks from the
list of ide disks and make it available via ataraid only.

OTOH I might have had a too long day and my memory is all wrong.

lukily I don't have to deal with this box any more ;-) and therefor can
not check this.
-- 
Goetz Bock       (c) 2003 as     blacknet.de - Munich - Germany   /"\
IT Consultant    GNU FDL 1.1    secure mobile Linux everNETting   \ /
                                                                   X
 ASCII Ribbon Campaign against HTML email & microsoft attachments / \

Re: [linux-lvm] Re: LVM writes on raw disk of ATA RAID Mirror

[Ewen McNeill]

In message <20030430011344.N16789@zealot.blacknet.de>, Goetz Bock writes:
>during my last/first/only time i had to deal with ataraid, and IIRC,
>the promise IDE driver had a compile time flag where it would not touch
>initialised raid arrays.
>aktivating this flag would simply remove the offending disks from the
>list of ide disks and make it available via ataraid only.

It would be nice if it were possible to do this.  Is this perhaps the
CONFIG_PDC202XX_FORCE flag (turned off, or on)?  

The Linux kernel configure help for it isn't very enlightening.  And as
best I can tell it affects only the flags passed to some of the Promise
cards during setup.  But interestingly _not_ the PDC20271 which is on
the Promise TX2000 card I have.  (Or the PDC20268, PDC20269, PDC20275,
or PDC20277.  It does affect the PDC20270 and PDC20276 though, and most
of the older PDC chipsets (in pdc202xx_old.h).)

Ewen

Re: [linux-lvm] Re: LVM writes on raw disk of ATA RAID Mirror

[Ewen McNeill]

In message <20030429232352.DC21EAE4F5@basilica.la.naos.co.nz>, Ewen McNeill writes:
>In message <20030430011344.N16789@zealot.blacknet.de>, Goetz Bock writes:
>>during my last/first/only time i had to deal with ataraid, and IIRC,
>>the promise IDE driver had a compile time flag where it would not touch
>>initialised raid arrays.
>>aktivating this flag would simply remove the offending disks from the
>>list of ide disks and make it available via ataraid only.
>
>It would be nice if it were possible to do this.  Is this perhaps the
>CONFIG_PDC202XX_FORCE flag (turned off, or on)?  

According to this message:

http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&selm=20030405112008%2462da%40gated-at.bofh.it

from Andre Hedrick (Linux IDE maintainer), the CONFIG_PDC202XX_FORCE
option is intended to force recognition of the Promise card's IDE
channels as IDE devices, in order to be able to use the Linux pdcraid
(ATA-RAID) driver.  (They are apparently semi-disabled in the PCI
configuration data, by default, so that only the Promise SCSI device
driver can find them.)  It should be on for using the pdcraid/ata-raid
support, and off if you are using the Promise "SCSI" device driver.
(And as I noted earlier, only certain chipsets seem to actually use this
option; the rest just default to forced on.)

This tends to confirm my view that if you use the ataraid driver, then
you'll see both the raw /dev/hd* devices, and also the /dev/ataraid/d0*
devices, since the ataraid/pdcraid driver appears to search the /dev/hd*
IDE controllers for likely devices based on partition signatures.

Which means that LVM needs to deal with the aliasing effects in some
manner, either by recognising the ataraid devices first and ignoring the
underlying devices, or by very loudly refusing to work with ataraid at
all.

Ewen

Re: [linux-lvm] Re: LVM writes on raw disk of ATA RAID Mirror

[Andrew Rechenberg]

vgscan has similar problems with MD devices in a software RAID10 (but
not RAID1) configuration.

In software RAID10 one creates a number of software RAID1 devices and
then stripes across those with RAID0.  For example if you have a 4 disk
RAID10 setup you would have /dev/md0 and /dev/md1 as RAID1 devices and
then /dev/md2 would be a stripe across /dev/md[0,1]

The problem occurs when vgcreate adds the LVM metadata to the device. 
When you run 'vgcreate somename0 /dev/md2' it adds the metadata to
/dev/md2, but since /dev/md0 is the first device in the RAID10 array,
the metadata is in the exact same location on /dev/md0 and /dev/md2. 
Upon reboot (or a subsequent vgscan), LVM complains that it can't find
all of the PE's on the volume group created because it's reading the
information from md0 instead of md2

I was trying to hack vgscan to get my setup working but I needed the box
working ASAP so time dictated that I use the KludgeMethod(tm).  Someone
on the list recommended that I move /dev/md0 before I run my vgscan. 
That worked in my case and I hacked my sysint script to move md0, run
vgscan, and then move it back.  Not elegant AT ALL, but it works.

I'm not sure if a similar quick fix could work for you or not.  You
would have to make sure that the ataraid driver "built" the
/dev/ataraid* entries, then move /dev/hd* somewhere, run vgscan, and
then move them back.

Make sure you test first and have a backup (do people still do those? :)
) before trying anything.

HTH,
Andy.


On Tue, 2003-04-29 at 20:30, Ewen McNeill wrote:
> In message <20030429232352.DC21EAE4F5@basilica.la.naos.co.nz>, Ewen McNeill writes:
> >In message <20030430011344.N16789@zealot.blacknet.de>, Goetz Bock writes:
> >>during my last/first/only time i had to deal with ataraid, and IIRC,
> >>the promise IDE driver had a compile time flag where it would not touch
> >>initialised raid arrays.
> >>aktivating this flag would simply remove the offending disks from the
> >>list of ide disks and make it available via ataraid only.
> >
> >It would be nice if it were possible to do this.  Is this perhaps the
> >CONFIG_PDC202XX_FORCE flag (turned off, or on)?  
> 
> According to this message:
> 
> http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&selm=20030405112008%2462da%40gated-at.bofh.it
> 
> from Andre Hedrick (Linux IDE maintainer), the CONFIG_PDC202XX_FORCE
> option is intended to force recognition of the Promise card's IDE
> channels as IDE devices, in order to be able to use the Linux pdcraid
> (ATA-RAID) driver.  (They are apparently semi-disabled in the PCI
> configuration data, by default, so that only the Promise SCSI device
> driver can find them.)  It should be on for using the pdcraid/ata-raid
> support, and off if you are using the Promise "SCSI" device driver.
> (And as I noted earlier, only certain chipsets seem to actually use this
> option; the rest just default to forced on.)
> 
> This tends to confirm my view that if you use the ataraid driver, then
> you'll see both the raw /dev/hd* devices, and also the /dev/ataraid/d0*
> devices, since the ataraid/pdcraid driver appears to search the /dev/hd*
> IDE controllers for likely devices based on partition signatures.
> 
> Which means that LVM needs to deal with the aliasing effects in some
> manner, either by recognising the ataraid devices first and ignoring the
> underlying devices, or by very loudly refusing to work with ataraid at
> all.
> 
> Ewen
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@sistina.com
> http://lists.sistina.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Re: LVM writes on raw disk of ATA RAID Mirror

[Ewen McNeill]

In message <1051724087.19901.287.camel@cinshrlnxws01.shermfin.com>, Andrew Reche
nberg writes:
>>[LVM 1.0.x; vgscan finds parts of RAID arrays and uses those instead]
>vgscan has similar problems with MD devices in a software RAID10 (but
>not RAID1) configuration. [...]

Interesting.  I've been told, and found in the source, that there were
some hacks in LVM to make LVM and md work in some common situations
(basically there's a loop after the first device scan which tries to
eliminate some of the duplicates; and there's a /* FIXME */ comment
immediately before that loop).  It seems that you've found one of
the situations where these hacks are too limited.

>I was trying to hack vgscan to get my setup working but I needed the box
>working ASAP so time dictated that I use the KludgeMethod(tm).  Someone
>on the list recommended that I move /dev/md0 before I run my vgscan. 
>[....]

That is really quite ugly :-)  I suspect something like that might work
for the ataraid case too, right up to the point that someone runs vgscan
for some reason without going through the "hide things from vgscan so it
doesn't get it wrong" ritual -- at which point (with a little prompting)
it silently swaps over to using part of the RAID array, and the RAID
array gets out of sync, and then on next reboot it swaps back to using
the RAID array, and the partitions are corrupt.

I'm afraid that's a little too much potential excitement for my liking!

I've been told that LVM 2 has some support where you can say what
devices to scan (and what devices not to scan).  I've not looked at LVM
2 yet, so I don't know how fine grained it is, but it might suit your
situation.

However after all this investigation I can't help thinking that the LVM
vgscan approach is broken by design, particularly to be run automatically
on startup.  The idea that one can somehow look through all the connected
devices and guess which one to use, and then automatically use those
guesses on the assumption they'll always be right, just seems to be asking
for trouble.  Static configuration files, and having these things under
manual control, seems a far more reliable way to approach the situation.

For now this ataraid system will be built without LVM, as it seems the
only way to be sure that the RAID array will actually always be used.

Thanks for your comments,

Ewen

Re: [linux-lvm] Re: LVM writes on raw disk of ATA RAID Mirror

[Alasdair G Kergon]

On Thu, May 01, 2003 at 07:30:30AM +1200, Ewen McNeill wrote:
> I've been told that LVM 2 has some support where you can say what
> devices to scan (and what devices not to scan).  

LVM2 offers selection by device type (i.e. the names that appear in
/proc/devices giving block device major numbers) and regular-expression 
based selection by device name.

Alasdair
-- 
agk@uk.sistina.com

Re: [linux-lvm] Re: LVM writes on raw disk of ATA RAID Mirror

[Andrew Rechenberg]

I forgot to mention that I move the md0 entry back into place after I
run vgscan.  It's worked quite well and across numerous reboots. 

It is VERY ugly, but it "Works For Me(TM)" :)

On Wed, 2003-04-30 at 15:30, Ewen McNeill wrote:
> In message <1051724087.19901.287.camel@cinshrlnxws01.shermfin.com>, Andrew Reche
> nberg writes:
> >>[LVM 1.0.x; vgscan finds parts of RAID arrays and uses those instead]
> >vgscan has similar problems with MD devices in a software RAID10 (but
> >not RAID1) configuration. [...]
> 
> Interesting.  I've been told, and found in the source, that there were
> some hacks in LVM to make LVM and md work in some common situations
> (basically there's a loop after the first device scan which tries to
> eliminate some of the duplicates; and there's a /* FIXME */ comment
> immediately before that loop).  It seems that you've found one of
> the situations where these hacks are too limited.
> 
> >I was trying to hack vgscan to get my setup working but I needed the box
> >working ASAP so time dictated that I use the KludgeMethod(tm).  Someone
> >on the list recommended that I move /dev/md0 before I run my vgscan. 
> >[....]
> 
> That is really quite ugly :-)  I suspect something like that might work
> for the ataraid case too, right up to the point that someone runs vgscan
> for some reason without going through the "hide things from vgscan so it
> doesn't get it wrong" ritual -- at which point (with a little prompting)
> it silently swaps over to using part of the RAID array, and the RAID
> array gets out of sync, and then on next reboot it swaps back to using
> the RAID array, and the partitions are corrupt.
> 
> I'm afraid that's a little too much potential excitement for my liking!
> 
> I've been told that LVM 2 has some support where you can say what
> devices to scan (and what devices not to scan).  I've not looked at LVM
> 2 yet, so I don't know how fine grained it is, but it might suit your
> situation.
> 
> However after all this investigation I can't help thinking that the LVM
> vgscan approach is broken by design, particularly to be run automatically
> on startup.  The idea that one can somehow look through all the connected
> devices and guess which one to use, and then automatically use those
> guesses on the assumption they'll always be right, just seems to be asking
> for trouble.  Static configuration files, and having these things under
> manual control, seems a far more reliable way to approach the situation.
> 
> For now this ataraid system will be built without LVM, as it seems the
> only way to be sure that the RAID array will actually always be used.
> 
> Thanks for your comments,
> 
> Ewen
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@sistina.com
> http://lists.sistina.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
-- 
Andrew Rechenberg <arechenberg@shermfin.com>
Infrastrucutre Team, Sherman Financial Group

[linux-lvm] LVM, GFS, and clustered mail servers

[Scott Parish]

This is a little off the typical discussion found on the list, and I apologize
in advance for the slight interruption.

I'm looking for anyone who has used LVM and Sistina's GFS to implement a
clustered mail server environment using commodity PCs.   I use the term cluster
here, somewhat loosely, to describe multiple machines providing the same or
similar service(s), in this case mail services (pop, imap, webmail, smtp).  I'm
particularly interested in your use of LVM/GFS in implementing a shared mail
store/filesystem. 

If you have such a setup, and could spare a few moments, I'd like to pick your
brain on your cluster configuration and experiences.  Feel free to reply to the
list or directly.

Some of the things I'd like to know are:

Describe your config, number of machines, adapters, disk subsystems
(manufacturer would be helpful for adapters and disk subsystem).

Software you're using to provide mail services?  (sendmail, qmail, cyrus,
courier, squirrelmail, etc.)

How many users do you serve?

How did you decide to distribute workload?  RR DNS?  Split services among
machines?  Linux Virtual Server?  Something else?

Do you share more than the mail filesystem among machines?  If so, what?

Do you have any performance numbers?

What snags did you hit?

If you had to implement again, which would you choose:  "big box" or the
cluster?

Anything else you'd like to volunteer.

Thanks in advance,
Scott

--
Scott Parish, Systems Administrator, Pittsburg State University
Thinking is the hardest work there is, which is probably the 
reason why so few engage in it.  -  Henry Ford