[linux-lvm] HUGE LVM log file...

[linux-lvm] HUGE LVM log file...

[Guennadi Liakhovetski]

Hello

This is a (rather) urgent issue, so, I am posting it to the 2 lists,
although, it might not be a bug.

My /var/log/lvm has grown up to 800MB+... and has filled up the disk...
well, almost... Since it's ext2, there's still some space, only available
for the root, and it is currently filling up that space too... What do I
do with it? Is it ok to chop off its head / remove it, do I need it at
all, how do I limit its size without a need to do it manually every time
(except cron-job)?

Please, help!

Thanks
Guennadi
___

Dr. Guennadi V. Liakhovetski
Department of Applied Mathematics
University of Sheffield, Sheffield, U.K.
email: g.liakhovetski@sheffield.ac.uk

Re: [linux-lvm] HUGE LVM log file...

[Heinz J. Mauelshagen]

On Tue, Jul 17, 2001 at 12:13:36PM +0100, Guennadi Liakhovetski wrote:
> Hello
> 
> This is a (rather) urgent issue, so, I am posting it to the 2 lists,
> although, it might not be a bug.
> 
> My /var/log/lvm has grown up to 800MB+... and has filled up the disk...
> well, almost... Since it's ext2, there's still some space, only available
> for the root, and it is currently filling up that space too... What do I
> do with it?

Remove it.
lvmsadc is probably filling it run by cron.

> Is it ok to chop off its head / remove it, do I need it at
> all, how do I limit its size without a need to do it manually every time
> (except cron-job)?

Remove lvmsadc from the crontab in case you don't like/need the log anyway.

> 
> Please, help!
> 
> Thanks
> Guennadi
> ___
> 
> Dr. Guennadi V. Liakhovetski
> Department of Applied Mathematics
> University of Sheffield, Sheffield, U.K.
> email: g.liakhovetski@sheffield.ac.uk
> 
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@sistina.com
> http://lists.sistina.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

-- 

Regards,
Heinz    -- The LVM Guy --

*** Software bugs are stupid.
    Nevertheless it needs not so stupid people to solve them ***

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Heinz Mauelshagen                                 Sistina Software Inc.
Senior Consultant/Developer                       Am Sonnenhang 11
                                                  56242 Marienrachdorf
                                                  Germany
Mauelshagen@Sistina.com                           +49 2626 141200
                                                       FAX 924446
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Re: [linux-lvm] HUGE LVM log file...

[Guennadi Liakhovetski]

> > My /var/log/lvm has grown up to 800MB+... and has filled up the disk...
>
> Remove it.
> lvmsadc is probably filling it run by cron.

Thanks! But why / what for is it needed and why is it created by default
without any size control?

Guennadi
___

Dr. Guennadi V. Liakhovetski
Department of Applied Mathematics
University of Sheffield, Sheffield, U.K.
email: g.liakhovetski@sheffield.ac.uk

Re: [linux-lvm] HUGE LVM log file...

[Heinz J. Mauelshagen]

On Tue, Jul 17, 2001 at 12:28:07PM +0100, Guennadi Liakhovetski wrote:
> > > My /var/log/lvm has grown up to 800MB+... and has filled up the disk...
> >
> > Remove it.
> > lvmsadc is probably filling it run by cron.
> 
> Thanks! But why / what for is it needed and why is it created by default
> without any size control?

It is not created/activated by default.
Someone must have put it into crontab.

But you are right: there's no size control so far.

> 
> Guennadi
> ___
> 
> Dr. Guennadi V. Liakhovetski
> Department of Applied Mathematics
> University of Sheffield, Sheffield, U.K.
> email: g.liakhovetski@sheffield.ac.uk
> 
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@sistina.com
> http://lists.sistina.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

-- 

Regards,
Heinz    -- The LVM Guy --

*** Software bugs are stupid.
    Nevertheless it needs not so stupid people to solve them ***

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Heinz Mauelshagen                                 Sistina Software Inc.
Senior Consultant/Developer                       Am Sonnenhang 11
                                                  56242 Marienrachdorf
                                                  Germany
Mauelshagen@Sistina.com                           +49 2626 141200
                                                       FAX 924446
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Re: [linux-lvm] HUGE LVM log file...

[Wolfgang Weisselberg]

Guennadi Liakhovetski (g.liakhovetski@ragingbull.com) wrote 23 lines:

> > > My /var/log/lvm has grown up to 800MB+... and has filled up the disk...

> > lvmsadc is probably filling it run by cron.

> Thanks! But why / what for is it needed and why is it created by default

man lvmsadc: 
       lvmsadc collects the read/write statistics of the  logical
       volume  manager  to the file "LogFilePath" or to stdout if
       no LogFilePath is given.  You should create a  cron  entry
       for lvmsadc which runs it at regular intervals eg. 10 min�
       utes.

That statistical info can be later used to e.g. decide which
LEs shoud have PEs at the faster 'begin' (outer edge) of the
HD.  An automatic LE mover program has not been written yet.

-Wolfgang

RE: [linux-lvm] HUGE LVM log file...

[Gonyou, Austin]

Is there a how-to or cookbook type doc which tells how/why you should setup
your LE/PE's? Outter tracks/inner tracks, speed areas, etc?
Theory around this is a good thing with relation to LVM and using it. Any
ideas on that?

-- 
Austin Gonyou
Systems Architect, CCNA
Coremetrics, Inc.
Phone: 512-796-9023
email: austin@coremetrics.com 

> -----Original Message-----
> From: Wolfgang Weisselberg [mailto:weissel@netcologne.de]
> Sent: Tuesday, July 17, 2001 4:29 PM
> To: linux-lvm@sistina.com
> Subject: Re: [linux-lvm] HUGE LVM log file...
> 
> 
> Guennadi Liakhovetski (g.liakhovetski@ragingbull.com) wrote 23 lines:
> 
> > > > My /var/log/lvm has grown up to 800MB+... and has 
> filled up the disk...
> 
> > > lvmsadc is probably filling it run by cron.
> 
> > Thanks! But why / what for is it needed and why is it 
> created by default
> 
> man lvmsadc: 
>        lvmsadc collects the read/write statistics of the  logical
>        volume  manager  to the file "LogFilePath" or to stdout if
>        no LogFilePath is given.  You should create a  cron  entry
>        for lvmsadc which runs it at regular intervals eg. 10 min­
>        utes.
> 
> That statistical info can be later used to e.g. decide which
> LEs shoud have PEs at the faster 'begin' (outer edge) of the
> HD.  An automatic LE mover program has not been written yet.
> 
> -Wolfgang
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@sistina.com
> http://lists.sistina.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html
> 

Re: [linux-lvm] HUGE LVM log file...

[Wolfgang Weisselberg]

Gonyou, Austin (austin@coremetrics.com) wrote 53 lines:

> Is there a how-to or cookbook type doc which tells how/why you should setup
> your LE/PE's? Outter tracks/inner tracks, speed areas, etc?

Just use it, unless your disk is a definite bottleneck, don't
worry about that.  That's the whole idea of LVM: don't worry;
just adjust the sizes as you need.

On the other hand I'd put a small(ish) /boot (or the complete /
) close to the front of the disk (lilo might still not like
high cylinders on all computers), put swap not on LVM and
close to the front of the HD[1].  You can always add swap
files/partitions on the fly, if you need them.

If you need to plan something, put the data/programs which are
small and heavily accessed close to the front, where swap is.
Slower data can have the rest.  But again, unless you have
*good* data, you are probably not only guessing wrong, but
also optimising at the completely wrong place.

[1] LVM is a tad slower than non-LVM[2].  And you want your
    primary swap to be FAST.  If you have multible fast disks,
    give each of them a swapspace.  By giving them all the same
    priority (see man swapon for more info) the kernel will use
    them just like a stripped volume, speeding it up further.

[2] it has to calculate the PE from the LE, which takes some
    small time

> Theory around this is a good thing with relation to LVM and using it. Any
> ideas on that?

Yes: Cluster the PEs of your most accessed LEs (minimize head
movement and thus seek time), spread them out evenly over all
your _fast_ disks (independent reading, independent -- thus
faster -- seeking, smaller 'most accessed' PE clusters == less
head movement == faster seeking) and move the PEs of your most
accessed LEs to the front of the disks (faster reading/writing).

And most important: Don't fix/tune it unless it's broken/a
proven and measured bottleneck.

-Wolfgang

Re: [linux-lvm] HUGE LVM log file...

[Joe Thornber]

On Tue, Jul 17, 2001 at 04:52:21PM -0500, Gonyou, Austin wrote:
> Is there a how-to or cookbook type doc which tells how/why you should setup
> your LE/PE's? Outter tracks/inner tracks, speed areas, etc?
> Theory around this is a good thing with relation to LVM and using it. Any
> ideas on that?

We've not had much feedback on this, I guess that people don't really
notice much of a difference, I certainly haven't.  The allocation
policy for PE's is to start at the beginning of the disk (which tend
to have faster data transfer rates) and work back.  Anybody done any
benchmarking in this area ?

- Joe

Re: [linux-lvm] HUGE LVM log file...

[Benoit SERRA]

At 18/07/01, you wrote:
>We've not had much feedback on this, I guess that people don't really
>notice much of a difference, I certainly haven't.  The allocation
>policy for PE's is to start at the beginning of the disk (which tend
>to have faster data transfer rates) and work back.  Anybody done any
>benchmarking in this area ?
>

Are you sure ?

In AIX, the system create PPS (PE) from the middle of the disk for 
performance reasons.

I heard on this list or on Linux-raid list that it's the middle of the disk 
which is the faster area.

Re: [linux-lvm] HUGE LVM log file...

[Joe Thornber]

On Wed, Jul 18, 2001 at 10:48:55AM +0200, Benoit SERRA wrote:
> At 18/07/01, you wrote:
> >We've not had much feedback on this, I guess that people don't really
> >notice much of a difference, I certainly haven't.  The allocation
> >policy for PE's is to start at the beginning of the disk (which tend
> >to have faster data transfer rates) and work back.  Anybody done any
> >benchmarking in this area ?
> >
> 
> Are you sure ?

No.

I'm just talking transfer rate here, and am just going on the typical
benchmark graphs you see in reviews. eg,

http://www6.tomshardware.com/storage/01q2/010614/barracuda-05.html#data_transfer_diagram

This comment is interesting:

"Seagate uses horizontal mapping with this drive, so that it reaches
maximum performance at the beginning of the medium, decreasing at the
end of the medium."

So obviously this varies from drive to drive.

> In AIX, the system create PPS (PE) from the middle of the disk for 
> performance reasons.

If people really think this would be of benefit we can always provide
another allocator.  Does it allocate to alternate sides of the middle ?

eg, middle, middle + 1, middle - 1, middle + 2 etc ... ?

If so you have to take the extra seek times into account if you're streaming
files that are bigger than the PE size.

This can also fragment your PV; if you create the first LV to take up
half the PV, and it puts this in the middle, a subsequent LV of the
same size ends up having a monster seek in the middle of it - though
it depends on your filesystem if this is an issue.
 
> I heard on this list or on Linux-raid list that it's the middle of the disk 
> which is the faster area.

Depends what you mean by 'faster', data transfer rate or seek time ?

- Joe

Re: [linux-lvm] HUGE LVM log file...

[Adam Cioccarelli]

actually on AIX you can choose where the LV starts using PPs from

	edge
	middle
	center
	inner-middle
	inner-edge

middle is the default but I think this is more so the average performance
of the LVs are better, but maybe not...

-------------------------------------------------------------------------------
Adam Cioccarelli (B.E Mechanical)                Adam.Cioccarelli@ecetra.com
Database Administrator			         Phone: +43 1 536 89 7725
						 Fax:   +43 1 536 89 7719
ecetra Central European e-Finance AG             Mobile:+43 664 181 4195
-------------------------------------------------------------------------------

On Wed, 18 Jul 2001, Benoit SERRA wrote:

> At 18/07/01, you wrote:
> >We've not had much feedback on this, I guess that people don't really
> >notice much of a difference, I certainly haven't.  The allocation
> >policy for PE's is to start at the beginning of the disk (which tend
> >to have faster data transfer rates) and work back.  Anybody done any
> >benchmarking in this area ?
> >
>
> Are you sure ?
>
> In AIX, the system create PPS (PE) from the middle of the disk for
> performance reasons.
>
> I heard on this list or on Linux-raid list that it's the middle of the disk
> which is the faster area.
>
>
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@sistina.com
> http://lists.sistina.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html
>

Re: [linux-lvm] HUGE LVM log file...

[Benoit SERRA]

At 18/07/01, you wrote:

>actually on AIX you can choose where the LV starts using PPs from
>
>         edge
>         middle
>         center
>         inner-middle
>         inner-edge
>
>middle is the default but I think this is more so the average performance
>of the LVs are better, but maybe not...

I think that middle is the default because IBM drives are more performants 
in the middle.

I never use another brand of disks in a RS/6000 box so I can't tell you 
more....

[linux-lvm] what happens when hdX get hdY in a VG

[Oliver Jovic]

Hi, :-)

well what happens when i have a VG build out of /dev/hd[e-h]1 (4
harddrives) and build in now a additional IDE controller and 4
additional harddrives and this new IDE controller wants these
driverletters now, so /dev/hde gets /dev/hdi and so on? Will a vgscan
in the startupscripts detect the right drives anyways due to the headers
of each harddisk and build the VG/LV as it was before just with different
driveletters?

If not, what do i need to do? Havent found much info about it, but well
must be also be honest, hadnt much time to look for it. ;-/ Sorry if
somebody asked this already.

Thanks in advance.

Oli

Re: [linux-lvm] what happens when hdX get hdY in a VG

[Joe Thornber]

On Wed, Jul 18, 2001 at 12:48:09PM +0200, Oliver Jovic wrote:
> Hi, :-)
> 
> well what happens when i have a VG build out of /dev/hd[e-h]1 (4
> harddrives) and build in now a additional IDE controller and 4
> additional harddrives and this new IDE controller wants these
> driverletters now, so /dev/hde gets /dev/hdi and so on? Will a vgscan
> in the startupscripts detect the right drives anyways due to the headers
> of each harddisk and build the VG/LV as it was before just with different
> driveletters?
> 
> If not, what do i need to do? Havent found much info about it, but well
> must be also be honest, hadnt much time to look for it. ;-/ Sorry if
> somebody asked this already.

vgscan should do the right thing.

- Joe

Re: [linux-lvm] HUGE LVM log file...

[Iain Campbell]

On Wednesday 18 July 2001 05:47, you wrote:
> At 18/07/01, you wrote:
> >actually on AIX you can choose where the LV starts using PPs from
> >
> >         edge
> >         middle
> >         center
> >         inner-middle
> >         inner-edge
> >
> >middle is the default but I think this is more so the average performance
> >of the LVs are better, but maybe not...
>
> I think that middle is the default because IBM drives are more performants
> in the middle.
>
> I never use another brand of disks in a RS/6000 box so I can't tell you
> more....
>

Actually, it's not as elegant as that. AIX simply divides the total number of 
PEs (or to use the AIX terminology, PPs) on the PV by five, thus allocating 
one fifth of the total drive capacity into each of the five disk "areas" 
mentioned in the earlier post. The problem is that there are more blocks on 
the outer edge of the platter than the inner (geometry 101 ;), so what AIX 
calls the "middle" of the platter really is more like one third or so from 
the outer edge. 

The theory is that data placed exactly in the middle of the platter has the 
lowest average seek time, and AIX simplistically calls this the "fastest" 
area of the disk. In fact the outer edge has the greatest peripheral speed 
and gives the fastest data transfer rate. So, is "speed" to you seek time or 
data throughput?

Furthermore, if you have alook at the thread elsewhere in this group on disk 
write ordering (64 bit scsi read/write) you begin to get a feel for the weird 
and wonderful things drives do with data in hardware, which may mask alot of 
the physical factors of where the data is  actually laid out on disk.

Finally, of course, for files that arre read/written repeatedly, the o/s 
buffer cache has a significant blurring effect on where the data may actually 
be written on disk.

iain 

Re: [linux-lvm] HUGE LVM log file...

[Heinz J. Mauelshagen]

On Tue, Jul 17, 2001 at 11:28:42PM +0200, Wolfgang Weisselberg wrote:
> Guennadi Liakhovetski (g.liakhovetski@ragingbull.com) wrote 23 lines:
> 
> > > > My /var/log/lvm has grown up to 800MB+... and has filled up the disk...
> 
> > > lvmsadc is probably filling it run by cron.
> 
> > Thanks! But why / what for is it needed and why is it created by default
> 
> man lvmsadc: 
>        lvmsadc collects the read/write statistics of the  logical
>        volume  manager  to the file "LogFilePath" or to stdout if
>        no LogFilePath is given.  You should create a  cron  entry
>        for lvmsadc which runs it at regular intervals eg. 10 min�
>        utes.

Well, it should rather say: "If you want to have I/O statistics at hand
for performance tuning, you could create a cron entry..."

Will change that.

> 
> That statistical info can be later used to e.g. decide which
> LEs shoud have PEs at the faster 'begin' (outer edge) of the
> HD.  An automatic LE mover program has not been written yet.

Well, it depends on the brand/model, if the outer edge is the fast area :-)

> 
> -Wolfgang
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@sistina.com
> http://lists.sistina.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

-- 

Regards,
Heinz    -- The LVM Guy --

*** Software bugs are stupid.
    Nevertheless it needs not so stupid people to solve them ***

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Heinz Mauelshagen                                 Sistina Software Inc.
Senior Consultant/Developer                       Am Sonnenhang 11
                                                  56242 Marienrachdorf
                                                  Germany
Mauelshagen@Sistina.com                           +49 2626 141200
                                                       FAX 924446
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Re: [linux-lvm] HUGE LVM log file...

[Steven Lembark]

> In AIX, the system create PPS (PE) from the middle of the disk for 
> performance reasons.

And systematically fry their performance in some cases.


The theory behind middle-tracks is reducing butterfly reads.
Eventually, however, the drive gets full and you end up with
butterflys anyway.

With newer (last 5 years+) drives the outer cyl's have more
sectors to keep the bit density constant.  This leaves you with
much more storate at the outside edge, which reduces seeks
which is faster (especially since at least SCSI drives normally
do cyl buffering).

For even finer control you can create fast/med/slow part's 
on the drive and then use them as PV's in different VG's.
Used carefully this can be a serious performance boost for
high-I/O systems.

sl

Re: [linux-lvm] HUGE LVM log file...

[Ragnar Kjørstad]

On Wed, Jul 18, 2001 at 10:33:21AM +0100, Joe Thornber wrote:
> This comment is interesting:
> 
> "Seagate uses horizontal mapping with this drive, so that it reaches
> maximum performance at the beginning of the medium, decreasing at the
> end of the medium."
> 
> So obviously this varies from drive to drive.

You can use zcav from the bonnie++ package
(http://www.coker.com.au/bonnie++) to find the transfer-rates for your
drive. I believe finding seek times is a little more tricky since it
depends on where you're seeking from and that will depend on your
application.



-- 
Ragnar Kjorstad
Big Storage

Re: [linux-lvm] HUGE LVM log file...

[lembark]

-- Ragnar Kjørstad <lvm@ragnark.vestdata.no> on 07/18/01 21:23:01 +0200

> On Wed, Jul 18, 2001 at 10:33:21AM +0100, Joe Thornber wrote:
>> This comment is interesting:
>> >> "Seagate uses horizontal mapping with this drive, so that it reaches
>> maximum performance at the beginning of the medium, decreasing at the
>> end of the medium."
>> >> So obviously this varies from drive to drive.
> > You can use zcav from the bonnie++ package
> (http://www.coker.com.au/bonnie++) to find the transfer-rates for your
> drive. I believe finding seek times is a little more tricky since it
> depends on where you're seeking from and that will depend on your
> application.

It is particularly tricky on SCSI's since they remap
bad blocks internally.  You can end up with a butterfly
read due to bad blocks without knowig it.  SCSI's and
the controllers can also silently do scatter/gather on
I/O which helps performance but can cause serious pain for benchmarks.

:-)