Which is better at vm, and why? 2.2 or 2.4

Which is better at vm, and why? 2.2 or 2.4

[Patrick McFarland]

[-- Attachment #1: Type: text/plain, Size: 213 bytes --]

Now, the great kernel hacker, ac, said that 2.2 is better at vm in low memory situations than 2.4 is. Why is this? Why hasnt someone fixed the 2.4 code?

-- 
Patrick "Diablo-D3" McFarland || unknown@panax.com

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Which is better at vm, and why? 2.2 or 2.4

[Alan Cox]

> Now, the great kernel hacker, ac, said that 2.2 is better at vm in low memo=
> ry situations than 2.4 is. Why is this? Why hasnt someone fixed the 2.4 cod=
> e?

Actually they have on thw whole. However VM tuning is a hard problem

RE: Which is better at vm, and why? 2.2 or 2.4

[M. Edward Borasky]

> -----Original Message-----
> From: linux-kernel-owner@vger.kernel.org
> [mailto:linux-kernel-owner@vger.kernel.org]On Behalf Of Alan Cox
> Sent: Saturday, October 13, 2001 10:16 AM
> To: Patrick McFarland
> Cc: linux-kernel@vger.kernel.org
> Subject: Re: Which is better at vm, and why? 2.2 or 2.4
> > Now, the great kernel hacker, ac, said that 2.2 is better at vm
> in low memo=
> > ry situations than 2.4 is. Why is this? Why hasnt someone fixed
> the 2.4 cod=
> > e?
> Actually they have on thw whole. However VM tuning is a hard problem

Ah! Finally the t-word!! Yes, VM "tuning" is a hard problem. Because any
full-strength operating system, whether Windows NT, Linux, some other flavor
of UNIX or even MVS, can be used to support a variety of computational
endeavors, it is almost impossible to come up with a fixed "algorithm" that
will effectively support all legitimate usage patterns while protecting
users as much as possible from pathological usage patterns. Therefore ...

Most operating systems allow one to *measure* performance variables and give
system managers *control knobs* they can use to tune policy to a given
usage. For example, I once worked on a system where there were three modes.
During the day, the system was tuned for interactive users, on the swing
shift it was tuned to a mix of batch jobs and system administration
functions like backups, and on the graveyard shift it ran nothing but huge
batch jobs.

Linux certainly has the measurement capabilities; I've been able to find
everything I need in /proc for the monitoring and analysis I need to do. On
the control knobs, I think Linux falls short relative to, say, Solaris,
Tru64, VMS and Windows 2000. Nearly all decisions seem to be "hard-wired" in
Linux, for example, the goodness boosts given to processes to promote soft
affinity, the time slice, and the fractions of memory allocated to the
various functions: buffers, cached, etc. They are set as #defines in header
files. Even having them as variables would be an improvement; then they
could be examined and modified with a debugger.

I would like to be able to set up a test system in my laboratory, fire up a
benchmark that emulates a real-world workload and tweak these parameters
somewhere in /proc in real time, while watching the response times of my
benchmark transactions. I can do this in Tru64, I can do this in a number of
other operating systems. Right now, for all practical purposes, when I want
to perform an experiment like this, I need to recompile, quite often, the
*entire* kernel, reboot and re-run my benchmark. In other words, if the
parameters were tunable, what now takes *days* to do could be accomplished
in hours, even minutes, with just a little up-front work.
--
M. Edward (Ed) Borasky, Chief Scientist, Borasky Research
http://www.borasky-research.net
mailto:znmeb@borasky-research.net
http://groups.yahoo.com/group/BoraskyResearchJournal

Q: How do you tell when a pineapple is ready to eat?
A: It picks up its knife and fork.

Re: Which is better at vm, and why? 2.2 or 2.4

[Patrick McFarland]

[-- Attachment #1: Type: text/plain, Size: 4060 bytes --]

Hmm, I see that as very bad. There should be a bunch of sysctls to do that easily. Also, I heard that 2.4 (and I'm assuming 2.2 as well) swaps pages on a last-used-age basis, instead of either a number-of-times-used or a hybrid of the two. That kinda seems stupid, especially if you get a bunch of apps running that just cycle thru pages, instead of the most used pages staying in memory, and the least used being swapped back and forth with about 2 or 3 megs of memory used to store the least used pages in memory

On 13-Oct-2001, M. Edward Borasky wrote:
> > -----Original Message-----
> > From: linux-kernel-owner@vger.kernel.org
> > [mailto:linux-kernel-owner@vger.kernel.org]On Behalf Of Alan Cox
> > Sent: Saturday, October 13, 2001 10:16 AM
> > To: Patrick McFarland
> > Cc: linux-kernel@vger.kernel.org
> > Subject: Re: Which is better at vm, and why? 2.2 or 2.4
> > > Now, the great kernel hacker, ac, said that 2.2 is better at vm
> > in low memo=
> > > ry situations than 2.4 is. Why is this? Why hasnt someone fixed
> > the 2.4 cod=
> > > e?
> > Actually they have on thw whole. However VM tuning is a hard problem
> 
> Ah! Finally the t-word!! Yes, VM "tuning" is a hard problem. Because any
> full-strength operating system, whether Windows NT, Linux, some other flavor
> of UNIX or even MVS, can be used to support a variety of computational
> endeavors, it is almost impossible to come up with a fixed "algorithm" that
> will effectively support all legitimate usage patterns while protecting
> users as much as possible from pathological usage patterns. Therefore ...
> 
> Most operating systems allow one to *measure* performance variables and give
> system managers *control knobs* they can use to tune policy to a given
> usage. For example, I once worked on a system where there were three modes.
> During the day, the system was tuned for interactive users, on the swing
> shift it was tuned to a mix of batch jobs and system administration
> functions like backups, and on the graveyard shift it ran nothing but huge
> batch jobs.
> 
> Linux certainly has the measurement capabilities; I've been able to find
> everything I need in /proc for the monitoring and analysis I need to do. On
> the control knobs, I think Linux falls short relative to, say, Solaris,
> Tru64, VMS and Windows 2000. Nearly all decisions seem to be "hard-wired" in
> Linux, for example, the goodness boosts given to processes to promote soft
> affinity, the time slice, and the fractions of memory allocated to the
> various functions: buffers, cached, etc. They are set as #defines in header
> files. Even having them as variables would be an improvement; then they
> could be examined and modified with a debugger.
> 
> I would like to be able to set up a test system in my laboratory, fire up a
> benchmark that emulates a real-world workload and tweak these parameters
> somewhere in /proc in real time, while watching the response times of my
> benchmark transactions. I can do this in Tru64, I can do this in a number of
> other operating systems. Right now, for all practical purposes, when I want
> to perform an experiment like this, I need to recompile, quite often, the
> *entire* kernel, reboot and re-run my benchmark. In other words, if the
> parameters were tunable, what now takes *days* to do could be accomplished
> in hours, even minutes, with just a little up-front work.
> --
> M. Edward (Ed) Borasky, Chief Scientist, Borasky Research
> http://www.borasky-research.net
> mailto:znmeb@borasky-research.net
> http://groups.yahoo.com/group/BoraskyResearchJournal
> 
> Q: How do you tell when a pineapple is ready to eat?
> A: It picks up its knife and fork.
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

-- 
Patrick "Diablo-D3" McFarland || unknown@panax.com

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Which is better at vm, and why? 2.2 or 2.4

[Rik van Riel]

On Sat, 13 Oct 2001, Patrick McFarland wrote:

> Hmm, I see that as very bad. There should be a bunch of sysctls to do
> that easily.

See /proc/sys/vm/* and the documentation ;)

> Also, I heard that 2.4 (and I'm assuming 2.2 as well) swaps pages on a
> last-used-age basis, instead of either a number-of-times-used or a
> hybrid of the two. That kinda seems stupid,

Don't worry since it's not true, at least the VM in the -ac
kernels _does_ use a hybrid of access recency and frequency
to determine page replacement.

The -linus kernel, however only has LRU-like selection.

At the moment the -linus kernel is faster than the -ac kernel
for some workloads. This may have something to do with better
clusterable IO ... when page replacement is less precise the
chance that IO is clusterable is probably larger due to the
way we scan.

I plan to do more explicit IO clustering in -ac to try and
remedy this difference.

regards,

Rik
-- 
DMCA, SSSCA, W3C?  Who cares?  http://thefreeworld.net/  (volunteers needed)

http://www.surriel.com/		http://distro.conectiva.com/

Re: Which is better at vm, and why? 2.2 or 2.4

[Patrick McFarland]

[-- Attachment #1: Type: text/plain, Size: 1606 bytes --]

Ill reiterate something here, im on a p133 with 16 megs. Yeah, the kind of the crappy ide controller that eats cpu time to swap. (Enough so that my mouse pointer will freeze in X that its swapping so much. Swapping is the only thing ive found that can pull that off) Swapping the least ammount would be the best for a box like that.

On 13-Oct-2001, Rik van Riel wrote:
> On Sat, 13 Oct 2001, Patrick McFarland wrote:
> 
> > Hmm, I see that as very bad. There should be a bunch of sysctls to do
> > that easily.
> 
> See /proc/sys/vm/* and the documentation ;)
> 
> > Also, I heard that 2.4 (and I'm assuming 2.2 as well) swaps pages on a
> > last-used-age basis, instead of either a number-of-times-used or a
> > hybrid of the two. That kinda seems stupid,
> 
> Don't worry since it's not true, at least the VM in the -ac
> kernels _does_ use a hybrid of access recency and frequency
> to determine page replacement.
> 
> The -linus kernel, however only has LRU-like selection.
> 
> At the moment the -linus kernel is faster than the -ac kernel
> for some workloads. This may have something to do with better
> clusterable IO ... when page replacement is less precise the
> chance that IO is clusterable is probably larger due to the
> way we scan.
> 
> I plan to do more explicit IO clustering in -ac to try and
> remedy this difference.
> 
> regards,
> 
> Rik
> -- 
> DMCA, SSSCA, W3C?  Who cares?  http://thefreeworld.net/  (volunteers needed)
> 
> http://www.surriel.com/		http://distro.conectiva.com/
> 

-- 
Patrick "Diablo-D3" McFarland || unknown@panax.com

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Which is better at vm, and why? 2.2 or 2.4

[Patrick McFarland]

[-- Attachment #1: Type: text/plain, Size: 2233 bytes --]

Also, I'd like to say about the documentation...

<quote>
Currently, these files are in /proc/sys/vm:
- bdflush
- buffermem
- freepages
- kswapd
- overcommit_memory
- page-cluster
- pagecache
- pagetable_cache
</quote>

but a simple ls of /proc/sys/vm reports:
bdflush  kswapd  overcommit_memory  page-cluster  pagetable_cache

Shouldnt the documentation be updated, seeing for the fact it was written in the 2.2.10 days?

On 13-Oct-2001, Patrick McFarland wrote:
> Ill reiterate something here, im on a p133 with 16 megs. Yeah, the kind of the crappy ide controller that eats cpu time to swap. (Enough so that my mouse pointer will freeze in X that its swapping so much. Swapping is the only thing ive found that can pull that off) Swapping the least ammount would be the best for a box like that.
> 
> On 13-Oct-2001, Rik van Riel wrote:
> > On Sat, 13 Oct 2001, Patrick McFarland wrote:
> > 
> > > Hmm, I see that as very bad. There should be a bunch of sysctls to do
> > > that easily.
> > 
> > See /proc/sys/vm/* and the documentation ;)
> > 
> > > Also, I heard that 2.4 (and I'm assuming 2.2 as well) swaps pages on a
> > > last-used-age basis, instead of either a number-of-times-used or a
> > > hybrid of the two. That kinda seems stupid,
> > 
> > Don't worry since it's not true, at least the VM in the -ac
> > kernels _does_ use a hybrid of access recency and frequency
> > to determine page replacement.
> > 
> > The -linus kernel, however only has LRU-like selection.
> > 
> > At the moment the -linus kernel is faster than the -ac kernel
> > for some workloads. This may have something to do with better
> > clusterable IO ... when page replacement is less precise the
> > chance that IO is clusterable is probably larger due to the
> > way we scan.
> > 
> > I plan to do more explicit IO clustering in -ac to try and
> > remedy this difference.
> > 
> > regards,
> > 
> > Rik
> > -- 
> > DMCA, SSSCA, W3C?  Who cares?  http://thefreeworld.net/  (volunteers needed)
> > 
> > http://www.surriel.com/		http://distro.conectiva.com/
> > 
> 
> -- 
> Patrick "Diablo-D3" McFarland || unknown@panax.com



-- 
Patrick "Diablo-D3" McFarland || unknown@panax.com

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Which is better at vm, and why? 2.2 or 2.4

[Rik van Riel]

On Sat, 13 Oct 2001, Patrick McFarland wrote:

> Shouldnt the documentation be updated, seeing for the fact it was
> written in the 2.2.10 days?

I've sent in the patch with updated (well, 2.4.2 time)
documentation a number of times around 2.4.2 / 2.4.3.

I'll update the docs for the -ac kernel and will give
Alan the new documentation.

As for Linus his kernel, complain to Andrea about the
complete lack of documentation.

regards,

Rik
-- 
DMCA, SSSCA, W3C?  Who cares?  http://thefreeworld.net/  (volunteers needed)

http://www.surriel.com/		http://distro.conectiva.com/

Re: Which is better at vm, and why? 2.2 or 2.4

[Patrick McFarland]

Could you put the updated vm.txt up somewhere for the world to read?
Atleast if the url is here, people could grep the mailing list for "vm doc" or something and find it that way.

On 13-Oct-2001, Rik van Riel wrote:
> On Sat, 13 Oct 2001, Patrick McFarland wrote:
> 
> > Shouldnt the documentation be updated, seeing for the fact it was
> > written in the 2.2.10 days?
> 
> I've sent in the patch with updated (well, 2.4.2 time)
> documentation a number of times around 2.4.2 / 2.4.3.
> 
> I'll update the docs for the -ac kernel and will give
> Alan the new documentation.
> 
> As for Linus his kernel, complain to Andrea about the
> complete lack of documentation.
> 
> regards,
> 
> Rik
> -- 
> DMCA, SSSCA, W3C?  Who cares?  http://thefreeworld.net/  (volunteers needed)
> 
> http://www.surriel.com/		http://distro.conectiva.com/
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

-- 
Patrick "Diablo-D3" McFarland || unknown@panax.com

Re: Which is better at vm, and why? 2.2 or 2.4

[Rik van Riel]

On Sat, 13 Oct 2001, Patrick McFarland wrote:

> Could you put the updated vm.txt up somewhere for the world to read?
> Atleast if the url is here, people could grep the mailing list for "vm
> doc" or something and find it that way.

The patch (for linux 2.4.2) has been up on my home page for
ages now:

	http://www.surriel.com/patches/

regards,

Rik
-- 
DMCA, SSSCA, W3C?  Who cares?  http://thefreeworld.net/  (volunteers needed)

http://www.surriel.com/		http://distro.conectiva.com/

Re: Which is better at vm, and why? 2.2 or 2.4

[Wilson]

----- Original Message -----
From: "Patrick McFarland" <unknown@panax.com>
To: "Rik van Riel" <riel@conectiva.com.br>
Cc: <linux-kernel@vger.kernel.org>
Sent: Saturday, October 13, 2001 2:53 PM
Subject: Re: Which is better at vm, and why? 2.2 or 2.4

>Also, I'd like to say about the documentation...
>
><quote>
>Currently, these files are in /proc/sys/vm:
>- bdflush
>- buffermem
>- freepages
>- kswapd
>- overcommit_memory
>- page-cluster
>- pagecache
>- pagetable_cache
></quote>
>
>but a simple ls of /proc/sys/vm reports:
>bdflush  kswapd  overcommit_memory  page-cluster  pagetable_cache
>
>Shouldnt the documentation be updated, seeing for the fact it was written
in the 2.2.10 days?

I must be confused.. What kernel are you running?
This is on 2.4.8-ac9:
[root@aeon /root]# ls /proc/sys/vm
bdflush    freepages  max_map_count  min-readahead      pagecache
pagetable_cache
buffermem  kswapd     max-readahead  overcommit_memory  page-cluster

Re: Which is better at vm, and why? 2.2 or 2.4

[[solid]]

froma  completely non-developer point of view:
to me it seems that th overall speed of 2.4 kernels is much faster,
even on machines like p133 16 mb ram. it may sound silly, but software
justs tends to do its work faster while using the 2.4 series.
especially operations like mounting a big filesystem(<5GB) which
happen almost immediatly, comparing to 5-10second times of mounting
10GB under 2.2. i even managed to work on a 386 20 mhz/4mb ram laptop
witch 8mb swap on a very slow disk, and it was quite workable,
although it had very few kernel options compiled in(well, there was
networking!:) and when i added plip it was just to big...but i thing
that the days of computers with 4 megs of ram are over now, and these
200-400KB difference in kernel image size doesn't make such a big
difference. the scary fact is, that the next kernel series(the stable
one after 2.5) might not fit on a floppy! :)
but for now...to me 2.4 seems the best choice for any kind of linux
box.

[solid]
Registered Linux user number 212159

Re: Which is better at vm, and why? 2.2 or 2.4

[Patrick McFarland]

Im using 2.4.12-linus

On 13-Oct-2001, Wilson wrote:
> ----- Original Message -----
> From: "Patrick McFarland" <unknown@panax.com>
> To: "Rik van Riel" <riel@conectiva.com.br>
> Cc: <linux-kernel@vger.kernel.org>
> Sent: Saturday, October 13, 2001 2:53 PM
> Subject: Re: Which is better at vm, and why? 2.2 or 2.4
> 
> >Also, I'd like to say about the documentation...
> >
> ><quote>
> >Currently, these files are in /proc/sys/vm:
> >- bdflush
> >- buffermem
> >- freepages
> >- kswapd
> >- overcommit_memory
> >- page-cluster
> >- pagecache
> >- pagetable_cache
> ></quote>
> >
> >but a simple ls of /proc/sys/vm reports:
> >bdflush  kswapd  overcommit_memory  page-cluster  pagetable_cache
> >
> >Shouldnt the documentation be updated, seeing for the fact it was written
> in the 2.2.10 days?
> 
> I must be confused.. What kernel are you running?
> This is on 2.4.8-ac9:
> [root@aeon /root]# ls /proc/sys/vm
> bdflush    freepages  max_map_count  min-readahead      pagecache
> pagetable_cache
> buffermem  kswapd     max-readahead  overcommit_memory  page-cluster
> 
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

-- 
Patrick "Diablo-D3" McFarland || unknown@panax.com

Re: Which is better at vm, and why? 2.2 or 2.4

[Rik van Riel]

On Sat, 13 Oct 2001, Patrick McFarland wrote:

> Ill reiterate something here, im on a p133 with 16 megs. Yeah, the
> kind of the crappy ide controller that eats cpu time to swap. (Enough
> so that my mouse pointer will freeze in X that its swapping so much.
> Swapping is the only thing ive found that can pull that off) Swapping
> the least ammount would be the best for a box like that.

Absolutely true.  Are you willing to help the VM developers test
their patches by seeing how well stuff runs on your box, so we
can try and make the VM work better for you ?

regards,

Rik
-- 
DMCA, SSSCA, W3C?  Who cares?  http://thefreeworld.net/  (volunteers needed)

http://www.surriel.com/		http://distro.conectiva.com/

RE: Which is better at vm, and why? 2.2 or 2.4

[M. Edward Borasky]

> -----Original Message-----
> From: linux-kernel-owner@vger.kernel.org
> [mailto:linux-kernel-owner@vger.kernel.org]On Behalf Of Patrick
> McFarland
> Sent: Saturday, October 13, 2001 11:17 AM
> To: M. Edward Borasky
> Cc: linux-kernel@vger.kernel.org
> Subject: Re: Which is better at vm, and why? 2.2 or 2.4
>
>
> Hmm, I see that as very bad. There should be a bunch of sysctls
> to do that easily. Also, I heard that 2.4 (and I'm assuming 2.2
> as well) swaps pages on a last-used-age basis, instead of either
> a number-of-times-used or a hybrid of the two. That kinda seems
> stupid, especially if you get a bunch of apps running that just
> cycle thru pages, instead of the most used pages staying in
> memory, and the least used being swapped back and forth with
> about 2 or 3 megs of memory used to store the least used pages in memory

What do you see as bad? The ability to change tuning parameters on the fly
or the lack of such ability? At the very least, I need to be able to poke
settings with a debugger, and a more systematic and user-friendly interface
would be preferable.

Your example is exactly what I'm talking about! Neither the applications nor
the kernel are "stupid"; they are doing exactly what they're being paid to
do. My job as a system tuner is to make sure that the users get the response
times and throughputs they need, given the inherent limits of how much they
can afford to pay for processors, memory and disk. As an aside, I also need
to be able to figure out what those limits are -- when it's time to add
capacity rather than just tune the system.
--
M. Edward (Ed) Borasky, Chief Scientist, Borasky Research
http://www.borasky-research.net
mailto:znmeb@borasky-research.net
http://groups.yahoo.com/group/BoraskyResearchJournal

Q: How do you tell when a pineapple is ready to eat?
A: It picks up its knife and fork.

Re: Which is better at vm, and why? 2.2 or 2.4

[Daniel Phillips]

On October 13, 2001 08:06 pm, M. Edward Borasky wrote:
> Linux certainly has the measurement capabilities; I've been able to find
> everything I need in /proc for the monitoring and analysis I need to do. On
> the control knobs, I think Linux falls short relative to, say, Solaris,
> Tru64, VMS and Windows 2000. Nearly all decisions seem to be "hard-wired" in
> Linux, for example, the goodness boosts given to processes to promote soft
> affinity, the time slice, and the fractions of memory allocated to the
> various functions: buffers, cached, etc. They are set as #defines in header
> files. Even having them as variables would be an improvement; then they
> could be examined and modified with a debugger.

It's because Linus wants it that way, with a view to encouraging the 
development of algorithms that work well across a broad range of 
configurations without requiring a lot of tuning.  So it's a case of short 
term pain for long term gain.

Keep in mind that once you start exposing tuning parameters you tend to get 
lots of user programs out there that break without the parameters, or if the 
parameters don't behave the same way across versions.  Official tuning 
parameters also get in the way of trying out new algorithms, which might not 
even support the old tweaks, for example.

> I would like to be able to set up a test system in my laboratory, fire up a
> benchmark that emulates a real-world workload and tweak these parameters
> somewhere in /proc in real time, while watching the response times of my
> benchmark transactions. I can do this in Tru64, I can do this in a number of
> other operating systems. Right now, for all practical purposes, when I want
> to perform an experiment like this, I need to recompile, quite often, the
> *entire* kernel, reboot and re-run my benchmark. In other words, if the
> parameters were tunable, what now takes *days* to do could be accomplished
> in hours, even minutes, with just a little up-front work.

So then you probably just want to grab one of the many patches that expose 
things through proc and use it as a jumping-off point to expose your own 
tweaks.  As you say, much faster than recompiling every time.

--
Daniel

Re: Which is better at vm, and why? 2.2 or 2.4

[Robert Love]

On Fri, 2001-10-19 at 20:38, Daniel Phillips wrote:
> Keep in mind that once you start exposing tuning parameters you tend to get 
> lots of user programs out there that break without the parameters, or if the 
> parameters don't behave the same way across versions.  Official tuning 
> parameters also get in the way of trying out new algorithms, which might not 
> even support the old tweaks, for example.

Agreed.  They also encourage people to write algorithms that are
suboptimal, but perform OK with proper tuning.  This, imho, is the
biggest argument against.

	Robert Love

Re: Which is better at vm, and why? 2.2 or 2.4

[Mike Fedyk]

On Fri, Oct 19, 2001 at 09:05:29PM -0400, Robert Love wrote:
> On Fri, 2001-10-19 at 20:38, Daniel Phillips wrote:
> > Keep in mind that once you start exposing tuning parameters you tend to get 
> > lots of user programs out there that break without the parameters, or if the 
> > parameters don't behave the same way across versions.  Official tuning 
> > parameters also get in the way of trying out new algorithms, which might not 
> > even support the old tweaks, for example.
> 
> Agreed.  They also encourage people to write algorithms that are
> suboptimal, but perform OK with proper tuning.  This, imho, is the
> biggest argument against.
> 

How does this differ when the tuning is hard coded?

There are always cases where the algo will fall over.

One thing I can say in favor of hard coded tuning is that it encourages the
cases where it does fall over to be reported, and possibly fixed.

Re: Which is better at vm, and why? 2.2 or 2.4

[Robert Love]

On Sat, 2001-10-20 at 15:56, Mike Fedyk wrote:

> On Fri, Oct 19, 2001 at 09:05:29PM -0400, Robert Love wrote:
>
> > Agreed.  They also encourage people to write algorithms that are
> > suboptimal, but perform OK with proper tuning.  This, imho, is the
> > biggest argument against.
>
> How does this differ when the tuning is hard coded?
> 
> There are always cases where the algo will fall over.
> 
> One thing I can say in favor of hard coded tuning is that it encourages the
> cases where it does fall over to be reported, and possibly fixed.

Because if the tunings are hard coded, developers will be encouraged to
make sure those tunings serve at least some common case, if not most
cases.  If they are tunable, you get "the default tuning works for me. 
play with the proc settings to get it right...".

I don't agree with that.  Code it right.  If it takes a different tuning
for every case, then ditch the algorithm and let's find one that works
right.  Kind of like the point you just mentioned...

	Robert Love

Re: Which is better at vm, and why? 2.2 or 2.4

[Mark Hahn]

> Now, the great kernel hacker, ac, said that 2.2 is better at vm in low
> memory situations than 2.4 is. Why is this? Why hasnt someone fixed the 2.4
> code? 

not to slight TGKH AC, but he's also the 2.2 maintainer; perhaps there's 
some paternal protectiveness there ;)

my test for VM is to compile a kernel on my crappy old BP6 with mem=64m;
I use a dedicated partition with a fresh ext2, unpack the same source tree,
make -j2 7 times, drop 1 outlier, and average:

2.2.19: 584.462user 57.492system 385.112elapsed 166.5%CPU
2.4.12: 582.318user 40.535system 337.093elapsed 184.5%CPU

notice that elapsed is noticably faster even than the 1+17 second
benefit to user and system times.  Rik's VM seems to be slightly
slower on this test.  with 128M, there's much less diference for 
any of the versions (and I don't have the patience for <64M.)

regards, mark hahn.

Re: Which is better at vm, and why? 2.2 or 2.4

[Mike Fedyk]

On Sat, Oct 13, 2001 at 01:48:05PM -0400, Mark Hahn wrote:
> > Now, the great kernel hacker, ac, said that 2.2 is better at vm in low
> > memory situations than 2.4 is. Why is this? Why hasnt someone fixed the 2.4
> > code? 
> 
> not to slight TGKH AC, but he's also the 2.2 maintainer; perhaps there's 
> some paternal protectiveness there ;)
> 
> my test for VM is to compile a kernel on my crappy old BP6 with mem=64m;
> I use a dedicated partition with a fresh ext2, unpack the same source tree,
> make -j2 7 times, drop 1 outlier, and average:
> 
> 2.2.19: 584.462user 57.492system 385.112elapsed 166.5%CPU
> 2.4.12: 582.318user 40.535system 337.093elapsed 184.5%CPU
>

Is this:
> 2.2.19: 
584.462user 
57.492system 
385.112elapsed 
166.5%CPU

> 2.4.12: 
582.318user 
40.535system 
337.093elapsed 
184.5%CPU

???

If so, then 2.4.12 won on user, system and elapsed.  What's with the CPU
percentage?  Are you on a dual system?

> notice that elapsed is noticably faster even than the 1+17 second
> benefit to user and system times.  Rik's VM seems to be slightly

No, that's Andrea's VM (since 2.4.10pre11).  Rik's is in 2.4.xx-ac.

Mike

Re: Which is better at vm, and why? 2.2 or 2.4

[Mark Hahn]

> > my test for VM is to compile a kernel on my crappy old BP6 with mem=64m;
> > I use a dedicated partition with a fresh ext2, unpack the same source tree,
> > make -j2 7 times, drop 1 outlier, and average:
> > 
> > 2.2.19: 584.462user 57.492system 385.112elapsed 166.5%CPU
> > 2.4.12: 582.318user 40.535system 337.093elapsed 184.5%CPU
> >
> 
> Is this:
> > 2.2.19: 
> 584.462user 
> 57.492system 
> 385.112elapsed 
> 166.5%CPU
> 
> > 2.4.12: 
> 582.318user 
> 40.535system 
> 337.093elapsed 
> 184.5%CPU
> 
> ???

what's the question?  you just reformatted my results.

> If so, then 2.4.12 won on user, system and elapsed.  

of course: that's the point.  but elapsed is where the big difference is,
and that's what's interesting, since it reflects less dead time due to 
smarter/less swapping.

> What's with the CPU
> percentage?  Are you on a dual system?

yes, of course: the bp6 is a dual, with cel/366's in this case.
I don't think SMPness is relevant here.

> > notice that elapsed is noticably faster even than the 1+17 second
> > benefit to user and system times.  Rik's VM seems to be slightly
> 
> No, that's Andrea's VM (since 2.4.10pre11).  Rik's is in 2.4.xx-ac.

no, my statement is correct; I merely didn't give stats for Rik's VM.

Re: Which is better at vm, and why? 2.2 or 2.4

[Patrick McFarland]

[-- Attachment #1: Type: text/plain, Size: 719 bytes --]

Hmm, it seems that I didnt realize I had to cc that to the list, because I belive this is something that should be on the list.

Anyhow, exactly how much tweeking did you do, and isnt the ac tree suppost to be unstable?

On 13-Oct-2001, Alan Cox wrote:
> > Now as of the tuning problem, that seems to be deep magic to me. How would =
> > one tune 2.4 for a p133 with 16 megs of memory? Or should I just give up an=
> > d try to tune 2.2?
> 
> I've been running 2.4.12-ac1 happily on both 20Mb and 32Mb boxes. The 
> performance of the kernel has been good. Indeed the tuning work I had to do
> on the 20Mb PC110 has been userspace.
> 
> Alan
> 

-- 
Patrick "Diablo-D3" McFarland || unknown@panax.com

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Which is better at vm, and why? 2.2 or 2.4

[Patrick McFarland]

[-- Attachment #1: Type: text/plain, Size: 638 bytes --]

Well, I dont actually need anything thats provided in 2.4 thats not provided in 2.2. I tend to use standard hardware. Would 2.2 be a better choice?

On 13-Oct-2001, Alan Cox wrote:
> > Hmm, it seems that I didnt realize I had to cc that to the list, because I =
> > belive this is something that should be on the list.
> > 
> > Anyhow, exactly how much tweeking did you do, and isnt the ac tree suppost =
> > to be unstable?
> 
> The -ac patches vary in stability. The release announcement for each one is
> intended to be a guide to how stable I expect it to be.
> 

-- 
Patrick "Diablo-D3" McFarland || unknown@panax.com

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]