pre5 VM feedback..

pre5 VM feedback..

[Jeff Garzik]

$!@#@! pre6 is already out :)

Anyway, this may be a totally subjective (and incorrect) perception, but
it seems to me like the recent 2.4.x-test kernels and thereafter start
swapping things out really quickly.  Case in point:  "diff -urN
linux.vanilla linux" command swaps out Konqueror and Netscape Mail, even
though I was using them only a few minutes ago.  Since hacking involves
a lot of "run this command, then do something else until it finishes" :)
my perception of the recent 2.4.x kernels is that my X apps are always
getting swapped out just because a big diff or make was the only active
process for ~5 minutes.

Anyway, the hard data.  The following is VM measurements taken during a
diff of two kernel trees (2.4.1-pre4 and pre5, in fact):

before diff
-----------
ps aux: http://gtf.org/garzik/vm1/before.ps.txt
meminfo: http://gtf.org/garzik/vm1/before.meminfo.txt

after diff
----------
ps aux: http://gtf.org/garzik/vm1/after.ps.txt
meminfo: http://gtf.org/garzik/vm1/after.meminfo.txt

"vmstat 3" was running in the background during the diff.  The following
is the output from that.  It should be noted that the final burst of
swapping came from my switching X desktops, forcing Netscape and
Konqueror to swap in.
http://gtf.org/garzik/vm1/vmstat.log

Obligatory dmesg output, which includes hardware info.  Summary: Dual
P-II w/ 128MB RAM
http://gtf.org/garzik/vm1/after.dmesg.txt

FWIW I am not attempting to imply that any particular conclusion should
be drawn from this data.  I just thought that the VM folks might find it
interesting and/or useful.

	Jeff


-- 
Jeff Garzik       | "You see, in this world there's two kinds of
Building 1024     |  people, my friend: Those with loaded guns
MandrakeSoft      |  and those who dig. You dig."  --Blondie
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: pre5 VM feedback..

[Linus Torvalds]

In article <3A63A9AE.345CBAF3@mandrakesoft.com>,
Jeff Garzik  <jgarzik@mandrakesoft.com> wrote:
>$!@#@! pre6 is already out :)

Yes, and for heavens sake don't use it, because the reiserfs merge got
some dirty inode logic wrong. pre7 fixes just that one line and should
be ok again.

>Anyway, this may be a totally subjective (and incorrect) perception, but
>it seems to me like the recent 2.4.x-test kernels and thereafter start
>swapping things out really quickly.  Case in point:  "diff -urN
>linux.vanilla linux" command swaps out Konqueror and Netscape Mail, even
>though I was using them only a few minutes ago.

Yes. It's really nice for some stuff, but a bit too aggressive for
normal use, I think.

If you want to play with tuning, I'd suggest something like

 - make SWAP_SHIFT bigger (try with 7 instead of 5)

 - do the "self-swap-out" only for __GFP_VM allocations, and add the
   __GFP_VM flag to all page fault allocations (ie __GPF_VM would be a
   flag that says "this allocation will grow my RSS"). 

The latter is kind of debatable - some allocations can't easily be put
in one category or the other (ie page cache growing - do we do it
because of the page cache or because we want to map the page?)

		Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: pre5 VM feedback..

[Jeff Garzik]

Linus Torvalds wrote:
> In article <3A63A9AE.345CBAF3@mandrakesoft.com>,
> Jeff Garzik  <jgarzik@mandrakesoft.com> wrote:
> >$!@#@! pre6 is already out :)

> Yes, and for heavens sake don't use it, [...]

Too late...  First and foremost, a correction:  The VM data I posted was
for pre1, not pre5.  Here is the VM data from a freshly rebooted pre6,
if that's worth anything:  http://gtf.org/garzik/vm2/

Hopefully pre6 will survive long enough to complete tulip testing :)

	Jeff


-- 
Jeff Garzik       | "You see, in this world there's two kinds of
Building 1024     |  people, my friend: Those with loaded guns
MandrakeSoft      |  and those who dig. You dig."  --Blondie
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: pre5 VM feedback..

[Linus Torvalds]

In article <3A63B9E8.F7E850F@mandrakesoft.com>,
Jeff Garzik  <jgarzik@mandrakesoft.com> wrote:
>Linus Torvalds wrote:
>> In article <3A63A9AE.345CBAF3@mandrakesoft.com>,
>> Jeff Garzik  <jgarzik@mandrakesoft.com> wrote:
>> >$!@#@! pre6 is already out :)
>
>> Yes, and for heavens sake don't use it, [...]
>
>Too late...  First and foremost, a correction:  The VM data I posted was
>for pre1, not pre5.  Here is the VM data from a freshly rebooted pre6,
>if that's worth anything:  http://gtf.org/garzik/vm2/
>
>Hopefully pre6 will survive long enough to complete tulip testing :)

Careful, careful, careful.

The problem is that pre6 survives quite nicely, but because of the silly
typo in __mark_inode_dirty() it won't actually mark inodes dirty. 
Unless you are actually using the newly integrated raiser-fs, in which
case you should be just peachy ;)

Note that the bug is not even noticeable under many loads - especially
if you have lots of memory.  The inodes just get happily cached for a
long time.  Which is why I could release a pre6, not even noticing that
it's strange and not writing out inode data to the disk. 

With a gig of RAM, inodes tend to cache really well. 

Now, I'm not saying your filesystem is toast.  I'm just saying that if
you booted up in pre6, I'd suggest a quick reboot into a better kernel
might be a good idea (be a jock, and do a sync and just push the reset
button to force a proper fsck when it comes up - just in case).

(And yes, I renamed the pre6's really quickly, so you had to be unlucky
to see them.)

			Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: pre5 VM feedback..

[Aaron Sethman]

On 15 Jan 2001, Linus Torvalds wrote:
> Now, I'm not saying your filesystem is toast.  I'm just saying that if
> you booted up in pre6, I'd suggest a quick reboot into a better kernel
> might be a good idea (be a jock, and do a sync and just push the reset
> button to force a proper fsck when it comes up - just in case).
> 
> (And yes, I renamed the pre6's really quickly, so you had to be unlucky
> to see them.)

Alas, I was one of those poor saps who got his / filesystem mangled a bit
by pre6.  Luckly nothing too horrid happened...just /etc/inittab happened
to not exist and other strangeness.  At least I know I am not losing my
mind now :)

Aaron

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: pre5 VM feedback..

[Marcelo Tosatti]

On 15 Jan 2001, Linus Torvalds wrote:

> In article <3A63A9AE.345CBAF3@mandrakesoft.com>,
> Jeff Garzik  <jgarzik@mandrakesoft.com> wrote:
> >$!@#@! pre6 is already out :)
> 
> Yes, and for heavens sake don't use it, because the reiserfs merge got
> some dirty inode logic wrong. pre7 fixes just that one line and should
> be ok again.
> 
> >Anyway, this may be a totally subjective (and incorrect) perception, but
> >it seems to me like the recent 2.4.x-test kernels and thereafter start
> >swapping things out really quickly.  Case in point:  "diff -urN
> >linux.vanilla linux" command swaps out Konqueror and Netscape Mail, even
> >though I was using them only a few minutes ago.
> 
> Yes. It's really nice for some stuff, but a bit too aggressive for
> normal use, I think.
> 
> If you want to play with tuning, I'd suggest something like
> 
>  - make SWAP_SHIFT bigger (try with 7 instead of 5)
> 
>  - do the "self-swap-out" only for __GFP_VM allocations, and add the
>    __GFP_VM flag to all page fault allocations (ie __GPF_VM would be a
>    flag that says "this allocation will grow my RSS"). 
> 
> The latter is kind of debatable - some allocations can't easily be put
> in one category or the other (ie page cache growing - do we do it
> because of the page cache or because we want to map the page?)

The swapin readahead code tries to allocate (1 << page_cluster) pages at
each swapin. 

This means there's a big chance of having (1 << page_cluster)
"self-swap-out"'s at each swapin if we're under low memory.

Nasty. 

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: pre5 VM feedback..

[Marcelo Tosatti]

On Fri, 19 Jan 2001, Marcelo Tosatti wrote:

> The swapin readahead code tries to allocate (1 << page_cluster) pages at
> each swapin. 
> 
> This means there's a big chance of having (1 << page_cluster)
> "self-swap-out"'s at each swapin if we're under low memory.
> 
> Nasty. 

Actually its even more nasty if we keep failing in refill_inactive_scan()

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/