Is swap == 2 * RAM a permanent thing?

Is swap == 2 * RAM a permanent thing?

[Mike A. Harris]

Is the fact that we're supposed to use double the RAM size as
swap a permanent thing or a temporary annoyance that will get
tweaked/fixed in the future at some point during 2.4.x perhaps?

What are the technical reasons behind this change?  Just curious
as I see a lot of people are complaining about having to
repartition (although a slower swap file could be used also).

I'm curious because I currently have 96Mb of RAM and 256Mb of
swap, but swap rarely if ever gets used, and performance is very
good.  This is with 2.2.18 I'm speaking.

I'm planning on upping my RAM to 256Mb or more in the near future
however, and going to 2.4.3 or 2.4.4 when released, and since
96Mb does the job for me already it would suck to have to
increase swap at the same time when it never gets used as it is
right now.

Would it be better to make part of RAM a ramdisk and swap to
that?  Sounds like we're going backwards IMHO, but I don't
understand the details, so I'll let someone that does explain
them to me.

Thanks in advance.



----------------------------------------------------------------------
    Mike A. Harris  -  Linux advocate  -  Free Software advocate
          This message is copyright 2001, all rights reserved.
  Views expressed are my own, not necessarily shared by my employer.
----------------------------------------------------------------------
If it weren't for C, we'd all be programming in BASI and OBOL.

Re: Is swap == 2 * RAM a permanent thing?

[Rik van Riel]

On Thu, 15 Mar 2001, Mike A. Harris wrote:

> Is the fact that we're supposed to use double the RAM size as
> swap a permanent thing or a temporary annoyance that will get
> tweaked/fixed in the future at some point during 2.4.x perhaps?
>
> What are the technical reasons behind this change?

The reason is that the Linux 2.4 kernel no longer reclaims swap
space on swapin (2.2 reclaimed swap space on write access, which
lead to fragmented swap space in lots of workloads).

This means that a lot of memory ends up "duplicated" in RAM and
in swap.

I plan on doing some code to reclaim swap space when we run out,
but Linus doesn't seem to like that idea very much. His argument
(when you're OOM, you should just fail instead of limp along)
makes a lot of sense, however, and the reclaiming of swap space
isn't really high on my TODO list ...

OTOH, for people who have swap < RAM and use it just as a small
overflow area, Linus' argument falls short, so I guess some time
in the future we will have code to reclaim swap space when needed.

regards,

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com.br/

Re: Is swap == 2 * RAM a permanent thing?

[Mike Harrold]

> The reason is that the Linux 2.4 kernel no longer reclaims swap
> space on swapin (2.2 reclaimed swap space on write access, which
> lead to fragmented swap space in lots of workloads).
> 
> This means that a lot of memory ends up "duplicated" in RAM and
> in swap.
> 
> I plan on doing some code to reclaim swap space when we run out,
> but Linus doesn't seem to like that idea very much. His argument
> (when you're OOM, you should just fail instead of limp along)
> makes a lot of sense, however, and the reclaiming of swap space
> isn't really high on my TODO list ...
> 
> OTOH, for people who have swap < RAM and use it just as a small
> overflow area, Linus' argument falls short, so I guess some time
> in the future we will have code to reclaim swap space when needed.

I have some questions on this.

1) If a process uses swap space and then later (after being paged
   into memory -- or even not) it completes, is killed, etc., is
   the swap space reclaimed then?

2) If a process uses swap, is paged into memory, and is then swapped
   out again, does it re-use the same swap as before?

If the answer to either question is no, then IMHO, that's a pretty
serious design flaw.

Regards,

/Mike

Re: Is swap == 2 * RAM a permanent thing?

[Rik van Riel]

On Thu, 15 Mar 2001, Mike Harrold wrote:

> 1) If a process uses swap space and then later (after being paged
>    into memory -- or even not) it completes, is killed, etc., is
>    the swap space reclaimed then?
>
> 2) If a process uses swap, is paged into memory, and is then swapped
>    out again, does it re-use the same swap as before?

Yes and yes.

Rik
--
Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml

Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: Is swap == 2 * RAM a permanent thing?

[christophe barbe]

Please Rik, could you explain what you mean with "reclaim swap space when we run out".
In my (limited) understanding, when there's no more free memory (ram and swap space), the kernel starts to kill process (and the choice is a difficult point).
Are you proposing to add an API to reclaim swap instead of killing process ?

Other thing (certainly related) what is supposed to have changed between 2.2 and 2.4.

Thanks,
Christophe

On jeu, 15 mar 2001 14:08:50 Rik van Riel wrote:
> On Thu, 15 Mar 2001, Mike A. Harris wrote:
> 
> > Is the fact that we're supposed to use double the RAM size as
> > swap a permanent thing or a temporary annoyance that will get
> > tweaked/fixed in the future at some point during 2.4.x perhaps?
> >
> > What are the technical reasons behind this change?
> 
> The reason is that the Linux 2.4 kernel no longer reclaims swap
> space on swapin (2.2 reclaimed swap space on write access, which
> lead to fragmented swap space in lots of workloads).
> 
> This means that a lot of memory ends up "duplicated" in RAM and
> in swap.
> 
> I plan on doing some code to reclaim swap space when we run out,
> but Linus doesn't seem to like that idea very much. His argument
> (when you're OOM, you should just fail instead of limp along)
> makes a lot of sense, however, and the reclaiming of swap space
> isn't really high on my TODO list ...
> 
> OTOH, for people who have swap < RAM and use it just as a small
> overflow area, Linus' argument falls short, so I guess some time
> in the future we will have code to reclaim swap space when needed.
> 
> regards,
> 
> Rik
> --
> Virtual memory is like a game you can't win;
> However, without VM there's truly nothing to lose...
> 
> 		http://www.surriel.com/
> http://www.conectiva.com/	http://distro.conectiva.com.br/
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 
-- 
Christophe Barbé
Software Engineer
Lineo High Availability Group
42-46, rue Médéric
92110 Clichy - France
phone (33).1.41.40.02.12
fax (33).1.41.40.02.01
www.lineo.com

Re: Is swap == 2 * RAM a permanent thing?

[Rik van Riel]

On Thu, 15 Mar 2001, christophe barbe wrote:

> Please Rik, could you explain what you mean with "reclaim swap
> space when we run out". In my (limited) understanding, when
> there's no more free memory (ram and swap space), the kernel
> starts to kill process (and the choice is a difficult point).
> Are you proposing to add an API to reclaim swap instead of
> killing process ?

When we swap something in from swap, it is in effect "duplicated"
in memory and swap. Freeing the swap space of these duplicates
will mean we have, effectively, more swap space.

Rik
--
Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml

Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: Is swap == 2 * RAM a permanent thing?

[christophe barbe]

Ok I understand better now.
So when we swap in, the place is still reserved in swap for the next time we swap off the same memory part.
The swap is freed only when the owner terminates.
Then when we need more memory, we need to swap off but we don't use the duplicated part.
I understand that it is to avoid fragmentation but this can lead to false OOM situation (false because we have free memory in swap but it's reserved to avoid fragmentation).

Christophe

On ven, 16 mar 2001 00:26:35 Rik van Riel wrote:
> On Thu, 15 Mar 2001, christophe barbe wrote:
> 
> > Please Rik, could you explain what you mean with "reclaim swap
> > space when we run out". In my (limited) understanding, when
> > there's no more free memory (ram and swap space), the kernel
> > starts to kill process (and the choice is a difficult point).
> > Are you proposing to add an API to reclaim swap instead of
> > killing process ?
> 
> When we swap something in from swap, it is in effect "duplicated"
> in memory and swap. Freeing the swap space of these duplicates
> will mean we have, effectively, more swap space.
> 
> Rik
> --
> Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml
> 
> Virtual memory is like a game you can't win;
> However, without VM there's truly nothing to lose...
> 
> 		http://www.surriel.com/
> http://www.conectiva.com/	http://distro.conectiva.com/
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 
-- 
Christophe Barbé
Software Engineer
Lineo High Availability Group
42-46, rue Médéric
92110 Clichy - France
phone (33).1.41.40.02.12
fax (33).1.41.40.02.01
www.lineo.com

Re: Is swap == 2 * RAM a permanent thing?

[James Lewis Nance]

On Thu, Mar 15, 2001 at 08:26:35PM -0300, Rik van Riel wrote:
> When we swap something in from swap, it is in effect "duplicated"
> in memory and swap. Freeing the swap space of these duplicates
> will mean we have, effectively, more swap space.

Hi Rik,
    Thanks for the explanation.  It brings another question to mind.  Lets
assume that I have two 16M processes and 32M of swap space.  Both the
processes have been swapped out at some point in time so the swap space is
full.  A third process is running and tries to allocate some memory, and
the kernel has no free pages.  Since swap is full, will the kernel kill my
process, or will it try and page out one of the processes that does have
space on swap?

Thanks,

Jim

Re: Is swap == 2 * RAM a permanent thing?

[Rik van Riel]

On Thu, 15 Mar 2001, James Lewis Nance wrote:
> On Thu, Mar 15, 2001 at 08:26:35PM -0300, Rik van Riel wrote:
> > When we swap something in from swap, it is in effect "duplicated"
> > in memory and swap. Freeing the swap space of these duplicates
> > will mean we have, effectively, more swap space.
>
>     Thanks for the explanation.  It brings another question to
> mind.  Lets assume that I have two 16M processes and 32M of swap
> space.  Both the processes have been swapped out at some point
> in time so the swap space is full.  A third process is running
> and tries to allocate some memory, and the kernel has no free
> pages.  Since swap is full, will the kernel kill my process, or
> will it try and page out one of the processes that does have
> space on swap?

It will end up swapping out the two processes which already have
space in swap ... even if the 3rd process is idle.

In that situation you could argue for 2 things:

1) the kernel should reclaim space when swap is full
2) you need more swap

I guess we'll want a bit of both, possibly with 1) being an
optional thing (since swap fragmentation could well be as
bad for performance as swapping out the wrong thing).

regards,

Rik
--
Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml

Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

kernel benchmark

[gis88530]

Hi,

I use kernprof+gprof to measure the 2.2.16 kernel,
but the scale is mini-second. 
So I use do_gettimeofday( ) kernel function call to measure 
the latency. (This function support micro-second scale.)

Moreover, I use SmartBits packet generator to generate
the specific network traffic load. The environment is
as follows. However, the result are very funny. I think 
that latency should increase progressively when load
increase, but the result are unable explaining.
Could you give me some hint? Thanks a lot.

1518byte packet
load    latency(us)
1%    13.1284
10%    14.1629
20%    12.6558
30%    11.1056
40%    10.7510
50%    10.4148
60%    10.3337
70%    10.1038
80%    10.1103
90%    10.3634
100%    11.2367

64byte packet
load    latency(us)
1%    3.6767
10%    2.7696
20%    4.3926
30%    2.8135
40%    8.2552
50%    5.3088
60%    9.3744
70%    23.6247
80%    8.5351
90%    9.7217
100%    13.065

Benchmark Environment:
+---smartbits<---+
|                         |
+---->Linux-----+


* The do_gettimeofday function call is as follows:
--------------
do_gettimeofday(&begin);
...
(kernel do something)
...
do_gettimeofday(&end);
if (end.tv_usec < begin.tv_usec) {
    end.tv_usec += 1000000; end.tv_sec--;
}
end.tv_sedc -= begin.tv_sec;
end.tv_usec -= begin.tv_usec;
return ((end.tv_sec*1000000) + end.tv_usec);

Re: Is swap == 2 * RAM a permanent thing?

[Ketil Froyn]

On Thu, 15 Mar 2001, Mike A. Harris wrote:

> Is the fact that we're supposed to use double the RAM size as
> swap a permanent thing or a temporary annoyance that will get
> tweaked/fixed in the future at some point during 2.4.x perhaps?

You're not supposed to do anything, that's just a general rule of thumb.
If your system hardly ever swaps, use a swapfile, because speed doesn't
matter a lot anyway.

> Would it be better to make part of RAM a ramdisk and swap to
> that?

No, don't do that. Physical memory is better than swap. Swap is a backup
for physical memory, so that you can run programs that use more memory
than you have. You don't really *need* swap, linux works fine without.

Ketil

RE: Is swap == 2 * RAM a permanent thing?

[Torrey Hoffman]

IIRC, when this discussion of swap size first came up, the general 
conclusion was NOT that you should have swap = 2 * RAM, but that you 
should have swap(2.4.x) = 2 * swap(2.2.x), that is, twice as much swap 
as you did under 2.2.x.

So if you never swapped at all under 2.2.x, you should not need any 
swap space in 2.4.x either. 

Is this correct?  

Also, what would be the consequences of not having "enough" swap?  
Just OOM faster?  Or more serious than that?

I have 512MB of RAM and rarely swap, so normally have just a 256MB
swap partition.  Is this bad?  It seems to work fine...

Thanks!

Torrey Hoffman

RE: Is swap == 2 * RAM a permanent thing?

[William T Wilson]

On Thu, 15 Mar 2001, Torrey Hoffman wrote:

> IIRC, when this discussion of swap size first came up, the general
> conclusion was NOT that you should have swap = 2 * RAM, but that you
> should have swap(2.4.x) = 2 * swap(2.2.x), that is, twice as much swap
> as you did under 2.2.x.

it seems to me that in 2.2.x it looks like this:

total usage == swap + RAM
under 2.4.x it looks like:
total usage == swap

> So if you never swapped at all under 2.2.x, you should not need any 
> swap space in 2.4.x either. 

Right.

RE: Is swap == 2 * RAM a permanent thing?

[Rik van Riel]

On Thu, 15 Mar 2001, William T Wilson wrote:

> it seems to me that in 2.2.x it looks like this:
>
> total usage == swap + RAM
> under 2.4.x it looks like:
> total usage == swap

  total usage == maximum(swap, ram)

Rik
--
Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml

Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: Is swap == 2 * RAM a permanent thing?

[LA Walsh]

The not reclaiming swap space is flawed in more than once instance.
Suppose my P1 and P2 have their swap reserved -- now both grow.
P3 is idle but can't fit in swap.  This is going to result in fragmentation
no?  How is this fragmentation less worse than just freeing swap.

Ever since Ram sizes got to about 256M, I've tended toward using swap spaces 
about half my RAM size -- thinking of swap as an 'overflow' place that
really shouldn't get used much if at all.  As you mention, not reclaiming
swap space, but having 'double-reservations' for previously swapped
programs becomes a problem fast in this situation.  Makes the swap
much less flexible.

-- 
L A Walsh                        | Trust Technology, Core Linux, SGI
law@sgi.com                      | Voice: (650) 933-5338

Re: Is swap == 2 * RAM a permanent thing?

[Rik van Riel]

On Thu, 15 Mar 2001, LA Walsh wrote:

> The not reclaiming swap space is flawed in more than once
> instance.

I want it fixed, but don't have much time for it now.

Patches are welcome, though.

regards,

Rik
--
Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml

Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: Is swap == 2 * RAM a permanent thing?

[Andrzej Krzysztofowicz]

"Rik van Riel wrote:"
>   total usage == maximum(swap, ram)

Does it mean that having  swap<RAM   you only lose some disk space ?

Andrzej
-- 
=======================================================================
  Andrzej M. Krzysztofowicz               ankry@mif.pg.gda.pl
  phone (48)(58) 347 14 61
Faculty of Applied Phys. & Math.,   Technical University of Gdansk

Re: Is swap == 2 * RAM a permanent thing?

[Rik van Riel]

On Thu, 15 Mar 2001, Andrzej Krzysztofowicz wrote:
> "Rik van Riel wrote:"
> >   total usage == maximum(swap, ram)
>
> Does it mean that having swap<RAM you only lose some disk space ?

If you actually "rely" on swap, yes.

If you usually don't swap but want to have it for "overflow"
in some situations, or if only a few things end up on swap,
then it's actually useful.

regards,

Rik
--
Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml

Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: Is swap == 2 * RAM a permanent thing?

[Guennadi Liakhovetski]

On Fri, 16 Mar 2001, Rik van Riel wrote:

> On Thu, 15 Mar 2001, Andrzej Krzysztofowicz wrote:
> > "Rik van Riel wrote:"
> > >   total usage == maximum(swap, ram)
> >
> > Does it mean that having swap<RAM you only lose some disk space ?
> 
> If you actually "rely" on swap, yes.

Sorry, I also don't understand this. Take a plain simple situation - your
RAM is full, swap (which is < RAM) empty, you start (or try to) another
application - will it start? If yes - it means you already have 
total > max(swap, RAM)
if you simply can't start another app in this situation - that'd be VERY
strange...

Guennadi

> 
> If you usually don't swap but want to have it for "overflow"
> in some situations, or if only a few things end up on swap,
> then it's actually useful.
> 
> regards,
> 
> Rik
> --
> Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml
> 
> Virtual memory is like a game you can't win;
> However, without VM there's truly nothing to lose...
> 
> 		http://www.surriel.com/
> http://www.conectiva.com/	http://distro.conectiva.com/
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

___

Dr. Guennadi V. Liakhovetski
Department of Applied Mathematics
University of Sheffield, U.K.
email: G.Liakhovetski@sheffield.ac.uk

Re: Is swap == 2 * RAM a permanent thing?

[Boris Pisarcik]

[-- Attachment #1: Type: text/plain, Size: 1449 bytes --]

On Thu, Mar 15, 2001 at 11:44:52PM -0300, Rik van Riel wrote:
> On Thu, 15 Mar 2001, William T Wilson wrote:
> 
> > it seems to me that in 2.2.x it looks like this:
> >
> > total usage == swap + RAM
> > under 2.4.x it looks like:
> > total usage == swap
> 
>   total usage == maximum(swap, ram)

Hi, 

Do you in fact talk about 
  1) curren usage == maximum(swap, ram)
     or
  2) virtual ram capacity == maximum(swap, ram)
?

I'm a bit confused.

My next question is: some time ago i've read, that code segments of process,
which comes from executable and should stay unmodified during process
duration, are not swapped into swap space, cause they can be restored
back from the executable. This should be ok, because in protect mode
no one can write into code seg. This does seem to be true for win, because
i cannot delete executable file when it's just run, but under linux
i can delete /bin/bash without any problem. 

Why this is so ?

Because of security ? Say my disk gets corrupted right at blocks executable
image si contained and swapping in page(s) from this errorneous area
should lock/corrupt system ?

Code content can be changed indirectly in case data or some read-write segment overlaps 
code segment. Does linux count with such a situation ? (may data segment
overlap code seg ?)



Thanks                                                                Boro


email: boris@acheron.sk 

[-- Attachment #2: Type: application/pgp-signature, Size: 240 bytes --]

Re: Is swap == 2 * RAM a permanent thing?

[Alexander Viro]

On Sat, 17 Mar 2001, Boris Pisarcik wrote:

> On Thu, Mar 15, 2001 at 11:44:52PM -0300, Rik van Riel wrote:
> i cannot delete executable file when it's just run, but under linux
> i can delete /bin/bash without any problem. 

You can't delete it. You can unlink it, but the file itself remains
alive until the last reference goes away. mapping counts as reference.