RFC: Remove swap file support

RFC: Remove swap file support

[Jeff Garzik]

Since you can make any file into a block device using loop,
is there any value to supporting swap files in 2.5?

swap files seem like a special case that is no longer necessary...

-- 
Jeff Garzik      | A recent study has shown that too much soup
Building 1024    | can cause malaise in laboratory mice.
MandrakeSoft     |

Re: RFC: Remove swap file support

[arjan]

In article <3B472C06.78A9530C@mandrakesoft.com> you wrote:
> Since you can make any file into a block device using loop,
> is there any value to supporting swap files in 2.5?

> swap files seem like a special case that is no longer necessary...

loop is always tricky re near-OOM deadlocks. It'll survive now, by sleeping
and waiting for memory, but swapping over that changes that equation.....

Re: RFC: Remove swap file support

[Fabbione]

HI Jeff,
	why you think people are still not using it????
I still do on most of my systems...

Fabbione

Jeff Garzik wrote:
> 
> Since you can make any file into a block device using loop,
> is there any value to supporting swap files in 2.5?
> 
> swap files seem like a special case that is no longer necessary...

Re: RFC: Remove swap file support

[Jeff Garzik]

Fabbione wrote:
>         why you think people are still not using it????
> I still do on most of my systems...

> Jeff Garzik wrote:
> > Since you can make any file into a block device using loop,
> > is there any value to supporting swap files in 2.5?
> >
> > swap files seem like a special case that is no longer necessary...

Re-read my message.  I do not say get rid of swap files, only swap file
code.  You can use loop for swap files.

-- 
Jeff Garzik      | A recent study has shown that too much soup
Building 1024    | can cause malaise in laboratory mice.
MandrakeSoft     |

Re: RFC: Remove swap file support

[Val Henson]

On Thu, Jul 12, 2001 at 09:54:35AM -0400, Jeff Garzik wrote:
> 
> Re-read my message.  I do not say get rid of swap files, only swap file
> code.  You can use loop for swap files.

Consider how often loop (doesn't) work.  Maybe doing this will
motivate people to keep loop working - or maybe they'll switch back to
swap partitions.

-VAL

Re: RFC: Remove swap file support

[Eric W. Biederman]

Jeff Garzik <jgarzik@mandrakesoft.com> writes:

> Since you can make any file into a block device using loop,
> is there any value to supporting swap files in 2.5?
> 
> swap files seem like a special case that is no longer necessary...

Yes, and no.  I'd say what we need to do is update rw_swap_page to
use the address space functions directly.  With block devices and
files going through the page cache in 2.5 that should remove any
special cases cleanly.

In 2.4 the swap code really hasn't been updated, the old code has only
been patched enough to work on 2.4.  This adds layers of work that we
really don't need to be doing.  Removing the extra redirection has the
potential to give a small performance boost to swapping.

The case to watch out for are deadlocks doing things like using
swapfiles on an NFS mount.  As you point out we can already do this
with the loop back devices so it isn't really a special case.  The
only new case I can see working are swapfiles with holes in them, or
swapfiles that do automatic compression.  I doubt those cases are
significant improvements but it looks like they will fall out
naturally. 

Eric

Re: RFC: Remove swap file support

[Eric W. Biederman]

Jeff Garzik <jgarzik@mandrakesoft.com> writes:

> Since you can make any file into a block device using loop,
> is there any value to supporting swap files in 2.5?
> 
> swap files seem like a special case that is no longer necessary...

Yes, and no.  I'd say what we need to do is update rw_swap_page to
use the address space functions directly.  With block devices and
files going through the page cache in 2.5 that should remove any
special cases cleanly.

In 2.4 the swap code really hasn't been updated, the old code has only
been patched enough to work on 2.4.  This adds layers of work that we
really don't need to be doing.  Removing the extra redirection has the
potential to give a small performance boost to swapping.

The case to watch out for are deadlocks doing things like using
swapfiles on an NFS mount.  As you point out we can already do this
with the loop back devices so it isn't really a special case.  The
only new case I can see working are swapfiles with holes in them, or
swapfiles that do automatic compression.  I doubt those cases are
significant improvements but it looks like they will fall out
naturally. 

Eric
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: RFC: Remove swap file support

[jlnance]

On Sat, Jul 14, 2001 at 12:07:38AM -0600, Eric W. Biederman wrote:
> 
> The case to watch out for are deadlocks doing things like using
> swapfiles on an NFS mount.  As you point out we can already do this
> with the loop back devices so it isn't really a special case.  The
> only new case I can see working are swapfiles with holes in them, or
> swapfiles that do automatic compression.  I doubt those cases are
> significant improvements but it looks like they will fall out
> naturally. 

The case of swap files with holes would be a nice thing to have.
It would effectivly give us a way to say "use the extra space on this
file system for swap" and at the same time the ability to set a limit
on how much space could be taken up by swap.  For example you could
create a totally sparse 1G file at bootup, and use it as a swap file.
If the system needed swap it could grow the file, but you would know
that it would never grow beyond 1G.

I dont know if the kernel should punch holes back into the swap file when
it no longer needed the space.  That would be nice, but it might be a lot
of work as I am not sure the VFS supports that.  You could accomplish the
same thing by having a daemon that added a new swap file every hour and
did a swapoff & rm of the old file.

Thanks,

Jim

Re: RFC: Remove swap file support

[Alan Cox]

> The case of swap files with holes would be a nice thing to have.
> It would effectivly give us a way to say "use the extra space on this
> file system for swap" and at the same time the ability to set a limit
> on how much space could be taken up by swap.  For example you could
> create a totally sparse 1G file at bootup, and use it as a swap file.
> If the system needed swap it could grow the file, but you would know
> that it would never grow beyond 1G.

Growing a swap file gets complex and complex where complexity is bad and 
resources are constrained due to memory pressure. 

We do need to sort this out for 2.5, for one the way that swap is 'different'
to the rest of the backing store is ugly in itself and causes a lot of
duplication and overcomplex code.

Really there should be no pages going to some anonymous magic 'swap' object.
Instead each virtual memory area should be backed by a file system object,
including a 'swapfs' - which might be the existing style of swap, might be
something stacked onto an existing fs that allocates and manages free space
or might be completely bizarre (eg a high speed SAN network swap protocol)

Re: RFC: Remove swap file support

[Eric W. Biederman]

Alan Cox <alan@lxorguk.ukuu.org.uk> writes:

> > The case of swap files with holes would be a nice thing to have.
> > It would effectivly give us a way to say "use the extra space on this
> > file system for swap" and at the same time the ability to set a limit
> > on how much space could be taken up by swap.  For example you could
> > create a totally sparse 1G file at bootup, and use it as a swap file.
> > If the system needed swap it could grow the file, but you would know
> > that it would never grow beyond 1G.
> 
> Growing a swap file gets complex and complex where complexity is bad and 
> resources are constrained due to memory pressure. 

I'll agree with that.  The big thing we lack in 2.4 is filesystem
space reservation.  It's the companion of allocate on write, and with
a little care could handle a lot of the bad cases with sparse files.
If done right we don't need to care how a file is layed out on disk.
We reserve the whole 1G during swapon, or more likely we reserve
storage from the swap page when we allocate it.  But this garantees we
won't have space problems later on during the write.  

> We do need to sort this out for 2.5, for one the way that swap is 'different'
> to the rest of the backing store is ugly in itself and causes a lot of
> duplication and overcomplex code.

Unifying the cases is good, but a lot of the over complex code is simply
because people just don't get around to updating the swap code to use
the new inferfaces when everything else is updated.  The code is just
patched to keep working when inerfaces are changed.

> Really there should be no pages going to some anonymous magic 'swap' object.
> Instead each virtual memory area should be backed by a file system object,
> including a 'swapfs' - which might be the existing style of swap, might be
> something stacked onto an existing fs that allocates and manages free space
> or might be completely bizarre (eg a high speed SAN network swap protocol)

Hmm.  I'm not certain what you are going for here.  Having swap going
to an address_space object is only a couple of days work to get
going, and gives the benefits you are describing.  Basically the only
case left in the code would be the swap file case.

Removing anonymous shared copy on write pages is something that might
be worth doing but it is something we need to examine very closely.
Over simplying things in the case of swap is very dangerous.  Last
time we tried we would up cache tons of worthless dead swap pages.

Eric

Re: RFC: Remove swap file support

[Chris Wedgwood]

On Sat, Jul 14, 2001 at 12:07:38AM -0600, Eric W. Biederman wrote:

    Yes, and no.  I'd say what we need to do is update rw_swap_page to
    use the address space functions directly.  With block devices and
    files going through the page cache in 2.5 that should remove any
    special cases cleanly.

Will block devices go through the page cache in 2.5.x?

I had hoped they would, that any block devices would just be
page-cache views of underlying character devices, thus allowing us to
remove the buffer-cache and the /dev/raw stuff.



  --cw

Re: RFC: Remove swap file support

[Chris Wedgwood]

On Sat, Jul 14, 2001 at 12:07:38AM -0600, Eric W. Biederman wrote:

    Yes, and no.  I'd say what we need to do is update rw_swap_page to
    use the address space functions directly.  With block devices and
    files going through the page cache in 2.5 that should remove any
    special cases cleanly.

Will block devices go through the page cache in 2.5.x?

I had hoped they would, that any block devices would just be
page-cache views of underlying character devices, thus allowing us to
remove the buffer-cache and the /dev/raw stuff.



  --cw
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: RFC: Remove swap file support

[Eric W. Biederman]

Chris Wedgwood <cw@f00f.org> writes:

> On Sat, Jul 14, 2001 at 12:07:38AM -0600, Eric W. Biederman wrote:
> 
>     Yes, and no.  I'd say what we need to do is update rw_swap_page to
>     use the address space functions directly.  With block devices and
>     files going through the page cache in 2.5 that should remove any
>     special cases cleanly.
> 
> Will block devices go through the page cache in 2.5.x?
> 
> I had hoped they would, that any block devices would just be
> page-cache views of underlying character devices, thus allowing us to
> remove the buffer-cache and the /dev/raw stuff.

<orcale>
Block devices will go through the page cache in 2.5.  It will take a
while for the buffer cache to go away completely, but it is there for
the code paths that haven't been updated.  Buffer heads will stay.

The /dev/raw stuff is for those users that don't want to the kernel to
cache their data and will continue to exist in some form.
</oracle>

I can't see how any device that doesn't support read or writing just a
byte can be a character device.

Eric

Re: RFC: Remove swap file support

[Eric W. Biederman]

Chris Wedgwood <cw@f00f.org> writes:

> On Sat, Jul 14, 2001 at 12:07:38AM -0600, Eric W. Biederman wrote:
> 
>     Yes, and no.  I'd say what we need to do is update rw_swap_page to
>     use the address space functions directly.  With block devices and
>     files going through the page cache in 2.5 that should remove any
>     special cases cleanly.
> 
> Will block devices go through the page cache in 2.5.x?
> 
> I had hoped they would, that any block devices would just be
> page-cache views of underlying character devices, thus allowing us to
> remove the buffer-cache and the /dev/raw stuff.

<orcale>
Block devices will go through the page cache in 2.5.  It will take a
while for the buffer cache to go away completely, but it is there for
the code paths that haven't been updated.  Buffer heads will stay.

The /dev/raw stuff is for those users that don't want to the kernel to
cache their data and will continue to exist in some form.
</oracle>

I can't see how any device that doesn't support read or writing just a
byte can be a character device.

Eric
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: RFC: Remove swap file support

[Chris Wedgwood]

On Sat, Jul 14, 2001 at 11:35:52AM -0600, Eric W. Biederman wrote:

    I can't see how any device that doesn't support read or writing
    just a byte can be a character device.

For requests smaller than the natural block size, you buffer and throw
away... this will surely suck for writing do a hd byte-by-byte though
:)



  --cw

Re: RFC: Remove swap file support

[Chris Wedgwood]

On Sat, Jul 14, 2001 at 11:35:52AM -0600, Eric W. Biederman wrote:

    I can't see how any device that doesn't support read or writing
    just a byte can be a character device.

For requests smaller than the natural block size, you buffer and throw
away... this will surely suck for writing do a hd byte-by-byte though
:)



  --cw
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: RFC: Remove swap file support

[Chris Wedgwood]

On Sat, Jul 14, 2001 at 11:35:52AM -0600, Eric W. Biederman wrote:

    I can't see how any device that doesn't support read or writing
    just a byte can be a character device.

Why not... -EINVAL if the read alignment or size is improper...



  --cw

Re: RFC: Remove swap file support

[Chris Wedgwood]

On Sat, Jul 14, 2001 at 11:35:52AM -0600, Eric W. Biederman wrote:

    I can't see how any device that doesn't support read or writing
    just a byte can be a character device.

Why not... -EINVAL if the read alignment or size is improper...



  --cw
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/