[Linux-ia64] mkswap fails to create large swap partitions on ia64 [patch]

[Linux-ia64] mkswap fails to create large swap partitions on ia64 [patch]

[Peter Chubb]

Package: util-linux
Version: 2.11n-4

Hi,
	Attempts to create large swap files/partitions fails on IA64,
because some things that are ints in mkswap source should be off_t or
long.

Here's a patch.


--- util-linux-2.11n/disk-utils/mkswap.c.orig	Thu Mar 15 21:09:57 2001
+++ util-linux-2.11n/disk-utils/mkswap.c	Thu Mar 14 10:52:09 2002
@@ -232,7 +232,7 @@
 /* patch from jj - why does this differ from the above? */
 /* 32bit kernels have a second limitation of 2GB, sparc64 is limited by
    the size of virtual address space allocation for vmalloc */
-#if defined(__alpha__)
+#if defined(__alpha__) || defined(__ia64__)
 #define V1_MAX_PAGES           ((1 << 24) - 1)
 #elif defined(__mips__)
 #define V1_MAX_PAGES           ((1 << 17) - 1)
@@ -373,7 +373,7 @@
 }
 
 static long
-valid_offset (int fd, int offset) {
+valid_offset (int fd, off_t offset) {
 	char ch;
 
 	if (lseek (fd, offset, 0) < 0)
@@ -383,16 +383,16 @@
 	return 1;
 }
 
-static int
+static off_t
 find_size (int fd) {
-	unsigned int high, low;
+	off_t high, low;
 
 	low = 0;
 	for (high = 1; high > 0 && valid_offset (fd, high); high *= 2)
 		low = high;
 	while (low < high - 1)
 	{
-		const int mid = (low + high) / 2;
+		const off_t mid = (low + high) / 2;
 
 		if (valid_offset (fd, mid))
 			low = mid;
@@ -432,9 +432,9 @@
 main(int argc, char ** argv) {
 	struct stat statbuf;
 	int i, sz;
-	int maxpages;
-	int goodpages;
-	int offset;
+	long maxpages;
+	long goodpages;
+	off_t offset;
 	int force = 0;
 	char *block_count = 0;
 	char *pp;
@@ -604,7 +604,7 @@
 	if (goodpages <= 0)
 		die(_("Unable to set up swap-space: unreadable"));
 	printf(_("Setting up swapspace version %d, size = %ld bytes\n"),
-		version, (long)(goodpages*pagesize));
+		version, (unsigned long)goodpages * pagesize);
 	write_signature((version = 0) ? "SWAP-SPACE" : "SWAPSPACE2");
 
 	offset = ((version = 0) ? 0 : 1024);

RE: [Linux-ia64] mkswap fails to create large swap partitions on

[Van Maren, Kevin]

> >>>>> "Peter" = Peter Chubb <peter@chubb.wattle.id.au> writes:
> 
> >>>>> "David" = David Mosberger <davidm@napali.hpl.hp.com> writes:
> >>>>> On Thu, 14 Mar 2002 10:54:21 +1100, Peter Chubb 
> <peter@chubb.wattle.id.au> said:
> 
> David> One question though: are you sure about the following line?
> 
> Peter> #define V1_MAX_PAGES ((1 << 24) - 1)
> 
> I've just traced through the code -- the limit should be the lesser of
> (1<<54)-1 and the number of shorts that can be vmalloced in one call,
> in the call
>    p->swap_map = vmalloc(maxpages * sizeof(short))
> in swapfiles.c near line 1029 (2.4.18)
> Unfortunately, I'm not sure (yet) what the maximum such number is.
> 
> Peter C

Peter,

Thanks for looking at this!  I'd been meaning to for a while -- with 32GB+
ram in a machine, 2GB swapfiles weren't really cutting it :-)  Especially
when the default kernel was limited to 8 swapfiles.

I looked through the code (mm/swapfile.c) a little, and I'm not claiming
it all makes sense to me -- how big is a "page" anyway -- but the code
helpfully calls SWAPSPACE2 V2 in the kernel and V1 in mkswap.

Version 1/2 (SWAPSPACE2) appears to not have the page limitation in the
kernel, except for the 54-bit one:
  maxpages = SWP_OFFSET(SWP_ENTRY(0,~0UL)) - 1;

However, the swap_map vmallocs a short (2 bytes) for every "maxpages",
but then the swap_map array is indexed by an int.  So it appears that
the "real" maximum is 2^31-1 (range of a signed int).  [As one example,
scan_swap_map returns an "int" index, and ints occur elsewhere as well.]

So we're okay until about a couple terrabytes is all, until we change
all the index variables to "long". :-)  But even one terrabyte is 16x 64GB.

Also, the kernel should probably try to use "some" of the swap space if it
can't vmalloc storage for the whole thing -- maybe cut p->max in half
until the vmalloc succeeds?  I see the case where you are low on memory
and can't add more swap because you can't allocate enough memory and
the only swapfile you have is 100GB...

Kevin

RE: [Linux-ia64] mkswap fails to create large swap partitions on

[Peter Chubb]

>>>>> "Kevin" = Van Maren <Van> writes:


Kevin> So we're okay until about a couple terrabytes is all, until we
Kevin> change all the index variables to "long". :-) But even one
Kevin> terrabyte is 16x 64GB.

I'm currently looking at some other problems which would also limit
the size of a single swap file/device to 1TB --- the genhd code limits
the size of a partition to 2^31-1 blocks, where a block is *known* to
be 512 bytes. (The 512 number is so embedded that's it's going to be a
major effort to remove, not only from the code, but from the brains of
the programmers... so the only thing we can do is change the size of
the index variables, and all the interfaces that use them).

So even if swap could be specified bigger than 1TB, you couldn't
create a volume that big to put it on (even with LVM).


Kevin> Also, the kernel should probably try to use "some" of the swap
Kevin> space if it can't vmalloc storage for the whole thing -- maybe
Kevin> cut p->max in half until the vmalloc succeeds?  I see the case
Kevin> where you are low on memory and can't add more swap because you
Kevin> can't allocate enough memory and the only swapfile you have is
Kevin> 100GB...

Interesting idea... I'll think about it.

Peter C
--
Working on the Gelato project 
http://www.gelato.org/ 
http://www.gelato.unsw.edu.au/  (coming soon!)

Re: [Linux-ia64] mkswap fails to create large swap partitions on ia64 [patch]

[Peter Chubb]

>>>>> "Peter" = Peter Chubb <peter@chubb.wattle.id.au> writes:

>>>>> "David" = David Mosberger <davidm@napali.hpl.hp.com> writes:
>>>>> On Thu, 14 Mar 2002 10:54:21 +1100, Peter Chubb <peter@chubb.wattle.id.au> said:

David> One question though: are you sure about the following line?

Peter> #define V1_MAX_PAGES ((1 << 24) - 1)

I've just traced through the code -- the limit should be the lesser of
(1<<54)-1 and the number of shorts that can be vmalloced in one call,
in the call
   p->swap_map = vmalloc(maxpages * sizeof(short))
in swapfiles.c near line 1029 (2.4.18)
Unfortunately, I'm not sure (yet) what the maximum such number is.

Peter C