Re: Value of TASK_UNMAPPED_SIZE on 2.4

Re: Value of TASK_UNMAPPED_SIZE on 2.4

[Kai Harrekilde-Petersen]

Andrea Arcangeli <andrea@suse.de> writes:

> On Fri, Nov 03, 2000 at 11:15:46AM -0800, Josue Emmanuel Amaro wrote:
> > (page.h).  This works out to be a value of 0x4000000.
> 						      ^ one more zero here
> > Are there any negative side effects in defining TASK_UNMAPPED_SIZE to 0x1000000?
> 
> I guess you mean 0x10000000. There's no risk in doing that. I also did another
> patch that moves the kernel away and allows 3.5G per process on IA32 via plain
> mmap or shmat, but it has the downside of reducing a lot the ZONE_NORMAL where
> on IA32 buffercache and skb still lives.

Is this available as a patch, or preferably as a compilation option to
the standard kernel?

The ASIC tools that we use wants all the memory they can grab, and the
extra few hundred megs can sometimes be _very_ critical.  On Solaris,
Synopsys' Design Compiler can allocate 3.8Gig and we've hit that limit
hard when working with toplevel netlists and SDF databases.

Thanks in advance,


Kai
-- 
Kai Harrekilde-Petersen
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Value of TASK_UNMAPPED_SIZE on 2.4

[Andrea Arcangeli]

On Fri, Nov 03, 2000 at 09:27:07PM +0100, Kai Harrekilde-Petersen wrote:
> Is this available as a patch, or preferably as a compilation option to

They're available here:

	ftp://ftp.us.kernel.org/pub/linux/kernel/people/andrea/patches/v2.2/2.2.14/bigmem-large-mapping-1.bz2
	ftp://ftp.us.kernel.org/pub/linux/kernel/people/andrea/patches/v2.2/2.2.14/bigmem-large-task-1.bz2

But they're against 2.2.x + bigmem. The first one is still valid (and it's
similar to the one discussed here). The second one doesn't apply to 2.4.x
and both vmlinux.lds and PAGE_OFFSET should be changed that way to
make it to work there.

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Value of TASK_UNMAPPED_SIZE on 2.4

[Josue Emmanuel Amaro]

[-- Attachment #1: Type: text/plain, Size: 1394 bytes --]

Andrea,

We will give it a try.

How difficult would it be to move that patch to 2.4?

It would be great if it could be a kernel configuration time option.

Regards,

Andrea Arcangeli wrote:

> On Fri, Nov 03, 2000 at 09:27:07PM +0100, Kai Harrekilde-Petersen wrote:
> > Is this available as a patch, or preferably as a compilation option to
>
> They're available here:
>
>         ftp://ftp.us.kernel.org/pub/linux/kernel/people/andrea/patches/v2.2/2.2.14/bigmem-large-mapping-1.bz2
>         ftp://ftp.us.kernel.org/pub/linux/kernel/people/andrea/patches/v2.2/2.2.14/bigmem-large-task-1.bz2
>
> But they're against 2.2.x + bigmem. The first one is still valid (and it's
> similar to the one discussed here). The second one doesn't apply to 2.4.x
> and both vmlinux.lds and PAGE_OFFSET should be changed that way to
> make it to work there.
>
> Andrea
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> Please read the FAQ at http://www.tux.org/lkml/

--
=======================================================================
  Josue Emmanuel Amaro                         Josue.Amaro@oracle.com
  Linux Products Manager                       Phone:   650.506.1239
  Intel and Linux Technologies Group           Fax:     650.413.0167
=======================================================================


[-- Attachment #2: Card for Josue Emmanuel Amaro --]
[-- Type: text/x-vcard, Size: 390 bytes --]

begin:vcard 
n:Amaro;Josue Emmanuel
tel;cell:650-245-5131
tel;fax:650-413-0167
tel;work:650-506-1239
x-mozilla-html:FALSE
url:http://www.oracle.com
org:Intel and Linux Technologies
version:2.1
email;internet:Josue.Amaro@oracle.com
title:Sr.Product Manager - Linux
adr;quoted-printable:;;500 Oracle Parkway=0D=0AMS1ip4;Redwood Shores;CA;94065;United States
fn:Josue Emmanuel Amaro
end:vcard

Re: Value of TASK_UNMAPPED_SIZE on 2.4

[Andrea Arcangeli]

On Fri, Nov 03, 2000 at 02:36:22PM -0800, Josue Emmanuel Amaro wrote:
> Andrea,
> 
> We will give it a try.
> 
> How difficult would it be to move that patch to 2.4?

I moved it to 2.4.0-test11-pre5 (should work with pre7 too):

	ftp://ftp.us.kernel.org/pub/linux/kernel/people/andrea/patches/v2.4/2.4.0-test11-pre5/per-process-3.5G-IA32-no-PAE-1

It won't work with PAE enabled though (64G option). Making it to work with PAE
enabled isn't much more complicated and I'll address that later (but
in the meantime you can use it on <= 4GB RAM machines).

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Value of TASK_UNMAPPED_SIZE on 2.4

[Julian Anastasov]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 548 bytes --]


	Hello,

On Fri, 3 Nov 2000, Josue Emmanuel Amaro wrote:

> It would be great if it could be a kernel configuration time option.

	Something like the attached old patch for 2.2. It is very
optimistic in using 64MB as min value for TASK_UNMAPPED_BASE while
the real min is above 128MB (where malloc starts). May be needs tuning for
other archs. You still need to select good value for TASK_SIZE/PAGE_OFFSET
at compile time. I run the patch safely with
echo 195887104 > /proc/sys/kernel/task_unmapped_base


Regards

--
Julian Anastasov <ja@ssi.bg>

[-- Attachment #2: /proc/sys/kernel/task_unmapped_base --]
[-- Type: TEXT/PLAIN, Size: 2139 bytes --]

--- linux/include/linux/sysctl.h.orig	Fri Dec 17 00:04:36 1999
+++ linux/include/linux/sysctl.h	Fri Dec 17 00:22:26 1999
@@ -106,6 +106,7 @@
 	KERN_SYSRQ=38,		/* int: Sysreq enable */
 	KERN_SHMALL=41,		/* int: maximum size of shared memory */
 	KERN_SPARC_STOP_A=44,	/* int: Sparc Stop-A enable */
+	KERN_TASK_UNMAPPED_BASE=45,
 };
 
 
--- linux/kernel/sysctl.c.orig	Fri Dec 17 00:04:36 1999
+++ linux/kernel/sysctl.c	Fri Dec 17 00:25:59 1999
@@ -51,6 +51,12 @@
 extern int shmall_max;
 #endif
 
+#if defined(CONFIG_2GB) || defined(CONFIG_3GB) || defined(CONFIG_1GB)
+extern int min_task_unmapped_base;
+extern int cur_task_unmapped_base;
+extern int max_task_unmapped_base;
+#endif
+
 #ifdef __sparc__
 extern char reboot_command [];
 extern int stop_a_enabled;
@@ -226,6 +232,11 @@
 	{KERN_SHMALL, "shmall", &shmall, sizeof (int),
 	 0644, NULL, &proc_dointvec_minmax, &sysctl_intvec,
 	 NULL, &zero_value, &shmall_max},
+#endif
+#if defined(CONFIG_2GB) || defined(CONFIG_3GB) || defined(CONFIG_1GB)
+	{KERN_TASK_UNMAPPED_BASE, "task_unmapped_base", &cur_task_unmapped_base, sizeof (int),
+	0644, NULL, &proc_dointvec_minmax, &sysctl_intvec,
+	NULL, &min_task_unmapped_base, &max_task_unmapped_base},
 #endif
 #ifdef CONFIG_MAGIC_SYSRQ
 	{KERN_SYSRQ, "sysrq", &sysrq_enabled, sizeof (int),
--- linux/mm/mmap.c.orig	Fri Dec 17 00:04:36 1999
+++ linux/mm/mmap.c	Fri Dec 17 00:28:11 1999
@@ -41,6 +41,12 @@
 
 int sysctl_overcommit_memory;
 
+#if defined(CONFIG_2GB) || defined(CONFIG_3GB) || defined(CONFIG_1GB)
+int min_task_unmapped_base = 0x04000000;
+int cur_task_unmapped_base = TASK_UNMAPPED_BASE;
+int max_task_unmapped_base = 0x7FC00000;
+#endif
+
 /* Check that a process has enough memory to allocate a
  * new virtual mapping.
  */
@@ -362,7 +368,11 @@
 	if (len > TASK_SIZE)
 		return 0;
 	if (!addr)
+#if defined(CONFIG_2GB) || defined(CONFIG_3GB) || defined(CONFIG_1GB)
+		addr = (unsigned long) cur_task_unmapped_base;
+#else
 		addr = TASK_UNMAPPED_BASE;
+#endif
 	addr = PAGE_ALIGN(addr);
 
 	for (vmm = find_vma(current->mm, addr); ; vmm = vmm->vm_next) {

Re: Value of TASK_UNMAPPED_SIZE on 2.4

[Andrea Arcangeli]

On Sat, Nov 04, 2000 at 01:09:42AM +0000, Julian Anastasov wrote:
> 	Something like the attached old patch for 2.2. It is very

It's not ok for 64bit archs.

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Value of TASK_UNMAPPED_SIZE on 2.4

[Julian Anastasov]

	Hello,

On Sat, 4 Nov 2000, Andrea Arcangeli wrote:

> On Sat, Nov 04, 2000 at 01:09:42AM +0000, Julian Anastasov wrote:
> > 	Something like the attached old patch for 2.2. It is very
>
> It's not ok for 64bit archs.

	Agreed. I see very different definitions for TASK_UNMAPPED_BASE
and ELF_ET_DYN_BASE in all archs and I'm not a guru to make this patch
for all archs but everyone can see the idea. May be the signed int max
value is not suitable for users that need large brk allocations with
more than 2GB on 32bit, i.e. cur_task_unmapped_base is enough. Using
machines with this RAM size and more are already a common practice but
I'm not sure if this feature is so useful compared to the difficulties
to implement it for all archs.

> Andrea


Regards

--
Julian Anastasov <ja@ssi.bg>

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Value of TASK_UNMAPPED_SIZE on 2.4

[Josue Emmanuel Amaro]

[-- Attachment #1: Type: text/plain, Size: 900 bytes --]

All,

TASK_UNMAPPED_SIZE is defined in include/asm-i386/processor.h as:

#define TASK_UNMAPPED_SIZE    (TASK_SIZE / 3)

The value of TASK_SIZE is defined as PAGE_OFFSET which is set to 0xC0000000
(page.h).  This works out to be a value of 0x4000000.

The question is:
Are there any negative side effects in defining TASK_UNMAPPED_SIZE to 0x1000000?

By doing this we allow a process to access more memory.  On Oracle it allows us
to grow our buffer size from 1.7 GB to 2.4 GB improving overall performance by
reducing I/O.

Thanks in advance,

--
=======================================================================
  Josue Emmanuel Amaro                         Josue.Amaro@oracle.com
  Linux Products Manager                       Phone:   650.506.1239
  Intel and Linux Technologies Group           Fax:     650.413.0167
=======================================================================


[-- Attachment #2: Card for Josue Emmanuel Amaro --]
[-- Type: text/x-vcard, Size: 390 bytes --]

begin:vcard 
n:Amaro;Josue Emmanuel
tel;cell:650-245-5131
tel;fax:650-413-0167
tel;work:650-506-1239
x-mozilla-html:FALSE
url:http://www.oracle.com
org:Intel and Linux Technologies
version:2.1
email;internet:Josue.Amaro@oracle.com
title:Sr.Product Manager - Linux
adr;quoted-printable:;;500 Oracle Parkway=0D=0AMS1ip4;Redwood Shores;CA;94065;United States
fn:Josue Emmanuel Amaro
end:vcard

Re: Value of TASK_UNMAPPED_SIZE on 2.4

[Andrea Arcangeli]

On Fri, Nov 03, 2000 at 11:15:46AM -0800, Josue Emmanuel Amaro wrote:
> (page.h).  This works out to be a value of 0x4000000.
						      ^ one more zero here
> Are there any negative side effects in defining TASK_UNMAPPED_SIZE to 0x1000000?

I guess you mean 0x10000000. There's no risk in doing that. I also did another
patch that moves the kernel away and allows 3.5G per process on IA32 via plain
mmap or shmat, but it has the downside of reducing a lot the ZONE_NORMAL where
on IA32 buffercache and skb still lives.

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Value of TASK_UNMAPPED_SIZE on 2.4

[Josue Emmanuel Amaro]

[-- Attachment #1: Type: text/plain, Size: 1467 bytes --]

All,

Additional follow up.

The idea of modifying this variable is to increase the amount of memory that a
process can use.  A database like Oracle can benefit from this because it allows
Oracle to create a bigger data buffer.

An obvious side effect of allocating more physical memory to a process is that there
is less available for other things like the kernel buffer cache.  This brings up a
question.  Would a write of a data block from the Oracle data buffer incur a "read
penalty" if the block is not in the buffer cache?

For example, a data block is read by Oracle into its buffer.  Since it is read
through the file system, this block is now present in the buffer cache too.  After a
while, and since we have allocated most of the memory to Oracle, that block is
removed/replaced from the cache.

Now a transaction modifies that block in Oracle's buffer and Oracle writes it to
disk.  Does the kernel firsts reads that block into the buffer cache, "read
penalty", and then writes it to disk? Or does it just write through the buffer cache
now placing the block in the kernel buffer cache?

Regards,

--
=======================================================================
  Josue Emmanuel Amaro                         Josue.Amaro@oracle.com
  Linux Products Manager                       Phone:   650.506.1239
  Intel and Linux Technologies Group           Fax:     650.413.0167
=======================================================================


[-- Attachment #2: Card for Josue Emmanuel Amaro --]
[-- Type: text/x-vcard, Size: 390 bytes --]

begin:vcard 
n:Amaro;Josue Emmanuel
tel;cell:650-245-5131
tel;fax:650-413-0167
tel;work:650-506-1239
x-mozilla-html:FALSE
url:http://www.oracle.com
org:Intel and Linux Technologies
version:2.1
email;internet:Josue.Amaro@oracle.com
title:Sr.Product Manager - Linux
adr;quoted-printable:;;500 Oracle Parkway=0D=0AMS1ip4;Redwood Shores;CA;94065;United States
fn:Josue Emmanuel Amaro
end:vcard