Memory allocation problem

Memory allocation problem

[anand kumar]

Hi,

We are developing a PCI driver for a specialized hardware which
needs blocks of physically contiguous memory regions of
32 KB. We need to allocate 514 such blocks for a total of 16 MB
We were using an ioctl implementation in the driver which uses
kmalloc() to allocate the required memory blocks. 
kmalloc()(GFP_KERNEL)
fails after allocating some 250 blocks of memory (probably due to 
fragmentation).
We then tried using __get_free_pages() and the result was the 
same.
Even though the free pages in zone NORMAL and DMA were 10000 and 
1500 respectively.

Are we hitting some limit because of fragmentation and are
not able to allocate 8 contiguous physical pages? We tried moving 
the
memory allocation in init_module and made the driver load during 
boot
time, during which allocation succeeds.

The kernel version we are using is 2.4.18 (Redhat 8.0) and the 
total
amount of memory available in the box is 128MB

Is there any other mechanism to allocate large amount of 
physically
contiguous memory blocks during normal run time of the driver? Is 
this
being addressed in later kernels.

Rgds
Anand




--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: Memory allocation problem

[Greg KH]

On Wed, Apr 30, 2003 at 10:14:38PM -0000, anand kumar wrote:
> 
> Is there any other mechanism to allocate large amount of 
> physically contiguous memory blocks during normal run time of the
> driver? Is this being addressed in later kernels.

Look at vmalloc().  It should do what you are looking for.

greg k-h
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: Memory allocation problem

[Christoph Hellwig]

On Wed, Apr 30, 2003 at 03:28:25PM -0700, Greg KH wrote:
> On Wed, Apr 30, 2003 at 10:14:38PM -0000, anand kumar wrote:
> > 
> > Is there any other mechanism to allocate large amount of 
> > physically contiguous memory blocks during normal run time of the
> > driver? Is this being addressed in later kernels.
> 
> Look at vmalloc().  It should do what you are looking for.

vmalloc is not physically continguos,  He could use bootmem
unless he wants his driver to work modular.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: Memory allocation problem

[Dave Hansen]

[-- Attachment #1: Type: text/plain, Size: 1079 bytes --]

anand kumar wrote:
> We are developing a PCI driver for a specialized hardware which
> needs blocks of physically contiguous memory regions of
> 32 KB. We need to allocate 514 such blocks for a total of 16 MB
> We were using an ioctl implementation in the driver which uses
> kmalloc() to allocate the required memory blocks. 
> kmalloc()(GFP_KERNEL)
> fails after allocating some 250 blocks of memory (probably due to 
> fragmentation).
> We then tried using __get_free_pages() and the result was the 
> same.

Well, kmalloc() falls back to __get_free_pages() eventually (after going
through the slab cache), so it isn't much of a surprise that both failed.

> Even though the free pages in zone NORMAL and DMA were 10000 and
> 1500 respectively.

Can you try this on a kernel that has /proc/buddyinfo?  It exports the
buddy allocators internal structures so that you can see the
fragmentation.  buddyinfo has been in 2.5 since ~2.5.35.  There may even
be a 2.4 version floating around...

If you can post some code too, it might be helpful.
-- 
Dave Hansen
haveblue@us.ibm.com

[-- Attachment #2: buddyinfo-2.5.34-0.patch --]
[-- Type: text/plain, Size: 3439 bytes --]

# This is a BitKeeper generated patch for the following project:
# Project Name: Linux kernel tree
# This patch format is intended for GNU patch command version 2.5 or higher.
# This patch includes the following deltas:
#	           ChangeSet	1.508.1.116 -> 1.513  
#	     mm/page_alloc.c	1.89.1.8 -> 1.92   
#	 fs/proc/proc_misc.c	1.34.1.2 -> 1.36   
#
# The following is the BitKeeper ChangeSet Log
# --------------------------------------------
# 02/09/09	haveblue@elm3b96.(none)	1.513
# Merge elm3b96.(none):/work/dave/bk/linux-2.5
# into elm3b96.(none):/work/dave/bk/linux-2.5-buddyinfo
# --------------------------------------------
#
diff -Nru a/fs/proc/proc_misc.c b/fs/proc/proc_misc.c
--- a/fs/proc/proc_misc.c	Wed Sep 11 10:45:01 2002
+++ b/fs/proc/proc_misc.c	Wed Sep 11 10:45:01 2002
@@ -208,6 +208,20 @@
 #undef K
 }
 
+extern struct seq_operations fragmentation_op;
+static int fragmentation_open(struct inode *inode, struct file *file)
+{
+	(void)inode;
+	return seq_open(file, &fragmentation_op);
+}
+
+static struct file_operations fragmentation_file_operations = {
+	open:		fragmentation_open,
+	read:		seq_read,
+	llseek:		seq_lseek,
+	release:	seq_release,
+};
+
 static int version_read_proc(char *page, char **start, off_t off,
 				 int count, int *eof, void *data)
 {
@@ -624,6 +638,7 @@
 	create_seq_entry("partitions", 0, &proc_partitions_operations);
 	create_seq_entry("interrupts", 0, &proc_interrupts_operations);
 	create_seq_entry("slabinfo",S_IWUSR|S_IRUGO,&proc_slabinfo_operations);
+	create_seq_entry("buddyinfo",S_IRUGO, &fragmentation_file_operations);
 #ifdef CONFIG_MODULES
 	create_seq_entry("modules", 0, &proc_modules_operations);
 	create_seq_entry("ksyms", 0, &proc_ksyms_operations);
diff -Nru a/mm/page_alloc.c b/mm/page_alloc.c
--- a/mm/page_alloc.c	Wed Sep 11 10:45:01 2002
+++ b/mm/page_alloc.c	Wed Sep 11 10:45:01 2002
@@ -949,3 +949,69 @@
 }
 
 __setup("memfrac=", setup_mem_frac);
+
+#ifdef CONFIG_PROC_FS
+
+#include <linux/seq_file.h>
+
+static void *frag_start(struct seq_file *m, loff_t *pos)
+{
+	pg_data_t *pgdat;
+	loff_t node = *pos;
+
+	for (pgdat = pgdat_list; pgdat && node; pgdat = pgdat->pgdat_next)
+		--node;
+
+	return pgdat;
+}
+
+static void *frag_next(struct seq_file *m, void *arg, loff_t *pos)
+{
+	pg_data_t *pgdat = (pg_data_t *)arg;
+
+	(*pos)++;
+	return pgdat->pgdat_next;
+}
+
+static void frag_stop(struct seq_file *m, void *arg)
+{
+}
+
+/* 
+ * This walks the freelist for each zone. Whilst this is slow, I'd rather 
+ * be slow here than slow down the fast path by keeping stats - mjbligh
+ */
+static int frag_show(struct seq_file *m, void *arg)
+{
+	pg_data_t *pgdat = (pg_data_t *)arg;
+	zone_t *zone, *node_zones = pgdat->node_zones;
+	unsigned long flags;
+	int order;
+
+	for (zone = node_zones; zone - node_zones < MAX_NR_ZONES; ++zone) {
+		if (!zone->size)
+			continue;
+
+		spin_lock_irqsave(&zone->lock, flags);
+		seq_printf(m, "Node %d, zone %8s ", pgdat->node_id, zone->name);
+		for (order = 0; order < MAX_ORDER; ++order) {
+			unsigned long nr_bufs = 0;
+			list_t *elem;
+			list_for_each(elem, &zone->free_area[order].free_list)
+				++nr_bufs;
+			seq_printf(m, "%6lu ", nr_bufs);
+		}
+		spin_unlock_irqrestore(&zone->lock, flags);
+		seq_putc(m, '\n');
+	}
+	return 0;
+}
+
+struct seq_operations fragmentation_op = {
+	start:	frag_start,
+	next:	frag_next,
+	stop:	frag_stop,
+	show:	frag_show,
+};
+
+#endif /* CONFIG_PROC_FS */

Re: Memory allocation problem

[Mark_H_Johnson]

>The kernel version we are using is 2.4.18 (Redhat 8.0) and the total
>amount of memory available in the box is 128MB

>Is there any other mechanism to allocate large amount of physically
>contiguous memory blocks during normal run time of the driver? Is this
>being addressed in later kernels.

I regularly use a patch (bigphysarea) recommended by Dolphin for use with
their SCI cards. The copy I use is from a relatively old kernel (2.4.4)
which applies with a few warnings but is otherwise OK. I did a quick search
with Google for
  bigphysarea linux 2.4.18
and found
  http://frmb.home.cern.ch/frmb/download/bigphysarea-2.4.18.patch
or a more readable page at
  http://frmb.home.cern.ch/frmb/linux.html
which appears to be a version updated for 2.4.18. I believe the original
patch is maintained at
  http://www.uni-paderborn.de/fachbereich/AG/heiss/linux/bigphysarea.html

There are apparently several drivers that already use this interface, but
it does require a patched kernel.

I am not aware of any effort to merge this into the main line kernel
(though I would certainly appreciate that).

--Mark H Johnson
  <mailto:Mark_H_Johnson@raytheon.com>


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Memory Allocation Problem

[Jeff Clark]

Using the latest cvs tarball I am able to create and start domains, but 
when I stop and/or destroy the domain the memory is never freed.  After 
stopping and starting my "test" domain a few times, I get an "Operation 
not permitted" message.  After running xc_physinfo.py I see that my 
available memory is exhausted (even though I only have dom0 running with 
256mb allocated and there is 512mb in the machine).

Also, any pointers for porting the gdth (raid) scsi driver?  I've 
figured out the obvious (rip out all the proc related stuff), but I'm a 
bit lost when it comes to the wait stuff.

-Jeff


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click

Re: Memory Allocation Problem

[Keir Fraser]

> Using the latest cvs tarball I am able to create and start domains, but 
> when I stop and/or destroy the domain the memory is never freed.  After 
> stopping and starting my "test" domain a few times, I get an "Operation 
> not permitted" message.  After running xc_physinfo.py I see that my 
> available memory is exhausted (even though I only have dom0 running with 
> 256mb allocated and there is 512mb in the machine).

Hmmm... yes I see this too. Any idea when it broke? Would it have been
one of yesterday's checkins?

> Also, any pointers for porting the gdth (raid) scsi driver?  I've 
> figured out the obvious (rip out all the proc related stuff), but I'm a 
> bit lost when it comes to the wait stuff.

We've tended to replace wait/completion stuff with spinning on a
flag. There are a few examples dotted around the SCSI code in Xen. If
you can wait a few weeks then the new IO world will be complete enough
for general use -- then you can use Linux drivers directly.

 -- Keir


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click

Re: Memory Allocation Problem

[Keir Fraser]

> 
> > Using the latest cvs tarball I am able to create and start domains, but 
> > when I stop and/or destroy the domain the memory is never freed.  After 
> > stopping and starting my "test" domain a few times, I get an "Operation 
> > not permitted" message.  After running xc_physinfo.py I see that my 
> > available memory is exhausted (even though I only have dom0 running with 
> > 256mb allocated and there is 512mb in the machine).

This bug is now fixed.

 -- Keir


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click

Memory Allocation Problem

[Ankit Jain]

      1 #include <stdio.h>
      2 int main()
      3 {
      4    double a[1450][1450];
      5
      6    a[1449][0] = 999;
      7    printf( "%lf\n", a[1449][0] );
      8    return 1;
      9 }


it gives segementation fault

if i use malloc also it gives wrong result

what to do?

ankit

________________________________________________________________________
Yahoo! Messenger - Communicate instantly..."Ping" 
your friends today! Download Messenger Now 
http://uk.messenger.yahoo.com/download/index.html

Re: Memory Allocation Problem

[Ankit Jain]

 --- Sriharsha Vedurmudi
<sriharsha.v@redpinesignals.com> wrote: 
> 
> Ankit Jain wrote:
> 
> >      1 #include <stdio.h>
> >      2 int main()
> >      3 {
> >      4    double a[1450][1450];
> >      5
> >      6    a[1449][0] = 999;
> >      7    printf( "%lf\n", a[1449][0] );
> >      8    return 1;
> >      9 }
> >

well i am sorry its return 0 only

i have 128 mb RAM. it gives segmentation fault while
running

i tried it on 512 MB Ram also it gives segemntation
fault. around 400 MB of Ram was free when i tried to
run that program

thanks 
ankit
> >
> >it gives segementation fault
> >  
> >
> I tried, but it ran well. Except ofcourse, I changed
> "return 1" to 
> "return 0" (you dont want to return an error from
> Main)
> 
> >if i use malloc also it gives wrong result
> >
> >what to do?
> >  
> >
> I guess your system is lacking the memory required
> to allocate 1450 * 
> 1450 * 4 bytes on stack. Try making it static.
> 
> -Harsha.
> 
> >ankit
> >
>
>________________________________________________________________________
> >Yahoo! Messenger - Communicate instantly..."Ping" 
> >your friends today! Download Messenger Now 
> >http://uk.messenger.yahoo.com/download/index.html
> >  
> >
>  

________________________________________________________________________
Yahoo! Messenger - Communicate instantly..."Ping" 
your friends today! Download Messenger Now 
http://uk.messenger.yahoo.com/download/index.html

RE: Memory Allocation Problem

[Chandan Chopra]

[-- Attachment #1: Type: text/plain, Size: 2137 bytes --]


Or workaround could be you make declaration of 'a' global.

-----Original Message-----
From: linux-c-programming-owner@vger.kernel.org
[mailto:linux-c-programming-owner@vger.kernel.org]On Behalf Of Suciu
Flavius
Sent: Wednesday, September 15, 2004 9:06 PM
To: linux-c-programming@vger.kernel.org
Cc: gcc-help@gcc.gnu.org
Subject: Re: Memory Allocation Problem


1450 * 1450 * 8 = 16 MEGA!!!!!!!!!
So, you need a 16 mega of stack which I think is a little too much  ;)

The solution is to malloc() but check if you have 16 mega of ram ;)


Ankit Jain wrote:
>       1 #include <stdio.h>
>       2 int main()
>       3 {
>       4    double a[1450][1450];
>       5
>       6    a[1449][0] = 999;
>       7    printf( "%lf\n", a[1449][0] );
>       8    return 1;
>       9 }
>
>
> it gives segementation fault
>
> if i use malloc also it gives wrong result
>
> what to do?
>
> ankit
>
> ________________________________________________________________________
> Yahoo! Messenger - Communicate instantly..."Ping"
> your friends today! Download Messenger Now
> http://uk.messenger.yahoo.com/download/index.html
>

-
To unsubscribe from this list: send the line "unsubscribe
linux-c-programming" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html



http://www.patni.com
World-Wide Partnerships. World-Class Solutions.
_____________________________________________________________________

This e-mail message may contain proprietary, confidential or legally
privileged information for the sole use of the person or entity to
whom this message was originally addressed. Any review, e-transmission
dissemination or other use of or taking of any action in reliance upon
this information by persons or entities other than the intended
recipient is prohibited. If you have received this e-mail in error
kindly delete  this e-mail from your records. If it appears that this
mail has been forwarded to you without proper authority, please notify
us immediately at netadmin@patni.com and delete this mail. 
_____________________________________________________________________

Re: Memory Allocation Problem

[Eljay Love-Jensen]

Hi Ankit,

Your OS probably does not provide that many megabytes of STACK space.

Allocate your enormous 2-dimensional array on the HEAP or in the BSS or in 
the DATA areas instead.

Or check out if your can adjust the size of your STACK to around 32 
MB.  That might be a linker option.  Depends on your OS.

HTH,
--Eljay

Re: Memory Allocation Problem

[Suciu Flavius]

1450 * 1450 * 8 = 16 MEGA!!!!!!!!!
So, you need a 16 mega of stack which I think is a little too much  ;)

The solution is to malloc() but check if you have 16 mega of ram ;)


Ankit Jain wrote:
>       1 #include <stdio.h>
>       2 int main()
>       3 {
>       4    double a[1450][1450];
>       5
>       6    a[1449][0] = 999;
>       7    printf( "%lf\n", a[1449][0] );
>       8    return 1;
>       9 }
> 
> 
> it gives segementation fault
> 
> if i use malloc also it gives wrong result
> 
> what to do?
> 
> ankit
> 
> ________________________________________________________________________
> Yahoo! Messenger - Communicate instantly..."Ping" 
> your friends today! Download Messenger Now 
> http://uk.messenger.yahoo.com/download/index.html
>