[Xenomai-help] Anti-memory-fragmentation tricks for my Xenomai process?

[Xenomai-help] Anti-memory-fragmentation tricks for my Xenomai process?

[Jeremy Friesner]

Hi all,

I've got a Xenomai-based program that consists of a Xenomai real-time thread, plus a number of non-realtime/regular Linux threads.

As the program runs, the regular Linux threads allocate and deallocate a fairly large number of 128kB buffers at various times, as well as doing various smaller allocations and deallocations of various sizes.  Pointers to these buffers are passed to the Xenomai real-time thread when appropriate, so that it can read data from the buffers as necessary.  (Steps are taken, of course, to make sure that the real-time thread stops reading from a buffer before it gets deallocated by a non-real-time thread).

This all works well, but there is one minor issue where I'd like to improve things:  after running for a while, the allocation and deallocation of 128kB buffers, interspersed with smaller allocations/deallocations, causes fragmentation in the process's heap, with the end result that the process appears to be using a lot more memory than it actually has allocated.  This doesn't cause real problems (AFAICT), but because the GUI for the system has a RAM Usage Indicator (that gets its info from /proc/meminfo, and displays total memory-usage as a percentage of physical RAM), users are prone to wonder why so much RAM is being used even when the program is idle.  The heap fragmentation makes it look like there is a memory leak, even when there technically isn't.

So my question is, is there some Xenomai-friendly way for me to allocate, say, 2GB of virtual address space, and manually control (from a non-realtime thread) when and how the underlying physical memory gets paged in and out of RAM?  That way I could do that at startup, and then allocate physical 128kB buffers as needed from that dedicated address space, without worrying about fragmentation caused by mixing the 128kB allocations with smaller ones in a single heap.

Note that I don't want to actually allocate 2GB of physical RAM at startup, since that would defeat the purpose of minimizing the program's RAM footprint  (User:  "why is it using 2GB already?  I haven't even done anything yet!").  So it seems like the required mlockall(MCL_CURRENT | MCL_FUTURE) command might cause me some headaches here.

I apologize if this is a newbiew/obvious question... any advice, or pointers to the web page(s) I should have read before posting will be welcomed.  :^)

Jeremy

Re: [Xenomai-help] Anti-memory-fragmentation tricks for my Xenomai process?

[Eric Noulard]

2011/7/8 Jeremy Friesner <jaf@domain.hid>:
> Hi all,
>
> I've got a Xenomai-based program that consists of a Xenomai real-time thread, plus a number of non-realtime/regular Linux threads.
>
> As the program runs, the regular Linux threads allocate and deallocate a fairly large number of 128kB buffers at various times, as well as doing various smaller allocations and deallocations of various sizes.  Pointers to these buffers are passed to the Xenomai real-time thread when appropriate, so that it can read data from the buffers as necessary.  (Steps are taken, of course, to make sure that the real-time thread stops reading from a buffer before it gets deallocated by a non-real-time thread).
>
> This all works well, but there is one minor issue where I'd like to improve things:  after running for a while, the allocation and deallocation of 128kB buffers, interspersed with smaller allocations/deallocations, causes fragmentation in the process's heap, with the end result that the process appears to be using a lot more memory than it actually has allocated.  This doesn't cause real problems (AFAICT), but because the GUI for the system has a RAM Usage Indicator (that gets its info from /proc/meminfo, and displays total memory-usage as a percentage of physical RAM), users are prone to wonder why so much RAM is being used even when the program is idle.  The heap fragmentation makes it look like there is a memory leak, even when there technically isn't.
>
> So my question is, is there some Xenomai-friendly way for me to allocate, say, 2GB of virtual address space, and manually control (from a non-realtime thread) when and how the underlying physical memory gets paged in and out of RAM?  That way I could do that at startup, and then allocate physical 128kB buffers as needed from that dedicated address space, without worrying about fragmentation caused by mixing the 128kB allocations with smaller ones in a single heap.
>
> Note that I don't want to actually allocate 2GB of physical RAM at startup, since that would defeat the purpose of minimizing the program's RAM footprint  (User:  "why is it using 2GB already?  I haven't even done anything yet!").  So it seems like the required mlockall(MCL_CURRENT | MCL_FUTURE) command might cause me some headaches here.
>
> I apologize if this is a newbiew/obvious question... any advice, or pointers to the web page(s) I should have read before posting will be welcomed.  :^)

If you know that the buffers you need have fixed size like 128kB then
you could write your own fixed-size buffer allocator.

Initially you create a memory zone whose size is say N*128kB + Management Space
(N of your blocks plus space for managing the blocks) and then you
maintain the list of used
block using your own void* get_block(), release_block(void*).
The "only" problem here is to chose N, which should be the maximum
number of 128kB memory block you need
at any time.

For the varying size part of the allocation, may be you can have a look at TLSF:
http://rtportal.upv.es/rtmalloc/

may be TLSF can handle the fixed sized block for you in a separate
memory pool as well.


-- 
Erk
Membre de l'April - « promouvoir et défendre le logiciel libre » -
http://www.april.org

Re: [Xenomai-help] Anti-memory-fragmentation tricks for my Xenomai process?

[Gilles Chanteperdrix]

On 07/08/2011 07:48 AM, Jeremy Friesner wrote:
> This all works well, but there is one minor issue where I'd like to
> improve things:  after running for a while, the allocation and
> deallocation of 128kB buffers, interspersed with smaller
> allocations/deallocations, causes fragmentation in the process's
> heap, with the end result that the process appears to be using a lot
> more memory than it actually has allocated.  This doesn't cause real
> problems (AFAICT), but because the GUI for the system has a RAM Usage
> Indicator (that gets its info from /proc/meminfo, and displays total
> memory-usage as a percentage of physical RAM), users are prone to
> wonder why so much RAM is being used even when the program is idle.
> The heap fragmentation makes it look like there is a memory leak,
> even when there technically isn't.

The size allocated by a process is more accurately measured with
/proc/self/maps or /proc/self/smaps than with /proc/meminfo. For the
128k buffers, you can mmap/munmap anonymous memory, this way, you will
only consume what you use. Malloc also has a threshold above which it
uses mmap, maybe you can change that threshold to be 128KiB (I think it
is 4MiB by default). See mallopt documentation.

-- 
					    Gilles.