* [Xenomai-help] Fwd: can't use message pipe - Cannot allocate memory
@ 2007-06-07 15:49 NZG
2007-06-07 17:03 ` Philippe Gerum
0 siblings, 1 reply; 12+ messages in thread
From: NZG @ 2007-06-07 15:49 UTC (permalink / raw)
To: xenomai
I'm trying to pass a message from user space to Xenomai and it doesn't seem
to be working. I keep getting an error when attempting to write to the pipe
in user space.
rt_pipe_write error
: Cannot allocate memory
This happens whether I use posix calls or real time calls.
If I change the poolsize to 0 for allocation from the heap, writes no longer
error out but the data is not received in real time space.
Does this work for anyone else? Anyone have any ideas I could try?
thx,
NZG
code specifics follow...................................................
3 differen POOLSIZE attempts
**************************************************
#define KOMMANDBUFFER 100
//#define POOLSIZE_CONTROL (sizeof(kommand_t)*KOMMANDBUFFER)
//#define POOLSIZE_CONTROL 0
#define POOLSIZE_CONTROL 1024
real-time thread
***********************************
if((err=rt_pipe_create (p, name, P_MINOR_AUTO, POOLSIZE_CONTROL))!=0){
error(0,-err, "could not create rt pipe");
return err;
}
/**
* retrieve a message from the message buffer
* and updates the kommand structure passed to it.
* @returns the type of command retrieved.
*
*/
static inline int getkommand(kommand_t *k){
int err;
err = rt_pipe_read(&command_pipe,k,sizeof(kommand_t),TM_NONBLOCK);
if(err<0)error(0,-err, "rt_pipe_read error\n");
return k->type;//return the type of message
}
while(getkommand(&kommand)!=KILLCYCLE);
user space using real time calls
***********************************
err=rt_pipe_write(&command_pipe, &k,sizeof(kommand_t),P_NORMAL);
if(err<0)error(0,-err, "rt_pipe_write error\n");
alternateuser space using posix calls
***********************************
int cfd = open(COMMANDPIPE, O_RDWR);
if(cfd<0){
printf("can't open %s (%s)\n",COMMANDPIPE, strerror(-cfd));
fflush(stdout);
return;
}
err=write(cfd,&k,sizeof(kommand_t));
if(err==0)printf("couldn't write to %s\n",COMMANDPIPE);
if(err<0)printf("err writing to %s (%s)\n",COMMANDPIPE,
strerror(errno));
if(close(cfd)<0)printf("couldn't close %s\n",COMMANDPIPE
-------------------------------------------------------
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [Xenomai-help] Fwd: can't use message pipe - Cannot allocate memory 2007-06-07 15:49 [Xenomai-help] Fwd: can't use message pipe - Cannot allocate memory NZG @ 2007-06-07 17:03 ` Philippe Gerum 2007-06-07 17:19 ` Philippe Gerum 0 siblings, 1 reply; 12+ messages in thread From: Philippe Gerum @ 2007-06-07 17:03 UTC (permalink / raw) To: NZG; +Cc: xenomai On Thu, 2007-06-07 at 10:49 -0500, NZG wrote: > I'm trying to pass a message from user space to Xenomai and it doesn't seem > to be working. I keep getting an error when attempting to write to the pipe > in user space. > > rt_pipe_write error > > : Cannot allocate memory > Issue confirmed here when passing a non-zero poolsize. Passing 0 does not seem to exhibit the problem. Could you re-check that both cases actually fail on your side? I have crafted a quick and dirty demo showing the problem. Until a non-zero value is passed to rt_pipe_create() to specify a local pool, everything is ok. (note: we could have used rt_pipe_write/rt_pipe_read indifferently, they both end up calling rt_pipe_send/receive). I'll have a look at this asap. --- /dev/null 2006-05-31 03:15:07.000000000 +0200 +++ pipe/module/module.c 2007-06-07 18:48:00.000000000 +0200 @@ -0,0 +1,90 @@ +#include <native/task.h> +#include <native/pipe.h> + +static RT_TASK task; + +static RT_PIPE pipe; + +#define ACK_STR "OK" + +static void test_task(void *cookie) +{ + RT_PIPE_MSG *msg, *ack; + ssize_t sz; + int s; + + printk("Pipe open: waiting...\n"); + + for (;;) { + sz = rt_pipe_receive(&pipe, &msg, TM_INFINITE); + + if (sz < 0) + break; + + printk("Read %d bytes => %.*s\n", sz, sz, P_MSGPTR(msg)); + + s = rt_pipe_free(&pipe, msg); + + if (s) { + printk("pipe free: failed %d\n", s); + goto fail; + } + + ack = rt_pipe_alloc(&pipe, 3); + + if (!ack) { + printk("pipe alloc: failed\n"); + goto fail; + } + + strcpy(P_MSGPTR(ack), ACK_STR); + + sz = rt_pipe_send(&pipe, ack, 3, 0); /* with \0 */ + + if (sz != sizeof(ACK_STR)) { + printk("pipe write: failed %d\n", sz); + goto fail; + } + } + + fail: + + return; +} + +int test_init_module(void) +{ + int s; + + s = rt_pipe_create(&pipe, "test", P_MINOR_AUTO, 0); + + if (s) { + printk("pipe open: failed %d\n", s); + goto fail; + } + + s = rt_task_create(&task, "pipe_task", 0, 1, 0); + + if (s) + printk("task create: failed %d\n", s); + else { + s = rt_task_start(&task, &test_task, NULL); + + if (s) + printk("task start: failed %d\n", s); + } + + fail: + return s; +} + +void test_cleanup_module(void) +{ + rt_pipe_delete(&pipe); + rt_task_delete(&task); +} + +module_init(test_init_module); +module_exit(test_cleanup_module); + +MODULE_LICENSE("GPL"); --- /dev/null 2006-05-31 03:15:07.000000000 +0200 +++ pipe/module/Makefile 2007-06-07 18:08:57.000000000 +0200 @@ -0,0 +1,5 @@ +EXTRA_CFLAGS += -Iinclude/xenomai + +obj-m += module_rt.o + +module_rt-objs := module.o --- /dev/null 2006-05-31 03:15:07.000000000 +0200 +++ pipe/pipe.c 2007-06-07 18:49:25.000000000 +0200 @@ -0,0 +1,37 @@ +#include <stdio.h> +#include <unistd.h> +#include <stdlib.h> +#include <fcntl.h> + +main() +{ + char buf[16]; + int fd, n; + + fd = open("/proc/xenomai/registry/native/pipes/test", O_RDWR); + + if (fd < 0) { + fprintf(stderr, "open(): %m\n"); + exit(1); + } + + printf("open(): %m, fd: %d\n", fd); + + n = write(fd, "Hello World", 11); + + if (n != 11) { + perror("write"); + exit(1); + } + + n = read(fd, buf, sizeof(buf)); + + if (n < 0) { + perror("write"); + exit(1); + } + + printf("Received %.*s, n=%d\n", n, buf, n); + + exit(0); +} -- Philippe. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Xenomai-help] Fwd: can't use message pipe - Cannot allocate memory 2007-06-07 17:03 ` Philippe Gerum @ 2007-06-07 17:19 ` Philippe Gerum 2007-06-07 18:11 ` NZG 0 siblings, 1 reply; 12+ messages in thread From: Philippe Gerum @ 2007-06-07 17:19 UTC (permalink / raw) To: NZG; +Cc: xenomai On Thu, 2007-06-07 at 19:03 +0200, Philippe Gerum wrote: > On Thu, 2007-06-07 at 10:49 -0500, NZG wrote: > > I'm trying to pass a message from user space to Xenomai and it doesn't seem > > to be working. I keep getting an error when attempting to write to the pipe > > in user space. > > > > rt_pipe_write error > > > > : Cannot allocate memory > > > > Issue confirmed here when passing a non-zero poolsize. Passing 0 does > not seem to exhibit the problem. Could you re-check that both cases > actually fail on your side? > > I have crafted a quick and dirty demo showing the problem. Until a > non-zero value is passed to rt_pipe_create() to specify a local pool, > everything is ok. (note: we could have used rt_pipe_write/rt_pipe_read > indifferently, they both end up calling rt_pipe_send/receive). > > I'll have a look at this asap. This is "normal" behaviour, albeit peculiar. Local pools for pipes are created on the fly, when the poolsize argument is non-zero, in which case, a real-time heap is created to manage the message pool behind the curtains. The init routine for real-time heaps will bail out when the required size is lower than two times the VM page size, plus some overhead induced by the meta-data, all rounded to the VM page size for your achitecture. In other words, on x86 for instance, you cannot create heaps smaller than ((4096 * 2) + align(overhead,4096)) bytes, which also apply to local message pools for pipes. Try passing 16384 to the poolsize argument to see if things get better. Obviously, if passing 0 still bugs for you, then we would still have another problem to solve. -- Philippe. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Xenomai-help] Fwd: can't use message pipe - Cannot allocate memory 2007-06-07 17:19 ` Philippe Gerum @ 2007-06-07 18:11 ` NZG 2007-06-07 18:22 ` Jan Kiszka 0 siblings, 1 reply; 12+ messages in thread From: NZG @ 2007-06-07 18:11 UTC (permalink / raw) To: Philippe Gerum; +Cc: xenomai Write does stop throwing errors with a pool of 16384. Still not receiving in the real time task though, I'll bang on it for a while to make sure it's not a PBKAC. thx, NZG On Thursday 07 June 2007 12:19 pm, Philippe Gerum wrote: > On Thu, 2007-06-07 at 19:03 +0200, Philippe Gerum wrote: > > On Thu, 2007-06-07 at 10:49 -0500, NZG wrote: > > > I'm trying to pass a message from user space to Xenomai and it doesn't > > > seem to be working. I keep getting an error when attempting to write to > > > the pipe in user space. > > > > > > rt_pipe_write error > > > > > > : Cannot allocate memory > > > > Issue confirmed here when passing a non-zero poolsize. Passing 0 does > > not seem to exhibit the problem. Could you re-check that both cases > > actually fail on your side? > > > > I have crafted a quick and dirty demo showing the problem. Until a > > non-zero value is passed to rt_pipe_create() to specify a local pool, > > everything is ok. (note: we could have used rt_pipe_write/rt_pipe_read > > indifferently, they both end up calling rt_pipe_send/receive). > > > > I'll have a look at this asap. > > This is "normal" behaviour, albeit peculiar. Local pools for pipes are > created on the fly, when the poolsize argument is non-zero, in which > case, a real-time heap is created to manage the message pool behind the > curtains. The init routine for real-time heaps will bail out when the > required size is lower than two times the VM page size, plus some > overhead induced by the meta-data, all rounded to the VM page size for > your achitecture. > > In other words, on x86 for instance, you cannot create heaps smaller > than ((4096 * 2) + align(overhead,4096)) bytes, which also apply to > local message pools for pipes. Try passing 16384 to the poolsize > argument to see if things get better. > > Obviously, if passing 0 still bugs for you, then we would still have > another problem to solve. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Xenomai-help] Fwd: can't use message pipe - Cannot allocate memory 2007-06-07 18:11 ` NZG @ 2007-06-07 18:22 ` Jan Kiszka 2007-06-07 18:37 ` NZG 0 siblings, 1 reply; 12+ messages in thread From: Jan Kiszka @ 2007-06-07 18:22 UTC (permalink / raw) To: NZG; +Cc: xenomai [-- Attachment #1: Type: text/plain, Size: 512 bytes --] NZG wrote: > Write does stop throwing errors with a pool of 16384. Just repeating for confirmation: rt_pipe_create with smaller, but non-zero pool sizes doesn't report some error? Actually, the rt_pipe_create code looks like it tries to round up appropriately. Then something's still fishy. > > Still not receiving in the real time task though, > I'll bang on it for a while to make sure it's not a PBKAC. Pipes are asymmetric: RT<->NRT only. Use rt_queue for inter-RT communication. Jan [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 250 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Xenomai-help] Fwd: can't use message pipe - Cannot allocate memory 2007-06-07 18:22 ` Jan Kiszka @ 2007-06-07 18:37 ` NZG 2007-06-07 19:27 ` Jan Kiszka 0 siblings, 1 reply; 12+ messages in thread From: NZG @ 2007-06-07 18:37 UTC (permalink / raw) To: Jan Kiszka; +Cc: xenomai > > Write does stop throwing errors with a pool of 16384. > > Just repeating for confirmation: rt_pipe_create with smaller, but > non-zero pool sizes doesn't report some error? Incorrect, if it's created with smaller it does report an error. If it's created with 0 however, it seems ok. (at least it doesn't throw an error) > Pipes are asymmetric: RT<->NRT only. RT<->NRT is my goal. essentially I 1. spawn off a real time task which creates a pipe to receive commands 2. throw commands at it from the user space thread which spawned the real time process (which is still NRT correct? I'm fuzzy on this). NZG ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Xenomai-help] Fwd: can't use message pipe - Cannot allocate memory 2007-06-07 18:37 ` NZG @ 2007-06-07 19:27 ` Jan Kiszka 2007-06-07 19:39 ` NZG ` (2 more replies) 0 siblings, 3 replies; 12+ messages in thread From: Jan Kiszka @ 2007-06-07 19:27 UTC (permalink / raw) To: NZG; +Cc: xenomai [-- Attachment #1: Type: text/plain, Size: 894 bytes --] NZG wrote: >>> Write does stop throwing errors with a pool of 16384. >> Just repeating for confirmation: rt_pipe_create with smaller, but >> non-zero pool sizes doesn't report some error? > Incorrect, if it's created with smaller it does report an error. If it's > created with 0 however, it seems ok. (at least it doesn't throw an error) You mean the write fails, but rt_pipe_create is fine. At least here, and Philippe noticed the same. Looks like the margin for minimal-sized heaps is broken. 2*PAGE_SIZE should be 3*PAGE_SIZE net space, Philippe? > >> Pipes are asymmetric: RT<->NRT only. > RT<->NRT is my goal. > essentially I > 1. spawn off a real time task which creates a pipe to receive commands > 2. throw commands at it from the user space thread which spawned the real time > process (which is still NRT correct? I'm fuzzy on this). It is NRT. Jan [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 250 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Xenomai-help] Fwd: can't use message pipe - Cannot allocate memory 2007-06-07 19:27 ` Jan Kiszka @ 2007-06-07 19:39 ` NZG 2007-06-07 20:15 ` Philippe Gerum 2007-06-07 22:29 ` Philippe Gerum 2 siblings, 0 replies; 12+ messages in thread From: NZG @ 2007-06-07 19:39 UTC (permalink / raw) To: Jan Kiszka; +Cc: xenomai > You mean the write fails, but rt_pipe_create is fine. At least here, and > Philippe noticed the same. Correct. thx, NZG ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Xenomai-help] Fwd: can't use message pipe - Cannot allocate memory 2007-06-07 19:27 ` Jan Kiszka 2007-06-07 19:39 ` NZG @ 2007-06-07 20:15 ` Philippe Gerum 2007-06-07 22:29 ` Philippe Gerum 2 siblings, 0 replies; 12+ messages in thread From: Philippe Gerum @ 2007-06-07 20:15 UTC (permalink / raw) To: Jan Kiszka; +Cc: xenomai On Thu, 2007-06-07 at 21:27 +0200, Jan Kiszka wrote: > NZG wrote: > >>> Write does stop throwing errors with a pool of 16384. > >> Just repeating for confirmation: rt_pipe_create with smaller, but > >> non-zero pool sizes doesn't report some error? > > Incorrect, if it's created with smaller it does report an error. If it's > > created with 0 however, it seems ok. (at least it doesn't throw an error) > > You mean the write fails, but rt_pipe_create is fine. At least here, and > Philippe noticed the same. Looks like the margin for minimal-sized heaps > is broken. 2*PAGE_SIZE should be 3*PAGE_SIZE net space, Philippe? Please leave me a few yea^H^H^Hmoments, I'm trying to understand my code... Mpf. > > > > >> Pipes are asymmetric: RT<->NRT only. > > RT<->NRT is my goal. > > essentially I > > 1. spawn off a real time task which creates a pipe to receive commands > > 2. throw commands at it from the user space thread which spawned the real time > > process (which is still NRT correct? I'm fuzzy on this). > > It is NRT. > > Jan > -- Philippe. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Xenomai-help] Fwd: can't use message pipe - Cannot allocate memory 2007-06-07 19:27 ` Jan Kiszka 2007-06-07 19:39 ` NZG 2007-06-07 20:15 ` Philippe Gerum @ 2007-06-07 22:29 ` Philippe Gerum 2007-06-07 22:47 ` Jan Kiszka 2 siblings, 1 reply; 12+ messages in thread From: Philippe Gerum @ 2007-06-07 22:29 UTC (permalink / raw) To: Jan Kiszka; +Cc: xenomai On Thu, 2007-06-07 at 21:27 +0200, Jan Kiszka wrote: > NZG wrote: > >>> Write does stop throwing errors with a pool of 16384. > >> Just repeating for confirmation: rt_pipe_create with smaller, but > >> non-zero pool sizes doesn't report some error? > > Incorrect, if it's created with smaller it does report an error. If it's > > created with 0 however, it seems ok. (at least it doesn't throw an error) > > You mean the write fails, but rt_pipe_create is fine. At least here, and > Philippe noticed the same. Looks like the margin for minimal-sized heaps > is broken. 2*PAGE_SIZE should be 3*PAGE_SIZE net space, Philippe? Actually, rt_pipe_create() already rounds this value to 3*PAGE_SIZE, right before calling the sysalloc service. Here is the sequence of events that leads to the situation Nathan is seeing: - Passing poolsize = 1024 to rt_pipe_create() creates a local heap of 12288 bytes (3 * PAGE_SZ), with 32 bytes of overhead taken from one of these pages to hold the meta-data. - 4112 bytes (4096 + sizeof(message header)) are then requested to this local pool to hold the internal streaming buffer (XENO_OPT_NATIVE_PIPE_BUFSZ), which ends up consuming two pages, i.e. 8192 bytes, from this pool. The reason for this is due to the way the McKusick allocation scheme we use works; basically, block sizes greater than the page size are always rounded to a multiple of the page size. Requested block sizes are always rounded up to the nearest power of two, which the allocator groups in pages holding blocks of the same size. This is not pretty wrt internal fragmentation, but quite efficient CPU-wise, when the page size is properly chosen wrt the most common allocation pattern, that is. Most importantly, blocks greater than a page will never lay on partially consumed pages (by other blocks). So, in our case, we started with three free pages, one already holds some meta-data, and we need two pages to fullfil the current allocation request. Therefore, after this request has suceeded, we have no page left in the pool. - When the write() call is issued, the pipe driver requests a 32 bytes block to hold the data moving from user-space to kernel space. Too bad, we have no other page left to dedicate to blocks holding that size, so no block is available, which in turn causes write() to return -ENOMEM. This situation illustrates the conflict which is raised when small heaps (1k) are mapped on large page sizes (4k). Even when rounding them to a (small) multiple of the page size, the pool might rapidly run short of free pages, depending on the allocation pattern. IOW, when configuring a heap, it is better to know which kind of block sizes are going to be requested from it, and reserve the appropriate number of pages for each different size when evaluating the total size of the heap. Yes, it's not that simple. No, I'm not that sorry. -- Philippe. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Xenomai-help] Fwd: can't use message pipe - Cannot allocate memory 2007-06-07 22:29 ` Philippe Gerum @ 2007-06-07 22:47 ` Jan Kiszka 2007-06-08 6:55 ` [Xenomai-core] " Philippe Gerum 0 siblings, 1 reply; 12+ messages in thread From: Jan Kiszka @ 2007-06-07 22:47 UTC (permalink / raw) To: rpm; +Cc: xenomai [-- Attachment #1: Type: text/plain, Size: 3456 bytes --] Philippe Gerum wrote: > On Thu, 2007-06-07 at 21:27 +0200, Jan Kiszka wrote: >> NZG wrote: >>>>> Write does stop throwing errors with a pool of 16384. >>>> Just repeating for confirmation: rt_pipe_create with smaller, but >>>> non-zero pool sizes doesn't report some error? >>> Incorrect, if it's created with smaller it does report an error. If it's >>> created with 0 however, it seems ok. (at least it doesn't throw an error) >> You mean the write fails, but rt_pipe_create is fine. At least here, and >> Philippe noticed the same. Looks like the margin for minimal-sized heaps >> is broken. 2*PAGE_SIZE should be 3*PAGE_SIZE net space, Philippe? > > Actually, rt_pipe_create() already rounds this value to 3*PAGE_SIZE, > right before calling the sysalloc service. > > Here is the sequence of events that leads to the situation Nathan is > seeing: > > - Passing poolsize = 1024 to rt_pipe_create() creates a local heap of > 12288 bytes (3 * PAGE_SZ), with 32 bytes of overhead taken from one of > these pages to hold the meta-data. > > - 4112 bytes (4096 + sizeof(message header)) are then requested to this > local pool to hold the internal streaming buffer > (XENO_OPT_NATIVE_PIPE_BUFSZ), which ends up consuming two pages, i.e. > 8192 bytes, from this pool. The reason for this is due to the way the > McKusick allocation scheme we use works; basically, block sizes greater > than the page size are always rounded to a multiple of the page size. > Requested block sizes are always rounded up to the nearest power of two, > which the allocator groups in pages holding blocks of the same size. > This is not pretty wrt internal fragmentation, but quite efficient > CPU-wise, when the page size is properly chosen wrt the most common > allocation pattern, that is. Most importantly, blocks greater than a > page will never lay on partially consumed pages (by other blocks). So, > in our case, we started with three free pages, one already holds some > meta-data, and we need two pages to fullfil the current allocation > request. Therefore, after this request has suceeded, we have no page > left in the pool. > > - When the write() call is issued, the pipe driver requests a 32 bytes > block to hold the data moving from user-space to kernel space. Too bad, > we have no other page left to dedicate to blocks holding that size, so > no block is available, which in turn causes write() to return -ENOMEM. Ah, I see. > > This situation illustrates the conflict which is raised when small heaps > (1k) are mapped on large page sizes (4k). Even when rounding them to a > (small) multiple of the page size, the pool might rapidly run short of > free pages, depending on the allocation pattern. Reminds me of the TLSF allocator claiming to perform smartly also on smaller hunks. But that code is still 32-bit-focused, and would still need someone to define and run comparative tests on representative Xenomai setups. :-/ > > IOW, when configuring a heap, it is better to know which kind of block > sizes are going to be requested from it, and reserve the appropriate > number of pages for each different size when evaluating the total size > of the heap. > > Yes, it's not that simple. No, I'm not that sorry. > For now, can we take XENO_OPT_NATIVE_PIPE_BUFSZ into account when picking a reasonable minimum size in rt_pipe_create on behalf of a close-fisted user? Jan [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 250 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Xenomai-core] [Xenomai-help] Fwd: can't use message pipe - Cannot allocate memory 2007-06-07 22:47 ` Jan Kiszka @ 2007-06-08 6:55 ` Philippe Gerum 0 siblings, 0 replies; 12+ messages in thread From: Philippe Gerum @ 2007-06-08 6:55 UTC (permalink / raw) To: Jan Kiszka; +Cc: xenomai, xenomai On Fri, 2007-06-08 at 00:47 +0200, Jan Kiszka wrote: > Philippe Gerum wrote: > > On Thu, 2007-06-07 at 21:27 +0200, Jan Kiszka wrote: > >> NZG wrote: > >>>>> Write does stop throwing errors with a pool of 16384. > >>>> Just repeating for confirmation: rt_pipe_create with smaller, but > >>>> non-zero pool sizes doesn't report some error? > >>> Incorrect, if it's created with smaller it does report an error. If it's > >>> created with 0 however, it seems ok. (at least it doesn't throw an error) > >> You mean the write fails, but rt_pipe_create is fine. At least here, and > >> Philippe noticed the same. Looks like the margin for minimal-sized heaps > >> is broken. 2*PAGE_SIZE should be 3*PAGE_SIZE net space, Philippe? > > > > Actually, rt_pipe_create() already rounds this value to 3*PAGE_SIZE, > > right before calling the sysalloc service. > > > > Here is the sequence of events that leads to the situation Nathan is > > seeing: > > > > - Passing poolsize = 1024 to rt_pipe_create() creates a local heap of > > 12288 bytes (3 * PAGE_SZ), with 32 bytes of overhead taken from one of > > these pages to hold the meta-data. > > > > - 4112 bytes (4096 + sizeof(message header)) are then requested to this > > local pool to hold the internal streaming buffer > > (XENO_OPT_NATIVE_PIPE_BUFSZ), which ends up consuming two pages, i.e. > > 8192 bytes, from this pool. The reason for this is due to the way the > > McKusick allocation scheme we use works; basically, block sizes greater > > than the page size are always rounded to a multiple of the page size. > > Requested block sizes are always rounded up to the nearest power of two, > > which the allocator groups in pages holding blocks of the same size. > > This is not pretty wrt internal fragmentation, but quite efficient > > CPU-wise, when the page size is properly chosen wrt the most common > > allocation pattern, that is. Most importantly, blocks greater than a > > page will never lay on partially consumed pages (by other blocks). So, > > in our case, we started with three free pages, one already holds some > > meta-data, and we need two pages to fullfil the current allocation > > request. Therefore, after this request has suceeded, we have no page > > left in the pool. > > > > - When the write() call is issued, the pipe driver requests a 32 bytes > > block to hold the data moving from user-space to kernel space. Too bad, > > we have no other page left to dedicate to blocks holding that size, so > > no block is available, which in turn causes write() to return -ENOMEM. > > Ah, I see. > > > > > This situation illustrates the conflict which is raised when small heaps > > (1k) are mapped on large page sizes (4k). Even when rounding them to a > > (small) multiple of the page size, the pool might rapidly run short of > > free pages, depending on the allocation pattern. > > Reminds me of the TLSF allocator claiming to perform smartly also on > smaller hunks. But that code is still 32-bit-focused, and would still > need someone to define and run comparative tests on representative > Xenomai setups. :-/ Yep, I see this patch series pending into my patch fridge right now. > > > > > IOW, when configuring a heap, it is better to know which kind of block > > sizes are going to be requested from it, and reserve the appropriate > > number of pages for each different size when evaluating the total size > > of the heap. > > > > Yes, it's not that simple. No, I'm not that sorry. > > > > For now, can we take XENO_OPT_NATIVE_PIPE_BUFSZ into account when > picking a reasonable minimum size in rt_pipe_create on behalf of a > close-fisted user? > Yes, the size of the streaming buffer has to be taken into account, it's clearly missing right now. I'm also going to use another page size than PAGE_SIZE, and this change may likely be applicable to almost all callers of xnheap_init() actually. The initial intent was to pick a value which would be consistent with the natural block size for the Linux VM, but thinking about it once more, it's mostly irrelevant, since the way the heap manager works is orthogonal with how kmalloc/vmalloc organizes the core memory anyway (i.e. heap memory is obtained from a single sysalloc call, so the worst-case incurred here is only having some external fragmentation due to the last VM page not being fully occupied by heap memory). 512 would be likely a better page size for most usage patterns; this would increase the overhead brought by the meta-data, but we are talking about 1 byte per addressable page within the heap, so this should not be that penalizing anyway. > Jan > -- Philippe. ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2007-06-08 6:55 UTC | newest] Thread overview: 12+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2007-06-07 15:49 [Xenomai-help] Fwd: can't use message pipe - Cannot allocate memory NZG 2007-06-07 17:03 ` Philippe Gerum 2007-06-07 17:19 ` Philippe Gerum 2007-06-07 18:11 ` NZG 2007-06-07 18:22 ` Jan Kiszka 2007-06-07 18:37 ` NZG 2007-06-07 19:27 ` Jan Kiszka 2007-06-07 19:39 ` NZG 2007-06-07 20:15 ` Philippe Gerum 2007-06-07 22:29 ` Philippe Gerum 2007-06-07 22:47 ` Jan Kiszka 2007-06-08 6:55 ` [Xenomai-core] " Philippe Gerum
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.