POSIX message queues should not allocate memory on send

POSIX message queues should not allocate memory on send

[Martijn Sipkema]

There is a problem with the current POSIX message queue
implementation. mq_send()/mq_timedsend() may not return
ENOMEM and this means memory for mq_maxmsg*mq_msgsize
will have to be allocated on queue creation. I think POSIX MSG
message passing being part of the REALTIME extensions this
makes sense. I've already mentioned this once to the implementors
of the current implementation, but they did not agree with my
reading of the standard.

The default mq_msgsize also seems a little large to me, but
I don't see why defaults are needed; if I understand the standard
correctly then creating a new message queue without mq_attr
should create an empty queue, which thus cannot be used to
pass messages.

--ms

Re: POSIX message queues should not allocate memory on send

[Jakub Jelinek]

On Fri, May 14, 2004 at 11:30:53AM +0100, Martijn Sipkema wrote:
> The default mq_msgsize also seems a little large to me, but
> I don't see why defaults are needed; if I understand the standard
> correctly then creating a new message queue without mq_attr
> should create an empty queue, which thus cannot be used to
> pass messages.

No idea where you found this.

"If attr is NULL, the message queue shall be created with
implementation-defined default message queue attributes."

Empty queue means a message queue which has no messages in it, not
that mq_msgsize and/or mq_maxmsg is 0.
And mq_open with mq_msgsize 0 and/or mq_maxmsg 0 must fail (with EINVAL),
so the implementation-defined defaults IMHO must be > 0 for both
limits.

"     The mq_open() function shall fail if:"
...
"     [EINVAL]
             O_CREAT was specified in oflag, the value of attr is not NULL,
	     and either mq_maxmsg or mq_msgsize was less than or equal to zero."

	Jakub

Re: POSIX message queues should not allocate memory on send

[Martijn Sipkema]

> On Fri, May 14, 2004 at 11:30:53AM +0100, Martijn Sipkema wrote:
> > The default mq_msgsize also seems a little large to me, but
> > I don't see why defaults are needed; if I understand the standard
> > correctly then creating a new message queue without mq_attr
> > should create an empty queue, which thus cannot be used to
> > pass messages.
>
> No idea where you found this.
>
> "If attr is NULL, the message queue shall be created with
> implementation-defined default message queue attributes."
>
> Empty queue means a message queue which has no messages in it, not
> that mq_msgsize and/or mq_maxmsg is 0.
> And mq_open with mq_msgsize 0 and/or mq_maxmsg 0 must fail (with EINVAL),
> so the implementation-defined defaults IMHO must be > 0 for both
> limits.
>
> "     The mq_open() function shall fail if:"
> ...
> "     [EINVAL]
>              O_CREAT was specified in oflag, the value of attr is not
NULL,
>      and either mq_maxmsg or mq_msgsize was less than or equal to zero."

You are correct; defaults are indeed needed. The current default value
for mq_msgsize seems rather large considering that mq_msgsize*mq_maxmsg
bytes will have to be allocated on queue creation. If variable sized large
payload messages are needed one might consider using shared memory in
combination with a message queue.

My main point was that mq_send()/mq_timedsend() may not return ENOMEM
and I am positive I did not misread the standard on that.

--ms

Re: POSIX message queues should not allocate memory on send

[Jakub Jelinek]

On Fri, May 14, 2004 at 12:09:46PM +0100, Martijn Sipkema wrote:
> You are correct; defaults are indeed needed. The current default value
> for mq_msgsize seems rather large considering that mq_msgsize*mq_maxmsg
> bytes will have to be allocated on queue creation. If variable sized large
> payload messages are needed one might consider using shared memory in
> combination with a message queue.
> 
> My main point was that mq_send()/mq_timedsend() may not return ENOMEM
> and I am positive I did not misread the standard on that.

Even that is not clear.
http://www.opengroup.org/onlinepubs/009695399/functions/xsh_chap02_03.html#tag_02_03
"   Implementations may support additional errors not included in this list,
    may generate errors included in this list under circumstances other than
    those described here, or may contain extensions or limitations that
    prevent some errors from occurring.  The ERRORS section on each
    reference page specifies whether an error shall be returned, or whether
    it may be returned.  Implementations shall not generate a different error
    number from the ones described here for error conditions described in
    this volume of IEEE Std 1003.1-2001, but may generate additional errors
    unless explicitly disallowed for a particular function."

Explicitely disallowed in the general section is only EINTR for the THR
option functions unless explitely listed for that function and nothing else.

I don't see ENOMEM explicitly forbidden for mq_send/mq_timedsend nor
any wording in mq_open description which would require the buffers to
be preallocated.

	Jakub

Re: POSIX message queues should not allocate memory on send

[Martijn Sipkema]

> On Fri, May 14, 2004 at 12:09:46PM +0100, Martijn Sipkema wrote:
> > You are correct; defaults are indeed needed. The current default value
> > for mq_msgsize seems rather large considering that mq_msgsize*mq_maxmsg
> > bytes will have to be allocated on queue creation. If variable sized
large
> > payload messages are needed one might consider using shared memory in
> > combination with a message queue.
> >
> > My main point was that mq_send()/mq_timedsend() may not return ENOMEM
> > and I am positive I did not misread the standard on that.
>
> Even that is not clear.
>
http://www.opengroup.org/onlinepubs/009695399/functions/xsh_chap02_03.html#tag_02_03
> "   Implementations may support additional errors not included in this
list,
>     may generate errors included in this list under circumstances other
than
>     those described here, or may contain extensions or limitations that
>     prevent some errors from occurring.  The ERRORS section on each
>     reference page specifies whether an error shall be returned, or
whether
>     it may be returned.  Implementations shall not generate a different
error
>     number from the ones described here for error conditions described in
>     this volume of IEEE Std 1003.1-2001, but may generate additional
errors
>     unless explicitly disallowed for a particular function."
>
> Explicitely disallowed in the general section is only EINTR for the THR
> option functions unless explitely listed for that function and nothing
else.
>
> I don't see ENOMEM explicitly forbidden for mq_send/mq_timedsend nor
> any wording in mq_open description which would require the buffers to
> be preallocated.

Indeed maybe returning ENOMEM from mq_send() is conforming to the
standard. Perhaps returning ENOMEM instead of ENOSPC from
mq_open() to indicate that there is unsufficient space for the creation of
a new queue isn't; I'm no longer sure.

What _is_ clear from the standard is that a queue is supposed to be
allocated on creation, even if it may not be required. Why else would
ENOSPC be explicitely listed for mq_open() when creating a new
queue, but not for mq_send()/mq_timedsend()?

What use is a message queue for a realtime application when on
mq_send() it may have to wait for memory allocation that may even
fail?

Even if POSIX allows errors to be returned that are not in the function's
ERRORS list, I don't think one should do this if it can be avoided.

--ms

Re: POSIX message queues should not allocate memory on send

[Ulrich Drepper]

Martijn Sipkema wrote:

> Indeed maybe returning ENOMEM from mq_send() is conforming to the
> standard.

Indeed, it is.


> What _is_ clear from the standard is that a queue is supposed to be
> allocated on creation, even if it may not be required.

Nothing is clear when it comes to the behavior.  There are certainly
some aspects of a possible implementation which the designers of the
APIs had in mind, but this is absolutely no requirement.


> What use is a message queue for a realtime application when on
> mq_send() it may have to wait for memory allocation that may even
> fail?

That's a completely different issue.  You want everybody to suffer
because of the requirements of have.  This is the same as if you would
demand the overcommit is disable.

Yes, there are situations where the allocation-on-demand is not
desirable and the POSIX API description is indeed hinting at such uses.
 But the APIs are useful even without having such hard requirements and
so far the people who developed the code have not seen this hard
realtime requirement.

If you have such requirements I suggest that you sit done and write some
extensions.  I.e., do not replace the current default behavior, but
instead make it possible to force pre-allocation.  I'd say the natural
way is to define a O_* flag which only mq_open() and mq_setattr()
understands.  The former will get it in the second parameter, the latter
via the mq_flags field in the incoming struct mq_attr structure.
Something like O_PREALLOCATE.  Again, the flag would only be needed for
message queues, so a bit other than the ones used by mq_open() can be
reused.


> Even if POSIX allows errors to be returned that are not in the function's
> ERRORS list, I don't think one should do this if it can be avoided.

Do your part to fulfill your niche-requirements.

-- 
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖

Re: POSIX message queues should not allocate memory on send

[Martijn Sipkema]

From: "Jakub Jelinek" <jakub@redhat.com>
> On Fri, May 14, 2004 at 12:09:46PM +0100, Martijn Sipkema wrote:
> > You are correct; defaults are indeed needed. The current default value
> > for mq_msgsize seems rather large considering that mq_msgsize*mq_maxmsg
> > bytes will have to be allocated on queue creation. If variable sized large
> > payload messages are needed one might consider using shared memory in
> > combination with a message queue.
> > 
> > My main point was that mq_send()/mq_timedsend() may not return ENOMEM
> > and I am positive I did not misread the standard on that.
> 
> Even that is not clear.
> http://www.opengroup.org/onlinepubs/009695399/functions/xsh_chap02_03.html#tag_02_03
> "   Implementations may support additional errors not included in this list,
>     may generate errors included in this list under circumstances other than
>     those described here, or may contain extensions or limitations that
>     prevent some errors from occurring.  The ERRORS section on each
>     reference page specifies whether an error shall be returned, or whether
>     it may be returned.  Implementations shall not generate a different error
>     number from the ones described here for error conditions described in
>     this volume of IEEE Std 1003.1-2001, but may generate additional errors
>     unless explicitly disallowed for a particular function."
> 
> Explicitely disallowed in the general section is only EINTR for the THR
> option functions unless explitely listed for that function and nothing else.
> 
> I don't see ENOMEM explicitly forbidden for mq_send/mq_timedsend nor
> any wording in mq_open description which would require the buffers to
> be preallocated.

You are correct, but from the mq_send/mq_timedsend DESCRIPTION:

``If the specified message queue is not full, mq_send() shall behave as if the
message is inserted into the message queue at the position indicated by the
msg_prio argument.''

and from ERRORS:

``The mq_send() and mq_timedsend() functions shall fail if:
[EAGAIN] The O_NONBLOCK flag is set in the message queue description
associated with mqdes, and the specified message queue is full.''

Thus, if full could be interpreted as to also mean that there is no memory
to store the message, returning ENOMEM is not allowed and EAGAIN
would have to be used in stead since an implementation isn't allowed to
use a different error number for the same error condition.

If full is taken to mean that the queue contains its maximum number
of messages, which I think is the correct meaning, then returning ENOMEM
is not allowed, i.e. the queue has to be preallocated, I think, since the
description states that a message shall be added if the queue is not full.

--ms

Re: POSIX message queues should not allocate memory on send

[Chris Wright]

* Martijn Sipkema (m.j.w.sipkema@student.tudelft.nl) wrote:
> You are correct; defaults are indeed needed. The current default value
> for mq_msgsize seems rather large considering that mq_msgsize*mq_maxmsg
> bytes will have to be allocated on queue creation. If variable sized large
> payload messages are needed one might consider using shared memory in
> combination with a message queue.

The defaults have been reduced in -mm tree which should make it to
mainline in relative near future.

#define DFLT_MSGMAX	10	/* max number of messages in each queue */
#define DFLT_MSGSIZEMAX	8192	/* max message size */

> My main point was that mq_send()/mq_timedsend() may not return ENOMEM
> and I am positive I did not misread the standard on that.

I'm not sure it's that clear, however, we somewhat adhered to this
principle when adding rlimits to mqueues, so rlimit checks are enforced
on mq_open() as if whole thing was pre-allocated.

thanks,
-chris
-- 
Linux Security Modules     http://lsm.immunix.org     http://lsm.bkbits.net