System V msg queue bugs in latest kernels

System V msg queue bugs in latest kernels

[Mark Swanson]

Hello,

ipcs (msg) gives incorrect results if used-bytes is above 65536. It
stays at 65536 even though messages are being read and removed from the
msg queue.

The sysv msg queue either ignores the /proc/sys/kernel/msgmnb value if
it is above 65536 or simply gets it wrong. Proof: I can place more than
msgmnb bytes in a queue. The behavior is not consistent, but 100%
reproducable. It's not consistent because if I use messages of about
1000-2000 bytes the msgmnb never gets bigger than 65536 (even if
/proc/sys/kernel/msgmnb is set to 1048576 - bug). However, if I use
small messages like 13 bytes I can get bizarre (wrong) ipcs results
like this:

used-bytes    messages
65536          65536


Why does Linux ignore the /proc/sys/kernel/msgmnb value - or seem to
partly ignore it if it is above 65536? I *really* need this to be
around a MB. Is there an undocumented limit here or is this a bug?

Thanks.





=====
A camel is ugly but useful; it may stink, and it may spit, but it'll get you where you're going. - Larry Wall -

__________________________________________________
Do You Yahoo!?
Get personalized email addresses from Yahoo! Mail - only $35 
a year!  http://personal.mail.yahoo.com/

Re: System V msg queue bugs in latest kernels

[Manfred Spraul]

Mark Swanson wrote:
> 
> Hello,
> 
> ipcs (msg) gives incorrect results if used-bytes is above 65536. It
> stays at 65536 even though messages are being read and removed from the
> msg queue.
>
I'm testing it.

Could you check /proc/sysvipc/msg?

I know that several API's have 16-bit numbers, perhaps wrong values are
returned to user space.

--
	Manfred

Re: System V msg queue bugs in latest kernels

[Mark Swanson]

You are right.
/proc/sysvipc/msg is correct. It shows:
cbytes: 1048575
qnum: 95325

ipcs shows:
used-bytes: 65535
messages: 65535

It's a 16-bit number issue.

--- Manfred Spraul <manfred@colorfullife.com> wrote:
> Mark Swanson wrote:
> > 
> > Hello,
> > 
> > ipcs (msg) gives incorrect results if used-bytes is above 65536. It
> > stays at 65536 even though messages are being read and removed from
> the
> > msg queue.
> >
> I'm testing it.
> 
> Could you check /proc/sysvipc/msg?
> 
> I know that several API's have 16-bit numbers, perhaps wrong values
> are
> returned to user space.
> 
> --
> 	Manfred


=====
A camel is ugly but useful; it may stink, and it may spit, but it'll get you where you're going. - Larry Wall -

__________________________________________________
Do You Yahoo!?
Get personalized email addresses from Yahoo! Mail - only $35 
a year!  http://personal.mail.yahoo.com/

Re: System V msg queue bugs in latest kernels

[Mark Swanson]

The exact error is in /usr/include/linux/msg.h

The three unsigned shorts should be unsigned int instead.
Would too many things break if this was changed?
Should user-space tools like ipcs be rewritten to use /proc/sysvipc
instead? (I notice that my old 2.2.14 kernel doesn't have
/proc/sysvipc...)

Thanks.

/* one msqid structure for each queue on the system */
struct msqid_ds {
    struct ipc_perm msg_perm;
    struct msg *msg_first;      /* first message on queue */
    struct msg *msg_last;       /* last message in queue */
    __kernel_time_t msg_stime;  /* last msgsnd time */
    __kernel_time_t msg_rtime;  /* last msgrcv time */
    __kernel_time_t msg_ctime;  /* last change time */
    struct wait_queue *wwait;
    struct wait_queue *rwait;
    unsigned short msg_cbytes;  /* current number of bytes on queue */
    unsigned short msg_qnum;    /* number of messages in queue */
    unsigned short msg_qbytes;  /* max number of bytes on queue */
    __kernel_ipc_pid_t msg_lspid;   /* pid of last msgsnd */
    __kernel_ipc_pid_t msg_lrpid;   /* last receive pid */
}; 




=====
A camel is ugly but useful; it may stink, and it may spit, but it'll get you where you're going. - Larry Wall -

__________________________________________________
Do You Yahoo!?
Get personalized email addresses from Yahoo! Mail - only $35 
a year!  http://personal.mail.yahoo.com/

Re: System V msg queue bugs in latest kernels

[Manfred Spraul]

[-- Attachment #1: Type: text/plain, Size: 979 bytes --]

Manfred Spraul wrote:
> 
> Mark Swanson wrote:
> >
> > Hello,
> >
> > ipcs (msg) gives incorrect results if used-bytes is above 65536. It
> > stays at 65536 even though messages are being read and removed from the
> > msg queue.
> >

Ok, does the value stay at 65536 or 65535?
It should stay at 65535 if you use a too old version of util-linux.

Please upgrade (see linux/Documentation/Changes)

The proc interface at /proc/sysvipc/msg should report the correct
numbers.

If you want to access values > 65535 from your app you have 2 options:

1) use the new msqid64_ds structure. You must pass IPC_64 to the msgctl
call. This is the only option if you need correct 32-bit uids.
Check the util-linux source, I don't have sample code.
msqid64_ds is only supported by the 2.4 kernel.

2) the old msqid_ds structure also support 32-bit queue length, an
unused field was reused. No support for 32-bit uids.

#define msg_lqbytes	__rwait;

I've attached my old sample code.
--
	Manfred

[-- Attachment #2: longqueue.c --]
[-- Type: text/plain, Size: 2435 bytes --]

/*
 * This code is public domain sample code.
 * Written by Manfred Spraul, 1999
 *
 * The application must be started by root or
 * setuid(root).
 *
 * $Header: /pub/cvs/ms/ipcmsg/longqueue.c,v 1.2 1999/10/09 23:27:54 manfreds Exp $
 */

#include <sys/msg.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>

/* result codes:
 * 0: success
 * 1: partial success, queue len now USHORT_MAX
 * 100: invalid parameters.
 * 101: other error
 * 256: fatal error, please delete the queue: queue len now 0.
 */

#define USHORT_MAX	0xFFff
struct queuelen {
	int llen;
	unsigned short slen;
};

struct msqid_ds g_q;

#define msg_lqbytes		__rwait
void failure(char* msg)
{
	printf(" unexpected error in %s.\n",msg);
	exit(101);
}

int init_ipc(int id)
{
	int res;
	
	res = msgget(id,0);
	if(res == -1)
		failure("findkey()");
	id = res;
	res = msgctl(id,IPC_STAT,&g_q);
	if(res == -1)
		failure("init_ipc()");
	return id;
}

void get_queuelen(int id,
		struct queuelen *out)
{
	int res;
	struct msqid_ds q;

	res = msgctl(id,IPC_STAT,&q);
	if(res == -1)
		failure("get_queuelen()");
	out->llen = q.msg_lqbytes;
	out->slen = q.msg_qbytes;
}

int set_queuelen(int id, int len)
{
	struct msqid_ds q;

	memcpy(&q,&g_q,sizeof(q));
	if(len > USHORT_MAX) {
		q.msg_qbytes = 0;
		q.msg_lqbytes = len;
	} else
	{
		q.msg_qbytes = len;
	}
	return msgctl(id,IPC_SET,&q);
}

int main(int argc,char** argv)
{
	int id;
	int len;
	struct queuelen prev;
	struct queuelen new;

	printf("longqueue <id> <len>\n");
	if(argc != 3) {
		printf("Invalid parameters.\n");
		return 100;
	}
	id = atoi(argv[1]);
	len = atoi(argv[2]);
	if(len <= 0) {
		printf("Invalid parameters.\n");
		return 100;
	}
	id = init_ipc(id);
	get_queuelen(id,&prev);
	if(set_queuelen(id,len) == -1)
		failure("set_queuelen()");
	if(len <= USHORT_MAX) {
out_success:
		get_queuelen(id,&prev);
		printf(" new queuelen: (%d,%d).\n",prev.slen,prev.llen);
		return 0;
	}
	/* the old Linux ipcmsg code doesn't support
	 * long queues. It interprets this as "queue len 0".
	 * Check for this, and try USHORT_MAX, then the original
	 * value.
	 */
	get_queuelen(id,&new);
	if(new.slen != 0)
		goto out_success;

	if(set_queuelen(id,USHORT_MAX) == -1) {
		if(set_queuelen(id,prev.slen) == -1) {
			printf(" fatal error. queue len now 0.\n");
			return 256;
		};
		failure("set_queuelen()");
	}
	get_queuelen(id,&new);
	printf(" new queuelen: (%d,%d).\n",prev.slen,prev.llen);
	return 1;
}

Re: System V msg queue bugs in latest kernels

[Christopher Allen Wing]

Manfred:

> If you want to access values > 65535 from your app you have 2 options:
> 
> 1) use the new msqid64_ds structure. You must pass IPC_64 to the msgctl
> call. This is the only option if you need correct 32-bit uids.

glibc 2.2 will support this natively without needing any changes to your
application (besides a recompile). struct msqid_ds in the glibc 2.2
headers corresponds to struct msqid64_ds in the kernel.

It would be a bad thing to require people to change their source code in
order to build against the improved sysvipc interface :)

(glibc 2.2 also handles the case of a non 2.4 kernel properly, by
detecting the lack of IPC_64 support and emulating it in software-- Jakub
Jelinek wrote this compatibility code because I was too lazy/didn't need
it myself)

-Chris Wing
wingc@engin.umich.edu