* [PATCH 0/5] RFC: KBUS messaging subsystem
@ 2011-02-27 18:11 Tony Ibbs
2011-02-27 18:11 ` [PATCH 1/5] Documentation for KBUS Tony Ibbs
` (2 more replies)
0 siblings, 3 replies; 15+ messages in thread
From: Tony Ibbs @ 2011-02-27 18:11 UTC (permalink / raw)
To: Linux-embedded; +Cc: Tibs at Kynesim, Richard Watts, Grant Likely
KBUS is a lightweight, Linux kernel mediated, message system,
particularly intended for use in embedded environments.
It is meant to be simple to use and understand. It is designed to
provide predictable message delivery, deterministic message ordering,
and a guaranteed reply for each request.
It is especially aimed at situations where existing solutions,
such as DBUS, cannot be used, typically because of system constraints.
We have various customers using KBUS in real life, and believe it to
be useful. I had a showcase table for KBUS at the ELCE in Cambridge,
October last year, and there seemed to be interest.
I am hoping to get a response from this list before attempting to send
to the kernel list, since I believe this list should contain people
who might be most interested in using it.
The KBUS project home page is at http://kbus-messaging.org/, from
which there are various useful links.
There is a working repository with Linux 2.6.37 patched for KBUS,
available via:
git pull git://github.com/crazyscot/linux-2.6-kbus.git kbus-2.6.37
This is a resubmission of the original RFC to the Linux-Embedded list,
having:
a) changed to use __u32 instead of uint32_t and
b) changed to use kernel messaging mechanisms (also removing
some messaging that should not be needed). These in response
to the suggestion of Sam Ravnborg
c) split into multiple in-line patches, as is normal practice,
and as requested by Grant Likely
Tibs (5):
Documentation for KBUS
External header file for KBUS
Kconfig files and Makefile for KBUS
Internal header file for KBUS
KBUS itself, the main source file
Documentation/Kbus.txt | 1222 +++++++++++
include/linux/kbus_defns.h | 666 ++++++
init/Kconfig | 3 +
ipc/Kconfig | 99 +
ipc/Makefile | 5 +
ipc/kbus.c | 4927 ++++++++++++++++++++++++++++++++++++++++++++
ipc/kbus_internal.h | 709 +++++++
7 files changed, 7631 insertions(+), 0 deletions(-)
create mode 100644 Documentation/Kbus.txt
create mode 100644 include/linux/kbus_defns.h
create mode 100644 ipc/Kconfig
create mode 100644 ipc/kbus.c
create mode 100644 ipc/kbus_internal.h
^ permalink raw reply [flat|nested] 15+ messages in thread* [PATCH 1/5] Documentation for KBUS 2011-02-27 18:11 [PATCH 0/5] RFC: KBUS messaging subsystem Tony Ibbs @ 2011-02-27 18:11 ` Tony Ibbs 2011-02-27 18:11 ` [PATCH 2/5] External header file " Tony Ibbs 2011-02-27 19:26 ` [PATCH 5/5] KBUS source file Tony Ibbs 2011-02-28 7:48 ` [PATCH 0/5] RFC: KBUS messaging subsystem Grant Likely 2 siblings, 1 reply; 15+ messages in thread From: Tony Ibbs @ 2011-02-27 18:11 UTC (permalink / raw) To: Linux-embedded; +Cc: Tibs at Kynesim, Richard Watts, Grant Likely From: Tibs <tibs@tonyibbs.co.uk> Signed-off-by: Tony Ibbs <tibs@tonyibbs.co.uk> --- Documentation/Kbus.txt | 1222 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 1222 insertions(+), 0 deletions(-) create mode 100644 Documentation/Kbus.txt diff --git a/Documentation/Kbus.txt b/Documentation/Kbus.txt new file mode 100644 index 0000000..7cf723f --- /dev/null +++ b/Documentation/Kbus.txt @@ -0,0 +1,1222 @@ +============================================= +KBUS -- Lightweight kernel-mediated messaging +============================================= + +Summary +======= +KBUS provides lightweight kernel-mediated messaging for Linux. + +* "lightweight" means that there is no intent to provide complex or + sophisticated mechanisms - if you need something more, consider DBUS or + other alternatives. + +* "kernel-mediated" means that the actual business of message passing and + message synchronisation is handled by a kernel module. + +* "for Linux" means what it says, since the Linux kernel is required. + +Initial use is expected to be in embedded systems. + +There is (at least initially) no intent to aim for a "fast" system - this is +not aimed at real-time systems. + +Although the implementation is kernel-mediated, there is a mechanism +("Limpets") for commnicating KBUS messages between buses and/or systems. + +Intentions +========== +KBUS is intended: + +* To be simple to use and simple to understand. +* To have a small codebase, written in C. +* To provide predictable message delivery. +* To give deterministic message ordering. +* To guarantee a reply to every request. + +It needs to be simple to use and understand because the expected users are +typically busy with other matters, and do not have time to spend learning +a complex messaging system. + +It needs to have a small codebase, written in C, because embedded systems +often lack resources, and may not have enough space for C++ libraries, or +messaging systems supporting more complex protocol stacks. + +Our own experience on embedded systems of various sizes indicates that +the last three points are especially important. + +Predictable message delivery means the user can know whether they can tell in +what circumstances messages will or will not be received. + +Deterministic message ordering means that all recipients of a given set of +messages will receive them in the same order as all other recpients (and this +will be the order in which the messages were sent). This is important when +several part of (for instance) an audio/video stack are interoperating. + +Guaranteeing that a request will always result in a reply means that the user +will be told if the intended replier has (for instance) crashed. This again +allows for simpler use of the system. + +The basics +========== +Python and C +------------ +Although the KBUS kernel module is written in C, the module tests are written +in Python, and there is a Python module providing useful interfaces, which is +expected to be the normal way of using KBUS from Python. + +There is also a C library (libkbus) which provides a similar level of +abstraction, so that C programmers can use KBUS without having to handle the +low level details of sockets and message datastructures. Note that the C +programer using KBUS does need to have some awareness of how KBUS messages +work in order to get memory management right. + +Messages +======== +Message names +------------- +All messages have names - for instance "$.Sensors.Kitchen". + +All message names start with "$.", followed by one or more alphanumeric words +separated by dots. There are two wildcard characters, "*" and "%", which can +be the last word of a name. + +Thus (in some notation or other):: + + name := '$.' [ word '.' ]+ ( word | '*' | '%' ) + word := alphanumerics + +Case is significant. There is probably a limit on the maximum size of a +subname, and also on the maximum length of a message name. + +Names form a name hierarchy or tree - so "$.Sensors" might have children +"$.Sensors.Kitchen" and "$.Sensors.Bedroom". + +If the last word of a name is "*", then this is a wildcard name that also +includes all the child names at that level and below -- i.e., all the names +that start with the name up to the "*". So "$.Sensors.*" includes +"$.Sensors.Kitchen", "$.Sensors.Bedroom", "$.Sensors.Kitchen.FireAlarm", +"$.Sensors.Kitchen.Toaster", "$.Sensors.Bedroom.FireAlarm", and so on. + +If the last word of a name is "%", then this is a wildcard name that also +includes all the child names at that level -- i.e., all the names obtained by +replacing the "%" by another word. So "$.Sensors.%" includes +"$.Sensors.Kitchen" and "$.Sensors.Bedroom", but not +"$.Sensors.Kitchen.Toaster". + +Message ids +----------- +Every message is expected to have a unique id. + +A message id is made up of two parts, a network id and a serial number. + +The network id is used to carry useful information when a message is +transferred from one KBUS system to another (for instance, over a bridge). By +default (for local messages) it is 0. + +A serial number is used to identify the particular message within a network. + +If a message is sent via KBUS with a network id of 0, then KBUS itself will +assign a new message id to the message, with the network id (still) 0, and +with the serial number one more than the last serial number assigned. Thus for +local messages, message ids ascend, and their order is deterministic. + +If a message is sent via KBUS with a non-zero network id, then KBUS does not +touch its message id. + +Network ids are represented textually as ``{n,s}``, where ``n`` is the +network id and ``s`` is the serial number. + + Message id {0,0} is reserved for use as an invalid message id. Both + network id and serial number are unsigned 32-bit integers. Note that this + means that local serial numbers will eventually wrap. + +Message content +--------------- +Messages are made of the following parts: + +:start and end guards: + + These are unsigned 32-bit words. 'start_guard' is notionally "Kbus", + and 'end_guard' (the 32 bit word after the rest of the message) is + notionally "subK". Obviously that depends on how one looks at the 32-bit + word. Every message shall start with a start guard and end with an end + guard (but see `Message implementation`_ for details). + + These provide some help in checking that a message is well formed, and in + particular the end guard helps to check for broken length fields. + + If the message layout changes in an incompatible manner (this has happened + once, and is strongly discouraged), then the start and end guards change. + +Unset +~~~~~ +Unset values are 0, or have zero length (as appropriate). + +It is not possible for a message name to be unset. + +The message header +~~~~~~~~~~~~~~~~~~ +:message id: identifies this particular message. This is made up of a network + id and a serial number, and is discussed in `Message ids`_. + + When replying to a message, copy this value into the 'In reply to' field. + +:in_reply_to: is the message id of the message that this is a reply to. + + This shall be set to 0 unless this message *is* a reply to a previous + message. In other words, if this value is non-0, then the message *is* a + reply. + +:to: is the Ksock id identifying who the message is to be sent to. + + When writing a new message, this should normally be set to 0, meaning + "anyone listening" (but see below if "state" is being maintained). + + When replying to a message, it shall be set to the 'from' value of the + orginal message. + + When constructing a request message (a message wanting a reply), then it can + be set to a specific replier's Ksock id. When such a message is sent, if the + replier bound (at that time) does not have that specific Ksock id, then the + send will fail. + +:from: indicates the Ksock id of the message's sender. + + When writing a new message, set this to 0, since KBUS will set it. + + When reading a message, this will have been set by KBUS. + +:orig_from: this indicates the original sender of a message, when being + transported via Limpet. This will be documented in more detail in the future. + +:final_to: this indicates the final target of a message, when being + transported via Limpet. This will be documented in more detail in the future. + +:extra: this is a zero field, for future expansion. KBUS will always set this + field to zero. + +:flags: indicates extra information about the message. See `Message Flags`_ + for detailed information. + + When writing a message, typical uses include: + + * the message is URGENT + * a reply is wanted + + When reading a message, typical uses include: + + * the message is URGENT + * a reply is wanted + * a reply is wanted from the specific reader + + The top 16 bits of the flags field is reserved for use by the user - KBUS + will not touch it. + +:name_length: is the length of the message name in bytes. This will always be + non-zero, as a message name must always be given. + +:data_length: is the length of the message data in bytes. It may be zero + if there is no data associated with this message. + +:name: identifies the message. It must be terminated with a + zero byte (as is normal for C - in the Python binding a normal Python string + can be used, and the this will be done for you). Byte ordering is according + to that of the platform. + + In an "entire" message (see `Message implementation`_ below) the name shall + be padded out to a multiple of 4 bytes. Neither the terminating zero byte + nor the padding are included in the name length. Padding should be with + zero bytes. + +:data: is optional. KBUS does not touch the content of the + data, but just copies it. Byte ordering is according to that of the + platform. + + In an "entire" message (see `Message implementation`_ below) the data shall, + if present, be padded out to a multiple of 4 bytes. This padding is not + included in the data length, and the padding bytes may be whatever byte + values are convenient to the user. KBUS does not guarantee to copy the exact + given padding bytes (in fact, current implementations just ignore them). + +Message implementation +~~~~~~~~~~~~~~~~~~~~~~ +There are two ways in which a message may be constructed, "pointy" and +"entire". +See the ``kbus_defns.h`` header file for details. + +.. note:: The Python binding hides most of the detail of the message + implementation from the user, so if you are using Python you may be able to + skip this section. + +In a "pointy" message, the ``name`` and ``data`` fields in the message header +are C pointers to the actual name and data. If there is no data, then the +``data`` field is NULL. This is probably the simplest form of message for a C +programmer to create. This might be represented as:: + + start_guard: 'Kbus' + id: (0,0) + in_reply_to: (0,0) + to: 0 + from: 0 + name_len: 6 + data_len: 0 + name: ---------------------------> "$.Fred" + data: NULL + end_guard: 'subK' + +or (with data):: + + start_guard: 'Kbus' + id: (0,0) + in_reply_to: (0,0) + to: 0 + from: 0 + name_len: 6 + data_len: 7 + name: ---------------------------> "$.Fred" + data: ---------------------------> "abc1234" + end_guard: 'subK' + +.. warning:: When writing a "pointy" message in C, be very careful not to + free the name and data between the ``write`` and the SEND, as it is + only when the message is sent that KBUS actually follows the ``name`` and + ``data`` pointers. + + *After* the SEND, KBUS will have taken its own copies of the name and + (any) data. + +In an "entire" message, both ``name`` and ``data`` fields are required to be +NULL. The message header is followed by the message name (padded as described +above), any message data (also padded), and another end guard. This might be +represented as:: + + start_guard: 'Kbus' + id: (0,0) + in_reply_to: (0,0) + to: 0 + from: 0 + name_len: 6 + data_len: 0 + name: NULL + data: NULL + end_guard: 'subK' + name_data: '$.Fred\x0\x0' + end_guard: 'subK' + +or (again with data):: + + start_guard: 'Kbus' + id: (0,0) + in_reply_to: (0,0) + to: 0 + from: 0 + name_len: 6 + data_len: 7 + name: NULL + data: NULL + end_guard: 'subK' + name_data: '$.Fred\x0\x0' + data_data: 'abc1234\x0' + end_guard: 'subK' + +Note that in these examples: + +1. The message name is padded out to 6 bytes of name, plus one of terminating + zero byte, plus another zero byte to make 8, but the message's ``name_len`` + is still 6. +2. When there is no data, there is no "data data" after the name data. +3. When there is data, the data is presented after the name, and is padded out + to a multiple of 4 bytes (but without the necessity for a terminating zero + byte, so it is possible to have no pad bytes if the data length is already + a multiple of 4). Again, the ``data_len`` always reflects the "real" data + length. +4. Although the data shown is presented as ASCII strings for these examples, + it really is just bytes, with no assumption of its content/meaning. + +When writing/sending messages, either form may be used (again, the "pointy" +form may be simpler for C programmers). + +When reading messages, however, the "entire" form is always returned - this +removes questions about needing to free multiple returned datastructures (for +instance, what to do if the user were to ask for the NEXTMSG, read a few +bytes, and then DISCARD the rest). + +Limits +~~~~~~ +Message names may not be shorter than 3 characters (since they must be at +least "$." plus another character). An arbitrary limit is also placed on the +maximum message length - this is currently 1000 characters, but may be +reviewed in the future. + +Message data may, of course, be of zero length. + +When reading a message, an "entire" message is always returned. + + .. note:: When using C to work with KBUS messages, it is generally + ill-advised to reference the message name and data "directly":: + + char *name = msg->name; + uint8_t *data = msg->data; + + since this will work for "pointy" messages, but not for "entire" + messages (where the ``name`` field will be NULL). Instead, it + is always better to do:: + + char *name = kbus_msg_name_ptr(msg); + uint8_t *data = kbus_msg_data_ptr(msg); + + regardless of the message type. + +Message flags +------------- +KBUS reserves the bottom 16 bits of the flags word for predefined purposes +(although not all of those bits are yet used), and guarantees not to touch the +top 16 bits, which are available for use by the programmer as a particular +application may wish. + +The WANT_A_REPLY bit is set by the sender to indicate that a +reply is wanted. This makes the message into a request. + + Note that setting the WANT_A_REPLY bit (i.e., a request) and + setting 'in_reply_to' (i.e., a reply) is bound to lead to + confusion, and the results are undefined (i.e., don't do it). + +The WANT_YOU_TO_REPLY bit is set by KBUS on a particular message +to indicate that the particular recipient is responsible for replying +to (this instance of the) message. Otherwise, KBUS clears it. + +The SYNTHETIC bit is set by KBUS when it generates a Status message, for +instance when a replier has gone away and will therefore not be sending a +reply to a request that has already been queued. + + Note that KBUS does not check that a sender has not set this + flag on a message, but doing so may lead to confusion. + +The URGENT bit is set by the sender if this message is to be +treated as urgent - i.e., it should be added to the *front* of the +recipient's message queue, not the back. + +Send flags +~~~~~~~~~~ +There are two "send" flags, ALL_OR_WAIT and ALL_OR_FAIL. +Either one may be set, or both may be unset. + + If both are set, the message will be rejected as invalid. + + Both flags are ignored in reply messages (i.e., messages with the + 'in_reply_to' field set). + +If a message has ALL_OR_FAIL set, then a SEND will only succeed if the message +could be added to all the (intended) recipient's message queues. Otherwise, +SEND returns -EBUSY. + +If a message has ALL_OR_WAIT set, then a SEND will only succeed if the message +could be added to all the (intended) recipient's message queues. Otherwise +SEND returns -EAGAIN. In this case, the message is still being sent, and the +caller should either call DISCARD (to drop it), or else use poll/select to +wait for the send to finish. It will not be possible to call "write" until the +send has completed or been discarded. + +These are primarily intended for use in debugging systems. In particular, note +that the mechanisms dealing with ALL_OR_WAIT internally are unlikely to be +very efficient. + +.. note:: The send flags will be less effective when messages are being + mediated via Limpets, as remote systems are involved. + +Things KBUS changes in a message +-------------------------------- +In general, KBUS leaves the content of a message alone - mostly so that an +individual KBUS module can "pass through" messages from another domain. +However, it does change: + +- the message id's serial number (but only if its network id is unset) +- the 'from' id (to indicate the Ksock this message was sent from) +- the WANT_YOU_TO_REPLY bit in the flags (set or cleared as appropriate) +- the SYNTHETIC bit, which will always be unset in a message sent by a + Sender + +KBUS will always set the 'extra' field to zero. + +Limpets will change: + +- the network id in any field that has one. +- the 'orig_from' and 'final_to' fields (which in general should only be + manipulated by Limpets). + +Types of message +================ +There are four basic message types: + +* Announcement -- a message aimed at any listeners, expecting no reply +* Request -- a message aimed at a replier, who is expected to reply +* Reply -- a reply to a request +* Status -- a message generated by KBUS + +The Python interface provides a Message base class, and subclasses thereof for +each of the "user" message types (but not currently for Status). + +Announcements +------------- +An announcement is the "plain" message type. It is a message that is being +sent for all bound listeners to "hear". + +When creating a new announcement message, it has: + + :message id: see `Message ids`_ + :in reply to: unset (it's not a reply) + :to: unset (all announcements are broadcast to any listeners) + :from: unset (KBUS will set it) + :flags: typically unset, see `Message flags`_ + :message name: as appropriate + :message data: as appropriate + +The Python interface provides an ``Announcement`` class to help in creating an +announcement message. + +Request message +--------------- +A request message is a message that wants a reply. + +Since only one Ksock may bind as a replier for a given message name, a +request message wants a reply from a single Ksock. By default, this is +whichever Ksock has bound to the message name at the moment of sending, but +see `Stateful transactions`_. + +When creating a new request message, it has: + + :message id: see `Message ids`_ + :in reply to: unset (it's not a reply) + :to: either unset, or a specific Ksock id if the request + should fail if that Ksock is (no longer) the replier + for this message name + :from: unset (KBUS will set it) + :flags: the "needs a reply" flag should be set. + KBUS will set the "you need to reply" flag in the + copy of the message delivered to its replier. + :message name: as appropriate + :message data: as appropriate + +When receiving a request message, the WANT_YOU_TO_REPLY flag will be set if it +is this recipient's responsibility to reply. + +The Python interface provides a ``Request`` class to help in creating a +request message. + +When a request message is sent, it is an error if there is no replier bound to +that message name. + +The message will, as normal, be delivered to all listeners, and will have the +"needs a reply" flag set wherever it is received. However, only the copy of +the message received by the replier will be marked with the WANT_YOU_TO_REPLY +flag. + + So, if a particular file descriptor is bound as listener and replier + for '$.Fred', it will receive two copies of the original message (one + marked as needing reply from that file descriptor). However, when the + reply is sent, only the "plain" listener will receive a copy of the reply + message. + +Reply message +------------- +A reply message is the expected response after reading a request message. + +A reply message is distinguished by having a non-zero 'in reply to' value. + +Each reply message is in response to a specific request, as indicated by the +'in reply to' field in the message. + +The replier is helped to remember that it needs to reply to a request, because +the request has the WANT_YOU_TO_REPLY flag set. + +When a reply is sent, all listeners for that message name will receive it. +However, the original replier will not. + +When creating a new reply message, it has: + + :message id: see `Message ids`_ + :in reply to: the request message's 'message id' + :to: the request message's 'from' id + :from: unset (KBUS will set it) + :flags: typically unset, see `Message flags`_ + :message name: the request message's 'message name' + :message data: as appropriate + +The Python interface provides a ``Reply`` class to help in creating a reply +message, but more usefully there is also a ``reply_to`` function that creates +a Reply Message from the original Request. + +Status message +-------------- +KBUS generates Status messages (also sometimes referred to as "synthetic" +messages) when a request message has been successfully sent, but the replier +is unable to reply (for instance, because it has closed its Ksock). KBUS thus +uses a Status message to provide the "reply" that it guarantees the sender +will get. + +As you might expect, a KBUS status message is thus (technically) a reply +message. + +A status message looks like: + + :message id: as normal + :in reply to: the 'message id' of the message whose sending or + processing caused this message. + :to: the Ksock id of the recipient of the message + :from: the Ksock id of the sender of the message - this will + be 0 if the sender is KBUS itself (which is assumed for + most exceptions) + :flags: typically unset, see `Message flags`_ + :message name: for KBUS exceptions, a message name in '$.KBUS.*' + :message data: for KBUS exceptions, normally absent + +KBUS status messages always have '$.KBUS.<something>' names (this may be a +multi-level <something>), and are always in response to a previous message, so +always have an 'in reply to'. + +Requests and Replies +-------------------- +KBUS guarantees that each Request will (eventually) be matched by a consequent +Reply (or Status [1]_) message, and only one such. + +The "normal" case is when the replier reads the request, and sends its own +reply back. + +If a Request message has been successfully SENT, there are the following other +cases to consider: + +1. The replier unbinds from that message name before reading the request + message from its queue. In this case, KBUS removes the message from the + repliers queue, and issues a "$.KBUS.Replier.Unbound" message. + +2. The replier closes itself (close the Ksock), but has not yet read the + message. In this case, KBUS issues a "$.KBUS.Replier.GoneAway" message. + +3. The replier closes itself (closes the Ksock), has read the message, but has + not yet (and now cannot) replied to it. In this case, KBUS issues a + "$.KBUS.Replier.Ignored" message. + +4. SEND did not complete, and the replier closes itself before the message can + be added to its message queue (by the POLL mechanism). In this case, KBUS + issues a "$.KBUS.Replier.Disappeared" message. + +5. SEND did not complete, and an error occurs when the POLL mechanims tries to + send the message. In this case, KBUS issues a "$.KBUS.ErrorSending" + message. + +In all these cases, the 'in_reply_to' field is set to the original request's +message id. In the first three cases, the 'from' field will be set to the +Ksock id of the (originally intended) replier. In the last two cases, that +information is not available, and a 'from' of 0 (indicating KBUS itself) is +used. + +.. [1] Remember that a Status message is essentially a specialisation of a + Reply message. + +.. note:: Limpets introduce some extra messages, which will be documented when + the proper Limpet documentation is written. + +KBUS end points - Ksocks +======================== +The KBUS devices +---------------- +Message interactions happen via the KBUS devices. Installing the KBUS kernel +module always creates ``/dev/kbus0``, it may also create ``/dev/kbus1``, and +so on. + + The number of devices to create is indicated by an argument at module + installation, for instance:: + + # insmod kbus.ko num_kbus_devices=10 + +Messages are sent by writing to a KBUS device, and received by reading from +the same device. A variety of useful ioctls are also provided. Each KBUS +device is independent - messages cannot be sent from ``/dev/kbus0`` to +``/dev/kbus1``, since there is no shared information. + +Ksocks +------ +Specifically, messages are written to and read from KBUS device file +descriptors. Each such is termed a *Ksock* - this is a simpler term than "file +descriptor", and has some resonance with "socket". + +Each Ksock may be any (one or more) of: + +* a Sender (opening the device for read/write) +* a Listener (only needing to open the device for read) +* a Replier (opening the device for read/write) + +Every Ksock has an id. This is a 32-bit unsigned number assigned by KBUS when +the device is opened. The value 0 is reserved for KBUS itself. + + The terms "listener id", "sender id", "replier id", etc., thus all refer + to a Ksock id, depending on what it is being used for. + +Senders +------- +Message senders are called "senders". A sender should open a Ksock for read +and write, as it may need to read replies and error/status messages. + +A message is sent by: + +1. Writing the message to the Ksock (using the standard ``write`` function) +2. Calling the SEND ioctl on the Ksock, to actually send the message. This + returns (via its arguments) the message id of the message sent. It also + returns status information about the send + + The status information is to be documented. + +The DISCARD ioctl can be used to "throw away" a partially written message, +before SEND has been called on it. + +If there are no listeners (of any type) bound to that message name, then the +message will be ignored. + +If the message is flagged as needing a reply, and there are no repliers bound +to that message name, then an error message will be sent to the sender, by +KBUS. + +It is not possible to send a message with a wildcard message name. + + As a restriction this makes the life of the implementor and documentor + easier. I believe it would also be confusing if provided. + +The sender does not need to bind to any message names in order to receive +error and status messages from KBUS. + +When a sender sends a Request, an internal note is made that it expects a +corresponding Reply (or possible a Status message from KBUS if the Replier +goes away or unbinds from that message name, before replying). A place for +that Reply is reserved in the sender's message queue. If the message queue +fills up (either with messages waiting to be read, or with reserved slots for +Replies), then the sender will not be able to send another Request until there +is room on the message queue again. + + Hopefully, this can be resolved by the sender reading a message off its + queue. However, if there are no messages to be read, and the queue is all + reserved for replies, the only solution is for the sender to wait for a + replier to send it something that it can then read. + +.. note:: What order do we describe things in? Don't forget: + + If the message being sent is a request, then the replier bound to that + message name will (presumably) write a reply to the request. Thus the normal + sequence for a request is likely to be: + + 1. write the request message + 2. read the reply + + The sender does *not* need to bind to anything in order to receive a reply to + a request it has sent. + + Of course, if a sender binds to listen to the name it uses for its + request, then it will get a copy of the request as sent, and it will + also get (an extra) copy of the reply. But see `Receiving messages once + only`_. + +Listeners +--------- +Message recipients are called "listeners". + +Listeners indicate that they want to receive particular messages, by using the +BIND ioctl on a Ksock to specify the name of the message that is to be +listened for. If the binding is to a wildcarded message name, then the +listener will receive all messages with names that match the wildcard. + +An ordinary listener will receive all messages with that name (sent to the +relevant Ksock). A listener may make more than one binding on the same Ksock +(indeed, it is allowed to bind to the same name more than once). + +Messages are received by: + +1. Using the NEXTMSG ioctl to request the next message (this also returns the + messages length in bytes) +2. Calling the standard ``read`` function to read the message data. + +If NEXTMSG is called again, the next message will be readied for reading, +whether the previous message has been read (or partially read) or not. + +If a listener no longer wants to receive a particular message name, then they +can unbind from it, using the UNBIND ioctl. The message name and flags used in +an UNBIND must match those in the corresponding BIND. Any messages in the +listener's message queue which match that unbinding will be removed from the +queue (i.e., the listener will not actually receive them). This does *not* +affect the message currently being read. + + Note that this has implication for binding and unbinding wildcards, + which must also match. + +Closing the Ksock also unbinds all the message bindings made on it. +It does not affect message bindings made on other Ksocks. + +Repliers +-------- +Repliers are a special sort of listener. + +For each message name, there may be a single "replier". A replier binds to a +message name in the same way as any other listener, but sets the "replier" +flag. If someone else has already bound to the same Ksock as a replier for +that message name, the request will fail. + +Repliers only receive Requests (messages that are marked as wanting a reply). + +A replier may (should? must?) reply to the request - this is done by sending +a Reply message through the Ksock from which the Request was read. + +It is perfectly legitimate to bind to a message as both replier and listener, +in which case two copies of the message will be read, once as replier, and +once as (just) listener (but see `Receiving messages once only`_). + + +When a request message is read by the appropriate replier, KBUS will mark +*that particular message* with the "you must reply" flag. This will not be set +on copies of that message read by any (non-replier) listeners. + + So, in the case where a Ksock is bound as replier and listener for the + same message name, only one of the two copies of the message received will + be marked as "you must reply". + +If a replier binds to a wildcarded message name, then they are the *default* +replier for any message names satisfying that wildcard. If another replier +binds to a more specific message name (matching that wildcard), +then the specific message name binding "wins" - the wildcard replier will no +longer receive that message name. + + In particular '$.Fred.Jim' is more specific than '$.Fred.%' which in turn + is more specific than '$.Fred.*' + +This means that if a wildcard replier wants to guarantee to see all the +messages matching their wildcard, they also need to bind as a listener for the +same wildcarded name. + +For example: + + Assume message names are of the form '$.Sensors.<Room>' or + '$.Sensors.<Room>.<Measurement>'. + + Replier 1 binds to '$.Sensors.*'. They will be the default replier for + all sensor requests. + + Replier 2 binds to '$.Sensors.%'. They will take over as the default + replier for any room specific requests. + + Replier 3 binds to '$.Sensors.Kitchen.Temperature'. They will take over as + the replier for the kitchen temperature. + + So: + + - A message named '$.Sensors.Kitchen.Temperature' will go to replier 3. + - A message named '$.Sensors.Kitchen' or '$.Sensors.LivingRoom' will go to + replier 2. + - A message named '$.Sensors.LivingRoom.Temperature' will go to replier 1. + +When a Replier is closed (technically, when its ``release`` function is +called by the kernel) KBUS traverses its outstanding message queue, and for +each Request that has not been answered, generates a Status message saying +that the Replier has "GoneAway". + +Similarly, if a Replier unbinds from replying to a mesage, KBUS traverses its +outstanding message queue, and for each Request that has not been answered, it +generates a Status message saying that it has "Unbound" from being a replier +for that message name. It also forgets the message, which it is now not going +to reply to. + +Lastly, when a Replier is closed, if it has read any Requests (technically, +called NEXTMSG to pop them from the message queue), but not actually replied +to them, then KBUS will send an "Ignored" Status message for each such +Request. + +More information +================ +Stateful transactions +--------------------- +It is possible to make stateful message transactions, by: + +1. sending a Request +2. receiving the Reply, and noting the Ksock id of the replier +3. sending another Request to that specific replier +4. and so on + +Sending a request to a particular Ksock will fail if that Ksock is no longer +bound as replier to the relevant message name. This allows a sender to +guarantee that it is communicating with a particular instance of the replier +for a message name. + +Queues filling up +----------------- +Messages are sent by a mechanism which: + +1. Checks the message is plausible (it has a plausible message name, + and the right sort of "shape") +2. If the message is a Request, checks that the sender has room on its message + queue for the (eventual) Reply. +3. Finds the Ksock ids of all the listeners and repliers bound to that + messages name +4. Adds the message to the queue for each such listener/replier + +This can cause problems if one of the queues is already full (allowing +infinite expansion of queues would also cause problems, of couse). + +If a *sender* attempts to send a Request, but does not have room on its +message queue for the (corresponding) Reply, then the message will not be +sent, and the send will fail. Note that the message id will not be set, and +the blocking behaviours defined below do not occur. + +If a *replier* cannot receive a particular message, because its queue is full, +then the message will not be sent, and the send will fail with an error. This +does, however, set the message id (and thus the "last message id" on the +sender). + +Moreover, a sender can indicate if it wants a message to be: + +1. Added to all the listener queues, regardless, in which case it will block + until that can be done (ALL_OR_WAIT, sender blocks) +2. Added to all the listener queues, and fail if that can't be done + (ALL_OR_FAIL) +3. Added to all the listener queues that have room (the default) + +See `Message flags`_ for more details. + +Urgent messages +--------------- +Messages may be flagged urgent. In this case they will be added to the front +of the destination message queue, rather than the end - in other words, they +will be the next message to be "popped" by NEXTMSG. + +Note that this means that if two urgent messages are sent to the same target, +and *then* a NEXTMSG/read occurs, the second urgent message will be popped and +read first. + +Select, write/send and "next message", blocking +----------------------------------------------- +.. warning:: At the moment, ``read`` and ``write`` are always non-blocking. + +``read`` returns more of the currently selected message, or EOF if there is no +more of that message to read (and thus also if there is no currently selected +message). The NEXTMSG ioctl is used to select ("pop") the next message. + +``write`` writes to the end of the currently-being-written message. The +DISCARD ioctl can be used to discard the data written so far, and the SEND +ioctl to send the (presumably completed message). Whilst the message is being +sent, it is not possible to use ``write``. + +Note that if SEND is used to send a Request, then KBUS ensures that there will +always be either a Reply or a Status message in response to that request. + +Specifically, if: + +1. The Replier "goes away" (and its "release" function is called) before + reading the Request (specifically, before calling NEXTMSG to pop it from + the message queue) +2. The Replier "goes away" (and its "release" function is called) before + replying to a Request that it has already read (i.e., used NEXTMSG to pop + from the message queue) +3. The Replier unbinds from that Request message name before reading the + Request (with the same caveat on what that means) +4. Select/poll attempts to send the Request, and discovers that the + Replier has disappeared since the initial SEND +5. Select/poll attempts to send the Request, and some other error occurs + +then KBUS will "reply" with an appropriate Status message. + +-------------------------------------------------- + +KBUS support its own particular variation on blocking of message sending. + +First of all, it supports use of "select" to determine if there are any +messages waiting to be read. So, for instance (in Python):: + + with Ksock(0,'rw') as sender: + with Ksock(0,'r') as listener: + (r,w,x) = select.select([listener],[],[],0) + assert r == [] + + listener.bind('$.Fred') + msg = Announcement('$.Fred','data') + sender.send_msg(msg) + + (r,w,x) = select.select([listener],[],[],0) + assert r == [listener] + +This simply checks if there is a message in the Ksock's message list, waiting +to be "popped" with NEXTMSG. + +Secondly, ``write``, SEND and DISCARD interact in what is hoped to be a +sensible manner. Specifically: + +* When SEND (i.e., the SEND ioctl) is called, KBUS can either: + + 1. Succeed in sending the message. The Ksock is now ready for ``write`` to + be called on it again. + 2. Failed in sending the message (possibly, if the message was a Request, + with EADDRNOTAVAIL, indicating that there is no Replier for that + Request). The Ksock is now ready for ``write`` to be called on it again. + 3. If the message was marked ALL_OR_WAIT, then it may fail with EAGAIN. + In this case, the Ksock is still in sending state, and an attempt to + call ``write`` will fail (with EALREADY). The caller can either use + DISCARD to discard the message, or use select/poll to wait for the + message to finish sending. + +Thus "select" for the write case checks whether it is allowed to call +"write" - for instance:: + + with Ksock(0,'rw') as sender: + write_list = [sender] + with Ksock(0,'r') as listener1: + write_list= [sender,listener1] + read_list = [listener1] + + (r,w,x) = select.select(read_list,write_list,[],0) + assert r == [] + assert w == [sender] + assert x == [] + + with Ksock(0,'rw') as listener2: + write_list.append(listener2) + read_list.append(listener2) + + (r,w,x) = select.select(read_list,write_list,[],0) + assert r == [] + assert len(w) == 2 + assert sender in w + assert listener2 in w + assert x == [] + +Receiving messages once only +---------------------------- +In normal usage (and by default), if a Ksock binds to a message name multiple +times, it will receive multiple copies of a message. This can happen: + +* explicitly (the Ksock deliberately and explicitly binds to the same name + more than once, seeking this effect). +* as a result of binding to a message name and a wildcard that includes the + same name, or two overlapping wildcards. +* as a result of binding as Replier to a name, and also as Listener to the + same name (possibly via a wildcard). In this case, multiple copies will + only be received when a Request with that name is made. + +Several programmers have complained that the last case, in particular, is very +inconvenient, and thus the "receive a message once only" facility has been +added. + +Using the MSGONCEONLY IOCTL, it is possible to tell a Ksock that only one copy +of a particular message should be received, even if multiple are "due". In the +case of the Replier/Listener copies, it will always be the message to which +the Replier should reply (the one with WANT_YOU_TO_REPLY set) that will be +received. + +Please use this facility with care, and only if you really need it. + +IOCTLS +------ +The KBUS ioctls are defined (with explanatory comments) in the kernel module +header file (``kbus_defns.h``). They are: + +:RESET: Currently has no effect +:BIND: Bind to a particular message name (possibly as replier). +:UNBIND: Unbind from a binding - must match exactly. +:KSOCKID: Determine the Ksock id of the Ksock used +:REPLIER: Determine who is bound as replier to a particular message + name. This returns 0 or the Ksock id of the replier. +:NEXTMSG: Pop the next message from the Ksock's message queue, ready + for reading (with ``read``), and return its length (in bytes). + If there is no next message, return a length of 0. + The length is always the length of an "entire" message (see + `Message implementation`_). +:LENLEFT: Determine how many bytes of the message currently being read + are still to read. +:SEND: Send the current outstanding message for this Ksock (i.e., the + bytes written to the Ksock since the last SEND or DISCARD). + Return the message id of the message, and maybe other status + information. +:DISCARD: Discard (throw away) the current outstanding message for this + Ksock (i.e., any bytes written to the Ksock since the last + SEND or DISCARD). +:LASTSENT: Determine the message id of the last message SENT on this + Ksock. +:MAXMSGS: Set the maximum length of the (read) message queue for this + KSOCK, and return the actual length that is set. An attempt + to set the queue length to 0 will just return the current + queue length. +:NUMMSGS: Determine how many messages are outstanding in this Ksock's + read queue. +:UNREPLIEDTO: Determines how many Requests (marked "WANT_YOU_TO_REPLY") + this Ksock still needs to reply to. This is primarily a + development tool. +:MSGONLYONCE: Determines whether only one copy of a message will be + received, even if the message name is bound to multiple times. + May also be used to query the current state. +:VERBOSE: Determines whether verbose kernel messages should be output or + not. Affects the *device* (the entire Ksock). + May also be used to query the current state. +:NEWDEVICE: Requests another KBUS device (``/dev/kbus/<n>``). The next + KBUS device number (up to a maximum of 255) will be allocated. + Returns the new device number. +:REPORTREPLIERBINDS: Request synthetic messages announcing Replier BIND/UNBIND + events. These are messages named "$.KBUS.ReplierBindEvent", + and are the only predefined messages with data. + Both Python and C bindings provide a useful function to + extract the ``is_bind``, ``binder`` and ``name`` values from + the data. + +/proc/kbus/bindings +------------------- +``/proc/kbus/bindings`` is a debugging aid for reporting the listener id, +exclusive flag and message name for each binding, for each kbus device. + +An example might be:: + + $ cat /proc/kbus/bindings + # <device> is bound to <Ksock-ID> in <process-PID> as <Replier|Listener> for <message-name> + 1: 1 22158 R $.Sensors.* + 1: 2 22158 R $.Sensors.Kitchen.Temperature + 1: 3 22158 L $.Sensors.* + 13: 4 22159 L $.Jim.* + 13: 1 22159 R $.Fred + 13: 1 22159 L $.Jim + 13: 14 23021 L $.Jim.* + +This describes two KBUS devices (``/dev/kbus1`` and ``/dev/kbus13``). + +The first has bindings on Ksock ids 1, 2 and 3, for the given message names. The +"R" indicates a replier binding, the "L" indicates a listener (non-replier) +binding. + +The second has bindings on Ksock ids 4, 1 and 14. The order of the bindings +reported is *not* particularly significant. + +Note that there is no communication between the two devices, so Ksock id 1 on +device 1 is not related to (and has no commonality with) Ksock id 1 on device +13. + +/proc/kbus/stats +---------------- +``/proc/kbus/stats`` is a debugging aid for reporting various statistics about +the KBUS devices and the Ksocks open on them. + +An example might be:: + + $ cat /proc/kbus/stats + dev 0: next file 5 next msg 8 unsent unbindings 0 + ksock 4 last msg 0:7 queue 1 of 100 + read byte 0 of 0, wrote byte 52 (max 60), sending + outstanding requests 0 (size 16, max 0), unsent replies 0 (max 0) + ksock 3 last msg 0:5 queue 0 of 1 + read byte 0 of 0, wrote byte 0 (max 0), not sending + outstanding requests 1 (size 16, max 0), unsent replies 0 (max 0) + +or:: + + $ cat /proc/kbus/stats + dev 0: next file 4 next msg 101 unsent unbindings 0 + ksock 3 last msg 0:0 queue 100 of 100 + read byte 0 of 0, wrote byte 0 (max 0), not sending + outstanding requests 0 (size 16, max 0), unsent replies 0 (max 0) + ksock 2 last msg 0:100 queue 0 of 100 + read byte 0 of 0, wrote byte 0 (max 0), not sending + outstanding requests 100 (size 102, max 92), unsent replies 0 (max 0) + + +Error numbers +------------- +The following error numbers get special use. In Python, they are all returned +as values inside the IOError exception. + + Since we're trying to fit into the normal Un*x convention that negative + values are error numbers, and since Un*x defines many of these for us, + it is natural to make use of the relevant definitions. However, this also + means that we are often using them in an unnatural sense. I've tried to + make the error numbers used bear at least a vague relationship to their + (mis)use in KBUS. + +:EADDRINUSE: On attempting to bind a message name as replier: There is + already a replier bound for this message +:EADDRNOTAVAIL: On attempting to send a Request message: There is no replier + bound for this message's name. + + On attempting to send a Reply message: The sender of the + original request (i.e., the Ksock mentioned as the ``to`` + in the Reply) is no longer connected. +:EALREADY: On attempting to write to a Ksock, when a previous send has + returned EAGAIN. Either DISCARD the message, or use + select/poll to wait for the send to complete, and write to be + allowed. +:EBADMSG: On attempting to bind, unbind or send a message: The message + name is not valid. On sending, this can also be because the + message name is a wildcard. +:EBUSY: On attempting to send, then: + + 1. For a request, the replier's message queue is full. + 2. For any message, with ALL_OR_FAIL set, one of the + targetted listener/replier queues was full. + +:ECONNREFUSED: On attempting to send a Reply, the intended recipient (the + notional original sender of the Request) is not expecting + a Reply with that message id in its 'in_reply_to'. Or, in + other words, this appears to be an attempt to reply to the + wrong message id or the wrong Ksock. +:EINVAL: Something went wrong (generic error). +:EMSGSIZE: On attempting to write a message: Data was written after + the end of the message (i.e., after the final end guard + of the message). +:ENAMETOOLONG: On attempting to bind, unbind or send a message: The message + name is too long. +:ENOENT: On attempting to open a Ksock: There is no such device + (normally because one has tried to open, for instance, + '/dev/kbus9' when there are only 3 KBUS devices). +:ENOLCK: On attempting to send a Request, when there is not enough room + in the sender's message queue to guarantee that it can + receive a reply for every Request already sent, *plus* this + one. If there are oustanding messages in the sender's message + queue, then the solution is to read some of them. Otherwise, + the sender will have to wait until one of the Repliers + replies to a previous Request (or goes away and KBUS replies + for it). + + When this error is received, the send has failed (just as if + the message was invalid). The sender is not left in "sending" + state, nor has the message been assigned a message id. + + Note that this is *not* EAGAIN, since we do not want to block + the sender (in the SEND) if it is up to the sender to perform + a read to sort things out. + +:ENOMSG: On attempting to send, when there is no message waiting to be + sent (either because there has been no write since the last + send, or because the message being written has been + discarded). +:EPIPE: On attempting to send 'to' a specific replier, the replier + with that id is no longer bound to the given message's name. + +:EFAULT: Memory allocation, copy from user space, or other such failed. This + is normally very bad, it should not happen, UNLESS it is the result + of calling an ioctl, when it indicates that the ioctl argument + cannot be accessed. + +:ENOMEM: Memory allocation failed (return NULL). This is normally very bad, + it should not happen. + +:EAGAIN: On attempting to send, the message being sent had ALL_OR_WAIT set, + and one of the targetted listener/replier queues was full. + + On attempting to unbind when Replier Bind Events have been + requested, one or more of the KSocks bound to receive + "$.KBUS.ReplierBindEvent" messages has a full message queue, + and thus cannot receive the unbind event. The unbind has not been + done. + +In the ``utils`` directory of the KBUS sources, there is a script called +``errno.py`` which takes an ``errno`` integer or name and prints out both the +"normal" meaning of that error number, and also (if there is one) the KBUS use +of it. For instance:: + + $ errno.py 1 + Error 1 (0x1) is EPERM: Operation not permitted + $ + $ errno.py EPIPE + EPIPE is error 32 (0x20): Broken pipe + + KBUS: + On attempting to send 'to' a specific replier, the replier with that id + is no longer bound to the given message's name. + -- 1.7.1 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 2/5] External header file for KBUS 2011-02-27 18:11 ` [PATCH 1/5] Documentation for KBUS Tony Ibbs @ 2011-02-27 18:11 ` Tony Ibbs 2011-02-27 18:11 ` [PATCH 3/5] Kconfig files and Makefile " Tony Ibbs 0 siblings, 1 reply; 15+ messages in thread From: Tony Ibbs @ 2011-02-27 18:11 UTC (permalink / raw) To: Linux-embedded; +Cc: Tibs at Kynesim, Richard Watts, Grant Likely From: Tibs <tibs@tonyibbs.co.uk> Signed-off-by: Tony Ibbs <tibs@tonyibbs.co.uk> --- include/linux/kbus_defns.h | 666 ++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 666 insertions(+), 0 deletions(-) create mode 100644 include/linux/kbus_defns.h diff --git a/include/linux/kbus_defns.h b/include/linux/kbus_defns.h new file mode 100644 index 0000000..82779a6 --- /dev/null +++ b/include/linux/kbus_defns.h @@ -0,0 +1,666 @@ +/* Kbus kernel module external headers + * + * This file provides the definitions (datastructures and ioctls) needed to + * communicate with the KBUS character device driver. + */ + +/* + * ***** BEGIN LICENSE BLOCK ***** + * Version: MPL 1.1 + * + * The contents of this file are subject to the Mozilla Public License Version + * 1.1 (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * http://www.mozilla.org/MPL/ + * + * Software distributed under the License is distributed on an "AS IS" basis, + * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License + * for the specific language governing rights and limitations under the + * License. + * + * The Original Code is the KBUS Lightweight Linux-kernel mediated + * message system + * + * The Initial Developer of the Original Code is Kynesim, Cambridge UK. + * Portions created by the Initial Developer are Copyright (C) 2009 + * the Initial Developer. All Rights Reserved. + * + * Contributor(s): + * Kynesim, Cambridge UK + * Tony Ibbs <tibs@tonyibbs.co.uk> + * + * Alternatively, the contents of this file may be used under the terms of the + * GNU Public License version 2 (the "GPL"), in which case the provisions of + * the GPL are applicable instead of the above. If you wish to allow the use + * of your version of this file only under the terms of the GPL and not to + * allow others to use your version of this file under the MPL, indicate your + * decision by deleting the provisions above and replace them with the notice + * and other provisions required by the GPL. If you do not delete the + * provisions above, a recipient may use your version of this file under either + * the MPL or the GPL. + * + * ***** END LICENSE BLOCK ***** + */ + +#ifndef _kbus_defns +#define _kbus_defns + +#if !__KERNEL__ && defined(__cplusplus) +extern "C" { +#endif + +#if __KERNEL__ +#include <linux/kernel.h> +#include <linux/ioctl.h> +#else +#include <stdint.h> +#include <sys/ioctl.h> +#endif + +/* + * A message id is made up of two fields. + * + * If the network id is 0, then it is up to us (KBUS) to assign the + * serial number. In other words, this is a local message. + * + * If the network id is non-zero, then this message is presumed to + * have originated from another "network", and we preserve both the + * network id and the serial number. + * + * The message id {0,0} is special and reserved (for use by KBUS). + */ +struct kbus_msg_id { + __u32 network_id; + __u32 serial_num; +}; + +/* + * kbus_orig_from is used for the "originally from" and "finally to" ids + * in the message header. These in turn are used when messages are + * being sent between KBUS systems (via KBUS "Limpets"). KBUS the kernel + * module transmits them, unaltered, but does not use them (although + * debug messages may report them). + * + * An "originally from" or "finally to" id is made up of two fields, the + * network id (which indicates the Limpet, if any, that originally gated the + * message), and a local id, which is the Ksock id of the original sender + * of the message, on its local KBUS. + * + * If the network id is 0, then the "originally from" id is not being used. + * + * Limpets and these fields are discussed in more detail in the userspace + * KBUS documentation - see http://kbus-messaging.org/ for pointers to + * more information. + */ +struct kbus_orig_from { + __u32 network_id; + __u32 local_id; +}; + +/* When the user asks to bind a message name to an interface, they use: */ +struct kbus_bind_request { + __u32 is_replier; /* are we a replier? */ + __u32 name_len; + char *name; +}; + +/* When the user requests the id of the replier to a message, they use: */ +struct kbus_bind_query { + __u32 return_id; + __u32 name_len; + char *name; +}; + +/* When the user writes/reads a message, they use: */ +struct kbus_message_header { + /* + * The guards + * ---------- + * + * * 'start_guard' is notionally "Kbus", and 'end_guard' (the 32 bit + * word after the rest of the message datastructure) is notionally + * "subK". Obviously that depends on how one looks at the 32-bit + * word. Every message datastructure shall start with a start guard + * and end with an end guard. + * + * These provide some help in checking that a message is well formed, + * and in particular the end guard helps to check for broken length + * fields. + * + * - 'id' identifies this particular message. + * + * When a user writes a new message, they should set this to {0,0}. + * KBUS will then set a new message id for the message. + * + * When a user reads a message, this will have been set by KBUS. + * + * When a user replies to a message, they should copy this value + * into the 'in_reply_to' field, so that the recipient will know + * what message this was a reply to. + * + * - 'in_reply_to' identifies the message this is a reply to. + * + * This shall be set to {0,0} unless this message *is* a reply to a + * previous message. In other words, if this value is non-0, then + * the message *is* a reply. + * + * - 'to' is who the message is to be sent to. + * + * When a user writes a new message, this should normally be set + * to {0,0}, meaning "anyone listening" (but see below if "state" + * is being maintained). + * + * When replying to a message, it shall be set to the 'from' value + * of the orginal message. + * + * When constructing a request message (a message wanting a reply), + * the user can set it to a specific replier id, to produce a stateful + * request. This is normally done by copying the 'from' of a previous + * Reply from the appropriate replier. When such a message is sent, + * if the replier bound (at that time) does not have that specific + * id, then the send will fail. + * + * Note that if 'to' is set, then 'orig_from' should also be set. + * + * - 'from' indicates who sent the message. + * + * When a user is writing a new message, they should set this + * to {0,0}. + * + * When a user is reading a message, this will have been set + * by KBUS. + * + * When a user replies to a message, the reply should have its + * 'to' set to the original messages 'from', and its 'from' set + * to {0,0} (see the "hmm" caveat under 'to' above, though). + * + * - 'orig_from' and 'final_to' are used when Limpets are mediating + * KBUS messages between KBUS devices (possibly on different + * machines). See the description by the datastructure definition + * above. The KBUS kernel preserves and propagates their values, + * but does not alter or use them. + * + * - 'extra' is currently unused, and KBUS will set it to zero. + * Future versions of KBUS may treat it differently. + * + * - 'flags' indicates the type of message. + * + * When a user writes a message, this can be used to indicate + * that: + * + * * the message is URGENT + * * a reply is wanted + * + * When a user reads a message, this indicates if: + * + * * the message is URGENT + * * a reply is wanted + * + * When a user writes a reply, this field should be set to 0. + * + * The top half of the 'flags' is not touched by KBUS, and may + * be used for any purpose the user wishes. + * + * - 'name_len' is the length of the message name in bytes. + * + * This must be non-zero. + * + * - 'data_len' is the length of the message data in bytes. It may be + * zero if there is no data. + * + * - 'name' is a pointer to the message name. This should be null + * terminated, as is the normal case for C strings. + * + * NB: If this is zero, then the name will be present, but after + * the end of this datastructure, and padded out to a multiple of + * four bytes (see kbus_entire_message). When doing this padding, + * remember to allow for the terminating null byte. If this field is + * zero, then 'data' shall also be zero. + * + * - 'data' is a pointer to the data. If there is no data (if + * 'data_len' is zero), then this shall also be zero. The data is + * not touched by KBUS, and may include any values. + * + * NB: If this is zero, then the data will occur immediately + * after the message name, padded out to a multiple of four bytes. + * See the note for 'name' above. + * + */ + __u32 start_guard; + struct kbus_msg_id id; /* Unique to this message */ + struct kbus_msg_id in_reply_to; /* Which message this is a reply to */ + __u32 to; /* 0 (empty) or a replier id */ + __u32 from; /* 0 (KBUS) or the sender's id */ + struct kbus_orig_from orig_from;/* Cross-network linkage */ + struct kbus_orig_from final_to; /* Cross-network linkage */ + __u32 extra; /* ignored field - future proofing */ + __u32 flags; /* Message type/flags */ + __u32 name_len; /* Message name's length, in bytes */ + __u32 data_len; /* Message length, also in bytes */ + char *name; + void *data; + __u32 end_guard; +}; + +#define KBUS_MSG_START_GUARD 0x7375624B +#define KBUS_MSG_END_GUARD 0x4B627573 + +/* + * When a message is returned by 'read', it is actually returned using the + * following datastructure, in which: + * + * - 'header.name' will point to 'rest[0]' + * - 'header.data' will point to 'rest[(header.name_len+3)/4]' + * + * followed by the name (padded to 4 bytes, remembering to allow for the + * terminating null byte), followed by the data (padded to 4 bytes) followed by + * (another) end_guard. + */ +struct kbus_entire_message { + struct kbus_message_header header; + __u32 rest[]; +}; + +/* + * We limit a message name to at most 1000 characters (some limit seems + * sensible, after all) + */ +#define KBUS_MAX_NAME_LEN 1000 + +/* + * The length (in bytes) of the name after padding, allowing for a terminating + * null byte. + */ +#define KBUS_PADDED_NAME_LEN(name_len) (4 * ((name_len + 1 + 3) / 4)) + +/* + * The length (in bytes) of the data after padding + */ +#define KBUS_PADDED_DATA_LEN(data_len) (4 * ((data_len + 3) / 4)) + +/* + * Given name_len (in bytes) and data_len (in bytes), return the + * length of the appropriate kbus_entire_message_struct, in bytes + * + * Note that we're allowing for a zero byte after the end of the message name. + * + * Remember that "sizeof" doesn't count the 'rest' field in our message + * structure. + */ +#define KBUS_ENTIRE_MSG_LEN(name_len, data_len) \ + (sizeof(struct kbus_entire_message) + \ + KBUS_PADDED_NAME_LEN(name_len) + \ + KBUS_PADDED_DATA_LEN(data_len) + 4) + +/* + * The message name starts at entire->rest[0]. + * The message data starts after the message name - given the message + * name's length (in bytes), that is at index: + */ +#define KBUS_ENTIRE_MSG_DATA_INDEX(name_len) ((name_len+1+3)/4) +/* + * Given the message name length (in bytes) and the message data length (also + * in bytes), the index of the entire message end guard is thus: + */ +#define KBUS_ENTIRE_MSG_END_GUARD_INDEX(name_len, data_len) \ + ((name_len+1+3)/4 + (data_len+3)/4) + +/* + * Find a pointer to the message's name. + * + * It's either the given name pointer, or just after the header (if the pointer + * is NULL) + */ +static inline char *kbus_msg_name_ptr(const struct kbus_message_header + *hdr) +{ + if (hdr->name) { + return hdr->name; + } else { + struct kbus_entire_message *entire; + entire = (struct kbus_entire_message *)hdr; + return (char *)&entire->rest[0]; + } +} + +/* + * Find a pointer to the message's data. + * + * It's either the given data pointer, or just after the name (if the pointer + * is NULL) + */ +static inline void *kbus_msg_data_ptr(const struct kbus_message_header + *hdr) +{ + if (hdr->data) { + return hdr->data; + } else { + struct kbus_entire_message *entire; + __u32 data_idx; + + entire = (struct kbus_entire_message *)hdr; + data_idx = KBUS_ENTIRE_MSG_DATA_INDEX(hdr->name_len); + return (void *)&entire->rest[data_idx]; + } +} + +/* + * Find a pointer to the message's (second/final) end guard. + */ +static inline __u32 *kbus_msg_end_ptr(struct kbus_entire_message + *entire) +{ + __u32 end_guard_idx = + KBUS_ENTIRE_MSG_END_GUARD_INDEX(entire->header.name_len, + entire->header.data_len); + return (__u32 *) &entire->rest[end_guard_idx]; +} + +/* + * Things KBUS changes in a message + * -------------------------------- + * In general, KBUS leaves the content of a message alone. However, it does + * change: + * + * - the message id (if id.network_id is unset - it assigns a new serial + * number unique to this message) + * - the from id (if from.network_id is unset - it sets the local_id to + * indicate the Ksock this message was sent from) + * - the KBUS_BIT_WANT_YOU_TO_REPLY bit in the flags (set or cleared + * as appropriate) + * - the SYNTHETIC bit, which KBUS will always unset in a user message + */ + +/* + * Flags for the message 'flags' word + * ---------------------------------- + * The KBUS_BIT_WANT_A_REPLY bit is set by the sender to indicate that a + * reply is wanted. This makes the message into a request. + * + * Note that setting the WANT_A_REPLY bit (i.e., a request) and + * setting 'in_reply_to' (i.e., a reply) is bound to lead to + * confusion, and the results are undefined (i.e., don't do it). + * + * The KBUS_BIT_WANT_YOU_TO_REPLY bit is set by KBUS on a particular message + * to indicate that the particular recipient is responsible for replying + * to (this instance of the) message. Otherwise, KBUS clears it. + * + * The KBUS_BIT_SYNTHETIC bit is set by KBUS when it generates a synthetic + * message (an exception, if you will), for instance when a replier has + * gone away and therefore a reply will never be generated for a request + * that has already been queued. + * + * Note that KBUS does not check that a sender has not set this + * on a message, but doing so may lead to confusion. + * + * The KBUS_BIT_URGENT bit is set by the sender if this message is to be + * treated as urgent - i.e., it should be added to the *front* of the + * recipient's message queue, not the back. + * + * Send flags + * ========== + * There are two "send" flags, KBUS_BIT_ALL_OR_WAIT and KBUS_BIT_ALL_OR_FAIL. + * Either one may be set, or both may be unset. + * + * If both bits are set, the message will be rejected as invalid. + * + * Both flags are ignored in reply messages (i.e., messages with the + * 'in_reply_to' field set). + * + * If both are unset, then a send will behave in the default manner. That is, + * the message will be added to a listener's queue if there is room but + * otherwise the listener will (silently) not receive the message. + * + * (Obviously, if the listener is a replier, and the message is a request, + * then a KBUS message will be synthesised in the normal manner when a + * request is lost.) + * + * If the KBUS_BIT_ALL_OR_WAIT bit is set, then a send should block until + * all recipients can be sent the message. Specifically, before the message is + * sent, all recipients must have room on their message queues for this + * message, and if they do not, the send will block until there is room for the + * message on all the queues. + * + * If the KBUS_BIT_ALL_OR_FAIL bit is set, then a send should fail if all + * recipients cannot be sent the message. Specifically, before the message is + * sent, all recipients must have room on their message queues for this + * message, and if they do not, the send will fail. + */ + +/* + * When a $.KBUS.ReplierBindEvent message is constructed, we use the + * following to encapsulate its data. + * + * This indicates whether it is a bind or unbind event, who is doing the + * bind or unbind, and for what message name. The message name is padded + * out to a multiple of four bytes, allowing for a terminating null byte, + * but the name length is the length without said padding (so, in C terms, + * strlen(name)). + * + * As for the message header data structure, the actual data "goes off the end" + * of the datastructure. + */ +struct kbus_replier_bind_event_data { + __u32 is_bind; /* 1=bind, 0=unbind */ + __u32 binder; /* Ksock id of binder */ + __u32 name_len; /* Length of name */ + __u32 rest[]; /* Message name */ +}; + +#if !__KERNEL__ +#define BIT(num) (((unsigned)1) << (num)) +#endif + +#define KBUS_BIT_WANT_A_REPLY BIT(0) +#define KBUS_BIT_WANT_YOU_TO_REPLY BIT(1) +#define KBUS_BIT_SYNTHETIC BIT(2) +#define KBUS_BIT_URGENT BIT(3) + +#define KBUS_BIT_ALL_OR_WAIT BIT(8) +#define KBUS_BIT_ALL_OR_FAIL BIT(9) + +/* + * Standard message names + * ====================== + * KBUS itself has some predefined message names. + * + * Synthetic Replies with no data + * ------------------------------ + * These are sent to the original Sender of a Request when KBUS knows that the + * Replier is not going to Reply. In all cases, you can identify which message + * they concern by looking at the "in_reply_to" field: + * + * * Replier.GoneAway - the Replier has gone away before reading the Request. + * * Replier.Ignored - the Replier has gone away after reading a Request, but + * before replying to it. + * * Replier.Unbound - the Replier has unbound (as Replier) from the message + * name, and is thus not going to reply to this Request in its unread message + * queue. + * * Replier.Disappeared - the Replier has disappeared when an attempt is made + * to send a Request whilst polling (i.e., after EAGAIN was returned from an + * earlier attempt to send a message). This typically means that the Ksock + * bound as Replier closed. + * * ErrorSending - an unexpected error occurred when trying to send a Request + * to its Replier whilst polling. + * + * Synthetic Announcements with no data + * ------------------------------------ + * * UnbindEventsLost - sent (instead of a Replier Bind Event) when the unbind + * events "set aside" list has filled up, and thus unbind events have been + * lost. + */ +#define KBUS_MSG_NAME_REPLIER_GONEAWAY "$.KBUS.Replier.GoneAway" +#define KBUS_MSG_NAME_REPLIER_IGNORED "$.KBUS.Replier.Ignored" +#define KBUS_MSG_NAME_REPLIER_UNBOUND "$.KBUS.Replier.Unbound" +#define KBUS_MSG_NAME_REPLIER_DISAPPEARED "$.KBUS.Replier.Disappeared" +#define KBUS_MSG_NAME_ERROR_SENDING "$.KBUS.ErrorSending" +#define KBUS_MSG_NAME_UNBIND_EVENTS_LOST "$.KBUS.UnbindEventsLost" + +/* + * Replier Bind Event + * ------------------ + * This is the only message name for which KBUS generates data -- see + * kbus_replier_bind_event_data. It is also the only message name which KBUS + * does not allow binding to as a Replier. + * + * This is the message that is sent when a Replier binds or unbinds to another + * message name, if the KBUS_IOC_REPORTREPLIERBINDS ioctl has been used to + * request such notification. + */ +#define KBUS_MSG_NAME_REPLIER_BIND_EVENT "$.KBUS.ReplierBindEvent" + +#define KBUS_IOC_MAGIC 'k' /* 0x6b - which seems fair enough for now */ +/* + * RESET: reserved for future use + */ +#define KBUS_IOC_RESET _IO(KBUS_IOC_MAGIC, 1) +/* + * BIND - bind a Ksock to a message name + * arg: struct kbus_bind_request, indicating what to bind to + * retval: 0 for success, negative for failure + */ +#define KBUS_IOC_BIND _IOW(KBUS_IOC_MAGIC, 2, char *) +/* + * UNBIND - unbind a Ksock from a message id + * arg: struct kbus_bind_request, indicating what to unbind from + * retval: 0 for success, negative for failure + */ +#define KBUS_IOC_UNBIND _IOW(KBUS_IOC_MAGIC, 3, char *) +/* + * KSOCKID - determine a Ksock's Ksock id + * + * The network_id for the current Ksock is, by definition, 0, so we don't need + * to return it. + * + * arg (out): __u32, indicating this Ksock's local_id + * retval: 0 for success, negative for failure + */ +#define KBUS_IOC_KSOCKID _IOR(KBUS_IOC_MAGIC, 4, char *) +/* + * REPLIER - determine the Ksock id of the replier for a message name + * arg: struct kbus_bind_query + * + * - on input, specify the message name to ask about. + * - on output, KBUS fills in the relevant Ksock id in the return_value, + * or 0 if there is no bound replier + * + * retval: 0 for success, negative for failure + */ +#define KBUS_IOC_REPLIER _IOWR(KBUS_IOC_MAGIC, 5, char *) +/* + * NEXTMSG - pop the next message from the read queue + * arg (out): __u32, number of bytes in the next message, 0 if there is no + * next message + * retval: 0 for success, negative for failure + */ +#define KBUS_IOC_NEXTMSG _IOR(KBUS_IOC_MAGIC, 6, char *) +/* + * LENLEFT - determine how many bytes are left to read of the current message + * arg (out): __u32, number of bytes left, 0 if there is no current read + * message + * retval: 1 if there was a message, 0 if there wasn't, negative for failure + */ +#define KBUS_IOC_LENLEFT _IOR(KBUS_IOC_MAGIC, 7, char *) +/* + * SEND - send the current message + * arg (out): struct kbus_msg_id, the message id of the sent message + * retval: 0 for success, negative for failure + */ +#define KBUS_IOC_SEND _IOR(KBUS_IOC_MAGIC, 8, char *) +/* + * DISCARD - discard the message currently being written (if any) + * arg: none + * retval: 0 for success, negative for failure + */ +#define KBUS_IOC_DISCARD _IO(KBUS_IOC_MAGIC, 9) +/* + * LASTSENT - determine the message id of the last message SENT + * arg (out): struct kbus_msg_id, {0,0} if there was no last message + * retval: 0 for success, negative for failure + */ +#define KBUS_IOC_LASTSENT _IOR(KBUS_IOC_MAGIC, 10, char *) +/* + * MAXMSGS - set the maximum number of messages on a Ksock read queue + * arg (in): __u32, the requested length of the read queue, or 0 to just + * request how many there are + * arg (out): __u32, the length of the read queue after this call has + * succeeded + * retval: 0 for success, negative for failure + */ +#define KBUS_IOC_MAXMSGS _IOWR(KBUS_IOC_MAGIC, 11, char *) +/* + * NUMMSGS - determine how many messages are in the read queue for this Ksock + * arg (out): __u32, the number of messages in the read queue. + * retval: 0 for success, negative for failure + */ +#define KBUS_IOC_NUMMSGS _IOR(KBUS_IOC_MAGIC, 12, char *) +/* + * UNREPLIEDTO - determine the number of requests (marked "WANT_YOU_TO_REPLY") + * which we still need to reply to. + * arg(out): __u32, said number + * retval: 0 for success, negative for failure + */ +#define KBUS_IOC_UNREPLIEDTO _IOR(KBUS_IOC_MAGIC, 13, char *) +/* + * MSGONLYONCE - should we receive a message only once? + * + * This IOCTL tells a Ksock whether it should only receive a particular message + * once, even if it is both a Replier and Listener for the message (in which + * case it will always get the message as Replier, if appropriate), or if it is + * registered as multiple Listeners for the message. + * + * arg(in): __u32, 1 to change to "only once", 0 to change to the default, + * 0xFFFFFFFF to just return the current/previous state. + * arg(out): __u32, the previous state. + * retval: 0 for success, negative for failure (-EINVAL if arg in was not one + * of the specified values) + */ +#define KBUS_IOC_MSGONLYONCE _IOWR(KBUS_IOC_MAGIC, 14, char *) +/* + * VERBOSE - should KBUS output verbose "printk" messages (for this device)? + * + * This IOCTL tells a Ksock whether it should output debugging messages. It is + * only effective if the kernel module has been built with the VERBOSE_DEBUGGING + * flag set. + * + * arg(in): __u32, 1 to change to "verbose", 0 to change to "quiet", + * 0xFFFFFFFF to just return the current/previous state. + * arg(out): __u32, the previous state. + * retval: 0 for success, negative for failure (-EINVAL if arg in was not one + * of the specified values) + */ +#define KBUS_IOC_VERBOSE _IOWR(KBUS_IOC_MAGIC, 15, char *) + +/* + * NEWDEVICE - request another KBUS device (/dev/kbus<n>). + * + * The next device number (up to a maximum of 255) will be allocated. + * + * arg(out): __u32, the new device number (<n>) + * retval: 0 for success, negative for failure + */ +#define KBUS_IOC_NEWDEVICE _IOR(KBUS_IOC_MAGIC, 16, char *) + +/* + * REPORTREPLIERBINDS - request synthetic messages announcing Replier + * bind/unbind events. + * + * If this flag is set, then when someone binds or unbinds to a message name as + * a Replier, KBUS will send out a synthetic Announcement of this fact. + * + * arg(in): __u32, 1 to change to "report", 0 to change to "do not report", + * 0xFFFFFFFF to just return the current/previous state. + * arg(out): __u32, the previous state. + * retval: 0 for success, negative for failure (-EINVAL if arg in was not one + * of the specified values) + */ +#define KBUS_IOC_REPORTREPLIERBINDS _IOWR(KBUS_IOC_MAGIC, 17, char *) + +/* If adding another IOCTL, remember to increment the next number! */ +#define KBUS_IOC_MAXNR 17 + +#if !__KERNEL__ && defined(__cplusplus) +} +#endif + +#endif /* _kbus_defns */ -- 1.7.1 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 3/5] Kconfig files and Makefile for KBUS 2011-02-27 18:11 ` [PATCH 2/5] External header file " Tony Ibbs @ 2011-02-27 18:11 ` Tony Ibbs 2011-02-27 18:11 ` [PATCH 4/5] Internal header file " Tony Ibbs 2011-02-27 18:19 ` [PATCH 3/5] Kconfig files and Makefile " Randy Dunlap 0 siblings, 2 replies; 15+ messages in thread From: Tony Ibbs @ 2011-02-27 18:11 UTC (permalink / raw) To: Linux-embedded; +Cc: Tibs at Kynesim, Richard Watts, Grant Likely From: Tibs <tibs@tonyibbs.co.uk> Amend the ipc Makefile. Add an ipc Kconfig for KBUS. Amend the init Kconfig appropriately. Signed-off-by: Tony Ibbs <tibs@tonyibbs.co.uk> --- init/Kconfig | 3 ++ ipc/Kconfig | 99 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ipc/Makefile | 5 +++ 3 files changed, 107 insertions(+), 0 deletions(-) create mode 100644 ipc/Kconfig diff --git a/init/Kconfig b/init/Kconfig index c972899..a31a6fe 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -339,6 +339,8 @@ config AUDIT_TREE depends on AUDITSYSCALL select FSNOTIFY +source "ipc/Kconfig" + source "kernel/irq/Kconfig" menu "RCU Subsystem" @@ -1305,3 +1307,4 @@ config PADATA bool source "kernel/Kconfig.locks" + diff --git a/ipc/Kconfig b/ipc/Kconfig new file mode 100644 index 0000000..46ebcda --- /dev/null +++ b/ipc/Kconfig @@ -0,0 +1,99 @@ +config KBUS + tristate "KBUS messaging system" + default m + ---help--- + KBUS is a lightweight messaging system, particularly aimed + at embedded platforms. This option provides the kernel support + for mediating messages between client processes. + + If you want KBUS support, you should say Y here. + + This support can also be built as a module. If so, the module + will be called kbus. + +if KBUS + +config KBUS_DEBUG + bool "Make KBUS debugging messages available" + default y + ---help--- + This is the master switch for KBUS debug kernel messages - + if turned off, all debug messages will be compiled out. + + In order for debugging messages to be emitted, they must + be enabled per-device with the KBUS_IOC_VERBOSE ioctl, + or by default by CONFIG_KBUS_DEBUG_DEFAULT_VERBOSE. + + If unsure, say Y. + +config KBUS_DEBUG_DEFAULT_VERBOSE + bool "Output KBUS debug messages by default" + depends on KBUS_DEBUG + default n + ---help--- + This switch is the default setting for whether KBUS devices will + emit debugging messages. Note that debug messages may be turned + on and off per-device at runtime with the KBUS_IOC_VERBOSE ioctl. + + If unsure, say N. + +config KBUS_DEF_NUM_DEVICES + int "Number of KBUS devices to auto-create" + default 1 + range 1 KBUS_MAX_NUM_DEVICES + ---help--- + This specifies the number of KBUS devices that will be automatically + created when the kbus subsystem initialises (when the module is + is loaded or the kernel booted, as appropriate). + + If kbus is built as a module, this number may also be given as a + parameter, for example kbus_num_devices=5. + +config KBUS_MAX_NUM_DEVICES + int "Maximum number of KBUS devices" + default 256 + range 1 2147483647 + # We don't impose a limit on the max, so if you've got enough + # RAM the only practical limit will be the (int) minor count + # passed to __register_chrdev_region. + ---help--- + Specify the maximum number of KBUS devices to support. + Note that this setting controls the size of an array of pointers + to in-kernel kbus structs; reducing it only saves a tiny amount + of RAM. + +config KBUS_DEF_MAX_MESSAGES + int "Default KBUS message queue size limit" + default 100 + range 1 2147483647 + ---help--- + Specify the default recipient message queue size limit. + This default is applied to all recipients (clients) whenever they + open or reopen a KBUS device node. + + Clients sending messages may specify their desired behaviour in + the event that any of the recipient(s)' message queues is full; + if a sender's own queue is full, it may not send a message + flagged as a Request. Refer to the documentation ("Queues filling + up") for details. + + Clients may change their own queue size limits at any time with the + KBUS_IOC_MAXMSGS ioctl. + +config KBUS_MAX_UNSENT_UNBIND_MESSAGES + int "Maximum number of unsent KBUS unbind event messages" + default 1000 + range 1 2147483647 + ---help--- + KBUS devices may request (by ioctl) that they want to receive + notifications when listeners unbind. If such a notification happens + at a time when the device's message queue is full, it is instead + saved internally and delivered later. This setting limits the number + of messages which may be queued internally in that way; if the + limit is reached, a special "there were more unbind events than + we were able to deliver" message is queued for that listener, + and no more internal events are sent to them until that message + has been delivered. + +endif # KBUS + diff --git a/ipc/Makefile b/ipc/Makefile index 9075e17..3b330eb 100644 --- a/ipc/Makefile +++ b/ipc/Makefile @@ -2,6 +2,10 @@ # Makefile for the linux ipc. # +ifeq ($(CONFIG_KBUS_DEBUG),y) + CFLAGS_kbus.o = -DDEBUG +endif + obj-$(CONFIG_SYSVIPC_COMPAT) += compat.o obj-$(CONFIG_SYSVIPC) += util.o msgutil.o msg.o sem.o shm.o ipcns_notifier.o syscall.o obj-$(CONFIG_SYSVIPC_SYSCTL) += ipc_sysctl.o @@ -9,4 +13,5 @@ obj_mq-$(CONFIG_COMPAT) += compat_mq.o obj-$(CONFIG_POSIX_MQUEUE) += mqueue.o msgutil.o $(obj_mq-y) obj-$(CONFIG_IPC_NS) += namespace.o obj-$(CONFIG_POSIX_MQUEUE_SYSCTL) += mq_sysctl.o +obj-$(CONFIG_KBUS) += kbus.o -- 1.7.1 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 4/5] Internal header file for KBUS 2011-02-27 18:11 ` [PATCH 3/5] Kconfig files and Makefile " Tony Ibbs @ 2011-02-27 18:11 ` Tony Ibbs 2011-02-27 18:19 ` [PATCH 3/5] Kconfig files and Makefile " Randy Dunlap 1 sibling, 0 replies; 15+ messages in thread From: Tony Ibbs @ 2011-02-27 18:11 UTC (permalink / raw) To: Linux-embedded; +Cc: Tibs at Kynesim, Richard Watts, Grant Likely From: Tibs <tibs@tonyibbs.co.uk> Signed-off-by: Tony Ibbs <tibs@tonyibbs.co.uk> --- ipc/kbus_internal.h | 709 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 709 insertions(+), 0 deletions(-) create mode 100644 ipc/kbus_internal.h diff --git a/ipc/kbus_internal.h b/ipc/kbus_internal.h new file mode 100644 index 0000000..5b711cd --- /dev/null +++ b/ipc/kbus_internal.h @@ -0,0 +1,709 @@ +/* KBUS kernel module - internal definitions + * + * This is a character device driver, providing the messaging support + * for KBUS. + * + * This header contains the definitions used internally by kbus.c. + * At the moment nothing else is expected to include this file. + * + * KBUS clients should include (at least) kbus_defns.h. + */ + +/* + * ***** BEGIN LICENSE BLOCK ***** + * Version: MPL 1.1 + * + * The contents of this file are subject to the Mozilla Public License Version + * 1.1 (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * http://www.mozilla.org/MPL/ + * + * Software distributed under the License is distributed on an "AS IS" basis, + * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License + * for the specific language governing rights and limitations under the + * License. + * + * The Original Code is the KBUS Lightweight Linux-kernel mediated + * message system + * + * The Initial Developer of the Original Code is Kynesim, Cambridge UK. + * Portions created by the Initial Developer are Copyright (C) 2009 + * the Initial Developer. All Rights Reserved. + * + * Contributor(s): + * Kynesim, Cambridge UK + * Tony Ibbs <tibs@tonyibbs.co.uk> + * + * Alternatively, the contents of this file may be used under the terms of the + * GNU Public License version 2 (the "GPL"), in which case the provisions of + * the GPL are applicable instead of the above. If you wish to allow the use + * of your version of this file only under the terms of the GPL and not to + * allow others to use your version of this file under the MPL, indicate your + * decision by deleting the provisions above and replace them with the notice + * and other provisions required by the GPL. If you do not delete the + * provisions above, a recipient may use your version of this file under either + * the MPL or the GPL. + * + * ***** END LICENSE BLOCK ***** + */ + +#ifndef _kbus_internal +#define _kbus_internal + +/* + * KBUS can support multiple devices, as /dev/kbus<N>. These all have + * the same major device number, and map to differing minor device + * numbers. <N> will also be the minor device number, but don't rely + * on that for anything. + * + * When KBUS starts up, it will always setup a single device (/dev/kbus0), + * but it can be asked to setup more - for instance: + * + * # insmod kbus.ko kbus_num_devices=5 + * + * There is also an IOCTL to allow user-space to request a new device as + * necessary. The hot plugging mechanisms should cause the device to appear "as + * if by magic". + * + * (This last means that we *could* default to setting up zero devices + * at module startup, and leave the user to ask for the first one, but + * that seems rather cruel.) + * + * We need to set a maximum number of KBUS devices (corresponding to a limit on + * minor device numbers). The obvious limit (corresponding to what we'd have + * got if we used the deprecated "register_chrdev" to setup our device) is 256, + * so we'll go with that. + */ +#define KBUS_MIN_NUM_DEVICES 1 + +#ifdef CONFIG_KBUS_MAX_NUM_DEVICES +#define KBUS_MAX_NUM_DEVICES CONFIG_KBUS_MAX_NUM_DEVICES +#else +#define KBUS_MAX_NUM_DEVICES 256 +#endif + +#ifndef CONFIG_KBUS_DEF_NUM_DEVICES +#define CONFIG_KBUS_DEF_NUM_DEVICES 1 +#endif + +/* + * Our initial array sizes could arguably be made configurable + * for tuning, if we discover this is useful + */ +#define KBUS_INIT_MSG_ID_MEMSIZE 16 +#define KBUS_INIT_LISTENER_ARRAY_SIZE 8 + +/* + * Setting CONFIG_KBUS_DEBUG will cause the Makefile + * to define DEBUG for us + */ +#ifdef DEBUG +#define kbus_maybe_dbg(kbus_dev, format, args...) do { \ + if ((kbus_dev)->verbose) \ + (void) dev_dbg((kbus_dev)->dev, format, ## args); \ +} while (0) +#else +#define kbus_maybe_dbg(kbus_dev, format, args...) ((void)0) +#endif + +/* + * This is really only directly useful if CONFIG_KBUS_DEBUG is on + */ +#ifdef CONFIG_KBUS_DEBUG_DEFAULT_VERBOSE +#define KBUS_DEFAULT_VERBOSE_SETTING true +#else +#define KBUS_DEFAULT_VERBOSE_SETTING false +#endif + +/* ========================================================================= */ + +/* We need a way of remembering message bindings */ +struct kbus_message_binding { + struct list_head list; + struct kbus_private_data *bound_to; /* who we're bound to */ + u32 bound_to_id; /* but the id is often useful */ + u32 is_replier; /* bound as a replier */ + u32 name_len; + char *name; /* the message name */ +}; + +/* + * For both keeping track of requests sent (to which we still want replies) + * and replies read (to which we haven't yet sent a reply), we need some + * means of remembering message ids. Since I'd rather not worry the rest of + * the code with how this is implemented (which is code for "I'll implement + * it very simply and worry about making it efficient/scalable later"), and + * since we always want to remember both the message ids and also how many + * there are, it seems sensible to bundle this up in its own datastructure. + */ +struct kbus_msg_id_mem { + u32 count; /* Number of entries in use */ + u32 size; /* Actual size of the array */ + u32 max_count; /* Max 'count' we've had */ + /* + * An array is probably the worst way to store a list of message ids, + * but it's *very simple*, and should work OK for a smallish number of + * message ids. So it's a place to start... + * + * Note the array may have "unused" slots, signified by message id {0:0} + */ + struct kbus_msg_id *ids; +}; + +/* An item in the list of requests that a Ksock has not yet replied to */ +struct kbus_unreplied_item { + struct list_head list; + struct kbus_msg_id id; /* the request's id */ + u32 from; /* the sender's id */ + struct kbus_name_ptr *name_ref; /* and its name... */ + u32 name_len; +}; + +/* + * The parts of a message being written to KBUS (via kbus_write[_parts]), + * or read by the user (via kbus_read) are: + * + * * the user-space message header - as in 'struct kbus_message_header' + * + * from which we copy various items into our own internal message header. + * + * For a "pointy" message, that is all there is. + * + * For an "entire" message, this is then followed by: + * + * * the message name + * * padding to bring that up to a NULL terminator and then a 4-byte boundary. + * + * If the "entire" message has data, then this is followed by: + * + * * N data parts (all but the last of size PART_LEN) + * * padding to bring that up to a 4-byte boundary. + * + * and finally, whether there was data or not: + * + * * the final end guard. + * + * Remember that kbus_read always delivers an "entire" message. + */ +enum kbus_msg_parts { + KBUS_PART_HDR = 0, + KBUS_PART_NAME, + KBUS_PART_NPAD, + KBUS_PART_DATA, + KBUS_PART_DPAD, + KBUS_PART_FINAL_GUARD +}; +/* N.B. New message parts require switch cases in kbus_msg_part_name() + * and kbus_write_parts(). + */ + +#define KBUS_NUM_PARTS (KBUS_PART_FINAL_GUARD+1) + +/* + * Replier typing. + * The higher the replier type, the more specific it is. + * We trust the binding mechanisms not to have created two replier + * bindings of the same type for the same name (so we shan't, for + * example, get '$.Fred.*' bound as replier twice). + */ + +enum kbus_replier_type { + UNSET = 0, + WILD_STAR, + WILD_PERCENT, + SPECIFIC +}; + +/* + * A reference counting wrapper for message data + * + * If 'as_pages' is false, then the data is stored as a single kmalloc'd + * entity, pointed to by 'parts[0]'. In this case, 'num_parts' will be 1, + * and 'last_page_len' will be the size of the allocated data. + * + * If 'as_pages' is true, then the data is stored as 'num_parts' pages, each + * pointed to by 'parts[n]'. The last page should be treated as being size + * 'last_page_len' (even if the implementation is not enforcing this). All + * other pages are of size PART_LEN. + * + * In either case, 'lengths[n]' is a "fill counter" for how many bytes of data + * are actually being stored in page 'n'. Once the data is all in place, this + * should be equal to PART_LEN or 'last_page_len' as appropriate. + * + * 'refcount' is a stanard kernel reference count for the data - when it reaches + * 0, everything (the parts, the arrays and the datastructure) gets freed. + */ +struct kbus_data_ptr { + int as_pages; + unsigned num_parts; + unsigned long *parts; + unsigned *lengths; + unsigned last_page_len; + struct kref refcount; +}; + +/* + * A reference counting wrapper for message names + * + * RATIONALE: + * + * When a message name is copied from user space, we take our first copy. + * + * In order to send a message to a Ksock, we use kbus_push_message(), which + * takes a copy of the message for each recipient. + * + * We also take a copy of the message name for our list of messages that have + * been read but not replied to. + * + * It makes sense to copy the message header, because the contents thereof are + * changed according to the particular recipient. + * + * Copying the message data (if any) is handled by the kbus_data_ptr, above, + * which provides reference counting. This makes sense because message data may + * be large. + * + * If we only have one recipient, copying the message name is not a big issue, + * but if there are many, we would prefer not to make many copies of the + * string. It is, perhaps, worth keeping a dictionary of message names. and + * referring to the name in that - but that's not an incremental change from + * the "simple copying" state we start from. + * + * The simplest change to make, which may have some benefit, is to reference + * count the names for an individual message, as is done for the message data. + * + * If we have a single recipient, we will have copied the string from user + * space, and also created the kbus_name_ptr datastructure - an overhead of 8 + * bytes. However, when we copy the message for the recipient, we do not need + * to copy the message name, so if the message name is more than 8 bytes, we + * have immediately made a gain (and experience shows that message names tend + * to be at least that long). + * + * As soon as we have more than one recipient, it becomes extremely likely that + * we have saved space, and we will definitely have saved allocations which + * could be fragmenting memory. So it sounds like a good thing to try. + * + * Also, if I later on want to store a hash code for the string (hoping to + * speed up comparisons), the new datastructure gives me somewhere to put it... + */ +struct kbus_name_ptr { + char *name; + struct kref refcount; +}; + +/* + * When the user reads a message from us, they receive a kbus_entire_message + * structure. + * + * When the user writes a message to us, they write a "pointy" message, using + * the kbus_message_header structure, or an "entire" message, using the + * kbus_entire_message structure. + * + * Within the kernel, all messages are held as "pointy" messages, but instead + * of direct pointers to the message name and data, we use reference counted + * pointers. + * + * Rather than overload the 'name' and 'data' pointer fields, with all the + * danger of getting it wrong that that implies, it seems simpler to have our + * own, internal to the kernel, clone of the datastructure, but with these + * fields defined correctly... + * + * Hmm. If we have the name and data references, perhaps we should move the + * name and data *lengths* into those same. + */ + +struct kbus_msg { + struct kbus_msg_id id; /* Unique to this message */ + struct kbus_msg_id in_reply_to; /* Which message this is a reply to */ + u32 to; /* 0 (empty) or a replier id */ + u32 from; /* 0 (KBUS) or the sender's id */ + struct kbus_orig_from orig_from; /* Cross-network linkage */ + struct kbus_orig_from final_to; /* Cross-network linkage */ + u32 extra; /* ignored field - future proofing */ + u32 flags; /* Message type/flags */ + u32 name_len; /* Message name's length, in bytes */ + u32 data_len; /* Message length, also in bytes */ + struct kbus_name_ptr *name_ref; + struct kbus_data_ptr *data_ref; +}; + +/* + * The current message that the user is reading (with kbus_read()) + * + * If 'msg' is NULL, then the data structure is "empty" (i.e., there is no + * message being read). + */ +struct kbus_read_msg { + struct kbus_entire_message user_hdr; /* the header for user space */ + + struct kbus_msg *msg; /* the internal message */ + char *parts[KBUS_NUM_PARTS]; + unsigned lengths[KBUS_NUM_PARTS]; + int which; /* The current item */ + u32 pos; /* How far they've read in it */ + /* + * If the current item is KBUS_PART_DATA then we 'ref_data_index' is + * which part of the data we're in, and 'pos' is how far we are through + * that particular item. + */ + u32 ref_data_index; +}; + +/* + * See kbus_write_parts() for how this data structure is actually used. + * + * If 'msg' is NULL, then the data structure is "empty" (i.e., there is no + * message being written). + * + * * 'is_finished' is true when we've got all the bytes for our message, + * and thus don't want any more. It's an error for the user to try to + * write more message after it is finished. + * + * For a "pointy" message, this is set immediately after the message header + * end guard is finished (the message name and any data aren't "pulled in" + * until the user does SEND). For an "entire" message, this is set after the + * final end guard is finished (so we will have the message name and any data + * in memory). + * + * * 'pointers_are_local' is true if the message's name and data have been + * transferred to kernel space (as reference counted entities), and false + * if they are (still) in user space. + * + * * 'hdr' is the message header, the shorter in-kernel version. + * + * * 'which' indicates which part of the message we think we're being given + * bytes for, from KBUS_PART_HDR through to (for an "entire" message) + * KBUS_PART_FINAL_GUARD. + * * 'pos' is the index of the next byte within the current part of whatever + * we're working on, as indicated by 'which'. Note that for message data, + * this is the index within the whole of the data (not the index within a + * data part). + * + * * If we're reading an "entire" message, then the message name gets written + * to 'ref_name', which is a reference-counted string. This is allocated to + * the correct size/shape for the entire message name, after the head has + * been read. + * + * The intention is that, if 'ref_name' is non-NULL, it should be legal + * to call 'kbus_lower_name_ref()' on it, to free its contents. + * + * * Similarly, 'ref_data' is reference-counted data, again allocated to the + * correct size/shape for the entire message data length, after the header + * has been read. The 'length' for each part is used to indicate how far + * through that part we have populated with bytes. + * + * The intention is that, if 'ref_data' is non-NULL, it should be legal + * to call 'kbus_lower_data_ref()' on it, to free its contents. + * + * 'ref_data_index' is then the index (starting at 0) of the referenced + * data part that we are populating. + */ +struct kbus_write_msg { + struct kbus_entire_message user_msg; /* from user space */ + struct kbus_msg *msg; /* our version of it */ + + u32 is_finished; + u32 pointers_are_local; + u32 guard; /* Whichever guard we're reading */ + char *user_name_ptr; /* User space name */ + void *user_data_ptr; /* User space data */ + enum kbus_msg_parts which; + u32 pos; + struct kbus_name_ptr *ref_name; + struct kbus_data_ptr *ref_data; + u32 ref_data_index; +}; + +/* + * The data for an unsent Replier Bind Event (in the unsent_unbind_msg_list) + * + * Note that 'binding' may theroretically be NULL (although I don't think this + * should ever actually happen). + */ +struct kbus_unsent_message_item { + struct list_head list; + struct kbus_private_data *send_to; /* who we want to send it to */ + u32 send_to_id; /* but the id is often useful */ + struct kbus_msg *msg; /* the message itself */ + struct kbus_message_binding *binding; /* and why we remembered it */ +}; + +/* + * This is the data for an individual Ksock + * + * Each time we open /dev/kbus<n>, we need to remember a unique id for + * our file-instance. Using 'filp' might work, but it's not something + * we have control over, and in particular, if the file is closed and + * then reopened, there's no guarantee that a particular value of 'filp' + * won't be used again. A simple serial number is safer. + * + * Each such "opening" also has a message queue associated with it. Any + * messages this "opening" has declared itself a listener (or replier) + * for will be added to that queue. + * + * 'id' is the unique id for this file descriptor - it enables stateful message + * transtions, etc. It is local to the particular KBUS device. + * + * 'last_msg_id_sent' is the message id of the last message that was + * (successfully) written to this file descriptor. It is needed when + * constructing a reply. + * + * We have a queue of messages waiting for us to read them, in 'message_queue'. + * 'message_count' is how many messages are in the queue, and 'max_messages' + * is an indication of how many messages we shall allow in the queue. + * + * Note that, however a message was originally sent to us, messages held + * internally are always a message header plus pointers to a message name and + * (optionally) message data. See kbus_send() for details. + */ +struct kbus_private_data { + struct list_head list; + struct kbus_dev *dev; /* Which device we are on */ + u32 id; /* Our own id */ + struct kbus_msg_id last_msg_id_sent; /* As it says - see above */ + u32 message_count; /* How many messages for us */ + u32 max_messages; /* How many messages allowed */ + struct list_head message_queue; /* Messages for us */ + + /* + * It's useful (for /proc/kbus/bindings) to remember the PID of the + * current process + */ + pid_t pid; + + /* Wait for something to appear in the message_queue */ + wait_queue_head_t read_wait; + + /* The message currently being read by the user */ + struct kbus_read_msg read; + + /* The message currently being written by the user */ + struct kbus_write_msg write; + + /* Are we currently sending that message? */ + int sending; + + /* + * Each request we send should (eventually) generate us a reply, or + * at worst a status message from KBUS itself telling us there isn't + * going to be one. So we need to ensure that there is room in our + * (as the sender) message queue to receive all/any such. + * + * Note that this *also* allows SEND to forbid sending a Reply to a + * Request that we did not receive (or to which we have already + * replied) + */ + struct kbus_msg_id_mem outstanding_requests; + + /* + * If we are a replier for a message, then KBUS wants to ensure + * that a reply is *definitely* made. If we release ourselves, then + * we're clearly not going to reply to any requests that we have + * read but not replied to, and KBUS would like to generate a status + * message for each such. So we need a list of the information needed + * to form such Status/Reply messages. + * + * (Thus we don't need the whole of the original message, since + * we're only *really* needing its name, its id and who its + * from -- given which its easiest just to keep the parts we + * *do* need, and ignore the data.) + * + * It was decided not to place a limit on the size of this list. + * Its size is limited by the ability of sender(s) to send + * requests, which in turn is limited by the the number of slots + * they can reserve for the replies to those requests in their + * own message queues. + * + * If a limit was imposed, then we would also need to stop a sender + * sending a request because the replier has too many replies + * outstanding (for instance, because it has gone to sleep). But + * then we'd assume that it is not responding to messages in + * general, and so its message queue would fill up, and that + * should be sufficient protection. + */ + struct list_head replies_unsent; + u32 num_replies_unsent; + u32 max_replies_unsent; + + /* + * Managing which messages a replier may reply to + * ---------------------------------------------- + * We need to police replying, such that a replier may only reply + * to requests that it has received (where "received" means "had + * placed into its message queue", because KBUS must reply for us + * if the particular Ksock is not going to). + * + * It is possible to do this using either the 'outstanding_requests' + * or the 'replies_unsent' list. + * + * Using the 'outstanding_requests' list means that when a replier + * wants to send a reply, it needs to look up who the original-sender + * is (from its Ksock id, in the "from" field of the message), and + * check against that. This is a bit inefficient. + * + * Using the 'replies_unsent' list means that when a replier wants + * to send a reply, it just needs to find the right message stub + * in said 'replies_unsent' list, and check that the reply *does* + * match the original request. This may be more efficient, depending. + * + * In fact, the 'outstanding_requests' list is used, simply because + * it was implemented first. + */ + + /* + * By default, if a Ksock binds to a message name as both Replier and + * Listener (typically by binding to a specific message name as Replier + * and to a wildcard including it as Listener), and a Reqest of that + * name is sent to that Ksock, it will get the message once as Replier + * (marked "WANT_YOU_TO_REPLY"), and once as listener. + * + * This is technically sensible, but can be irritating to the end user + * who really often only wants to receive the message once. + * + * If "messages_only_once" is set, then when a message is about to be + * put onto a Ksocks message queue, it will only be added if it (i.e., + * a message with the same id) has not already just been added. This + * is safe because Requests to the specific Replier are always dealt + * with first. + * + * As a side-effect, which I think also makes sense, this will also + * mean that if a Listener has bound to the same message name multiple + * times (as a Listener), then they will only get the message once. + */ + int messages_only_once; + /* + * Messages can be added to either end of our message queue (i.e., + * depending on whether they're urgent or not). This means that the + * "only once" mechanism needs to check both ends of the queue (which + * is a pain). Or we can just remember the message id of the last + * message pushed onto the queue. Which is much simpler. + */ + struct kbus_msg_id msg_id_just_pushed; + + /* + * If this flag is set, then we may have outstanding Replier Unbound + * Event messages (kept on a list on our device). These must be read + * before any "normal" messages (on our message_queue) get read. + */ + int maybe_got_unsent_unbind_msgs; +}; + +/* What is a sensible number for the default maximum number of messages? */ +#ifndef CONFIG_KBUS_DEF_MAX_MESSAGES +#define CONFIG_KBUS_DEF_MAX_MESSAGES 100 +#endif + +/* + * What about the maximum number of unsent unbind event messages? + * This may want to be quite large, to allow for Limpets with momentary + * network outages. + * + * The default value is probably too small, but experimantation is + * needed to determine a more sensible value. + */ +#ifndef CONFIG_KBUS_MAX_UNSENT_UNBIND_MESSAGES +#define CONFIG_KBUS_MAX_UNSENT_UNBIND_MESSAGES 1000 +#endif + +/* Information belonging to each /dev/kbus<N> device */ +struct kbus_dev { + struct cdev cdev; /* Character device data */ + struct device *dev; /* Our very selves */ + + u32 index; /* Which /dev/kbus<n> device we are */ + + /* + * The Big Lock + * We use a single mutex for all purposes, and all locking is done + * at the "top level", i.e., in the externally called functions. + * This simplifies the design of the internal (list processing, + * etc.) functions, at the possible cost of making interaction + * with KBUS, in general, slower. + * + * On the other hand, we favour reliable over fast. + */ + struct mutex mux; + + /* Who has bound to receive which messages in what manner */ + struct list_head bound_message_list; + + /* + * The actual Ksock entries (one per 'open("/dev/kbus<n>")') + * This is to allow us to find the 'kbus_private_data' instances, + * so that we can get at all the message queues. The details of + * how we do this are *definitely* going to change... + */ + struct list_head open_ksock_list; + + /* Has one of our Ksocks made space available in its message queue? */ + wait_queue_head_t write_wait; + + /* + * Each open file descriptor needs an internal id - this is used + * when binding messages to listeners, but is also needed when we + * want to reply. We reserve the id 0 as a special value ("none"). + */ + u32 next_ksock_id; + + /* + * Every message sent has a unique id (again, unique per device). + */ + u32 next_msg_serial_num; + + /* Are we wanting debugging messages? */ + u32 verbose; + + /* + * Are we wanting to send a synthetic message for each Replier + * bind/unbind? */ + u32 report_replier_binds; + + /* + * If Replier (un)bind events have been requested, then when + * kbus_release is called, a message must be sent for each Replier that + * is (of necessity) unbound from the Ksock being released. For a + * normal unbound, if any of the Repliers doesn't have room in its + * message queue for such an event, then the unbind fails with -EAGAIN. + * This isn't acceptable for kbus_release (apart from anything else, + * the release might be due to the original program falling over). + * It's not acceptable to fail to send the messages (that's a general + * KBUS principle). + * + * The only sensible solution seems to be to put the messages we'd + * like to have sent onto a set-aside list, and mark each recipient + * as having messages thereon. Then, each time a Ksock looks for a + * new message, it should first check to see if it might have one + * on the set-aside list, and if it does, read that instead. + * + * Once we're doing this, though, we need some limit on how big that + * set-aside list may grow (to allow for user processes that keep + * binding and falling over!). When the list gets "too long", we set a + * "gone tragically wrong" flag, and instead of adding more unbind + * events, we instead add a single "gone tragically wrong" message for + * each Ksock. We don't revert to remembering unbind events again until + * the list has been emptied. + */ + struct list_head unsent_unbind_msg_list; + u32 unsent_unbind_msg_count; + int unsent_unbind_is_tragic; +}; + +/* + * Each entry in a message queue holds a single message, and a pointer to + * the message name binding that caused it to be added to the list. This + * makes it simple to remove messages from the queue if the message name + * binding is unbound. The binding shall be NULL for: + * + * * Replies + * * KBUS "synthetic" messages, which are also (essentialy) Replies + */ +struct kbus_message_queue_item { + struct list_head list; + struct kbus_msg *msg; + struct kbus_message_binding *binding; +}; + +/* The sizes of the parts in our reference counted data */ +#define KBUS_PART_LEN PAGE_SIZE +#define KBUS_PAGE_THRESHOLD (PAGE_SIZE >> 1) + +#endif /* _kbus_internal */ -- 1.7.1 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH 3/5] Kconfig files and Makefile for KBUS 2011-02-27 18:11 ` [PATCH 3/5] Kconfig files and Makefile " Tony Ibbs 2011-02-27 18:11 ` [PATCH 4/5] Internal header file " Tony Ibbs @ 2011-02-27 18:19 ` Randy Dunlap 2011-02-27 19:36 ` Tony Ibbs 1 sibling, 1 reply; 15+ messages in thread From: Randy Dunlap @ 2011-02-27 18:19 UTC (permalink / raw) To: Tony Ibbs; +Cc: Linux-embedded, Tibs at Kynesim, Richard Watts, Grant Likely On Sun, 27 Feb 2011 18:11:46 +0000 Tony Ibbs wrote: > From: Tibs <tibs@tonyibbs.co.uk> > > Amend the ipc Makefile. > Add an ipc Kconfig for KBUS. > Amend the init Kconfig appropriately. > > Signed-off-by: Tony Ibbs <tibs@tonyibbs.co.uk> > --- > init/Kconfig | 3 ++ > ipc/Kconfig | 99 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > ipc/Makefile | 5 +++ > 3 files changed, 107 insertions(+), 0 deletions(-) > create mode 100644 ipc/Kconfig > > diff --git a/init/Kconfig b/init/Kconfig > index c972899..a31a6fe 100644 > --- a/init/Kconfig > +++ b/init/Kconfig > @@ -339,6 +339,8 @@ config AUDIT_TREE > depends on AUDITSYSCALL > select FSNOTIFY > > +source "ipc/Kconfig" > + > source "kernel/irq/Kconfig" > > menu "RCU Subsystem" > @@ -1305,3 +1307,4 @@ config PADATA > bool > > source "kernel/Kconfig.locks" > + You can drop that line. > diff --git a/ipc/Kconfig b/ipc/Kconfig > new file mode 100644 > index 0000000..46ebcda > --- /dev/null > +++ b/ipc/Kconfig > @@ -0,0 +1,99 @@ > +config KBUS > + tristate "KBUS messaging system" > + default m Use tabs to indent kconfig entry text, like above (not spaces). Please do not enable new configs unless they are absolutely required. > + ---help--- > + KBUS is a lightweight messaging system, particularly aimed > + at embedded platforms. This option provides the kernel support > + for mediating messages between client processes. > + > + If you want KBUS support, you should say Y here. > + > + This support can also be built as a module. If so, the module > + will be called kbus. --- ~Randy *** Remember to use Documentation/SubmitChecklist when testing your code *** ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 3/5] Kconfig files and Makefile for KBUS 2011-02-27 18:19 ` [PATCH 3/5] Kconfig files and Makefile " Randy Dunlap @ 2011-02-27 19:36 ` Tony Ibbs 0 siblings, 0 replies; 15+ messages in thread From: Tony Ibbs @ 2011-02-27 19:36 UTC (permalink / raw) To: Randy Dunlap; +Cc: Linux-embedded, Tibs at Kynesim, Richard Watts, Grant Likely On 27 Feb 2011, at 18:19, Randy Dunlap wrote: > On Sun, 27 Feb 2011 18:11:46 +0000 Tony Ibbs wrote: >> >> source "kernel/Kconfig.locks" >> + > > You can drop that line. Thanks. >> + tristate "KBUS messaging system" >> + default m > > Use tabs to indent kconfig entry text, like above (not spaces). Drat. Thank you. > Please do not enable new configs unless they are absolutely required. And that , of course, makes sense. Thank you again. Tibs ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 5/5] KBUS source file 2011-02-27 18:11 [PATCH 0/5] RFC: KBUS messaging subsystem Tony Ibbs 2011-02-27 18:11 ` [PATCH 1/5] Documentation for KBUS Tony Ibbs @ 2011-02-27 19:26 ` Tony Ibbs 2011-02-27 20:25 ` Randy Dunlap 2011-02-28 19:23 ` Sam Ravnborg 2011-02-28 7:48 ` [PATCH 0/5] RFC: KBUS messaging subsystem Grant Likely 2 siblings, 2 replies; 15+ messages in thread From: Tony Ibbs @ 2011-02-27 19:26 UTC (permalink / raw) To: Tony Ibbs; +Cc: Linux-embedded, Tibs at Kynesim, Richard Watts, Grant Likely Apologies, my fifth patch (the actual KBUS source file) appears to be too long - I hadn't realise the 100,000 character limit on majordomo messages. The file as submitted is that at: https://github.com/crazyscot/linux-2.6-kbus/blob/6cc3262bd3e28757179dd8a0d30298dfb2f70845/ipc/kbus.c If there is a better way for me to submit this, please let me know. The source is still Signed-off-by: Tony Ibbs <tibs@tonyibbs.co.uk> ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 5/5] KBUS source file 2011-02-27 19:26 ` [PATCH 5/5] KBUS source file Tony Ibbs @ 2011-02-27 20:25 ` Randy Dunlap 2011-02-28 19:23 ` Sam Ravnborg 1 sibling, 0 replies; 15+ messages in thread From: Randy Dunlap @ 2011-02-27 20:25 UTC (permalink / raw) To: Tony Ibbs; +Cc: Linux-embedded, Tibs at Kynesim, Richard Watts, Grant Likely On Sun, 27 Feb 2011 19:26:49 +0000 Tony Ibbs wrote: > Apologies, my fifth patch (the actual KBUS source file) appears to be > too long - I hadn't realise the 100,000 character limit on majordomo > messages. The file as submitted is that at: > > https://github.com/crazyscot/linux-2.6-kbus/blob/6cc3262bd3e28757179dd8a0d30298dfb2f70845/ipc/kbus.c > > If there is a better way for me to submit this, please let me know. > > The source is still > Signed-off-by: Tony Ibbs <tibs@tonyibbs.co.uk>-- linux-kernel@vger.kernel.org allows up to 400 KB iirc, though it might be 300 KB. --- ~Randy *** Remember to use Documentation/SubmitChecklist when testing your code *** ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 5/5] KBUS source file 2011-02-27 19:26 ` [PATCH 5/5] KBUS source file Tony Ibbs 2011-02-27 20:25 ` Randy Dunlap @ 2011-02-28 19:23 ` Sam Ravnborg 2011-02-28 22:25 ` Tony Ibbs 1 sibling, 1 reply; 15+ messages in thread From: Sam Ravnborg @ 2011-02-28 19:23 UTC (permalink / raw) To: Tony Ibbs; +Cc: Linux-embedded, Tibs at Kynesim, Richard Watts, Grant Likely On Sun, Feb 27, 2011 at 07:26:49PM +0000, Tony Ibbs wrote: > Apologies, my fifth patch (the actual KBUS source file) appears to be > too long - I hadn't realise the 100,000 character limit on majordomo > messages. The file as submitted is that at: > > https://github.com/crazyscot/linux-2.6-kbus/blob/6cc3262bd3e28757179dd8a0d30298dfb2f70845/ipc/kbus.c You are using /proc/* to something that is not process related. At lkml you will be requested to use something else. You can conisder splitting this up in another file so you in the Makefile can use it conditionally. > If there is a better way for me to submit this, please let me know. One way you would not like.... Let the first patches introduce core-functionality and add features in later patches - touching the same file many times. Also please submit the Makefile/Kconfig stuff in the _last_ mail. Otherwise you would ruin bisect because we cannot build kbus until you have the last files included. Sam ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 5/5] KBUS source file 2011-02-28 19:23 ` Sam Ravnborg @ 2011-02-28 22:25 ` Tony Ibbs 2011-03-01 5:42 ` Sam Ravnborg 0 siblings, 1 reply; 15+ messages in thread From: Tony Ibbs @ 2011-02-28 22:25 UTC (permalink / raw) To: Sam Ravnborg; +Cc: Linux-embedded, Tibs at Kynesim, Richard Watts, Grant Likely On 28 Feb 2011, at 19:23, Sam Ravnborg wrote: > You are using /proc/* to something that is not process related. > At lkml you will be requested to use something else. OK. That's mainly through ignorance of what I *should* be using instead (and perhaps poor reading of Linux Device Drivers). Do you have an opinion on where such information should be going? > You can conisder splitting this up in another file so you > in the Makefile can use it conditionally. That makes sense. >> If there is a better way for me to submit this, please let me know. > > One way you would not like.... > Let the first patches introduce core-functionality and add features > in later patches - touching the same file many times. Hmm. That's very sensible. Assuming I can figure out a sensible way to do it (which I assume is what you expected me to object). It would certainly be interesting to do (sort of reverse archaeology), but would take a while, and it would, of course, also mean rewriting all of my tests to only test whatever is still there (I'm assuming a source file is only interesting if it actually *works*). It would, of course, also have the advantage of presenting the "simplest possible" idea of KBUS, followed by "but it appears also to be necessary to do this" patches. Which is presumably why you suggest it. > Also please submit the Makefile/Kconfig stuff in the _last_ mail. > Otherwise you would ruin bisect because we cannot build kbus until > you have the last files included. Ah. Not something I have experience of. I shall make it so. Thank you, Tibs ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 5/5] KBUS source file 2011-02-28 22:25 ` Tony Ibbs @ 2011-03-01 5:42 ` Sam Ravnborg 0 siblings, 0 replies; 15+ messages in thread From: Sam Ravnborg @ 2011-03-01 5:42 UTC (permalink / raw) To: Tony Ibbs; +Cc: Linux-embedded, Tibs at Kynesim, Richard Watts, Grant Likely On Mon, Feb 28, 2011 at 10:25:41PM +0000, Tony Ibbs wrote: > > On 28 Feb 2011, at 19:23, Sam Ravnborg wrote: > > > You are using /proc/* to something that is not process related. > > At lkml you will be requested to use something else. > > OK. That's mainly through ignorance of what I *should* be using > instead (and perhaps poor reading of Linux Device Drivers). Do you > have an opinion on where such information should be going? I think sysfs is the better place. But my experience in this ares is limited. > >> If there is a better way for me to submit this, please let me know. > > > > One way you would not like.... > > Let the first patches introduce core-functionality and add features > > in later patches - touching the same file many times. > > Hmm. That's very sensible. Assuming I can figure out a sensible way > to do it (which I assume is what you expected me to object). It is a lot of work when you already have something that works. But sometimes it helps to do so. I have previously done a complete rewrite in the process (becuase my initial version was bad). Sam ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 0/5] RFC: KBUS messaging subsystem 2011-02-27 18:11 [PATCH 0/5] RFC: KBUS messaging subsystem Tony Ibbs 2011-02-27 18:11 ` [PATCH 1/5] Documentation for KBUS Tony Ibbs 2011-02-27 19:26 ` [PATCH 5/5] KBUS source file Tony Ibbs @ 2011-02-28 7:48 ` Grant Likely 2011-02-28 7:52 ` Grant Likely 2 siblings, 1 reply; 15+ messages in thread From: Grant Likely @ 2011-02-28 7:48 UTC (permalink / raw) To: Tony Ibbs; +Cc: Linux-embedded, Tibs at Kynesim, Richard Watts On Sun, Feb 27, 2011 at 06:11:43PM +0000, Tony Ibbs wrote: > KBUS is a lightweight, Linux kernel mediated, message system, > particularly intended for use in embedded environments. > > It is meant to be simple to use and understand. It is designed to > provide predictable message delivery, deterministic message ordering, > and a guaranteed reply for each request. > > It is especially aimed at situations where existing solutions, > such as DBUS, cannot be used, typically because of system constraints. > > We have various customers using KBUS in real life, and believe it to > be useful. I had a showcase table for KBUS at the ELCE in Cambridge, > October last year, and there seemed to be interest. > > I am hoping to get a response from this list before attempting to send > to the kernel list, since I believe this list should contain people > who might be most interested in using it. Just go ahead and post to lkml. There isn't a whole lot of point in holding back from there, and that is the only way you'll get the level of review that you need to even be considered for merging. g. ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 0/5] RFC: KBUS messaging subsystem 2011-02-28 7:48 ` [PATCH 0/5] RFC: KBUS messaging subsystem Grant Likely @ 2011-02-28 7:52 ` Grant Likely 2011-02-28 19:13 ` Tony Ibbs 0 siblings, 1 reply; 15+ messages in thread From: Grant Likely @ 2011-02-28 7:52 UTC (permalink / raw) To: Tony Ibbs; +Cc: Linux-embedded, Tibs at Kynesim, Richard Watts On Mon, Feb 28, 2011 at 12:48 AM, Grant Likely <grant.likely@secretlab.ca> wrote: > On Sun, Feb 27, 2011 at 06:11:43PM +0000, Tony Ibbs wrote: >> KBUS is a lightweight, Linux kernel mediated, message system, >> particularly intended for use in embedded environments. >> >> It is meant to be simple to use and understand. It is designed to >> provide predictable message delivery, deterministic message ordering, >> and a guaranteed reply for each request. >> >> It is especially aimed at situations where existing solutions, >> such as DBUS, cannot be used, typically because of system constraints. >> >> We have various customers using KBUS in real life, and believe it to >> be useful. I had a showcase table for KBUS at the ELCE in Cambridge, >> October last year, and there seemed to be interest. >> >> I am hoping to get a response from this list before attempting to send >> to the kernel list, since I believe this list should contain people >> who might be most interested in using it. > > Just go ahead and post to lkml. There isn't a whole lot of point in > holding back from there, and that is the only way you'll get the level > of review that you need to even be considered for merging. In fact, I think I'll hold off on reviewing this series until lkml is cc'd so that I don't end up repeating myself. :-) g. ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 0/5] RFC: KBUS messaging subsystem 2011-02-28 7:52 ` Grant Likely @ 2011-02-28 19:13 ` Tony Ibbs 0 siblings, 0 replies; 15+ messages in thread From: Tony Ibbs @ 2011-02-28 19:13 UTC (permalink / raw) To: Grant Likely; +Cc: Linux-embedded, Tibs at Kynesim, Richard Watts On 28 Feb 2011, at 07:52, Grant Likely wrote: > On Mon, Feb 28, 2011 at 12:48 AM, Grant Likely > <grant.likely@secretlab.ca> wrote: >> >> Just go ahead and post to lkml. There isn't a whole lot of point in >> holding back from there, and that is the only way you'll get the level >> of review that you need to even be considered for merging. > > In fact, I think I'll hold off on reviewing this series until lkml is > cc'd so that I don't end up repeating myself. :-) That echoes what someone else (offline) was saying earlier today. I think you are clearly right. Later in the week, unfortunately, though. Tibs ^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2011-03-01 5:42 UTC | newest] Thread overview: 15+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2011-02-27 18:11 [PATCH 0/5] RFC: KBUS messaging subsystem Tony Ibbs 2011-02-27 18:11 ` [PATCH 1/5] Documentation for KBUS Tony Ibbs 2011-02-27 18:11 ` [PATCH 2/5] External header file " Tony Ibbs 2011-02-27 18:11 ` [PATCH 3/5] Kconfig files and Makefile " Tony Ibbs 2011-02-27 18:11 ` [PATCH 4/5] Internal header file " Tony Ibbs 2011-02-27 18:19 ` [PATCH 3/5] Kconfig files and Makefile " Randy Dunlap 2011-02-27 19:36 ` Tony Ibbs 2011-02-27 19:26 ` [PATCH 5/5] KBUS source file Tony Ibbs 2011-02-27 20:25 ` Randy Dunlap 2011-02-28 19:23 ` Sam Ravnborg 2011-02-28 22:25 ` Tony Ibbs 2011-03-01 5:42 ` Sam Ravnborg 2011-02-28 7:48 ` [PATCH 0/5] RFC: KBUS messaging subsystem Grant Likely 2011-02-28 7:52 ` Grant Likely 2011-02-28 19:13 ` Tony Ibbs
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).