why is it not allowed to add a new socket protocol family as an external module?

why is it not allowed to add a new socket protocol family as an external module?

[Chris Friesen]

Hi,

I was just wondering why the kernel doesn't allow a new network protocol 
family to be loaded as as a kernel module built outside the kernel 
source tree.

Most of the networking code would seem to allow this, but the check for
"if (ops->family >= NPROTO)" in sock_register() means that only protocol 
families defined in socket.h can be registered.

Was this intentional for ideological reasons, or has nobody wanted to be 
able to add arbitrary network protocols without patching the kernel and 
rebuilding?

Thanks,
Chris

Re: why is it not allowed to add a new socket protocol family as an external module?

[Stephen Hemminger]

On Wed, 20 Feb 2013 10:56:13 -0600
Chris Friesen <chris.friesen@genband.com> wrote:

> Hi,
> 
> I was just wondering why the kernel doesn't allow a new network protocol 
> family to be loaded as as a kernel module built outside the kernel 
> source tree.
> 
> Most of the networking code would seem to allow this, but the check for
> "if (ops->family >= NPROTO)" in sock_register() means that only protocol 
> families defined in socket.h can be registered.
> 
> Was this intentional for ideological reasons, or has nobody wanted to be 
> able to add arbitrary network protocols without patching the kernel and 
> rebuilding?
> 

If you want an answer, to the question, use a tool like cscope and
learn to read the kernel code. There are several tables of pointers sized by NPROTO.

Re: why is it not allowed to add a new socket protocol family as an external module?

[Chris Friesen]

On 02/20/2013 05:23 PM, Stephen Hemminger wrote:
> On Wed, 20 Feb 2013 10:56:13 -0600
> Chris Friesen<chris.friesen@genband.com>  wrote:
>
>> Hi,
>>
>> I was just wondering why the kernel doesn't allow a new network protocol
>> family to be loaded as as a kernel module built outside the kernel
>> source tree.

> If you want an answer, to the question, use a tool like cscope and
> learn to read the kernel code. There are several tables of pointers sized by NPROTO.

That's a bit insulting, don't you think?

Yes, there are a number of tables sized by NPROTO/AF_MAX/PF_MAX (and the 
fact that we use all three within the kernel is kind of sad) but there 
is no technical reason why we couldn't extend those dynamically if 
desired (with a linked list of additional protocols, perhaps).

Hence my question--is the restriction for an ideological reason or 
simply because nobody thought it was worth the effort?

Chris

Re: why is it not allowed to add a new socket protocol family as an external module?

[David Miller]

From: Chris Friesen <chris.friesen@genband.com>
Date: Wed, 20 Feb 2013 18:44:50 -0600

> On 02/20/2013 05:23 PM, Stephen Hemminger wrote:
>> On Wed, 20 Feb 2013 10:56:13 -0600
>> Chris Friesen<chris.friesen@genband.com>  wrote:
>>
>>> Hi,
>>>
>>> I was just wondering why the kernel doesn't allow a new network
>>> protocol
>>> family to be loaded as as a kernel module built outside the kernel
>>> source tree.
> 
>> If you want an answer, to the question, use a tool like cscope and
>> learn to read the kernel code. There are several tables of pointers
>> sized by NPROTO.
> 
> That's a bit insulting, don't you think?

Absolutely not insulting at all.

The list is _NOT_ a place to go when you're just too damn lazy to take
the 10 seconds it would have taken to answer your question with a
quick NPROTO grep on the kernel sources.

Stephen's response to you was therefore %100 appropriate, and I would
have told you likewise if I had been the first to respond.

Re: why is it not allowed to add a new socket protocol family as an external module?

[Chris Friesen]

On 02/20/2013 07:05 PM, David Miller wrote:
> From: Chris Friesen<chris.friesen@genband.com> Date: Wed, 20 Feb 2013
> 18:44:50 -0600
>
>> On 02/20/2013 05:23 PM, Stephen Hemminger wrote:
>>> On Wed, 20 Feb 2013 10:56:13 -0600 Chris
>>> Friesen<chris.friesen@genband.com>   wrote:
>>>
>>>> Hi,
>>>>
>>>> I was just wondering why the kernel doesn't allow a new
>>>> network protocol family to be loaded as as a kernel module
>>>> built outside the kernel source tree.
>>
>>> If you want an answer, to the question, use a tool like cscope
>>> and learn to read the kernel code. There are several tables of
>>> pointers sized by NPROTO.
>>
>> That's a bit insulting, don't you think?
>
> Absolutely not insulting at all.
>
> The list is _NOT_ a place to go when you're just too damn lazy to
> take the 10 seconds it would have taken to answer your question with
> a quick NPROTO grep on the kernel sources.
>
> Stephen's response to you was therefore %100 appropriate, and I
> would have told you likewise if I had been the first to respond.

As the rest of my reply to Stephen should have made clear, I in fact did 
read the code prior to asking the question.  My question was not about 
the details of the current implementation, but rather the rationale 
behind them.

I'm fully aware that there are statically-sized tables in the current 
code, and thus the current code does not support dynamically adding 
protocols.  However, it would be reasonably straightforward to extend 
the static tables with some form of dynamic data structure to handle 
adding new network protocols at runtime.

If the current restriction is intentional (to minimize the likelihood 
of non-GPL'd network protocols, for instance), then there's no point in 
anyone submitting patches to add support for dynamic network protocols. 
  If it's just because nobody has bothered to do it yet, then it might 
be an interesting project.

Chris

Re: why is it not allowed to add a new socket protocol family as an external module?

[Eric Dumazet]

On Wed, 2013-02-20 at 18:44 -0600, Chris Friesen wrote:

> Yes, there are a number of tables sized by NPROTO/AF_MAX/PF_MAX (and the 
> fact that we use all three within the kernel is kind of sad) but there 
> is no technical reason why we couldn't extend those dynamically if 
> desired (with a linked list of additional protocols, perhaps).
> 
> Hence my question--is the restriction for an ideological reason or 
> simply because nobody thought it was worth the effort?
> 

I guess nobody did the preliminary work.

lockdep might be the tricky part.

net/core/sock.c:197:static const char *const af_family_key_strings[AF_MAX+1] = {
net/core/sock.c:211:  "sk_lock-AF_NFC"   , "sk_lock-AF_MAX"
net/core/sock.c:213:static const char *const af_family_slock_key_strings[AF_MAX+1] = {
net/core/sock.c:227:  "slock-AF_NFC"   , "slock-AF_MAX"
net/core/sock.c:229:static const char *const af_family_clock_key_strings[AF_MAX+1] = {
net/core/sock.c:243:  "clock-AF_NFC"   , "clock-AF_MAX"
net/core/sock.c:250:static struct lock_class_key af_callback_keys[AF_MAX];

Re: why is it not allowed to add a new socket protocol family as an external module?

[Chris Friesen]

On 02/20/2013 07:39 PM, Eric Dumazet wrote:
> On Wed, 2013-02-20 at 18:44 -0600, Chris Friesen wrote:
>
>> Yes, there are a number of tables sized by NPROTO/AF_MAX/PF_MAX (and the
>> fact that we use all three within the kernel is kind of sad) but there
>> is no technical reason why we couldn't extend those dynamically if
>> desired (with a linked list of additional protocols, perhaps).
>>
>> Hence my question--is the restriction for an ideological reason or
>> simply because nobody thought it was worth the effort?
>>
>
> I guess nobody did the preliminary work.
>
> lockdep might be the tricky part.
>
> net/core/sock.c:197:static const char *const af_family_key_strings[AF_MAX+1] = {
> net/core/sock.c:211:  "sk_lock-AF_NFC"   , "sk_lock-AF_MAX"
> net/core/sock.c:213:static const char *const af_family_slock_key_strings[AF_MAX+1] = {
> net/core/sock.c:227:  "slock-AF_NFC"   , "slock-AF_MAX"
> net/core/sock.c:229:static const char *const af_family_clock_key_strings[AF_MAX+1] = {
> net/core/sock.c:243:  "clock-AF_NFC"   , "clock-AF_MAX"
> net/core/sock.c:250:static struct lock_class_key af_callback_keys[AF_MAX];

Unless I'm missing something that looks straightforward.

When registering dynamically the new protocol would need to specify one 
string, the protocol name.  ("AF_MYPROTOCOL" or something).  The three 
key strings are derived from that.

The three lock_class_key structs (corresponding to 
af_family_keys/af_family_slock_keys/af_callback_keys) can be 
auto-allocated by the network core at dynamic registration time.

sock_lock_init(), sk_clone_lock(), and sock_init_data() would need 
special-casing for protocol number larger than AF_MAX.  The most 
readable would probably be to do the mapping from family to key/string 
in helper functions rather than hard-coding a table offset right in 
these routines.

Yes, there would be a runtime penalty, but it should be minimal.  For 
static protocols it would be a few "sk->sk_family < AF_MAX" checks.  For 
dynamic protocols they might need to walk a linked list, which in all 
probability would be very short.

Chris

Re: why is it not allowed to add a new socket protocol family as an external module?

[Stephen Hemminger]

On Thu, 21 Feb 2013 09:47:33 -0600
Chris Friesen <chris.friesen@genband.com> wrote:

> On 02/20/2013 07:39 PM, Eric Dumazet wrote:
> > On Wed, 2013-02-20 at 18:44 -0600, Chris Friesen wrote:
> >
> >> Yes, there are a number of tables sized by NPROTO/AF_MAX/PF_MAX (and the
> >> fact that we use all three within the kernel is kind of sad) but there
> >> is no technical reason why we couldn't extend those dynamically if
> >> desired (with a linked list of additional protocols, perhaps).
> >>
> >> Hence my question--is the restriction for an ideological reason or
> >> simply because nobody thought it was worth the effort?
> >>
> >
> > I guess nobody did the preliminary work.
> >
> > lockdep might be the tricky part.
> >
> > net/core/sock.c:197:static const char *const af_family_key_strings[AF_MAX+1] = {
> > net/core/sock.c:211:  "sk_lock-AF_NFC"   , "sk_lock-AF_MAX"
> > net/core/sock.c:213:static const char *const af_family_slock_key_strings[AF_MAX+1] = {
> > net/core/sock.c:227:  "slock-AF_NFC"   , "slock-AF_MAX"
> > net/core/sock.c:229:static const char *const af_family_clock_key_strings[AF_MAX+1] = {
> > net/core/sock.c:243:  "clock-AF_NFC"   , "clock-AF_MAX"
> > net/core/sock.c:250:static struct lock_class_key af_callback_keys[AF_MAX];
> 
> Unless I'm missing something that looks straightforward.
> 
> When registering dynamically the new protocol would need to specify one 
> string, the protocol name.  ("AF_MYPROTOCOL" or something).  The three 
> key strings are derived from that.
> 
> The three lock_class_key structs (corresponding to 
> af_family_keys/af_family_slock_keys/af_callback_keys) can be 
> auto-allocated by the network core at dynamic registration time.
> 
> sock_lock_init(), sk_clone_lock(), and sock_init_data() would need 
> special-casing for protocol number larger than AF_MAX.  The most 
> readable would probably be to do the mapping from family to key/string 
> in helper functions rather than hard-coding a table offset right in 
> these routines.
> 
> Yes, there would be a runtime penalty, but it should be minimal.  For 
> static protocols it would be a few "sk->sk_family < AF_MAX" checks.  For 
> dynamic protocols they might need to walk a linked list, which in all 
> probability would be very short.

It is not impossible to make this dynamic, you would need to make the table an allocated
object and use proper locking like RCU. Oh, and because it is using GPL, the symbols
would have to be EXPORT_SYMBOL_GPL(), so any dream of proprietary stacks there would
be skating on even thinner ice. 
The lockdep stuff makes it more complicated but not impossible. 

The bigger issue is how would you manage statically assigned id's which
are not visible int headers or kernel source. How would you keep AF_VENDOR_PROTOCOL1 from
not colliding with AF_VENDOR_PROTOCOL2?

Re: why is it not allowed to add a new socket protocol family as an external module?

[Chris Friesen]

On 02/21/2013 10:04 AM, Stephen Hemminger wrote:

> It is not impossible to make this dynamic, you would need to make the table an allocated
> object and use proper locking like RCU. Oh, and because it is using GPL, the symbols
> would have to be EXPORT_SYMBOL_GPL(), so any dream of proprietary stacks there would
> be skating on even thinner ice.
> The lockdep stuff makes it more complicated but not impossible.

I'm not even thinking about closed-source stacks.  In our particular 
case the use-case is highly proprietary (not to mention legacy and 
obscure) but GPL-compliant.

> The bigger issue is how would you manage statically assigned id's which
> are not visible int headers or kernel source. How would you keep AF_VENDOR_PROTOCOL1 from
> not colliding with AF_VENDOR_PROTOCOL2?

I wouldn't expect many people to have multiple dynamic protocols loaded, 
so I'm not sure this would be a big problem.

For a fully generic solution I think we'd need to do dynamic numbering 
and export the mapping somewhere like /sys/class/net or 
/proc/sys/net/core/.  This would makes things a bit non-standard for the 
userspace code, but would allow arbitrary numbers of dynamic protocols 
without collision.

Or maybe we get really crazy and do protocol namespaces. :)

Chris

Re: why is it not allowed to add a new socket protocol family as an external module?

[Eric Dumazet]

On Thu, 2013-02-21 at 09:47 -0600, Chris Friesen wrote:
> On 02/20/2013 07:39 PM, Eric Dumazet wrote:
> > On Wed, 2013-02-20 at 18:44 -0600, Chris Friesen wrote:
> >
> >> Yes, there are a number of tables sized by NPROTO/AF_MAX/PF_MAX (and the
> >> fact that we use all three within the kernel is kind of sad) but there
> >> is no technical reason why we couldn't extend those dynamically if
> >> desired (with a linked list of additional protocols, perhaps).
> >>
> >> Hence my question--is the restriction for an ideological reason or
> >> simply because nobody thought it was worth the effort?
> >>
> >
> > I guess nobody did the preliminary work.
> >
> > lockdep might be the tricky part.
> >
> > net/core/sock.c:197:static const char *const af_family_key_strings[AF_MAX+1] = {
> > net/core/sock.c:211:  "sk_lock-AF_NFC"   , "sk_lock-AF_MAX"
> > net/core/sock.c:213:static const char *const af_family_slock_key_strings[AF_MAX+1] = {
> > net/core/sock.c:227:  "slock-AF_NFC"   , "slock-AF_MAX"
> > net/core/sock.c:229:static const char *const af_family_clock_key_strings[AF_MAX+1] = {
> > net/core/sock.c:243:  "clock-AF_NFC"   , "clock-AF_MAX"
> > net/core/sock.c:250:static struct lock_class_key af_callback_keys[AF_MAX];
> 
> Unless I'm missing something that looks straightforward.
> 
> When registering dynamically the new protocol would need to specify one 
> string, the protocol name.  ("AF_MYPROTOCOL" or something).  The three 
> key strings are derived from that.
> 
> The three lock_class_key structs (corresponding to 
> af_family_keys/af_family_slock_keys/af_callback_keys) can be 
> auto-allocated by the network core at dynamic registration time.

Nope, this was the point I specifically raised but you missed it

Take a look at kernel/lockdep.c, lines 2981-2988

All the other stuff you mention seems pretty obvious.

Re: why is it not allowed to add a new socket protocol family as an external module?

[Chris Friesen]

On 02/21/2013 10:43 AM, Eric Dumazet wrote:
> On Thu, 2013-02-21 at 09:47 -0600, Chris Friesen wrote:

>> The three lock_class_key structs (corresponding to
>> af_family_keys/af_family_slock_keys/af_callback_keys) can be
>> auto-allocated by the network core at dynamic registration time.
>
> Nope, this was the point I specifically raised but you missed it
>
> Take a look at kernel/lockdep.c, lines 2981-2988
>
> All the other stuff you mention seems pretty obvious.

Ah, sorry.  I'm not familiar with the guts of lockdep, so I was not 
aware that the keys needed to be static.

Given that the lockdep code considers module addresses to be okay, it 
should work to make the keys static in the KLM implementing the new 
protocol and pass the addresses in at registration time.

Chris