Understanding execution context of netfilter hooks

Understanding execution context of netfilter hooks

[Stefan Hellkvist]

Hi,

Could anyone help me understand the execution context under which
netfilter hooks are being executed? I played around with some code in
order to learn things and noticed that the code executed differently
in a netfilter hook than in, for instance, the init method of a module
and I fail to understand why that is (possibly due to lack of
understanding of the kernel in general).

I can give a very simplified example. Take the following rediculous
code which reads a few bytes from a file in the file system (yes, a
very unlikely example I know, but the question about writing or
reading files from kernel space is not in my interest right now):

static void
readshadow() {
  struct file *fp;
  char buf[1024];

  fp = filp_open("/etc/shadow", O_RDONLY, 0);
  if (fp != NULL) {
    int retval = kernel_read(fp, 0, buf, 20);
    if (retval != 20) {
      printk("disaster!\n");
    }
    buf[20] = '\0';
    printk("first 20 chars: \"%s\"\n", buf);
    filp_close(fp, 0);
  }
}


The code opens a file, reads a few bytes from it and then closes the
file after having logged the bytes with printk.

If this method is run within for instance the init method of the
module it performs as expected (it reads the file and shows the right
bytes in the log). If it however is called from within a netfilter
hook (registered for instance on NF_INET_LOCAL_OUT), such as this:

static unsigned int
hook(unsigned int hooknum,
     struct sk_buff *skb,
     const struct net_device *in,
     const struct net_device *out,
     int (*okfn) (struct sk_buff *)) {

  readshadow();
  return NF_ACCEPT;
}


Then it will fail with an "Oops: 0000 [#2]" "BUG: unable to handle
kernel NULL pointer dereference at 0000000f" when you trigger it with
some network traffic. The last call in the stack-trace of the OOPS
shows vfs_read (called by kernel_read) being the culprit (possibly
because it's being executed in the wrong context).

So, the question, as mentioned in the beginning. What context is the
hook executing in which causes the code to behave differently from
when it executed in the module init method for instance?

Kindest regards,
Stefan
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Understanding execution context of netfilter hooks

[Jan Engelhardt]

On Friday 2009-07-17 18:50, Stefan Hellkvist wrote:
>
>Could anyone help me understand the execution context under which
>netfilter hooks are being executed?

If I am not mistaken, SIRQ context, including the no-nos that
come with it. Reading files sure is one.

>Then it will fail with an "Oops: 0000 [#2]" "BUG: unable to handle
>kernel NULL pointer dereference at 0000000f" when you trigger it with
>some network traffic. The last call in the stack-trace of the OOPS
>shows vfs_read (called by kernel_read) being the culprit (possibly
>because it's being executed in the wrong context).
>
>So, the question, as mentioned in the beginning. What context is the
>hook executing in which causes the code to behave differently from
>when it executed in the module init method for instance?

module_init either runs as user or kthread context. Your reading
of files would most likely also fail in the latter.

Re: Understanding execution context of netfilter hooks

[Stefan Hellkvist]

On Fri, Jul 17, 2009 at 9:19 PM, Jan Engelhardt<jengelh@medozas.de> wrote:

>>So, the question, as mentioned in the beginning. What context is the
>>hook executing in which causes the code to behave differently from
>>when it executed in the module init method for instance?
>
> module_init either runs as user or kthread context. Your reading
> of files would most likely also fail in the latter.

I see, then this problem is not really netfilter specific I guess. I
guess I have many things to learn here. I suppose exec is also running
in user context when it reads files with kernel_read then.

If anyone knows of any good online documentation or book regarding
these different contexts I would be grateful for the info. Otherwise I
thank you for the feedback.

/Stefan

Re: Understanding execution context of netfilter hooks

[Jan Engelhardt]

>>>So, the question, as mentioned in the beginning. What context is the
>>>hook executing in which causes the code to behave differently from
>>>when it executed in the module init method for instance?
>>
>> module_init either runs as user or kthread context. Your reading
>> of files would most likely also fail in the latter.
>
>I see, then this problem is not really netfilter specific I guess.

Nope. You could move your reading to netif_receive_skb and still get the 
same crash.

>If anyone knows of any good online documentation or book regarding
>these different contexts I would be grateful for the info. Otherwise I
>thank you for the feedback.

Chapters 5-8,10,12 of LDD3 (and perhaps more, I did not bother besides a 
quick grep) talk about it now and again.

Re: Understanding execution context of netfilter hooks

[Stephen Hemminger]

On Fri, 17 Jul 2009 18:50:52 +0200
Stefan Hellkvist <hellkvist@gmail.com> wrote:

> Hi,
> 
> Could anyone help me understand the execution context under which
> netfilter hooks are being executed? I played around with some code in
> order to learn things and noticed that the code executed differently
> in a netfilter hook than in, for instance, the init method of a module
> and I fail to understand why that is (possibly due to lack of
> understanding of the kernel in general).
> 
> I can give a very simplified example. Take the following rediculous
> code which reads a few bytes from a file in the file system (yes, a
> very unlikely example I know, but the question about writing or
> reading files from kernel space is not in my interest right now):
> 
> static void
> readshadow() {
>   struct file *fp;
>   char buf[1024];
> 
>   fp = filp_open("/etc/shadow", O_RDONLY, 0);
>   if (fp != NULL) {
>     int retval = kernel_read(fp, 0, buf, 20);
>     if (retval != 20) {
>       printk("disaster!\n");
>     }
>     buf[20] = '\0';
>     printk("first 20 chars: \"%s\"\n", buf);
>     filp_close(fp, 0);
>   }
> }


One of the repeating mantra's of kernel development is:
 "Don't do file i/o in kernel code."

Your code is wrong not just because kernel i/o can sleep,
but also because there really is not just one namespace,
so what is /etc/shadow!

The right way to do something like this is to read/parse
the file in a utility and pass the necessary data into the
kernel module through other mechanisms (/proc,netlink, debugfs, ...)
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Understanding execution context of netfilter hooks

[Stefan Hellkvist]

On Jul 18, 2009, at 12:57 AM, Stephen Hemminger wrote:

>>
>>     buf[20] = '\0';
>>     printk("first 20 chars: \"%s\"\n", buf);
>>     filp_close(fp, 0);
>>   }
>> }
>
>
> One of the repeating mantra's of kernel development is:
> "Don't do file i/o in kernel code."
>
> Your code is wrong not just because kernel i/o can sleep,
> but also because there really is not just one namespace,
> so what is /etc/shadow!
>
> The right way to do something like this is to read/parse
> the file in a utility and pass the necessary data into the
> kernel module through other mechanisms (/proc,netlink, debugfs, ...)

Yes, I am well aware of the actual "badness" of the code in question.  
I was more curious and wondering why it behaved differently in  
different parts of the kernel but I kind of understand that now. The  
real code I'm working on is doing something similar as to what you  
describe with a utility in user space. I was also using  
netfilter_queue at one stage but for some reason I could not make that  
stable. With high traffic the netfilter_queue just stopped working for  
some reason and not because of buffer overrun (as far as I could tell)  
but due to something else that I never understood (probably some error  
in my code) so I went for something similar but simpler and more  
targeted for my needs.

/Stefan