Is there a way to build a cross reference with kernel file

Is there a way to build a cross reference with kernel file

[Jimmy Pan]

It seems we can use ctags, while, I don't really know how to implement it.
I don't want to use the online cross reference every time...
Thanks.

Is there a way to build a cross reference with kernel file

[Alexandre Courbot]

On Tue, Oct 25, 2011 at 12:00 AM, Jimmy Pan <dspjm1@gmail.com> wrote:
> It seems we can use ctags, while, I don't really know how to implement it.
> I don't want to use the online cross reference every time...

I personally use KDevelop to navigate the kernel. It's not perfect yet
and a little bit tricky to configure correctly, but it's still pretty
efficient and I work on improving it. I wrote about it recently:
http://www.gnurou.org/code/kdevelop-kernel

CTags is also known to work well, actually there is even a 'make tags'
target in the kernel's Makefile.

Hope this helps,
Alex.

Is there a way to build a cross reference with kernel file

[Jeff Haran]

> -----Original Message-----
> From: kernelnewbies-bounces at kernelnewbies.org [mailto:kernelnewbies-
> bounces at kernelnewbies.org] On Behalf Of Alexandre Courbot
> Sent: Monday, October 24, 2011 7:05 PM
> To: Jimmy Pan
> Cc: Kernelnewbies at kernelnewbies.org
> Subject: Re: Is there a way to build a cross reference with kernel
file
> 
> On Tue, Oct 25, 2011 at 12:00 AM, Jimmy Pan <dspjm1@gmail.com> wrote:
> > It seems we can use ctags, while, I don't really know how to
implement it.
> > I don't want to use the online cross reference every time...
> 
> I personally use KDevelop to navigate the kernel. It's not perfect yet
> and a little bit tricky to configure correctly, but it's still pretty
> efficient and I work on improving it. I wrote about it recently:
> http://www.gnurou.org/code/kdevelop-kernel
> 
> CTags is also known to work well, actually there is even a 'make tags'
> target in the kernel's Makefile.
> 
> Hope this helps,
> Alex.

I have found that one of the hardest things to deal with when trying to
understand kernel code (or the code for any big project written in C) is
trying to find out where a given structure field is used. For instance,
say you want to find out where the cb field of a sk_buff structure is
referenced. If I use cscope on my kernel source tree and enter "cb"
under "Find this C symbol:", it provides 2596 references. The vast
majority of those references will have nothing to do with sk_buff
structures because cb is used as a variable or field name all over the
place, usually to mean some sort of pointer to a callback function, but
I'd have to sift through all 2596 of them to find the references to the
"right" cb.

I tried kdevelop to solve this same problem. It doesn't seem to provide
any references to structure field names. Hover the mouse cursor over a
field name in kdevelop and nothing pops up like it would for say a local
or global variable name. At least it didn't when I tried it.

Has anybody found a tool the solves this kind of problem well?
Specifically, given a struct foo containing a field named bar, show me
all of the code that references the bar field of struct foos but not the
bar fields of other structures nor instances of variables that are
themselves named bar.

I realize that with things like arbitrary type casting and the C
pre-processor, doing this kind of thing with 100% accuracy could be
quite difficult. But even if it was only partially successful at
eliminating the false positives that come with tools like cscope, it
could save a lot of time.

Thanks,

Jeff Haran

Is there a way to build a cross reference with kernel file

[Javier Martinez Canillas]

On Sat, Dec 17, 2011 at 1:58 AM, Jeff Haran <jharan@bytemobile.com> wrote:
>> -----Original Message-----
>> From: kernelnewbies-bounces at kernelnewbies.org [mailto:kernelnewbies-
>> bounces at kernelnewbies.org] On Behalf Of Alexandre Courbot
>> Sent: Monday, October 24, 2011 7:05 PM
>> To: Jimmy Pan
>> Cc: Kernelnewbies at kernelnewbies.org
>> Subject: Re: Is there a way to build a cross reference with kernel
> file
>>
>> On Tue, Oct 25, 2011 at 12:00 AM, Jimmy Pan <dspjm1@gmail.com> wrote:
>> > It seems we can use ctags, while, I don't really know how to
> implement it.
>> > I don't want to use the online cross reference every time...
>>
>> I personally use KDevelop to navigate the kernel. It's not perfect yet
>> and a little bit tricky to configure correctly, but it's still pretty
>> efficient and I work on improving it. I wrote about it recently:
>> http://www.gnurou.org/code/kdevelop-kernel
>>
>> CTags is also known to work well, actually there is even a 'make tags'
>> target in the kernel's Makefile.
>>
>> Hope this helps,
>> Alex.
>
> I have found that one of the hardest things to deal with when trying to
> understand kernel code (or the code for any big project written in C) is
> trying to find out where a given structure field is used. For instance,
> say you want to find out where the cb field of a sk_buff structure is
> referenced. If I use cscope on my kernel source tree and enter "cb"
> under "Find this C symbol:", it provides 2596 references. The vast
> majority of those references will have nothing to do with sk_buff
> structures because cb is used as a variable or field name all over the
> place, usually to mean some sort of pointer to a callback function, but
> I'd have to sift through all 2596 of them to find the references to the
> "right" cb.
>
> I tried kdevelop to solve this same problem. It doesn't seem to provide
> any references to structure field names. Hover the mouse cursor over a
> field name in kdevelop and nothing pops up like it would for say a local
> or global variable name. At least it didn't when I tried it.
>
> Has anybody found a tool the solves this kind of problem well?
> Specifically, given a struct foo containing a field named bar, show me
> all of the code that references the bar field of struct foos but not the
> bar fields of other structures nor instances of variables that are
> themselves named bar.
>
> I realize that with things like arbitrary type casting and the C
> pre-processor, doing this kind of thing with 100% accuracy could be
> quite difficult. But even if it was only partially successful at
> eliminating the false positives that come with tools like cscope, it
> could save a lot of time.
>

Well you can always rely on conventions. For the sk_buff example,
generally the kernel uses sk_buff pointers and the variable name is
almost always skb. So git grep skb->cb is a good approach to this.

Also, I think that the first thing one have to learn in the kernel is
the project layout. Where everything is placed and why. I recommend
that the first thing someone getting into kernel dev is getting
familiar with each sub-dir.

One should be able to ask questions such as: Where are the input
drivers? Where is the code for a given filesystem or the support for
an architecture?

So, in your example I would do something like this:

$ git grep "skb->cb" net/

If I want to know the usage of that field in the network core or

$ git grep "skb->cb" drivers/net/

If I want to know how drivers used it.

Again, ETAGS and tools like that are good for pointing where a symbol
is defined, but its usage is a little more trickier. Like you said the
kernel uses _lots_ of functions pointers so I don't think that a tool
that shows me every place a given function is called will be easier
(if possible) to implement.

Hope it helps,

-- 
Javier Mart?nez Canillas
(+34) 682 39 81 69
Barcelona, Spain

Is there a way to build a cross reference with kernel file

[Jeff Haran]

> -----Original Message-----
> From: Javier Martinez Canillas [mailto:martinez.javier at gmail.com]
> Sent: Friday, December 16, 2011 6:04 PM
> To: Jeff Haran
> Cc: Alexandre Courbot; Jimmy Pan; Kernelnewbies at kernelnewbies.org
> Subject: Re: Is there a way to build a cross reference with kernel file
> 
> On Sat, Dec 17, 2011 at 1:58 AM, Jeff Haran <jharan@bytemobile.com>
> wrote:
> >> -----Original Message-----
> >> From: kernelnewbies-bounces at kernelnewbies.org
> [mailto:kernelnewbies-
> >> bounces at kernelnewbies.org] On Behalf Of Alexandre Courbot
> >> Sent: Monday, October 24, 2011 7:05 PM
> >> To: Jimmy Pan
> >> Cc: Kernelnewbies at kernelnewbies.org
> >> Subject: Re: Is there a way to build a cross reference with kernel
> > file
> >>
> >> On Tue, Oct 25, 2011 at 12:00 AM, Jimmy Pan <dspjm1@gmail.com>
> wrote:
> >> > It seems we can use ctags, while, I don't really know how to
> > implement it.
> >> > I don't want to use the online cross reference every time...
> >>
> >> I personally use KDevelop to navigate the kernel. It's not perfect yet
> >> and a little bit tricky to configure correctly, but it's still pretty
> >> efficient and I work on improving it. I wrote about it recently:
> >> http://www.gnurou.org/code/kdevelop-kernel
> >>
> >> CTags is also known to work well, actually there is even a 'make tags'
> >> target in the kernel's Makefile.
> >>
> >> Hope this helps,
> >> Alex.
> >
...
> > Has anybody found a tool the solves this kind of problem well?
> > Specifically, given a struct foo containing a field named bar, show me
> > all of the code that references the bar field of struct foos but not the
> > bar fields of other structures nor instances of variables that are
> > themselves named bar.
> >
> > I realize that with things like arbitrary type casting and the C
> > pre-processor, doing this kind of thing with 100% accuracy could be
> > quite difficult. But even if it was only partially successful at
> > eliminating the false positives that come with tools like cscope, it
> > could save a lot of time.
> >
> 
> Well you can always rely on conventions. For the sk_buff example,
> generally the kernel uses sk_buff pointers and the variable name is
> almost always skb. So git grep skb->cb is a good approach to this.

Problem is one can't count on these conventions being followed. There are pointers to struct sk_buffs that aren't named skb (for example, struct sk_buff *skbuff and struct sk_buff *skb2 appear in several places) so by assuming they are all named skb will mean you miss some that might be important.

... 

> Again, ETAGS and tools like that are good for pointing where a symbol
> is defined, but its usage is a little more trickier. Like you said the
> kernel uses _lots_ of functions pointers so I don't think that a tool
> that shows me every place a given function is called will be easier
> (if possible) to implement.

Yes, that's another problem. And agreed, something like this would likely be difficult to put together given the way C works.

> Hope it helps,
> 
> --
> Javier Mart?nez Canillas
> (+34) 682 39 81 69
> Barcelona, Spain

Well, it helps in the sense that I think it answers my question. No such tool seems to exist.

Thanks,

Jeff Haran

Is there a way to build a cross reference with kernel file

[Dave Hylands]

Hi Jeff,

On Fri, Dec 16, 2011 at 6:24 PM, Jeff Haran <jharan@bytemobile.com> wrote:

> > -----Original Message-----
> > From: Javier Martinez Canillas [mailto:martinez.javier at gmail.com]
> > Sent: Friday, December 16, 2011 6:04 PM
> > To: Jeff Haran
> > Cc: Alexandre Courbot; Jimmy Pan; Kernelnewbies at kernelnewbies.org
> > Subject: Re: Is there a way to build a cross reference with kernel file
> >
> > On Sat, Dec 17, 2011 at 1:58 AM, Jeff Haran <jharan@bytemobile.com>
> > wrote:
> > >> -----Original Message-----
> > >> From: kernelnewbies-bounces at kernelnewbies.org
> > [mailto:kernelnewbies-
> > >> bounces at kernelnewbies.org] On Behalf Of Alexandre Courbot
> > >> Sent: Monday, October 24, 2011 7:05 PM
> > >> To: Jimmy Pan
> > >> Cc: Kernelnewbies at kernelnewbies.org
> > >> Subject: Re: Is there a way to build a cross reference with kernel
> > > file
> > >>
> > >> On Tue, Oct 25, 2011 at 12:00 AM, Jimmy Pan <dspjm1@gmail.com>
> > wrote:
> > >> > It seems we can use ctags, while, I don't really know how to
> > > implement it.
> > >> > I don't want to use the online cross reference every time...
> > >>
> > >> I personally use KDevelop to navigate the kernel. It's not perfect yet
> > >> and a little bit tricky to configure correctly, but it's still pretty
> > >> efficient and I work on improving it. I wrote about it recently:
> > >> http://www.gnurou.org/code/kdevelop-kernel
> > >>
> > >> CTags is also known to work well, actually there is even a 'make tags'
> > >> target in the kernel's Makefile.
> > >>
> > >> Hope this helps,
> > >> Alex.
> > >
> ...
> > > Has anybody found a tool the solves this kind of problem well?
> > > Specifically, given a struct foo containing a field named bar, show me
> > > all of the code that references the bar field of struct foos but not
> the
> > > bar fields of other structures nor instances of variables that are
> > > themselves named bar.
>

It's not free, but I use Visual SlickEdit, and it is capable of doing what
you want.
www.slickedit.com

I tried what you suggested, and clicked on the cb field inside the sk_buff
struct and asked for all references to that. I checked through most of what
it returned and they all seemed to be valid.

-- 
Dave Hylands
Shuswap, BC, Canada
http://www.davehylands.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20111217/dc919370/attachment.html 

Is there a way to build a cross reference with kernel file

[Alexandre Courbot]

On Dec 17, 2011 9:58 AM, "Jeff Haran" <jharan@bytemobile.com> wrote:
> I tried kdevelop to solve this same problem. It doesn't seem to provide
> any references to structure field names. Hover the mouse cursor over a
> field name in kdevelop and nothing pops up like it would for say a local
> or global variable name. At least it didn't when I tried it.

Actually, KDevelop does that, and pretty well even. The problem is that the
source files cannot be correctly parsed as of now because many macro are
not defined. For instance, __KERNEL__ must exist for many core kernel
include files to be parsed correctly, and the header file generated from
your kernel configuration must also be parsed automatically (or you will
miss things like CONFIG_PM and such and some parts of the code that depend
on these macros will appear as dead code). But for other stuff, it really
does a great job at infering the types and only giving you relevant
references.

I am currently writing a couple of patches to improve kernel parsing (e.g.
C99 designated initializers are not supported yet), and also started a
kernel project plugin to solve the macros problem and only parse the files
and drivers that are relevant for a given kernel configuration, but this
will still take some time.

If you are looking for a better grep, you may want to give a try to
coccigrep, part of the coccinelle suite. It supposingly does wonders,
unfortunately I did not manage to compile it and did not look further but
maybe you will have more luck.

Alex.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20111217/0b4d8ae7/attachment.html 

Is there a way to build a cross reference with kernel file

[Mulyadi Santosa]

On Mon, Oct 24, 2011 at 22:00, Jimmy Pan <dspjm1@gmail.com> wrote:
> It seems we can use ctags, while, I don't really know how to implement it.
> I don't want to use the online cross reference every time...
> Thanks.

or you can try "make cscope" and then browse your kernel code using
cscope. Google for it :)


-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

Is there a way to build a cross reference with kernel file

[Javier Martinez Canillas]

Hello Jimmy,

On Mon, Oct 24, 2011 at 5:00 PM, Jimmy Pan <dspjm1@gmail.com> wrote:
> It seems we can use ctags, while, I don't really know how to implement it.
> I don't want to use the online cross reference every time...
> Thanks.
>

Emacs + etags + cscope is for my the best approach to navigate the kernel.

A few months ago I wrote a blog post about this, you could take a look:

http://martinezjavier.blogspot.com/2011/07/emacs-configuration-for-linux-kernel.html

Hope it helps,

-- 
Javier Mart?nez Canillas
(+34) 682 39 81 69
Barcelona, Spain