sparse ctags

sparse ctags

[Aneesh Kumar]

Hi,

Is there a document how to use this ? Does this work on multiple files
? Looking at the source i see it opening tags in truncate mode

-aneesh

Re: sparse ctags

[Josh Triplett]

On Tue, 2007-06-26 at 14:15 +0530, Aneesh Kumar wrote:
> Is there a document how to use this ? Does this work on multiple files
> ? Looking at the source i see it opening tags in truncate mode

No documentation, unfortunately.  The Sparse ctags doesn't take any
options.  It should work on multiple files by simply giving all the
files on one ctags command line, with something like:
ctags $(find -name '*.c')

However, it seems to have some problems at the moment running on even
*one* file:

$ ./ctags ctags.c
unknown symbol const namespace:2 type:17

$ ./ctags validation/context.c
unknown symbol const namespace:2 type:17

type 17 refers to SYM_KEYWORD, which seems consistent with "unknown
symbol const", though I don't know why ctags would look at const as a
symbol.

I've CCed Christopher Li, the author of the Sparse ctags implementation.
Chris, can you look into this problem and suggest a possible solution?

- Josh Triplett

Re: sparse ctags

[Josh Triplett]

[-- Attachment #1.1: Type: text/plain, Size: 3202 bytes --]

Josh Triplett wrote:
> On Tue, 2007-06-26 at 14:15 +0530, Aneesh Kumar wrote:
>> Is there a document how to use this ? Does this work on multiple files
>> ? Looking at the source i see it opening tags in truncate mode
> 
> No documentation, unfortunately.  The Sparse ctags doesn't take any
> options.  It should work on multiple files by simply giving all the
> files on one ctags command line, with something like:
> ctags $(find -name '*.c')
> 
> However, it seems to have some problems at the moment running on even
> *one* file:
> 
> $ ./ctags ctags.c
> unknown symbol const namespace:2 type:17
> 
> $ ./ctags validation/context.c
> unknown symbol const namespace:2 type:17
> 
> type 17 refers to SYM_KEYWORD, which seems consistent with "unknown
> symbol const", though I don't know why ctags would look at const as a
> symbol.

Looks like some enums grew new values and ctags didn't stay up to date.  I
think this patch fixes that particular problem:

diff --git a/ctags.c b/ctags.c
index f5b8fc7..63f59a5 100644
--- a/ctags.c
+++ b/ctags.c
@@ -111,6 +111,8 @@ static void examine_symbol(struct symbol *sym)
 		return;
 	if (sym->ident && sym->ident->reserved)
 		return;
+	if (sym->type == SYM_KEYWORD)
+		return;
 
 	add_tag(sym);
 	base = sym->ctype.base_type;
@@ -158,6 +160,8 @@ static void examine_namespace(struct symbol *sym)
 		return;
 
 	switch(sym->namespace) {
+	case NS_KEYWORD:
+		return;
 	case NS_LABEL:
 		sym->kind = 'l';
 		break;

With that patch, ctags successfully ran on individual files.  I then observed
some unusual symbols in the tags files:

add_dirafter    preprocessor    0;"     ^@      file:
add_include     preprocessor    0;"     ^@      file:
add_isystem     preprocessor    0;"     ^@      file:
define  preprocessor    0;"     ^@      file:
elif    preprocessor    0;"     ^@      file:
endif   preprocessor    0;"     ^@      file:
error   preprocessor    0;"     ^@      file:
ifdef   preprocessor    0;"     ^@      file:
ifndef  preprocessor    0;"     ^@      file:
include preprocessor    0;"     ^@      file:
include_next    preprocessor    0;"     ^@      file:
line    preprocessor    0;"     ^@      file:
nostdinc        preprocessor    0;"     ^@      file:
pragma  preprocessor    0;"     ^@      file:
split_include   preprocessor    0;"     ^@      file:
strong_define   preprocessor    0;"     ^@      file:
strong_undef    preprocessor    0;"     ^@      file:
undef   preprocessor    0;"     ^@      file:
warning preprocessor    0;"     ^@      file:
weak_define     preprocessor    0;"     ^@      file:

^@ here signifies a null.

After some further investigation, I think ctags really shouldn't look at
SYM_PREPROCESSOR or NS_PREPROCESSOR either.  I came up with the attached
patch, which I will apply soon unless someone shouts.  Chris?

This still doesn't fix running ctags on multiple files; ctags seems to get
confused in much the same way sparse does when run on multiple files: it spews
a pile of redefinition errors.

After we get ctags fixed up, clearly it needs some tests in the test suite so
this doesn't happen again.

- Josh Triplett

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0001-ctags-Handle-some-new-namespaces-and-symbol-types.patch --]
[-- Type: text/x-patch; name="0001-ctags-Handle-some-new-namespaces-and-symbol-types.patch", Size: 1164 bytes --]

From 3624a543e3884e2faa9980a2ecb0c10cda646671 Mon Sep 17 00:00:00 2001
From: Josh Triplett <josh@freedesktop.org>
Date: Mon, 6 Aug 2007 20:37:25 -0700
Subject: [PATCH] ctags: Handle some new namespaces and symbol types.

ctags didn't handle SYM_KEYWORD, SYM_PROCESSOR, or NS_KEYWORD, and didn't
handle NS_PREPROCESSOR correctly.

Signed-off-by: Josh Triplett <josh@freedesktop.org>
---
 ctags.c |    7 +++++--
 1 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/ctags.c b/ctags.c
index f5b8fc7..7e129a6 100644
--- a/ctags.c
+++ b/ctags.c
@@ -111,6 +111,8 @@ static void examine_symbol(struct symbol *sym)
 		return;
 	if (sym->ident && sym->ident->reserved)
 		return;
+	if (sym->type == SYM_KEYWORD || sym->type == SYM_PREPROCESSOR)
+		return;
 
 	add_tag(sym);
 	base = sym->ctype.base_type;
@@ -158,11 +160,12 @@ static void examine_namespace(struct symbol *sym)
 		return;
 
 	switch(sym->namespace) {
+	case NS_KEYWORD:
+	case NS_PREPROCESSOR:
+		return;
 	case NS_LABEL:
 		sym->kind = 'l';
 		break;
-	case NS_PREPROCESSOR:
-		break;
 	case NS_MACRO:
 	case NS_UNDEF:
 		sym->kind = 'd';
-- 
1.5.2.4


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 252 bytes --]

Re: sparse ctags

[Christopher Li]

On 8/6/07, Josh Triplett <josh@freedesktop.org> wrote:
>
> Looks like some enums grew new values and ctags didn't stay up to date.  I
> think this patch fixes that particular problem:
>
> diff --git a/ctags.c b/ctags.c
> index f5b8fc7..63f59a5 100644
> --- a/ctags.c
> +++ b/ctags.c
> @@ -111,6 +111,8 @@ static void examine_symbol(struct symbol *sym)
>                 return;
>         if (sym->ident && sym->ident->reserved)
>                 return;
> +       if (sym->type == SYM_KEYWORD)
> +               return;
>
>         add_tag(sym);
>         base = sym->ctype.base_type;
> @@ -158,6 +160,8 @@ static void examine_namespace(struct symbol *sym)
>                 return;
>
>         switch(sym->namespace) {
> +       case NS_KEYWORD:
> +               return;
>         case NS_LABEL:
>                 sym->kind = 'l';
>                 break;
>

That looks good to me.

> After some further investigation, I think ctags really shouldn't look at
> SYM_PREPROCESSOR or NS_PREPROCESSOR either.  I came up with the attached
> patch, which I will apply soon unless someone shouts.  Chris?

I think SYM_PREPROCESSOR is also used by macros with NS_MACRO.
You just need to strip out the NS_PREPROCESSOR. I guess you try to
skip the symbol with NS_SYMBOL: and SYM_PREPROCESSOR at the same
time.

I am not sure about the test for "sym->type == SYM_KEYWORD",
do you see some case trigger that?

Sorry, I just find out that I get drop out from the sparse mail list
for a while and miss
a lot of interesting discussions. Blame my mail server.

>
> This still doesn't fix running ctags on multiple files; ctags seems to get
> confused in much the same way sparse does when run on multiple files: it spews
> a pile of redefinition errors.

Should make sparse able to clean up itself. I will see what I can do.

Chris