Re: [Patch] Scripting engine

[Vladimir Serbinenko <phcoder@gmail.com>]

Marco Gerards wrote:

>
>Can't this be done in the same bison parser?  Or what do you mean?
>  
>
I don't think so. Even the tokenization is very differetn. Ex:
2+3*5 is one token for normal parsing, but five for arithmethic
and yylex knows nothing about what parsing is now. If it would always
issue 5 tokens it would be impossible to difference
2+3*5 and 2 + 3 * 5 And yylex normally must not issue the space -
it would make bison parser too complicated

>I noticed that you have hand written yylex, wouldn't it be a lot
>easier to use flex for this?  In that case you can easily detect
>tokens using regular expressions.  This makes maintaining the
>tokenizer really easy.
>  
>
For bash regular expression is not really needed. There are only 3 types
of tokens:
reserved words (if, ...), normal words (WORD) and command separators (;
and ENTER).
All token except WORD could be easily recognized by simple comparing
(check_for_keyword function). But WORD needs special treatment because
its value
is partially unknown that's why I use superchar structure (it keeps
known parts as well as
necessary information to determine the value of unknown parts)
                                 
                                                                        
                              Vladimir