Re: dependency tee from c parser entities downto token

[Konrad Eisele <eiselekd@gmail.com>]

On 05/05/2012 02:32 AM, Josh Triplett wrote:
> On Fri, May 04, 2012 at 03:30:21PM -0700, Christopher Li wrote:
>> On Fri, May 4, 2012 at 1:53 PM, Konrad Eisele<eiselekd@gmail.com>  wrote:
>>> make C=2:
>>>
>>> original sparse:
>>> real    17m54.997s
>>> user    15m25.181s
>>> sys     2m11.281s
>>>
>>> decpp-sparse from "git clone git://git.code.sf.net/p/decpp/code decpp "
>>> real    18m29.748s
>>> user    16m18.155s
>>> sys     2m13.221s
>>>
>>> But decpp is not written with performance in common cases in mind.
>>> The 2 runs probably also depend on other factors too.
>>> I cant think that 4 bytes extra for each token can have a big impact,
>>> if I would implement it that way (it is not in decpp).
>>
>> The deal breaker is not able to free token list if other program using
>> sparse don't need it.
>
>> From the top of token.h:
>
> /*
>   * Basic tokenization structures. NOTE! Those tokens had better
>   * be pretty small, since we're going to keep them all in memory
>   * indefinitely.
>
> Has that changed?  If so, perhaps the comment needs fixing.  If not, I
> suspect the problem lies elsewhere; perhaps in the extra levels of
> indirection introduced by the patch, rather than in the extra memory
> usage.

The benchmarking of decpp's sparse is not based on the patch I sent,
rather decpp's sparse does a lot more tagging to every token with a lot
of extra data. And without performance in mind.Programmed with the
common cases performance in mind it shouldnt have an impact at all...
I think I understood Christophers suggestion now and migh implement
it that way..
-- Konrad



>
> - Josh Triplett
>