Re: [PATCH 0/10] Sparse linker

["Alexey Zaytsev" <alexey.zaytsev@gmail.com>]

On Thu, Sep 4, 2008 at 5:16 AM, Christopher Li <sparse@chrisli.org> wrote:
> Oops, forget to CC the list.
>
> Chris
>
> ---------- Forwarded message ----------
> From: Chris Li <sparse@chrisli.org>
> Date: Wed, Sep 3, 2008 at 6:08 PM
> Subject: Re: [PATCH 0/10] Sparse linker
> To: alexey.zaytsev@gmail.com
>
>
> On Wed, Sep 3, 2008 at 2:55 PM,  <alexey.zaytsev@gmail.com> wrote:
>
>> more on the subject, I now agree that we should include the
>> intermediate code representation into the object files.
>
> Good.
>
>> for this is a four byte overhead prepended to every
>> serializable structure by the allocation wrapper. Also, you
>
> I would rather not have that 4 byte prepended to every
> structure. Serialize is just one short stage of the life cycle
> of those c structures. Having the permanent extra space
> for just that is unnecessary. That 4 bytes meta data also
> limits what C structure you can work on. All you need
> is being able to map a point into some serialize object
> to keep track which object is tracked and which one is not.
>
> After you serialized the data. The meta data can be drop
> completely. So the price to pay is for every unknown object
> pointer, you need to do a dictionary look up. Only during
> the dumping stage. But that price is actually very small,
> when you dumping objects. You are mostly limit by the disk
> any way. The plus side is: you can work with any objects.
> You don't need to waste extra memory for serialization
> when you are not doing serialization. You can leave the
> object allocation code unchanged.

Thanks for the comment, I will look into this idea. I just
realized that I'm actually unable to serialize stand-alone
arrays right now. The array members should be there,
but not sequentially.

>
>> have to use a macro when declaring a serializable structure
>> (or an array of such) statically. One limitation I was unable
>> to overcome is the inability to work with structures used both
>> stand-alone and embedded into bigger ones. Luckily, we have no
>
> Like the ctype member inside the "struct symbol"?

I still hope I made no mistake that struct ctype is used stand-alone
only as a temporary variable, and is never pointed to from an other
data structure that we would like to serialize.

>
>> list and serializing it [PATCH 07]. The linker needs to dlopen
>
> Do you use the stander share library for dlopen and dynamic linking
> the sparse objects?

If I understand the question right, no. Every "sparse object" .so has a
"struct ptr_list *symbols" entry (in fact, the only non-static entry) that
points to the serialized ptr list of the "struct sold_symbol". The linker
dlopen()'s the .so and hooks to the entry, for every input object file.
After that, it simply calls ptr_list_concat() on the opened symbol lists,
and serializes the resulting combined list. There is of course nothing
wrong if we modify the data obtained from the .so, as it is cow-mmaped.

>
>
>> the input "sparse objects", merge the symbol lists, and
>> serialize the result [PATCH 08]. The generated code compilation
>> is handled by the cgcc, cld and car wrappers [PATCH 09]. To
>> look up symbols in sparse object files, a simple program is
>> included [PATCH 10].
>
> Do you dump your sparse object in ELF format?
Well, I serialize the data into C, and then compile it into .so, if
that was the question. You might want to apply the first patch
and look at the serialization-test output.

>
> Chris
>