ELF Notes for kernel image

ELF Notes for kernel image

[Carl Patenaude Poulin]

Hi all,

For my undergraduate project, I am developing a simple
paravirtualized x64 Xen unikernel. I've been hitting some issues
that I hope you can help me with!

I'm trying to figure out how to equip an ELF binary with an ELF note
that marks it as a valid Xen image. I've found the following
resources:

http://wiki.xenproject.org/wiki/X86_Paravirtualised_Memory_Management#Start_Of_Day
http://www.netbsd.org/docs/kernel/elf-notes.html#note-creation
http://xenbits.xen.org/docs/4.3-testing/hypercall/include,public,elfnote.h.html

Unfortunately, they are quite hard to decipher! I've tried to
summarize what little information I've extracted. Can you please
verify that the following is accurate, and maybe help me fill in the
blanks or point me to additional resources?

* A "PT_NOTE section" is any ELF header section which conforms to
the PT_NOTE specification. It is typically named ".note.*" and
marked allocatable.
* A PT_NOTE section is a series of "notes", or key-value pairs, each
formatted as `key_size: int32, value_size: int32, value_type: int32,
key: (int32|int64|asciz), value: (int32|int64|asciz)`.
* The Xen PT_NOTE section is named ".note.Xen", and its keys are
listen (in order) in $XEN_ROOT/xen/include/public/elfnote.h.

My questions:
* Every PT_NOTE entry contains a type field. What does it do, and
how do I decide what goes in it?
* In particular, what are the types of the various entries in the
Xen PT_NOTES (as seen in elfnote.h)?
* After all of this is done, how can I verify that my compiled
binary indeed has a valid PT_NOTE entry for Xen? Does `readelf` have
a switch that's particularly appropriate for this?

Thanks in advance,

Carl Patenaude Poulin
B Eng Software Engineering undergrad
McGill University

Re: ELF Notes for kernel image

[Andrew Cooper]

On 25/10/2015 00:07, Carl Patenaude Poulin wrote:
> Hi all,
>
> For my undergraduate project, I am developing a simple
> paravirtualized x64 Xen unikernel. I've been hitting some issues
> that I hope you can help me with!
>
> I'm trying to figure out how to equip an ELF binary with an ELF note
> that marks it as a valid Xen image. I've found the following
> resources:
>
> http://wiki.xenproject.org/wiki/X86_Paravirtualised_Memory_Management#Start_Of_Day
> http://www.netbsd.org/docs/kernel/elf-notes.html#note-creation
> http://xenbits.xen.org/docs/4.3-testing/hypercall/include,public,elfnote.h.html
>
> Unfortunately, they are quite hard to decipher!

I have to concur.  The Xen elf note documentation is in a poor state.

I have recently made a set of unikernels from scratch for a different
project, and I had to reverse engineer some bits of the elfnote layout
from the domain building code.

>  I've tried to
> summarize what little information I've extracted. Can you please
> verify that the following is accurate, and maybe help me fill in the
> blanks or point me to additional resources?
>
> * A "PT_NOTE section" is any ELF header section which conforms to
> the PT_NOTE specification. It is typically named ".note.*" and
> marked allocatable.
> * A PT_NOTE section is a series of "notes", or key-value pairs, each
> formatted as `key_size: int32, value_size: int32, value_type: int32,
> key: (int32|int64|asciz), value: (int32|int64|asciz)`.

This is not the PT_NOTE specification.

Notes are:
name size, [4 bytes]
desc size, [4 bytes]
type [4 bytes]
name [variable length, typically NUL terminated ASCII string, padded to
4 byte]
desc [variable length, padded to 4 bytes]

For Xen notes, 'name' is always "Xen" to indicate the specifying entity
(i.e. us).
'type' is the integer index from elfnote.h, and is effectively the key.
'desc' is the value associated with 'type'.  The type of data in 'desc'
is specified by 'type'

> * The Xen PT_NOTE section is named ".note.Xen", and its keys are
> listen (in order) in $XEN_ROOT/xen/include/public/elfnote.h.

There is no ordering requirements about notes, not an expectation that
all will be filled in.  Very few are actually required to get a Paravirt
kernel started.

>
> My questions:
> * Every PT_NOTE entry contains a type field. What does it do, and
> how do I decide what goes in it?

I hope these are now clear, given the explanation above.

> * In particular, what are the types of the various entries in the
> Xen PT_NOTES (as seen in elfnote.h)?

An interpretation of the text describing the 'type'.  Mostly NUL
terminated ASCII strings, but not always.

> * After all of this is done, how can I verify that my compiled
> binary indeed has a valid PT_NOTE entry for Xen? Does `readelf` have
> a switch that's particularly appropriate for this?

There is a utility called readnotes (tools/xcutils/readnotes.c in the
Xen source tree) which can parse and print the Xen elf notes, similar to
`readelf`

To get you started,
http://xenbits.xen.org/gitweb/?p=people/andrewcoop/xen-test-framework.git;a=blob;f=arch/x86/boot/head_pv.S;h=078d6b85af704840a0abb700a4d5e40253c21fdf;hb=HEAD
are the minimal set of elfnotes to get a PV guest booted.  (The
hypercall page note isn't strictly needed, but you would silly not to
include it).

Alternatively,
http://lxr.free-electrons.com/source/arch/x86/xen/xen-head.S is the set
of elfnotes used by Linux.

~Andrew

Re: ELF Notes for kernel image

[Carl Patenaude Poulin]

Hi Andrew,

Thank you so much for your help! In particular, I find your
xen-test-framework code to be very instructive.

I have a question about that code, though. I can't find any place
where you're specifying that .note should be loaded as a PT_NOTE ELF
header. Do you do that anywhere? Or is it done implicitly somehow?

On Sun, Oct 25, 2015 at 2:45 PM, Andrew Cooper
<andrew.cooper3@citrix.com> wrote:
> On 25/10/2015 00:07, Carl Patenaude Poulin wrote:
>> Hi all,
>>
>> For my undergraduate project, I am developing a simple
>> paravirtualized x64 Xen unikernel. I've been hitting some issues
>> that I hope you can help me with!
>>
>> I'm trying to figure out how to equip an ELF binary with an ELF note
>> that marks it as a valid Xen image. I've found the following
>> resources:
>>
>> http://wiki.xenproject.org/wiki/X86_Paravirtualised_Memory_Management#Start_Of_Day
>> http://www.netbsd.org/docs/kernel/elf-notes.html#note-creation
>> http://xenbits.xen.org/docs/4.3-testing/hypercall/include,public,elfnote.h.html
>>
>> Unfortunately, they are quite hard to decipher!
>
> I have to concur.  The Xen elf note documentation is in a poor state.
>
> I have recently made a set of unikernels from scratch for a different
> project, and I had to reverse engineer some bits of the elfnote layout
> from the domain building code.
>
>>  I've tried to
>> summarize what little information I've extracted. Can you please
>> verify that the following is accurate, and maybe help me fill in the
>> blanks or point me to additional resources?
>>
>> * A "PT_NOTE section" is any ELF header section which conforms to
>> the PT_NOTE specification. It is typically named ".note.*" and
>> marked allocatable.
>> * A PT_NOTE section is a series of "notes", or key-value pairs, each
>> formatted as `key_size: int32, value_size: int32, value_type: int32,
>> key: (int32|int64|asciz), value: (int32|int64|asciz)`.
>
> This is not the PT_NOTE specification.
>
> Notes are:
> name size, [4 bytes]
> desc size, [4 bytes]
> type [4 bytes]
> name [variable length, typically NUL terminated ASCII string, padded to
> 4 byte]
> desc [variable length, padded to 4 bytes]
>
> For Xen notes, 'name' is always "Xen" to indicate the specifying entity
> (i.e. us).
> 'type' is the integer index from elfnote.h, and is effectively the key.
> 'desc' is the value associated with 'type'.  The type of data in 'desc'
> is specified by 'type'
>
>> * The Xen PT_NOTE section is named ".note.Xen", and its keys are
>> listen (in order) in $XEN_ROOT/xen/include/public/elfnote.h.
>
> There is no ordering requirements about notes, not an expectation that
> all will be filled in.  Very few are actually required to get a Paravirt
> kernel started.
>
>>
>> My questions:
>> * Every PT_NOTE entry contains a type field. What does it do, and
>> how do I decide what goes in it?
>
> I hope these are now clear, given the explanation above.
>
>> * In particular, what are the types of the various entries in the
>> Xen PT_NOTES (as seen in elfnote.h)?
>
> An interpretation of the text describing the 'type'.  Mostly NUL
> terminated ASCII strings, but not always.
>
>> * After all of this is done, how can I verify that my compiled
>> binary indeed has a valid PT_NOTE entry for Xen? Does `readelf` have
>> a switch that's particularly appropriate for this?
>
> There is a utility called readnotes (tools/xcutils/readnotes.c in the
> Xen source tree) which can parse and print the Xen elf notes, similar to
> `readelf`
>
> To get you started,
> http://xenbits.xen.org/gitweb/?p=people/andrewcoop/xen-test-framework.git;a=blob;f=arch/x86/boot/head_pv.S;h=078d6b85af704840a0abb700a4d5e40253c21fdf;hb=HEAD
> are the minimal set of elfnotes to get a PV guest booted.  (The
> hypercall page note isn't strictly needed, but you would silly not to
> include it).
>
> Alternatively,
> http://lxr.free-electrons.com/source/arch/x86/xen/xen-head.S is the set
> of elfnotes used by Linux.
>
> ~Andrew

Re: ELF Notes for kernel image

[Andrew Cooper]

On 27/10/15 16:03, Carl Patenaude Poulin wrote:
> Hi Andrew,
>
> Thank you so much for your help! In particular, I find your
> xen-test-framework code to be very instructive.

It is intended to be so, but I have a few more bits and pieces to
complete before I declare it v1 and formally release it to the public. 
(Looks like that boat has sailed now.)

> I have a question about that code, though. I can't find any place
> where you're specifying that .note should be loaded as a PT_NOTE ELF
> header. Do you do that anywhere? Or is it done implicitly somehow?

The linker file (arch/x86/link.lds.S) places all .note.* sections into
the top level .note section.

~Andrew

Re: ELF Notes for kernel image

[Lars Kurth]

Guys,
given that you "concur [that] the Xen elf note documentation is in a poor state", I was wondering whether we could fix this based on this discussion. 
Lars

> On 27 Oct 2015, at 16:17, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> 
> On 27/10/15 16:03, Carl Patenaude Poulin wrote:
>> Hi Andrew,
>> 
>> Thank you so much for your help! In particular, I find your
>> xen-test-framework code to be very instructive.
> 
> It is intended to be so, but I have a few more bits and pieces to
> complete before I declare it v1 and formally release it to the public. 
> (Looks like that boat has sailed now.)
> 
>> I have a question about that code, though. I can't find any place
>> where you're specifying that .note should be loaded as a PT_NOTE ELF
>> header. Do you do that anywhere? Or is it done implicitly somehow?
> 
> The linker file (arch/x86/link.lds.S) places all .note.* sections into
> the top level .note section.
> 
> ~Andrew
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel