xen kernel: build failure

xen kernel: build failure

[Christoph Egger]

Hi,

Compiling the xen kernel fails with:

xen/arch/x86/domain.c: In function 'alloc_domain_struct'
xen/arch/x86/domain.c:191: error: negative width in bit-field '<anonymous>'

Removing the line

BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);

makes xen kernel compile again.


Christoph


-- 
---to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Einsteinring 24, 85689 Dornach b. Muenchen
Geschaeftsfuehrer: Alberto Bozzo, Andrew Bowd
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632

Re: xen kernel: build failure

[Keir Fraser]

On 25/10/2011 12:08, "Christoph Egger" <Christoph.Egger@amd.com> wrote:

> 
> Hi,
> 
> Compiling the xen kernel fails with:
> 
> xen/arch/x86/domain.c: In function 'alloc_domain_struct'
> xen/arch/x86/domain.c:191: error: negative width in bit-field '<anonymous>'

Problem is that struct domain has grown bigger than a page for some reason,
in your build environment.

I can't reproduce this.

> Removing the line
> 
> BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
> 
> makes xen kernel compile again.

But not actually work properly. We only allocate a single page for the
domain struct. If the struct is bigger than a page, you'll get memory
corruption at run time.

 -- Keir

> 
> Christoph
> 

Re: xen kernel: build failure

[Christoph Egger]

On 10/25/11 14:35, Keir Fraser wrote:
> On 25/10/2011 12:08, "Christoph Egger"<Christoph.Egger@amd.com>  wrote:
>
>>
>> Hi,
>>
>> Compiling the xen kernel fails with:
>>
>> xen/arch/x86/domain.c: In function 'alloc_domain_struct'
>> xen/arch/x86/domain.c:191: error: negative width in bit-field '<anonymous>'
>
> Problem is that struct domain has grown bigger than a page for some reason,
> in your build environment.

Oh, local changes caused this. Thanks for the hint.

Christoph


>
> I can't reproduce this.
>
>> Removing the line
>>
>> BUILD_BUG_ON(sizeof(*d)>  PAGE_SIZE);
>>
>> makes xen kernel compile again.
>
> But not actually work properly. We only allocate a single page for the
> domain struct. If the struct is bigger than a page, you'll get memory
> corruption at run time.
>
>   -- Keir
>
>>
>> Christoph
>>
>
>
>


-- 
---to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Einsteinring 24, 85689 Dornach b. Muenchen
Geschaeftsfuehrer: Alberto Bozzo, Andrew Bowd
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632

Re: xen kernel: build failure

[Jean Guyader]

On 25 October 2011 13:35, Keir Fraser <keir.xen@gmail.com> wrote:
> On 25/10/2011 12:08, "Christoph Egger" <Christoph.Egger@amd.com> wrote:
>
>>
>> Hi,
>>
>> Compiling the xen kernel fails with:
>>
>> xen/arch/x86/domain.c: In function 'alloc_domain_struct'
>> xen/arch/x86/domain.c:191: error: negative width in bit-field '<anonymous>'
>
> Problem is that struct domain has grown bigger than a page for some reason,
> in your build environment.
>
> I can't reproduce this.
>
>> Removing the line
>>
>> BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
>>
>> makes xen kernel compile again.
>
> But not actually work properly. We only allocate a single page for the
> domain struct. If the struct is bigger than a page, you'll get memory
> corruption at run time.
>

Is there a reason for that? What would be the recommended to add something
into the struct domain now if we can't make it bigger than a page.

Jean

>  -- Keir
>
>>
>> Christoph
>>
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>

Re: xen kernel: build failure

[Jan Beulich]

>>> On 23.11.11 at 17:31, Jean Guyader <jean.guyader@gmail.com> wrote:
> On 25 October 2011 13:35, Keir Fraser <keir.xen@gmail.com> wrote:
>> On 25/10/2011 12:08, "Christoph Egger" <Christoph.Egger@amd.com> wrote:
>>
>>>
>>> Hi,
>>>
>>> Compiling the xen kernel fails with:
>>>
>>> xen/arch/x86/domain.c: In function 'alloc_domain_struct'
>>> xen/arch/x86/domain.c:191: error: negative width in bit-field '<anonymous>'
>>
>> Problem is that struct domain has grown bigger than a page for some reason,
>> in your build environment.
>>
>> I can't reproduce this.
>>
>>> Removing the line
>>>
>>> BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
>>>
>>> makes xen kernel compile again.
>>
>> But not actually work properly. We only allocate a single page for the
>> domain struct. If the struct is bigger than a page, you'll get memory
>> corruption at run time.
>>
> 
> Is there a reason for that?

Of course there is: Post-boot there shouldn't be any allocations of
order greater than zero. This is a generally advisable thing, given that
Xen can't reclaim memory arbitrarily from domains, and in particular
also necessary for tmem.

> What would be the recommended to add something
> into the struct domain now if we can't make it bigger than a page.

Put a pointer to your data (or larger parts that are already there)
instead of the data itself into struct domain, and allocate it
separately. (You may have seen a patch from Olaf Hering late
yesterday or earlier today that moved the mem_event pieces out
of there for this very reason.)

Re: xen kernel: build failure

[Keir Fraser]

On 23/11/2011 16:31, "Jean Guyader" <jean.guyader@gmail.com> wrote:

>>> Removing the line
>>> 
>>> BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
>>> 
>>> makes xen kernel compile again.
>> 
>> But not actually work properly. We only allocate a single page for the
>> domain struct. If the struct is bigger than a page, you'll get memory
>> corruption at run time.
>> 
> 
> Is there a reason for that? What would be the recommended to add something
> into the struct domain now if we can't make it bigger than a page.

If the thing being added is sufficiently large and self-contained, it could
be allocated separately and a pointer added to the domain struct. Else a
large logical piece of the domain struct needs to be split off. Would have
to check what would be a nice large piece -- arch.{hvm,pv} for example. Jan
might also have an opinion, as he already did some work on shrinking the
domain struct.

 -- Keir

Re: xen kernel: build failure

[Jean Guyader]

On 23/11 04:41, Jan Beulich wrote:
> >>> On 23.11.11 at 17:31, Jean Guyader <jean.guyader@gmail.com> wrote:
> > On 25 October 2011 13:35, Keir Fraser <keir.xen@gmail.com> wrote:
> >> On 25/10/2011 12:08, "Christoph Egger" <Christoph.Egger@amd.com> wrote:
> >>
> >>>
> >>> Hi,
> >>>
> >>> Compiling the xen kernel fails with:
> >>>
> >>> xen/arch/x86/domain.c: In function 'alloc_domain_struct'
> >>> xen/arch/x86/domain.c:191: error: negative width in bit-field '<anonymous>'
> >>
> >> Problem is that struct domain has grown bigger than a page for some reason,
> >> in your build environment.
> >>
> >> I can't reproduce this.
> >>
> >>> Removing the line
> >>>
> >>> BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
> >>>
> >>> makes xen kernel compile again.
> >>
> >> But not actually work properly. We only allocate a single page for the
> >> domain struct. If the struct is bigger than a page, you'll get memory
> >> corruption at run time.
> >>
> > 
> > Is there a reason for that?
> 
> Of course there is: Post-boot there shouldn't be any allocations of
> order greater than zero. This is a generally advisable thing, given that
> Xen can't reclaim memory arbitrarily from domains, and in particular
> also necessary for tmem.
> 
> > What would be the recommended to add something
> > into the struct domain now if we can't make it bigger than a page.
> 
> Put a pointer to your data (or larger parts that are already there)
> instead of the data itself into struct domain, and allocate it
> separately. (You may have seen a patch from Olaf Hering late
> yesterday or earlier today that moved the mem_event pieces out
> of there for this very reason.)
> 
> 

I've printed sizeof (struct domain) on boot with xen-unstable and it's
exactly a PAGE big. That mean that if I want a add a value to the struct
domain (event a pointer) I will have to make some room first.

Jean

Re: xen kernel: build failure

[Jean Guyader]

On 28/11 11:59, Jean Guyader wrote:
> On 23/11 04:41, Jan Beulich wrote:
> > >>> On 23.11.11 at 17:31, Jean Guyader <jean.guyader@gmail.com> wrote:
> > > On 25 October 2011 13:35, Keir Fraser <keir.xen@gmail.com> wrote:
> > >> On 25/10/2011 12:08, "Christoph Egger" <Christoph.Egger@amd.com> wrote:
> > >>
> > >>>
> > >>> Hi,
> > >>>
> > >>> Compiling the xen kernel fails with:
> > >>>
> > >>> xen/arch/x86/domain.c: In function 'alloc_domain_struct'
> > >>> xen/arch/x86/domain.c:191: error: negative width in bit-field '<anonymous>'
> > >>
> > >> Problem is that struct domain has grown bigger than a page for some reason,
> > >> in your build environment.
> > >>
> > >> I can't reproduce this.
> > >>
> > >>> Removing the line
> > >>>
> > >>> BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
> > >>>
> > >>> makes xen kernel compile again.
> > >>
> > >> But not actually work properly. We only allocate a single page for the
> > >> domain struct. If the struct is bigger than a page, you'll get memory
> > >> corruption at run time.
> > >>
> > > 
> > > Is there a reason for that?
> > 
> > Of course there is: Post-boot there shouldn't be any allocations of
> > order greater than zero. This is a generally advisable thing, given that
> > Xen can't reclaim memory arbitrarily from domains, and in particular
> > also necessary for tmem.
> > 
> > > What would be the recommended to add something
> > > into the struct domain now if we can't make it bigger than a page.
> > 
> > Put a pointer to your data (or larger parts that are already there)
> > instead of the data itself into struct domain, and allocate it
> > separately. (You may have seen a patch from Olaf Hering late
> > yesterday or earlier today that moved the mem_event pieces out
> > of there for this very reason.)
> > 
> > 
> 
> I've printed sizeof (struct domain) on boot with xen-unstable and it's
> exactly a PAGE big. That mean that if I want a add a value to the struct
> domain (event a pointer) I will have to make some room first.
> 

I can see two candidates, that could be turned into pointers in arch_domain
for x86:

    /* nestedhvm: translate l2 guest physical to host physical */
    struct p2m_domain *nested_p2m[MAX_NESTEDP2M];
    mm_lock_t nested_p2m_lock;
or
    struct PITState vpit;

Let me know which one I should pick. Maybe I should modify both to save
some space for later.

Jean

Re: xen kernel: build failure

[Jan Beulich]

>>> On 28.11.11 at 13:16, Jean Guyader <jean.guyader@eu.citrix.com> wrote:
> On 28/11 11:59, Jean Guyader wrote:
>> On 23/11 04:41, Jan Beulich wrote:
>> > >>> On 23.11.11 at 17:31, Jean Guyader <jean.guyader@gmail.com> wrote:
>> > > On 25 October 2011 13:35, Keir Fraser <keir.xen@gmail.com> wrote:
>> > >> On 25/10/2011 12:08, "Christoph Egger" <Christoph.Egger@amd.com> wrote:
>> > >>
>> > >>>
>> > >>> Hi,
>> > >>>
>> > >>> Compiling the xen kernel fails with:
>> > >>>
>> > >>> xen/arch/x86/domain.c: In function 'alloc_domain_struct'
>> > >>> xen/arch/x86/domain.c:191: error: negative width in bit-field '<anonymous>'
>> > >>
>> > >> Problem is that struct domain has grown bigger than a page for some 
> reason,
>> > >> in your build environment.
>> > >>
>> > >> I can't reproduce this.
>> > >>
>> > >>> Removing the line
>> > >>>
>> > >>> BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
>> > >>>
>> > >>> makes xen kernel compile again.
>> > >>
>> > >> But not actually work properly. We only allocate a single page for the
>> > >> domain struct. If the struct is bigger than a page, you'll get memory
>> > >> corruption at run time.
>> > >>
>> > > 
>> > > Is there a reason for that?
>> > 
>> > Of course there is: Post-boot there shouldn't be any allocations of
>> > order greater than zero. This is a generally advisable thing, given that
>> > Xen can't reclaim memory arbitrarily from domains, and in particular
>> > also necessary for tmem.
>> > 
>> > > What would be the recommended to add something
>> > > into the struct domain now if we can't make it bigger than a page.
>> > 
>> > Put a pointer to your data (or larger parts that are already there)
>> > instead of the data itself into struct domain, and allocate it
>> > separately. (You may have seen a patch from Olaf Hering late
>> > yesterday or earlier today that moved the mem_event pieces out
>> > of there for this very reason.)
>> > 
>> > 
>> 
>> I've printed sizeof (struct domain) on boot with xen-unstable and it's
>> exactly a PAGE big. That mean that if I want a add a value to the struct
>> domain (event a pointer) I will have to make some room first.
>> 
> 
> I can see two candidates, that could be turned into pointers in arch_domain
> for x86:
> 
>     /* nestedhvm: translate l2 guest physical to host physical */
>     struct p2m_domain *nested_p2m[MAX_NESTEDP2M];
>     mm_lock_t nested_p2m_lock;
> or
>     struct PITState vpit;
> 
> Let me know which one I should pick. Maybe I should modify both to save
> some space for later.

As written earlier, I think you should leverage Olaf Hering's patch to
split out the three "struct mem_event_domain" instances. Other
candidates would be the watchdog related items or the evtchn array
(the latter would need to be converted to a simple pointer anyway
at some point in order to support more than 4096 event channels
per domain).

Jan