Xen 4.6.0-rc1 build with lock_profile=y crash_debug=y, frame_pointer=y and domain.c:241: error: negative width in bit-field ‘<anonymous>’

Xen 4.6.0-rc1 build with lock_profile=y crash_debug=y, frame_pointer=y and domain.c:241: error: negative width in bit-field ‘<anonymous>’

[Konrad Rzeszutek Wilk]

I am troubleshooting an locking issue and figured I would enable extra options.

But now I am hitting this build issue:

domain.c:241: error: negative width in bit-field ‘<anonymous>’

Which is:

 229 struct domain *alloc_domain_struct(void)
 230 {
 231     struct domain *d;
...
 241     BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);

Thoughts?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.6.0-rc1 build with lock_profile=y crash_debug=y, frame_pointer=y and domain.c:241: error: negative width in bit-field ‘<anonymous>’

[Andrew Cooper]

On 25/08/15 17:43, Konrad Rzeszutek Wilk wrote:
> I am troubleshooting an locking issue and figured I would enable extra options.
>
> But now I am hitting this build issue:
>
> domain.c:241: error: negative width in bit-field ‘<anonymous>’
>
> Which is:
>
>  229 struct domain *alloc_domain_struct(void)
>  230 {
>  231     struct domain *d;
> ...
>  241     BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
>
> Thoughts?

That is to catch the case where sizeof struct domain exceeds PAGE_SIZE.

The logic behind this was to prevent needing order 1 allocations for
domains or vcpus, and therefore allocation failures in heavily memory
fragmented situations.

It means we will probably need to find some other areas of struct domain
to shrink, or move out into a separate xmalloc().

For straight debugging, it is acceptable to drop the BUILD_BUG_ON().  If
we can't drop the size of struct domain, we might want to move to using
vmalloc() instead.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.6.0-rc1 build with lock_profile=y crash_debug=y, frame_pointer=y and domain.c:241: error: negative width in bit-field ‘<anonymous>’

[Konrad Rzeszutek Wilk]

On Tue, Aug 25, 2015 at 05:48:58PM +0100, Andrew Cooper wrote:
> On 25/08/15 17:43, Konrad Rzeszutek Wilk wrote:
> > I am troubleshooting an locking issue and figured I would enable extra options.
> >
> > But now I am hitting this build issue:
> >
> > domain.c:241: error: negative width in bit-field ‘<anonymous>’
> >
> > Which is:
> >
> >  229 struct domain *alloc_domain_struct(void)
> >  230 {
> >  231     struct domain *d;
> > ...
> >  241     BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
> >
> > Thoughts?
> 
> That is to catch the case where sizeof struct domain exceeds PAGE_SIZE.
> 
> The logic behind this was to prevent needing order 1 allocations for
> domains or vcpus, and therefore allocation failures in heavily memory
> fragmented situations.
> 
> It means we will probably need to find some other areas of struct domain
> to shrink, or move out into a separate xmalloc().
> 
> For straight debugging, it is acceptable to drop the BUILD_BUG_ON().  If
> we can't drop the size of struct domain, we might want to move to using
> vmalloc() instead.

It is big. pahole says:

   /* size: 4352, cachelines: 68, members: 74 */ 
    /* sum members: 4238, holes: 9, sum holes: 34 */ 
    /* padding: 80 */ 
    /* paddings: 1, sum paddings: 4 */

Would this patch be OK then:
diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index 045f6ff..cc9ce0b 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -238,7 +238,9 @@ struct domain *alloc_domain_struct(void)
     if ( unlikely(!bits) )
          bits = _domain_struct_bits();
 
+#ifndef LOCK_PROFILE
     BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
+#endif
     d = alloc_xenheap_pages(0, MEMF_bits(bits));
     if ( d != NULL )
         clear_page(d);

(not compile tested nor runtime tested)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.6.0-rc1 build with lock_profile=y crash_debug=y, frame_pointer=y and domain.c:241: error: negative width in bit-field ‘<anonymous>’

[Andrew Cooper]

On 25/08/15 18:09, Konrad Rzeszutek Wilk wrote:
> On Tue, Aug 25, 2015 at 05:48:58PM +0100, Andrew Cooper wrote:
>> On 25/08/15 17:43, Konrad Rzeszutek Wilk wrote:
>>> I am troubleshooting an locking issue and figured I would enable extra options.
>>>
>>> But now I am hitting this build issue:
>>>
>>> domain.c:241: error: negative width in bit-field ‘<anonymous>’
>>>
>>> Which is:
>>>
>>>  229 struct domain *alloc_domain_struct(void)
>>>  230 {
>>>  231     struct domain *d;
>>> ...
>>>  241     BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
>>>
>>> Thoughts?
>> That is to catch the case where sizeof struct domain exceeds PAGE_SIZE.
>>
>> The logic behind this was to prevent needing order 1 allocations for
>> domains or vcpus, and therefore allocation failures in heavily memory
>> fragmented situations.
>>
>> It means we will probably need to find some other areas of struct domain
>> to shrink, or move out into a separate xmalloc().
>>
>> For straight debugging, it is acceptable to drop the BUILD_BUG_ON().  If
>> we can't drop the size of struct domain, we might want to move to using
>> vmalloc() instead.
> It is big. pahole says:
>
>    /* size: 4352, cachelines: 68, members: 74 */ 
>     /* sum members: 4238, holes: 9, sum holes: 34 */ 
>     /* padding: 80 */ 
>     /* paddings: 1, sum paddings: 4 */

So by rearranging to reduce padding, it would easily fit ;p

>
> Would this patch be OK then:
> diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
> index 045f6ff..cc9ce0b 100644
> --- a/xen/arch/x86/domain.c
> +++ b/xen/arch/x86/domain.c
> @@ -238,7 +238,9 @@ struct domain *alloc_domain_struct(void)
>      if ( unlikely(!bits) )
>           bits = _domain_struct_bits();
>  
> +#ifndef LOCK_PROFILE
>      BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
> +#endif
>      d = alloc_xenheap_pages(0, MEMF_bits(bits));
>      if ( d != NULL )
>          clear_page(d);
>
> (not compile tested nor runtime tested)

Either remove it locally for debugging, or use something like

#if sizeof(*d) <= PAGE_SIZE
... alloc_xenheap_pages(0, MEMF_bits(bits));
#else
... vzalloc(sizeof(*d), ...);
#endif

Although this would require passing memflags to the v?alloc()
infrastructure.

Jan: While reading the code, I note that the bits restriction is not
required if CONFIG_BIGMEM, and should probably become conditional.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.6.0-rc1 build with lock_profile=y crash_debug=y, frame_pointer=y and domain.c:241: error: negative width in bit-field ‘<anonymous>’

[Konrad Rzeszutek Wilk]

On Tue, Aug 25, 2015 at 06:41:06PM +0100, Andrew Cooper wrote:
> On 25/08/15 18:09, Konrad Rzeszutek Wilk wrote:
> > On Tue, Aug 25, 2015 at 05:48:58PM +0100, Andrew Cooper wrote:
> >> On 25/08/15 17:43, Konrad Rzeszutek Wilk wrote:
> >>> I am troubleshooting an locking issue and figured I would enable extra options.
> >>>
> >>> But now I am hitting this build issue:
> >>>
> >>> domain.c:241: error: negative width in bit-field ‘<anonymous>’
> >>>
> >>> Which is:
> >>>
> >>>  229 struct domain *alloc_domain_struct(void)
> >>>  230 {
> >>>  231     struct domain *d;
> >>> ...
> >>>  241     BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
> >>>
> >>> Thoughts?
> >> That is to catch the case where sizeof struct domain exceeds PAGE_SIZE.
> >>
> >> The logic behind this was to prevent needing order 1 allocations for
> >> domains or vcpus, and therefore allocation failures in heavily memory
> >> fragmented situations.
> >>
> >> It means we will probably need to find some other areas of struct domain
> >> to shrink, or move out into a separate xmalloc().
> >>
> >> For straight debugging, it is acceptable to drop the BUILD_BUG_ON().  If
> >> we can't drop the size of struct domain, we might want to move to using
> >> vmalloc() instead.
> > It is big. pahole says:
> >
> >    /* size: 4352, cachelines: 68, members: 74 */ 
> >     /* sum members: 4238, holes: 9, sum holes: 34 */ 
> >     /* padding: 80 */ 
> >     /* paddings: 1, sum paddings: 4 */
> 
> So by rearranging to reduce padding, it would easily fit ;p
> 
> >
> > Would this patch be OK then:
> > diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
> > index 045f6ff..cc9ce0b 100644
> > --- a/xen/arch/x86/domain.c
> > +++ b/xen/arch/x86/domain.c
> > @@ -238,7 +238,9 @@ struct domain *alloc_domain_struct(void)
> >      if ( unlikely(!bits) )
> >           bits = _domain_struct_bits();
> >  
> > +#ifndef LOCK_PROFILE
> >      BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
> > +#endif
> >      d = alloc_xenheap_pages(0, MEMF_bits(bits));
> >      if ( d != NULL )
> >          clear_page(d);
> >
> > (not compile tested nor runtime tested)
> 
> Either remove it locally for debugging, or use something like
> 
> #if sizeof(*d) <= PAGE_SIZE
> ... alloc_xenheap_pages(0, MEMF_bits(bits));
> #else
> ... vzalloc(sizeof(*d), ...);
> #endif
> 
> Although this would require passing memflags to the v?alloc()
> infrastructure.


With
commit fd280f57975d813cbacb3e5d31843ec8bd5b52e7
Author: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Date:   Tue Aug 25 16:50:03 2015 -0400

    lock: Try build
    Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index 045f6ff..a36a5d8 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -238,8 +238,12 @@ struct domain *alloc_domain_struct(void)
     if ( unlikely(!bits) )
          bits = _domain_struct_bits();
 
+#ifndef LOCK_PROFILE
     BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
     d = alloc_xenheap_pages(0, MEMF_bits(bits));
+#else
+    d = vzalloc(sizeof(*d));
+#endif
     if ( d != NULL )
         clear_page(d);
     return d;
@@ -248,7 +252,11 @@ struct domain *alloc_domain_struct(void)
 void free_domain_struct(struct domain *d)
 {
     lock_profile_deregister_struct(LOCKPROF_TYPE_PERDOM, d);
+#ifndef LOCK_PROFILE
     free_xenheap_page(d);
+#else
+    vfree(d);
+#endif
 }
 
 struct vcpu *alloc_vcpu_struct(void)

it blows up later:

(XEN) Xen version 4.6.0-rc (konrad@(none)) (gcc (GCC) 4.4.4 20100503 (Red Hat 4.4.4-2)) debug=y Tue Aug 25 16:51:ign-buses,verbose com1=115200,8n1,pci console=com1 tmem=1 tmem_compress=1 tmem_dedup=1 sync_console dom0_max_vcpus=1 cpufreq=xen:performance dom0_mem=max:2G apic=debug loglvl=all guest_loglvl=all
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN)  Found 0 MBR signatures
(XEN)  Found 2 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 000000000009a800 (usable)
(XEN)  000000000009a800 - 00000000000a0000 (reserved)
(XEN)  00000000000e0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 0000000020000000 (usable)
(XEN)  0000000020000000 - 0000000020200000 (reserved)
(XEN)  0000000020200000 - 0000000040000000 (usable)
(XEN)  0000000040000000 - 0000000040200000 (reserved)
(XEN)  0000000040200000 - 000000009e856000 (usable)
(XEN)  000000009e856000 - 000000009e85f000 (ACPI data)
(XEN)  000000009e85f000 - 000000009e8aa000 (ACPI NVS)
(XEN)  000000009e8aa000 - 000000009e8b2000 (usable)
(XEN)  000000009e8b2000 - 000000009e9a5000 (reserved)
(XEN)  000000009e9a5000 - 000000009e9a7000 (usable)
(XEN)  000000009e9a7000 - 000000009ebc6000 (reserved)
(XEN)  000000009ebc6000 - 000000009ebc7000 (usable)
(XEN)  000000009ebc7000 - 000000009ebd7000 (reserved)
(XEN)  000000009ebd7000 - 000000009ebf5000 (ACPI NVS)
(XEN)  000000009ebf5000 - 000000009ec19000 (reserved)
(XEN)  000000009ec19000 - 000000009ec5c000 (ACPI NVS)
(XEN)  000000009ec5c000 - 000000009ee7c000 (reserved)
(XEN)  000000009ee7c000 - 000000009f000000 (usable)
(XEN)  000000009f800000 - 00000000bfa00000 (reserved)
(XEN)  00000000fed1c000 - 00000000fed40000 (reserved)
(XEN)  00000000ff000000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 000000043e600000 (usable)
(XEN) ACPI: RSDP 000F0450, 0024 (r2  INTEL)
(XEN) ACPI: XSDT 9E856070, 0064 (r1 INTEL  DQ67SW    1072009 AMI     10013)
(XEN) ACPI: FACP 9E85DBC0, 00F4 (r4 INTEL  DQ67SW    1072009 AMI     10013)
(XEN) ACPI: DSDT 9E856168, 7A54 (r2 INTEL  DQ67SW         16 INTL 20051117)
(XEN) ACPI: FACS 9EBDBF80, 0040
(XEN) ACPI: APIC 9E85DCB8, 0072 (r3 INTEL  DQ67SW    1072009 AMI     10013)
(XEN) ACPI: TCPA 9E85DD30, 0032 (r2 INTEL  DQ67SW          1 MSFT  1000013)
(XEN) ACPI: SSDT 9E85DD68, 0102 (r1 INTEL  DQ67SW          1 MSFT  3000001)
(XEN) ACPI: MCFG 9E85DE70, 003C (r1 INTEL  DQ67SW    1072009 MSFT       97)
(XEN) ACPI: HPET 9E85DEB0, 0038 (r1 INTEL  DQ67SW    1072009 AMI.        4)
(XEN) ACPI: ASF! 9E85DEE8, 00A0 (r32 INTEL  DQ67SW          1 TFSM    F4240)
(XEN) ACPI: DMAR 9E85DF88, 00E8 (r1 INTEL  DQ67SW          1 INTL        1)
(XEN) System RAM: 15819MB (16199164kB)
(XEN) No NUMA configuration found
(XEN) Faking a node at 0000000000000000-000000043e600000
(XEN) Domain heap initialised
(XEN) Allocated console ring of 1048576 KiB.
(XEN) found SMP MP-table at 000fda50
(XEN) DMI 2.6 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x4_id[0x01] lapic_id[0x00] enabled)
(XEN) Processor #0 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
(XEN) Processor #2 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled)
(XEN) Processor #4 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled)
(XEN) Processor #6 6:10 APIC version 21
(XEN) ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000
(XEN) ERST table was not found
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 4 CPUs (0 hotplug CPUs)
(XEN) NR_CPUS:384 nr_cpumask_bits:64
(XEN) IRQ limits: 24 GSI, 760 MSI/MSI-X
(XEN) Switched to APIC driver x2apic_cluster.
(XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
(XEN) mce_intel.c:735: MCA Capability: BCAST 1 SER 0 CMCI 1 firstbank 0 extended MCE MSR 0
(XEN) Intel machine check reporting enabled
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 3292.563 MHz processor.
(XEN) Assertion '(page->count_info & ~PGC_xen_heap) == 0' failed at mm.c:449
(XEN) ----CPU:    0
(XEN) RIP:    e008:[<ffff82d080175336>] share_xen_page_with_guest+0x178/0x20f
(XEN) RFLAGS: 0000000000010082   CONTEXT: hypervisor
(XEN) rax: 8000000000000001   rbx: ffff82c00020b000   rcx: 0000000000000001
(XEN) rdx: bfffffffffffffff   rsi: ffff82d080353218   rdi: 000000007c9455a3
(XEN) rbp: ffff82d0802f7de8   rsp: ffff82d0802f7dc8   r8:  ffff82e000000000
(XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
(XEN) r12: ffff82e000001340   r13: ffff82c00020b030   r14: 0000000000000000
(XEN) r15: 0000000000020000   cr0: 000000008005003b   cr4: 00000000000406e0
(XEN) cr3: 000000009e6a0000   cr2: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
(XEN) Xen stack trace from rsp=ffff82d0802f7dc8:
(XEN)    000000000000009a 0000000000000003 0000000000000100 ffff82d080330c60
(XEN)    ffff82d0802f7e28 ffff82d0802c2a23 ffff83042f6c0eb0 ffff82d0802f0000
(XEN)    ffff830000073fb0 ffff83042f6c0eb0 ffff82d08028eca8 ffff82d080353420
(XEN)    ffff82d0802f7f08 ffff82d0802ca10f 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 ffffffd0802f0000 ffff82d08026777a
(XEN)    00000000012dd000 0000000000000000 00000000ffffffff 0000000000100000
(XEN)    0000000300000000 ffff830000000006 0000000000253580 00000004ffffffff
(XEN)    ffff830000073e80 ffff82d080353420 0000000800000000 000000010000006e
(XEN)    0000000000000003 00000000000002f8 0000000000000000 0000000000000000
(XEN)    000000000000180a 00000000010ff9c0 0000000000000000 00000000016d5018
(XEN)    0000000000000000 ffff82d080100073 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    ffff83009e9a5000 0000000000000000 0000000000000000
(XEN) Xen call trace:
(XEN)    [<ffff82d080175336>] share_xen_page_with_guest+0x178/0x20f
(XEN)    [<ffff82d0802c2a23>] arch_init_memory+0x2a2/0x55f
(XEN)    [<ffff82d0802ca10f>] __start_xen+0x4aa4/0x509e
(XEN)    [<ffff82d080100073>] __high_start+0x53/0x58
(XEN) 
(XEN) 
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Assertion '(page->count_info & ~PGC_xen_heap) == 0' failed at mm.c:449
(XEN) ****************************************
(XEN) 
(XEN) Reboot in five seconds...
(XEN) Resetting with ACPI MEMORY or I/O RESET_REG.

telnet> Connection closed.
[Connecting to system 31 ]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.6.0-rc1 build with lock_profile=y crash_debug=y, frame_pointer=y and domain.c:241: error: negative width in bit-field ‘<anonymous>’

[Konrad Rzeszutek Wilk]

On Tue, Aug 25, 2015 at 03:52:15PM -0400, Konrad Rzeszutek Wilk wrote:
> On Tue, Aug 25, 2015 at 06:41:06PM +0100, Andrew Cooper wrote:
> > On 25/08/15 18:09, Konrad Rzeszutek Wilk wrote:
> > > On Tue, Aug 25, 2015 at 05:48:58PM +0100, Andrew Cooper wrote:
> > >> On 25/08/15 17:43, Konrad Rzeszutek Wilk wrote:
> > >>> I am troubleshooting an locking issue and figured I would enable extra options.
> > >>>
> > >>> But now I am hitting this build issue:
> > >>>
> > >>> domain.c:241: error: negative width in bit-field ‘<anonymous>’
> > >>>
> > >>> Which is:
> > >>>
> > >>>  229 struct domain *alloc_domain_struct(void)
> > >>>  230 {
> > >>>  231     struct domain *d;
> > >>> ...
> > >>>  241     BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
> > >>>
> > >>> Thoughts?
> > >> That is to catch the case where sizeof struct domain exceeds PAGE_SIZE.
> > >>
> > >> The logic behind this was to prevent needing order 1 allocations for
> > >> domains or vcpus, and therefore allocation failures in heavily memory
> > >> fragmented situations.
> > >>
> > >> It means we will probably need to find some other areas of struct domain
> > >> to shrink, or move out into a separate xmalloc().
> > >>
> > >> For straight debugging, it is acceptable to drop the BUILD_BUG_ON().  If
> > >> we can't drop the size of struct domain, we might want to move to using
> > >> vmalloc() instead.
> > > It is big. pahole says:
> > >
> > >    /* size: 4352, cachelines: 68, members: 74 */ 
> > >     /* sum members: 4238, holes: 9, sum holes: 34 */ 
> > >     /* padding: 80 */ 
> > >     /* paddings: 1, sum paddings: 4 */
> > 
> > So by rearranging to reduce padding, it would easily fit ;p
> > 
> > >
> > > Would this patch be OK then:
> > > diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
> > > index 045f6ff..cc9ce0b 100644
> > > --- a/xen/arch/x86/domain.c
> > > +++ b/xen/arch/x86/domain.c
> > > @@ -238,7 +238,9 @@ struct domain *alloc_domain_struct(void)
> > >      if ( unlikely(!bits) )
> > >           bits = _domain_struct_bits();
> > >  
> > > +#ifndef LOCK_PROFILE
> > >      BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
> > > +#endif
> > >      d = alloc_xenheap_pages(0, MEMF_bits(bits));
> > >      if ( d != NULL )
> > >          clear_page(d);
> > >
> > > (not compile tested nor runtime tested)
> > 
> > Either remove it locally for debugging, or use something like

I forgot to mention that by removing it locally Xen ends up halting at:

(XEN) Detected 3292.657 MHz processor.

..

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.6.0-rc1 build with lock_profile=y crash_debug=y, frame_pointer=y and domain.c:241: error: negative width in bit-field ‘<anonymous>’

[Jan Beulich]

>>> On 25.08.15 at 21:54, <konrad.wilk@oracle.com> wrote:
> On Tue, Aug 25, 2015 at 03:52:15PM -0400, Konrad Rzeszutek Wilk wrote:
>> On Tue, Aug 25, 2015 at 06:41:06PM +0100, Andrew Cooper wrote:
>> > On 25/08/15 18:09, Konrad Rzeszutek Wilk wrote:
>> > > --- a/xen/arch/x86/domain.c
>> > > +++ b/xen/arch/x86/domain.c
>> > > @@ -238,7 +238,9 @@ struct domain *alloc_domain_struct(void)
>> > >      if ( unlikely(!bits) )
>> > >           bits = _domain_struct_bits();
>> > >  
>> > > +#ifndef LOCK_PROFILE
>> > >      BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
>> > > +#endif
>> > >      d = alloc_xenheap_pages(0, MEMF_bits(bits));
>> > >      if ( d != NULL )
>> > >          clear_page(d);
>> > >
>> > > (not compile tested nor runtime tested)
>> > 
>> > Either remove it locally for debugging, or use something like
> 
> I forgot to mention that by removing it locally Xen ends up halting at:
> 
> (XEN) Detected 3292.657 MHz processor.

Sure - as Andrew said on irc, neither simply removing it nor replacing
it by vmalloc() can actually work. At the very least you'd want the
above to become

#ifndef LOCK_PROFILE
    BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
    d = alloc_xenheap_pages(0, MEMF_bits(bits));
#else
    BUILD_BUG_ON(sizeof(*d) > 2 * PAGE_SIZE);
    d = alloc_xenheap_pages(1, MEMF_bits(bits));
#endif

But preferably we'd find ways to shrink struct domain back to a size
where all build options work.

Jan

Re: Xen 4.6.0-rc1 build with lock_profile=y crash_debug=y, frame_pointer=y and domain.c:241: error: negative width in bit-field ‘<anonymous>’

[Konrad Rzeszutek Wilk]

On Wed, Aug 26, 2015 at 01:32:54AM -0600, Jan Beulich wrote:
> >>> On 25.08.15 at 21:54, <konrad.wilk@oracle.com> wrote:
> > On Tue, Aug 25, 2015 at 03:52:15PM -0400, Konrad Rzeszutek Wilk wrote:
> >> On Tue, Aug 25, 2015 at 06:41:06PM +0100, Andrew Cooper wrote:
> >> > On 25/08/15 18:09, Konrad Rzeszutek Wilk wrote:
> >> > > --- a/xen/arch/x86/domain.c
> >> > > +++ b/xen/arch/x86/domain.c
> >> > > @@ -238,7 +238,9 @@ struct domain *alloc_domain_struct(void)
> >> > >      if ( unlikely(!bits) )
> >> > >           bits = _domain_struct_bits();
> >> > >  
> >> > > +#ifndef LOCK_PROFILE
> >> > >      BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
> >> > > +#endif
> >> > >      d = alloc_xenheap_pages(0, MEMF_bits(bits));
> >> > >      if ( d != NULL )
> >> > >          clear_page(d);
> >> > >
> >> > > (not compile tested nor runtime tested)
> >> > 
> >> > Either remove it locally for debugging, or use something like
> > 
> > I forgot to mention that by removing it locally Xen ends up halting at:
> > 
> > (XEN) Detected 3292.657 MHz processor.
> 
> Sure - as Andrew said on irc, neither simply removing it nor replacing
> it by vmalloc() can actually work. At the very least you'd want the
> above to become

I ended up doing:

>From 52548151b6db3724fdafd005f52fc1aaefa65eff Mon Sep 17 00:00:00 2001
From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Date: Tue, 25 Aug 2015 21:25:54 -0400
Subject: [PATCH] x86: Allow 'struct domain' to expand past PAGE_SIZE

If we are building with lock profilling enabled - as that
expands the spinlock structures considerably.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 xen/arch/x86/domain.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index 045f6ff..7df58d8 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -237,9 +237,10 @@ struct domain *alloc_domain_struct(void)
 
     if ( unlikely(!bits) )
          bits = _domain_struct_bits();
-
+#ifndef LOCK_PROFILE
     BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
-    d = alloc_xenheap_pages(0, MEMF_bits(bits));
+#endif
+    d = alloc_xenheap_pages(get_order_from_bytes(sizeof(*d)), MEMF_bits(bits));
     if ( d != NULL )
         clear_page(d);
     return d;
@@ -261,7 +262,7 @@ struct vcpu *alloc_vcpu_struct(void)
      * structure must satisfy this restriction. Thus we specify MEMF_bits(32).
      */
     BUILD_BUG_ON(sizeof(*v) > PAGE_SIZE);
-    v = alloc_xenheap_pages(0, MEMF_bits(32));
+    v = alloc_xenheap_pages(get_order_from_bytes(sizeof(*v)), MEMF_bits(32));
     if ( v != NULL )
         clear_page(v);
     return v;
-- 
2.1.0


which made it compile and boot. However the moment I ran xenlockprof I got
this nice splash:

(XEN) CPU:    0
(XEN) RIP:    e008:[<ffff82d08012e0fa>] spinlocerate+0x5b/0x96
(XEN) RFLAGS: 0000000000010202   CONTEXT: hypervisor (d0v0)
(XEN) rax: ffff83009e6f7e40   rbx: c2c2c2c2c2c2c2c2   rcx: ffff83009e6f7d48
(XEN) rdx: 000000000000003d   rsi: 0000000000000001   rdi: c2c2c2c2c2c2c2c2
(XEN) rbp: ffff83009e6f7d38   rsp: ffff83009e6f7cf8   r8:  00000000deadbeef
(XEN) r9:  00000000deadbeef   r10: ffff82d0802542a0   r11: 0000000000000286
(XEN) r12: ffff830413381078   r13: 0000000000000001   r14: ffff82d08012da37
(XEN) r15: ffff83009e6f7d48   cr0: 0000000080050033   cr4: 00000000000426e0
(XEN) cr3: 0000000411303000   cr2: 00007f42d4a6e9a0
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
(XEN) Xen stack trace from rsp=ffff83009e6f7cf8:
(XEN)    0000000000411fc1 ffff82d080327ef0 ffff830400000000 ffff83009e6f7e40
(XEN)    00007f42d4ea4004 ffff83009e6f7f18 ffff83009e6f7f18 ffff82d08031be00
(XEN)    ffff83009e6f7d68 ffff82d08012e1e5 ffff83009e6f7e40 ffff82d000000000
(XEN)    0000000000000000 ffffffffffffffff ffff83009e6f7ef8 ffff82d08012f3fa
(XEN)    ffff83009e6f7d98 ffff83042c186000 ffff83009e8b0000 ffff83009e6f7d98
(XEN)    ffff83009e6f7f18 ffff83009e6f7f18 ffff83009e6f7f18 ffff83009e6f7f18
(XEN)    ffff83009e6f7f18 ffff83009e6f7f18 ffff83009e6f7f18 0000000000000000
(XEN)    ffff83009e6f7eb0 ffff83009e6f7ea8 ffff88008020ab10 0000000000411fc1
(XEN)    0000000000000000 ffff830000000007 0000000300007ff0 ffff820040008000
(XEN)    ffff83009e8b0000 8000000402e3d067 0000000400000002 ffff83009e6f7f18
(XEN)    0000000c0000000f 0000000000000002 00007ffd0000003e 0000000000000000
(XEN)    0000000000000000 00007f42d4a5a8e8 00000000010eb050 0000000000000000
(XEN)    00007ffd86253bd0 00000000004008f0 00007ffd86253cb0 00007f42d4ca07c5
(XEN)    00000000010eb050 00007f42d47ae695 00007ffd86253b68 00007ffd86253b68
(XEN)    00000000010eb050 0000000000000033 ffff83009e6f7ed8 ffff83009e8b0000
(XEN)    ffff880068795900 ffff88007395dd88 0000000000000003 00007ffd86253900
(XEN)    00007cff619080c7 ffff82d0802396d2 ffffffff8100146a 0000000000000023
(XEN)    0000000000000000 0000000000000000 00007ffd86253cb0 00000000004008f0
(XEN)    ffff8800686d3e68 00007ffd86253900 0000000000000286 00007ffd862539b0
(XEN)    00007ffd86253b68 00007f42d4c95256 0000000000000023 ffffffff8100146a
(XEN) Xen call trace:
(XEN)    [<ffff82d08012e0fa>] spinlock_profile_iterate+0x5b/0x96
(XEN)    [<ffff82d08012e1e5>] spinlock_profile_control+0x52/0x6c
(XEN)    [<ffff82d08012f3fa>] do_sysctl+0x33a/0x10c0
(XEN)    [<ffff82d0802396d2>] lstar_enter+0xe2/0x13c
(XEN) 
(XEN) 
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) GENERAL PROTECTION FAULT
(XEN) [error_code=0000]
(XEN) ****************************************
(XEN) 
(XEN) Reboot in five seconds...
(XEN) Resetting with ACPI MEMORY or I/O RESET_REG.

Off to figure this one out (http://darnok.org/xen-syms is where the binary is
in case any one else wants to jump on this) next week.

Re: Xen 4.6.0-rc1 build with lock_profile=y crash_debug=y, frame_pointer=y and domain.c:241: error: negative width in bit-field ‘<anonymous>’

[Wei Liu]

On Wed, Aug 26, 2015 at 01:37:12PM -0400, Konrad Rzeszutek Wilk wrote:
> On Wed, Aug 26, 2015 at 01:32:54AM -0600, Jan Beulich wrote:
> > >>> On 25.08.15 at 21:54, <konrad.wilk@oracle.com> wrote:
> > > On Tue, Aug 25, 2015 at 03:52:15PM -0400, Konrad Rzeszutek Wilk wrote:
> > >> On Tue, Aug 25, 2015 at 06:41:06PM +0100, Andrew Cooper wrote:
> > >> > On 25/08/15 18:09, Konrad Rzeszutek Wilk wrote:
> > >> > > --- a/xen/arch/x86/domain.c
> > >> > > +++ b/xen/arch/x86/domain.c
> > >> > > @@ -238,7 +238,9 @@ struct domain *alloc_domain_struct(void)
> > >> > >      if ( unlikely(!bits) )
> > >> > >           bits = _domain_struct_bits();
> > >> > >  
> > >> > > +#ifndef LOCK_PROFILE
> > >> > >      BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
> > >> > > +#endif
> > >> > >      d = alloc_xenheap_pages(0, MEMF_bits(bits));
> > >> > >      if ( d != NULL )
> > >> > >          clear_page(d);
> > >> > >
> > >> > > (not compile tested nor runtime tested)
> > >> > 
> > >> > Either remove it locally for debugging, or use something like
> > > 
> > > I forgot to mention that by removing it locally Xen ends up halting at:
> > > 
> > > (XEN) Detected 3292.657 MHz processor.
> > 
> > Sure - as Andrew said on irc, neither simply removing it nor replacing
> > it by vmalloc() can actually work. At the very least you'd want the
> > above to become
> 
> I ended up doing:
> 
> >From 52548151b6db3724fdafd005f52fc1aaefa65eff Mon Sep 17 00:00:00 2001
> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Date: Tue, 25 Aug 2015 21:25:54 -0400
> Subject: [PATCH] x86: Allow 'struct domain' to expand past PAGE_SIZE
> 
> If we are building with lock profilling enabled - as that
> expands the spinlock structures considerably.
> 
> Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> ---
>  xen/arch/x86/domain.c | 7 ++++---
>  1 file changed, 4 insertions(+), 3 deletions(-)
> 
> diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
> index 045f6ff..7df58d8 100644
> --- a/xen/arch/x86/domain.c
> +++ b/xen/arch/x86/domain.c
> @@ -237,9 +237,10 @@ struct domain *alloc_domain_struct(void)
>  
>      if ( unlikely(!bits) )
>           bits = _domain_struct_bits();
> -
> +#ifndef LOCK_PROFILE
>      BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
> -    d = alloc_xenheap_pages(0, MEMF_bits(bits));
> +#endif
> +    d = alloc_xenheap_pages(get_order_from_bytes(sizeof(*d)), MEMF_bits(bits));
>      if ( d != NULL )
>          clear_page(d);
>      return d;
> @@ -261,7 +262,7 @@ struct vcpu *alloc_vcpu_struct(void)
>       * structure must satisfy this restriction. Thus we specify MEMF_bits(32).
>       */
>      BUILD_BUG_ON(sizeof(*v) > PAGE_SIZE);
> -    v = alloc_xenheap_pages(0, MEMF_bits(32));
> +    v = alloc_xenheap_pages(get_order_from_bytes(sizeof(*v)), MEMF_bits(32));
>      if ( v != NULL )
>          clear_page(v);

clear_page() works on PAGE_SIZE page so if you allocate order 2 page it
is not sufficient.

Wei.

>      return v;
> -- 
> 2.1.0
> 
> 
> which made it compile and boot. However the moment I ran xenlockprof I got
> this nice splash:
> 
> (XEN) CPU:    0
> (XEN) RIP:    e008:[<ffff82d08012e0fa>] spinlocerate+0x5b/0x96
> (XEN) RFLAGS: 0000000000010202   CONTEXT: hypervisor (d0v0)
> (XEN) rax: ffff83009e6f7e40   rbx: c2c2c2c2c2c2c2c2   rcx: ffff83009e6f7d48
> (XEN) rdx: 000000000000003d   rsi: 0000000000000001   rdi: c2c2c2c2c2c2c2c2
> (XEN) rbp: ffff83009e6f7d38   rsp: ffff83009e6f7cf8   r8:  00000000deadbeef
> (XEN) r9:  00000000deadbeef   r10: ffff82d0802542a0   r11: 0000000000000286
> (XEN) r12: ffff830413381078   r13: 0000000000000001   r14: ffff82d08012da37
> (XEN) r15: ffff83009e6f7d48   cr0: 0000000080050033   cr4: 00000000000426e0
> (XEN) cr3: 0000000411303000   cr2: 00007f42d4a6e9a0
> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
> (XEN) Xen stack trace from rsp=ffff83009e6f7cf8:
> (XEN)    0000000000411fc1 ffff82d080327ef0 ffff830400000000 ffff83009e6f7e40
> (XEN)    00007f42d4ea4004 ffff83009e6f7f18 ffff83009e6f7f18 ffff82d08031be00
> (XEN)    ffff83009e6f7d68 ffff82d08012e1e5 ffff83009e6f7e40 ffff82d000000000
> (XEN)    0000000000000000 ffffffffffffffff ffff83009e6f7ef8 ffff82d08012f3fa
> (XEN)    ffff83009e6f7d98 ffff83042c186000 ffff83009e8b0000 ffff83009e6f7d98
> (XEN)    ffff83009e6f7f18 ffff83009e6f7f18 ffff83009e6f7f18 ffff83009e6f7f18
> (XEN)    ffff83009e6f7f18 ffff83009e6f7f18 ffff83009e6f7f18 0000000000000000
> (XEN)    ffff83009e6f7eb0 ffff83009e6f7ea8 ffff88008020ab10 0000000000411fc1
> (XEN)    0000000000000000 ffff830000000007 0000000300007ff0 ffff820040008000
> (XEN)    ffff83009e8b0000 8000000402e3d067 0000000400000002 ffff83009e6f7f18
> (XEN)    0000000c0000000f 0000000000000002 00007ffd0000003e 0000000000000000
> (XEN)    0000000000000000 00007f42d4a5a8e8 00000000010eb050 0000000000000000
> (XEN)    00007ffd86253bd0 00000000004008f0 00007ffd86253cb0 00007f42d4ca07c5
> (XEN)    00000000010eb050 00007f42d47ae695 00007ffd86253b68 00007ffd86253b68
> (XEN)    00000000010eb050 0000000000000033 ffff83009e6f7ed8 ffff83009e8b0000
> (XEN)    ffff880068795900 ffff88007395dd88 0000000000000003 00007ffd86253900
> (XEN)    00007cff619080c7 ffff82d0802396d2 ffffffff8100146a 0000000000000023
> (XEN)    0000000000000000 0000000000000000 00007ffd86253cb0 00000000004008f0
> (XEN)    ffff8800686d3e68 00007ffd86253900 0000000000000286 00007ffd862539b0
> (XEN)    00007ffd86253b68 00007f42d4c95256 0000000000000023 ffffffff8100146a
> (XEN) Xen call trace:
> (XEN)    [<ffff82d08012e0fa>] spinlock_profile_iterate+0x5b/0x96
> (XEN)    [<ffff82d08012e1e5>] spinlock_profile_control+0x52/0x6c
> (XEN)    [<ffff82d08012f3fa>] do_sysctl+0x33a/0x10c0
> (XEN)    [<ffff82d0802396d2>] lstar_enter+0xe2/0x13c
> (XEN) 
> (XEN) 
> (XEN) ****************************************
> (XEN) Panic on CPU 0:
> (XEN) GENERAL PROTECTION FAULT
> (XEN) [error_code=0000]
> (XEN) ****************************************
> (XEN) 
> (XEN) Reboot in five seconds...
> (XEN) Resetting with ACPI MEMORY or I/O RESET_REG.
> 
> Off to figure this one out (http://darnok.org/xen-syms is where the binary is
> in case any one else wants to jump on this) next week.

Re: Xen 4.6.0-rc1 build with lock_profile=y crash_debug=y, frame_pointer=y and domain.c:241: error: negative width in bit-field ‘<anonymous>’

[Jan Beulich]

>>> On 25.08.15 at 19:41, <andrew.cooper3@citrix.com> wrote:
> Jan: While reading the code, I note that the bits restriction is not
> required if CONFIG_BIGMEM, and should probably become conditional.

Indeed - patch about to be sent (but probably not qualifying for 4.6).

Jan

Re: Xen 4.6.0-rc1 build with lock_profile=y crash_debug=y, frame_pointer=y and domain.c:241: error: negative width in bit-field ‘<anonymous>’

[Jan Beulich]

>>> On 25.08.15 at 19:41, <andrew.cooper3@citrix.com> wrote:
> On 25/08/15 18:09, Konrad Rzeszutek Wilk wrote:
>> On Tue, Aug 25, 2015 at 05:48:58PM +0100, Andrew Cooper wrote:
>>> On 25/08/15 17:43, Konrad Rzeszutek Wilk wrote:
>>>> I am troubleshooting an locking issue and figured I would enable extra 
> options.
>>>>
>>>> But now I am hitting this build issue:
>>>>
>>>> domain.c:241: error: negative width in bit-field ‘<anonymous>’
>>>>
>>>> Which is:
>>>>
>>>>  229 struct domain *alloc_domain_struct(void)
>>>>  230 {
>>>>  231     struct domain *d;
>>>> ...
>>>>  241     BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
>>>>
>>>> Thoughts?
>>> That is to catch the case where sizeof struct domain exceeds PAGE_SIZE.
>>>
>>> The logic behind this was to prevent needing order 1 allocations for
>>> domains or vcpus, and therefore allocation failures in heavily memory
>>> fragmented situations.
>>>
>>> It means we will probably need to find some other areas of struct domain
>>> to shrink, or move out into a separate xmalloc().
>>>
>>> For straight debugging, it is acceptable to drop the BUILD_BUG_ON().  If
>>> we can't drop the size of struct domain, we might want to move to using
>>> vmalloc() instead.
>> It is big. pahole says:
>>
>>    /* size: 4352, cachelines: 68, members: 74 */ 
>>     /* sum members: 4238, holes: 9, sum holes: 34 */ 
>>     /* padding: 80 */ 
>>     /* paddings: 1, sum paddings: 4 */
> 
> So by rearranging to reduce padding, it would easily fit ;p

How that? 4238 > 4096.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.6.0-rc1 build with lock_profile=y crash_debug=y, frame_pointer=y and domain.c:241: error: negative width in bit-field ‘<anonymous>’

[Wei Liu]

On Tue, Aug 25, 2015 at 12:43:38PM -0400, Konrad Rzeszutek Wilk wrote:
> I am troubleshooting an locking issue and figured I would enable extra options.
>
> But now I am hitting this build issue:
>
> domain.c:241: error: negative width in bit-field ‘<anonymous>’
>
> Which is:
>
>  229 struct domain *alloc_domain_struct(void)
>  230 {
>  231     struct domain *d;
> ...
>  241     BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
>

In xen/include/xen/lib.h:

 14 #if __GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 6)
 15 /* Force a compilation error if condition is true */
 16 #define BUILD_BUG_ON(cond) ({ _Static_assert(!(cond), "!(" #cond ")"); })
 17
 18 /* Force a compilation error if condition is true, but also produce a
 19    result (of value 0 and type size_t), so the expression can be used
 20    e.g. in a structure initializer (or where-ever else comma expressions
 21    aren't permitted). */
 22 #define BUILD_BUG_ON_ZERO(cond) \
 23     sizeof(struct { _Static_assert(!(cond), "!(" #cond ")"); })
 24 #else
 25 #define BUILD_BUG_ON_ZERO(cond) sizeof(struct { int:-!!(cond); })
 26 #define BUILD_BUG_ON(cond) ((void)BUILD_BUG_ON_ZERO(cond))
 27 #endif

So you're using very old gcc and your struct domain is larger than 1
page.

Wei.

> Thoughts?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel