Re: [PATCH 2/6] xen/iommu: Move dom0 setup code out of __init

[Daniel De Graaf <dgdegra@tycho.nsa.gov>]

On 03/05/2014 04:56 AM, Jan Beulich wrote:
>>>> On 04.03.14 at 23:51, Daniel De Graaf <dgdegra@tycho.nsa.gov> wrote:
>> This is required to allow the hardware domain to be built by dom0 after
>> the __init sections have been discarded.
>
> Can you quantify the amount of code/data getting moved?

It looks like about one page of code and very little data (fits in slack space):

objdump -x old-xen-syms:
Idx Name          Size      VMA               LMA               File off  Algn
   0 .text         00138312  ffff82d080100000  ffff82d080100000  00001000  2**12
                   CONTENTS, ALLOC, LOAD, CODE
   1 .rodata       0004a830  ffff82d080238320  ffff82d080238320  00139320  2**5
                   CONTENTS, ALLOC, LOAD, READONLY, DATA
   2 .data.read_mostly 0000b1d8  ffff82d080282b80  ffff82d080282b80  00183b80  2**7
                   CONTENTS, ALLOC, LOAD, DATA
   3 .data         0000e828  ffff82d08028e000  ffff82d08028e000  0018f000  2**12
                   CONTENTS, ALLOC, LOAD, DATA
   4 .init.text    0002e96b  ffff82d08029d000  ffff82d08029d000  0019e000  2**4
                   CONTENTS, ALLOC, LOAD, READONLY, CODE
   5 .init.data    0000f638  ffff82d0802cb980  ffff82d0802cb980  001cc980  2**5
                   CONTENTS, ALLOC, LOAD, DATA
   6 .init.setup   000010c0  ffff82d0802dafc0  ffff82d0802dafc0  001dbfc0  2**5
                   CONTENTS, ALLOC, LOAD, DATA
   7 .initcall.init 00000210  ffff82d0802dc080  ffff82d0802dc080  001dd080  2**3
                   CONTENTS, ALLOC, LOAD, DATA
   8 .xsm_initcall.init 00000008  ffff82d0802dc290  ffff82d0802dc290  001dd290  2**3
                   CONTENTS, ALLOC, LOAD, DATA
   9 .bss          0004c900  ffff82d0802e0000  ffff82d0802e0000  001dd298  2**7
                   ALLOC

objdump -x new-xen-syms:
   0 .text         00139312  ffff82d080100000  ffff82d080100000  00001000  2**12
                   CONTENTS, ALLOC, LOAD, CODE
   1 .rodata       0004a830  ffff82d080239320  ffff82d080239320  0013a320  2**5
                   CONTENTS, ALLOC, LOAD, READONLY, DATA
   2 .data.read_mostly 0000b1d8  ffff82d080283b80  ffff82d080283b80  00184b80  2**7
                   CONTENTS, ALLOC, LOAD, DATA
   3 .data         0000e828  ffff82d08028f000  ffff82d08028f000  00190000  2**12
                   CONTENTS, ALLOC, LOAD, DATA
   4 .init.text    0002dfcb  ffff82d08029e000  ffff82d08029e000  0019f000  2**4
                   CONTENTS, ALLOC, LOAD, READONLY, CODE
   5 .init.data    0000f5f8  ffff82d0802cbfe0  ffff82d0802cbfe0  001ccfe0  2**5
                   CONTENTS, ALLOC, LOAD, DATA
   6 .init.setup   000010c0  ffff82d0802db5e0  ffff82d0802db5e0  001dc5e0  2**5
                   CONTENTS, ALLOC, LOAD, DATA
   7 .initcall.init 00000210  ffff82d0802dc6a0  ffff82d0802dc6a0  001dd6a0  2**3
                   CONTENTS, ALLOC, LOAD, DATA
   8 .xsm_initcall.init 00000008  ffff82d0802dc8b0  ffff82d0802dc8b0  001dd8b0  2**3
                   CONTENTS, ALLOC, LOAD, DATA
   9 .bss          0004c980  ffff82d0802e0000  ffff82d0802e0000  001dd8b8  2**7
                   ALLOC

> Overall I wonder what the purpose is of allowing the hardware
> domain to be created later, rather than keeping Dom0 for this
> purpose. After all booting the system is very much hardware
> control. Once a control domain was spawned, Dom0 could then
> be revoked the right to create further domains.

One reason is to maintain a small trusted computing base for critical
components of the platform, such as the control domains and the vTPM
infrastructure.  We would prefer to avoid as much parsing of unmeasured
(and therefore untrusted) data as possible in the early boot process
because any vulnerability in these parsers becomes an attack vector for
components depending on them.

When using TXT/TBOOT as the basis for trust, the Xen hypervisor and
domain 0 are measured and may be considered more trusted than the state
of hardware and any structures supplied by the BIOS that could be
modified by a bootloader or other unmeasured "gap code".  When a domain
0 running Linux boots, it parses the ACPI tables and enumerates hardware
based on them prior to executing any user-space code.  Exploits using
malicious ACPI tables can be used for privilege escalation (see
CVE-2010-4347 for an example), so if the domain building process were
run from the initrd, all domains on the system would be vulnerable to an
attack using this.  ACPI is not the only attack vector here; malicious
hardware is very difficult to defend the hardware domain from once the
IOMMU begins allowing DMA access to the domain's memory.

A possible alternative that does not involve hypervisor modification
would be to have the domain builder use kexec() after building domains
and de-privileging itself.  This requires a more complex domain builder,
and makes a number of security assertions more difficult to evaluate
from the security policy.  With a separate domain builder and hardware
domain, certain ranges of memory (such as the TPM MMIO pages) can be
restricted from ever being accessible to the hardware domain in the
security policy.  Using separate domains also avoids the possibility
that the domain builder neglects to clean up a mapping to a just-created
domain prior to kexec which could be used to later change that domain's
behavior; while a given version of the domain builder may not have such
bugs, new features (such as the ability to change the Xenstore start
info page after event channels are added) can introduce bugs that
silently violate the stated security policy.

>> --- a/xen/drivers/passthrough/vtd/x86/vtd.c
>> +++ b/xen/drivers/passthrough/vtd/x86/vtd.c
>> @@ -36,7 +36,7 @@
>>    * iommu_inclusive_mapping: when set, all memory below 4GB is included in dom0
>>    * 1:1 iommu mappings except xen and unusable regions.
>>    */
>> -static bool_t __initdata iommu_inclusive_mapping = 1;
>> +static bool_t iommu_inclusive_mapping = 1;
>
> __read_mostly?

Yes. I used __read_mostly in the size tests above, and will in v2.

> Jan

-- 
Daniel De Graaf
National Security Agency