Livepatch for Xen 4.9

Livepatch for Xen 4.9

[Konrad Rzeszutek Wilk]

Hey!

[CC-ing xen-devel]

Xen 4.8-rc1 is out and means taking a break from some of the Livepatch hypervisor
parts for me.

My plan for 4.8 is to concentrate on any livepatch fallout and doing OSSTest along
with Marcos (CC-ed) and see if we can wrestle it to expand on what
we want to have done.

However going forward (Xen 4.9) I believe the top issues we need
to get addressed are:

 a) "A better mechanism to "mask" NMIs during patching. The existing mechanism looses
   NMI if they have been sent and we don't have a mechanism to replay them. Note that
   this is also fixes alternative section patching. Could (like Linux) annotate handlers don't get patched."
   (https://wiki.xenproject.org/wiki/LivePatch).
 b) Restart the shrinking of code using__LINE__
 c) When figuring out the new_addr, take into account name being <symbol>+<offset>.
 d) Make asm code be in its own section. That eases the livepatch tools work in figuring out a change.
    See https://lkml.org/lkml/2009/2/24/364
 e) ? 

 g) Make XENPF_get_symbol also include Live Patch symbols.


I was wondering if folks could put in their preference and what they are thinking
to work on during 4.9?


During 4.8 I took a stab at a) and c).

The 'a)' is a thorny one as:
 1) If we get an NMI during patching we better not be patching MCE/NMI code! This could be avoided
    by annotating all the MCE/NMI code and have a 'blacklist' of functions that MUST NEVER BE PATCHED.
    But that may be difficult as that code can reach in many parts (especially MCE which wants to
    send an event channel, hence touches normal event channel code).

 2) We could also do some form of restartable patching. That is seed the code (where we are going to
    put a trampoline) with 'CC'. Then do memcpy over the the 'CC' the new instructions (jump). If the
    NMI/MCE handler hits that code it would call the int3 - which we expand now to take over and check
    whether the EIP is in the location which we just seeded with 'CC' - and if so it can memcpy the
    trampoline code in (with a slight twist - we first memcpy the displacement, so the start of a function
    would be say: CC 00 23 00 10 - and then we do a single write to replace 'CC' with 'E9').
   
    We could also (to combat bitrotting) change this a bit more - the debugger handler is in charge of
    actually memcpying and the arch_livepatch_apply just calls 'int 3' (after it has seeded the area with 'CC').

    Either of this would would require some global livepatch->debugger handler handover structure with
    the EIP to patch over and the insns.


The 'c)' needs a bit more work.


Also I was thinking we can drop the IRC meeting we have setup. It has been quite useful during the
starting stage to re-sync patches but at this point I think emails are more suited?


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Livepatch for Xen 4.9

[Jan Beulich]

>>> Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> 10/03/16 4:18 PM >>>
>2) We could also do some form of restartable patching. That is seed the code (where we are going to
>put a trampoline) with 'CC'. Then do memcpy over the the 'CC' the new instructions (jump). If the
>NMI/MCE handler hits that code it would call the int3 - which we expand now to take over and check
>whether the EIP is in the location which we just seeded with 'CC' - and if so it can memcpy the
>trampoline code in (with a slight twist - we first memcpy the displacement, so the start of a function
>would be say: CC 00 23 00 10 - and then we do a single write to replace 'CC' with 'E9').
   
Careful here: How do you mean to return from the int3 handler? You mustn't use IRET
there, or else you unmask further NMIs.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Livepatch for Xen 4.9

[Andrew Cooper]

On 03/10/16 15:16, Konrad Rzeszutek Wilk wrote:
> Hey!
>
> [CC-ing xen-devel]
>
> Xen 4.8-rc1 is out and means taking a break from some of the Livepatch hypervisor
> parts for me.
>
> My plan for 4.8 is to concentrate on any livepatch fallout and doing OSSTest along
> with Marcos (CC-ed) and see if we can wrestle it to expand on what
> we want to have done.
>
> However going forward (Xen 4.9) I believe the top issues we need
> to get addressed are:
>
>  a) "A better mechanism to "mask" NMIs during patching. The existing mechanism looses
>    NMI if they have been sent and we don't have a mechanism to replay them. Note that
>    this is also fixes alternative section patching. Could (like Linux) annotate handlers don't get patched."
>    (https://wiki.xenproject.org/wiki/LivePatch).

You cant mask NMIs, and as we have alternatives at the head of the
entrypoints, we need to work towards making patching safe on these
paths.  The traditional way is with 0xcc and magic in the debug trap
handler to take over the responsibility of patching.

>  b) Restart the shrinking of code using__LINE__

+1 (shame these patches missed 4.8)

>  c) When figuring out the new_addr, take into account name being <symbol>+<offset>.
>  d) Make asm code be in its own section. That eases the livepatch tools work in figuring out a change.
>     See https://lkml.org/lkml/2009/2/24/364

d.1) Reducing the quantity of ASM code outright.

As a start, {,compat_}create_bounce_frame() should definitely be written
in C, and would half the quantity of runtime ASM we have.  (Worse, we
already have C versions of create_bounce_frame() with
ever-so-slighty-different semantics).  I also have my eye on the general
exception handling path, which I think can safely move up into C.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Livepatch for Xen 4.9

[Ross Lagerwall]

On 10/03/2016 03:16 PM, Konrad Rzeszutek Wilk wrote:
> Hey!
>
> [CC-ing xen-devel]
>
> Xen 4.8-rc1 is out and means taking a break from some of the Livepatch hypervisor
> parts for me.

Thanks for all the work you put in!

>
> My plan for 4.8 is to concentrate on any livepatch fallout and doing OSSTest along
> with Marcos (CC-ed) and see if we can wrestle it to expand on what
> we want to have done.
>
> However going forward (Xen 4.9) I believe the top issues we need
> to get addressed are:
>
>  a) "A better mechanism to "mask" NMIs during patching. The existing mechanism looses
>    NMI if they have been sent and we don't have a mechanism to replay them. Note that
>    this is also fixes alternative section patching. Could (like Linux) annotate handlers don't get patched."
>    (https://wiki.xenproject.org/wiki/LivePatch).
>  b) Restart the shrinking of code using__LINE__
>  c) When figuring out the new_addr, take into account name being <symbol>+<offset>.
>  d) Make asm code be in its own section. That eases the livepatch tools work in figuring out a change.
>     See https://lkml.org/lkml/2009/2/24/364
>  e) ?
>
>  g) Make XENPF_get_symbol also include Live Patch symbols.
>
>
> I was wondering if folks could put in their preference and what they are thinking
> to work on during 4.9?
>
>

I'm going to try implement signature verification of livepatch modules. 
The bits for signing modules are already in livepatch-build-tools, so it 
just requires the hypervisor to verify the signature. How hard can that 
be? :-)

>
>
> Also I was thinking we can drop the IRC meeting we have setup. It has been quite useful during the
> starting stage to re-sync patches but at this point I think emails are more suited?
>

Sure.

-- 
Ross Lagerwall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel