[PATCH] i386 & x86_64: Live Patching Funcion on 2.6.11.7

[PATCH] i386 & x86_64: Live Patching Funcion on 2.6.11.7

[Takashi Ikebe]

Hello,
This patch add function called "Live patching" which is defined on
OSDL's carrier grade linux requiremnt definition to linux 2.6.11.7 kernel.
The live patching allows process to patch on-line (without restarting
process) on i386 and x86_64 architectures, by overwriting jump assembly
code on entry point of functions which you want to fix, to patched
functions.
The live patching function is very common on high-availability system
such as carrier system, and this patch realize it also on linux.
(Patch & process restart time is very critical on such high-availability
system, live patch allows you to milliseconds order process stopping
time to apply new patch.)

The basis is below:
1. Live patch command loads the patch modules to target process's memory
area,
2. Live patch command resolve patch symbol.
3. Live patch command overwrite jump code to the entry point of function
which you want to fix, to the patch module's symbol.

Kernel patch and user mode tools are required, and both of them are
available at http://pannus.sourceforge.net
Please take a look and give us comments!

This patch add following system calls and function.
o mmap3: maps patch to target process's memory area with security check.
o accesspvm: access(read/write) target process's memory area.
o init_pend: initialization of live patch sequence on target process.
o rt_handlereturn: run initialize root of each patch (same as signal
handler).
o check_init: check that the initialization is finished or not.
o munmap3: unmap patch from target process's memory area.


---
Takashi Ikebe
NTT Network Service Systems Laboratories
9-11, Midori-Cho 3-Chome Musashino-Shi,
Tokyo 180-8585 Japan
Tel : +81 422 59 4246, Fax : +81 422 60 4012
e-mail : ikebe.takashi@lab.ntt.co.jp

Re: [PATCH] i386 & x86_64: Live Patching Funcion on 2.6.11.7

[David S. Miller]

Takashi-san, have you ever investigated using kprobes to
implement this feature?  It seems a perfect fit, and would
allow support on several architectures other than just x86
and x86_64.

If kprobes does not meet your needs completely, it could
be trivially extended to do so.

I think implementing something like this from scratch is
not a good idea when we have much of the needed logic and
infrastructure already.

Re: [PATCH] i386 & x86_64: Live Patching Funcion on 2.6.11.7

[Daniel Jacobowitz]

On Sat, Apr 16, 2005 at 11:44:39PM -0700, David S. Miller wrote:
> 
> Takashi-san, have you ever investigated using kprobes to
> implement this feature?  It seems a perfect fit, and would
> allow support on several architectures other than just x86
> and x86_64.
> 
> If kprobes does not meet your needs completely, it could
> be trivially extended to do so.
> 
> I think implementing something like this from scratch is
> not a good idea when we have much of the needed logic and
> infrastructure already.

Takashi-san's description was not very clear, but it sounds like it's a
patching mechanism for userspace applications - not for kernel space.
So kprobes would not be a good fit.

If I'm right, I'm not sure why some of the bits of it were done
separately instead of via the existing ptrace mechanism.  And GDB
would appreciate a mechanism for mmap/munmap/mprotect in a debugged
process, also.

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: [PATCH] i386 & x86_64: Live Patching Funcion on 2.6.11.7

[David S. Miller]

On Sun, 17 Apr 2005 14:51:43 -0400
Daniel Jacobowitz <dan@debian.org> wrote:

> Takashi-san's description was not very clear, but it sounds like it's a
> patching mechanism for userspace applications - not for kernel space.
> So kprobes would not be a good fit.

I saw the presentation of this stuff at the Linux Kernel conference
last year in Tokyo.  I'm pretty sure it's for the kernel. :-)

Re: [PATCH] i386 & x86_64: Live Patching Funcion on 2.6.11.7

[Takashi Ikebe]

Daniel-san, David-san,

Pannus project has two targets.
One is user-mode application live patching, and the other one is kernel 
live patching.
What we posted now is user-mode application live patching function.

 >If I'm right, I'm not sure why some of the bits of it were done
 >separately instead of via the existing ptrace mechanism.  And GDB
 >would appreciate a mechanism for mmap/munmap/mprotect in a debugged
 >process, also.

Daniel-san,
GDB based approach seems not fit to our requirements. GDB(ptrace) based 
functions are basically need to be done when target process is stopping. 
 From our experience, sometimes patches became to dozens to hundreds at 
one patching, and in this case GDB based approach cause target process's 
availability descent.

Patch exceeds 50k, so I cut comments and separate architecture, and post 
as in line.

David S. Miller wrote:
> On Sun, 17 Apr 2005 14:51:43 -0400
> Daniel Jacobowitz <dan@debian.org> wrote:
> 
> 
>>Takashi-san's description was not very clear, but it sounds like it's a
>>patching mechanism for userspace applications - not for kernel space.
>>So kprobes would not be a good fit.
> 
> 
> I saw the presentation of this stuff at the Linux Kernel conference
> last year in Tokyo.  I'm pretty sure it's for the kernel. :-)
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-- 
Takashi Ikebe
NTT Network Service Systems Laboratories
9-11, Midori-Cho 3-Chome Musashino-Shi,
Tokyo 180-8585 Japan
Tel : +81 422 59 4246, Fax : +81 422 60 4012
e-mail : ikebe.takashi@lab.ntt.co.jp

Re: [PATCH] i386 & x86_64: Live Patching Funcion on 2.6.11.7

[Daniel Jacobowitz]

On Mon, Apr 18, 2005 at 10:41:23AM +0900, Takashi Ikebe wrote:
> Daniel-san,
> GDB based approach seems not fit to our requirements. GDB(ptrace) based 
> functions are basically need to be done when target process is stopping. 
> From our experience, sometimes patches became to dozens to hundreds at 
> one patching, and in this case GDB based approach cause target process's 
> availability descent.

That's right, it does require the target process be stopped.  If it
isn't stopped how do you know it isn't executing the same instruction
you're currently patching?

Even with hundreds of kilobytes of patch, I have trouble imagining this
takes a substantial amount of time.

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: [PATCH] i386 & x86_64: Live Patching Funcion on 2.6.11.7

[Takashi Ikebe]

Daniel Jacobowitz wrote:

>On Mon, Apr 18, 2005 at 10:41:23AM +0900, Takashi Ikebe wrote:
>  
>
>>Daniel-san,
>>GDB based approach seems not fit to our requirements. GDB(ptrace) based 
>>functions are basically need to be done when target process is stopping. 
>>From our experience, sometimes patches became to dozens to hundreds at 
>>one patching, and in this case GDB based approach cause target process's 
>>availability descent.
>>    
>>
>
>That's right, it does require the target process be stopped.  If it
>isn't stopped how do you know it isn't executing the same instruction
>you're currently patching?
>
>Even with hundreds of kilobytes of patch, I have trouble imagining this
>takes a substantial amount of time.
>  
>
Pannus patch does not require target process stop while loading(mapping)
patch module to target process memory,
but as you described, target process stopping is needed whenever check
EIP not to conflict, and overwrite jump assembly code.(This makes only
few milliseconds target process stopping. Even on hundreds, it only
takes dozens mill-seconds yet.)
Typically telecoms application needs soft real time, and has timeout.
This kind of framework can not stop target process so long(Should be
dozens milliseconds at worst).
We want not to stop target process whenever patch module is loading....
we want not to stop target process as possible as.

-- 
Takashi Ikebe
NTT Network Service Systems Laboratories
9-11, Midori-Cho 3-Chome Musashino-Shi,
Tokyo 180-8585 Japan
Tel : +81 422 59 4246, Fax : +81 422 60 4012
e-mail : ikebe.takashi@lab.ntt.co.jp

Re: [PATCH] i386 & x86_64: Live Patching Funcion on 2.6.11.7

[Andi Kleen]

"David S. Miller" <davem@davemloft.net> writes:

> Takashi-san, have you ever investigated using kprobes to
> implement this feature?  It seems a perfect fit, and would
> allow support on several architectures other than just x86
> and x86_64.
>
> If kprobes does not meet your needs completely, it could
> be trivially extended to do so.

kprobes would require an exception for each patchpoint because
it uses an trap instruction. 

Probably a bit too costly for commonly used functions. Especially
on a P4 exceptions are quite costly. 

But you could add lightweight kprobes that just use jmp 
for it in the kprobes  framework, that might be useful for other stuff too

-Andi