Re: [Ksummit-2010-discuss] checkpoint-restart: naked patch

[Tejun Heo <tj@kernel.org>]

Hello,

On 11/17/2010 06:29 AM, Anton Blanchard wrote:
> It seems like there are a number of questions around the utility of
> C/R so I'd like to take a step back from the technical discussion
> around implementation and hopefully convince you, Tejun (and anyone
> else interested) that C/R is something we want to solve in Linux.

I'm not arguing CR isn't that useful.  My argument was that it's a
solution for a fairly niche problems and that the implementation isn't
transparent at all for general use cases.

> Here at IBM we are working on the next generation of HPC systems. One
> example of this will be the NCSA Bluewaters supercomputer:

And, yeah, I agree that it is a very useful thing for HPC.

> You could argue that we should just add C/R capability to every HPC
> application and library people care about or rework them to be
> fault tolerant in software. Unfortunately I don't see either as being
> viable. There are so many applications, libraries and even programming
> languages in use for HPC that it would be a losing battle. If we
> did go down this route we would also be unable to leverage C/R for
> anything else. I can understand the concern around finding a general
> purpose case, but I do believe many other solid uses for C/R outside of
> HPC will emerge. For example, there was interest from the embedded guys
> during the KS discussion and I can easily imagine using C/R to bring up
> firefox faster on a TV.

Thanks for pointing out the use cases although for the last one it
would be much wiser to just use webkit.

> The problems found in HPC often turn into more general problems down
> the track. I think back to the heated discussions we had around SMP back
> in the early 2000s when we had 32 core POWER4s and SGI had similar sized
> machines. Now a 24 core machine fits in 1U and can be purchased for
> under $5k. NUMA support, CPU affinity and multi queue scheduling are
> other areas that initially had a very small user base but have since
> become important features for many users.

Sure, the pointy edges can discover general problems of future early.
At the same time, they also encounter problems which no one else would
care about ever, so in itself it isn't much of an argument.  I'm no
analyst and it is very difficult to foretell the future but comparing
CR to SMP and NUMA doesn't seem too valid to me.  In-kernel CR is
sandwiched between userland CR and virtualization.  Its problem space
is shrinking, not expanding.

Having a generally accepted standard CR implementation would certainly
be very nice and I understand that CR would be a much better fit for
HPC than virtualization, but I fail to see why it should be
implemented in kernel when userland implementation which doesn't
extend the kernel in any way already achieves most of what HPC
workload requires.  In this already sizeable thread, the only benefits
presented seem to be the ability to cover some more corner cases and
remote use cases in slightly more transparent manner.  Those are very
weak arguments for something as intrusive and backwards (in that it
dumps kernel states in binary blobs unrestrained by ABI) as in-kernel
CR and, as such, I don't really see the in-kernel CR surviving as a
mainline feature.

So, I think the best recourse would be identifying the specific
features which would help userland CR and improve them.  The in-kernel
CR people have been working on the problem for a long time now and
gotta know which parts are tricky and how to solve them.  In fact, I
don't think the work would be that widely different.  It would be
harder but those changes would benefit other use cases too instead of
only useful for in-kernel CR.

Thanks.

-- 
tejun