crash proof / recovery from power loss

crash proof / recovery from power loss

[Sascha Silbe]

[-- Attachment #1: Type: text/plain, Size: 413 bytes --]

Hi!


I'm going to use git as a back end for the Sugar data store / Journal. 
[1]
One thing I'm worried about is how it's going to behave if a crash (e.g. 
power loss - no corruption) occurs.
What can happen in such a case? Is it always recoverable and if so, how?


[1] http://wiki.sugarlabs.org/go/Version_support_for_datastore

CU Sascha

-- 
http://sascha.silbe.org/
http://www.infra-silbe.de/

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 489 bytes --]

Re: crash proof / recovery from power loss

[Jakub Narebski]

Sascha Silbe <sascha-ml-devel-vcs-git@silbe.org> writes:

> I'm going to use git as a back end for the Sugar data store /
> Journal. [1]
>
> One thing I'm worried about is how it's going to behave if a crash
> (e.g. power loss - no corruption) occurs.
> What can happen in such a case? Is it always recoverable and if so, how?
> 
> 
> [1] http://wiki.sugarlabs.org/go/Version_support_for_datastore

In Git all operations are atomic.  

In most cases Git uses the following 'trick': it first
downloads/saves/creates data, and only then automatically update
information that it is available.  So crash would result in some
unreachanble objects, which would go away on prune.

In some cases Git uses locks to ensure atomicity.  I don't know what
happens with locks after a crash...

-- 
Jakub Narebski
Poland
ShadeHawk on #git

Re: crash proof / recovery from power loss

[Martin Langhoff]

On Tue, Jun 23, 2009 at 11:05 PM, Sascha Silbe
<sascha-ml-devel-vcs-git@silbe.org> wrote:
> I'm going to use git as a back end for the Sugar data store / Journal. [1]
> One thing I'm worried about is how it's going to behave if a crash (e.g. power loss - no corruption) occurs.
> What can happen in such a case? Is it always recoverable and if so, how?

Git is outrageously good in keeping things sane. On that front, you
won't see any problems.

Now... there is one area where it will hurt lots: storing large binary
blobs. Git assumes that every file is very small in comparison with
available physical memory.

 - Sugar utilities already can create files larger than available
memory. Think Record.xo, etoys, etc.

 - Many sugar target platforms are _very_ memory constrained. XO-1
being the prime example.

Lots of what git does would be good in the Journal. But some of the
core assumptions are not well matched.

There are some tunables in git that might help. And maybe you can
sidestep the harder corners of the problem by having a threshold, and
handling large files separately.

hth,




m
--
martin.langhoff@gmail.com
martin@laptop.org -- School Server Architect
- ask interesting questions
- don't get distracted with shiny stuff  - working code first
- http://wiki.laptop.org/go/User:Martinlanghoff