[0/3] Distributed storage. Mirror algo extension for automatic recovery.

[0/3] Distributed storage. Mirror algo extension for automatic recovery.

[Evgeniy Polyakov]

Hi.

I'm pleased to announce sixth release of the distributed storage
subsystem, which allows to form a storage on top of remote and local
nodes, which in turn can be exported to another storage as a node to
form tree-like storages.

This release includes mirroring algorithm extension, which allows to
store 'age' of the given node on the underlying media.

In this case, if failed node gets new media, which does not contain 
correct 'age' (unique id assigned to the whole storage during 
initialization time), the whole node will be marked as dirty and
eventually resynced.

This allows to have completely transparent failure recovery - failed
node can be just turned off, its hardware fixed and then turned on. DST
core will detect connection reset and automatically reconnect when node
is ready and resync if needed without any special administrator's steps.

This patchset has been split into 4 parts:
0 - this introduction
1 - core files
2 - network state machine
3 - documentation and algorithms

Hope they all will find its way into mail lists.

Further TODO list includes:
* new redundancy algorithm (complex, low priority)
* some thoughts about distributed filesystem tightly connected to DST

Thank you.

Homepage:
http://tservice.net.ru/~s0mbre/old/?section=projects&item=dst

Signed-off-by: Evgeniy Polyakov <johnpol@2ka.mipt.ru>

-- 
	Evgeniy Polyakov

Re: [0/3] Distributed storage. Mirror algo extension for automatic recovery.

[Andrew Morton]

On Thu, 18 Oct 2007 23:17:41 +0400
Evgeniy Polyakov <johnpol@2ka.mipt.ru> wrote:

> I'm pleased to announce sixth release of the distributed storage
> subsystem, which allows to form a storage on top of remote and local
> nodes, which in turn can be exported to another storage as a node to
> form tree-like storages.

I went back and re-read last month's discussion and I'm not seeing any
reason why we shouldn't start thinking about merging this.

How close is it to that stage?  A peek at your development blog indicates
that things are still changing at a moderate rate?

Re: [0/3] Distributed storage. Mirror algo extension for automatic recovery.

[Evgeniy Polyakov]

Hi Andrew.

On Fri, Oct 26, 2007 at 01:55:38PM -0700, Andrew Morton (akpm@linux-foundation.org) wrote:
> > I'm pleased to announce sixth release of the distributed storage
> > subsystem, which allows to form a storage on top of remote and local
> > nodes, which in turn can be exported to another storage as a node to
> > form tree-like storages.
> 
> I went back and re-read last month's discussion and I'm not seeing any
> reason why we shouldn't start thinking about merging this.
> 
> How close is it to that stage?  A peek at your development blog indicates
> that things are still changing at a moderate rate?

I completed storage layer development itself, the only remaining todo item
is to implement new redundancy algorithm, but I did not see major demand
on that, so it will stay for now with low priority.

I will use DST as a transport layer for distributed filesystem, and
probably that will require additional features, I have no clean design so
far, but right now I have nothing in the pipe to commit to DST.


-- 
	Evgeniy Polyakov