feature request: read/write priorities on multiple devices/raid1

feature request: read/write priorities on multiple devices/raid1

[Kaspar Schleiser]

Hello everybody!

This is my first post on the list, let me know if I'm not following any
rules.

What do you think about adding priorities for raid1 arrays?

For example, always use /dev/ram0 (ramdisk) for reading/writing, but
nonetheless write everything additionally on /dev/sda1. Restore to
/dev/ram0 after mount if it is empty.

Or use /dev/sd[c-h], maybe a raid 0 of fast read access/slow random write
flash media for reading, but /dev/sda1 (a reasonably fast HD) for writing.
make sure written data get's replicated on the raid0.

The linux md driver supports that in some way using the --write-mostly and
--write-behind options, but that's not very flexible.

Cheers
Kaspar Schleiser

Re: feature request: read/write priorities on multiple devices/raid1

[Jeff Schroeder]

On Sep 25, 2008, at 2:19 AM, "Kaspar Schleiser" <kaspar@schleiser.de>  
wrote:

> Hello everybody!
>
> This is my first post on the list, let me know if I'm not following  
> any
> rules.
>
> What do you think about adding priorities for raid1 arrays?
>
> For example, always use /dev/ram0 (ramdisk) for reading/writing, but
> nonetheless write everything additionally on /dev/sda1. Restore to
> /dev/ram0 after mount if it is empty.
>
> Or use /dev/sd[c-h], maybe a raid 0 of fast read access/slow random  
> write
> flash media for reading, but /dev/sda1 (a reasonably fast HD) for  
> writing.
> make sure written data get's replicated on the raid0.
>
> The linux md driver supports that in some way using the --write- 
> mostly and
> --write-behind options, but that's not very flexible.
>
> Cheers
> Kaspar Schleiser
>

Sounds roughly like a project posted to LKML not that long ago by  
Daniel Phillips. Google "LKML Daniel Phillips ramback".

Sent from my iPhone 

Re: feature request: read/write priorities on multiple devices/raid1

[Chris Mason]

On Thu, 2008-09-25 at 11:19 +0200, Kaspar Schleiser wrote:
> Hello everybody!
> 
> This is my first post on the list, let me know if I'm not following any
> rules.
> 
> What do you think about adding priorities for raid1 arrays?
> 
> For example, always use /dev/ram0 (ramdisk) for reading/writing, but
> nonetheless write everything additionally on /dev/sda1. Restore to
> /dev/ram0 after mount if it is empty.
> 

I don't plan on adding support for non-persistent storage, but there
will be the idea of a fast frontend device that is basically a cache for
slower devices.

-chris

Re: feature request: read/write priorities on multiple devices/raid1

[Miguel Sousa Filipe]

Hi all,

On Thu, Sep 25, 2008 at 2:10 PM, Chris Mason <chris.mason@oracle.com> wrote:
> On Thu, 2008-09-25 at 11:19 +0200, Kaspar Schleiser wrote:
>> Hello everybody!
>>
>> This is my first post on the list, let me know if I'm not following any
>> rules.
>>
>> What do you think about adding priorities for raid1 arrays?
>>
>> For example, always use /dev/ram0 (ramdisk) for reading/writing, but
>> nonetheless write everything additionally on /dev/sda1. Restore to
>> /dev/ram0 after mount if it is empty.
>>
>
> I don't plan on adding support for non-persistent storage, but there
> will be the idea of a fast frontend device that is basically a cache for
> slower devices.

Something like yet another caching/buffer layer between the buffer
cache and the storage devices?

I recall reading something similiar to that in a listing of new
features of a recent revision of ZFS ?

-- 
Miguel Sousa Filipe

Re: feature request: read/write priorities on multiple devices/raid1

[Xavier Nicollet]

Le 25 septembre 2008 =E0 16:31, Miguel Sousa Filipe a =E9crit:
> On Thu, Sep 25, 2008 at 2:10 PM, Chris Mason <chris.mason@oracle.com>=
 wrote:
> > I don't plan on adding support for non-persistent storage, but ther=
e
> > will be the idea of a fast frontend device that is basically a cach=
e for
> > slower devices.
>=20
> Something like yet another caching/buffer layer between the buffer
> cache and the storage devices?
>=20
> I recall reading something similiar to that in a listing of new
> features of a recent revision of ZFS ?

Reminds me WAFL from netapp.

http://media.netapp.com/documents/wp_3002.pdf

Transactions are written to NVRAM, then flushed every 10 seconds.
If some files are created then imediately deleted, it won't even touch
the disk.

As the server "knows" the underlining RAID structure, it is able to
choose how to flush the data for maximum disk throughput.

I am not a specialist, but this document is worth reading IMHO.

--=20
Xavier Nicollet
http://nicollet.jeru.org/
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: feature request: read/write priorities on multiple devices/raid1

[Chris Mason]

On Thu, 2008-09-25 at 16:31 +0100, Miguel Sousa Filipe wrote:
> Hi all,
> 
> On Thu, Sep 25, 2008 at 2:10 PM, Chris Mason <chris.mason@oracle.com> wrote:
> > On Thu, 2008-09-25 at 11:19 +0200, Kaspar Schleiser wrote:
> >> Hello everybody!
> >>
> >> This is my first post on the list, let me know if I'm not following any
> >> rules.
> >>
> >> What do you think about adding priorities for raid1 arrays?
> >>
> >> For example, always use /dev/ram0 (ramdisk) for reading/writing, but
> >> nonetheless write everything additionally on /dev/sda1. Restore to
> >> /dev/ram0 after mount if it is empty.
> >>
> >
> > I don't plan on adding support for non-persistent storage, but there
> > will be the idea of a fast frontend device that is basically a cache for
> > slower devices.
> 
> Something like yet another caching/buffer layer between the buffer
> cache and the storage devices?
> 

Not really a caching layer, more of a block allocation group that when
full migrates old blocks to a different block allocation group.

> I recall reading something similiar to that in a listing of new
> features of a recent revision of ZFS ?
> 

Sorry, no idea.

-chris

Re: feature request: read/write priorities on multiple devices/raid1

[Chris Samuel]

[-- Attachment #1: Type: text/plain, Size: 776 bytes --]

/* Sorry for replying to an ancient post, was away travelling! */

On Fri, 26 Sep 2008 1:31:41 am Miguel Sousa Filipe wrote:

> I recall reading something similiar to that in a listing of new
> features of a recent revision of ZFS ?

That's correct, there's a good blog [1] from one of the developers describing 
what they did and the improvements they saw.   And yes, they checksum the data 
on the SSD too, in case you share Val's worries [2]. :-)

[1] - http://blogs.sun.com/brendan/date/20080722
[2] - http://valhenson.livejournal.com/25228.html

cheers,
Chris
-- 
 Chris Samuel  :  http://www.csamuel.org/  :  Melbourne, VIC

This email may come with a PGP signature as a file. Do not panic.
For more info see: http://en.wikipedia.org/wiki/OpenPGP


[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 481 bytes --]