support for external persistent cache

support for external persistent cache

[Cory Coager]

I was wondering if it would be possible to add support for specifying a 
device to use for read/write persistent cache.  The reason why I'm 
asking is because I see physical ram disks with battery backups 
available recently.  These ram disks are relatively cheap and could add 
great performance for write through without the fear of data loss.

Any interest in adding support for this?

Re: support for external persistent cache

[Roberto Spadim]

i think that is a fileystem cache
not a md raid cache
i'm right?

2011/1/19 Cory Coager <ccoager@gmail.com>:
> I was wondering if it would be possible to add support for specifying a
> device to use for read/write persistent cache.  The reason why I'm asking is
> because I see physical ram disks with battery backups available recently.
>  These ram disks are relatively cheap and could add great performance for
> write through without the fear of data loss.
>
> Any interest in adding support for this?
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
Roberto Spadim
Spadim Technology / SPAEmpresarial
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: support for external persistent cache

[Roberto Spadim]

reading again... should you use a good ups system with information
about % of batery, like a notebook?
read/write cache can be change with async or sync options on filesystem mount
batery support is done with ups system
maybe with some time of use (10 years) the baterry of you ramdisk
should be replaced, will you replace it? is easier replace a ups
batery or a ramdisk specific batery?
check at internet about 12vdc atx power supply, and use a charger and
a car batery to supply power to your computer, it's a very nice
solution =)


2011/1/19 Roberto Spadim <roberto@spadim.com.br>:
> i think that is a fileystem cache
> not a md raid cache
> i'm right?
>
> 2011/1/19 Cory Coager <ccoager@gmail.com>:
>> I was wondering if it would be possible to add support for specifying a
>> device to use for read/write persistent cache.  The reason why I'm asking is
>> because I see physical ram disks with battery backups available recently.
>>  These ram disks are relatively cheap and could add great performance for
>> write through without the fear of data loss.
>>
>> Any interest in adding support for this?
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
>
>
> --
> Roberto Spadim
> Spadim Technology / SPAEmpresarial
>



-- 
Roberto Spadim
Spadim Technology / SPAEmpresarial
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: support for external persistent cache

[Cory Coager]

On 01/18/2011 10:17 PM, Roberto Spadim wrote:
> reading again... should you use a good ups system with information
> about % of batery, like a notebook?
> read/write cache can be change with async or sync options on filesystem mount
> batery support is done with ups system
> maybe with some time of use (10 years) the baterry of you ramdisk
> should be replaced, will you replace it? is easier replace a ups
> batery or a ramdisk specific batery?
> check at internet about 12vdc atx power supply, and use a charger and
> a car batery to supply power to your computer, it's a very nice
> solution =)
Sure, I'll replace the battery in the ram disk.  Not much different than 
a hardware raid card with a battery in it.

I think you're missing my point.  A UPS isn't going to save your file 
system if the machine dies mid-write.

Re: support for external persistent cache

[Roberto Spadim]

ok,
but if you don´t sync file system (remove from memory and put in disk
controller)
you will lost information with or without a batery

how to don´t lose information?
don´t power down you memory,cpu, disk controller (sata controler, raid
controller, or anyother) and disks (does it have a batery? a super
capacitor?)
if you power down, be sure that all memory was send to disk controller
and that disk controller have energy (batery or capacitor) to send
information to disks (they need batery or capacitor too)

right?
so, a ups can power cpu, memory, disk controller and disks with only
one batery (not a batery for each device cpu,memory,disk,disk
controller)
the best world could be a non volatile memory (250mb/s flash 4kb
block) with the speed of volatile memory (10000mb/s ddr3 i don´t know
the block size)


2011/1/19 Cory Coager <ccoager@gmail.com>:
> On 01/18/2011 10:17 PM, Roberto Spadim wrote:
>>
>> reading again... should you use a good ups system with information
>> about % of batery, like a notebook?
>> read/write cache can be change with async or sync options on filesystem
>> mount
>> batery support is done with ups system
>> maybe with some time of use (10 years) the baterry of you ramdisk
>> should be replaced, will you replace it? is easier replace a ups
>> batery or a ramdisk specific batery?
>> check at internet about 12vdc atx power supply, and use a charger and
>> a car batery to supply power to your computer, it's a very nice
>> solution =)
>
> Sure, I'll replace the battery in the ram disk.  Not much different than a
> hardware raid card with a battery in it.
>
> I think you're missing my point.  A UPS isn't going to save your file system
> if the machine dies mid-write.
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
Roberto Spadim
Spadim Technology / SPAEmpresarial
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: support for external persistent cache

[Cory Coager]

On Wed, Jan 19, 2011 at 01:19:18PM -0200, Roberto Spadim wrote:
> ok,
> but if you don?t sync file system (remove from memory and put in disk
> controller)
> you will lost information with or without a batery
> 
> how to don?t lose information?
> don?t power down you memory,cpu, disk controller (sata controler, raid
> controller, or anyother) and disks (does it have a batery? a super
> capacitor?)
> if you power down, be sure that all memory was send to disk controller
> and that disk controller have energy (batery or capacitor) to send
> information to disks (they need batery or capacitor too)
> 
> right?
> so, a ups can power cpu, memory, disk controller and disks with only
> one batery (not a batery for each device cpu,memory,disk,disk
> controller)
> the best world could be a non volatile memory (250mb/s flash 4kb
> block) with the speed of volatile memory (10000mb/s ddr3 i don?t know
> the block size)

It would have to work the same as a hardware RAID controller.
Information is first written to the cache then synced to the
disk.  If the data is in the cache but not on the disk, the
machine loses power, next boot up the software raid would need a
way to flush the data from the ram disk to the disk.  Of course
this would require the battery be in working condition, as with
any hardware.

Hopefully I've explained that well enough.  Perhaps it will be
better to see the hardware I'm talking about:
http://techreport.com/articles.x/16255

Re: support for external persistent cache

[Roberto Spadim]

ok, your hardware have:
cpu, memory, disk controller, disks

and you computer have:
cpu, memory, disk controler (your hardware)

if your computer cache don´t sync to your disk controller you will
lose information....

check that *memory, is the volatille memory and *disk controller is
the non volatille memory
if you tell me that you will never have a *memory, and you have always
a non volatille memory, no problem, you will never need a kernel
load... just a boot loader that read previous memory information and
start in that point... why don´t do this? non volatille memory is not
as fast as volatille memory
got the problem?


2011/1/19 Cory Coager <ccoager@gmail.com>:
> On Wed, Jan 19, 2011 at 01:19:18PM -0200, Roberto Spadim wrote:
>> ok,
>> but if you don?t sync file system (remove from memory and put in disk
>> controller)
>> you will lost information with or without a batery
>>
>> how to don?t lose information?
>> don?t power down you memory,cpu, disk controller (sata controler, raid
>> controller, or anyother) and disks (does it have a batery? a super
>> capacitor?)
>> if you power down, be sure that all memory was send to disk controller
>> and that disk controller have energy (batery or capacitor) to send
>> information to disks (they need batery or capacitor too)
>>
>> right?
>> so, a ups can power cpu, memory, disk controller and disks with only
>> one batery (not a batery for each device cpu,memory,disk,disk
>> controller)
>> the best world could be a non volatile memory (250mb/s flash 4kb
>> block) with the speed of volatile memory (10000mb/s ddr3 i don?t know
>> the block size)
>
> It would have to work the same as a hardware RAID controller.
> Information is first written to the cache then synced to the
> disk.  If the data is in the cache but not on the disk, the
> machine loses power, next boot up the software raid would need a
> way to flush the data from the ram disk to the disk.  Of course
> this would require the battery be in working condition, as with
> any hardware.
>
> Hopefully I've explained that well enough.  Perhaps it will be
> better to see the hardware I'm talking about:
> http://techreport.com/articles.x/16255
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
Roberto Spadim
Spadim Technology / SPAEmpresarial
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: support for external persistent cache

[Roberto Spadim]

don´t forget that you can use ramdisks.... just read how to select the
right memory, and the right position before initialize you ramdisk

2011/1/19 Roberto Spadim <roberto@spadim.com.br>:
> ok, your hardware have:
> cpu, memory, disk controller, disks
>
> and you computer have:
> cpu, memory, disk controler (your hardware)
>
> if your computer cache don´t sync to your disk controller you will
> lose information....
>
> check that *memory, is the volatille memory and *disk controller is
> the non volatille memory
> if you tell me that you will never have a *memory, and you have always
> a non volatille memory, no problem, you will never need a kernel
> load... just a boot loader that read previous memory information and
> start in that point... why don´t do this? non volatille memory is not
> as fast as volatille memory
> got the problem?
>
>
> 2011/1/19 Cory Coager <ccoager@gmail.com>:
>> On Wed, Jan 19, 2011 at 01:19:18PM -0200, Roberto Spadim wrote:
>>> ok,
>>> but if you don?t sync file system (remove from memory and put in disk
>>> controller)
>>> you will lost information with or without a batery
>>>
>>> how to don?t lose information?
>>> don?t power down you memory,cpu, disk controller (sata controler, raid
>>> controller, or anyother) and disks (does it have a batery? a super
>>> capacitor?)
>>> if you power down, be sure that all memory was send to disk controller
>>> and that disk controller have energy (batery or capacitor) to send
>>> information to disks (they need batery or capacitor too)
>>>
>>> right?
>>> so, a ups can power cpu, memory, disk controller and disks with only
>>> one batery (not a batery for each device cpu,memory,disk,disk
>>> controller)
>>> the best world could be a non volatile memory (250mb/s flash 4kb
>>> block) with the speed of volatile memory (10000mb/s ddr3 i don?t know
>>> the block size)
>>
>> It would have to work the same as a hardware RAID controller.
>> Information is first written to the cache then synced to the
>> disk.  If the data is in the cache but not on the disk, the
>> machine loses power, next boot up the software raid would need a
>> way to flush the data from the ram disk to the disk.  Of course
>> this would require the battery be in working condition, as with
>> any hardware.
>>
>> Hopefully I've explained that well enough.  Perhaps it will be
>> better to see the hardware I'm talking about:
>> http://techreport.com/articles.x/16255
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
>
>
> --
> Roberto Spadim
> Spadim Technology / SPAEmpresarial
>



-- 
Roberto Spadim
Spadim Technology / SPAEmpresarial
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: support for external persistent cache

[Roberto Spadim]

look this:
http://www.ramsan.com/products/4
http://www.ramsan.com/products/2


2011/1/19 Roberto Spadim <roberto@spadim.com.br>:
> don´t forget that you can use ramdisks.... just read how to select the
> right memory, and the right position before initialize you ramdisk
>
> 2011/1/19 Roberto Spadim <roberto@spadim.com.br>:
>> ok, your hardware have:
>> cpu, memory, disk controller, disks
>>
>> and you computer have:
>> cpu, memory, disk controler (your hardware)
>>
>> if your computer cache don´t sync to your disk controller you will
>> lose information....
>>
>> check that *memory, is the volatille memory and *disk controller is
>> the non volatille memory
>> if you tell me that you will never have a *memory, and you have always
>> a non volatille memory, no problem, you will never need a kernel
>> load... just a boot loader that read previous memory information and
>> start in that point... why don´t do this? non volatille memory is not
>> as fast as volatille memory
>> got the problem?
>>
>>
>> 2011/1/19 Cory Coager <ccoager@gmail.com>:
>>> On Wed, Jan 19, 2011 at 01:19:18PM -0200, Roberto Spadim wrote:
>>>> ok,
>>>> but if you don?t sync file system (remove from memory and put in disk
>>>> controller)
>>>> you will lost information with or without a batery
>>>>
>>>> how to don?t lose information?
>>>> don?t power down you memory,cpu, disk controller (sata controler, raid
>>>> controller, or anyother) and disks (does it have a batery? a super
>>>> capacitor?)
>>>> if you power down, be sure that all memory was send to disk controller
>>>> and that disk controller have energy (batery or capacitor) to send
>>>> information to disks (they need batery or capacitor too)
>>>>
>>>> right?
>>>> so, a ups can power cpu, memory, disk controller and disks with only
>>>> one batery (not a batery for each device cpu,memory,disk,disk
>>>> controller)
>>>> the best world could be a non volatile memory (250mb/s flash 4kb
>>>> block) with the speed of volatile memory (10000mb/s ddr3 i don?t know
>>>> the block size)
>>>
>>> It would have to work the same as a hardware RAID controller.
>>> Information is first written to the cache then synced to the
>>> disk.  If the data is in the cache but not on the disk, the
>>> machine loses power, next boot up the software raid would need a
>>> way to flush the data from the ram disk to the disk.  Of course
>>> this would require the battery be in working condition, as with
>>> any hardware.
>>>
>>> Hopefully I've explained that well enough.  Perhaps it will be
>>> better to see the hardware I'm talking about:
>>> http://techreport.com/articles.x/16255
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>
>>
>>
>> --
>> Roberto Spadim
>> Spadim Technology / SPAEmpresarial
>>
>
>
>
> --
> Roberto Spadim
> Spadim Technology / SPAEmpresarial
>



-- 
Roberto Spadim
Spadim Technology / SPAEmpresarial
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: support for external persistent cache

[Cory Coager]

On Wed, Jan 19, 2011 at 02:16:08PM -0200, Roberto Spadim wrote:
> ok, your hardware have:
> cpu, memory, disk controller, disks
> 
> and you computer have:
> cpu, memory, disk controler (your hardware)
> 
> if your computer cache don?t sync to your disk controller you will
> lose information....
> 
> check that *memory, is the volatille memory and *disk controller is
> the non volatille memory
> if you tell me that you will never have a *memory, and you have always
> a non volatille memory, no problem, you will never need a kernel
> load... just a boot loader that read previous memory information and
> start in that point... why don?t do this? non volatille memory is not
> as fast as volatille memory
> got the problem?

Sorry, we're still not on the same page.  I would use both a ramdisk
and sata disk.  The ramdisk would act as a persistent cache
(with the battery) for the sata disks.  Enabling write through
would write to the cache first then sync to the sata disks.

Understand what I'm getting at now?

Re: support for external persistent cache

[Roberto Spadim]

yes, a jornaling but using ssd first and sata after
it´s not raid feature...
it´s per disk filesystem feature...
we could implement jornaling in raid too...
but´s more inteligent (easy) at application level (filesystem)
you could write first at network (if it´s faster than sata) and after
at sata (if it´s slower than network)

it´s not a raid feature... it´s a per device feature got it?
maybe a device (not raid..) for a cache division on devices
for example


/dev/sda (sata 1terabyte 100mb/s)
/dev/sdb (ssd 1gigabyte 1000mb/s)

/dev/cache_a (a mix of sata and ssd with sata size 1terabyte, and
mixed speed (memory, ssd, sata))

cache_a device should know that
early reads/write should be writen to sdb
time in time it should sync at sda

the same happens with memory (ram memory) but it´s volatille (diferent
than ssd that´s not volatille)

/dev/sdb should be sync
/dev/sda should be async (since /dev/sdb make it safe to use async)


that´s you intention? i don´t know if linux have it, anyone know?


2011/1/19 Cory Coager <ccoager@gmail.com>:
> On Wed, Jan 19, 2011 at 02:16:08PM -0200, Roberto Spadim wrote:
>> ok, your hardware have:
>> cpu, memory, disk controller, disks
>>
>> and you computer have:
>> cpu, memory, disk controler (your hardware)
>>
>> if your computer cache don?t sync to your disk controller you will
>> lose information....
>>
>> check that *memory, is the volatille memory and *disk controller is
>> the non volatille memory
>> if you tell me that you will never have a *memory, and you have always
>> a non volatille memory, no problem, you will never need a kernel
>> load... just a boot loader that read previous memory information and
>> start in that point... why don?t do this? non volatille memory is not
>> as fast as volatille memory
>> got the problem?
>
> Sorry, we're still not on the same page.  I would use both a ramdisk
> and sata disk.  The ramdisk would act as a persistent cache
> (with the battery) for the sata disks.  Enabling write through
> would write to the cache first then sync to the sata disks.
>
> Understand what I'm getting at now?
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
Roberto Spadim
Spadim Technology / SPAEmpresarial
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: support for external persistent cache

[Roberto Spadim]

if /dev/sdb get full, /dev/sda must be sync before continue...
it´s a background thread
but it works like linux memory cache, but linux use ram (volatille)
you want a volatille (ssd)

it´s like this:
http://www.seagate.com/www/en-us/products/laptops/laptop-hdd

2011/1/19 Roberto Spadim <roberto@spadim.com.br>:
> yes, a jornaling but using ssd first and sata after
> it´s not raid feature...
> it´s per disk filesystem feature...
> we could implement jornaling in raid too...
> but´s more inteligent (easy) at application level (filesystem)
> you could write first at network (if it´s faster than sata) and after
> at sata (if it´s slower than network)
>
> it´s not a raid feature... it´s a per device feature got it?
> maybe a device (not raid..) for a cache division on devices
> for example
>
>
> /dev/sda (sata 1terabyte 100mb/s)
> /dev/sdb (ssd 1gigabyte 1000mb/s)
>
> /dev/cache_a (a mix of sata and ssd with sata size 1terabyte, and
> mixed speed (memory, ssd, sata))
>
> cache_a device should know that
> early reads/write should be writen to sdb
> time in time it should sync at sda
>
> the same happens with memory (ram memory) but it´s volatille (diferent
> than ssd that´s not volatille)
>
> /dev/sdb should be sync
> /dev/sda should be async (since /dev/sdb make it safe to use async)
>
>
> that´s you intention? i don´t know if linux have it, anyone know?
>
>
> 2011/1/19 Cory Coager <ccoager@gmail.com>:
>> On Wed, Jan 19, 2011 at 02:16:08PM -0200, Roberto Spadim wrote:
>>> ok, your hardware have:
>>> cpu, memory, disk controller, disks
>>>
>>> and you computer have:
>>> cpu, memory, disk controler (your hardware)
>>>
>>> if your computer cache don?t sync to your disk controller you will
>>> lose information....
>>>
>>> check that *memory, is the volatille memory and *disk controller is
>>> the non volatille memory
>>> if you tell me that you will never have a *memory, and you have always
>>> a non volatille memory, no problem, you will never need a kernel
>>> load... just a boot loader that read previous memory information and
>>> start in that point... why don?t do this? non volatille memory is not
>>> as fast as volatille memory
>>> got the problem?
>>
>> Sorry, we're still not on the same page.  I would use both a ramdisk
>> and sata disk.  The ramdisk would act as a persistent cache
>> (with the battery) for the sata disks.  Enabling write through
>> would write to the cache first then sync to the sata disks.
>>
>> Understand what I'm getting at now?
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
>
>
> --
> Roberto Spadim
> Spadim Technology / SPAEmpresarial
>



-- 
Roberto Spadim
Spadim Technology / SPAEmpresarial
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: support for external persistent cache

[Cory Coager]

On Wed, Jan 19, 2011 at 02:44:31PM -0200, Roberto Spadim wrote:
> yes, a jornaling but using ssd first and sata after
> it?s not raid feature...
> it?s per disk filesystem feature...
> we could implement jornaling in raid too...
> but?s more inteligent (easy) at application level (filesystem)
> you could write first at network (if it?s faster than sata) and after
> at sata (if it?s slower than network)
> 
> it?s not a raid feature... it?s a per device feature got it?
> maybe a device (not raid..) for a cache division on devices
> for example
> 
> 
> /dev/sda (sata 1terabyte 100mb/s)
> /dev/sdb (ssd 1gigabyte 1000mb/s)
> 
> /dev/cache_a (a mix of sata and ssd with sata size 1terabyte, and
> mixed speed (memory, ssd, sata))
> 
> cache_a device should know that
> early reads/write should be writen to sdb
> time in time it should sync at sda
> 
> the same happens with memory (ram memory) but it?s volatille (diferent
> than ssd that?s not volatille)
> 
> /dev/sdb should be sync
> /dev/sda should be async (since /dev/sdb make it safe to use async)
> 
> 
> that?s you intention? i don?t know if linux have it, anyone know?
I would use a physical ramdisk over a ssd but thats besides the point.
They are still disk drives to the OS.

Yes, I guess it would be similar to journaling but done in RAID
itself.  I don't believe it exists yet, thats why I'm asking if
we could add support for this?

Re: support for external persistent cache

[Roberto Spadim]

check:
Key Specifications

    * 500GB, 320GB and 250GB hard drive capacity options   (hard disk
like, maybe 100mb/s with 7200rpm   - it´s you hd disk (/dev/sda in my
example) )
    * 7200-RPM spindle speed
    * 4GB SLC NAND solid state memory   (SSD like, maybe 300mb/s   -
it´s you ssd cache disk (/dev/sdb in my example) )
    * 32MB of drive-level cache  (is it ddr3? ddr3 is 10000mb/s, maybe
less, sata is 3gb/s    - it´s the linux/computer ram memory)
    * SATA 3Gb/s with Native Command Queuing (3000mb/s)

what this disk do?

operational system write to (hd) and read from (hd)

hd:
write/read in memory
if not in memory, read from ssd
if not in ssd, read from hd

write can be async
when loser energy, save memory to ssd, 32mb is fast to write, with
300mb/s ssd speed you can use a super capacitor and have 1 second of
life...
after write to ssd, write to hd





2011/1/19 Roberto Spadim <roberto@spadim.com.br>:
> if /dev/sdb get full, /dev/sda must be sync before continue...
> it´s a background thread
> but it works like linux memory cache, but linux use ram (volatille)
> you want a volatille (ssd)
>
> it´s like this:
> http://www.seagate.com/www/en-us/products/laptops/laptop-hdd
>
> 2011/1/19 Roberto Spadim <roberto@spadim.com.br>:
>> yes, a jornaling but using ssd first and sata after
>> it´s not raid feature...
>> it´s per disk filesystem feature...
>> we could implement jornaling in raid too...
>> but´s more inteligent (easy) at application level (filesystem)
>> you could write first at network (if it´s faster than sata) and after
>> at sata (if it´s slower than network)
>>
>> it´s not a raid feature... it´s a per device feature got it?
>> maybe a device (not raid..) for a cache division on devices
>> for example
>>
>>
>> /dev/sda (sata 1terabyte 100mb/s)
>> /dev/sdb (ssd 1gigabyte 1000mb/s)
>>
>> /dev/cache_a (a mix of sata and ssd with sata size 1terabyte, and
>> mixed speed (memory, ssd, sata))
>>
>> cache_a device should know that
>> early reads/write should be writen to sdb
>> time in time it should sync at sda
>>
>> the same happens with memory (ram memory) but it´s volatille (diferent
>> than ssd that´s not volatille)
>>
>> /dev/sdb should be sync
>> /dev/sda should be async (since /dev/sdb make it safe to use async)
>>
>>
>> that´s you intention? i don´t know if linux have it, anyone know?
>>
>>
>> 2011/1/19 Cory Coager <ccoager@gmail.com>:
>>> On Wed, Jan 19, 2011 at 02:16:08PM -0200, Roberto Spadim wrote:
>>>> ok, your hardware have:
>>>> cpu, memory, disk controller, disks
>>>>
>>>> and you computer have:
>>>> cpu, memory, disk controler (your hardware)
>>>>
>>>> if your computer cache don?t sync to your disk controller you will
>>>> lose information....
>>>>
>>>> check that *memory, is the volatille memory and *disk controller is
>>>> the non volatille memory
>>>> if you tell me that you will never have a *memory, and you have always
>>>> a non volatille memory, no problem, you will never need a kernel
>>>> load... just a boot loader that read previous memory information and
>>>> start in that point... why don?t do this? non volatille memory is not
>>>> as fast as volatille memory
>>>> got the problem?
>>>
>>> Sorry, we're still not on the same page.  I would use both a ramdisk
>>> and sata disk.  The ramdisk would act as a persistent cache
>>> (with the battery) for the sata disks.  Enabling write through
>>> would write to the cache first then sync to the sata disks.
>>>
>>> Understand what I'm getting at now?
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>
>>
>>
>> --
>> Roberto Spadim
>> Spadim Technology / SPAEmpresarial
>>
>
>
>
> --
> Roberto Spadim
> Spadim Technology / SPAEmpresarial
>



-- 
Roberto Spadim
Spadim Technology / SPAEmpresarial
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: support for external persistent cache

[Roberto Spadim]

i think we could make it
but not a raid feature
a linux feature
for example

we could get ramdisk, nbd (network block device), ssd, hd, scsi, raid
and select what is first cache, and what is real value

after this device is created we could use it with raid...


but.... the problem persist
if we lost energy, we can´t confirm that ram memory is sync to disk,
just if we don´t use memory cache... in other words we will work with
the best cache disk speed)

since ram can get 32GB on big home computer
and ssd can get 1terabyte on big home computer
and hdd can get more than 1terabyte on big home computer

we just need time to sync 32GB from memory to ssd
what´s the ssd speed? 300mb/s? we need 106,6666666666667 seconds to
sync memory to ssd, can you buy a ups with 107 seconds of life with a
serial cable (usb maybe) that inform that we are without energy?





2011/1/19 Cory Coager <ccoager@gmail.com>:
> On Wed, Jan 19, 2011 at 02:44:31PM -0200, Roberto Spadim wrote:
>> yes, a jornaling but using ssd first and sata after
>> it?s not raid feature...
>> it?s per disk filesystem feature...
>> we could implement jornaling in raid too...
>> but?s more inteligent (easy) at application level (filesystem)
>> you could write first at network (if it?s faster than sata) and after
>> at sata (if it?s slower than network)
>>
>> it?s not a raid feature... it?s a per device feature got it?
>> maybe a device (not raid..) for a cache division on devices
>> for example
>>
>>
>> /dev/sda (sata 1terabyte 100mb/s)
>> /dev/sdb (ssd 1gigabyte 1000mb/s)
>>
>> /dev/cache_a (a mix of sata and ssd with sata size 1terabyte, and
>> mixed speed (memory, ssd, sata))
>>
>> cache_a device should know that
>> early reads/write should be writen to sdb
>> time in time it should sync at sda
>>
>> the same happens with memory (ram memory) but it?s volatille (diferent
>> than ssd that?s not volatille)
>>
>> /dev/sdb should be sync
>> /dev/sda should be async (since /dev/sdb make it safe to use async)
>>
>>
>> that?s you intention? i don?t know if linux have it, anyone know?
> I would use a physical ramdisk over a ssd but thats besides the point.
> They are still disk drives to the OS.
>
> Yes, I guess it would be similar to journaling but done in RAID
> itself.  I don't believe it exists yet, thats why I'm asking if
> we could add support for this?
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
Roberto Spadim
Spadim Technology / SPAEmpresarial
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: support for external persistent cache

[Roberto Spadim]

hummm....
thinking again.....

what´s the today raid1 problem?
it´s based on minimal disk size
for example...
/dev/sda 1gb
/dev/sdb 1tb
we can have a 1gb raid only

what you want is:
1tb raid0
with a automatic /dev/sda to /dev/sdb write


could we call this raid0-cache?
i think we could implement it in raid....

check this fuse (filesystem) implementation: http://www.furquim.org/chironfs/
it´s based on filesystem (ok it´s a raid1, not a cache like, but´s
limited to max filesystem size, not the minimal filesystem size... ok
it´s not secure to use it since small filesystem don´t have all
information, but´s nice)

anyone want to make raid0-cache implementation?


2011/1/19 Roberto Spadim <roberto@spadim.com.br>:
> i think we could make it
> but not a raid feature
> a linux feature
> for example
>
> we could get ramdisk, nbd (network block device), ssd, hd, scsi, raid
> and select what is first cache, and what is real value
>
> after this device is created we could use it with raid...
>
>
> but.... the problem persist
> if we lost energy, we can´t confirm that ram memory is sync to disk,
> just if we don´t use memory cache... in other words we will work with
> the best cache disk speed)
>
> since ram can get 32GB on big home computer
> and ssd can get 1terabyte on big home computer
> and hdd can get more than 1terabyte on big home computer
>
> we just need time to sync 32GB from memory to ssd
> what´s the ssd speed? 300mb/s? we need 106,6666666666667 seconds to
> sync memory to ssd, can you buy a ups with 107 seconds of life with a
> serial cable (usb maybe) that inform that we are without energy?
>
>
>
>
>
> 2011/1/19 Cory Coager <ccoager@gmail.com>:
>> On Wed, Jan 19, 2011 at 02:44:31PM -0200, Roberto Spadim wrote:
>>> yes, a jornaling but using ssd first and sata after
>>> it?s not raid feature...
>>> it?s per disk filesystem feature...
>>> we could implement jornaling in raid too...
>>> but?s more inteligent (easy) at application level (filesystem)
>>> you could write first at network (if it?s faster than sata) and after
>>> at sata (if it?s slower than network)
>>>
>>> it?s not a raid feature... it?s a per device feature got it?
>>> maybe a device (not raid..) for a cache division on devices
>>> for example
>>>
>>>
>>> /dev/sda (sata 1terabyte 100mb/s)
>>> /dev/sdb (ssd 1gigabyte 1000mb/s)
>>>
>>> /dev/cache_a (a mix of sata and ssd with sata size 1terabyte, and
>>> mixed speed (memory, ssd, sata))
>>>
>>> cache_a device should know that
>>> early reads/write should be writen to sdb
>>> time in time it should sync at sda
>>>
>>> the same happens with memory (ram memory) but it?s volatille (diferent
>>> than ssd that?s not volatille)
>>>
>>> /dev/sdb should be sync
>>> /dev/sda should be async (since /dev/sdb make it safe to use async)
>>>
>>>
>>> that?s you intention? i don?t know if linux have it, anyone know?
>> I would use a physical ramdisk over a ssd but thats besides the point.
>> They are still disk drives to the OS.
>>
>> Yes, I guess it would be similar to journaling but done in RAID
>> itself.  I don't believe it exists yet, thats why I'm asking if
>> we could add support for this?
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
>
>
> --
> Roberto Spadim
> Spadim Technology / SPAEmpresarial
>



-- 
Roberto Spadim
Spadim Technology / SPAEmpresarial
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: support for external persistent cache

[Cory Coager]

On Wed, Jan 19, 2011 at 03:04:09PM -0200, Roberto Spadim wrote:
> check:
> Key Specifications
> 
>     * 500GB, 320GB and 250GB hard drive capacity options   (hard disk
> like, maybe 100mb/s with 7200rpm   - it?s you hd disk (/dev/sda in my
> example) )
>     * 7200-RPM spindle speed
>     * 4GB SLC NAND solid state memory   (SSD like, maybe 300mb/s   -
> it?s you ssd cache disk (/dev/sdb in my example) )
>     * 32MB of drive-level cache  (is it ddr3? ddr3 is 10000mb/s, maybe
> less, sata is 3gb/s    - it?s the linux/computer ram memory)
>     * SATA 3Gb/s with Native Command Queuing (3000mb/s)
> 
> what this disk do?
> 
> operational system write to (hd) and read from (hd)
> 
> hd:
> write/read in memory
> if not in memory, read from ssd
> if not in ssd, read from hd
> 
> write can be async
> when loser energy, save memory to ssd, 32mb is fast to write, with
> 300mb/s ssd speed you can use a super capacitor and have 1 second of
> life...
> after write to ssd, write to hd

Sounds about right.  Can we make it happen?

Re: support for external persistent cache

[Roberto Spadim]

maybe it´s the intention of  'mcachefs'
http://sourceforge.net/apps/mediawiki/fuse/index.php?title=FileSystems



2011/1/19 Roberto Spadim <roberto@spadim.com.br>:
> hummm....
> thinking again.....
>
> what´s the today raid1 problem?
> it´s based on minimal disk size
> for example...
> /dev/sda 1gb
> /dev/sdb 1tb
> we can have a 1gb raid only
>
> what you want is:
> 1tb raid0
> with a automatic /dev/sda to /dev/sdb write
>
>
> could we call this raid0-cache?
> i think we could implement it in raid....
>
> check this fuse (filesystem) implementation: http://www.furquim.org/chironfs/
> it´s based on filesystem (ok it´s a raid1, not a cache like, but´s
> limited to max filesystem size, not the minimal filesystem size... ok
> it´s not secure to use it since small filesystem don´t have all
> information, but´s nice)
>
> anyone want to make raid0-cache implementation?
>
>
> 2011/1/19 Roberto Spadim <roberto@spadim.com.br>:
>> i think we could make it
>> but not a raid feature
>> a linux feature
>> for example
>>
>> we could get ramdisk, nbd (network block device), ssd, hd, scsi, raid
>> and select what is first cache, and what is real value
>>
>> after this device is created we could use it with raid...
>>
>>
>> but.... the problem persist
>> if we lost energy, we can´t confirm that ram memory is sync to disk,
>> just if we don´t use memory cache... in other words we will work with
>> the best cache disk speed)
>>
>> since ram can get 32GB on big home computer
>> and ssd can get 1terabyte on big home computer
>> and hdd can get more than 1terabyte on big home computer
>>
>> we just need time to sync 32GB from memory to ssd
>> what´s the ssd speed? 300mb/s? we need 106,6666666666667 seconds to
>> sync memory to ssd, can you buy a ups with 107 seconds of life with a
>> serial cable (usb maybe) that inform that we are without energy?
>>
>>
>>
>>
>>
>> 2011/1/19 Cory Coager <ccoager@gmail.com>:
>>> On Wed, Jan 19, 2011 at 02:44:31PM -0200, Roberto Spadim wrote:
>>>> yes, a jornaling but using ssd first and sata after
>>>> it?s not raid feature...
>>>> it?s per disk filesystem feature...
>>>> we could implement jornaling in raid too...
>>>> but?s more inteligent (easy) at application level (filesystem)
>>>> you could write first at network (if it?s faster than sata) and after
>>>> at sata (if it?s slower than network)
>>>>
>>>> it?s not a raid feature... it?s a per device feature got it?
>>>> maybe a device (not raid..) for a cache division on devices
>>>> for example
>>>>
>>>>
>>>> /dev/sda (sata 1terabyte 100mb/s)
>>>> /dev/sdb (ssd 1gigabyte 1000mb/s)
>>>>
>>>> /dev/cache_a (a mix of sata and ssd with sata size 1terabyte, and
>>>> mixed speed (memory, ssd, sata))
>>>>
>>>> cache_a device should know that
>>>> early reads/write should be writen to sdb
>>>> time in time it should sync at sda
>>>>
>>>> the same happens with memory (ram memory) but it?s volatille (diferent
>>>> than ssd that?s not volatille)
>>>>
>>>> /dev/sdb should be sync
>>>> /dev/sda should be async (since /dev/sdb make it safe to use async)
>>>>
>>>>
>>>> that?s you intention? i don?t know if linux have it, anyone know?
>>> I would use a physical ramdisk over a ssd but thats besides the point.
>>> They are still disk drives to the OS.
>>>
>>> Yes, I guess it would be similar to journaling but done in RAID
>>> itself.  I don't believe it exists yet, thats why I'm asking if
>>> we could add support for this?
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>
>>
>>
>> --
>> Roberto Spadim
>> Spadim Technology / SPAEmpresarial
>>
>
>
>
> --
> Roberto Spadim
> Spadim Technology / SPAEmpresarial
>



-- 
Roberto Spadim
Spadim Technology / SPAEmpresarial
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: support for external persistent cache

[Roberto Spadim]

i´m not a good person to implement raid at kernel level, maybe at fuse level

neil have more time in this forum than me
neilb@suse.de

i´m new here =) hihi

i will make a new post this is very big...


2011/1/19 Cory Coager <ccoager@gmail.com>:
> On Wed, Jan 19, 2011 at 03:04:09PM -0200, Roberto Spadim wrote:
>> check:
>> Key Specifications
>>
>>     * 500GB, 320GB and 250GB hard drive capacity options   (hard disk
>> like, maybe 100mb/s with 7200rpm   - it?s you hd disk (/dev/sda in my
>> example) )
>>     * 7200-RPM spindle speed
>>     * 4GB SLC NAND solid state memory   (SSD like, maybe 300mb/s   -
>> it?s you ssd cache disk (/dev/sdb in my example) )
>>     * 32MB of drive-level cache  (is it ddr3? ddr3 is 10000mb/s, maybe
>> less, sata is 3gb/s    - it?s the linux/computer ram memory)
>>     * SATA 3Gb/s with Native Command Queuing (3000mb/s)
>>
>> what this disk do?
>>
>> operational system write to (hd) and read from (hd)
>>
>> hd:
>> write/read in memory
>> if not in memory, read from ssd
>> if not in ssd, read from hd
>>
>> write can be async
>> when loser energy, save memory to ssd, 32mb is fast to write, with
>> 300mb/s ssd speed you can use a super capacitor and have 1 second of
>> life...
>> after write to ssd, write to hd
>
> Sounds about right.  Can we make it happen?
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
Roberto Spadim
Spadim Technology / SPAEmpresarial
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: support for external persistent cache

[Cory Coager]

On Wed, Jan 19, 2011 at 03:15:40PM -0200, Roberto Spadim wrote:
> hummm....
> thinking again.....
> 
> what?s the today raid1 problem?
> it?s based on minimal disk size
> for example...
> /dev/sda 1gb
> /dev/sdb 1tb
> we can have a 1gb raid only
> 
> what you want is:
> 1tb raid0
> with a automatic /dev/sda to /dev/sdb write
> 
> 
> could we call this raid0-cache?
> i think we could implement it in raid....
> 
> check this fuse (filesystem) implementation: http://www.furquim.org/chironfs/
> it?s based on filesystem (ok it?s a raid1, not a cache like, but?s
> limited to max filesystem size, not the minimal filesystem size... ok
> it?s not secure to use it since small filesystem don?t have all
> information, but?s nice)
> 
> anyone want to make raid0-cache implementation?

I have two comments on this.

I don't think the cache device should be the same size as your
disk storage.  It should be a small cache size similar to
hardware RAID.

Also, we'd need a way to flush the data from the cache device
on bootup.

Re: support for external persistent cache

[Roberto Spadim]

please check my email =)
let´s end this email?

2011/1/19 Cory Coager <ccoager@gmail.com>:
> On Wed, Jan 19, 2011 at 03:15:40PM -0200, Roberto Spadim wrote:
>> hummm....
>> thinking again.....
>>
>> what?s the today raid1 problem?
>> it?s based on minimal disk size
>> for example...
>> /dev/sda 1gb
>> /dev/sdb 1tb
>> we can have a 1gb raid only
>>
>> what you want is:
>> 1tb raid0
>> with a automatic /dev/sda to /dev/sdb write
>>
>>
>> could we call this raid0-cache?
>> i think we could implement it in raid....
>>
>> check this fuse (filesystem) implementation: http://www.furquim.org/chironfs/
>> it?s based on filesystem (ok it?s a raid1, not a cache like, but?s
>> limited to max filesystem size, not the minimal filesystem size... ok
>> it?s not secure to use it since small filesystem don?t have all
>> information, but?s nice)
>>
>> anyone want to make raid0-cache implementation?
>
> I have two comments on this.
>
> I don't think the cache device should be the same size as your
> disk storage.  It should be a small cache size similar to
> hardware RAID.
>
> Also, we'd need a way to flush the data from the cache device
> on bootup.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
Roberto Spadim
Spadim Technology / SPAEmpresarial
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html