Write cache on SATA drives?

Write cache on SATA drives?

[Edwards, Scott (GE Healthcare)]

Hello,

Can anyone tell me if the write cache on SATA drives are handled such that journaling file systems work correctly?  Back in the 2.6.5 - 2.6.7 time frame I had to disable the write cache to get ext3 to not trash things when the power was lost.  With kernel 2.6.10 would I still need to disable the write cache?

Thanks
  -Scott

Re: Write cache on SATA drives?

[Jens Axboe]

On Thu, Jun 30 2005, Edwards, Scott (GE Healthcare) wrote:
> Hello,
> 
> Can anyone tell me if the write cache on SATA drives are handled such
> that journaling file systems work correctly?  Back in the 2.6.5 -
> 2.6.7 time frame I had to disable the write cache to get ext3 to not
> trash things when the power was lost.  With kernel 2.6.10 would I
> still need to disable the write cache?

With 2.6.12.x it should work, if you use ext3 or reiser and the
appropriate mount options (-o barrier=1 for ext3, barrier=flush for
reiserfs). 2.6.11 and earlier does not work on SATA.

-- 
Jens Axboe

Re: Write cache on SATA drives?

[Matthias Andree]

Jens Axboe <axboe@suse.de> writes:

> On Thu, Jun 30 2005, Edwards, Scott (GE Healthcare) wrote:
>> Hello,
>> 
>> Can anyone tell me if the write cache on SATA drives are handled such
>> that journaling file systems work correctly?  Back in the 2.6.5 -
>> 2.6.7 time frame I had to disable the write cache to get ext3 to not
>> trash things when the power was lost.  With kernel 2.6.10 would I
>> still need to disable the write cache?
>
> With 2.6.12.x it should work, if you use ext3 or reiser and the
> appropriate mount options (-o barrier=1 for ext3, barrier=flush for
> reiserfs). 2.6.11 and earlier does not work on SATA.

I presume this means "libata" exclusively, no?

Which is the oldest version where this works

1. for SCSI (perhaps by adaptor)?

2. for traditional IDE (such as VIA 82*, PIIX_*)?

3. for RAID?

4. for XFS or JFS?

I have started to compile a list of requirements,
<http://home.pages.de/~mandree/linux/kernel/safe-write-caches.html>

Contributions solicited.

-- 
Matthias Andree

Re: Write cache on SATA drives?

[Jens Axboe]

On Fri, Jul 01 2005, Matthias Andree wrote:
> Jens Axboe <axboe@suse.de> writes:
> 
> > On Thu, Jun 30 2005, Edwards, Scott (GE Healthcare) wrote:
> >> Hello,
> >> 
> >> Can anyone tell me if the write cache on SATA drives are handled such
> >> that journaling file systems work correctly?  Back in the 2.6.5 -
> >> 2.6.7 time frame I had to disable the write cache to get ext3 to not
> >> trash things when the power was lost.  With kernel 2.6.10 would I
> >> still need to disable the write cache?
> >
> > With 2.6.12.x it should work, if you use ext3 or reiser and the
> > appropriate mount options (-o barrier=1 for ext3, barrier=flush for
> > reiserfs). 2.6.11 and earlier does not work on SATA.
> 
> I presume this means "libata" exclusively, no?

It means "SCSI without queueing" which I'm assuming is basically only
libata at this moment, at least if you count setups that have a non-zero
userbase :)

> Which is the oldest version where this works
> 
> 1. for SCSI (perhaps by adaptor)?

2.6.12, with the above restriction. I hope to lift that for .13/14, with
Tejuns barrier updates.

> 2. for traditional IDE (such as VIA 82*, PIIX_*)?

In SUSE kernels, for many years. Since 2.6.7/8'ish in Linus' kernels.

> 3. for RAID?

Not really safe, only for fsync.

> 4. for XFS or JFS?

XFS and JFS need minor updates, similar to what ext3 and reiser got.
Should take someone familiar with the code no more than an hour to do,
nobody has done it yet though.

> <http://home.pages.de/~mandree/linux/kernel/safe-write-caches.html>

One important update, the 'oe' in my name is not an ascii safe variant
of the real name, it really is spelled that way :)

-- 
Jens Axboe

Re: Write cache on SATA drives?

[Matthias Andree]

Jens Axboe schrieb am 2005-07-01:

> On Fri, Jul 01 2005, Matthias Andree wrote:
> > Jens Axboe <axboe@suse.de> writes:
> > 
> > > On Thu, Jun 30 2005, Edwards, Scott (GE Healthcare) wrote:
> > >> Hello,
> > >> 
> > >> Can anyone tell me if the write cache on SATA drives are handled such
> > >> that journaling file systems work correctly?  Back in the 2.6.5 -
> > >> 2.6.7 time frame I had to disable the write cache to get ext3 to not
> > >> trash things when the power was lost.  With kernel 2.6.10 would I
> > >> still need to disable the write cache?
> > >
> > > With 2.6.12.x it should work, if you use ext3 or reiser and the
> > > appropriate mount options (-o barrier=1 for ext3, barrier=flush for
> > > reiserfs). 2.6.11 and earlier does not work on SATA.
> > 
> > I presume this means "libata" exclusively, no?
> 
> It means "SCSI without queueing" which I'm assuming is basically only
> libata at this moment, at least if you count setups that have a non-zero
> userbase :)
> 
> > Which is the oldest version where this works
> > 
> > 1. for SCSI (perhaps by adaptor)?
> 
> 2.6.12, with the above restriction. I hope to lift that for .13/14, with
> Tejuns barrier updates.

The "SCSI without queueing" isn't clear to me. What system exactly is
meant "without queueing"?

Does this mean SCSI with TCQ is safe?

Does this means SCSI is safe only when TCQ isn't used?

Does this apply to internals of the SCSI host adaptor driver?

I'm not acquainted with kernel/block I/O queueing internals.

> > 2. for traditional IDE (such as VIA 82*, PIIX_*)?
> 
> In SUSE kernels, for many years. Since 2.6.7/8'ish in Linus' kernels.

I don't care for vendor kernels. Is listing 2.6.8 safe?

> > <http://home.pages.de/~mandree/linux/kernel/safe-write-caches.html>
> 
> One important update, the 'oe' in my name is not an ascii safe variant
> of the real name, it really is spelled that way :)

Fixed.

Thanks for your help.

-- 
Matthias Andree

Re: Write cache on SATA drives?

[Jens Axboe]

On Fri, Jul 01 2005, Matthias Andree wrote:
> Jens Axboe schrieb am 2005-07-01:
> 
> > On Fri, Jul 01 2005, Matthias Andree wrote:
> > > Jens Axboe <axboe@suse.de> writes:
> > > 
> > > > On Thu, Jun 30 2005, Edwards, Scott (GE Healthcare) wrote:
> > > >> Hello,
> > > >> 
> > > >> Can anyone tell me if the write cache on SATA drives are handled such
> > > >> that journaling file systems work correctly?  Back in the 2.6.5 -
> > > >> 2.6.7 time frame I had to disable the write cache to get ext3 to not
> > > >> trash things when the power was lost.  With kernel 2.6.10 would I
> > > >> still need to disable the write cache?
> > > >
> > > > With 2.6.12.x it should work, if you use ext3 or reiser and the
> > > > appropriate mount options (-o barrier=1 for ext3, barrier=flush for
> > > > reiserfs). 2.6.11 and earlier does not work on SATA.
> > > 
> > > I presume this means "libata" exclusively, no?
> > 
> > It means "SCSI without queueing" which I'm assuming is basically only
> > libata at this moment, at least if you count setups that have a non-zero
> > userbase :)
> > 
> > > Which is the oldest version where this works
> > > 
> > > 1. for SCSI (perhaps by adaptor)?
> > 
> > 2.6.12, with the above restriction. I hope to lift that for .13/14, with
> > Tejuns barrier updates.
> 
> The "SCSI without queueing" isn't clear to me. What system exactly is
> meant "without queueing"?
> 
> Does this mean SCSI with TCQ is safe?
> 
> Does this means SCSI is safe only when TCQ isn't used?
> 
> Does this apply to internals of the SCSI host adaptor driver?
> 
> I'm not acquainted with kernel/block I/O queueing internals.

SCSI without TCQ or with TCQ depth=1 should be safe, provided the driver
never reorders a command once it has received it.

> > > 2. for traditional IDE (such as VIA 82*, PIIX_*)?
> > 
> > In SUSE kernels, for many years. Since 2.6.7/8'ish in Linus' kernels.
> 
> I don't care for vendor kernels. Is listing 2.6.8 safe?

Check the changelogs, I can't remember exactly if it was 2.6.7 or 2.6.8
(or perhaps .6...).

-- 
Jens Axboe

Re: Write cache on SATA drives?

[Matthias Andree]

Jens Axboe <axboe@suse.de> writes:

>> I'm not acquainted with kernel/block I/O queueing internals.
>
> SCSI without TCQ or with TCQ depth=1 should be safe, provided the driver
> never reorders a command once it has received it.

Wouldn't we rather use *deep* TCQ and set write cache enable to 0 then?

Does anybody have benchmarks with TCQ depth = 1/WCE = 1 vs. TCQ = <deep>
(64+)/WCE = 0? I only have ancient SCSI drives in test machines, so my
testing wouldn't be representative.

>> > > 2. for traditional IDE (such as VIA 82*, PIIX_*)?
>> > 
>> > In SUSE kernels, for many years. Since 2.6.7/8'ish in Linus' kernels.
>> 
>> I don't care for vendor kernels. Is listing 2.6.8 safe?
>
> Check the changelogs, I can't remember exactly if it was 2.6.7 or 2.6.8
> (or perhaps .6...).

Hm. Any buzz^Wkeywords to look for? "barrier" doesn't seem to be the
right word, it yields only memory barriers and an as-iosched fix in
2.6.7.  I checked ChangeLog-2.6.[56789].

-- 
Matthias Andree

Re: Write cache on SATA drives?

[Jens Axboe]

On Sat, Jul 02 2005, Matthias Andree wrote:
> Jens Axboe <axboe@suse.de> writes:
> 
> >> I'm not acquainted with kernel/block I/O queueing internals.
> >
> > SCSI without TCQ or with TCQ depth=1 should be safe, provided the driver
> > never reorders a command once it has received it.
> 
> Wouldn't we rather use *deep* TCQ and set write cache enable to 0 then?

Depends, if you can't use ordered tags it wont help you.

> Does anybody have benchmarks with TCQ depth = 1/WCE = 1 vs. TCQ = <deep>
> (64+)/WCE = 0? I only have ancient SCSI drives in test machines, so my
> testing wouldn't be representative.

I posted some for NCQ, see lkml/linux-ide.

> >> > > 2. for traditional IDE (such as VIA 82*, PIIX_*)?
> >> > 
> >> > In SUSE kernels, for many years. Since 2.6.7/8'ish in Linus' kernels.
> >> 
> >> I don't care for vendor kernels. Is listing 2.6.8 safe?
> >
> > Check the changelogs, I can't remember exactly if it was 2.6.7 or 2.6.8
> > (or perhaps .6...).
> 
> Hm. Any buzz^Wkeywords to look for? "barrier" doesn't seem to be the
> right word, it yields only memory barriers and an as-iosched fix in
> 2.6.7.  I checked ChangeLog-2.6.[56789].

Grep for blk_queue_ordered or similar in the patch.

-- 
Jens Axboe

Re: Write cache on SATA drives?

[Matthias Andree]

Jens Axboe <axboe@suse.de> writes:

>> Hm. Any buzz^Wkeywords to look for? "barrier" doesn't seem to be the
>> right word, it yields only memory barriers and an as-iosched fix in
>> 2.6.7.  I checked ChangeLog-2.6.[56789].
>
> Grep for blk_queue_ordered or similar in the patch.

patch-2.6.9 adds this function -- and a call to it to ide-disk.c. Have
reiserfs and ext3fs supported this from the very beginning (i. e. 2.6.9)
or is this an issue that affected only the lower layers anyhow?

/**
 * blk_queue_ordered - does this queue support ordered writes
 * @q:     the request queue
 * @flag:  see below
 *
 * Description:
 *   For journalled file systems, doing ordered writes on a commit
 *   block instead of explicitly doing wait_on_buffer (which is bad
 *   for performance) can be a big win. Block drivers supporting this
 *   feature should call this function and indicate so.
 *
 **/
void blk_queue_ordered(request_queue_t *q, int flag)
...

-- 
Matthias Andree

Re: Write cache on SATA drives?

[Jens Axboe]

On Sun, Jul 03 2005, Matthias Andree wrote:
> Jens Axboe <axboe@suse.de> writes:
> 
> >> Hm. Any buzz^Wkeywords to look for? "barrier" doesn't seem to be the
> >> right word, it yields only memory barriers and an as-iosched fix in
> >> 2.6.7.  I checked ChangeLog-2.6.[56789].
> >
> > Grep for blk_queue_ordered or similar in the patch.
> 
> patch-2.6.9 adds this function -- and a call to it to ide-disk.c. Have

Ah ok, 2.6.9 it is then.

> reiserfs and ext3fs supported this from the very beginning (i. e. 2.6.9)
> or is this an issue that affected only the lower layers anyhow?

They have.

-- 
Jens Axboe