RFC: detection of silent corruption via ATA long sector reads

RFC: detection of silent corruption via ATA long sector reads

[Greg Freemyer]

All,

On the mdraid list, there was a recent thread about using raid
functionality to detect / repair silent corruption.

The issues brought up were that a lot of silent data corruption occurs
when cables, controllers, power supplies, ram, cache, etc. goes bad.

It made me think about another option for detecting silent corruption
I have not seen discussed, but maybe I missed it.

Aiui, the ATA spec allows for the reading of a long sector as well as
the normal 512 byte sector.  When you get a long sector you also get
the CRC (or whatever checksum data there is on the disk that allows
the drive itself to detect media errors).

I don't have any idea how easy or hard it would be to do, but I would
like to see the entire block subsystem enhanced to optionally allow
long sector reads to be used in a "paranoid" fashion.

Effectively it would be:

1) Read long sector from drive:  verify CRC in kernel.  This tests
most everything on the i/o path.

2) maintain CRC type information in block subsystem.  Verify no
corruption just before handing off to userspace.  This would
potentially identify CPU/cache/RAM failures.

Mark Lord has implemented long sector reads via hdparm.  Mark can you
comment on the feasibility of this idea?

Thanks
Greg
-- 
Greg Freemyer
Litigation Triage Solutions Specialist
http://www.linkedin.com/in/gregfreemyer
First 99 Days Litigation White Paper -
http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf

The Norcross Group
The Intersection of Evidence & Technology
http://www.norcrossgroup.com

Re: RFC: detection of silent corruption via ATA long sector reads

[Robert Hancock]

Greg Freemyer wrote:
> All,
> 
> On the mdraid list, there was a recent thread about using raid
> functionality to detect / repair silent corruption.
> 
> The issues brought up were that a lot of silent data corruption occurs
> when cables, controllers, power supplies, ram, cache, etc. goes bad.
> 
> It made me think about another option for detecting silent corruption
> I have not seen discussed, but maybe I missed it.
> 
> Aiui, the ATA spec allows for the reading of a long sector as well as
> the normal 512 byte sector.  When you get a long sector you also get
> the CRC (or whatever checksum data there is on the disk that allows
> the drive itself to detect media errors).
> 
> I don't have any idea how easy or hard it would be to do, but I would
> like to see the entire block subsystem enhanced to optionally allow
> long sector reads to be used in a "paranoid" fashion.
> 
> Effectively it would be:
> 
> 1) Read long sector from drive:  verify CRC in kernel.  This tests
> most everything on the i/o path.
> 
> 2) maintain CRC type information in block subsystem.  Verify no
> corruption just before handing off to userspace.  This would
> potentially identify CPU/cache/RAM failures.

Even if the drive supports those commands the problem is the CRC/ECC 
data is in a vendor-specific format, so it couldn't be processed 
generically.

RE: Re: RFC: detection of silent corruption via ATA long sector reads

[David Lethe]

> -----Original Message-----
> From: linux-raid-owner@vger.kernel.org [mailto:linux-raid-
> owner@vger.kernel.org] On Behalf Of Robert Hancock
> Sent: Friday, December 26, 2008 4:16 PM
> To: linux-raid@vger.kernel.org
> Cc: linux-kernel@vger.kernel.org
> Subject: Re: RFC: detection of silent corruption via ATA long sector
> reads
> 
> Greg Freemyer wrote:
> > All,
> >
> > On the mdraid list, there was a recent thread about using raid
> > functionality to detect / repair silent corruption.
> >
> > The issues brought up were that a lot of silent data corruption
> occurs
> > when cables, controllers, power supplies, ram, cache, etc. goes bad.
> >
> > It made me think about another option for detecting silent
corruption
> > I have not seen discussed, but maybe I missed it.
> >
> > Aiui, the ATA spec allows for the reading of a long sector as well
as
> > the normal 512 byte sector.  When you get a long sector you also get
> > the CRC (or whatever checksum data there is on the disk that allows
> > the drive itself to detect media errors).
> >
> > I don't have any idea how easy or hard it would be to do, but I
would
> > like to see the entire block subsystem enhanced to optionally allow
> > long sector reads to be used in a "paranoid" fashion.
> >
> > Effectively it would be:
> >
> > 1) Read long sector from drive:  verify CRC in kernel.  This tests
> > most everything on the i/o path.
> >
> > 2) maintain CRC type information in block subsystem.  Verify no
> > corruption just before handing off to userspace.  This would
> > potentially identify CPU/cache/RAM failures.
> 
> Even if the drive supports those commands the problem is the CRC/ECC
> data is in a vendor-specific format, so it couldn't be processed
> generically.
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid"
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Many of the RAID appliance/subsystem vendors format the disks to 520 or
528
Bytes/sector, but expose 512-byte blocks to the user.  The ECC logic is
done
by the firmware ... or if this ever gets implemented, would be done by
the
LINUX kernel.  True there are some issues with many of the cheap
consumer
class drives not supporting anything but 512-byte blocks, but we
shouldn't
code to lowest common denominator.

With 1TB SATA disks selling for $99, then it isn't as if the extra 8-16
bytes
for ECC on the disk drive is going to be a problem.

David

Re: RFC: detection of silent corruption via ATA long sector reads

[Mark Lord]

Greg Freemyer wrote:
> All,
> 
> On the mdraid list, there was a recent thread about using raid
> functionality to detect / repair silent corruption.
> 
> The issues brought up were that a lot of silent data corruption occurs
> when cables, controllers, power supplies, ram, cache, etc. goes bad.
> 
> It made me think about another option for detecting silent corruption
> I have not seen discussed, but maybe I missed it.
> 
> Aiui, the ATA spec allows for the reading of a long sector as well as
> the normal 512 byte sector.  When you get a long sector you also get
> the CRC (or whatever checksum data there is on the disk that allows
> the drive itself to detect media errors).
> 
> I don't have any idea how easy or hard it would be to do, but I would
> like to see the entire block subsystem enhanced to optionally allow
> long sector reads to be used in a "paranoid" fashion.
> 
> Effectively it would be:
> 
> 1) Read long sector from drive:  verify CRC in kernel.  This tests
> most everything on the i/o path.
> 
> 2) maintain CRC type information in block subsystem.  Verify no
> corruption just before handing off to userspace.  This would
> potentially identify CPU/cache/RAM failures.
> 
> Mark Lord has implemented long sector reads via hdparm.  Mark can you
> comment on the feasibility of this idea?
..

The ATA READ/WRITE LONG commands have been obsoleted in the past few ATA specs,
even though most drives continue to implement them.

But not a good avenue.

There's a separate effort, involving drive vendors and kernel hackers,
to provide end-to-end CRC protection of data.  I forget what it was called,
but that's the future of this stuff for high-reliability requirements.

Cheers

Re: RFC: detection of silent corruption via ATA long sector reads

[Sitsofe Wheeler]

> Is there a more comprehensive write-up / resource that describes the
> current status of the overall INTEGRITY support is, especially as it
> relates to ATA devices?


Did you check the kernel notes on kernelnewbies when the feature went in - 
http://kernelnewbies.org/Linux_2_6_27 ?


      

Re: RFC: detection of silent corruption via ATA long sector reads

[Greg Freemyer]

On Fri, Jan 2, 2009 at 3:43 PM, Sitsofe Wheeler <sitsofe@yahoo.com> wrote:
>> Is there a more comprehensive write-up / resource that describes the
>> current status of the overall INTEGRITY support is, especially as it
>> relates to ATA devices?
>
>
> Did you check the kernel notes on kernelnewbies when the feature went in -
> http://kernelnewbies.org/Linux_2_6_27 ?

Interesting read, but it does not really answer the questions I posed.

I did look through the 2.6.27 source I have handy and the only call to
blk_integrity_register() is in./drivers/scsi/sd_dif.c.

That leaves me with the impression that there are not any ATA devices
claiming support yet.

Greg
-- 
Greg Freemyer
Litigation Triage Solutions Specialist
http://www.linkedin.com/in/gregfreemyer
First 99 Days Litigation White Paper -
http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf

The Norcross Group
The Intersection of Evidence & Technology
http://www.norcrossgroup.com