Blockbusting news, this is important (Re: Why are bad disk sectors numbered strangely, and what happens to them?)

["Norman Diamond" <ndiamond@wta.att.ne.jp>]

Friends in the disk drive section at Toshiba said this:

When a drive tries to read a block, if it detects errors, it retries up to
255 times.  If a retry succeeds then the block gets reallocated.  IF 255
RETRIES FAIL THEN THE BLOCK DOES NOT GET REALLOCATED.

This was so unbelievable to that I had to confirm this with them in
different words.  In case of a temporary error, the drive provides the
recovered data as the result of the read operation and the drive writes the
data to a reallocated sector.  In case of a permanent error, the block is
assumed bad, and of course the data are lost.  Since the data are assumed
lost, the drive keeps the defective LBA sector number associated with the
same defective physical block and it does not reallocate the defective
block.

I explained to them why the LBA sector number should still get reallocated
even though the data are lost.  When the sector isn't reallocated, I could
repartition the drive and reformat the partition and the OS wouldn't know
about the defective block so the OS would try again to use it.  At first
they did not believe I could do this, but I explained to them that I'm still
able to delete partitions and create new partitions etc., and then they
understood.

They also said that a write operation has a chance of getting the bad block
reallocated.  The conditions for reallocation on write are similar but not
identical to the conditions for reallocate on read.  During a write
operation if a sector is determined to be permanently bad (255 failing
retries) then it is likely to be reallocated, unlike a read.  But I'm not
sure if this is guaranteed or not.  We agreed that we should try it on my
bad sector, but if the drive again detects a permantent error then it will
not reallocate the sector.  First I still want to find which file contains
the sector; I haven't had time for this on weekdays.

When I ran the "long" S.M.A.R.T. self-test, the number of reallocated
sectors and number of reallocation events both increased from 1 to 2, but
the known bad sector remained bad.  This is entirely because of the behavior
as designed.  The self-test detected a temporary error in some other
unrelated sector, rescued the data in that unreported sector number, and
reallocated it.  That was only a coincidence.  The known bad sector was
detected yet again as permanently bad and was not reallocated.

In this mailing list there has been some discussion of whether file systems
should keep lists of known bad blocks and hide those bad blocks from
ordinary operations in ordinary usage.  Of course historically this was
always necessary.  As someone else mentioned, and I've done it too, when
formatting a disk drive, type in the list of known bad block numbers that
were printed on a piece of paper that came with the drive.

In modern times, some people think that this shouldn't be necessary because
the drive already does its best to reallocate bad blocks.  WRONG.  THE BAD
BLOCK LIST REMAINS AS NECESSARY AS IT ALWAYS WAS.

This design might change in the future, but it might not.  My friends are
afraid that they might lose their jobs if they try to suggest such a change
in the high-level design of disk drive corporate politics.  I only hope this
posting doesn't get them fired.  (This is not a frivolous concern by the
way.  The myth of lifetime employment is a less pervasive myth than it used
to be, and Toshiba is pretty much average in both world and Japanese
standards for corporate politics.)

Regarding finding which file contains the known bad sector, someone in this
mailing list said that the badblocks program could help, but the manual page
for the badblocks program doesn't give any clues as to how it would help.
I'm still doing find of all files in the partition and cp them to /dev/null.

Meanwhile, yes we do need to record those bad block lists and try to never
let them get allocated to user-visible files.