Re: 2.4.35 SMP: ext3_readdir: bad entry in directory #323888: rec_len is smaller than minimal

[Willy Tarreau <w@1wt.eu>]

Hi Ville,

On Tue, Sep 18, 2007 at 07:33:26PM +0300, Ville Herva wrote:
> On Tue, Sep 18, 2007 at 06:22:56PM +0200, you [Jan Kara] wrote:
> > > Sorry for the sparse details, but when you have these kind of problems on
> > > live servers, you tend to forget the debuggability...
> >   Yes, I can understand that :). It's just that now it's hard to find
> > out what has really happened. Anyway, thanks for your report.
> 
> If we are really lucky or unlucky it will happen again. 
> 
> Zeroed-out block just might be a kernel problem (SMP race, whatever) -
> random corruption would more likely be a hardware problem. There were no IO
> error either. But, 2.4 ext3 has been pretty extensively tested, so I don't
> suppose that's likely either. And judging from
> http://www.kernel.org/pub/linux/kernel/v2.4/ChangeLog-2.4.3[45] there
> haven't been many changes to ext3 lately either.

Thanks for your report. Unfortunately, I've rechecked the recent changelogs
and see nothing related either. At least, in order to keep trace of the
incident, would you please post some info about your config (CPU, RAM,
chipset, .config, gcc, and any possible patches you may have applied) ?
Maybe some of these info may remind old bad memories to some people.

Also, do you know if this server has ECC memory ? I would more easily
bet for side effects of one random bit flip in memory than for some
massive block corruption.

I vaguely remember about very old reports of people sometimes observing
zeroed out blocks during writes, which were attributed to chipset bugs
if my memory serves me. But I would rule this out as recent chipsets
look more stable than 5-10 years ago !

Regards,
Willy