Putrid Elevator Behavior 2.4.18/19

Jeff V. Merkey
2002-03-20T18:49:58Z

Jens/Linux,

The elevator code is malfunctioning in 2.4.18/19-pre when we start
reaching the upward limits with multiple 3Ware adapters
running together. We started seeing the problem when we went to
64 K aligned writes with sustained > 200 MB/S writes to
multiple 3Ware adapters.

We have verified that the 3Ware adapters are not holding the request
off, but that one of the requests is getting severely starved and
does not get posted to the 3Ware adapters until thousands of IOs
have gone before it.

The basic symptom is a lower offset 4K write gets hung in the elevator
as it traverses a very long list of requests being written linearly
to a disk device. Both Darren and I have seen this problem in NetWare
with remirroring, which is why we went to the A/B alternating
list to prevent this type of starvation. There are a very small number
of reads being posted during this test to update meta-data.

The data that is being held off is meta-data that occupies a lower sector
offset on the device. This startvation error is very troublesome and
results in certain sectors not being freed up as anticipated, which
results in a fatal error for our system. The elevator ends of getting
very far behind.

By way of example, this delayed write is held off for several **MINUTES**.
This is severaly broken.

Please let me know what other information you would like Darren and I
to run down and provide. We are at present coding some changes into
your elevator to implement an A and B list so this startvation problem
is completely avoided.

Please advise,

Jeff