Re: mptbase Code={Abort} (LSI SAS1068 PCI-X Fusion-MPT)

Re: mptbase Code={Abort} (LSI SAS1068 PCI-X Fusion-MPT)

[Richard Scobie]

I received a reply from LSI:

"It seems there is some bad cable or connectivity or bad device in the
topology. The loginfo 0x31123000 says the I/O is aborted due to a Stuck
Link. The loginfo 0x31120403 specifies the I/O is aborted due to frame
transfer errors."

So far doing large dd reads and writes, I have been unable to generate 
any of these errors, it is only during the initial md RAID5 sync up, so 
I guess this is going to be a little tricky to pin down.

Regards,

Richard

RE: mptbase Code={Abort} (LSI SAS1068 PCI-X Fusion-MPT)

[Moore, Eric]

On Tuesday, April 22, 2008 1:29 PM,  Richard Scobie wrote:
> I received a reply from LSI:
> 
> "It seems there is some bad cable or connectivity or bad device in the
> topology. The loginfo 0x31123000 says the I/O is aborted due 
> to a Stuck
> Link. The loginfo 0x31120403 specifies the I/O is aborted due to frame
> transfer errors."
> 
> So far doing large dd reads and writes, I have been unable to 
> generate 
> any of these errors, it is only during the initial md RAID5 
> sync up, so 
> I guess this is going to be a little tricky to pin down.
> 

The loginfo bits are defined in a header inside the linux kernel
sources.  

Location:  /usr/src/linux/drivers/message/fusion/lsi/mpi_log_sas.h

If you search for 3000, you will find
PL_LOGINFO_CODE_WRONG_REL_OFF_OR_FRAME_LENGTH.

There's no match for 403, so I guess this iz a new define not included
in my version of the headers.

Eric

Re: mptbase Code={Abort} (LSI SAS1068 PCI-X Fusion-MPT)

[Richard Scobie]

Moore, Eric wrote:

> The loginfo bits are defined in a header inside the linux kernel
> sources.  
> 
> Location:  /usr/src/linux/drivers/message/fusion/lsi/mpi_log_sas.h
> 
> If you search for 3000, you will find
> PL_LOGINFO_CODE_WRONG_REL_OFF_OR_FRAME_LENGTH.
> 
> There's no match for 403, so I guess this iz a new define not included
> in my version of the headers.
> 
> Eric

Thanks Eric. I doubt I can source a new backplane to swap out, so I'll 
try to find out if there is a more recent firmware for it and try that.

Regards,

Richard

Re: mptbase Code={Abort} (LSI SAS1068 PCI-X Fusion-MPT)

[Sebastian Wasilewski]

[-- Attachment #1: Type: text/plain, Size: 1239 bytes --]

Richard Scobie wrote:
> I received a reply from LSI:
> 
> "It seems there is some bad cable or connectivity or bad device in the
> topology. The loginfo 0x31123000 says the I/O is aborted due to a Stuck
> Link. The loginfo 0x31120403 specifies the I/O is aborted due to frame
> transfer errors."
> 
> So far doing large dd reads and writes, I have been unable to generate 
> any of these errors, it is only during the initial md RAID5 sync up, so 
> I guess this is going to be a little tricky to pin down.
> 

Thanks for the info.

There is still one thing I do not understand. I have made a test -- I 
have connected three SATA hard drives directly to the controller using 
cable supplied with the controller. The error still occurred. Then I 
have changed the cable to the one supplied with other controller, as 
well as hard drives (but model was still the same) and the error message 
was still occurring.

I have changed controller (I have a few of them) but again, the problem 
still existed.

The only unusual thing I have noticed during the test, was that all 
ST31000340NS drives  worked at 1.5Gbps, not 3.0Gbps link speed.

I have made plenty of the dd tests and all were fine (both with NCQ and 
without).


Regards,
Sebastian

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/x-pkcs7-signature, Size: 4968 bytes --]

Re: mptbase Code={Abort} (LSI SAS1068 PCI-X Fusion-MPT)

[Richard Scobie]

Sebastian Wasilewski said:

There is still one thing I do not understand. I have made a test -- I
have connected three SATA hard drives directly to the controller using
cable supplied with the controller. The error still occurred. Then I
have changed the cable to the one supplied with other controller, as
well as hard drives (but model was still the same) and the error message
was still occurring.

I have changed controller (I have a few of them) but again, the problem
still existed.

------------------------------

If anyone else can replicate these errors using Sebastian's setup above 
I would interested in hearing.

I have a case open with LSI but apparently my HBA is a "prototype" board 
and this is complicating things. I have had no indication whether they 
can replicate this issue or not.

If anyone else from LSI is on this list and can wade through SR# 
2-209998499  I'd be interested in any suggestions. Time is running out 
for me and I will probably have to go with someone elses HBA.

Apologies for this mail not threading correctly, as I am not subscribed. 
The start of it can be found here:

http://marc.info/?l=linux-scsi&m=120805128416791&w=2

and carries on here:

http://marc.info/?l=linux-scsi&m=120869748422095&w=2

Regards,

Richard

RE: mptbase Code={Abort} (LSI SAS1068 PCI-X Fusion-MPT)

[Moore, Eric]

> If anyone else from LSI is on this list and can wade through SR# 
> 2-209998499  I'd be interested in any suggestions. Time is 
> running out 
> for me and I will probably have to go with someone elses HBA.
> 

Richard, those SR issues are not accessable by developers at LSI.
Those are created by the tech support group, used internally within the
support group.    The developers is what your going to find on the
linux-scsi@ forum, like me and Sathya.  We are using clear case, which
is a different database.    If you give me the contact name whom
assigned you that SR number, I will see what I can do.

Eric

Re: mptbase Code={Abort} (LSI SAS1068 PCI-X Fusion-MPT)

[Richard Scobie]

Moore, Eric wrote:

> Richard, those SR issues are not accessable by developers at LSI.
> Those are created by the tech support group, used internally within the
> support group.    The developers is what your going to find on the
> linux-scsi@ forum, like me and Sathya.  We are using clear case, which
> is a different database.    If you give me the contact name whom
> assigned you that SR number, I will see what I can do.
> 
> Eric

Hi Eric,

Many thanks. Ken Stewart is the man to talk to, otherwise I can forward 
you some long, back and forth emails.

Regards,

Richard

Re: mptbase Code={Abort} (LSI SAS1068 PCI-X Fusion-MPT)

[Richard Scobie]

I have a case open with LSI at the moment on this. I'll let you know 
what they tell me.

Regards,

Richard

Re: mptbase Code={Abort}

[Richard Scobie]

 > Could you let me know when he replies, please? I would like know
 > whether it is something serious or can be ignored.

Certainly. I think this is certainly impacting performance badly, 
judging by the LED's on each drive - each time an error occurs, it can 
take up to 10 seconds for them toi settle back to a stable pattern of 
activity.

 > I have noticed that turning off NCQ significantly reduces amount of
 > the errors.

I have already tried this and found that the errors were slightly more 
frequent, but I had updated the HBA BIOS and firmware before I set the 
queues to 1, so I'm not sure which change had an effect.

Regards,

Richard

Re: mptbase Code={Abort}

[Sebastian Wasilewski]

[-- Attachment #1: Type: text/plain, Size: 547 bytes --]

Richard Scobie wrote:
> 
> I have already tried this and found that the errors were slightly more 
> frequent, but I had updated the HBA BIOS and firmware before I set the 
> queues to 1, so I'm not sure which change had an effect.

In my case, setting it reduced amount of errors from ~100 to ~30 during 
full raid rebuild. But anyway -- it is still 30 to more.

I have made an updated of the BIOS and firmware as a first thing after I 
have connected controllers so I cannot compare results with the old BIOS 
unfortunately.

Regards,
Sebastian

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/x-pkcs7-signature, Size: 4968 bytes --]

Re: mptbase Code={Abort} (LSI SAS1068 PCI-X Fusion-MPT)

[Sebastian Wasilewski]

[-- Attachment #1: Type: text/plain, Size: 517 bytes --]

Hi All,

The problem still exists. With NCQ turned off as well as with NCQ turned on.

# dmesg  |grep 'Code={Abort}, SubCode(0x3000)'
mptbase: ioc0: LogInfo(0x31123000): Originator={PL}, Code={Abort}, 
SubCode(0x3000)
mptbase: ioc0: LogInfo(0x31123000): Originator={PL}, Code={Abort}, 
SubCode(0x3000)
mptbase: ioc0: LogInfo(0x31123000): Originator={PL}, Code={Abort}, 
SubCode(0x3000)
(...)

# dmesg  |grep 'Code={Abort}, SubCode(0x3000)' |wc -l
377

Does anyone know, what that messages mean?

Regards,
Sebastian



[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/x-pkcs7-signature, Size: 4968 bytes --]