kernel 2.6.22 pdflush consumes ~90% cpu

kernel 2.6.22 pdflush consumes ~90% cpu

[Muralitharan Perumal]

[-- Attachment #1: Type: text/plain, Size: 1401 bytes --]

Hi,

I am currently working on an embedded platform involving XFS as the
preferred fs. Earlier, ext3 was used but now replaced with XFS. The unit
is working fine with ext3 but after replacing with XFS, I am seeing a
few issues:

a) Occassionally I am getting the following error when we write files to
the fs:

attempt to access beyond end of device
sda1: rw=0, want=10487141907480064, limit=9766912
I/O error in filesystem ("sda1") meta-data dev sda1 block 0x254200001abdf8       ("xfs_trans_read_buf") error 5 buf count 4096
attempt to access beyond end of device
sda1: rw=0, want=10487141907480064, limit=9766912
I/O error in filesystem ("sda1") meta-data dev sda1 block 0x254200001abdf8       ("xfs_trans_read_buf") error 5 buf count 4096

b) After running for about 20 to 25 minutes, the pdflush consumes about 90% cpu and stuck there in D state:

Is there any similar issue seen with kernel 2.6.22? Am I missing any XFS patches that could fix this?

Any pointers would be helpful.

Thanks,
Murali


IMPORTANT NOTICE - this e-mail and any attachments hereto are strictly confidential and intended solely for the addressee. If you are not the intended recipient, please notify the sender to report the error and delete this communication immediately. You must not disclose, forward or copy this e-mail or any attachments hereto without the prior consent of the sender.

[-- Attachment #2: 0x1B79B2F8.asc --]
[-- Type: application/pgp-keys, Size: 3149 bytes --]

[-- Attachment #3: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

RE: kernel 2.6.22 pdflush consumes ~90% cpu

[Muralitharan Perumal]

Hi Dave,

Is this related to log buffer size when formatting and mounting as I am just using the default 32KB buffer size?

Any pointers will be helpful.


-----Original Message-----
From: xfs-bounces@oss.sgi.com [mailto:xfs-bounces@oss.sgi.com] On Behalf Of Muralitharan Perumal
Sent: 13 January 2014 14:59
To: xfs@oss.sgi.com
Subject: kernel 2.6.22 pdflush consumes ~90% cpu

Hi,

I am currently working on an embedded platform involving XFS as the preferred fs. Earlier, ext3 was used but now replaced with XFS. The unit is working fine with ext3 but after replacing with XFS, I am seeing a few issues:

a) Occassionally I am getting the following error when we write files to the fs:

attempt to access beyond end of device
sda1: rw=0, want=10487141907480064, limit=9766912
I/O error in filesystem ("sda1") meta-data dev sda1 block 0x254200001abdf8       ("xfs_trans_read_buf") error 5 buf count 4096
attempt to access beyond end of device
sda1: rw=0, want=10487141907480064, limit=9766912
I/O error in filesystem ("sda1") meta-data dev sda1 block 0x254200001abdf8       ("xfs_trans_read_buf") error 5 buf count 4096

b) After running for about 20 to 25 minutes, the pdflush consumes about 90% cpu and stuck there in D state:

Is there any similar issue seen with kernel 2.6.22? Am I missing any XFS patches that could fix this?

Any pointers would be helpful.

Thanks,
Murali


IMPORTANT NOTICE - this e-mail and any attachments hereto are strictly confidential and intended solely for the addressee. If you are not the intended recipient, please notify the sender to report the error and delete this communication immediately. You must not disclose, forward or copy this e-mail or any attachments hereto without the prior consent of the sender.
IMPORTANT NOTICE - this e-mail and any attachments hereto are strictly confidential and intended solely for the addressee. If you are not the intended recipient, please notify the sender to report the error and delete this communication immediately. You must not disclose, forward or copy this e-mail or any attachments hereto without the prior consent of the sender.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: kernel 2.6.22 pdflush consumes ~90% cpu

[Eric Sandeen]

On 1/13/14, 8:58 AM, Muralitharan Perumal wrote:
> Hi,
> 
> I am currently working on an embedded platform involving XFS as the
> preferred fs. Earlier, ext3 was used but now replaced with XFS. The unit
> is working fine with ext3 but after replacing with XFS, I am seeing a
> few issues:

a 7 year old kernel on an embedded platform?  Nobody here will be able
to spend much time on this I'm afraid.

> a) Occassionally I am getting the following error when we write files to
> the fs:
> 
> attempt to access beyond end of device
> sda1: rw=0, want=10487141907480064, limit=9766912
> I/O error in filesystem ("sda1") meta-data dev sda1 block 0x254200001abdf8       ("xfs_trans_read_buf") error 5 buf count 4096
> attempt to access beyond end of device
> sda1: rw=0, want=10487141907480064, limit=9766912
> I/O error in filesystem ("sda1") meta-data dev sda1 block 0x254200001abdf8       ("xfs_trans_read_buf") error 5 buf count 4096

looks like on-disk corruption.  xfs_repair is probably warranted.

> b) After running for about 20 to 25 minutes, the pdflush consumes about 90% cpu and stuck there in D state:

do a sysrq-w to get backtraces of all blocked tasks, and that might yield a clue for you.

-Eric
 
> Is there any similar issue seen with kernel 2.6.22? Am I missing any XFS patches that could fix this?
> 
> Any pointers would be helpful.
> 
> Thanks,
> Murali
> 
> 
> IMPORTANT NOTICE - this e-mail and any attachments hereto are strictly confidential and intended solely for the addressee. If you are not the intended recipient, please notify the sender to report the error and delete this communication immediately. You must not disclose, forward or copy this e-mail or any attachments hereto without the prior consent of the sender.
> 
> 
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: kernel 2.6.22 pdflush consumes ~90% cpu

[Dave Chinner]

On Mon, Jan 13, 2014 at 02:58:54PM +0000, Muralitharan Perumal wrote:
> Hi,
> 
> I am currently working on an embedded platform involving XFS as the
> preferred fs. Earlier, ext3 was used but now replaced with XFS. The unit
> is working fine with ext3 but after replacing with XFS, I am seeing a
> few issues:
> 
> a) Occassionally I am getting the following error when we write files to
> the fs:
> 
> attempt to access beyond end of device
> sda1: rw=0, want=10487141907480064, limit=9766912
> I/O error in filesystem ("sda1") meta-data dev sda1 block 0x254200001abdf8       ("xfs_trans_read_buf") error 5 buf count 4096
> attempt to access beyond end of device
> sda1: rw=0, want=10487141907480064, limit=9766912
> I/O error in filesystem ("sda1") meta-data dev sda1 block 0x254200001abdf8       ("xfs_trans_read_buf") error 5 buf count 4096

It's trying to write way beyond the end of the device, hence the
error. I'd suggest updating to xfsprogs 3.11 and running
xfs_repair -n /dev/sda1 on the device and seeing if it finds a
corrupt block map on an inode somewhere.

> b) After running for about 20 to 25 minutes, the pdflush consumes about 90% cpu and stuck there in D state:

It can't be stuck in an uninterruptible sleep (D state) and
consuming CPU at the same time....

> Is there any similar issue seen with kernel 2.6.22? Am I missing any XFS patches that could fix this?

That's a 6 year old kernel. I'd suggest that an upgrade to something
more recent might be in order...

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs