* sizing log - is there a too big? @ 2013-06-26 23:56 aurfalien 2013-06-27 1:48 ` Dave Chinner 0 siblings, 1 reply; 4+ messages in thread From: aurfalien @ 2013-06-26 23:56 UTC (permalink / raw) To: xfs Hi all, Wondering if my log being just under 2GB is a bad idea. Noticing flush-253:2/kcopyd which is my XFS file system getting really high load avg and wait times via top). Doing a simple rsync over NFS and after a bit, the system gets to a load of 24.... yikes... Upon killing the rsync, I am seeing loads going down to sub 1 after about 10 min. I have repeated this to verify 10 min. - aurf _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: sizing log - is there a too big? 2013-06-26 23:56 sizing log - is there a too big? aurfalien @ 2013-06-27 1:48 ` Dave Chinner 2013-06-27 15:58 ` aurfalien 0 siblings, 1 reply; 4+ messages in thread From: Dave Chinner @ 2013-06-27 1:48 UTC (permalink / raw) To: aurfalien; +Cc: xfs On Wed, Jun 26, 2013 at 04:56:31PM -0700, aurfalien wrote: > Hi all, > > Wondering if my log being just under 2GB is a bad idea. > > Noticing flush-253:2/kcopyd which is my XFS file system getting > really high load avg and wait times via top). What has the log size got to do with something that is happening at the block layer? What's your storage config? > Doing a simple rsync over NFS and after a bit, the system gets to a load of 24.... yikes... Let me guess - 24 nfsds blocked waiting for kcopyd to do it's stuff? Load average going up when the NFS server is busy generally means your IO subsystem is heavily loaded - it's not uncommon to see large NFS servers that are extremely busy sustain load averages over a 100 (or even 1000) for hours/days on end.... > Upon killing the rsync, I am seeing loads going down to sub 1 > after about 10 min. I have repeated this to verify 10 min. Sure. Processes blocked on IO contribute to the load average. Kill the IO load, and the load average will return to nothing in 10-15 minutes. Cheers, Dave. -- Dave Chinner david@fromorbit.com _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: sizing log - is there a too big? 2013-06-27 1:48 ` Dave Chinner @ 2013-06-27 15:58 ` aurfalien 2013-06-28 1:56 ` Dave Chinner 0 siblings, 1 reply; 4+ messages in thread From: aurfalien @ 2013-06-27 15:58 UTC (permalink / raw) To: Dave Chinner; +Cc: xfs On Jun 26, 2013, at 6:48 PM, Dave Chinner wrote: > On Wed, Jun 26, 2013 at 04:56:31PM -0700, aurfalien wrote: >> Hi all, >> >> Wondering if my log being just under 2GB is a bad idea. >> >> Noticing flush-253:2/kcopyd which is my XFS file system getting >> really high load avg and wait times via top). > > What has the log size got to do with something that is happening at > the block layer? What's your storage config? > >> Doing a simple rsync over NFS and after a bit, the system gets to a load of 24.... yikes... > > Let me guess - 24 nfsds blocked waiting for kcopyd to do it's stuff? > > Load average going up when the NFS server is busy generally means > your IO subsystem is heavily loaded - it's not uncommon to see large > NFS servers that are extremely busy sustain load averages over a > 100 (or even 1000) for hours/days on end.... > >> Upon killing the rsync, I am seeing loads going down to sub 1 >> after about 10 min. I have repeated this to verify 10 min. > > Sure. Processes blocked on IO contribute to the load average. Kill > the IO load, and the load average will return to nothing in 10-15 > minutes. Not so fast my fine feathered friend. Same work load, same hardware. Only diff is; External log, its 2GB And its Centos 6.4 which was previously 5.9. Its 16 drives in a hw raid 6, but 2 are for the mirrored journal and 1 for hot spare so 13 spindles. Before it was 14 spindles in raid 6 and 2 hot spare. I did power cycle it late last night and will more closely observe today. Very odd. - aurf _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: sizing log - is there a too big? 2013-06-27 15:58 ` aurfalien @ 2013-06-28 1:56 ` Dave Chinner 0 siblings, 0 replies; 4+ messages in thread From: Dave Chinner @ 2013-06-28 1:56 UTC (permalink / raw) To: aurfalien; +Cc: xfs On Thu, Jun 27, 2013 at 08:58:06AM -0700, aurfalien wrote: > > On Jun 26, 2013, at 6:48 PM, Dave Chinner wrote: > > > On Wed, Jun 26, 2013 at 04:56:31PM -0700, aurfalien wrote: > >> Hi all, > >> > >> Wondering if my log being just under 2GB is a bad idea. > >> > >> Noticing flush-253:2/kcopyd which is my XFS file system getting > >> really high load avg and wait times via top). > > > > What has the log size got to do with something that is happening at > > the block layer? What's your storage config? > > > >> Doing a simple rsync over NFS and after a bit, the system gets to a load of 24.... yikes... > > > > Let me guess - 24 nfsds blocked waiting for kcopyd to do it's stuff? > > > > Load average going up when the NFS server is busy generally means > > your IO subsystem is heavily loaded - it's not uncommon to see large > > NFS servers that are extremely busy sustain load averages over a > > 100 (or even 1000) for hours/days on end.... > > > >> Upon killing the rsync, I am seeing loads going down to sub 1 > >> after about 10 min. I have repeated this to verify 10 min. > > > > Sure. Processes blocked on IO contribute to the load average. Kill > > the IO load, and the load average will return to nothing in 10-15 > > minutes. > > > Not so fast my fine feathered friend. > > Same work load, same hardware. > > Only diff is; > > External log, its 2GB > And its Centos 6.4 which was previously 5.9. Oh, you're comparing behaviour between kernels 5 years in age difference. Well, things change, and a change of load average for the same workload between very different kernels is no unexpected. Cheers, Dave. -- Dave Chinner david@fromorbit.com _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2013-06-28 1:56 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2013-06-26 23:56 sizing log - is there a too big? aurfalien 2013-06-27 1:48 ` Dave Chinner 2013-06-27 15:58 ` aurfalien 2013-06-28 1:56 ` Dave Chinner
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox