* [PATCH] Printk for ENOSPC due to lack of inodes @ 2012-02-27 0:37 Raghavendra D Prabhu 2012-03-07 17:39 ` Eric Sandeen 0 siblings, 1 reply; 6+ messages in thread From: Raghavendra D Prabhu @ 2012-02-27 0:37 UTC (permalink / raw) To: xfs [-- Attachment #1.1: Type: text/plain, Size: 2625 bytes --] Hi, While diagnosing a MySQL crash (on a Centos 5.7 box), I noticed that it had failed with ENOSPC earlier; it was rebooted after that; now after reboot, even though space was there, ENOSPC was showing up, I also did df -i and it showed inodes available. At this point, mounting with inode64 option was tried, which fixed it. http://oss.sgi.com/archives/xfs/2011-03/msg00299.html helped me here. So, I have attached a patch here. ===================================================================== When a ENOSPC is encountered and it is due to lack of inodes (particularly without inode64), it is not possible to detect this (df -i doesn't help here), so adding a printk which can aid in detecting this. Signed-off-by: Raghavendra D Prabhu <rprabhu@wnohang.net> --- fs/xfs/xfs_qm.c | 3 +++ fs/xfs/xfs_vnodeops.c | 10 ++++++++-- 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/fs/xfs/xfs_qm.c b/fs/xfs/xfs_qm.c index c872fea..fbefa87 100644 --- a/fs/xfs/xfs_qm.c +++ b/fs/xfs/xfs_qm.c @@ -987,6 +987,9 @@ xfs_qm_qino_alloc( error = xfs_dir_ialloc(&tp, NULL, S_IFREG, 1, 0, 0, 1, ip, &committed); if (error) { + if (error == ENOSPC) + xfs_err(mp, "Out of inodes: Required %d, Current %llu, Maximum %llu", + XFS_IALLOC_INODES(mp), mp->m_sb.sb_icount, mp->m_maxicount); xfs_trans_cancel(tp, XFS_TRANS_RELEASE_LOG_RES | XFS_TRANS_ABORT); return error; diff --git a/fs/xfs/xfs_vnodeops.c b/fs/xfs/xfs_vnodeops.c index ebdb888..7542c36 100644 --- a/fs/xfs/xfs_vnodeops.c +++ b/fs/xfs/xfs_vnodeops.c @@ -946,8 +946,11 @@ xfs_create( error = xfs_dir_ialloc(&tp, dp, mode, is_dir ? 2 : 1, rdev, prid, resblks > 0, &ip, &committed); if (error) { - if (error == ENOSPC) + if (error == ENOSPC) { + xfs_err(mp, "Out of inodes: Required %d, Current %llu, Maximum %llu", + XFS_IALLOC_INODES(mp), mp->m_sb.sb_icount, mp->m_maxicount); goto out_trans_cancel; + } goto out_trans_abort; } @@ -1610,8 +1613,11 @@ xfs_symlink( error = xfs_dir_ialloc(&tp, dp, S_IFLNK | (mode & ~S_IFMT), 1, 0, prid, resblks > 0, &ip, NULL); if (error) { - if (error == ENOSPC) + if (error == ENOSPC) { + xfs_err(mp, "Out of inodes: Required %d, Current %llu, Maximum %llu", + XFS_IALLOC_INODES(mp), mp->m_sb.sb_icount, mp->m_maxicount); goto error_return; + } goto error1; } -- 1.7.9.2 Regards, -- Raghavendra Prabhu GPG Id : 0xD72BE977 Fingerprint: B93F EBCB 8E05 7039 CD3C A4B8 A616 DCA1 D72B E977 www: wnohang.net [-- Attachment #1.2: Type: application/pgp-signature, Size: 490 bytes --] [-- Attachment #2: Type: text/plain, Size: 121 bytes --] _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH] Printk for ENOSPC due to lack of inodes 2012-02-27 0:37 [PATCH] Printk for ENOSPC due to lack of inodes Raghavendra D Prabhu @ 2012-03-07 17:39 ` Eric Sandeen 2012-05-31 19:36 ` Raghavendra D Prabhu 0 siblings, 1 reply; 6+ messages in thread From: Eric Sandeen @ 2012-03-07 17:39 UTC (permalink / raw) To: xfs On 2/26/12 6:37 PM, Raghavendra D Prabhu wrote: > Hi, > > While diagnosing a MySQL crash (on a Centos 5.7 box), I noticed that > it had failed with ENOSPC earlier; it was rebooted after that; now > after reboot, even though space was there, ENOSPC was showing up, I > also did df -i and it showed inodes available. At this point, > mounting with inode64 option was tried, which fixed it. > http://oss.sgi.com/archives/xfs/2011-03/msg00299.html helped me > here. Yeah, that's kind of a bummer. And given the semi-ugly situation we're in with inode32, maybe a syslog message would be good. But a few things; I don't think we want to warn on every inode-allocation ENOSPC. Ideally I'd probably do a WARN_ON_ONCE or a ratelimited printk. xfs_warn_once_per_fs()? :) I'd probably also want to only do it in the case where the ENOSPC was due to either inode32, or due to fragmented freespace. Without looking very hard yet; could this be done in xfs_dialloc() so that the exact reason for the ENOSPC can be issued? (maxicount, inode32, or no free contiguous space...) -Eric > > So, I have attached a patch here. > > ===================================================================== > > > When a ENOSPC is encountered and it is due to lack of inodes (particularly > without inode64), it is not possible to detect this (df -i doesn't help here), > so adding a printk which can aid in detecting this. > > Signed-off-by: Raghavendra D Prabhu <rprabhu@wnohang.net> > --- > fs/xfs/xfs_qm.c | 3 +++ > fs/xfs/xfs_vnodeops.c | 10 ++++++++-- > 2 files changed, 11 insertions(+), 2 deletions(-) > > diff --git a/fs/xfs/xfs_qm.c b/fs/xfs/xfs_qm.c > index c872fea..fbefa87 100644 > --- a/fs/xfs/xfs_qm.c > +++ b/fs/xfs/xfs_qm.c > @@ -987,6 +987,9 @@ xfs_qm_qino_alloc( > > error = xfs_dir_ialloc(&tp, NULL, S_IFREG, 1, 0, 0, 1, ip, &committed); > if (error) { > + if (error == ENOSPC) > + xfs_err(mp, "Out of inodes: Required %d, Current %llu, Maximum %llu", > + XFS_IALLOC_INODES(mp), mp->m_sb.sb_icount, mp->m_maxicount); > xfs_trans_cancel(tp, XFS_TRANS_RELEASE_LOG_RES | > XFS_TRANS_ABORT); > return error; > diff --git a/fs/xfs/xfs_vnodeops.c b/fs/xfs/xfs_vnodeops.c > index ebdb888..7542c36 100644 > --- a/fs/xfs/xfs_vnodeops.c > +++ b/fs/xfs/xfs_vnodeops.c > @@ -946,8 +946,11 @@ xfs_create( > error = xfs_dir_ialloc(&tp, dp, mode, is_dir ? 2 : 1, rdev, > prid, resblks > 0, &ip, &committed); > if (error) { > - if (error == ENOSPC) > + if (error == ENOSPC) { > + xfs_err(mp, "Out of inodes: Required %d, Current %llu, Maximum %llu", > + XFS_IALLOC_INODES(mp), mp->m_sb.sb_icount, mp->m_maxicount); > goto out_trans_cancel; > + } > goto out_trans_abort; > } > > @@ -1610,8 +1613,11 @@ xfs_symlink( > error = xfs_dir_ialloc(&tp, dp, S_IFLNK | (mode & ~S_IFMT), 1, 0, > prid, resblks > 0, &ip, NULL); > if (error) { > - if (error == ENOSPC) > + if (error == ENOSPC) { > + xfs_err(mp, "Out of inodes: Required %d, Current %llu, Maximum %llu", > + XFS_IALLOC_INODES(mp), mp->m_sb.sb_icount, mp->m_maxicount); > goto error_return; > + } > goto error1; > } > > > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Re: [PATCH] Printk for ENOSPC due to lack of inodes 2012-03-07 17:39 ` Eric Sandeen @ 2012-05-31 19:36 ` Raghavendra D Prabhu 2012-06-01 3:31 ` Eric Sandeen 0 siblings, 1 reply; 6+ messages in thread From: Raghavendra D Prabhu @ 2012-05-31 19:36 UTC (permalink / raw) To: Eric Sandeen; +Cc: xfs [-- Attachment #1.1: Type: text/plain, Size: 4649 bytes --] Hi, Thanks for the review. I have now moved it inside xfs_dialloc. Along with adding the message, I noticed that the loop while (!agi->agi_freecount) { } is redundant when noroom=1 and okalloc=0. Also, xfs_ialloc_ag_alloc function in the loop calls ============ if (mp->m_maxicount && mp->m_sb.sb_icount + XFS_IALLOC_INODES(mp) > mp->m_maxicount) { =============== condition again. So I have moved xfs_tran_brelse etc. into the condition along with message. Is this logic valid? If it is, then I will look into rate-limiting the message etc. * On Wed, Mar 07, 2012 at 11:39:24AM -0600, Eric Sandeen <sandeen@sandeen.net> wrote: >On 2/26/12 6:37 PM, Raghavendra D Prabhu wrote: >> Hi, >> >> While diagnosing a MySQL crash (on a Centos 5.7 box), I noticed that >> it had failed with ENOSPC earlier; it was rebooted after that; now >> after reboot, even though space was there, ENOSPC was showing up, I >> also did df -i and it showed inodes available. At this point, >> mounting with inode64 option was tried, which fixed it. >> http://oss.sgi.com/archives/xfs/2011-03/msg00299.html helped me >> here. > >Yeah, that's kind of a bummer. And given the semi-ugly situation >we're in with inode32, maybe a syslog message would be good. > >But a few things; I don't think we want to warn on every inode-allocation >ENOSPC. Ideally I'd probably do a WARN_ON_ONCE or a ratelimited printk. >xfs_warn_once_per_fs()? :) > >I'd probably also want to only do it in the case where the ENOSPC was due >to either inode32, or due to fragmented freespace. > >Without looking very hard yet; could this be done in xfs_dialloc() so that >the exact reason for the ENOSPC can be issued? (maxicount, inode32, or >no free contiguous space...) > >-Eric > >> >> So, I have attached a patch here. >> >> ===================================================================== >> >> >> When a ENOSPC is encountered and it is due to lack of inodes (particularly >> without inode64), it is not possible to detect this (df -i doesn't help here), >> so adding a printk which can aid in detecting this. >> >> Signed-off-by: Raghavendra D Prabhu <rprabhu@wnohang.net> >> --- >> fs/xfs/xfs_qm.c | 3 +++ >> fs/xfs/xfs_vnodeops.c | 10 ++++++++-- >> 2 files changed, 11 insertions(+), 2 deletions(-) >> >> diff --git a/fs/xfs/xfs_qm.c b/fs/xfs/xfs_qm.c >> index c872fea..fbefa87 100644 >> --- a/fs/xfs/xfs_qm.c >> +++ b/fs/xfs/xfs_qm.c >> @@ -987,6 +987,9 @@ xfs_qm_qino_alloc( >> >> error = xfs_dir_ialloc(&tp, NULL, S_IFREG, 1, 0, 0, 1, ip, &committed); >> if (error) { >> + if (error == ENOSPC) >> + xfs_err(mp, "Out of inodes: Required %d, Current %llu, Maximum %llu", >> + XFS_IALLOC_INODES(mp), mp->m_sb.sb_icount, mp->m_maxicount); >> xfs_trans_cancel(tp, XFS_TRANS_RELEASE_LOG_RES | >> XFS_TRANS_ABORT); >> return error; >> diff --git a/fs/xfs/xfs_vnodeops.c b/fs/xfs/xfs_vnodeops.c >> index ebdb888..7542c36 100644 >> --- a/fs/xfs/xfs_vnodeops.c >> +++ b/fs/xfs/xfs_vnodeops.c >> @@ -946,8 +946,11 @@ xfs_create( >> error = xfs_dir_ialloc(&tp, dp, mode, is_dir ? 2 : 1, rdev, >> prid, resblks > 0, &ip, &committed); >> if (error) { >> - if (error == ENOSPC) >> + if (error == ENOSPC) { >> + xfs_err(mp, "Out of inodes: Required %d, Current %llu, Maximum %llu", >> + XFS_IALLOC_INODES(mp), mp->m_sb.sb_icount, mp->m_maxicount); >> goto out_trans_cancel; >> + } >> goto out_trans_abort; >> } >> >> @@ -1610,8 +1613,11 @@ xfs_symlink( >> error = xfs_dir_ialloc(&tp, dp, S_IFLNK | (mode & ~S_IFMT), 1, 0, >> prid, resblks > 0, &ip, NULL); >> if (error) { >> - if (error == ENOSPC) >> + if (error == ENOSPC) { >> + xfs_err(mp, "Out of inodes: Required %d, Current %llu, Maximum %llu", >> + XFS_IALLOC_INODES(mp), mp->m_sb.sb_icount, mp->m_maxicount); >> goto error_return; >> + } >> goto error1; >> } >> >> >> >> _______________________________________________ >> xfs mailing list >> xfs@oss.sgi.com >> http://oss.sgi.com/mailman/listinfo/xfs > >_______________________________________________ >xfs mailing list >xfs@oss.sgi.com >http://oss.sgi.com/mailman/listinfo/xfs > Regards, -- Raghavendra Prabhu GPG Id : 0xD72BE977 Fingerprint: B93F EBCB 8E05 7039 CD3C A4B8 A616 DCA1 D72B E977 www: wnohang.net [-- Attachment #1.2: Type: application/pgp-signature, Size: 490 bytes --] [-- Attachment #2: Type: text/plain, Size: 121 bytes --] _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] Printk for ENOSPC due to lack of inodes 2012-05-31 19:36 ` Raghavendra D Prabhu @ 2012-06-01 3:31 ` Eric Sandeen 2012-06-01 6:07 ` Raghavendra D Prabhu 2012-06-05 6:46 ` Raghavendra D Prabhu 0 siblings, 2 replies; 6+ messages in thread From: Eric Sandeen @ 2012-06-01 3:31 UTC (permalink / raw) To: xfs On 5/31/12 2:36 PM, Raghavendra D Prabhu wrote: > Hi, > > Thanks for the review. I have now moved it inside xfs_dialloc. Along with adding the message, I noticed that the loop > > while (!agi->agi_freecount) { > } > > is redundant when noroom=1 and okalloc=0. > > Also, xfs_ialloc_ag_alloc function in the loop calls > ============ > if (mp->m_maxicount && > mp->m_sb.sb_icount + XFS_IALLOC_INODES(mp) > mp->m_maxicount) { > > =============== > > condition again. > > So I have moved xfs_tran_brelse etc. into the condition along with message. > > Is this logic valid? If it is, then I will look into rate-limiting the message etc. It'd be easiest to understand this new change as a patch rather than as a description. If you are changing logic or flow in addition to adding the messages, it should almost certainly be sent as more than one patch. Thanks, -eric _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Re: [PATCH] Printk for ENOSPC due to lack of inodes 2012-06-01 3:31 ` Eric Sandeen @ 2012-06-01 6:07 ` Raghavendra D Prabhu 2012-06-05 6:46 ` Raghavendra D Prabhu 1 sibling, 0 replies; 6+ messages in thread From: Raghavendra D Prabhu @ 2012-06-01 6:07 UTC (permalink / raw) To: Eric Sandeen; +Cc: xfs [-- Attachment #1.1: Type: text/plain, Size: 1424 bytes --] Hi, * On Thu, May 31, 2012 at 10:31:21PM -0500, Eric Sandeen <sandeen@sandeen.net> wrote: >On 5/31/12 2:36 PM, Raghavendra D Prabhu wrote: >> Hi, >> >> Thanks for the review. I have now moved it inside xfs_dialloc. Along with adding the message, I noticed that the loop >> >> while (!agi->agi_freecount) { >> } >> >> is redundant when noroom=1 and okalloc=0. >> >> Also, xfs_ialloc_ag_alloc function in the loop calls >> ============ >> if (mp->m_maxicount && >> mp->m_sb.sb_icount + XFS_IALLOC_INODES(mp) > mp->m_maxicount) { >> >> =============== >> >> condition again. >> >> So I have moved xfs_tran_brelse etc. into the condition along with message. >> >> Is this logic valid? If it is, then I will look into rate-limiting the message etc. > >It'd be easiest to understand this new change as a patch rather than as a description. Yeah, I would certainly send it as a patch. This was just for RFC. :) > >If you are changing logic or flow in addition to adding the messages, it should almost certainly be sent as more than one patch. Thanks, I will make it so. > >Thanks, > >-eric > >_______________________________________________ >xfs mailing list >xfs@oss.sgi.com >http://oss.sgi.com/mailman/listinfo/xfs > Regards, -- Raghavendra Prabhu GPG Id : 0xD72BE977 Fingerprint: B93F EBCB 8E05 7039 CD3C A4B8 A616 DCA1 D72B E977 www: wnohang.net [-- Attachment #1.2: Type: application/pgp-signature, Size: 490 bytes --] [-- Attachment #2: Type: text/plain, Size: 121 bytes --] _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Re: [PATCH] Printk for ENOSPC due to lack of inodes 2012-06-01 3:31 ` Eric Sandeen 2012-06-01 6:07 ` Raghavendra D Prabhu @ 2012-06-05 6:46 ` Raghavendra D Prabhu 1 sibling, 0 replies; 6+ messages in thread From: Raghavendra D Prabhu @ 2012-06-05 6:46 UTC (permalink / raw) To: Eric Sandeen; +Cc: xfs [-- Attachment #1.1: Type: text/plain, Size: 1510 bytes --] Hi, Thanks for reviewing earlier. I have sent the patch series here from: http://oss.sgi.com/archives/xfs/2012-06/msg00020.html to http://oss.sgi.com/archives/xfs/2012-06/msg00023.html * On Thu, May 31, 2012 at 10:31:21PM -0500, Eric Sandeen <sandeen@sandeen.net> wrote: >On 5/31/12 2:36 PM, Raghavendra D Prabhu wrote: >> Hi, >> >> Thanks for the review. I have now moved it inside xfs_dialloc. Along with adding the message, I noticed that the loop >> >> while (!agi->agi_freecount) { >> } >> >> is redundant when noroom=1 and okalloc=0. >> >> Also, xfs_ialloc_ag_alloc function in the loop calls >> ============ >> if (mp->m_maxicount && >> mp->m_sb.sb_icount + XFS_IALLOC_INODES(mp) > mp->m_maxicount) { >> >> =============== >> >> condition again. >> >> So I have moved xfs_tran_brelse etc. into the condition along with message. >> >> Is this logic valid? If it is, then I will look into rate-limiting the message etc. > >It'd be easiest to understand this new change as a patch rather than as a description. > >If you are changing logic or flow in addition to adding the messages, it should almost certainly be sent as more than one patch. > >Thanks, > >-eric > >_______________________________________________ >xfs mailing list >xfs@oss.sgi.com >http://oss.sgi.com/mailman/listinfo/xfs > Regards, -- Raghavendra Prabhu GPG Id : 0xD72BE977 Fingerprint: B93F EBCB 8E05 7039 CD3C A4B8 A616 DCA1 D72B E977 www: wnohang.net [-- Attachment #1.2: Type: application/pgp-signature, Size: 490 bytes --] [-- Attachment #2: Type: text/plain, Size: 121 bytes --] _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2012-06-05 6:46 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2012-02-27 0:37 [PATCH] Printk for ENOSPC due to lack of inodes Raghavendra D Prabhu 2012-03-07 17:39 ` Eric Sandeen 2012-05-31 19:36 ` Raghavendra D Prabhu 2012-06-01 3:31 ` Eric Sandeen 2012-06-01 6:07 ` Raghavendra D Prabhu 2012-06-05 6:46 ` Raghavendra D Prabhu
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox