* 2.6.24.3 nfs server on xfs keeps producing nfsd: non-standard errno: -117 @ 2008-03-17 23:27 Stuart Rowan 2008-03-18 0:28 ` Timothy Shimmin 0 siblings, 1 reply; 6+ messages in thread From: Stuart Rowan @ 2008-03-17 23:27 UTC (permalink / raw) To: xfs Hi, Firstly thanks for the great filesystem and apologies if this ends up being NFS rather than XFS being weird! I'm not subscribed so please do keep me CC'd. I have *millions* of lines of (>200k per minute according to syslog): nfsd: non-standard errno: -117 being sent out of dmesg Now errno 117 is #define EUCLEAN 117 /* Structure needs cleaning */ which seems to be only used from a quick grep by XFS and JFFS and smbfs. My nfs server export two locations /home /home/archive both of these are XFS partitions, hence my suspicion that the -117 is coming from XFS. xfs_repair -n says the filesystems are clean xfs_repair has been run multiple times to completion on the filesystems, all is fine. The XFS partitions are lvm volumes as follows data/home 900G data/archive 400G The volume group, data, is sda3 sda3 is a 6 drive 3ware 9550SXU-8LP RAID10 array The NFS server is currently in use (indeed the message only starts once clients connect) and works absolutely fine. How do I find out what (if anything) is wrong with my filesystem / appropriately silence this message? Many thanks, Stu. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: 2.6.24.3 nfs server on xfs keeps producing nfsd: non-standard errno: -117 2008-03-17 23:27 2.6.24.3 nfs server on xfs keeps producing nfsd: non-standard errno: -117 Stuart Rowan @ 2008-03-18 0:28 ` Timothy Shimmin 2008-03-18 1:07 ` Stuart Rowan 2008-03-18 13:49 ` Stuart Rowan 0 siblings, 2 replies; 6+ messages in thread From: Timothy Shimmin @ 2008-03-18 0:28 UTC (permalink / raw) To: strr-debian; +Cc: xfs Hi Stuart, Stuart Rowan wrote: > Hi, > > Firstly thanks for the great filesystem and apologies if this ends up > being NFS rather than XFS being weird! I'm not subscribed so please do > keep me CC'd. > > I have *millions* of lines of (>200k per minute according to syslog): > nfsd: non-standard errno: -117 > being sent out of dmesg > > Now errno 117 is > #define EUCLEAN 117 /* Structure needs cleaning */ > which seems to be only used from a quick grep by XFS and JFFS and smbfs. > > In XFS we mapped EFSCORRUPTED to EUCLEAN as EFSCORRUPTED didn't exist on Linux. However, normally if this error is encountered in XFS then we output an appropriate msg to the syslog. Our default error level is 3 and most reports are rated at 1 so should show up I would have thought. --Tim > My nfs server export two locations > /home > /home/archive > both of these are XFS partitions, hence my suspicion that the -117 is > coming from XFS. > > xfs_repair -n says the filesystems are clean > xfs_repair has been run multiple times to completion on the filesystems, > all is fine. > > The XFS partitions are lvm volumes as follows > data/home 900G > data/archive 400G > The volume group, data, is sda3 > sda3 is a 6 drive 3ware 9550SXU-8LP RAID10 array > > The NFS server is currently in use (indeed the message only starts once > clients connect) and works absolutely fine. > > How do I find out what (if anything) is wrong with my filesystem / > appropriately silence this message? > > Many thanks, > Stu. > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: 2.6.24.3 nfs server on xfs keeps producing nfsd: non-standard errno: -117 2008-03-18 0:28 ` Timothy Shimmin @ 2008-03-18 1:07 ` Stuart Rowan 2008-03-18 13:49 ` Stuart Rowan 1 sibling, 0 replies; 6+ messages in thread From: Stuart Rowan @ 2008-03-18 1:07 UTC (permalink / raw) To: Timothy Shimmin; +Cc: xfs Hi Tim, Timothy Shimmin wrote: > Hi Stuart, > > Stuart Rowan wrote: >> Hi, >> >> Firstly thanks for the great filesystem and apologies if this ends up >> being NFS rather than XFS being weird! I'm not subscribed so please do >> keep me CC'd. >> >> I have *millions* of lines of (>200k per minute according to syslog): >> nfsd: non-standard errno: -117 >> being sent out of dmesg >> >> Now errno 117 is >> #define EUCLEAN 117 /* Structure needs cleaning */ >> which seems to be only used from a quick grep by XFS and JFFS and smbfs. >> >> > In XFS we mapped EFSCORRUPTED to EUCLEAN as EFSCORRUPTED > didn't exist on Linux. > However, normally if this error is encountered in XFS then > we output an appropriate msg to the syslog. > Our default error level is 3 and most reports are rated at 1 > so should show up I would have thought. > > --Tim > Thanks for the swift reply -- reading previous mailing list posts, that was my expectation too! I've attached the relevant section of /var/log/kern.log inline below -- there is no error message! Is there another way of inspecting what (if anything) a given XFS file-system thinks is wrong with it. Thanks, Stu. Mar 17 23:01:50 evenlode kernel: SGI XFS with ACLs, security attributes, realtime, large block/inode numbers, no debug enabled Mar 17 23:01:50 evenlode kernel: SGI XFS Quota Management subsystem Mar 17 23:01:50 evenlode kernel: Filesystem "dm-0": Disabling barriers, not supported by the underlying device Mar 17 23:01:50 evenlode kernel: XFS mounting filesystem dm-0 Mar 17 23:01:50 evenlode kernel: Ending clean XFS mount for filesystem: dm-0 Mar 17 23:01:50 evenlode kernel: Filesystem "dm-1": Disabling barriers, not supported by the underlying device Mar 17 23:01:50 evenlode kernel: XFS mounting filesystem dm-1 Mar 17 23:01:50 evenlode kernel: Ending clean XFS mount for filesystem: dm-1 Mar 17 23:01:50 evenlode kernel: e1000: eth0: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX Mar 17 23:01:50 evenlode kernel: RPC: Registered udp transport module. Mar 17 23:01:50 evenlode kernel: RPC: Registered tcp transport module. Mar 17 23:01:50 evenlode kernel: NET: Registered protocol family 10 Mar 17 23:01:50 evenlode kernel: lo: Disabled Privacy Extensions Mar 17 23:01:50 evenlode kernel: IA-32 Microcode Update Driver: v1.14a <tigran@aivazian.fsnet.co.uk> Mar 17 23:01:51 evenlode kernel: Installing knfsd (copyright (C) 1996 okir@monad.swb.de). Mar 17 23:01:51 evenlode kernel: NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory Mar 17 23:01:51 evenlode kernel: NFSD: starting 90-second grace period Mar 17 23:01:57 evenlode kernel: eth0: no IPv6 routers present Mar 17 23:03:04 evenlode kernel: 3w-9xxx: scsi0: AEN: INFO (0x04:0x000C): Initialize started:unit=0, subunit=0. Mar 17 23:03:04 evenlode kernel: 3w-9xxx: scsi0: AEN: INFO (0x04:0x000C): Initialize started:unit=0, subunit=2. Mar 17 23:04:40 evenlode kernel: nfsd: non-standard errno: -117 Mar 17 23:05:11 evenlode last message repeated 93970 times Mar 17 23:06:12 evenlode last message repeated 188363 times mount: /dev/mapper/data-home on /home type xfs (rw,logbufs=8,usrquota) /dev/mapper/data-archive on /home/archive type xfs (rw,logbufs=8) evenlode:~# xfs_info /home meta-data=/dev/data/home isize=256 agcount=32, agsize=7372784 blks = sectsz=512 attr=0 data = bsize=4096 blocks=235929088, imaxpct=25 = sunit=16 swidth=48 blks naming =version 2 bsize=4096 log =internal bsize=4096 blocks=32768, version=2 = sectsz=512 sunit=16 blks, lazy-count=0 realtime =none extsz=65536 blocks=0, rtextents=0 evenlode:~# xfs_info /home/archive meta-data=/dev/data/archive isize=256 agcount=16, agsize=6553600 blks = sectsz=512 attr=0 data = bsize=4096 blocks=104857600, imaxpct=25 = sunit=16 swidth=48 blks naming =version 2 bsize=4096 log =internal bsize=4096 blocks=32768, version=2 = sectsz=512 sunit=16 blks, lazy-count=0 realtime =none extsz=65536 blocks=0, rtextents=0 ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: 2.6.24.3 nfs server on xfs keeps producing nfsd: non-standard errno: -117 2008-03-18 0:28 ` Timothy Shimmin 2008-03-18 1:07 ` Stuart Rowan @ 2008-03-18 13:49 ` Stuart Rowan 2008-03-20 1:09 ` Timothy Shimmin 1 sibling, 1 reply; 6+ messages in thread From: Stuart Rowan @ 2008-03-18 13:49 UTC (permalink / raw) To: xfs; +Cc: Timothy Shimmin Timothy Shimmin wrote, on 18/03/08 00:28: > Hi Stuart, > > Stuart Rowan wrote: >> Hi, >> >> Firstly thanks for the great filesystem and apologies if this ends up >> being NFS rather than XFS being weird! I'm not subscribed so please do >> keep me CC'd. >> >> I have *millions* of lines of (>200k per minute according to syslog): >> nfsd: non-standard errno: -117 >> being sent out of dmesg >> >> Now errno 117 is >> #define EUCLEAN 117 /* Structure needs cleaning */ >> which seems to be only used from a quick grep by XFS and JFFS and smbfs. >> >> > In XFS we mapped EFSCORRUPTED to EUCLEAN as EFSCORRUPTED > didn't exist on Linux. > However, normally if this error is encountered in XFS then > we output an appropriate msg to the syslog. > Our default error level is 3 and most reports are rated at 1 > so should show up I would have thought. > > --Tim > >> My nfs server export two locations >> /home >> /home/archive >> both of these are XFS partitions, hence my suspicion that the -117 is >> coming from XFS. >> >> xfs_repair -n says the filesystems are clean >> xfs_repair has been run multiple times to completion on the >> filesystems, all is fine. >> >> The XFS partitions are lvm volumes as follows >> data/home 900G >> data/archive 400G >> The volume group, data, is sda3 >> sda3 is a 6 drive 3ware 9550SXU-8LP RAID10 array >> >> The NFS server is currently in use (indeed the message only starts >> once clients connect) and works absolutely fine. >> >> How do I find out what (if anything) is wrong with my filesystem / >> appropriately silence this message? >> >> Many thanks, >> Stu. >> > > I briefly changed the sysctl fs.xfs.error_level to 6 and then back to 3 It gives the following message and backtrace > Mar 18 13:35:15 evenlode kernel: nfsd: non-standard errno: -117 > Mar 18 13:35:15 evenlode kernel: 0x0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > Mar 18 13:35:15 evenlode kernel: Filesystem "dm-0": XFS internal error xfs_itobp at line 360 of file fs/xfs/xfs_inode.c. Caller 0xffffffff8821224d > Mar 18 13:35:15 evenlode kernel: Pid: 2791, comm: nfsd Not tainted 2.6.24.3-generic #1 > Mar 18 13:35:15 evenlode kernel: > Mar 18 13:35:15 evenlode kernel: Call Trace: > Mar 18 13:35:15 evenlode kernel: [<ffffffff8821224d>] :xfs:xfs_iread+0x71/0x1e8 > Mar 18 13:35:15 evenlode kernel: [<ffffffff8820f784>] :xfs:xfs_itobp+0x141/0x17b > Mar 18 13:35:15 evenlode kernel: [<ffffffff8821224d>] :xfs:xfs_iread+0x71/0x1e8 > Mar 18 13:35:15 evenlode kernel: [<ffffffff8821224d>] :xfs:xfs_iread+0x71/0x1e8 > Mar 18 13:35:15 evenlode kernel: [<ffffffff8820d7c9>] :xfs:xfs_iget_core+0x352/0x63a > Mar 18 13:35:15 evenlode kernel: [<ffffffff8029095f>] alloc_inode+0x152/0x1c2 > Mar 18 13:35:15 evenlode kernel: [<ffffffff8820db4c>] :xfs:xfs_iget+0x9b/0x13f > Mar 18 13:35:15 evenlode kernel: [<ffffffff882243d1>] :xfs:xfs_vget+0x4d/0xbb > Mar 18 13:35:15 evenlode kernel: [<ffffffff8822f0d2>] :xfs:xfs_nfs_get_inode+0x2e/0x42 > Mar 18 13:35:15 evenlode kernel: [<ffffffff8822f1f9>] :xfs:xfs_fs_fh_to_dentry+0x64/0x97 > Mar 18 13:35:15 evenlode kernel: [<ffffffff88332554>] :exportfs:exportfs_decode_fh+0x30/0x1dc > Mar 18 13:35:15 evenlode kernel: [<ffffffff88347c16>] :nfsd:nfsd_acceptable+0x0/0xca > Mar 18 13:35:15 evenlode kernel: [<ffffffff8023b11a>] set_current_groups+0x148/0x153 > Mar 18 13:35:15 evenlode kernel: [<ffffffff8834df5f>] :nfsd:nfsd_setuser+0x11c/0x171 > Mar 18 13:35:15 evenlode kernel: [<ffffffff88347762>] :nfsd:nfsd_setuser_and_check_port+0x52/0x57 > Mar 18 13:35:15 evenlode kernel: [<ffffffff88347edb>] :nfsd:fh_verify+0x1fb/0x4a4 > Mar 18 13:35:15 evenlode kernel: [<ffffffff88261024>] :sunrpc:svc_tcp_recvfrom+0x7ab/0x843 > Mar 18 13:35:15 evenlode kernel: [<ffffffff88349141>] :nfsd:nfsd_open+0x1f/0x170 > Mar 18 13:35:15 evenlode kernel: [<ffffffff88349538>] :nfsd:nfsd_read+0x7f/0xc4 > Mar 18 13:35:15 evenlode kernel: [<ffffffff88350321>] :nfsd:nfsd3_proc_read+0x117/0x15a > Mar 18 13:35:15 evenlode kernel: [<ffffffff8834524e>] :nfsd:nfsd_dispatch+0xde/0x1c2 > Mar 18 13:35:15 evenlode kernel: [<ffffffff8825e192>] :sunrpc:svc_process+0x3f7/0x6e9 > Mar 18 13:35:15 evenlode kernel: [<ffffffff803fba8d>] __down_read+0x12/0x9a > Mar 18 13:35:15 evenlode kernel: [<ffffffff8834582f>] :nfsd:nfsd+0x191/0x2ae > Mar 18 13:35:15 evenlode kernel: [<ffffffff8020cbe8>] child_rip+0xa/0x12 > Mar 18 13:35:15 evenlode kernel: [<ffffffff8834569e>] :nfsd:nfsd+0x0/0x2ae > Mar 18 13:35:15 evenlode kernel: [<ffffffff8020cbde>] child_rip+0x0/0x12 > Mar 18 13:35:15 evenlode kernel: Does that help? Thanks, Stu. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: 2.6.24.3 nfs server on xfs keeps producing nfsd: non-standard errno: -117 2008-03-18 13:49 ` Stuart Rowan @ 2008-03-20 1:09 ` Timothy Shimmin 2008-03-20 8:25 ` XFS internal error xfs_itobp at line 360 of file fs/xfs/xfs_inode.c. (was Re: 2.6.24.3 nfs server on xfs keeps producing nfsd: non-standard errno: -117) Stuart Rowan 0 siblings, 1 reply; 6+ messages in thread From: Timothy Shimmin @ 2008-03-20 1:09 UTC (permalink / raw) To: strr-debian; +Cc: xfs Stuart Rowan wrote: > Timothy Shimmin wrote, on 18/03/08 00:28: >> Hi Stuart, >> >> Stuart Rowan wrote: >>> >>> I have *millions* of lines of (>200k per minute according to syslog): >>> nfsd: non-standard errno: -117 >>> being sent out of dmesg >>> >>> Now errno 117 is >>> #define EUCLEAN 117 /* Structure needs cleaning */ >>> >> In XFS we mapped EFSCORRUPTED to EUCLEAN as EFSCORRUPTED >> didn't exist on Linux. >> However, normally if this error is encountered in XFS then >> we output an appropriate msg to the syslog. >> Our default error level is 3 and most reports are rated at 1 >> so should show up I would have thought. >> >> --Tim >> >>> >>> xfs_repair -n says the filesystems are clean >>> xfs_repair has been run multiple times to completion on the >>> filesystems, all is fine. >>> >>> The NFS server is currently in use (indeed the message only starts >>> once clients connect) and works absolutely fine. >>> >>> How do I find out what (if anything) is wrong with my filesystem / >>> appropriately silence this message? >>> >> > I briefly changed the sysctl fs.xfs.error_level to 6 and then back to 3 > Good idea (I was thinking about that :-). Somehow, your subject line referring to 2.6.24 didn't stick in my brain (that's pretty old). So I was looking at recent code which I can't see has this error case from xfs_itobp() (it is now in xfs_imap_to_bp()). Looking at old code, I see 2 EFSCORRUPTED paths with the following one triggering at XFS_ERRLEVEL_HIGH (and presumably why you didn't see it until now) ... montep |1.198| | /* montep |1.198| | * Validate the magic number and version of every inode in the buffer montep |1.198| | * (if DEBUG kernel) or the first inode in the buffer, otherwise. montep |1.198| | */ nathans |1.303|2.4.x-xfs:slinx:74929a |#ifdef DEBUG montep |1.198| | ni = BBTOB(imap.im_len) >> mp->m_sb.sb_inodelog; montep |1.198| |#else montep |1.198| | ni = 1; montep |1.198| |#endif montep |1.198| | for (i = 0; i < ni; i++) { doucette |1.245|irix6.5f:irix:09146b | int di_ok; doucette |1.245|irix6.5f:irix:09146b | xfs_dinode_t *dip; doucette |1.245|irix6.5f:irix:09146b | lord |1.292|2.4.0-test1-xfs:slinx:65571a| dip = (xfs_dinode_t *)xfs_buf_offset(bp, montep |1.198| | (i << mp->m_sb.sb_inodelog)); dxm |1.285|2.4.0-test1-xfs:slinx:62350a| di_ok = INT_GET(dip->di_core.di_magic, ARCH_CONVERT) == XFS_DINODE_MAGIC && dxm |1.285|2.4.0-test1-xfs:slinx:62350a| XFS_DINODE_GOOD_VERSION(INT_GET(dip->di_core.di_version, ARCH_CONVERT)); overby |1.362|2.4.x-xfs:slinx:136445a | if (unlikely(XFS_TEST_ERROR(!di_ok, mp, XFS_ERRTAG_ITOBP_INOTOBP, overby |1.362|2.4.x-xfs:slinx:136445a | XFS_RANDOM_ITOBP_INOTOBP))) { montep |1.198| |#ifdef DEBUG nathans |1.337|2.4.x-xfs:slinx:119399a | prdev("bad inode magic/vsn daddr 0x%llx #%d (magic=%x)", nathans |1.337|2.4.x-xfs:slinx:119399a | mp->m_dev, (unsigned long long)imap.im_blkno, i, nathans |1.303|2.4.x-xfs:slinx:74929a | INT_GET(dip->di_core.di_magic, ARCH_CONVERT)); montep |1.198| |#endif lord |1.376|2.4.x-xfs:slinx:150747a | XFS_CORRUPTION_ERROR("xfs_itobp", XFS_ERRLEVEL_HIGH, overby |1.362|2.4.x-xfs:slinx:136445a | mp, dip); montep |1.198| | xfs_trans_brelse(tp, bp); sup |1.216| | return XFS_ERROR(EFSCORRUPTED); montep |1.198| | } ajs |1.143| | } So the first inode in the buffer has the wrong magic# or version#. I'm surprised that this wasn't picked up by repair or check. --Tim > It gives the following message and backtrace > >> Mar 18 13:35:15 evenlode kernel: nfsd: non-standard errno: -117 >> Mar 18 13:35:15 evenlode kernel: 0x0: 00 00 00 00 00 00 00 00 00 00 00 >> 00 00 00 00 00 Mar 18 13:35:15 evenlode kernel: Filesystem "dm-0": XFS >> internal error xfs_itobp at line 360 of file fs/xfs/xfs_inode.c. >> Caller 0xffffffff8821224d >> Mar 18 13:35:15 evenlode kernel: Pid: 2791, comm: nfsd Not tainted >> 2.6.24.3-generic #1 >> Mar 18 13:35:15 evenlode kernel: Mar 18 13:35:15 evenlode kernel: Call >> Trace: >> Mar 18 13:35:15 evenlode kernel: [<ffffffff8821224d>] >> :xfs:xfs_iread+0x71/0x1e8 >> Mar 18 13:35:15 evenlode kernel: [<ffffffff8820f784>] >> :xfs:xfs_itobp+0x141/0x17b >> Mar 18 13:35:15 evenlode kernel: [<ffffffff8821224d>] >> :xfs:xfs_iread+0x71/0x1e8 >> Mar 18 13:35:15 evenlode kernel: [<ffffffff8821224d>] >> :xfs:xfs_iread+0x71/0x1e8 >> Mar 18 13:35:15 evenlode kernel: [<ffffffff8820d7c9>] >> :xfs:xfs_iget_core+0x352/0x63a >> Mar 18 13:35:15 evenlode kernel: [<ffffffff8029095f>] >> alloc_inode+0x152/0x1c2 >> Mar 18 13:35:15 evenlode kernel: [<ffffffff8820db4c>] >> :xfs:xfs_iget+0x9b/0x13f >> Mar 18 13:35:15 evenlode kernel: [<ffffffff882243d1>] >> :xfs:xfs_vget+0x4d/0xbb > > > Does that help? > > Thanks, > Stu. ^ permalink raw reply [flat|nested] 6+ messages in thread
* XFS internal error xfs_itobp at line 360 of file fs/xfs/xfs_inode.c. (was Re: 2.6.24.3 nfs server on xfs keeps producing nfsd: non-standard errno: -117) 2008-03-20 1:09 ` Timothy Shimmin @ 2008-03-20 8:25 ` Stuart Rowan 0 siblings, 0 replies; 6+ messages in thread From: Stuart Rowan @ 2008-03-20 8:25 UTC (permalink / raw) To: Timothy Shimmin; +Cc: strr-debian, xfs On Thu, 20 Mar 2008, Timothy Shimmin wrote: > Stuart Rowan wrote: >> Timothy Shimmin wrote, on 18/03/08 00:28: >>> Hi Stuart, >>> >>> Stuart Rowan wrote: >>>> >>>> I have *millions* of lines of (>200k per minute according to syslog): >>>> nfsd: non-standard errno: -117 >>>> being sent out of dmesg >>>> >>>> Now errno 117 is >>>> #define EUCLEAN 117 /* Structure needs cleaning */ >>>> >>> In XFS we mapped EFSCORRUPTED to EUCLEAN as EFSCORRUPTED >>> didn't exist on Linux. >>> However, normally if this error is encountered in XFS then >>> we output an appropriate msg to the syslog. >>> Our default error level is 3 and most reports are rated at 1 >>> so should show up I would have thought. >>> >>> --Tim >>> >>>> >>>> xfs_repair -n says the filesystems are clean >>>> xfs_repair has been run multiple times to completion on the filesystems, >>>> all is fine. >>>> >>>> The NFS server is currently in use (indeed the message only starts once >>>> clients connect) and works absolutely fine. >>>> >>>> How do I find out what (if anything) is wrong with my filesystem / >>>> appropriately silence this message? >>>> >>> >> I briefly changed the sysctl fs.xfs.error_level to 6 and then back to 3 >> > Good idea (I was thinking about that :-). > > Somehow, your subject line referring to 2.6.24 didn't stick in > my brain (that's pretty old). > So I was looking at recent code which I can't see has this error > case from xfs_itobp() (it is now in xfs_imap_to_bp()). > Pretty old for you, latest released Linux kernel to me :-P > Looking at old code, I see 2 EFSCORRUPTED paths with the following > one triggering at XFS_ERRLEVEL_HIGH (and presumably why you didn't > see it until now) ... > > montep |1.198| | /* > montep |1.198| | * Validate the magic number > and version of every inode in the buffer > montep |1.198| | * (if DEBUG kernel) or the > first inode in the buffer, otherwise. > montep |1.198| | */ > nathans |1.303|2.4.x-xfs:slinx:74929a |#ifdef DEBUG > montep |1.198| | ni = BBTOB(imap.im_len) >> > mp->m_sb.sb_inodelog; > montep |1.198| |#else > montep |1.198| | ni = 1; > montep |1.198| |#endif > montep |1.198| | for (i = 0; i < ni; i++) { > doucette |1.245|irix6.5f:irix:09146b | int > di_ok; > doucette |1.245|irix6.5f:irix:09146b | xfs_dinode_t *dip; > doucette |1.245|irix6.5f:irix:09146b | > lord |1.292|2.4.0-test1-xfs:slinx:65571a| dip = (xfs_dinode_t > *)xfs_buf_offset(bp, > montep |1.198| | > (i << mp->m_sb.sb_inodelog)); > dxm |1.285|2.4.0-test1-xfs:slinx:62350a| di_ok = > INT_GET(dip->di_core.di_magic, ARCH_CONVERT) == XFS_DINODE_MAGIC && > dxm |1.285|2.4.0-test1-xfs:slinx:62350a| > XFS_DINODE_GOOD_VERSION(INT_GET(dip->di_core.di_version, ARCH_CONVERT)); > overby |1.362|2.4.x-xfs:slinx:136445a | if > (unlikely(XFS_TEST_ERROR(!di_ok, mp, XFS_ERRTAG_ITOBP_INOTOBP, > overby |1.362|2.4.x-xfs:slinx:136445a | > XFS_RANDOM_ITOBP_INOTOBP))) { > montep |1.198| |#ifdef DEBUG > nathans |1.337|2.4.x-xfs:slinx:119399a | prdev("bad > inode magic/vsn daddr 0x%llx #%d (magic=%x)", > nathans |1.337|2.4.x-xfs:slinx:119399a | > mp->m_dev, (unsigned long long)imap.im_blkno, i, > nathans |1.303|2.4.x-xfs:slinx:74929a | > INT_GET(dip->di_core.di_magic, ARCH_CONVERT)); > montep |1.198| |#endif > lord |1.376|2.4.x-xfs:slinx:150747a | > XFS_CORRUPTION_ERROR("xfs_itobp", XFS_ERRLEVEL_HIGH, > overby |1.362|2.4.x-xfs:slinx:136445a | > mp, dip); > montep |1.198| | > xfs_trans_brelse(tp, bp); > sup |1.216| | return > XFS_ERROR(EFSCORRUPTED); > montep |1.198| | } > ajs |1.143| | } > > So the first inode in the buffer has the wrong magic# or version#. > I'm surprised that this wasn't picked up by repair or check. > > --Tim > I have some more information! The server, evenlode, was previously serving NFS exports of ext3 filesystems. Last week we rsycned the data to the new server running XFS. Eventually I spotted the high error rate was linked to the volume of NFS read calls (200k / minute). A quick tcpdump gave me a couple of likely looking hosts. I logged into one (bonny) and found gnome-panel using 100% CPU. I killed that and these messages have now reduced to a handful an hour. That gnome-panel will have had the NFS server and underlying NFS backing filesystem (ext3-> XFS) changed underneath it. So my questions ... Is it possible that the errors are related to duff request data being sent by the NFS clients because they are still referencing e.g. inodes as they were when the NFS server was ext3 backed? Is it also possible that things like the rather high request rate (200k/sec) although that's reduced now, made a race in e.g. the XFS code triggerable? As you say, it's rather suprising that this sort of issue is not being caught by xfs_repair (-n) and that's what leads me to suspect something else at play ... Cheers, Stu. >> It gives the following message and backtrace >> >>> Mar 18 13:35:15 evenlode kernel: nfsd: non-standard errno: -117 >>> Mar 18 13:35:15 evenlode kernel: 0x0: 00 00 00 00 00 00 00 00 00 00 00 00 >>> 00 00 00 00 Mar 18 13:35:15 evenlode kernel: Filesystem "dm-0": XFS >>> internal error xfs_itobp at line 360 of file fs/xfs/xfs_inode.c. Caller >>> 0xffffffff8821224d >>> Mar 18 13:35:15 evenlode kernel: Pid: 2791, comm: nfsd Not tainted >>> 2.6.24.3-generic #1 >>> Mar 18 13:35:15 evenlode kernel: Mar 18 13:35:15 evenlode kernel: Call >>> Trace: >>> Mar 18 13:35:15 evenlode kernel: [<ffffffff8821224d>] >>> :xfs:xfs_iread+0x71/0x1e8 >>> Mar 18 13:35:15 evenlode kernel: [<ffffffff8820f784>] >>> :xfs:xfs_itobp+0x141/0x17b >>> Mar 18 13:35:15 evenlode kernel: [<ffffffff8821224d>] >>> :xfs:xfs_iread+0x71/0x1e8 >>> Mar 18 13:35:15 evenlode kernel: [<ffffffff8821224d>] >>> :xfs:xfs_iread+0x71/0x1e8 >>> Mar 18 13:35:15 evenlode kernel: [<ffffffff8820d7c9>] >>> :xfs:xfs_iget_core+0x352/0x63a >>> Mar 18 13:35:15 evenlode kernel: [<ffffffff8029095f>] >>> alloc_inode+0x152/0x1c2 >>> Mar 18 13:35:15 evenlode kernel: [<ffffffff8820db4c>] >>> :xfs:xfs_iget+0x9b/0x13f >>> Mar 18 13:35:15 evenlode kernel: [<ffffffff882243d1>] >>> :xfs:xfs_vget+0x4d/0xbb >> >> >> Does that help? >> >> Thanks, >> Stu. > > > ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2008-03-20 8:25 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2008-03-17 23:27 2.6.24.3 nfs server on xfs keeps producing nfsd: non-standard errno: -117 Stuart Rowan 2008-03-18 0:28 ` Timothy Shimmin 2008-03-18 1:07 ` Stuart Rowan 2008-03-18 13:49 ` Stuart Rowan 2008-03-20 1:09 ` Timothy Shimmin 2008-03-20 8:25 ` XFS internal error xfs_itobp at line 360 of file fs/xfs/xfs_inode.c. (was Re: 2.6.24.3 nfs server on xfs keeps producing nfsd: non-standard errno: -117) Stuart Rowan
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox