* Re: stale nfs file handle with exported loopback mounts
@ 2007-10-31 22:50 devzero
2007-11-01 4:26 ` Neil Brown
0 siblings, 1 reply; 17+ messages in thread
From: devzero @ 2007-10-31 22:50 UTC (permalink / raw)
To: J. Bruce Fields; +Cc: Neil Brown, NFS
ok, i just wanted to tell that this isn`t the right way to go imho.
some time ago i have tested exporting a parent dir containing several loopb=
ack mounted iso images with some pre-1.1.0 nfs-utils version and it worked =
- so =EC wonder why it now seems to have issues as things should have gone =
stable.....
> -----Urspr=FCngliche Nachricht-----
> Von: "J. Bruce Fields" <bfields@fieldses.org>
> Gesendet: 31.10.07 23:39:46
> An: devzero@web.de
> CC: NFS@lists.sourceforge.net, Neil Brown <neilb@suse.de>
> Betreff: Re: [NFS] stale nfs file handle with exported loopback mounts
> =
> On Wed, Oct 31, 2007 at 11:19:43PM +0100, devzero@web.de wrote:
> > >If you add an explicit export for each one, =
> > you mean, i should export each of those ~500 loopback mounted iso image=
s ?
> > come on, that`s not what admins or users want, isn`t it ?
> =
> No, of course not; I was just suggesting it as a way to confirm that's
> where the problem is.
> =
> --b.
> =
> > =
> > > -----Urspr=FCngliche Nachricht-----
> > > Von: "J. Bruce Fields" <bfields@fieldses.org>
> > > Gesendet: 31.10.07 21:58:00
> > > An: devzero@web.de
> > > CC: Neil Brown <neilb@suse.de>, NFS@lists.sourceforge.net
> > > Betreff: Re: [NFS] stale nfs file handle with exported loopback mounts
> > =
> > =
> > > =
> > > On Wed, Oct 31, 2007 at 09:46:13PM +0100, devzero@web.de wrote:
> > > > hi !
> > > > =
> > > > i tried latest grml build (lenny/sid) as server today (suse 9.3 as =
client).
> > > > =
> > > > error still existing, but a little bit different:
> > > > =
> > > > if i cd to the loopback-mount dir`s on the client, i see the conten=
ts of the parent directory, i.e. i get a recursion.
> > > =
> > > That suggests something funky in the way we're identifying those
> > > filesystems in the filehandle. If you add an explicit export for each
> > > one, each with its own "fsid=3Dxyz" option (with xyz whatever positive
> > > integer you'd like, as long as it's difficult for export), does the
> > > problem go away??
> > > =
> > > --b.
> > > =
> > > > =
> > > > regards
> > > > roland
> > > > =
> > > > =
> > > > =
> > > > > -----Urspr=FCngliche Nachricht-----
> > > > > Von: devzero@web.de
> > > > > Gesendet: 30.10.07 21:05:56
> > > > > An: Neil Brown <neilb@suse.de>
> > > > > CC: NFS@lists.sourceforge.net
> > > > > Betreff: Re: [NFS] stale nfs file handle with exported loopback m=
ounts
> > > > =
> > > > =
> > > > > =
> > > > > > I recommend replacing subtree_check with no_subtree_check, but =
it
> > > > > > shouldn't make an important difference in this case.
> > > > > =
> > > > > ok, i leave it as is.
> > > > > =
> > > > > > This should work with nfs-utils 1.1.0 or later. With earlier r=
eleases
> > > > > > you need to explicitly export the subordinate filesystems too.
> > > > > > =
> > > > > =
> > > > > mhh - opensuse doesn`t have nfs-utils package, but it has nfs-cli=
ent-1.1.0-8 which looks like they repackaged nfs-utils 1.1.0
> > > > > =
> > > > > > It is a little odd that the errors are inconsistent.
> > > > > =
> > > > > ok, but only a minor issue, if an issue at all, isn`t it ?
> > > > > =
> > > > > > Can you find any log messages from mountd in syslog? What do t=
hey
> > > > > > say?
> > > > > =
> > > > > yes, on the client i`m getting :
> > > > > =
> > > > > Jun 3 21:36:01 linux kernel: nfs_update_inode: inode number mism=
atch
> > > > > Jun 3 21:36:01 linux kernel: expected (0:11/0x2), got (0:11/0x13=
881)
> > > > > Jun 3 21:36:01 linux kernel: nfs_update_inode: inode number mism=
atch
> > > > > Jun 3 21:36:01 linux kernel: expected (0:11/0x2), got (0:11/0x13=
881)
> > > > > Jun 3 21:36:17 linux kernel: nfs_update_inode: inode number mism=
atch
> > > > > Jun 3 21:36:17 linux kernel: expected (0:11/0x2), got (0:11/0x13=
881)
> > > > > Jun 3 21:36:17 linux kernel: nfs_update_inode: inode number mism=
atch
> > > > > Jun 3 21:36:17 linux kernel: expected (0:11/0x2), got (0:11/0x13=
881)
> > > > > Jun 3 21:36:20 linux kernel: nfs_update_inode: inode number mism=
atch
> > > > > Jun 3 21:36:20 linux kernel: expected (0:11/0x2), got (0:11/0x13=
881)
> > > > > Jun 3 21:36:20 linux kernel: nfs_update_inode: inode number mism=
atch
> > > > > =
> > > > > =
> > > > > no error message on the server:
> > > > > Oct 26 10:09:31 opensuse103 mountd[4293]: authenticated unmount r=
equest from 10.0.0.40:1014 for /mnt (/mnt)
> > > > > Oct 26 10:10:07 opensuse103 mountd[4293]: authenticated mount req=
uest from 10.0.0.40:612 for /mnt (/mnt)
> > > > > Oct 26 10:10:43 opensuse103 mountd[4293]: authenticated unmount r=
equest from 10.0.0.40:623 for /mnt (/mnt)
> > > > > Oct 26 10:10:52 opensuse103 mountd[4293]: authenticated mount req=
uest from 10.0.0.40:624 for /mnt (/mnt)
> > > > > =
> > > > > > Also what does
> > > > > > cat /proc/fs/nfsd/exports
> > > > > > =
> > > > > > on the server show.
> > > > > =
> > > > > opensuse103:~ # cat /proc/fs/nfsd/exports
> > > > > # Version 1.1
> > > > > # Path Client(Flags) # IPs
> > > > > /mnt/iso1 *(ro,no_root_squash,sync,wdelay,crossmnt,uuid=3D2=
c49fef2:ba464293:9b2bf2b8:322ccbcb)
> > > > > /mnt/iso3 *(ro,no_root_squash,sync,wdelay,crossmnt,uuid=3D2=
7ae9c67:0b794b36:8b5e9e17:37b569eb)
> > > > > /mnt *(ro,no_root_squash,sync,wdelay,crossmnt,uuid=3D08164ee4:=
2db141eb:ac961701:49c74396)
> > > > > /mnt/iso2 *(ro,no_root_squash,sync,wdelay,crossmnt,uuid=3D2=
aad6ea5:a05d4441:b94c48e6:e5d9981e)
> > > > > =
> > > > > =
> > > > > > Finally, a tcpdump:
> > > > > > =
> > > > > > tcpdump -s 0 -w /tmp/tcpdump port 2049
> > > > > > =
> > > > > > while you run the experiment might help.
> > > > > =
> > > > > ah - this seems to give a hint, but i don`t have a clue why the s=
erver (10.0.0.30) is telling the client (10.0.0.40) a "RPC Version mismatch=
".
> > > > > I also tried --no-nfs-version 4 for rpc.mountd (setting in /etc/=
sysconfig/nfs), but this didn`t make a difference.
> > > > > =
> > > > > here is the tcpdump output - i did =
> > > > > =
> > > > > - mount
> > > > > - ls / ls -la / cd to subdirs
> > > > > =
> > > > > 10:15:06.480542 IP 10.0.0.40.0 > 10.0.0.30.2049: 0 null
> > > > > 10:15:06.480572 IP 10.0.0.30.2049 > 10.0.0.40.0: reply ERR 0: RPC=
Version mismatch (167772160-0)
> > > > > 10:15:06.480765 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 2031114=
913 win 1460 <nop,nop,timestamp 686128 737918>
> > > > > 10:15:06.480821 IP 10.0.0.40.2079804678 > 10.0.0.30.2049: 108 fsi=
nfo fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743963CD8=
1908
> > > > > 10:15:06.480833 IP 10.0.0.30.2049 > 10.0.0.40.1022: . ack 108 win=
181 <nop,nop,timestamp 737918 686128>
> > > > > 10:15:06.513365 IP 10.0.0.30.2049 > 10.0.0.40.2079804678: reply o=
k 84 fsinfo rtmax 65536 rtpref 65536 wtmax 65536 wtpref 65536 dtpref 4096
> > > > > 10:15:06.514408 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 85 win =
1460 <nop,nop,timestamp 686160 737926>
> > > > > 10:15:06.514902 IP 10.0.0.40.2096581894 > 10.0.0.30.2049: 108 get=
attr fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743963CD=
81908
> > > > > 10:15:06.515256 IP 10.0.0.30.2049 > 10.0.0.40.2096581894: reply o=
k 116 getattr DIR 40755 ids 0/0 sz 4096
> > > > > 10:15:06.553865 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 201 win=
1460 <nop,nop,timestamp 686201 737926>
> > > > > 10:16:03.091784 IP 10.0.0.40.2113359110 > 10.0.0.30.2049: 108 get=
attr fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C74396472=
19C8C
> > > > > 10:16:03.093366 IP 10.0.0.30.2049 > 10.0.0.40.2113359110: reply o=
k 116 getattr DIR 40755 ids 0/0 sz 4096
> > > > > 10:16:03.096046 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 317 win=
1460 <nop,nop,timestamp 743610 752071>
> > > > > 10:16:03.097370 IP 10.0.0.40.2130136326 > 10.0.0.30.2049: 112 acc=
ess fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000=
001F 001f
> > > > > 10:16:03.098156 IP 10.0.0.30.2049 > 10.0.0.40.2130136326: reply o=
k 124 access c 0003
> > > > > 10:16:03.156812 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 441 win=
1460 <nop,nop,timestamp 743651 752072>
> > > > > 10:16:08.967804 IP 10.0.0.40.2146913542 > 10.0.0.30.2049: 108 get=
attr fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C74396000=
00000
> > > > > 10:16:08.968305 IP 10.0.0.30.2049 > 10.0.0.40.2146913542: reply o=
k 116 getattr DIR 40755 ids 0/0 sz 4096
> > > > > 10:16:08.974285 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 557 win=
1460 <nop,nop,timestamp 749477 753540>
> > > > > 10:16:08.975739 IP 10.0.0.40.2163690758 > 10.0.0.30.2049: 132 rea=
ddirplus fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439=
600000000 512 bytes @ 0
> > > > > 10:16:08.975985 IP 10.0.0.30.2049 > 10.0.0.40.2163690758: reply o=
k 1448 readdirplus
> > > > > 10:16:08.976510 IP 10.0.0.30.2049 > 10.0.0.40.1684108288: reply U=
nknown rpc response code=3D2021855861 340
> > > > > 10:16:08.982238 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 2345 wi=
n 2184 <nop,nop,timestamp 749479 753541>
> > > > > 10:16:08.984415 IP 10.0.0.40.2180467974 > 10.0.0.30.2049: 116 loo=
kup fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000=
0004 "iso3"
> > > > > 10:16:08.984931 IP 10.0.0.30.2049 > 10.0.0.40.2180467974: reply o=
k 232 lookup fh Unknown/0100060027AE9C670B794B368B5E9E1737B569EB00000001000=
00002000041ED
> > > > > 10:16:09.008819 IP 10.0.0.40.2197245190 > 10.0.0.30.2049: 104 get=
attr fh Unknown/0100060027AE9C670B794B368B5E9E1737B569EB0000000FDD22CC3C700=
2F0AC
> > > > > 10:16:09.010946 IP 10.0.0.30.2049 > 10.0.0.40.2197245190: reply o=
k 188 getattr REG 2 ids 5/0 sz 0
> > > > > 10:16:09.032541 IP 10.0.0.40.2214022406 > 10.0.0.30.2049: 128 get=
attr fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E66=
D0700
> > > > > 10:16:09.033405 IP 10.0.0.30.2049 > 10.0.0.40.2214022406: reply o=
k 188 getattr REG 1 ids 1/0 sz 0
> > > > > 10:16:09.033490 IP 10.0.0.40.2230799622 > 10.0.0.30.2049: 128 get=
attr fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E56=
D0700
> > > > > 10:16:09.036811 IP 10.0.0.30.2049 > 10.0.0.40.2230799622: reply o=
k 188 getattr REG 1 ids 1/0 sz 0
> > > > > 10:16:09.037823 IP 10.0.0.40.2247576838 > 10.0.0.30.2049: 108 get=
attr fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C74396000=
00000
> > > > > 10:16:09.039817 IP 10.0.0.30.2049 > 10.0.0.40.2247576838: reply o=
k 116 getattr DIR 40755 ids 0/0 sz 4096
> > > > > 10:16:09.040423 IP 10.0.0.40.2264354054 > 10.0.0.30.2049: 112 get=
attr fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C74396000=
0000F
> > > > > 10:16:09.040856 IP 10.0.0.30.2049 > 10.0.0.40.2264354054: reply o=
k 188 getattr REG 2 ids 5/0 sz 0
> > > > > 10:16:09.041590 IP 10.0.0.40.2281131270 > 10.0.0.30.2049: 116 loo=
kup fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000=
0004 "iso2"
> > > > > 10:16:09.041920 IP 10.0.0.30.2049 > 10.0.0.40.2281131270: reply o=
k 232 lookup fh Unknown/010006002AAD6EA5A05D4441B94C48E6E5D9981E00000001000=
00002000041ED
> > > > > 10:16:09.049633 IP 10.0.0.40.2297908486 > 10.0.0.30.2049: 104 get=
attr fh Unknown/010006002AAD6EA5A05D4441B94C48E6E5D9981E0000000F5FDC8445432=
6E193
> > > > > 10:16:09.049781 IP 10.0.0.30.2049 > 10.0.0.40.2297908486: reply o=
k 188 getattr REG 2 ids 5/0 sz 0
> > > > > 10:16:09.063218 IP 10.0.0.40.2314685702 > 10.0.0.30.2049: 128 get=
attr fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E46=
D0700
> > > > > 10:16:09.072505 IP 10.0.0.30.2049 > 10.0.0.40.2314685702: reply o=
k 188 getattr REG 1 ids 1/0 sz 0
> > > > > 10:16:09.091698 IP 10.0.0.40.2331462918 > 10.0.0.30.2049: 124 get=
attr fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E46=
D0700
> > > > > 10:16:09.092022 IP 10.0.0.30.2049 > 10.0.0.40.2331462918: reply o=
k 116 getattr REG 100644 ids 0/0 sz 1048576
> > > > > 10:16:09.128971 IP 10.0.0.40.2348240134 > 10.0.0.30.2049: 128 get=
attr fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E36=
D0700
> > > > > 10:16:09.129304 IP 10.0.0.30.2049 > 10.0.0.40.2348240134: reply o=
k 188 getattr REG 1 ids 1/0 sz 0
> > > > > 10:16:09.184155 IP 10.0.0.40.2365017350 > 10.0.0.30.2049: 124 get=
attr fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E36=
D0700
> > > > > 10:16:09.184582 IP 10.0.0.30.2049 > 10.0.0.40.2365017350: reply o=
k 116 getattr REG 100644 ids 0/0 sz 1048576
> > > > > 10:16:09.189234 IP 10.0.0.40.2381794566 > 10.0.0.30.2049: 116 loo=
kup fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000=
0004 "iso1"
> > > > > 10:16:09.189435 IP 10.0.0.30.2049 > 10.0.0.40.2381794566: reply o=
k 232 lookup fh Unknown/010006002C49FEF2BA4642939B2BF2B8322CCBCB00000001000=
00002000041ED
> > > > > 10:16:09.193476 IP 10.0.0.40.2398571782 > 10.0.0.30.2049: 104 get=
attr fh Unknown/010006002C49FEF2BA4642939B2BF2B8322CCBCB0000000F58896A884A0=
C62B7
> > > > > 10:16:09.193652 IP 10.0.0.30.2049 > 10.0.0.40.2398571782: reply o=
k 188 getattr REG 2 ids 5/0 sz 0
> > > > > 10:16:09.194937 IP 10.0.0.40.2415348998 > 10.0.0.30.2049: 128 get=
attr fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E76=
D0700
> > > > > 10:16:09.195033 IP 10.0.0.30.2049 > 10.0.0.40.2415348998: reply o=
k 188 getattr REG 1 ids 1/0 sz 0
> > > > > 10:16:09.195230 IP 10.0.0.40.2432126214 > 10.0.0.30.2049: 128 get=
attr fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E86=
D0700
> > > > > 10:16:09.323635 IP 10.0.0.30.2049 > 10.0.0.40.1022: . ack 2572 wi=
n 416 <nop,nop,timestamp 753629 749566>
> > > > > 10:16:09.324345 IP 10.0.0.30.2049 > 10.0.0.40.2432126214: reply o=
k 188 getattr REG 1 ids 1/0 sz 0
> > > > > 10:16:09.341475 IP 10.0.0.40.2448903430 > 10.0.0.30.2049: 128 get=
attr fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E26=
D0700
> > > > > 10:16:09.341524 IP 10.0.0.30.2049 > 10.0.0.40.1022: . ack 2700 wi=
n 449 <nop,nop,timestamp 753632 749707>
> > > > > 10:16:09.341928 IP 10.0.0.30.2049 > 10.0.0.40.2448903430: reply o=
k 188 getattr REG 1 ids 1/0 sz 0
> > > > > 10:16:09.342117 IP 10.0.0.40.2465680646 > 10.0.0.30.2049: 124 get=
attr fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E26=
D0700
> > > > > 10:16:09.343404 IP 10.0.0.30.2049 > 10.0.0.40.2465680646: reply o=
k 116 getattr REG 100644 ids 0/0 sz 1048576
> > > > > 10:16:09.389316 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 5573 wi=
n 2184 <nop,nop,timestamp 749751 753633>
> > > > > 10:16:13.449513 IP 10.0.0.40.2482457862 > 10.0.0.30.2049: 108 get=
attr fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C74396472=
19CB5
> > > > > 10:16:13.449815 IP 10.0.0.30.2049 > 10.0.0.40.2482457862: reply o=
k 116 getattr DIR 40755 ids 0/0 sz 4096
> > > > > 10:16:13.452344 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 5689 wi=
n 2184 <nop,nop,timestamp 753943 754660>
> > > > > 10:16:13.453973 IP 10.0.0.40.2499235078 > 10.0.0.30.2049: 100 get=
attr fh Unknown/010006002C49FEF2BA4642939B2BF2B8322CCBCB47219C8C00000000472=
19C8C
> > > > > 10:16:13.454154 IP 10.0.0.30.2049 > 10.0.0.40.2499235078: reply o=
k 116 getattr DIR 40755 ids 0/0 sz 4096
> > > > > 10:16:13.456194 IP 10.0.0.40.2516012294 > 10.0.0.30.2049: 108 get=
attr fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C74396472=
19C8C
> > > > > 10:16:13.456361 IP 10.0.0.30.2049 > 10.0.0.40.2516012294: reply o=
k 116 getattr DIR 40755 ids 0/0 sz 4096
> > > > > 10:16:13.458282 IP 10.0.0.40.2532789510 > 10.0.0.30.2049: 116 loo=
kup fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000=
0004 "iso1"
> > > > > 10:16:13.458461 IP 10.0.0.30.2049 > 10.0.0.40.2532789510: reply o=
k 232 lookup fh Unknown/010006002C49FEF2BA4642939B2BF2B8322CCBCB00000001000=
00002000041ED
> > > > > 10:16:13.510637 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 6153 wi=
n 2184 <nop,nop,timestamp 753985 754662>
> > > > > 10:16:14.030110 IP 10.0.0.40.2549566726 > 10.0.0.30.2049: 112 acc=
ess fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000=
001F 001f
> > > > > 10:16:14.030927 IP 10.0.0.30.2049 > 10.0.0.40.2549566726: reply o=
k 124 access c 0003
> > > > > 10:16:14.033436 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 6277 wi=
n 2184 <nop,nop,timestamp 754548 754805>
> > > > > 10:16:14.034732 IP 10.0.0.40.2566343942 > 10.0.0.30.2049: 108 get=
attr fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C74396000=
00000
> > > > > 10:16:14.034980 IP 10.0.0.30.2049 > 10.0.0.40.2566343942: reply o=
k 116 getattr DIR 40755 ids 0/0 sz 4096
> > > > > 10:16:14.037319 IP 10.0.0.40.2583121158 > 10.0.0.30.2049: 112 acc=
ess fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000=
001F 001f
> > > > > 10:16:14.037486 IP 10.0.0.30.2049 > 10.0.0.40.2583121158: reply o=
k 124 access c 0003
> > > > > 10:16:14.040323 IP 10.0.0.40.2599898374 > 10.0.0.30.2049: 132 rea=
ddirplus fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439=
600000000 512 bytes @ 0
> > > > > 10:16:14.040554 IP 10.0.0.30.2049 > 10.0.0.40.2599898374: reply o=
k 1448 readdirplus
> > > > > 10:16:14.041020 IP 10.0.0.30.2049 > 10.0.0.40.1684108288: reply U=
nknown rpc response code=3D2021855861 340
> > > > > 10:16:14.043583 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 8305 wi=
n 2908 <nop,nop,timestamp 754550 754808>
> > > > > 10:16:14.045104 IP 10.0.0.40.2616675590 > 10.0.0.30.2049: 112 acc=
ess fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000=
001F 001f
> > > > > 10:16:14.045402 IP 10.0.0.30.2049 > 10.0.0.40.2616675590: reply o=
k 124 access c 0003
> > > > > 10:16:14.047830 IP 10.0.0.40.2633452806 > 10.0.0.30.2049: 112 acc=
ess fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000=
001F 001f
> > > > > 10:16:14.048039 IP 10.0.0.30.2049 > 10.0.0.40.2633452806: reply o=
k 124 access c 0003
> > > > > 10:16:14.099385 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 8553 wi=
n 2908 <nop,nop,timestamp 754592 754810>
> > > > > 10:16:14.293714 IP 10.0.0.40.2650230022 > 10.0.0.30.2049: 108 get=
attr fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C74396000=
00000
> > > > > 10:16:14.294019 IP 10.0.0.30.2049 > 10.0.0.40.2650230022: reply o=
k 116 getattr DIR 40755 ids 0/0 sz 4096
> > > > > 10:16:14.297263 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 8669 wi=
n 2908 <nop,nop,timestamp 754763 754871>
> > > > > 10:16:14.297272 IP 10.0.0.40.2667007238 > 10.0.0.30.2049: 112 acc=
ess fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000=
001F 001f
> > > > > 10:16:14.297466 IP 10.0.0.30.2049 > 10.0.0.40.2667007238: reply o=
k 124 access c 0003
> > > > > 10:16:14.297586 IP 10.0.0.40.2683784454 > 10.0.0.30.2049: 112 acc=
ess fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000=
001F 001f
> > > > > 10:16:14.297987 IP 10.0.0.30.2049 > 10.0.0.40.2683784454: reply o=
k 124 access c 0003
> > > > > 10:16:14.301165 IP 10.0.0.40.2700561670 > 10.0.0.30.2049: 104 acc=
ess fh Unknown/010006002C49FEF2BA4642939B2BF2B8322CCBCB0000001F47219C8C0000=
0000 001f
> > > > > 10:16:14.301405 IP 10.0.0.30.2049 > 10.0.0.40.2700561670: reply o=
k 124 access c 0003
> > > > > 10:16:14.351229 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 9041 wi=
n 2908 <nop,nop,timestamp 754810 754873>
> > > > > 10:16:14.842633 IP 10.0.0.40.2717338886 > 10.0.0.30.2049: 100 get=
attr fh Unknown/010006002C49FEF2BA4642939B2BF2B8322CCBCB0000000047219C8C000=
00000
> > > > > 10:16:14.851711 IP 10.0.0.30.2049 > 10.0.0.40.2717338886: reply o=
k 116 getattr DIR 40755 ids 0/0 sz 4096
> > > > > 10:16:14.856846 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 9157 wi=
n 2908 <nop,nop,timestamp 755272 755008>
> > > > > =
> > > > > =
> > > > > > > does somebody have such setup up and running and can tell his=
distro / kernel and nfs-utils version ?
> > > > > > > maybe i change distro then.
> > > > > > =
> > > > > > I doubt that it is a distro-specific thing. As long as you have
> > > > > > nfs-utils-1.1.0 it should work. I don't have a 10.3 box
> > > > > > set up yet, but it works fine on Debian/unstable for me.
> > > > > =
> > > > > ok, will try this on debian.
> > > > > =
> > > > > > Maybe try adding the "no_root_squash" export option.
> > > > > no difference
> > > > > =
> > > > > > What does "ls -l /export" on the server show?
> > > > > nothing unusual. no errors, just the dirs/mountpoints
> > > > > =
> > > > > Thanks for your help!
> > > > > =
> > > > > regards
> > > > > roland
> > > > > =
> > > > > =
> > > > > > =
> > > > > > On Saturday October 27, devzero@web.de wrote:
> > > > > > > Hello !
> > > > > > > =
> > > > > > > with 2.6.22 i`m trying to export loopback mounted iso-images.
> > > > > > > =
> > > > > > > this is /etc/exports:
> > > > > > > =
> > > > > > > /export *(ro,crossmnt,subtree_check)
> > > > > > =
> > > > > > I recommend replacing subtree_check with no_subtree_check, but =
it
> > > > > > shouldn't make an important difference in this case.
> > > > > > =
> > > > > > =
> > > > > > This should work with nfs-utils 1.1.0 or later. With earlier r=
eleases
> > > > > > you need to explicitly export the subordinate filesystems too.
> > > > > > =
> > > > > > > =
> > > > > > > in /export, i have loopback mounted iso-images
> > > > > > > =
> > > > > > > after mounting on the client side under /mnt (tried one older=
and one recent system) , i`m getting:
> > > > > > > =
> > > > > > > vmhost:/mnt # ls -la
> > > > > > > /bin/ls: iso1: Input/output error
> > > > > > > /bin/ls: iso2 Input/output error
> > > > > > > /bin/ls: iso3: Input/output error
> > > > > > > total 10128
> > > > > > > drwxrwxrwt 18 root root 270336 Oct 26 08:45 .
> > > > > > > drwxrwxrwt 186 root root 20760 Oct 27 17:45 ..
> > > > > > > drwxr-xr-x 2 root root 16384 Jan 1 1970 iso1
> > > > > > > drwxr-xr-x 2 root root 16384 Jan 1 1970 iso2
> > > > > > > drwxr-xr-x 2 root root 16384 Jan 1 1970 iso3
> > > > > > > =
> > > > > > > vmhost:/mnt/iso1 # ls
> > > > > > > /bin/ls: .: Stale NFS file handle
> > > > > > > vmhost:/mnt/iso1 # ls -la
> > > > > > > /bin/ls: .: Input/output error
> > > > > > =
> > > > > > It is a little odd that the errors are inconsistent.
> > > > > > =
> > > > > > Can you find any log messages from mountd in syslog? What do t=
hey
> > > > > > say?
> > > > > > Also what does
> > > > > > cat /proc/fs/nfsd/exports
> > > > > > =
> > > > > > on the server show.
> > > > > > =
> > > > > > Finally, a tcpdump:
> > > > > > =
> > > > > > tcpdump -s 0 -w /tmp/tcpdump port 2049
> > > > > > =
> > > > > > while you run the experiment might help.
> > > > > > =
> > > > > > > =
> > > > > > > i`m unsure if i should blame suse here (it`s an opensuse 10.3=
box which seems to have nfs-utils 1.1.0)
> > > > > > > =
> > > > > > > does somebody have such setup up and running and can tell his=
distro / kernel and nfs-utils version ?
> > > > > > > maybe i change distro then.
> > > > > > =
> > > > > > I doubt that it is a distro-specific thing. As long as you have
> > > > > > nfs-utils-1.1.0 it should work. I don't have a 10.3 box
> > > > > > set up yet, but it works fine on Debian/unstable for me.
> > > > > > =
> > > > > > Maybe try adding the "no_root_squash" export option.
> > > > > > What does "ls -l /export" on the server show?
> > > > > > =
> > > > > > NeilBrown
> > > > > > =
> > > > > =
> > > > > =
> > > > =
> > > > =
> > > > ___________________________________________________________________=
__
> > > > Der WEB.DE SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu spare=
n!
> > > > http://smartsurfer.web.de/?mc=3D100071&distributionid=3D000000000066
> > > > =
> > > > =
> > > > -------------------------------------------------------------------=
------
> > > > This SF.net email is sponsored by: Splunk Inc.
> > > > Still grepping through log files to find problems? Stop.
> > > > Now Search log events and configuration files using AJAX and a brow=
ser.
> > > > Download your FREE copy of Splunk now >> http://get.splunk.com/
> > > > _______________________________________________
> > > > NFS maillist - NFS@lists.sourceforge.net
> > > > https://lists.sourceforge.net/lists/listinfo/nfs
> > > =
> > =
> > =
> > _______________________________________________________________________=
___
> > Erweitern Sie FreeMail zu einem noch leistungsst=E4rkeren E-Mail-Postfa=
ch! =
> > Mehr Infos unter http://produkte.web.de/club/?mc=3D021131
> > =
> =
_______________________________________________________________________
Jetzt neu! Sch=FCtzen Sie Ihren PC mit McAfee und WEB.DE. 3 Monate
kostenlos testen. http://www.pc-sicherheit.web.de/startseite/?mc=3D022220
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: stale nfs file handle with exported loopback mounts
2007-10-31 22:50 stale nfs file handle with exported loopback mounts devzero
@ 2007-11-01 4:26 ` Neil Brown
0 siblings, 0 replies; 17+ messages in thread
From: Neil Brown @ 2007-11-01 4:26 UTC (permalink / raw)
To: devzero; +Cc: J. Bruce Fields, NFS
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=unknown, Size: 1728 bytes --]
On Wednesday October 31, devzero@web.de wrote:
> ok, i just wanted to tell that this isn`t the right way to go imho.
>
> some time ago i have tested exporting a parent dir containing
> several loopback mounted iso images with some pre-1.1.0 nfs-utils
> version and it worked - so ì wonder why it now seems to have issues
> as things should have gone stable.....
We have a way of breaking things sometimes.... It's called
"progress". :-)
The short answer is that there is a bug in mountd which is fixed by
this patch:
diff --git a/utils/mountd/cache.c b/utils/mountd/cache.c
index ce1a5a9..fd317cd 100644
--- a/utils/mountd/cache.c
+++ b/utils/mountd/cache.c
@@ -508,7 +508,7 @@ void nfsd_fh(FILE *f)
*/
qword_printint(f, 0x7fffffff);
if (found)
- qword_print(f, found->e_path);
+ qword_print(f, found_path);
qword_eol(f);
out:
free(found_path);
The longer answer is that there is also a bug in "mount.nfs" which is
unrelated but was slowing me down in chasing this bug, and there is
also a bug in the NFS client which was causing my client oops and need
a reboot every time I triggered this bug in mountd, which further
slowed me down.
The effect of this bug in mountd is that when the NFS client calls
GETATTR on the root of the subordinate filesystem (e.g. your
loop-mounted isos), it got attr information about the parent. ie. the
top-level exported filesystem (/export in your case I think).
This has a different 'fsid' than the nfs client was expecting and
the NFS client got confused in various ways.
Thanks for your problem report - it helped find 3 bugs!
I'll get proper patches or bug report off to the relevant maintainers
shortly.
NeilBrown
[-- Attachment #2: Type: text/plain, Size: 314 bytes --]
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
[-- Attachment #3: Type: text/plain, Size: 140 bytes --]
_______________________________________________
NFS maillist - NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: stale nfs file handle with exported loopback mounts
@ 2007-11-10 15:14 devzero
0 siblings, 0 replies; 17+ messages in thread
From: devzero @ 2007-11-10 15:14 UTC (permalink / raw)
To: Neil Brown; +Cc: J. Bruce Fields, NFS
Hi Neil, =
as Andreas told that the kernel fix will be in one of the next openSuSE ker=
nel updates i`m wondering if that also applies to the mountd patch.
Will there be an update package for nfs-utils or is this just too minor iss=
ue for releasing an update package ?
regards
roland
> -----Urspr=FCngliche Nachricht-----
> Von: Neil Brown <neilb@suse.de>
> Gesendet: 01.11.07 05:26:50
> An: devzero@web.de
> CC: "J. Bruce Fields" <bfields@fieldses.org>, NFS@lists.sourceforge.net
> Betreff: Re: [NFS] stale nfs file handle with exported loopback mounts
> =
> On Wednesday October 31, devzero@web.de wrote:
> > ok, i just wanted to tell that this isn`t the right way to go imho.
> > =
> > some time ago i have tested exporting a parent dir containing
> > several loopback mounted iso images with some pre-1.1.0 nfs-utils
> > version and it worked - so =EC wonder why it now seems to have issues
> > as things should have gone stable..... =
> =
> We have a way of breaking things sometimes.... It's called
> "progress". :-)
> =
> The short answer is that there is a bug in mountd which is fixed by
> this patch:
> =
> diff --git a/utils/mountd/cache.c b/utils/mountd/cache.c
> index ce1a5a9..fd317cd 100644
> --- a/utils/mountd/cache.c
> +++ b/utils/mountd/cache.c
> @@ -508,7 +508,7 @@ void nfsd_fh(FILE *f)
> */
> qword_printint(f, 0x7fffffff);
> if (found)
> - qword_print(f, found->e_path);
> + qword_print(f, found_path);
> qword_eol(f);
> out:
> free(found_path);
> =
> =
> The longer answer is that there is also a bug in "mount.nfs" which is
> unrelated but was slowing me down in chasing this bug, and there is
> also a bug in the NFS client which was causing my client oops and need
> a reboot every time I triggered this bug in mountd, which further
> slowed me down.
> =
> The effect of this bug in mountd is that when the NFS client calls
> GETATTR on the root of the subordinate filesystem (e.g. your
> loop-mounted isos), it got attr information about the parent. ie. the
> top-level exported filesystem (/export in your case I think).
> This has a different 'fsid' than the nfs client was expecting and
> the NFS client got confused in various ways.
> =
> Thanks for your problem report - it helped find 3 bugs!
> =
> I'll get proper patches or bug report off to the relevant maintainers
> shortly.
> =
> NeilBrown
> =
_____________________________________________________________________
Der WEB.DE SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen!
http://smartsurfer.web.de/?mc=3D100071&distributionid=3D000000000066
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: stale nfs file handle with exported loopback mounts
@ 2007-11-02 19:37 devzero
2007-11-02 19:42 ` J. Bruce Fields
0 siblings, 1 reply; 17+ messages in thread
From: devzero @ 2007-11-02 19:37 UTC (permalink / raw)
To: J. Bruce Fields; +Cc: Neil Brown, NFS
so, you mean this is a problem of the client or some "fix it at the client =
end" ?
that would be bad, since it would mean i need to touch all our linux system=
s which want to access our next-generation :) cd-rom server.... =
anyway =
> > > after mounting for the first time on the client, i`m getting "Invalid=
argument" for each loopback-mounted dir, if i do an ls -la on /mnt.
> > > this only happens _once_ and seems to be a server problem, because i =
can reboot the client and remount , i never see that errors again.
wondering, why doesn`t it happen again, even if i reboot the client.....
regards
roland
> -----Urspr=FCngliche Nachricht-----
> Von: "J. Bruce Fields" <bfields@fieldses.org>
> Gesendet: 02.11.07 20:24:52
> An: devzero@web.de
> CC: Neil Brown <neilb@suse.de>, NFS@lists.sourceforge.net
> Betreff: Re: [NFS] stale nfs file handle with exported loopback mounts
> =
> On Fri, Nov 02, 2007 at 03:23:17PM -0400, J. Bruce Fields wrote:
> > On Fri, Nov 02, 2007 at 08:06:58PM +0100, devzero@web.de wrote:
> > > hi!
> > > =
> > > it seems i was having weird mail problems with sending mails trough m=
y webmailer - at least two followups with attachments seem to be lost on se=
nding and are not in my sent folder anymore....
> > > =
> > > anyway - here is a second try, but probably worse than what i have wr=
itten before :)
> > > =
> > > =
> > > first off, thanks for the patch Neil, things look _much_ better now a=
nd exporting loopback mounts now basiscally works again.
> > > nice to see that my posting helped finding bugs.
> > > =
> > > maybe i have two more bugs for you :)
> > > =
> > > i have loopback mounts on the server and exported the parent dir with=
crossmnt option.
> > > =
> > > after mounting for the first time on the client, i`m getting "Invalid=
argument" for each loopback-mounted dir, if i do an ls -la on /mnt.
> > > this only happens _once_ and seems to be a server problem, because i =
can reboot the client and remount , i never see that errors again.
> > =
> > >From a quick look at the trace (thanks)--there's some getacl calls that
> > return EINVAL, then a few milliseconds later a second reply returns with
> > the original data. That looks suspiciously like a reply being sent when
> > the request was also deferred pending the upcall to deal with the newly
> > encountered filesystem. Sure enough, in
> > fs/nfs/nfs3acl.c:nfsd3_proc_getacl(), there's
> > =
> > if ((nfserr =3D fh_verify(rqstp, &resp->fh, 0, MAY_NOP)))
> > RETURN_STATUS(nfserr_inval);
> > =
> > Change that nfserr_inval to an nfserr (here and in
> > fs/nfs/nfs2acl.c:nfsd_proc_getacl()), and maybe the problem will go
> > away.
> =
> (I've been seeing some odd spurious replayed replies in the nfsv4 case
> as well, by the way--I suspect it's a similar problem, but haven't had
> the chance to track it down yet....)
> =
> --b.
> =
_____________________________________________________________________
Der WEB.DE SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen!
http://smartsurfer.web.de/?mc=3D100071&distributionid=3D000000000066
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: stale nfs file handle with exported loopback mounts
2007-11-02 19:37 devzero
@ 2007-11-02 19:42 ` J. Bruce Fields
2007-11-04 20:30 ` J. Bruce Fields
0 siblings, 1 reply; 17+ messages in thread
From: J. Bruce Fields @ 2007-11-02 19:42 UTC (permalink / raw)
To: devzero; +Cc: Neil Brown, NFS, Andreas Gruenbacher
On Fri, Nov 02, 2007 at 08:37:12PM +0100, devzero@web.de wrote:
> so, you mean this is a problem of the client or some "fix it at the
> client end" ?
>
> that would be bad, since it would mean i need to touch all our linux
> systems which want to access our next-generation :) cd-rom server....
No, it's definitely a server-side problem. Oh, I see, I miswrote the
path; I meant fs/nfsd/nfs3acl.c, not fs/nfs/nfs3acl.c....
How about this?
--b.
>From 7fc982f90d404003329934ae3b786c5e4efb0c0a Mon Sep 17 00:00:00 2001
From: J. Bruce Fields <bfields@citi.umich.edu>
Date: Fri, 2 Nov 2007 15:36:08 -0400
Subject: [PATCH] knfsd: fix spurious EINVAL errors on first access of new filesystem
The v2/v3 acl code in nfsd is translating any return from fh_verify() to
nfserr_inval. This is particularly unfortunate in the case of an
nfserr_dropit return, which is an internal error meant to indicate to
callers that this request has been deferred and should just be dropped
pending the results of an upcall to mountd.
Thanks to Roland <devzero@web.de> for bug report and data collection.
Cc: Roland <devzero@web.de>
Cc: Andreas Gruenbacher <agruen@suse.de>
Signed-off-by: J. Bruce Fields <bfields@citi.umich.edu>
---
fs/nfsd/nfs2acl.c | 2 +-
fs/nfsd/nfs3acl.c | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/fs/nfsd/nfs2acl.c b/fs/nfsd/nfs2acl.c
index bb3e188..d5fca59 100644
--- a/fs/nfsd/nfs2acl.c
+++ b/fs/nfsd/nfs2acl.c
@@ -41,7 +41,7 @@ static __be32 nfsacld_proc_getacl(struct svc_rqst * rqstp,
fh = fh_copy(&resp->fh, &argp->fh);
if ((nfserr = fh_verify(rqstp, &resp->fh, 0, MAY_NOP)))
- RETURN_STATUS(nfserr_inval);
+ RETURN_STATUS(nfserr);
if (argp->mask & ~(NFS_ACL|NFS_ACLCNT|NFS_DFACL|NFS_DFACLCNT))
RETURN_STATUS(nfserr_inval);
diff --git a/fs/nfsd/nfs3acl.c b/fs/nfsd/nfs3acl.c
index 3e3f2de..7cb39e7 100644
--- a/fs/nfsd/nfs3acl.c
+++ b/fs/nfsd/nfs3acl.c
@@ -40,7 +40,7 @@ static __be32 nfsd3_proc_getacl(struct svc_rqst * rqstp,
RETURN_STATUS(nfserr_inval);
if (argp->mask & ~(NFS_ACL|NFS_ACLCNT|NFS_DFACL|NFS_DFACLCNT))
- RETURN_STATUS(nfserr_inval);
+ RETURN_STATUS(nfserr);
resp->mask = argp->mask;
if (resp->mask & (NFS_ACL|NFS_ACLCNT)) {
--
1.5.3.4.208.gc990
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs
^ permalink raw reply related [flat|nested] 17+ messages in thread* Re: stale nfs file handle with exported loopback mounts
2007-11-02 19:42 ` J. Bruce Fields
@ 2007-11-04 20:30 ` J. Bruce Fields
2007-11-05 9:59 ` Andreas Gruenbacher
0 siblings, 1 reply; 17+ messages in thread
From: J. Bruce Fields @ 2007-11-04 20:30 UTC (permalink / raw)
To: devzero; +Cc: Neil Brown, NFS, Andreas Gruenbacher
Oops--thanks to Roland for pointing out a mistake in the nfs3acl.c piece
of the previous version of this patch.
--b.
>From c5ac46d1b7022f726606f92033bebd9fdc6ae343 Mon Sep 17 00:00:00 2001
From: J. Bruce Fields <bfields@citi.umich.edu>
Date: Fri, 2 Nov 2007 15:36:08 -0400
Subject: [PATCH] knfsd: fix spurious EINVAL errors on first access of new filesystem
The v2/v3 acl code in nfsd is translating any return from fh_verify() to
nfserr_inval. This is particularly unfortunate in the case of an
nfserr_dropit return, which is an internal error meant to indicate to
callers that this request has been deferred and should just be dropped
pending the results of an upcall to mountd.
Thanks to Roland <devzero@web.de> for bug report and data collection.
Cc: Roland <devzero@web.de>
Cc: Andreas Gruenbacher <agruen@suse.de>
Signed-off-by: J. Bruce Fields <bfields@citi.umich.edu>
---
fs/nfsd/nfs2acl.c | 2 +-
fs/nfsd/nfs3acl.c | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/fs/nfsd/nfs2acl.c b/fs/nfsd/nfs2acl.c
index bb3e188..d5fca59 100644
--- a/fs/nfsd/nfs2acl.c
+++ b/fs/nfsd/nfs2acl.c
@@ -41,7 +41,7 @@ static __be32 nfsacld_proc_getacl(struct svc_rqst * rqstp,
fh = fh_copy(&resp->fh, &argp->fh);
if ((nfserr = fh_verify(rqstp, &resp->fh, 0, MAY_NOP)))
- RETURN_STATUS(nfserr_inval);
+ RETURN_STATUS(nfserr);
if (argp->mask & ~(NFS_ACL|NFS_ACLCNT|NFS_DFACL|NFS_DFACLCNT))
RETURN_STATUS(nfserr_inval);
diff --git a/fs/nfsd/nfs3acl.c b/fs/nfsd/nfs3acl.c
index 3e3f2de..b647f2f 100644
--- a/fs/nfsd/nfs3acl.c
+++ b/fs/nfsd/nfs3acl.c
@@ -37,7 +37,7 @@ static __be32 nfsd3_proc_getacl(struct svc_rqst * rqstp,
fh = fh_copy(&resp->fh, &argp->fh);
if ((nfserr = fh_verify(rqstp, &resp->fh, 0, MAY_NOP)))
- RETURN_STATUS(nfserr_inval);
+ RETURN_STATUS(nfserr);
if (argp->mask & ~(NFS_ACL|NFS_ACLCNT|NFS_DFACL|NFS_DFACLCNT))
RETURN_STATUS(nfserr_inval);
--
1.5.3.5.475.g477d-dirty
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: stale nfs file handle with exported loopback mounts
@ 2007-11-02 19:06 devzero
2007-11-02 19:23 ` J. Bruce Fields
0 siblings, 1 reply; 17+ messages in thread
From: devzero @ 2007-11-02 19:06 UTC (permalink / raw)
To: Neil Brown; +Cc: J. Bruce Fields, NFS
hi!
it seems i was having weird mail problems with sending mails trough my webm=
ailer - at least two followups with attachments seem to be lost on sending =
and are not in my sent folder anymore....
anyway - here is a second try, but probably worse than what i have written =
before :)
first off, thanks for the patch Neil, things look _much_ better now and exp=
orting loopback mounts now basiscally works again.
nice to see that my posting helped finding bugs.
maybe i have two more bugs for you :)
i have loopback mounts on the server and exported the parent dir with cross=
mnt option.
after mounting for the first time on the client, i`m getting "Invalid argum=
ent" for each loopback-mounted dir, if i do an ls -la on /mnt.
this only happens _once_ and seems to be a server problem, because i can re=
boot the client and remount , i never see that errors again.
besides that, all seems to work fine.
as neil suggested, i have made a tcpdump of this available at:
http://82.141.46.148/bugs/nfs/tcpdump.out.bz2
furthermore, there is a very strange performance issue i was able to track =
down to uuid/blkid support.
i recognized this issue when i exported a directory containing a very large=
number of loopback mounts via crossmnt export option.
ls -la on the clients mountpoint seemed to hung and i could see mountd bein=
g busy, eating 100% cpu for quite a while.
the time needed for ls to finish seems to grow exponentially with the numbe=
r of loopback-mounts inside the exported directory - i also tried with 1000=
loopback mounts and mountd being busy for several minutes with this.
i have made a strace of mountd available at:
http://82.141.46.148/bugs/nfs/mountd.strace.txt.bz2
you can see that mountd seems to be busy doing the same things over and ove=
r again, looks that it does stat64() for all devices in /etc/blkid.tab for =
each loopback mount, weird.
here is some "strace -c -p $PID_OF_MOUNTD" for comparison - without uuid/b=
lkid support compiled in it looks like this:
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
73.23 0.147722 2 66313 stat64
10.37 0.020923 20 1031 write
5.54 0.011179 23 494 select
3.82 0.007699 5 1546 read
3.04 0.006137 8 773 time
2.18 0.004393 6 769 lstat64
1.08 0.002182 4 519 munmap
0.40 0.000797 1 1035 close
0.29 0.000594 1 1034 open
0.04 0.000089 0 1036 fstat64
0.00 0.000000 0 2 alarm
0.00 0.000000 0 3 _llseek
0.00 0.000000 0 1 fdatasync
0.00 0.000000 0 2 poll
0.00 0.000000 0 2 rt_sigaction
0.00 0.000000 0 521 mmap2
0.00 0.000000 0 2 fcntl64
0.00 0.000000 0 1 socket
0.00 0.000000 0 1 connect
0.00 0.000000 0 1 accept
0.00 0.000000 0 2 send
------ ----------- ----------- --------- --------- ----------------
100.00 0.201715 75088 total
this is an strace -c when uuid/blkid support is being compiled in:
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
61.64 1.008158 2 550916 stat64
21.67 0.354441 9 37662 read
5.65 0.092476 15 6377 getdents64
4.06 0.066381 3 21395 8232 open
1.62 0.026485 2 13169 fstat64
1.36 0.022237 2 13164 close
1.22 0.020025 2 8414 lstat64
1.15 0.018805 4 4415 munmap
0.27 0.004382 17 258 rename
0.26 0.004329 17 258 unlink
0.26 0.004305 2 2101 write
0.23 0.003786 1 4380 fcntl64
0.18 0.002899 11 262 select
0.18 0.002883 11 258 access
0.11 0.001857 0 4417 mmap2
0.11 0.001765 0 4652 time
0.01 0.000237 1 258 link
0.00 0.000041 0 258 lseek
0.00 0.000000 0 2 alarm
0.00 0.000000 0 2 brk
0.00 0.000000 0 1 gettimeofday
0.00 0.000000 0 258 fchmod
0.00 0.000000 0 265 _llseek
0.00 0.000000 0 1 fdatasync
0.00 0.000000 0 2 poll
0.00 0.000000 0 1 prctl
0.00 0.000000 0 2 rt_sigaction
0.00 0.000000 0 1 getuid32
0.00 0.000000 0 1 getgid32
0.00 0.000000 0 1 geteuid32
0.00 0.000000 0 1 getegid32
0.00 0.000000 0 1 futex
0.00 0.000000 0 1 socket
0.00 0.000000 0 1 connect
0.00 0.000000 0 1 accept
0.00 0.000000 0 2 send
------ ----------- ----------- --------- --------- ----------------
100.00 1.635492 673158 8232 total
=
as you can see there is an unusual high number of stat64() calls
server is opensuse 10.3 , client is suse 9.3 professional
if i can help resolving this issue, tell me what to do :)
regards
roland
> -----Urspr=FCngliche Nachricht-----
> Von: Neil Brown <neilb@suse.de>
> Gesendet: 01.11.07 05:26:50
> An: devzero@web.de
> CC: "J. Bruce Fields" <bfields@fieldses.org>, NFS@lists.sourceforge.net
> Betreff: Re: [NFS] stale nfs file handle with exported loopback mounts
> =
> On Wednesday October 31, devzero@web.de wrote:
> > ok, i just wanted to tell that this isn`t the right way to go imho.
> > =
> > some time ago i have tested exporting a parent dir containing
> > several loopback mounted iso images with some pre-1.1.0 nfs-utils
> > version and it worked - so =EC wonder why it now seems to have issues
> > as things should have gone stable..... =
> =
> We have a way of breaking things sometimes.... It's called
> "progress". :-)
> =
> The short answer is that there is a bug in mountd which is fixed by
> this patch:
> =
> diff --git a/utils/mountd/cache.c b/utils/mountd/cache.c
> index ce1a5a9..fd317cd 100644
> --- a/utils/mountd/cache.c
> +++ b/utils/mountd/cache.c
> @@ -508,7 +508,7 @@ void nfsd_fh(FILE *f)
> */
> qword_printint(f, 0x7fffffff);
> if (found)
> - qword_print(f, found->e_path);
> + qword_print(f, found_path);
> qword_eol(f);
> out:
> free(found_path);
> =
> =
> The longer answer is that there is also a bug in "mount.nfs" which is
> unrelated but was slowing me down in chasing this bug, and there is
> also a bug in the NFS client which was causing my client oops and need
> a reboot every time I triggered this bug in mountd, which further
> slowed me down.
> =
> The effect of this bug in mountd is that when the NFS client calls
> GETATTR on the root of the subordinate filesystem (e.g. your
> loop-mounted isos), it got attr information about the parent. ie. the
> top-level exported filesystem (/export in your case I think).
> This has a different 'fsid' than the nfs client was expecting and
> the NFS client got confused in various ways.
> =
> Thanks for your problem report - it helped find 3 bugs!
> =
> I'll get proper patches or bug report off to the relevant maintainers
> shortly.
> =
> NeilBrown
> =
_____________________________________________________________________
Der WEB.DE SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen!
http://smartsurfer.web.de/?mc=3D100071&distributionid=3D000000000066
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: stale nfs file handle with exported loopback mounts
2007-11-02 19:06 devzero
@ 2007-11-02 19:23 ` J. Bruce Fields
2007-11-02 19:24 ` J. Bruce Fields
0 siblings, 1 reply; 17+ messages in thread
From: J. Bruce Fields @ 2007-11-02 19:23 UTC (permalink / raw)
To: devzero; +Cc: Neil Brown, NFS
On Fri, Nov 02, 2007 at 08:06:58PM +0100, devzero@web.de wrote:
> hi!
>
> it seems i was having weird mail problems with sending mails trough my webmailer - at least two followups with attachments seem to be lost on sending and are not in my sent folder anymore....
>
> anyway - here is a second try, but probably worse than what i have written before :)
>
>
> first off, thanks for the patch Neil, things look _much_ better now and exporting loopback mounts now basiscally works again.
> nice to see that my posting helped finding bugs.
>
> maybe i have two more bugs for you :)
>
> i have loopback mounts on the server and exported the parent dir with crossmnt option.
>
> after mounting for the first time on the client, i`m getting "Invalid argument" for each loopback-mounted dir, if i do an ls -la on /mnt.
> this only happens _once_ and seems to be a server problem, because i can reboot the client and remount , i never see that errors again.
>From a quick look at the trace (thanks)--there's some getacl calls that
return EINVAL, then a few milliseconds later a second reply returns with
the original data. That looks suspiciously like a reply being sent when
the request was also deferred pending the upcall to deal with the newly
encountered filesystem. Sure enough, in
fs/nfs/nfs3acl.c:nfsd3_proc_getacl(), there's
if ((nfserr = fh_verify(rqstp, &resp->fh, 0, MAY_NOP)))
RETURN_STATUS(nfserr_inval);
Change that nfserr_inval to an nfserr (here and in
fs/nfs/nfs2acl.c:nfsd_proc_getacl()), and maybe the problem will go
away.
--b.
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: stale nfs file handle with exported loopback mounts
2007-11-02 19:23 ` J. Bruce Fields
@ 2007-11-02 19:24 ` J. Bruce Fields
0 siblings, 0 replies; 17+ messages in thread
From: J. Bruce Fields @ 2007-11-02 19:24 UTC (permalink / raw)
To: devzero; +Cc: Neil Brown, NFS
On Fri, Nov 02, 2007 at 03:23:17PM -0400, J. Bruce Fields wrote:
> On Fri, Nov 02, 2007 at 08:06:58PM +0100, devzero@web.de wrote:
> > hi!
> >
> > it seems i was having weird mail problems with sending mails trough my webmailer - at least two followups with attachments seem to be lost on sending and are not in my sent folder anymore....
> >
> > anyway - here is a second try, but probably worse than what i have written before :)
> >
> >
> > first off, thanks for the patch Neil, things look _much_ better now and exporting loopback mounts now basiscally works again.
> > nice to see that my posting helped finding bugs.
> >
> > maybe i have two more bugs for you :)
> >
> > i have loopback mounts on the server and exported the parent dir with crossmnt option.
> >
> > after mounting for the first time on the client, i`m getting "Invalid argument" for each loopback-mounted dir, if i do an ls -la on /mnt.
> > this only happens _once_ and seems to be a server problem, because i can reboot the client and remount , i never see that errors again.
>
> >From a quick look at the trace (thanks)--there's some getacl calls that
> return EINVAL, then a few milliseconds later a second reply returns with
> the original data. That looks suspiciously like a reply being sent when
> the request was also deferred pending the upcall to deal with the newly
> encountered filesystem. Sure enough, in
> fs/nfs/nfs3acl.c:nfsd3_proc_getacl(), there's
>
> if ((nfserr = fh_verify(rqstp, &resp->fh, 0, MAY_NOP)))
> RETURN_STATUS(nfserr_inval);
>
> Change that nfserr_inval to an nfserr (here and in
> fs/nfs/nfs2acl.c:nfsd_proc_getacl()), and maybe the problem will go
> away.
(I've been seeing some odd spurious replayed replies in the nfsv4 case
as well, by the way--I suspect it's a similar problem, but haven't had
the chance to track it down yet....)
--b.
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: stale nfs file handle with exported loopback mounts
@ 2007-10-31 22:19 devzero
2007-10-31 22:39 ` J. Bruce Fields
0 siblings, 1 reply; 17+ messages in thread
From: devzero @ 2007-10-31 22:19 UTC (permalink / raw)
To: J. Bruce Fields; +Cc: Neil Brown, NFS
>If you add an explicit export for each one, =
you mean, i should export each of those ~500 loopback mounted iso images ?
come on, that`s not what admins or users want, isn`t it ?
> -----Urspr=FCngliche Nachricht-----
> Von: "J. Bruce Fields" <bfields@fieldses.org>
> Gesendet: 31.10.07 21:58:00
> An: devzero@web.de
> CC: Neil Brown <neilb@suse.de>, NFS@lists.sourceforge.net
> Betreff: Re: [NFS] stale nfs file handle with exported loopback mounts
> =
> On Wed, Oct 31, 2007 at 09:46:13PM +0100, devzero@web.de wrote:
> > hi !
> > =
> > i tried latest grml build (lenny/sid) as server today (suse 9.3 as clie=
nt).
> > =
> > error still existing, but a little bit different:
> > =
> > if i cd to the loopback-mount dir`s on the client, i see the contents o=
f the parent directory, i.e. i get a recursion.
> =
> That suggests something funky in the way we're identifying those
> filesystems in the filehandle. If you add an explicit export for each
> one, each with its own "fsid=3Dxyz" option (with xyz whatever positive
> integer you'd like, as long as it's difficult for export), does the
> problem go away??
> =
> --b.
> =
> > =
> > regards
> > roland
> > =
> > =
> > =
> > > -----Urspr=FCngliche Nachricht-----
> > > Von: devzero@web.de
> > > Gesendet: 30.10.07 21:05:56
> > > An: Neil Brown <neilb@suse.de>
> > > CC: NFS@lists.sourceforge.net
> > > Betreff: Re: [NFS] stale nfs file handle with exported loopback mounts
> > =
> > =
> > > =
> > > > I recommend replacing subtree_check with no_subtree_check, but it
> > > > shouldn't make an important difference in this case.
> > > =
> > > ok, i leave it as is.
> > > =
> > > > This should work with nfs-utils 1.1.0 or later. With earlier relea=
ses
> > > > you need to explicitly export the subordinate filesystems too.
> > > > =
> > > =
> > > mhh - opensuse doesn`t have nfs-utils package, but it has nfs-client-=
1.1.0-8 which looks like they repackaged nfs-utils 1.1.0
> > > =
> > > > It is a little odd that the errors are inconsistent.
> > > =
> > > ok, but only a minor issue, if an issue at all, isn`t it ?
> > > =
> > > > Can you find any log messages from mountd in syslog? What do they
> > > > say?
> > > =
> > > yes, on the client i`m getting :
> > > =
> > > Jun 3 21:36:01 linux kernel: nfs_update_inode: inode number mismatch
> > > Jun 3 21:36:01 linux kernel: expected (0:11/0x2), got (0:11/0x13881)
> > > Jun 3 21:36:01 linux kernel: nfs_update_inode: inode number mismatch
> > > Jun 3 21:36:01 linux kernel: expected (0:11/0x2), got (0:11/0x13881)
> > > Jun 3 21:36:17 linux kernel: nfs_update_inode: inode number mismatch
> > > Jun 3 21:36:17 linux kernel: expected (0:11/0x2), got (0:11/0x13881)
> > > Jun 3 21:36:17 linux kernel: nfs_update_inode: inode number mismatch
> > > Jun 3 21:36:17 linux kernel: expected (0:11/0x2), got (0:11/0x13881)
> > > Jun 3 21:36:20 linux kernel: nfs_update_inode: inode number mismatch
> > > Jun 3 21:36:20 linux kernel: expected (0:11/0x2), got (0:11/0x13881)
> > > Jun 3 21:36:20 linux kernel: nfs_update_inode: inode number mismatch
> > > =
> > > =
> > > no error message on the server:
> > > Oct 26 10:09:31 opensuse103 mountd[4293]: authenticated unmount reque=
st from 10.0.0.40:1014 for /mnt (/mnt)
> > > Oct 26 10:10:07 opensuse103 mountd[4293]: authenticated mount request=
from 10.0.0.40:612 for /mnt (/mnt)
> > > Oct 26 10:10:43 opensuse103 mountd[4293]: authenticated unmount reque=
st from 10.0.0.40:623 for /mnt (/mnt)
> > > Oct 26 10:10:52 opensuse103 mountd[4293]: authenticated mount request=
from 10.0.0.40:624 for /mnt (/mnt)
> > > =
> > > > Also what does
> > > > cat /proc/fs/nfsd/exports
> > > > =
> > > > on the server show.
> > > =
> > > opensuse103:~ # cat /proc/fs/nfsd/exports
> > > # Version 1.1
> > > # Path Client(Flags) # IPs
> > > /mnt/iso1 *(ro,no_root_squash,sync,wdelay,crossmnt,uuid=3D2c49f=
ef2:ba464293:9b2bf2b8:322ccbcb)
> > > /mnt/iso3 *(ro,no_root_squash,sync,wdelay,crossmnt,uuid=3D27ae9=
c67:0b794b36:8b5e9e17:37b569eb)
> > > /mnt *(ro,no_root_squash,sync,wdelay,crossmnt,uuid=3D08164ee4:2db1=
41eb:ac961701:49c74396)
> > > /mnt/iso2 *(ro,no_root_squash,sync,wdelay,crossmnt,uuid=3D2aad6=
ea5:a05d4441:b94c48e6:e5d9981e)
> > > =
> > > =
> > > > Finally, a tcpdump:
> > > > =
> > > > tcpdump -s 0 -w /tmp/tcpdump port 2049
> > > > =
> > > > while you run the experiment might help.
> > > =
> > > ah - this seems to give a hint, but i don`t have a clue why the serve=
r (10.0.0.30) is telling the client (10.0.0.40) a "RPC Version mismatch".
> > > I also tried --no-nfs-version 4 for rpc.mountd (setting in /etc/sysc=
onfig/nfs), but this didn`t make a difference.
> > > =
> > > here is the tcpdump output - i did =
> > > =
> > > - mount
> > > - ls / ls -la / cd to subdirs
> > > =
> > > 10:15:06.480542 IP 10.0.0.40.0 > 10.0.0.30.2049: 0 null
> > > 10:15:06.480572 IP 10.0.0.30.2049 > 10.0.0.40.0: reply ERR 0: RPC Ver=
sion mismatch (167772160-0)
> > > 10:15:06.480765 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 2031114913 =
win 1460 <nop,nop,timestamp 686128 737918>
> > > 10:15:06.480821 IP 10.0.0.40.2079804678 > 10.0.0.30.2049: 108 fsinfo =
fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743963CD81908
> > > 10:15:06.480833 IP 10.0.0.30.2049 > 10.0.0.40.1022: . ack 108 win 181=
<nop,nop,timestamp 737918 686128>
> > > 10:15:06.513365 IP 10.0.0.30.2049 > 10.0.0.40.2079804678: reply ok 84=
fsinfo rtmax 65536 rtpref 65536 wtmax 65536 wtpref 65536 dtpref 4096
> > > 10:15:06.514408 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 85 win 1460=
<nop,nop,timestamp 686160 737926>
> > > 10:15:06.514902 IP 10.0.0.40.2096581894 > 10.0.0.30.2049: 108 getattr=
fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743963CD81908
> > > 10:15:06.515256 IP 10.0.0.30.2049 > 10.0.0.40.2096581894: reply ok 11=
6 getattr DIR 40755 ids 0/0 sz 4096
> > > 10:15:06.553865 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 201 win 146=
0 <nop,nop,timestamp 686201 737926>
> > > 10:16:03.091784 IP 10.0.0.40.2113359110 > 10.0.0.30.2049: 108 getattr=
fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439647219C8C
> > > 10:16:03.093366 IP 10.0.0.30.2049 > 10.0.0.40.2113359110: reply ok 11=
6 getattr DIR 40755 ids 0/0 sz 4096
> > > 10:16:03.096046 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 317 win 146=
0 <nop,nop,timestamp 743610 752071>
> > > 10:16:03.097370 IP 10.0.0.40.2130136326 > 10.0.0.30.2049: 112 access =
fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000001F=
001f
> > > 10:16:03.098156 IP 10.0.0.30.2049 > 10.0.0.40.2130136326: reply ok 12=
4 access c 0003
> > > 10:16:03.156812 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 441 win 146=
0 <nop,nop,timestamp 743651 752072>
> > > 10:16:08.967804 IP 10.0.0.40.2146913542 > 10.0.0.30.2049: 108 getattr=
fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439600000000
> > > 10:16:08.968305 IP 10.0.0.30.2049 > 10.0.0.40.2146913542: reply ok 11=
6 getattr DIR 40755 ids 0/0 sz 4096
> > > 10:16:08.974285 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 557 win 146=
0 <nop,nop,timestamp 749477 753540>
> > > 10:16:08.975739 IP 10.0.0.40.2163690758 > 10.0.0.30.2049: 132 readdir=
plus fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C74396000=
00000 512 bytes @ 0
> > > 10:16:08.975985 IP 10.0.0.30.2049 > 10.0.0.40.2163690758: reply ok 14=
48 readdirplus
> > > 10:16:08.976510 IP 10.0.0.30.2049 > 10.0.0.40.1684108288: reply Unkno=
wn rpc response code=3D2021855861 340
> > > 10:16:08.982238 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 2345 win 21=
84 <nop,nop,timestamp 749479 753541>
> > > 10:16:08.984415 IP 10.0.0.40.2180467974 > 10.0.0.30.2049: 116 lookup =
fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439600000004=
"iso3"
> > > 10:16:08.984931 IP 10.0.0.30.2049 > 10.0.0.40.2180467974: reply ok 23=
2 lookup fh Unknown/0100060027AE9C670B794B368B5E9E1737B569EB000000010000000=
2000041ED
> > > 10:16:09.008819 IP 10.0.0.40.2197245190 > 10.0.0.30.2049: 104 getattr=
fh Unknown/0100060027AE9C670B794B368B5E9E1737B569EB0000000FDD22CC3C7002F0AC
> > > 10:16:09.010946 IP 10.0.0.30.2049 > 10.0.0.40.2197245190: reply ok 18=
8 getattr REG 2 ids 5/0 sz 0
> > > 10:16:09.032541 IP 10.0.0.40.2214022406 > 10.0.0.30.2049: 128 getattr=
fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E66D0700
> > > 10:16:09.033405 IP 10.0.0.30.2049 > 10.0.0.40.2214022406: reply ok 18=
8 getattr REG 1 ids 1/0 sz 0
> > > 10:16:09.033490 IP 10.0.0.40.2230799622 > 10.0.0.30.2049: 128 getattr=
fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E56D0700
> > > 10:16:09.036811 IP 10.0.0.30.2049 > 10.0.0.40.2230799622: reply ok 18=
8 getattr REG 1 ids 1/0 sz 0
> > > 10:16:09.037823 IP 10.0.0.40.2247576838 > 10.0.0.30.2049: 108 getattr=
fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439600000000
> > > 10:16:09.039817 IP 10.0.0.30.2049 > 10.0.0.40.2247576838: reply ok 11=
6 getattr DIR 40755 ids 0/0 sz 4096
> > > 10:16:09.040423 IP 10.0.0.40.2264354054 > 10.0.0.30.2049: 112 getattr=
fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000000F
> > > 10:16:09.040856 IP 10.0.0.30.2049 > 10.0.0.40.2264354054: reply ok 18=
8 getattr REG 2 ids 5/0 sz 0
> > > 10:16:09.041590 IP 10.0.0.40.2281131270 > 10.0.0.30.2049: 116 lookup =
fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439600000004=
"iso2"
> > > 10:16:09.041920 IP 10.0.0.30.2049 > 10.0.0.40.2281131270: reply ok 23=
2 lookup fh Unknown/010006002AAD6EA5A05D4441B94C48E6E5D9981E000000010000000=
2000041ED
> > > 10:16:09.049633 IP 10.0.0.40.2297908486 > 10.0.0.30.2049: 104 getattr=
fh Unknown/010006002AAD6EA5A05D4441B94C48E6E5D9981E0000000F5FDC84454326E193
> > > 10:16:09.049781 IP 10.0.0.30.2049 > 10.0.0.40.2297908486: reply ok 18=
8 getattr REG 2 ids 5/0 sz 0
> > > 10:16:09.063218 IP 10.0.0.40.2314685702 > 10.0.0.30.2049: 128 getattr=
fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E46D0700
> > > 10:16:09.072505 IP 10.0.0.30.2049 > 10.0.0.40.2314685702: reply ok 18=
8 getattr REG 1 ids 1/0 sz 0
> > > 10:16:09.091698 IP 10.0.0.40.2331462918 > 10.0.0.30.2049: 124 getattr=
fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E46D0700
> > > 10:16:09.092022 IP 10.0.0.30.2049 > 10.0.0.40.2331462918: reply ok 11=
6 getattr REG 100644 ids 0/0 sz 1048576
> > > 10:16:09.128971 IP 10.0.0.40.2348240134 > 10.0.0.30.2049: 128 getattr=
fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E36D0700
> > > 10:16:09.129304 IP 10.0.0.30.2049 > 10.0.0.40.2348240134: reply ok 18=
8 getattr REG 1 ids 1/0 sz 0
> > > 10:16:09.184155 IP 10.0.0.40.2365017350 > 10.0.0.30.2049: 124 getattr=
fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E36D0700
> > > 10:16:09.184582 IP 10.0.0.30.2049 > 10.0.0.40.2365017350: reply ok 11=
6 getattr REG 100644 ids 0/0 sz 1048576
> > > 10:16:09.189234 IP 10.0.0.40.2381794566 > 10.0.0.30.2049: 116 lookup =
fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439600000004=
"iso1"
> > > 10:16:09.189435 IP 10.0.0.30.2049 > 10.0.0.40.2381794566: reply ok 23=
2 lookup fh Unknown/010006002C49FEF2BA4642939B2BF2B8322CCBCB000000010000000=
2000041ED
> > > 10:16:09.193476 IP 10.0.0.40.2398571782 > 10.0.0.30.2049: 104 getattr=
fh Unknown/010006002C49FEF2BA4642939B2BF2B8322CCBCB0000000F58896A884A0C62B7
> > > 10:16:09.193652 IP 10.0.0.30.2049 > 10.0.0.40.2398571782: reply ok 18=
8 getattr REG 2 ids 5/0 sz 0
> > > 10:16:09.194937 IP 10.0.0.40.2415348998 > 10.0.0.30.2049: 128 getattr=
fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E76D0700
> > > 10:16:09.195033 IP 10.0.0.30.2049 > 10.0.0.40.2415348998: reply ok 18=
8 getattr REG 1 ids 1/0 sz 0
> > > 10:16:09.195230 IP 10.0.0.40.2432126214 > 10.0.0.30.2049: 128 getattr=
fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E86D0700
> > > 10:16:09.323635 IP 10.0.0.30.2049 > 10.0.0.40.1022: . ack 2572 win 41=
6 <nop,nop,timestamp 753629 749566>
> > > 10:16:09.324345 IP 10.0.0.30.2049 > 10.0.0.40.2432126214: reply ok 18=
8 getattr REG 1 ids 1/0 sz 0
> > > 10:16:09.341475 IP 10.0.0.40.2448903430 > 10.0.0.30.2049: 128 getattr=
fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E26D0700
> > > 10:16:09.341524 IP 10.0.0.30.2049 > 10.0.0.40.1022: . ack 2700 win 44=
9 <nop,nop,timestamp 753632 749707>
> > > 10:16:09.341928 IP 10.0.0.30.2049 > 10.0.0.40.2448903430: reply ok 18=
8 getattr REG 1 ids 1/0 sz 0
> > > 10:16:09.342117 IP 10.0.0.40.2465680646 > 10.0.0.30.2049: 124 getattr=
fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E26D0700
> > > 10:16:09.343404 IP 10.0.0.30.2049 > 10.0.0.40.2465680646: reply ok 11=
6 getattr REG 100644 ids 0/0 sz 1048576
> > > 10:16:09.389316 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 5573 win 21=
84 <nop,nop,timestamp 749751 753633>
> > > 10:16:13.449513 IP 10.0.0.40.2482457862 > 10.0.0.30.2049: 108 getattr=
fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439647219CB5
> > > 10:16:13.449815 IP 10.0.0.30.2049 > 10.0.0.40.2482457862: reply ok 11=
6 getattr DIR 40755 ids 0/0 sz 4096
> > > 10:16:13.452344 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 5689 win 21=
84 <nop,nop,timestamp 753943 754660>
> > > 10:16:13.453973 IP 10.0.0.40.2499235078 > 10.0.0.30.2049: 100 getattr=
fh Unknown/010006002C49FEF2BA4642939B2BF2B8322CCBCB47219C8C0000000047219C8C
> > > 10:16:13.454154 IP 10.0.0.30.2049 > 10.0.0.40.2499235078: reply ok 11=
6 getattr DIR 40755 ids 0/0 sz 4096
> > > 10:16:13.456194 IP 10.0.0.40.2516012294 > 10.0.0.30.2049: 108 getattr=
fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439647219C8C
> > > 10:16:13.456361 IP 10.0.0.30.2049 > 10.0.0.40.2516012294: reply ok 11=
6 getattr DIR 40755 ids 0/0 sz 4096
> > > 10:16:13.458282 IP 10.0.0.40.2532789510 > 10.0.0.30.2049: 116 lookup =
fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439600000004=
"iso1"
> > > 10:16:13.458461 IP 10.0.0.30.2049 > 10.0.0.40.2532789510: reply ok 23=
2 lookup fh Unknown/010006002C49FEF2BA4642939B2BF2B8322CCBCB000000010000000=
2000041ED
> > > 10:16:13.510637 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 6153 win 21=
84 <nop,nop,timestamp 753985 754662>
> > > 10:16:14.030110 IP 10.0.0.40.2549566726 > 10.0.0.30.2049: 112 access =
fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000001F=
001f
> > > 10:16:14.030927 IP 10.0.0.30.2049 > 10.0.0.40.2549566726: reply ok 12=
4 access c 0003
> > > 10:16:14.033436 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 6277 win 21=
84 <nop,nop,timestamp 754548 754805>
> > > 10:16:14.034732 IP 10.0.0.40.2566343942 > 10.0.0.30.2049: 108 getattr=
fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439600000000
> > > 10:16:14.034980 IP 10.0.0.30.2049 > 10.0.0.40.2566343942: reply ok 11=
6 getattr DIR 40755 ids 0/0 sz 4096
> > > 10:16:14.037319 IP 10.0.0.40.2583121158 > 10.0.0.30.2049: 112 access =
fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000001F=
001f
> > > 10:16:14.037486 IP 10.0.0.30.2049 > 10.0.0.40.2583121158: reply ok 12=
4 access c 0003
> > > 10:16:14.040323 IP 10.0.0.40.2599898374 > 10.0.0.30.2049: 132 readdir=
plus fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C74396000=
00000 512 bytes @ 0
> > > 10:16:14.040554 IP 10.0.0.30.2049 > 10.0.0.40.2599898374: reply ok 14=
48 readdirplus
> > > 10:16:14.041020 IP 10.0.0.30.2049 > 10.0.0.40.1684108288: reply Unkno=
wn rpc response code=3D2021855861 340
> > > 10:16:14.043583 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 8305 win 29=
08 <nop,nop,timestamp 754550 754808>
> > > 10:16:14.045104 IP 10.0.0.40.2616675590 > 10.0.0.30.2049: 112 access =
fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000001F=
001f
> > > 10:16:14.045402 IP 10.0.0.30.2049 > 10.0.0.40.2616675590: reply ok 12=
4 access c 0003
> > > 10:16:14.047830 IP 10.0.0.40.2633452806 > 10.0.0.30.2049: 112 access =
fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000001F=
001f
> > > 10:16:14.048039 IP 10.0.0.30.2049 > 10.0.0.40.2633452806: reply ok 12=
4 access c 0003
> > > 10:16:14.099385 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 8553 win 29=
08 <nop,nop,timestamp 754592 754810>
> > > 10:16:14.293714 IP 10.0.0.40.2650230022 > 10.0.0.30.2049: 108 getattr=
fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439600000000
> > > 10:16:14.294019 IP 10.0.0.30.2049 > 10.0.0.40.2650230022: reply ok 11=
6 getattr DIR 40755 ids 0/0 sz 4096
> > > 10:16:14.297263 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 8669 win 29=
08 <nop,nop,timestamp 754763 754871>
> > > 10:16:14.297272 IP 10.0.0.40.2667007238 > 10.0.0.30.2049: 112 access =
fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000001F=
001f
> > > 10:16:14.297466 IP 10.0.0.30.2049 > 10.0.0.40.2667007238: reply ok 12=
4 access c 0003
> > > 10:16:14.297586 IP 10.0.0.40.2683784454 > 10.0.0.30.2049: 112 access =
fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000001F=
001f
> > > 10:16:14.297987 IP 10.0.0.30.2049 > 10.0.0.40.2683784454: reply ok 12=
4 access c 0003
> > > 10:16:14.301165 IP 10.0.0.40.2700561670 > 10.0.0.30.2049: 104 access =
fh Unknown/010006002C49FEF2BA4642939B2BF2B8322CCBCB0000001F47219C8C00000000=
001f
> > > 10:16:14.301405 IP 10.0.0.30.2049 > 10.0.0.40.2700561670: reply ok 12=
4 access c 0003
> > > 10:16:14.351229 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 9041 win 29=
08 <nop,nop,timestamp 754810 754873>
> > > 10:16:14.842633 IP 10.0.0.40.2717338886 > 10.0.0.30.2049: 100 getattr=
fh Unknown/010006002C49FEF2BA4642939B2BF2B8322CCBCB0000000047219C8C00000000
> > > 10:16:14.851711 IP 10.0.0.30.2049 > 10.0.0.40.2717338886: reply ok 11=
6 getattr DIR 40755 ids 0/0 sz 4096
> > > 10:16:14.856846 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 9157 win 29=
08 <nop,nop,timestamp 755272 755008>
> > > =
> > > =
> > > > > does somebody have such setup up and running and can tell his dis=
tro / kernel and nfs-utils version ?
> > > > > maybe i change distro then.
> > > > =
> > > > I doubt that it is a distro-specific thing. As long as you have
> > > > nfs-utils-1.1.0 it should work. I don't have a 10.3 box
> > > > set up yet, but it works fine on Debian/unstable for me.
> > > =
> > > ok, will try this on debian.
> > > =
> > > > Maybe try adding the "no_root_squash" export option.
> > > no difference
> > > =
> > > > What does "ls -l /export" on the server show?
> > > nothing unusual. no errors, just the dirs/mountpoints
> > > =
> > > Thanks for your help!
> > > =
> > > regards
> > > roland
> > > =
> > > =
> > > > =
> > > > On Saturday October 27, devzero@web.de wrote:
> > > > > Hello !
> > > > > =
> > > > > with 2.6.22 i`m trying to export loopback mounted iso-images.
> > > > > =
> > > > > this is /etc/exports:
> > > > > =
> > > > > /export *(ro,crossmnt,subtree_check)
> > > > =
> > > > I recommend replacing subtree_check with no_subtree_check, but it
> > > > shouldn't make an important difference in this case.
> > > > =
> > > > =
> > > > This should work with nfs-utils 1.1.0 or later. With earlier relea=
ses
> > > > you need to explicitly export the subordinate filesystems too.
> > > > =
> > > > > =
> > > > > in /export, i have loopback mounted iso-images
> > > > > =
> > > > > after mounting on the client side under /mnt (tried one older and=
one recent system) , i`m getting:
> > > > > =
> > > > > vmhost:/mnt # ls -la
> > > > > /bin/ls: iso1: Input/output error
> > > > > /bin/ls: iso2 Input/output error
> > > > > /bin/ls: iso3: Input/output error
> > > > > total 10128
> > > > > drwxrwxrwt 18 root root 270336 Oct 26 08:45 .
> > > > > drwxrwxrwt 186 root root 20760 Oct 27 17:45 ..
> > > > > drwxr-xr-x 2 root root 16384 Jan 1 1970 iso1
> > > > > drwxr-xr-x 2 root root 16384 Jan 1 1970 iso2
> > > > > drwxr-xr-x 2 root root 16384 Jan 1 1970 iso3
> > > > > =
> > > > > vmhost:/mnt/iso1 # ls
> > > > > /bin/ls: .: Stale NFS file handle
> > > > > vmhost:/mnt/iso1 # ls -la
> > > > > /bin/ls: .: Input/output error
> > > > =
> > > > It is a little odd that the errors are inconsistent.
> > > > =
> > > > Can you find any log messages from mountd in syslog? What do they
> > > > say?
> > > > Also what does
> > > > cat /proc/fs/nfsd/exports
> > > > =
> > > > on the server show.
> > > > =
> > > > Finally, a tcpdump:
> > > > =
> > > > tcpdump -s 0 -w /tmp/tcpdump port 2049
> > > > =
> > > > while you run the experiment might help.
> > > > =
> > > > > =
> > > > > i`m unsure if i should blame suse here (it`s an opensuse 10.3 box=
which seems to have nfs-utils 1.1.0)
> > > > > =
> > > > > does somebody have such setup up and running and can tell his dis=
tro / kernel and nfs-utils version ?
> > > > > maybe i change distro then.
> > > > =
> > > > I doubt that it is a distro-specific thing. As long as you have
> > > > nfs-utils-1.1.0 it should work. I don't have a 10.3 box
> > > > set up yet, but it works fine on Debian/unstable for me.
> > > > =
> > > > Maybe try adding the "no_root_squash" export option.
> > > > What does "ls -l /export" on the server show?
> > > > =
> > > > NeilBrown
> > > > =
> > > =
> > > =
> > =
> > =
> > _____________________________________________________________________
> > Der WEB.DE SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen!
> > http://smartsurfer.web.de/?mc=3D100071&distributionid=3D000000000066
> > =
> > =
> > -----------------------------------------------------------------------=
--
> > This SF.net email is sponsored by: Splunk Inc.
> > Still grepping through log files to find problems? Stop.
> > Now Search log events and configuration files using AJAX and a browser.
> > Download your FREE copy of Splunk now >> http://get.splunk.com/
> > _______________________________________________
> > NFS maillist - NFS@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/nfs
> =
__________________________________________________________________________
Erweitern Sie FreeMail zu einem noch leistungsst=E4rkeren E-Mail-Postfach! =
Mehr Infos unter http://produkte.web.de/club/?mc=3D021131
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: stale nfs file handle with exported loopback mounts
2007-10-31 22:19 devzero
@ 2007-10-31 22:39 ` J. Bruce Fields
0 siblings, 0 replies; 17+ messages in thread
From: J. Bruce Fields @ 2007-10-31 22:39 UTC (permalink / raw)
To: devzero; +Cc: Neil Brown, NFS
T24gV2VkLCBPY3QgMzEsIDIwMDcgYXQgMTE6MTk6NDNQTSArMDEwMCwgZGV2emVyb0B3ZWIuZGUg
d3JvdGU6Cj4gPklmIHlvdSBhZGQgYW4gZXhwbGljaXQgZXhwb3J0IGZvciBlYWNoIG9uZSwgCj4g
eW91IG1lYW4sIGkgc2hvdWxkIGV4cG9ydCBlYWNoIG9mIHRob3NlIH41MDAgbG9vcGJhY2sgbW91
bnRlZCBpc28gaW1hZ2VzID8KPiBjb21lIG9uLCB0aGF0YHMgbm90IHdoYXQgYWRtaW5zIG9yIHVz
ZXJzIHdhbnQsIGlzbmB0IGl0ID8KCk5vLCBvZiBjb3Vyc2Ugbm90OyBJIHdhcyBqdXN0IHN1Z2dl
c3RpbmcgaXQgYXMgYSB3YXkgdG8gY29uZmlybSB0aGF0J3MKd2hlcmUgdGhlIHByb2JsZW0gaXMu
CgotLWIuCgo+IAo+ID4gLS0tLS1VcnNwcsO8bmdsaWNoZSBOYWNocmljaHQtLS0tLQo+ID4gVm9u
OiAiSi4gQnJ1Y2UgRmllbGRzIiA8YmZpZWxkc0BmaWVsZHNlcy5vcmc+Cj4gPiBHZXNlbmRldDog
MzEuMTAuMDcgMjE6NTg6MDAKPiA+IEFuOiBkZXZ6ZXJvQHdlYi5kZQo+ID4gQ0M6IE5laWwgQnJv
d24gPG5laWxiQHN1c2UuZGU+LCBORlNAbGlzdHMuc291cmNlZm9yZ2UubmV0Cj4gPiBCZXRyZWZm
OiBSZTogW05GU10gc3RhbGUgbmZzIGZpbGUgaGFuZGxlIHdpdGggZXhwb3J0ZWQgbG9vcGJhY2sg
bW91bnRzCj4gCj4gCj4gPiAKPiA+IE9uIFdlZCwgT2N0IDMxLCAyMDA3IGF0IDA5OjQ2OjEzUE0g
KzAxMDAsIGRldnplcm9Ad2ViLmRlIHdyb3RlOgo+ID4gPiBoaSAhCj4gPiA+IAo+ID4gPiBpIHRy
aWVkIGxhdGVzdCBncm1sIGJ1aWxkIChsZW5ueS9zaWQpIGFzIHNlcnZlciB0b2RheSAoc3VzZSA5
LjMgYXMgY2xpZW50KS4KPiA+ID4gCj4gPiA+IGVycm9yIHN0aWxsIGV4aXN0aW5nLCBidXQgYSBs
aXR0bGUgYml0IGRpZmZlcmVudDoKPiA+ID4gCj4gPiA+IGlmIGkgY2QgdG8gdGhlIGxvb3BiYWNr
LW1vdW50IGRpcmBzIG9uIHRoZSBjbGllbnQsIGkgc2VlIHRoZSBjb250ZW50cyBvZiB0aGUgcGFy
ZW50IGRpcmVjdG9yeSwgaS5lLiBpIGdldCBhIHJlY3Vyc2lvbi4KPiA+IAo+ID4gVGhhdCBzdWdn
ZXN0cyBzb21ldGhpbmcgZnVua3kgaW4gdGhlIHdheSB3ZSdyZSBpZGVudGlmeWluZyB0aG9zZQo+
ID4gZmlsZXN5c3RlbXMgaW4gdGhlIGZpbGVoYW5kbGUuICBJZiB5b3UgYWRkIGFuIGV4cGxpY2l0
IGV4cG9ydCBmb3IgZWFjaAo+ID4gb25lLCBlYWNoIHdpdGggaXRzIG93biAiZnNpZD14eXoiIG9w
dGlvbiAod2l0aCB4eXogd2hhdGV2ZXIgcG9zaXRpdmUKPiA+IGludGVnZXIgeW91J2QgbGlrZSwg
YXMgbG9uZyBhcyBpdCdzIGRpZmZpY3VsdCBmb3IgZXhwb3J0KSwgZG9lcyB0aGUKPiA+IHByb2Js
ZW0gZ28gYXdheT8/Cj4gPiAKPiA+IC0tYi4KPiA+IAo+ID4gPiAKPiA+ID4gcmVnYXJkcwo+ID4g
PiByb2xhbmQKPiA+ID4gCj4gPiA+IAo+ID4gPiAKPiA+ID4gPiAtLS0tLVVyc3Byw7xuZ2xpY2hl
IE5hY2hyaWNodC0tLS0tCj4gPiA+ID4gVm9uOiBkZXZ6ZXJvQHdlYi5kZQo+ID4gPiA+IEdlc2Vu
ZGV0OiAzMC4xMC4wNyAyMTowNTo1Ngo+ID4gPiA+IEFuOiBOZWlsIEJyb3duIDxuZWlsYkBzdXNl
LmRlPgo+ID4gPiA+IENDOiBORlNAbGlzdHMuc291cmNlZm9yZ2UubmV0Cj4gPiA+ID4gQmV0cmVm
ZjogUmU6IFtORlNdIHN0YWxlIG5mcyBmaWxlIGhhbmRsZSB3aXRoIGV4cG9ydGVkIGxvb3BiYWNr
IG1vdW50cwo+ID4gPiAKPiA+ID4gCj4gPiA+ID4gCj4gPiA+ID4gPiBJIHJlY29tbWVuZCByZXBs
YWNpbmcgc3VidHJlZV9jaGVjayB3aXRoIG5vX3N1YnRyZWVfY2hlY2ssIGJ1dCBpdAo+ID4gPiA+
ID4gc2hvdWxkbid0IG1ha2UgYW4gaW1wb3J0YW50IGRpZmZlcmVuY2UgaW4gdGhpcyBjYXNlLgo+
ID4gPiA+IAo+ID4gPiA+IG9rLCBpIGxlYXZlIGl0IGFzIGlzLgo+ID4gPiA+IAo+ID4gPiA+ID4g
VGhpcyBzaG91bGQgd29yayB3aXRoIG5mcy11dGlscyAxLjEuMCBvciBsYXRlci4gIFdpdGggZWFy
bGllciByZWxlYXNlcwo+ID4gPiA+ID4geW91IG5lZWQgdG8gZXhwbGljaXRseSBleHBvcnQgdGhl
IHN1Ym9yZGluYXRlIGZpbGVzeXN0ZW1zIHRvby4KPiA+ID4gPiA+IAo+ID4gPiA+IAo+ID4gPiA+
IG1oaCAtIG9wZW5zdXNlIGRvZXNuYHQgaGF2ZSBuZnMtdXRpbHMgcGFja2FnZSwgYnV0IGl0IGhh
cyBuZnMtY2xpZW50LTEuMS4wLTggd2hpY2ggbG9va3MgbGlrZSB0aGV5IHJlcGFja2FnZWQgbmZz
LXV0aWxzIDEuMS4wCj4gPiA+ID4gCj4gPiA+ID4gPiBJdCBpcyBhIGxpdHRsZSBvZGQgdGhhdCB0
aGUgZXJyb3JzIGFyZSBpbmNvbnNpc3RlbnQuCj4gPiA+ID4gCj4gPiA+ID4gb2ssIGJ1dCBvbmx5
IGEgbWlub3IgaXNzdWUsIGlmIGFuIGlzc3VlIGF0IGFsbCwgaXNuYHQgaXQgPwo+ID4gPiA+IAo+
ID4gPiA+ID4gQ2FuIHlvdSBmaW5kIGFueSBsb2cgbWVzc2FnZXMgZnJvbSBtb3VudGQgaW4gc3lz
bG9nPyAgV2hhdCBkbyB0aGV5Cj4gPiA+ID4gPiBzYXk/Cj4gPiA+ID4gCj4gPiA+ID4geWVzLCBv
biB0aGUgY2xpZW50IGlgbSBnZXR0aW5nIDoKPiA+ID4gPiAKPiA+ID4gPiBKdW4gIDMgMjE6MzY6
MDEgbGludXgga2VybmVsOiBuZnNfdXBkYXRlX2lub2RlOiBpbm9kZSBudW1iZXIgbWlzbWF0Y2gK
PiA+ID4gPiBKdW4gIDMgMjE6MzY6MDEgbGludXgga2VybmVsOiBleHBlY3RlZCAoMDoxMS8weDIp
LCBnb3QgKDA6MTEvMHgxMzg4MSkKPiA+ID4gPiBKdW4gIDMgMjE6MzY6MDEgbGludXgga2VybmVs
OiBuZnNfdXBkYXRlX2lub2RlOiBpbm9kZSBudW1iZXIgbWlzbWF0Y2gKPiA+ID4gPiBKdW4gIDMg
MjE6MzY6MDEgbGludXgga2VybmVsOiBleHBlY3RlZCAoMDoxMS8weDIpLCBnb3QgKDA6MTEvMHgx
Mzg4MSkKPiA+ID4gPiBKdW4gIDMgMjE6MzY6MTcgbGludXgga2VybmVsOiBuZnNfdXBkYXRlX2lu
b2RlOiBpbm9kZSBudW1iZXIgbWlzbWF0Y2gKPiA+ID4gPiBKdW4gIDMgMjE6MzY6MTcgbGludXgg
a2VybmVsOiBleHBlY3RlZCAoMDoxMS8weDIpLCBnb3QgKDA6MTEvMHgxMzg4MSkKPiA+ID4gPiBK
dW4gIDMgMjE6MzY6MTcgbGludXgga2VybmVsOiBuZnNfdXBkYXRlX2lub2RlOiBpbm9kZSBudW1i
ZXIgbWlzbWF0Y2gKPiA+ID4gPiBKdW4gIDMgMjE6MzY6MTcgbGludXgga2VybmVsOiBleHBlY3Rl
ZCAoMDoxMS8weDIpLCBnb3QgKDA6MTEvMHgxMzg4MSkKPiA+ID4gPiBKdW4gIDMgMjE6MzY6MjAg
bGludXgga2VybmVsOiBuZnNfdXBkYXRlX2lub2RlOiBpbm9kZSBudW1iZXIgbWlzbWF0Y2gKPiA+
ID4gPiBKdW4gIDMgMjE6MzY6MjAgbGludXgga2VybmVsOiBleHBlY3RlZCAoMDoxMS8weDIpLCBn
b3QgKDA6MTEvMHgxMzg4MSkKPiA+ID4gPiBKdW4gIDMgMjE6MzY6MjAgbGludXgga2VybmVsOiBu
ZnNfdXBkYXRlX2lub2RlOiBpbm9kZSBudW1iZXIgbWlzbWF0Y2gKPiA+ID4gPiAKPiA+ID4gPiAK
PiA+ID4gPiBubyBlcnJvciBtZXNzYWdlIG9uIHRoZSBzZXJ2ZXI6Cj4gPiA+ID4gT2N0IDI2IDEw
OjA5OjMxIG9wZW5zdXNlMTAzIG1vdW50ZFs0MjkzXTogYXV0aGVudGljYXRlZCB1bm1vdW50IHJl
cXVlc3QgZnJvbSAxMC4wLjAuNDA6MTAxNCBmb3IgL21udCAoL21udCkKPiA+ID4gPiBPY3QgMjYg
MTA6MTA6MDcgb3BlbnN1c2UxMDMgbW91bnRkWzQyOTNdOiBhdXRoZW50aWNhdGVkIG1vdW50IHJl
cXVlc3QgZnJvbSAxMC4wLjAuNDA6NjEyIGZvciAvbW50ICgvbW50KQo+ID4gPiA+IE9jdCAyNiAx
MDoxMDo0MyBvcGVuc3VzZTEwMyBtb3VudGRbNDI5M106IGF1dGhlbnRpY2F0ZWQgdW5tb3VudCBy
ZXF1ZXN0IGZyb20gMTAuMC4wLjQwOjYyMyBmb3IgL21udCAoL21udCkKPiA+ID4gPiBPY3QgMjYg
MTA6MTA6NTIgb3BlbnN1c2UxMDMgbW91bnRkWzQyOTNdOiBhdXRoZW50aWNhdGVkIG1vdW50IHJl
cXVlc3QgZnJvbSAxMC4wLjAuNDA6NjI0IGZvciAvbW50ICgvbW50KQo+ID4gPiA+IAo+ID4gPiA+
ID4gQWxzbyB3aGF0IGRvZXMKPiA+ID4gPiA+ICAgIGNhdCAvcHJvYy9mcy9uZnNkL2V4cG9ydHMK
PiA+ID4gPiA+IAo+ID4gPiA+ID4gb24gdGhlIHNlcnZlciBzaG93Lgo+ID4gPiA+IAo+ID4gPiA+
IG9wZW5zdXNlMTAzOn4gIyBjYXQgL3Byb2MvZnMvbmZzZC9leHBvcnRzCj4gPiA+ID4gIyBWZXJz
aW9uIDEuMQo+ID4gPiA+ICMgUGF0aCBDbGllbnQoRmxhZ3MpICMgSVBzCj4gPiA+ID4gL21udC9p
c28xICAgICAgICoocm8sbm9fcm9vdF9zcXVhc2gsc3luYyx3ZGVsYXksY3Jvc3NtbnQsdXVpZD0y
YzQ5ZmVmMjpiYTQ2NDI5Mzo5YjJiZjJiODozMjJjY2JjYikKPiA+ID4gPiAvbW50L2lzbzMgICAg
ICAgKihybyxub19yb290X3NxdWFzaCxzeW5jLHdkZWxheSxjcm9zc21udCx1dWlkPTI3YWU5YzY3
OjBiNzk0YjM2OjhiNWU5ZTE3OjM3YjU2OWViKQo+ID4gPiA+IC9tbnQgICAgKihybyxub19yb290
X3NxdWFzaCxzeW5jLHdkZWxheSxjcm9zc21udCx1dWlkPTA4MTY0ZWU0OjJkYjE0MWViOmFjOTYx
NzAxOjQ5Yzc0Mzk2KQo+ID4gPiA+IC9tbnQvaXNvMiAgICAgICAqKHJvLG5vX3Jvb3Rfc3F1YXNo
LHN5bmMsd2RlbGF5LGNyb3NzbW50LHV1aWQ9MmFhZDZlYTU6YTA1ZDQ0NDE6Yjk0YzQ4ZTY6ZTVk
OTk4MWUpCj4gPiA+ID4gCj4gPiA+ID4gCj4gPiA+ID4gPiBGaW5hbGx5LCBhIHRjcGR1bXA6Cj4g
PiA+ID4gPiAKPiA+ID4gPiA+ICAgdGNwZHVtcCAtcyAwIC13IC90bXAvdGNwZHVtcCBwb3J0IDIw
NDkKPiA+ID4gPiA+IAo+ID4gPiA+ID4gd2hpbGUgeW91IHJ1biB0aGUgZXhwZXJpbWVudCBtaWdo
dCBoZWxwLgo+ID4gPiA+IAo+ID4gPiA+IGFoIC0gdGhpcyBzZWVtcyB0byBnaXZlIGEgaGludCwg
YnV0IGkgZG9uYHQgaGF2ZSBhIGNsdWUgd2h5IHRoZSBzZXJ2ZXIgKDEwLjAuMC4zMCkgaXMgdGVs
bGluZyB0aGUgY2xpZW50ICgxMC4wLjAuNDApIGEgIlJQQyBWZXJzaW9uIG1pc21hdGNoIi4KPiA+
ID4gPiBJIGFsc28gdHJpZWQgIC0tbm8tbmZzLXZlcnNpb24gNCBmb3IgcnBjLm1vdW50ZCAoc2V0
dGluZyBpbiAvZXRjL3N5c2NvbmZpZy9uZnMpLCBidXQgdGhpcyBkaWRuYHQgbWFrZSBhIGRpZmZl
cmVuY2UuCj4gPiA+ID4gCj4gPiA+ID4gaGVyZSBpcyB0aGUgdGNwZHVtcCBvdXRwdXQgLSBpIGRp
ZCAKPiA+ID4gPiAKPiA+ID4gPiAtIG1vdW50Cj4gPiA+ID4gLSBscyAvIGxzIC1sYSAvIGNkIHRv
IHN1YmRpcnMKPiA+ID4gPiAKPiA+ID4gPiAxMDoxNTowNi40ODA1NDIgSVAgMTAuMC4wLjQwLjAg
PiAxMC4wLjAuMzAuMjA0OTogMCBudWxsCj4gPiA+ID4gMTA6MTU6MDYuNDgwNTcyIElQIDEwLjAu
MC4zMC4yMDQ5ID4gMTAuMC4wLjQwLjA6IHJlcGx5IEVSUiAwOiBSUEMgVmVyc2lvbiBtaXNtYXRj
aCAoMTY3NzcyMTYwLTApCj4gPiA+ID4gMTA6MTU6MDYuNDgwNzY1IElQIDEwLjAuMC40MC4xMDIy
ID4gMTAuMC4wLjMwLjIwNDk6IC4gYWNrIDIwMzExMTQ5MTMgd2luIDE0NjAgPG5vcCxub3AsdGlt
ZXN0YW1wIDY4NjEyOCA3Mzc5MTg+Cj4gPiA+ID4gMTA6MTU6MDYuNDgwODIxIElQIDEwLjAuMC40
MC4yMDc5ODA0Njc4ID4gMTAuMC4wLjMwLjIwNDk6IDEwOCBmc2luZm8gZmggVW5rbm93bi8wMTAw
MDcwMDgxMzgwMTAwMDAwMDAwMDAwODE2NEVFNDJEQjE0MUVCQUM5NjE3MDE0OUM3NDM5NjNDRDgx
OTA4Cj4gPiA+ID4gMTA6MTU6MDYuNDgwODMzIElQIDEwLjAuMC4zMC4yMDQ5ID4gMTAuMC4wLjQw
LjEwMjI6IC4gYWNrIDEwOCB3aW4gMTgxIDxub3Asbm9wLHRpbWVzdGFtcCA3Mzc5MTggNjg2MTI4
Pgo+ID4gPiA+IDEwOjE1OjA2LjUxMzM2NSBJUCAxMC4wLjAuMzAuMjA0OSA+IDEwLjAuMC40MC4y
MDc5ODA0Njc4OiByZXBseSBvayA4NCBmc2luZm8gcnRtYXggNjU1MzYgcnRwcmVmIDY1NTM2IHd0
bWF4IDY1NTM2IHd0cHJlZiA2NTUzNiBkdHByZWYgNDA5Ngo+ID4gPiA+IDEwOjE1OjA2LjUxNDQw
OCBJUCAxMC4wLjAuNDAuMTAyMiA+IDEwLjAuMC4zMC4yMDQ5OiAuIGFjayA4NSB3aW4gMTQ2MCA8
bm9wLG5vcCx0aW1lc3RhbXAgNjg2MTYwIDczNzkyNj4KPiA+ID4gPiAxMDoxNTowNi41MTQ5MDIg
SVAgMTAuMC4wLjQwLjIwOTY1ODE4OTQgPiAxMC4wLjAuMzAuMjA0OTogMTA4IGdldGF0dHIgZmgg
VW5rbm93bi8wMTAwMDcwMDgxMzgwMTAwMDAwMDAwMDAwODE2NEVFNDJEQjE0MUVCQUM5NjE3MDE0
OUM3NDM5NjNDRDgxOTA4Cj4gPiA+ID4gMTA6MTU6MDYuNTE1MjU2IElQIDEwLjAuMC4zMC4yMDQ5
ID4gMTAuMC4wLjQwLjIwOTY1ODE4OTQ6IHJlcGx5IG9rIDExNiBnZXRhdHRyIERJUiA0MDc1NSBp
ZHMgMC8wIHN6IDQwOTYKPiA+ID4gPiAxMDoxNTowNi41NTM4NjUgSVAgMTAuMC4wLjQwLjEwMjIg
PiAxMC4wLjAuMzAuMjA0OTogLiBhY2sgMjAxIHdpbiAxNDYwIDxub3Asbm9wLHRpbWVzdGFtcCA2
ODYyMDEgNzM3OTI2Pgo+ID4gPiA+IDEwOjE2OjAzLjA5MTc4NCBJUCAxMC4wLjAuNDAuMjExMzM1
OTExMCA+IDEwLjAuMC4zMC4yMDQ5OiAxMDggZ2V0YXR0ciBmaCBVbmtub3duLzAxMDAwNzAwODEz
ODAxMDAwMDAwMDAwMDA4MTY0RUU0MkRCMTQxRUJBQzk2MTcwMTQ5Qzc0Mzk2NDcyMTlDOEMKPiA+
ID4gPiAxMDoxNjowMy4wOTMzNjYgSVAgMTAuMC4wLjMwLjIwNDkgPiAxMC4wLjAuNDAuMjExMzM1
OTExMDogcmVwbHkgb2sgMTE2IGdldGF0dHIgRElSIDQwNzU1IGlkcyAwLzAgc3ogNDA5Ngo+ID4g
PiA+IDEwOjE2OjAzLjA5NjA0NiBJUCAxMC4wLjAuNDAuMTAyMiA+IDEwLjAuMC4zMC4yMDQ5OiAu
IGFjayAzMTcgd2luIDE0NjAgPG5vcCxub3AsdGltZXN0YW1wIDc0MzYxMCA3NTIwNzE+Cj4gPiA+
ID4gMTA6MTY6MDMuMDk3MzcwIElQIDEwLjAuMC40MC4yMTMwMTM2MzI2ID4gMTAuMC4wLjMwLjIw
NDk6IDExMiBhY2Nlc3MgZmggVW5rbm93bi8wMTAwMDcwMDgxMzgwMTAwMDAwMDAwMDAwODE2NEVF
NDJEQjE0MUVCQUM5NjE3MDE0OUM3NDM5NjAwMDAwMDFGIDAwMWYKPiA+ID4gPiAxMDoxNjowMy4w
OTgxNTYgSVAgMTAuMC4wLjMwLjIwNDkgPiAxMC4wLjAuNDAuMjEzMDEzNjMyNjogcmVwbHkgb2sg
MTI0IGFjY2VzcyBjIDAwMDMKPiA+ID4gPiAxMDoxNjowMy4xNTY4MTIgSVAgMTAuMC4wLjQwLjEw
MjIgPiAxMC4wLjAuMzAuMjA0OTogLiBhY2sgNDQxIHdpbiAxNDYwIDxub3Asbm9wLHRpbWVzdGFt
cCA3NDM2NTEgNzUyMDcyPgo+ID4gPiA+IDEwOjE2OjA4Ljk2NzgwNCBJUCAxMC4wLjAuNDAuMjE0
NjkxMzU0MiA+IDEwLjAuMC4zMC4yMDQ5OiAxMDggZ2V0YXR0ciBmaCBVbmtub3duLzAxMDAwNzAw
ODEzODAxMDAwMDAwMDAwMDA4MTY0RUU0MkRCMTQxRUJBQzk2MTcwMTQ5Qzc0Mzk2MDAwMDAwMDAK
PiA+ID4gPiAxMDoxNjowOC45NjgzMDUgSVAgMTAuMC4wLjMwLjIwNDkgPiAxMC4wLjAuNDAuMjE0
NjkxMzU0MjogcmVwbHkgb2sgMTE2IGdldGF0dHIgRElSIDQwNzU1IGlkcyAwLzAgc3ogNDA5Ngo+
ID4gPiA+IDEwOjE2OjA4Ljk3NDI4NSBJUCAxMC4wLjAuNDAuMTAyMiA+IDEwLjAuMC4zMC4yMDQ5
OiAuIGFjayA1NTcgd2luIDE0NjAgPG5vcCxub3AsdGltZXN0YW1wIDc0OTQ3NyA3NTM1NDA+Cj4g
PiA+ID4gMTA6MTY6MDguOTc1NzM5IElQIDEwLjAuMC40MC4yMTYzNjkwNzU4ID4gMTAuMC4wLjMw
LjIwNDk6IDEzMiByZWFkZGlycGx1cyBmaCBVbmtub3duLzAxMDAwNzAwODEzODAxMDAwMDAwMDAw
MDA4MTY0RUU0MkRCMTQxRUJBQzk2MTcwMTQ5Qzc0Mzk2MDAwMDAwMDAgNTEyIGJ5dGVzIEAgMAo+
ID4gPiA+IDEwOjE2OjA4Ljk3NTk4NSBJUCAxMC4wLjAuMzAuMjA0OSA+IDEwLjAuMC40MC4yMTYz
NjkwNzU4OiByZXBseSBvayAxNDQ4IHJlYWRkaXJwbHVzCj4gPiA+ID4gMTA6MTY6MDguOTc2NTEw
IElQIDEwLjAuMC4zMC4yMDQ5ID4gMTAuMC4wLjQwLjE2ODQxMDgyODg6IHJlcGx5IFVua25vd24g
cnBjIHJlc3BvbnNlIGNvZGU9MjAyMTg1NTg2MSAzNDAKPiA+ID4gPiAxMDoxNjowOC45ODIyMzgg
SVAgMTAuMC4wLjQwLjEwMjIgPiAxMC4wLjAuMzAuMjA0OTogLiBhY2sgMjM0NSB3aW4gMjE4NCA8
bm9wLG5vcCx0aW1lc3RhbXAgNzQ5NDc5IDc1MzU0MT4KPiA+ID4gPiAxMDoxNjowOC45ODQ0MTUg
SVAgMTAuMC4wLjQwLjIxODA0Njc5NzQgPiAxMC4wLjAuMzAuMjA0OTogMTE2IGxvb2t1cCBmaCBV
bmtub3duLzAxMDAwNzAwODEzODAxMDAwMDAwMDAwMDA4MTY0RUU0MkRCMTQxRUJBQzk2MTcwMTQ5
Qzc0Mzk2MDAwMDAwMDQgImlzbzMiCj4gPiA+ID4gMTA6MTY6MDguOTg0OTMxIElQIDEwLjAuMC4z
MC4yMDQ5ID4gMTAuMC4wLjQwLjIxODA0Njc5NzQ6IHJlcGx5IG9rIDIzMiBsb29rdXAgZmggVW5r
bm93bi8wMTAwMDYwMDI3QUU5QzY3MEI3OTRCMzY4QjVFOUUxNzM3QjU2OUVCMDAwMDAwMDEwMDAw
MDAwMjAwMDA0MUVECj4gPiA+ID4gMTA6MTY6MDkuMDA4ODE5IElQIDEwLjAuMC40MC4yMTk3MjQ1
MTkwID4gMTAuMC4wLjMwLjIwNDk6IDEwNCBnZXRhdHRyIGZoIFVua25vd24vMDEwMDA2MDAyN0FF
OUM2NzBCNzk0QjM2OEI1RTlFMTczN0I1NjlFQjAwMDAwMDBGREQyMkNDM0M3MDAyRjBBQwo+ID4g
PiA+IDEwOjE2OjA5LjAxMDk0NiBJUCAxMC4wLjAuMzAuMjA0OSA+IDEwLjAuMC40MC4yMTk3MjQ1
MTkwOiByZXBseSBvayAxODggZ2V0YXR0ciBSRUcgMiBpZHMgNS8wIHN6IDAKPiA+ID4gPiAxMDox
NjowOS4wMzI1NDEgSVAgMTAuMC4wLjQwLjIyMTQwMjI0MDYgPiAxMC4wLjAuMzAuMjA0OTogMTI4
IGdldGF0dHIgZmggVW5rbm93bi8wMTAwMDcwMjgxMzgwMTAwMDAwMDAwMDAwODE2NEVFNDJEQjE0
MUVCQUM5NjE3MDE0OUM3NDM5NkU2NkQwNzAwCj4gPiA+ID4gMTA6MTY6MDkuMDMzNDA1IElQIDEw
LjAuMC4zMC4yMDQ5ID4gMTAuMC4wLjQwLjIyMTQwMjI0MDY6IHJlcGx5IG9rIDE4OCBnZXRhdHRy
IFJFRyAxIGlkcyAxLzAgc3ogMAo+ID4gPiA+IDEwOjE2OjA5LjAzMzQ5MCBJUCAxMC4wLjAuNDAu
MjIzMDc5OTYyMiA+IDEwLjAuMC4zMC4yMDQ5OiAxMjggZ2V0YXR0ciBmaCBVbmtub3duLzAxMDAw
NzAyODEzODAxMDAwMDAwMDAwMDA4MTY0RUU0MkRCMTQxRUJBQzk2MTcwMTQ5Qzc0Mzk2RTU2RDA3
MDAKPiA+ID4gPiAxMDoxNjowOS4wMzY4MTEgSVAgMTAuMC4wLjMwLjIwNDkgPiAxMC4wLjAuNDAu
MjIzMDc5OTYyMjogcmVwbHkgb2sgMTg4IGdldGF0dHIgUkVHIDEgaWRzIDEvMCBzeiAwCj4gPiA+
ID4gMTA6MTY6MDkuMDM3ODIzIElQIDEwLjAuMC40MC4yMjQ3NTc2ODM4ID4gMTAuMC4wLjMwLjIw
NDk6IDEwOCBnZXRhdHRyIGZoIFVua25vd24vMDEwMDA3MDA4MTM4MDEwMDAwMDAwMDAwMDgxNjRF
RTQyREIxNDFFQkFDOTYxNzAxNDlDNzQzOTYwMDAwMDAwMAo+ID4gPiA+IDEwOjE2OjA5LjAzOTgx
NyBJUCAxMC4wLjAuMzAuMjA0OSA+IDEwLjAuMC40MC4yMjQ3NTc2ODM4OiByZXBseSBvayAxMTYg
Z2V0YXR0ciBESVIgNDA3NTUgaWRzIDAvMCBzeiA0MDk2Cj4gPiA+ID4gMTA6MTY6MDkuMDQwNDIz
IElQIDEwLjAuMC40MC4yMjY0MzU0MDU0ID4gMTAuMC4wLjMwLjIwNDk6IDExMiBnZXRhdHRyIGZo
IFVua25vd24vMDEwMDA3MDA4MTM4MDEwMDAwMDAwMDAwMDgxNjRFRTQyREIxNDFFQkFDOTYxNzAx
NDlDNzQzOTYwMDAwMDAwRgo+ID4gPiA+IDEwOjE2OjA5LjA0MDg1NiBJUCAxMC4wLjAuMzAuMjA0
OSA+IDEwLjAuMC40MC4yMjY0MzU0MDU0OiByZXBseSBvayAxODggZ2V0YXR0ciBSRUcgMiBpZHMg
NS8wIHN6IDAKPiA+ID4gPiAxMDoxNjowOS4wNDE1OTAgSVAgMTAuMC4wLjQwLjIyODExMzEyNzAg
PiAxMC4wLjAuMzAuMjA0OTogMTE2IGxvb2t1cCBmaCBVbmtub3duLzAxMDAwNzAwODEzODAxMDAw
MDAwMDAwMDA4MTY0RUU0MkRCMTQxRUJBQzk2MTcwMTQ5Qzc0Mzk2MDAwMDAwMDQgImlzbzIiCj4g
PiA+ID4gMTA6MTY6MDkuMDQxOTIwIElQIDEwLjAuMC4zMC4yMDQ5ID4gMTAuMC4wLjQwLjIyODEx
MzEyNzA6IHJlcGx5IG9rIDIzMiBsb29rdXAgZmggVW5rbm93bi8wMTAwMDYwMDJBQUQ2RUE1QTA1
RDQ0NDFCOTRDNDhFNkU1RDk5ODFFMDAwMDAwMDEwMDAwMDAwMjAwMDA0MUVECj4gPiA+ID4gMTA6
MTY6MDkuMDQ5NjMzIElQIDEwLjAuMC40MC4yMjk3OTA4NDg2ID4gMTAuMC4wLjMwLjIwNDk6IDEw
NCBnZXRhdHRyIGZoIFVua25vd24vMDEwMDA2MDAyQUFENkVBNUEwNUQ0NDQxQjk0QzQ4RTZFNUQ5
OTgxRTAwMDAwMDBGNUZEQzg0NDU0MzI2RTE5Mwo+ID4gPiA+IDEwOjE2OjA5LjA0OTc4MSBJUCAx
MC4wLjAuMzAuMjA0OSA+IDEwLjAuMC40MC4yMjk3OTA4NDg2OiByZXBseSBvayAxODggZ2V0YXR0
ciBSRUcgMiBpZHMgNS8wIHN6IDAKPiA+ID4gPiAxMDoxNjowOS4wNjMyMTggSVAgMTAuMC4wLjQw
LjIzMTQ2ODU3MDIgPiAxMC4wLjAuMzAuMjA0OTogMTI4IGdldGF0dHIgZmggVW5rbm93bi8wMTAw
MDcwMjgxMzgwMTAwMDAwMDAwMDAwODE2NEVFNDJEQjE0MUVCQUM5NjE3MDE0OUM3NDM5NkU0NkQw
NzAwCj4gPiA+ID4gMTA6MTY6MDkuMDcyNTA1IElQIDEwLjAuMC4zMC4yMDQ5ID4gMTAuMC4wLjQw
LjIzMTQ2ODU3MDI6IHJlcGx5IG9rIDE4OCBnZXRhdHRyIFJFRyAxIGlkcyAxLzAgc3ogMAo+ID4g
PiA+IDEwOjE2OjA5LjA5MTY5OCBJUCAxMC4wLjAuNDAuMjMzMTQ2MjkxOCA+IDEwLjAuMC4zMC4y
MDQ5OiAxMjQgZ2V0YXR0ciBmaCBVbmtub3duLzAxMDAwNzAyODEzODAxMDAwMDAwMDAwMDA4MTY0
RUU0MkRCMTQxRUJBQzk2MTcwMTQ5Qzc0Mzk2RTQ2RDA3MDAKPiA+ID4gPiAxMDoxNjowOS4wOTIw
MjIgSVAgMTAuMC4wLjMwLjIwNDkgPiAxMC4wLjAuNDAuMjMzMTQ2MjkxODogcmVwbHkgb2sgMTE2
IGdldGF0dHIgUkVHIDEwMDY0NCBpZHMgMC8wIHN6IDEwNDg1NzYKPiA+ID4gPiAxMDoxNjowOS4x
Mjg5NzEgSVAgMTAuMC4wLjQwLjIzNDgyNDAxMzQgPiAxMC4wLjAuMzAuMjA0OTogMTI4IGdldGF0
dHIgZmggVW5rbm93bi8wMTAwMDcwMjgxMzgwMTAwMDAwMDAwMDAwODE2NEVFNDJEQjE0MUVCQUM5
NjE3MDE0OUM3NDM5NkUzNkQwNzAwCj4gPiA+ID4gMTA6MTY6MDkuMTI5MzA0IElQIDEwLjAuMC4z
MC4yMDQ5ID4gMTAuMC4wLjQwLjIzNDgyNDAxMzQ6IHJlcGx5IG9rIDE4OCBnZXRhdHRyIFJFRyAx
IGlkcyAxLzAgc3ogMAo+ID4gPiA+IDEwOjE2OjA5LjE4NDE1NSBJUCAxMC4wLjAuNDAuMjM2NTAx
NzM1MCA+IDEwLjAuMC4zMC4yMDQ5OiAxMjQgZ2V0YXR0ciBmaCBVbmtub3duLzAxMDAwNzAyODEz
ODAxMDAwMDAwMDAwMDA4MTY0RUU0MkRCMTQxRUJBQzk2MTcwMTQ5Qzc0Mzk2RTM2RDA3MDAKPiA+
ID4gPiAxMDoxNjowOS4xODQ1ODIgSVAgMTAuMC4wLjMwLjIwNDkgPiAxMC4wLjAuNDAuMjM2NTAx
NzM1MDogcmVwbHkgb2sgMTE2IGdldGF0dHIgUkVHIDEwMDY0NCBpZHMgMC8wIHN6IDEwNDg1NzYK
PiA+ID4gPiAxMDoxNjowOS4xODkyMzQgSVAgMTAuMC4wLjQwLjIzODE3OTQ1NjYgPiAxMC4wLjAu
MzAuMjA0OTogMTE2IGxvb2t1cCBmaCBVbmtub3duLzAxMDAwNzAwODEzODAxMDAwMDAwMDAwMDA4
MTY0RUU0MkRCMTQxRUJBQzk2MTcwMTQ5Qzc0Mzk2MDAwMDAwMDQgImlzbzEiCj4gPiA+ID4gMTA6
MTY6MDkuMTg5NDM1IElQIDEwLjAuMC4zMC4yMDQ5ID4gMTAuMC4wLjQwLjIzODE3OTQ1NjY6IHJl
cGx5IG9rIDIzMiBsb29rdXAgZmggVW5rbm93bi8wMTAwMDYwMDJDNDlGRUYyQkE0NjQyOTM5QjJC
RjJCODMyMkNDQkNCMDAwMDAwMDEwMDAwMDAwMjAwMDA0MUVECj4gPiA+ID4gMTA6MTY6MDkuMTkz
NDc2IElQIDEwLjAuMC40MC4yMzk4NTcxNzgyID4gMTAuMC4wLjMwLjIwNDk6IDEwNCBnZXRhdHRy
IGZoIFVua25vd24vMDEwMDA2MDAyQzQ5RkVGMkJBNDY0MjkzOUIyQkYyQjgzMjJDQ0JDQjAwMDAw
MDBGNTg4OTZBODg0QTBDNjJCNwo+ID4gPiA+IDEwOjE2OjA5LjE5MzY1MiBJUCAxMC4wLjAuMzAu
MjA0OSA+IDEwLjAuMC40MC4yMzk4NTcxNzgyOiByZXBseSBvayAxODggZ2V0YXR0ciBSRUcgMiBp
ZHMgNS8wIHN6IDAKPiA+ID4gPiAxMDoxNjowOS4xOTQ5MzcgSVAgMTAuMC4wLjQwLjI0MTUzNDg5
OTggPiAxMC4wLjAuMzAuMjA0OTogMTI4IGdldGF0dHIgZmggVW5rbm93bi8wMTAwMDcwMjgxMzgw
MTAwMDAwMDAwMDAwODE2NEVFNDJEQjE0MUVCQUM5NjE3MDE0OUM3NDM5NkU3NkQwNzAwCj4gPiA+
ID4gMTA6MTY6MDkuMTk1MDMzIElQIDEwLjAuMC4zMC4yMDQ5ID4gMTAuMC4wLjQwLjI0MTUzNDg5
OTg6IHJlcGx5IG9rIDE4OCBnZXRhdHRyIFJFRyAxIGlkcyAxLzAgc3ogMAo+ID4gPiA+IDEwOjE2
OjA5LjE5NTIzMCBJUCAxMC4wLjAuNDAuMjQzMjEyNjIxNCA+IDEwLjAuMC4zMC4yMDQ5OiAxMjgg
Z2V0YXR0ciBmaCBVbmtub3duLzAxMDAwNzAyODEzODAxMDAwMDAwMDAwMDA4MTY0RUU0MkRCMTQx
RUJBQzk2MTcwMTQ5Qzc0Mzk2RTg2RDA3MDAKPiA+ID4gPiAxMDoxNjowOS4zMjM2MzUgSVAgMTAu
MC4wLjMwLjIwNDkgPiAxMC4wLjAuNDAuMTAyMjogLiBhY2sgMjU3MiB3aW4gNDE2IDxub3Asbm9w
LHRpbWVzdGFtcCA3NTM2MjkgNzQ5NTY2Pgo+ID4gPiA+IDEwOjE2OjA5LjMyNDM0NSBJUCAxMC4w
LjAuMzAuMjA0OSA+IDEwLjAuMC40MC4yNDMyMTI2MjE0OiByZXBseSBvayAxODggZ2V0YXR0ciBS
RUcgMSBpZHMgMS8wIHN6IDAKPiA+ID4gPiAxMDoxNjowOS4zNDE0NzUgSVAgMTAuMC4wLjQwLjI0
NDg5MDM0MzAgPiAxMC4wLjAuMzAuMjA0OTogMTI4IGdldGF0dHIgZmggVW5rbm93bi8wMTAwMDcw
MjgxMzgwMTAwMDAwMDAwMDAwODE2NEVFNDJEQjE0MUVCQUM5NjE3MDE0OUM3NDM5NkUyNkQwNzAw
Cj4gPiA+ID4gMTA6MTY6MDkuMzQxNTI0IElQIDEwLjAuMC4zMC4yMDQ5ID4gMTAuMC4wLjQwLjEw
MjI6IC4gYWNrIDI3MDAgd2luIDQ0OSA8bm9wLG5vcCx0aW1lc3RhbXAgNzUzNjMyIDc0OTcwNz4K
PiA+ID4gPiAxMDoxNjowOS4zNDE5MjggSVAgMTAuMC4wLjMwLjIwNDkgPiAxMC4wLjAuNDAuMjQ0
ODkwMzQzMDogcmVwbHkgb2sgMTg4IGdldGF0dHIgUkVHIDEgaWRzIDEvMCBzeiAwCj4gPiA+ID4g
MTA6MTY6MDkuMzQyMTE3IElQIDEwLjAuMC40MC4yNDY1NjgwNjQ2ID4gMTAuMC4wLjMwLjIwNDk6
IDEyNCBnZXRhdHRyIGZoIFVua25vd24vMDEwMDA3MDI4MTM4MDEwMDAwMDAwMDAwMDgxNjRFRTQy
REIxNDFFQkFDOTYxNzAxNDlDNzQzOTZFMjZEMDcwMAo+ID4gPiA+IDEwOjE2OjA5LjM0MzQwNCBJ
UCAxMC4wLjAuMzAuMjA0OSA+IDEwLjAuMC40MC4yNDY1NjgwNjQ2OiByZXBseSBvayAxMTYgZ2V0
YXR0ciBSRUcgMTAwNjQ0IGlkcyAwLzAgc3ogMTA0ODU3Ngo+ID4gPiA+IDEwOjE2OjA5LjM4OTMx
NiBJUCAxMC4wLjAuNDAuMTAyMiA+IDEwLjAuMC4zMC4yMDQ5OiAuIGFjayA1NTczIHdpbiAyMTg0
IDxub3Asbm9wLHRpbWVzdGFtcCA3NDk3NTEgNzUzNjMzPgo+ID4gPiA+IDEwOjE2OjEzLjQ0OTUx
MyBJUCAxMC4wLjAuNDAuMjQ4MjQ1Nzg2MiA+IDEwLjAuMC4zMC4yMDQ5OiAxMDggZ2V0YXR0ciBm
aCBVbmtub3duLzAxMDAwNzAwODEzODAxMDAwMDAwMDAwMDA4MTY0RUU0MkRCMTQxRUJBQzk2MTcw
MTQ5Qzc0Mzk2NDcyMTlDQjUKPiA+ID4gPiAxMDoxNjoxMy40NDk4MTUgSVAgMTAuMC4wLjMwLjIw
NDkgPiAxMC4wLjAuNDAuMjQ4MjQ1Nzg2MjogcmVwbHkgb2sgMTE2IGdldGF0dHIgRElSIDQwNzU1
IGlkcyAwLzAgc3ogNDA5Ngo+ID4gPiA+IDEwOjE2OjEzLjQ1MjM0NCBJUCAxMC4wLjAuNDAuMTAy
MiA+IDEwLjAuMC4zMC4yMDQ5OiAuIGFjayA1Njg5IHdpbiAyMTg0IDxub3Asbm9wLHRpbWVzdGFt
cCA3NTM5NDMgNzU0NjYwPgo+ID4gPiA+IDEwOjE2OjEzLjQ1Mzk3MyBJUCAxMC4wLjAuNDAuMjQ5
OTIzNTA3OCA+IDEwLjAuMC4zMC4yMDQ5OiAxMDAgZ2V0YXR0ciBmaCBVbmtub3duLzAxMDAwNjAw
MkM0OUZFRjJCQTQ2NDI5MzlCMkJGMkI4MzIyQ0NCQ0I0NzIxOUM4QzAwMDAwMDAwNDcyMTlDOEMK
PiA+ID4gPiAxMDoxNjoxMy40NTQxNTQgSVAgMTAuMC4wLjMwLjIwNDkgPiAxMC4wLjAuNDAuMjQ5
OTIzNTA3ODogcmVwbHkgb2sgMTE2IGdldGF0dHIgRElSIDQwNzU1IGlkcyAwLzAgc3ogNDA5Ngo+
ID4gPiA+IDEwOjE2OjEzLjQ1NjE5NCBJUCAxMC4wLjAuNDAuMjUxNjAxMjI5NCA+IDEwLjAuMC4z
MC4yMDQ5OiAxMDggZ2V0YXR0ciBmaCBVbmtub3duLzAxMDAwNzAwODEzODAxMDAwMDAwMDAwMDA4
MTY0RUU0MkRCMTQxRUJBQzk2MTcwMTQ5Qzc0Mzk2NDcyMTlDOEMKPiA+ID4gPiAxMDoxNjoxMy40
NTYzNjEgSVAgMTAuMC4wLjMwLjIwNDkgPiAxMC4wLjAuNDAuMjUxNjAxMjI5NDogcmVwbHkgb2sg
MTE2IGdldGF0dHIgRElSIDQwNzU1IGlkcyAwLzAgc3ogNDA5Ngo+ID4gPiA+IDEwOjE2OjEzLjQ1
ODI4MiBJUCAxMC4wLjAuNDAuMjUzMjc4OTUxMCA+IDEwLjAuMC4zMC4yMDQ5OiAxMTYgbG9va3Vw
IGZoIFVua25vd24vMDEwMDA3MDA4MTM4MDEwMDAwMDAwMDAwMDgxNjRFRTQyREIxNDFFQkFDOTYx
NzAxNDlDNzQzOTYwMDAwMDAwNCAiaXNvMSIKPiA+ID4gPiAxMDoxNjoxMy40NTg0NjEgSVAgMTAu
MC4wLjMwLjIwNDkgPiAxMC4wLjAuNDAuMjUzMjc4OTUxMDogcmVwbHkgb2sgMjMyIGxvb2t1cCBm
aCBVbmtub3duLzAxMDAwNjAwMkM0OUZFRjJCQTQ2NDI5MzlCMkJGMkI4MzIyQ0NCQ0IwMDAwMDAw
MTAwMDAwMDAyMDAwMDQxRUQKPiA+ID4gPiAxMDoxNjoxMy41MTA2MzcgSVAgMTAuMC4wLjQwLjEw
MjIgPiAxMC4wLjAuMzAuMjA0OTogLiBhY2sgNjE1MyB3aW4gMjE4NCA8bm9wLG5vcCx0aW1lc3Rh
bXAgNzUzOTg1IDc1NDY2Mj4KPiA+ID4gPiAxMDoxNjoxNC4wMzAxMTAgSVAgMTAuMC4wLjQwLjI1
NDk1NjY3MjYgPiAxMC4wLjAuMzAuMjA0OTogMTEyIGFjY2VzcyBmaCBVbmtub3duLzAxMDAwNzAw
ODEzODAxMDAwMDAwMDAwMDA4MTY0RUU0MkRCMTQxRUJBQzk2MTcwMTQ5Qzc0Mzk2MDAwMDAwMUYg
MDAxZgo+ID4gPiA+IDEwOjE2OjE0LjAzMDkyNyBJUCAxMC4wLjAuMzAuMjA0OSA+IDEwLjAuMC40
MC4yNTQ5NTY2NzI2OiByZXBseSBvayAxMjQgYWNjZXNzIGMgMDAwMwo+ID4gPiA+IDEwOjE2OjE0
LjAzMzQzNiBJUCAxMC4wLjAuNDAuMTAyMiA+IDEwLjAuMC4zMC4yMDQ5OiAuIGFjayA2Mjc3IHdp
biAyMTg0IDxub3Asbm9wLHRpbWVzdGFtcCA3NTQ1NDggNzU0ODA1Pgo+ID4gPiA+IDEwOjE2OjE0
LjAzNDczMiBJUCAxMC4wLjAuNDAuMjU2NjM0Mzk0MiA+IDEwLjAuMC4zMC4yMDQ5OiAxMDggZ2V0
YXR0ciBmaCBVbmtub3duLzAxMDAwNzAwODEzODAxMDAwMDAwMDAwMDA4MTY0RUU0MkRCMTQxRUJB
Qzk2MTcwMTQ5Qzc0Mzk2MDAwMDAwMDAKPiA+ID4gPiAxMDoxNjoxNC4wMzQ5ODAgSVAgMTAuMC4w
LjMwLjIwNDkgPiAxMC4wLjAuNDAuMjU2NjM0Mzk0MjogcmVwbHkgb2sgMTE2IGdldGF0dHIgRElS
IDQwNzU1IGlkcyAwLzAgc3ogNDA5Ngo+ID4gPiA+IDEwOjE2OjE0LjAzNzMxOSBJUCAxMC4wLjAu
NDAuMjU4MzEyMTE1OCA+IDEwLjAuMC4zMC4yMDQ5OiAxMTIgYWNjZXNzIGZoIFVua25vd24vMDEw
MDA3MDA4MTM4MDEwMDAwMDAwMDAwMDgxNjRFRTQyREIxNDFFQkFDOTYxNzAxNDlDNzQzOTYwMDAw
MDAxRiAwMDFmCj4gPiA+ID4gMTA6MTY6MTQuMDM3NDg2IElQIDEwLjAuMC4zMC4yMDQ5ID4gMTAu
MC4wLjQwLjI1ODMxMjExNTg6IHJlcGx5IG9rIDEyNCBhY2Nlc3MgYyAwMDAzCj4gPiA+ID4gMTA6
MTY6MTQuMDQwMzIzIElQIDEwLjAuMC40MC4yNTk5ODk4Mzc0ID4gMTAuMC4wLjMwLjIwNDk6IDEz
MiByZWFkZGlycGx1cyBmaCBVbmtub3duLzAxMDAwNzAwODEzODAxMDAwMDAwMDAwMDA4MTY0RUU0
MkRCMTQxRUJBQzk2MTcwMTQ5Qzc0Mzk2MDAwMDAwMDAgNTEyIGJ5dGVzIEAgMAo+ID4gPiA+IDEw
OjE2OjE0LjA0MDU1NCBJUCAxMC4wLjAuMzAuMjA0OSA+IDEwLjAuMC40MC4yNTk5ODk4Mzc0OiBy
ZXBseSBvayAxNDQ4IHJlYWRkaXJwbHVzCj4gPiA+ID4gMTA6MTY6MTQuMDQxMDIwIElQIDEwLjAu
MC4zMC4yMDQ5ID4gMTAuMC4wLjQwLjE2ODQxMDgyODg6IHJlcGx5IFVua25vd24gcnBjIHJlc3Bv
bnNlIGNvZGU9MjAyMTg1NTg2MSAzNDAKPiA+ID4gPiAxMDoxNjoxNC4wNDM1ODMgSVAgMTAuMC4w
LjQwLjEwMjIgPiAxMC4wLjAuMzAuMjA0OTogLiBhY2sgODMwNSB3aW4gMjkwOCA8bm9wLG5vcCx0
aW1lc3RhbXAgNzU0NTUwIDc1NDgwOD4KPiA+ID4gPiAxMDoxNjoxNC4wNDUxMDQgSVAgMTAuMC4w
LjQwLjI2MTY2NzU1OTAgPiAxMC4wLjAuMzAuMjA0OTogMTEyIGFjY2VzcyBmaCBVbmtub3duLzAx
MDAwNzAwODEzODAxMDAwMDAwMDAwMDA4MTY0RUU0MkRCMTQxRUJBQzk2MTcwMTQ5Qzc0Mzk2MDAw
MDAwMUYgMDAxZgo+ID4gPiA+IDEwOjE2OjE0LjA0NTQwMiBJUCAxMC4wLjAuMzAuMjA0OSA+IDEw
LjAuMC40MC4yNjE2Njc1NTkwOiByZXBseSBvayAxMjQgYWNjZXNzIGMgMDAwMwo+ID4gPiA+IDEw
OjE2OjE0LjA0NzgzMCBJUCAxMC4wLjAuNDAuMjYzMzQ1MjgwNiA+IDEwLjAuMC4zMC4yMDQ5OiAx
MTIgYWNjZXNzIGZoIFVua25vd24vMDEwMDA3MDA4MTM4MDEwMDAwMDAwMDAwMDgxNjRFRTQyREIx
NDFFQkFDOTYxNzAxNDlDNzQzOTYwMDAwMDAxRiAwMDFmCj4gPiA+ID4gMTA6MTY6MTQuMDQ4MDM5
IElQIDEwLjAuMC4zMC4yMDQ5ID4gMTAuMC4wLjQwLjI2MzM0NTI4MDY6IHJlcGx5IG9rIDEyNCBh
Y2Nlc3MgYyAwMDAzCj4gPiA+ID4gMTA6MTY6MTQuMDk5Mzg1IElQIDEwLjAuMC40MC4xMDIyID4g
MTAuMC4wLjMwLjIwNDk6IC4gYWNrIDg1NTMgd2luIDI5MDggPG5vcCxub3AsdGltZXN0YW1wIDc1
NDU5MiA3NTQ4MTA+Cj4gPiA+ID4gMTA6MTY6MTQuMjkzNzE0IElQIDEwLjAuMC40MC4yNjUwMjMw
MDIyID4gMTAuMC4wLjMwLjIwNDk6IDEwOCBnZXRhdHRyIGZoIFVua25vd24vMDEwMDA3MDA4MTM4
MDEwMDAwMDAwMDAwMDgxNjRFRTQyREIxNDFFQkFDOTYxNzAxNDlDNzQzOTYwMDAwMDAwMAo+ID4g
PiA+IDEwOjE2OjE0LjI5NDAxOSBJUCAxMC4wLjAuMzAuMjA0OSA+IDEwLjAuMC40MC4yNjUwMjMw
MDIyOiByZXBseSBvayAxMTYgZ2V0YXR0ciBESVIgNDA3NTUgaWRzIDAvMCBzeiA0MDk2Cj4gPiA+
ID4gMTA6MTY6MTQuMjk3MjYzIElQIDEwLjAuMC40MC4xMDIyID4gMTAuMC4wLjMwLjIwNDk6IC4g
YWNrIDg2Njkgd2luIDI5MDggPG5vcCxub3AsdGltZXN0YW1wIDc1NDc2MyA3NTQ4NzE+Cj4gPiA+
ID4gMTA6MTY6MTQuMjk3MjcyIElQIDEwLjAuMC40MC4yNjY3MDA3MjM4ID4gMTAuMC4wLjMwLjIw
NDk6IDExMiBhY2Nlc3MgZmggVW5rbm93bi8wMTAwMDcwMDgxMzgwMTAwMDAwMDAwMDAwODE2NEVF
NDJEQjE0MUVCQUM5NjE3MDE0OUM3NDM5NjAwMDAwMDFGIDAwMWYKPiA+ID4gPiAxMDoxNjoxNC4y
OTc0NjYgSVAgMTAuMC4wLjMwLjIwNDkgPiAxMC4wLjAuNDAuMjY2NzAwNzIzODogcmVwbHkgb2sg
MTI0IGFjY2VzcyBjIDAwMDMKPiA+ID4gPiAxMDoxNjoxNC4yOTc1ODYgSVAgMTAuMC4wLjQwLjI2
ODM3ODQ0NTQgPiAxMC4wLjAuMzAuMjA0OTogMTEyIGFjY2VzcyBmaCBVbmtub3duLzAxMDAwNzAw
ODEzODAxMDAwMDAwMDAwMDA4MTY0RUU0MkRCMTQxRUJBQzk2MTcwMTQ5Qzc0Mzk2MDAwMDAwMUYg
MDAxZgo+ID4gPiA+IDEwOjE2OjE0LjI5Nzk4NyBJUCAxMC4wLjAuMzAuMjA0OSA+IDEwLjAuMC40
MC4yNjgzNzg0NDU0OiByZXBseSBvayAxMjQgYWNjZXNzIGMgMDAwMwo+ID4gPiA+IDEwOjE2OjE0
LjMwMTE2NSBJUCAxMC4wLjAuNDAuMjcwMDU2MTY3MCA+IDEwLjAuMC4zMC4yMDQ5OiAxMDQgYWNj
ZXNzIGZoIFVua25vd24vMDEwMDA2MDAyQzQ5RkVGMkJBNDY0MjkzOUIyQkYyQjgzMjJDQ0JDQjAw
MDAwMDFGNDcyMTlDOEMwMDAwMDAwMCAwMDFmCj4gPiA+ID4gMTA6MTY6MTQuMzAxNDA1IElQIDEw
LjAuMC4zMC4yMDQ5ID4gMTAuMC4wLjQwLjI3MDA1NjE2NzA6IHJlcGx5IG9rIDEyNCBhY2Nlc3Mg
YyAwMDAzCj4gPiA+ID4gMTA6MTY6MTQuMzUxMjI5IElQIDEwLjAuMC40MC4xMDIyID4gMTAuMC4w
LjMwLjIwNDk6IC4gYWNrIDkwNDEgd2luIDI5MDggPG5vcCxub3AsdGltZXN0YW1wIDc1NDgxMCA3
NTQ4NzM+Cj4gPiA+ID4gMTA6MTY6MTQuODQyNjMzIElQIDEwLjAuMC40MC4yNzE3MzM4ODg2ID4g
MTAuMC4wLjMwLjIwNDk6IDEwMCBnZXRhdHRyIGZoIFVua25vd24vMDEwMDA2MDAyQzQ5RkVGMkJB
NDY0MjkzOUIyQkYyQjgzMjJDQ0JDQjAwMDAwMDAwNDcyMTlDOEMwMDAwMDAwMAo+ID4gPiA+IDEw
OjE2OjE0Ljg1MTcxMSBJUCAxMC4wLjAuMzAuMjA0OSA+IDEwLjAuMC40MC4yNzE3MzM4ODg2OiBy
ZXBseSBvayAxMTYgZ2V0YXR0ciBESVIgNDA3NTUgaWRzIDAvMCBzeiA0MDk2Cj4gPiA+ID4gMTA6
MTY6MTQuODU2ODQ2IElQIDEwLjAuMC40MC4xMDIyID4gMTAuMC4wLjMwLjIwNDk6IC4gYWNrIDkx
NTcgd2luIDI5MDggPG5vcCxub3AsdGltZXN0YW1wIDc1NTI3MiA3NTUwMDg+Cj4gPiA+ID4gCj4g
PiA+ID4gCj4gPiA+ID4gPiA+IGRvZXMgc29tZWJvZHkgaGF2ZSBzdWNoIHNldHVwIHVwIGFuZCBy
dW5uaW5nIGFuZCBjYW4gdGVsbCBoaXMgZGlzdHJvIC8ga2VybmVsIGFuZCBuZnMtdXRpbHMgdmVy
c2lvbiA/Cj4gPiA+ID4gPiA+IG1heWJlIGkgY2hhbmdlIGRpc3RybyB0aGVuLgo+ID4gPiA+ID4g
Cj4gPiA+ID4gPiBJIGRvdWJ0IHRoYXQgaXQgaXMgYSBkaXN0cm8tc3BlY2lmaWMgdGhpbmcuICBB
cyBsb25nIGFzIHlvdSBoYXZlCj4gPiA+ID4gPiBuZnMtdXRpbHMtMS4xLjAgaXQgc2hvdWxkIHdv
cmsuICBJIGRvbid0IGhhdmUgYSAxMC4zIGJveAo+ID4gPiA+ID4gc2V0IHVwIHlldCwgYnV0IGl0
IHdvcmtzIGZpbmUgb24gRGViaWFuL3Vuc3RhYmxlIGZvciBtZS4KPiA+ID4gPiAKPiA+ID4gPiBv
aywgd2lsbCB0cnkgdGhpcyBvbiBkZWJpYW4uCj4gPiA+ID4gCj4gPiA+ID4gPiBNYXliZSB0cnkg
YWRkaW5nIHRoZSAibm9fcm9vdF9zcXVhc2giIGV4cG9ydCBvcHRpb24uCj4gPiA+ID4gbm8gZGlm
ZmVyZW5jZQo+ID4gPiA+IAo+ID4gPiA+ID4gV2hhdCBkb2VzICJscyAtbCAvZXhwb3J0IiBvbiB0
aGUgc2VydmVyIHNob3c/Cj4gPiA+ID4gbm90aGluZyB1bnVzdWFsLiBubyBlcnJvcnMsIGp1c3Qg
dGhlIGRpcnMvbW91bnRwb2ludHMKPiA+ID4gPiAKPiA+ID4gPiBUaGFua3MgZm9yIHlvdXIgaGVs
cCEKPiA+ID4gPiAKPiA+ID4gPiByZWdhcmRzCj4gPiA+ID4gcm9sYW5kCj4gPiA+ID4gCj4gPiA+
ID4gCj4gPiA+ID4gPiAKPiA+ID4gPiA+IE9uIFNhdHVyZGF5IE9jdG9iZXIgMjcsIGRldnplcm9A
d2ViLmRlIHdyb3RlOgo+ID4gPiA+ID4gPiBIZWxsbyAhCj4gPiA+ID4gPiA+IAo+ID4gPiA+ID4g
PiB3aXRoIDIuNi4yMiBpYG0gdHJ5aW5nIHRvIGV4cG9ydCBsb29wYmFjayBtb3VudGVkIGlzby1p
bWFnZXMuCj4gPiA+ID4gPiA+IAo+ID4gPiA+ID4gPiB0aGlzIGlzIC9ldGMvZXhwb3J0czoKPiA+
ID4gPiA+ID4gCj4gPiA+ID4gPiA+IC9leHBvcnQgKihybyxjcm9zc21udCxzdWJ0cmVlX2NoZWNr
KQo+ID4gPiA+ID4gCj4gPiA+ID4gPiBJIHJlY29tbWVuZCByZXBsYWNpbmcgc3VidHJlZV9jaGVj
ayB3aXRoIG5vX3N1YnRyZWVfY2hlY2ssIGJ1dCBpdAo+ID4gPiA+ID4gc2hvdWxkbid0IG1ha2Ug
YW4gaW1wb3J0YW50IGRpZmZlcmVuY2UgaW4gdGhpcyBjYXNlLgo+ID4gPiA+ID4gCj4gPiA+ID4g
PiAKPiA+ID4gPiA+IFRoaXMgc2hvdWxkIHdvcmsgd2l0aCBuZnMtdXRpbHMgMS4xLjAgb3IgbGF0
ZXIuICBXaXRoIGVhcmxpZXIgcmVsZWFzZXMKPiA+ID4gPiA+IHlvdSBuZWVkIHRvIGV4cGxpY2l0
bHkgZXhwb3J0IHRoZSBzdWJvcmRpbmF0ZSBmaWxlc3lzdGVtcyB0b28uCj4gPiA+ID4gPiAKPiA+
ID4gPiA+ID4gCj4gPiA+ID4gPiA+IGluIC9leHBvcnQsIGkgaGF2ZSBsb29wYmFjayBtb3VudGVk
IGlzby1pbWFnZXMKPiA+ID4gPiA+ID4gCj4gPiA+ID4gPiA+IGFmdGVyIG1vdW50aW5nIG9uIHRo
ZSBjbGllbnQgc2lkZSB1bmRlciAvbW50ICh0cmllZCBvbmUgb2xkZXIgYW5kIG9uZSByZWNlbnQg
c3lzdGVtKSAsIGlgbSBnZXR0aW5nOgo+ID4gPiA+ID4gPiAKPiA+ID4gPiA+ID4gdm1ob3N0Oi9t
bnQgIyBscyAtbGEKPiA+ID4gPiA+ID4gL2Jpbi9sczogaXNvMTogSW5wdXQvb3V0cHV0IGVycm9y
Cj4gPiA+ID4gPiA+IC9iaW4vbHM6IGlzbzIgIElucHV0L291dHB1dCBlcnJvcgo+ID4gPiA+ID4g
PiAvYmluL2xzOiBpc28zOiBJbnB1dC9vdXRwdXQgZXJyb3IKPiA+ID4gPiA+ID4gdG90YWwgMTAx
MjgKPiA+ID4gPiA+ID4gZHJ3eHJ3eHJ3dCAgIDE4IHJvb3Qgcm9vdCAgMjcwMzM2IE9jdCAyNiAw
ODo0NSAuCj4gPiA+ID4gPiA+IGRyd3hyd3hyd3QgIDE4NiByb290IHJvb3QgICAyMDc2MCBPY3Qg
MjcgMTc6NDUgLi4KPiA+ID4gPiA+ID4gZHJ3eHIteHIteCAgICAyIHJvb3Qgcm9vdCAgIDE2Mzg0
IEphbiAgMSAgMTk3MCBpc28xCj4gPiA+ID4gPiA+IGRyd3hyLXhyLXggICAgMiByb290IHJvb3Qg
ICAxNjM4NCBKYW4gIDEgIDE5NzAgaXNvMgo+ID4gPiA+ID4gPiBkcnd4ci14ci14ICAgIDIgcm9v
dCByb290ICAgMTYzODQgSmFuICAxICAxOTcwIGlzbzMKPiA+ID4gPiA+ID4gCj4gPiA+ID4gPiA+
IHZtaG9zdDovbW50L2lzbzEgIyBscwo+ID4gPiA+ID4gPiAvYmluL2xzOiAuOiBTdGFsZSBORlMg
ZmlsZSBoYW5kbGUKPiA+ID4gPiA+ID4gdm1ob3N0Oi9tbnQvaXNvMSAjIGxzIC1sYQo+ID4gPiA+
ID4gPiAvYmluL2xzOiAuOiBJbnB1dC9vdXRwdXQgZXJyb3IKPiA+ID4gPiA+IAo+ID4gPiA+ID4g
SXQgaXMgYSBsaXR0bGUgb2RkIHRoYXQgdGhlIGVycm9ycyBhcmUgaW5jb25zaXN0ZW50Lgo+ID4g
PiA+ID4gCj4gPiA+ID4gPiBDYW4geW91IGZpbmQgYW55IGxvZyBtZXNzYWdlcyBmcm9tIG1vdW50
ZCBpbiBzeXNsb2c/ICBXaGF0IGRvIHRoZXkKPiA+ID4gPiA+IHNheT8KPiA+ID4gPiA+IEFsc28g
d2hhdCBkb2VzCj4gPiA+ID4gPiAgICBjYXQgL3Byb2MvZnMvbmZzZC9leHBvcnRzCj4gPiA+ID4g
PiAKPiA+ID4gPiA+IG9uIHRoZSBzZXJ2ZXIgc2hvdy4KPiA+ID4gPiA+IAo+ID4gPiA+ID4gRmlu
YWxseSwgYSB0Y3BkdW1wOgo+ID4gPiA+ID4gCj4gPiA+ID4gPiAgIHRjcGR1bXAgLXMgMCAtdyAv
dG1wL3RjcGR1bXAgcG9ydCAyMDQ5Cj4gPiA+ID4gPiAKPiA+ID4gPiA+IHdoaWxlIHlvdSBydW4g
dGhlIGV4cGVyaW1lbnQgbWlnaHQgaGVscC4KPiA+ID4gPiA+IAo+ID4gPiA+ID4gPiAKPiA+ID4g
PiA+ID4gaWBtIHVuc3VyZSBpZiBpIHNob3VsZCBibGFtZSBzdXNlIGhlcmUgKGl0YHMgYW4gb3Bl
bnN1c2UgMTAuMyBib3ggd2hpY2ggc2VlbXMgdG8gaGF2ZSBuZnMtdXRpbHMgMS4xLjApCj4gPiA+
ID4gPiA+IAo+ID4gPiA+ID4gPiBkb2VzIHNvbWVib2R5IGhhdmUgc3VjaCBzZXR1cCB1cCBhbmQg
cnVubmluZyBhbmQgY2FuIHRlbGwgaGlzIGRpc3RybyAvIGtlcm5lbCBhbmQgbmZzLXV0aWxzIHZl
cnNpb24gPwo+ID4gPiA+ID4gPiBtYXliZSBpIGNoYW5nZSBkaXN0cm8gdGhlbi4KPiA+ID4gPiA+
IAo+ID4gPiA+ID4gSSBkb3VidCB0aGF0IGl0IGlzIGEgZGlzdHJvLXNwZWNpZmljIHRoaW5nLiAg
QXMgbG9uZyBhcyB5b3UgaGF2ZQo+ID4gPiA+ID4gbmZzLXV0aWxzLTEuMS4wIGl0IHNob3VsZCB3
b3JrLiAgSSBkb24ndCBoYXZlIGEgMTAuMyBib3gKPiA+ID4gPiA+IHNldCB1cCB5ZXQsIGJ1dCBp
dCB3b3JrcyBmaW5lIG9uIERlYmlhbi91bnN0YWJsZSBmb3IgbWUuCj4gPiA+ID4gPiAKPiA+ID4g
PiA+IE1heWJlIHRyeSBhZGRpbmcgdGhlICJub19yb290X3NxdWFzaCIgZXhwb3J0IG9wdGlvbi4K
PiA+ID4gPiA+IFdoYXQgZG9lcyAibHMgLWwgL2V4cG9ydCIgb24gdGhlIHNlcnZlciBzaG93Pwo+
ID4gPiA+ID4gCj4gPiA+ID4gPiBOZWlsQnJvd24KPiA+ID4gPiA+IAo+ID4gPiA+IAo+ID4gPiA+
IAo+ID4gPiAKPiA+ID4gCj4gPiA+IF9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwo+ID4gPiBEZXIgV0VCLkRFIFNtYXJ0
U3VyZmVyIGhpbGZ0IGJpcyB6dSA3MCUgSWhyZXIgT25saW5la29zdGVuIHp1IHNwYXJlbiEKPiA+
ID4gaHR0cDovL3NtYXJ0c3VyZmVyLndlYi5kZS8/bWM9MTAwMDcxJmRpc3RyaWJ1dGlvbmlkPTAw
MDAwMDAwMDA2Ngo+ID4gPiAKPiA+ID4gCj4gPiA+IC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KPiA+ID4gVGhp
cyBTRi5uZXQgZW1haWwgaXMgc3BvbnNvcmVkIGJ5OiBTcGx1bmsgSW5jLgo+ID4gPiBTdGlsbCBn
cmVwcGluZyB0aHJvdWdoIGxvZyBmaWxlcyB0byBmaW5kIHByb2JsZW1zPyAgU3RvcC4KPiA+ID4g
Tm93IFNlYXJjaCBsb2cgZXZlbnRzIGFuZCBjb25maWd1cmF0aW9uIGZpbGVzIHVzaW5nIEFKQVgg
YW5kIGEgYnJvd3Nlci4KPiA+ID4gRG93bmxvYWQgeW91ciBGUkVFIGNvcHkgb2YgU3BsdW5rIG5v
dyA+PiBodHRwOi8vZ2V0LnNwbHVuay5jb20vCj4gPiA+IF9fX19fX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fCj4gPiA+IE5GUyBtYWlsbGlzdCAgLSAgTkZTQGxpc3Rz
LnNvdXJjZWZvcmdlLm5ldAo+ID4gPiBodHRwczovL2xpc3RzLnNvdXJjZWZvcmdlLm5ldC9saXN0
cy9saXN0aW5mby9uZnMKPiA+IAo+IAo+IAo+IF9fX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCj4gRXJ3ZWl0ZXJu
IFNpZSBGcmVlTWFpbCB6dSBlaW5lbSBub2NoIGxlaXN0dW5nc3N0w6Rya2VyZW4gRS1NYWlsLVBv
c3RmYWNoIQkJCj4gTWVociBJbmZvcyB1bnRlciBodHRwOi8vcHJvZHVrdGUud2ViLmRlL2NsdWIv
P21jPTAyMTEzMQo+IAoKLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQpUaGlzIFNGLm5ldCBlbWFpbCBpcyBzcG9u
c29yZWQgYnk6IFNwbHVuayBJbmMuClN0aWxsIGdyZXBwaW5nIHRocm91Z2ggbG9nIGZpbGVzIHRv
IGZpbmQgcHJvYmxlbXM/ICBTdG9wLgpOb3cgU2VhcmNoIGxvZyBldmVudHMgYW5kIGNvbmZpZ3Vy
YXRpb24gZmlsZXMgdXNpbmcgQUpBWCBhbmQgYSBicm93c2VyLgpEb3dubG9hZCB5b3VyIEZSRUUg
Y29weSBvZiBTcGx1bmsgbm93ID4+IGh0dHA6Ly9nZXQuc3BsdW5rLmNvbS8KX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KTkZTIG1haWxsaXN0ICAtICBORlNA
bGlzdHMuc291cmNlZm9yZ2UubmV0Cmh0dHBzOi8vbGlzdHMuc291cmNlZm9yZ2UubmV0L2xpc3Rz
L2xpc3RpbmZvL25mcwo=
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: stale nfs file handle with exported loopback mounts
@ 2007-10-31 20:46 devzero
2007-10-31 20:57 ` J. Bruce Fields
0 siblings, 1 reply; 17+ messages in thread
From: devzero @ 2007-10-31 20:46 UTC (permalink / raw)
To: Neil Brown; +Cc: NFS
hi !
i tried latest grml build (lenny/sid) as server today (suse 9.3 as client).
error still existing, but a little bit different:
if i cd to the loopback-mount dir`s on the client, i see the contents of th=
e parent directory, i.e. i get a recursion.
regards
roland
> -----Urspr=FCngliche Nachricht-----
> Von: devzero@web.de
> Gesendet: 30.10.07 21:05:56
> An: Neil Brown <neilb@suse.de>
> CC: NFS@lists.sourceforge.net
> Betreff: Re: [NFS] stale nfs file handle with exported loopback mounts
> =
> > I recommend replacing subtree_check with no_subtree_check, but it
> > shouldn't make an important difference in this case.
> =
> ok, i leave it as is.
> =
> > This should work with nfs-utils 1.1.0 or later. With earlier releases
> > you need to explicitly export the subordinate filesystems too.
> > =
> =
> mhh - opensuse doesn`t have nfs-utils package, but it has nfs-client-1.1.=
0-8 which looks like they repackaged nfs-utils 1.1.0
> =
> > It is a little odd that the errors are inconsistent.
> =
> ok, but only a minor issue, if an issue at all, isn`t it ?
> =
> > Can you find any log messages from mountd in syslog? What do they
> > say?
> =
> yes, on the client i`m getting :
> =
> Jun 3 21:36:01 linux kernel: nfs_update_inode: inode number mismatch
> Jun 3 21:36:01 linux kernel: expected (0:11/0x2), got (0:11/0x13881)
> Jun 3 21:36:01 linux kernel: nfs_update_inode: inode number mismatch
> Jun 3 21:36:01 linux kernel: expected (0:11/0x2), got (0:11/0x13881)
> Jun 3 21:36:17 linux kernel: nfs_update_inode: inode number mismatch
> Jun 3 21:36:17 linux kernel: expected (0:11/0x2), got (0:11/0x13881)
> Jun 3 21:36:17 linux kernel: nfs_update_inode: inode number mismatch
> Jun 3 21:36:17 linux kernel: expected (0:11/0x2), got (0:11/0x13881)
> Jun 3 21:36:20 linux kernel: nfs_update_inode: inode number mismatch
> Jun 3 21:36:20 linux kernel: expected (0:11/0x2), got (0:11/0x13881)
> Jun 3 21:36:20 linux kernel: nfs_update_inode: inode number mismatch
> =
> =
> no error message on the server:
> Oct 26 10:09:31 opensuse103 mountd[4293]: authenticated unmount request f=
rom 10.0.0.40:1014 for /mnt (/mnt)
> Oct 26 10:10:07 opensuse103 mountd[4293]: authenticated mount request fro=
m 10.0.0.40:612 for /mnt (/mnt)
> Oct 26 10:10:43 opensuse103 mountd[4293]: authenticated unmount request f=
rom 10.0.0.40:623 for /mnt (/mnt)
> Oct 26 10:10:52 opensuse103 mountd[4293]: authenticated mount request fro=
m 10.0.0.40:624 for /mnt (/mnt)
> =
> > Also what does
> > cat /proc/fs/nfsd/exports
> > =
> > on the server show.
> =
> opensuse103:~ # cat /proc/fs/nfsd/exports
> # Version 1.1
> # Path Client(Flags) # IPs
> /mnt/iso1 *(ro,no_root_squash,sync,wdelay,crossmnt,uuid=3D2c49fef2:=
ba464293:9b2bf2b8:322ccbcb)
> /mnt/iso3 *(ro,no_root_squash,sync,wdelay,crossmnt,uuid=3D27ae9c67:=
0b794b36:8b5e9e17:37b569eb)
> /mnt *(ro,no_root_squash,sync,wdelay,crossmnt,uuid=3D08164ee4:2db141eb=
:ac961701:49c74396)
> /mnt/iso2 *(ro,no_root_squash,sync,wdelay,crossmnt,uuid=3D2aad6ea5:=
a05d4441:b94c48e6:e5d9981e)
> =
> =
> > Finally, a tcpdump:
> > =
> > tcpdump -s 0 -w /tmp/tcpdump port 2049
> > =
> > while you run the experiment might help.
> =
> ah - this seems to give a hint, but i don`t have a clue why the server (1=
0.0.0.30) is telling the client (10.0.0.40) a "RPC Version mismatch".
> I also tried --no-nfs-version 4 for rpc.mountd (setting in /etc/sysconfi=
g/nfs), but this didn`t make a difference.
> =
> here is the tcpdump output - i did =
> =
> - mount
> - ls / ls -la / cd to subdirs
> =
> 10:15:06.480542 IP 10.0.0.40.0 > 10.0.0.30.2049: 0 null
> 10:15:06.480572 IP 10.0.0.30.2049 > 10.0.0.40.0: reply ERR 0: RPC Version=
mismatch (167772160-0)
> 10:15:06.480765 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 2031114913 win =
1460 <nop,nop,timestamp 686128 737918>
> 10:15:06.480821 IP 10.0.0.40.2079804678 > 10.0.0.30.2049: 108 fsinfo fh U=
nknown/01000700813801000000000008164EE42DB141EBAC96170149C743963CD81908
> 10:15:06.480833 IP 10.0.0.30.2049 > 10.0.0.40.1022: . ack 108 win 181 <no=
p,nop,timestamp 737918 686128>
> 10:15:06.513365 IP 10.0.0.30.2049 > 10.0.0.40.2079804678: reply ok 84 fsi=
nfo rtmax 65536 rtpref 65536 wtmax 65536 wtpref 65536 dtpref 4096
> 10:15:06.514408 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 85 win 1460 <no=
p,nop,timestamp 686160 737926>
> 10:15:06.514902 IP 10.0.0.40.2096581894 > 10.0.0.30.2049: 108 getattr fh =
Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743963CD81908
> 10:15:06.515256 IP 10.0.0.30.2049 > 10.0.0.40.2096581894: reply ok 116 ge=
tattr DIR 40755 ids 0/0 sz 4096
> 10:15:06.553865 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 201 win 1460 <n=
op,nop,timestamp 686201 737926>
> 10:16:03.091784 IP 10.0.0.40.2113359110 > 10.0.0.30.2049: 108 getattr fh =
Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439647219C8C
> 10:16:03.093366 IP 10.0.0.30.2049 > 10.0.0.40.2113359110: reply ok 116 ge=
tattr DIR 40755 ids 0/0 sz 4096
> 10:16:03.096046 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 317 win 1460 <n=
op,nop,timestamp 743610 752071>
> 10:16:03.097370 IP 10.0.0.40.2130136326 > 10.0.0.30.2049: 112 access fh U=
nknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000001F 001f
> 10:16:03.098156 IP 10.0.0.30.2049 > 10.0.0.40.2130136326: reply ok 124 ac=
cess c 0003
> 10:16:03.156812 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 441 win 1460 <n=
op,nop,timestamp 743651 752072>
> 10:16:08.967804 IP 10.0.0.40.2146913542 > 10.0.0.30.2049: 108 getattr fh =
Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439600000000
> 10:16:08.968305 IP 10.0.0.30.2049 > 10.0.0.40.2146913542: reply ok 116 ge=
tattr DIR 40755 ids 0/0 sz 4096
> 10:16:08.974285 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 557 win 1460 <n=
op,nop,timestamp 749477 753540>
> 10:16:08.975739 IP 10.0.0.40.2163690758 > 10.0.0.30.2049: 132 readdirplus=
fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000000=
0 512 bytes @ 0
> 10:16:08.975985 IP 10.0.0.30.2049 > 10.0.0.40.2163690758: reply ok 1448 r=
eaddirplus
> 10:16:08.976510 IP 10.0.0.30.2049 > 10.0.0.40.1684108288: reply Unknown r=
pc response code=3D2021855861 340
> 10:16:08.982238 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 2345 win 2184 <=
nop,nop,timestamp 749479 753541>
> 10:16:08.984415 IP 10.0.0.40.2180467974 > 10.0.0.30.2049: 116 lookup fh U=
nknown/01000700813801000000000008164EE42DB141EBAC96170149C7439600000004 "is=
o3"
> 10:16:08.984931 IP 10.0.0.30.2049 > 10.0.0.40.2180467974: reply ok 232 lo=
okup fh Unknown/0100060027AE9C670B794B368B5E9E1737B569EB0000000100000002000=
041ED
> 10:16:09.008819 IP 10.0.0.40.2197245190 > 10.0.0.30.2049: 104 getattr fh =
Unknown/0100060027AE9C670B794B368B5E9E1737B569EB0000000FDD22CC3C7002F0AC
> 10:16:09.010946 IP 10.0.0.30.2049 > 10.0.0.40.2197245190: reply ok 188 ge=
tattr REG 2 ids 5/0 sz 0
> 10:16:09.032541 IP 10.0.0.40.2214022406 > 10.0.0.30.2049: 128 getattr fh =
Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E66D0700
> 10:16:09.033405 IP 10.0.0.30.2049 > 10.0.0.40.2214022406: reply ok 188 ge=
tattr REG 1 ids 1/0 sz 0
> 10:16:09.033490 IP 10.0.0.40.2230799622 > 10.0.0.30.2049: 128 getattr fh =
Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E56D0700
> 10:16:09.036811 IP 10.0.0.30.2049 > 10.0.0.40.2230799622: reply ok 188 ge=
tattr REG 1 ids 1/0 sz 0
> 10:16:09.037823 IP 10.0.0.40.2247576838 > 10.0.0.30.2049: 108 getattr fh =
Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439600000000
> 10:16:09.039817 IP 10.0.0.30.2049 > 10.0.0.40.2247576838: reply ok 116 ge=
tattr DIR 40755 ids 0/0 sz 4096
> 10:16:09.040423 IP 10.0.0.40.2264354054 > 10.0.0.30.2049: 112 getattr fh =
Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000000F
> 10:16:09.040856 IP 10.0.0.30.2049 > 10.0.0.40.2264354054: reply ok 188 ge=
tattr REG 2 ids 5/0 sz 0
> 10:16:09.041590 IP 10.0.0.40.2281131270 > 10.0.0.30.2049: 116 lookup fh U=
nknown/01000700813801000000000008164EE42DB141EBAC96170149C7439600000004 "is=
o2"
> 10:16:09.041920 IP 10.0.0.30.2049 > 10.0.0.40.2281131270: reply ok 232 lo=
okup fh Unknown/010006002AAD6EA5A05D4441B94C48E6E5D9981E0000000100000002000=
041ED
> 10:16:09.049633 IP 10.0.0.40.2297908486 > 10.0.0.30.2049: 104 getattr fh =
Unknown/010006002AAD6EA5A05D4441B94C48E6E5D9981E0000000F5FDC84454326E193
> 10:16:09.049781 IP 10.0.0.30.2049 > 10.0.0.40.2297908486: reply ok 188 ge=
tattr REG 2 ids 5/0 sz 0
> 10:16:09.063218 IP 10.0.0.40.2314685702 > 10.0.0.30.2049: 128 getattr fh =
Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E46D0700
> 10:16:09.072505 IP 10.0.0.30.2049 > 10.0.0.40.2314685702: reply ok 188 ge=
tattr REG 1 ids 1/0 sz 0
> 10:16:09.091698 IP 10.0.0.40.2331462918 > 10.0.0.30.2049: 124 getattr fh =
Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E46D0700
> 10:16:09.092022 IP 10.0.0.30.2049 > 10.0.0.40.2331462918: reply ok 116 ge=
tattr REG 100644 ids 0/0 sz 1048576
> 10:16:09.128971 IP 10.0.0.40.2348240134 > 10.0.0.30.2049: 128 getattr fh =
Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E36D0700
> 10:16:09.129304 IP 10.0.0.30.2049 > 10.0.0.40.2348240134: reply ok 188 ge=
tattr REG 1 ids 1/0 sz 0
> 10:16:09.184155 IP 10.0.0.40.2365017350 > 10.0.0.30.2049: 124 getattr fh =
Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E36D0700
> 10:16:09.184582 IP 10.0.0.30.2049 > 10.0.0.40.2365017350: reply ok 116 ge=
tattr REG 100644 ids 0/0 sz 1048576
> 10:16:09.189234 IP 10.0.0.40.2381794566 > 10.0.0.30.2049: 116 lookup fh U=
nknown/01000700813801000000000008164EE42DB141EBAC96170149C7439600000004 "is=
o1"
> 10:16:09.189435 IP 10.0.0.30.2049 > 10.0.0.40.2381794566: reply ok 232 lo=
okup fh Unknown/010006002C49FEF2BA4642939B2BF2B8322CCBCB0000000100000002000=
041ED
> 10:16:09.193476 IP 10.0.0.40.2398571782 > 10.0.0.30.2049: 104 getattr fh =
Unknown/010006002C49FEF2BA4642939B2BF2B8322CCBCB0000000F58896A884A0C62B7
> 10:16:09.193652 IP 10.0.0.30.2049 > 10.0.0.40.2398571782: reply ok 188 ge=
tattr REG 2 ids 5/0 sz 0
> 10:16:09.194937 IP 10.0.0.40.2415348998 > 10.0.0.30.2049: 128 getattr fh =
Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E76D0700
> 10:16:09.195033 IP 10.0.0.30.2049 > 10.0.0.40.2415348998: reply ok 188 ge=
tattr REG 1 ids 1/0 sz 0
> 10:16:09.195230 IP 10.0.0.40.2432126214 > 10.0.0.30.2049: 128 getattr fh =
Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E86D0700
> 10:16:09.323635 IP 10.0.0.30.2049 > 10.0.0.40.1022: . ack 2572 win 416 <n=
op,nop,timestamp 753629 749566>
> 10:16:09.324345 IP 10.0.0.30.2049 > 10.0.0.40.2432126214: reply ok 188 ge=
tattr REG 1 ids 1/0 sz 0
> 10:16:09.341475 IP 10.0.0.40.2448903430 > 10.0.0.30.2049: 128 getattr fh =
Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E26D0700
> 10:16:09.341524 IP 10.0.0.30.2049 > 10.0.0.40.1022: . ack 2700 win 449 <n=
op,nop,timestamp 753632 749707>
> 10:16:09.341928 IP 10.0.0.30.2049 > 10.0.0.40.2448903430: reply ok 188 ge=
tattr REG 1 ids 1/0 sz 0
> 10:16:09.342117 IP 10.0.0.40.2465680646 > 10.0.0.30.2049: 124 getattr fh =
Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E26D0700
> 10:16:09.343404 IP 10.0.0.30.2049 > 10.0.0.40.2465680646: reply ok 116 ge=
tattr REG 100644 ids 0/0 sz 1048576
> 10:16:09.389316 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 5573 win 2184 <=
nop,nop,timestamp 749751 753633>
> 10:16:13.449513 IP 10.0.0.40.2482457862 > 10.0.0.30.2049: 108 getattr fh =
Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439647219CB5
> 10:16:13.449815 IP 10.0.0.30.2049 > 10.0.0.40.2482457862: reply ok 116 ge=
tattr DIR 40755 ids 0/0 sz 4096
> 10:16:13.452344 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 5689 win 2184 <=
nop,nop,timestamp 753943 754660>
> 10:16:13.453973 IP 10.0.0.40.2499235078 > 10.0.0.30.2049: 100 getattr fh =
Unknown/010006002C49FEF2BA4642939B2BF2B8322CCBCB47219C8C0000000047219C8C
> 10:16:13.454154 IP 10.0.0.30.2049 > 10.0.0.40.2499235078: reply ok 116 ge=
tattr DIR 40755 ids 0/0 sz 4096
> 10:16:13.456194 IP 10.0.0.40.2516012294 > 10.0.0.30.2049: 108 getattr fh =
Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439647219C8C
> 10:16:13.456361 IP 10.0.0.30.2049 > 10.0.0.40.2516012294: reply ok 116 ge=
tattr DIR 40755 ids 0/0 sz 4096
> 10:16:13.458282 IP 10.0.0.40.2532789510 > 10.0.0.30.2049: 116 lookup fh U=
nknown/01000700813801000000000008164EE42DB141EBAC96170149C7439600000004 "is=
o1"
> 10:16:13.458461 IP 10.0.0.30.2049 > 10.0.0.40.2532789510: reply ok 232 lo=
okup fh Unknown/010006002C49FEF2BA4642939B2BF2B8322CCBCB0000000100000002000=
041ED
> 10:16:13.510637 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 6153 win 2184 <=
nop,nop,timestamp 753985 754662>
> 10:16:14.030110 IP 10.0.0.40.2549566726 > 10.0.0.30.2049: 112 access fh U=
nknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000001F 001f
> 10:16:14.030927 IP 10.0.0.30.2049 > 10.0.0.40.2549566726: reply ok 124 ac=
cess c 0003
> 10:16:14.033436 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 6277 win 2184 <=
nop,nop,timestamp 754548 754805>
> 10:16:14.034732 IP 10.0.0.40.2566343942 > 10.0.0.30.2049: 108 getattr fh =
Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439600000000
> 10:16:14.034980 IP 10.0.0.30.2049 > 10.0.0.40.2566343942: reply ok 116 ge=
tattr DIR 40755 ids 0/0 sz 4096
> 10:16:14.037319 IP 10.0.0.40.2583121158 > 10.0.0.30.2049: 112 access fh U=
nknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000001F 001f
> 10:16:14.037486 IP 10.0.0.30.2049 > 10.0.0.40.2583121158: reply ok 124 ac=
cess c 0003
> 10:16:14.040323 IP 10.0.0.40.2599898374 > 10.0.0.30.2049: 132 readdirplus=
fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000000=
0 512 bytes @ 0
> 10:16:14.040554 IP 10.0.0.30.2049 > 10.0.0.40.2599898374: reply ok 1448 r=
eaddirplus
> 10:16:14.041020 IP 10.0.0.30.2049 > 10.0.0.40.1684108288: reply Unknown r=
pc response code=3D2021855861 340
> 10:16:14.043583 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 8305 win 2908 <=
nop,nop,timestamp 754550 754808>
> 10:16:14.045104 IP 10.0.0.40.2616675590 > 10.0.0.30.2049: 112 access fh U=
nknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000001F 001f
> 10:16:14.045402 IP 10.0.0.30.2049 > 10.0.0.40.2616675590: reply ok 124 ac=
cess c 0003
> 10:16:14.047830 IP 10.0.0.40.2633452806 > 10.0.0.30.2049: 112 access fh U=
nknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000001F 001f
> 10:16:14.048039 IP 10.0.0.30.2049 > 10.0.0.40.2633452806: reply ok 124 ac=
cess c 0003
> 10:16:14.099385 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 8553 win 2908 <=
nop,nop,timestamp 754592 754810>
> 10:16:14.293714 IP 10.0.0.40.2650230022 > 10.0.0.30.2049: 108 getattr fh =
Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439600000000
> 10:16:14.294019 IP 10.0.0.30.2049 > 10.0.0.40.2650230022: reply ok 116 ge=
tattr DIR 40755 ids 0/0 sz 4096
> 10:16:14.297263 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 8669 win 2908 <=
nop,nop,timestamp 754763 754871>
> 10:16:14.297272 IP 10.0.0.40.2667007238 > 10.0.0.30.2049: 112 access fh U=
nknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000001F 001f
> 10:16:14.297466 IP 10.0.0.30.2049 > 10.0.0.40.2667007238: reply ok 124 ac=
cess c 0003
> 10:16:14.297586 IP 10.0.0.40.2683784454 > 10.0.0.30.2049: 112 access fh U=
nknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000001F 001f
> 10:16:14.297987 IP 10.0.0.30.2049 > 10.0.0.40.2683784454: reply ok 124 ac=
cess c 0003
> 10:16:14.301165 IP 10.0.0.40.2700561670 > 10.0.0.30.2049: 104 access fh U=
nknown/010006002C49FEF2BA4642939B2BF2B8322CCBCB0000001F47219C8C00000000 001f
> 10:16:14.301405 IP 10.0.0.30.2049 > 10.0.0.40.2700561670: reply ok 124 ac=
cess c 0003
> 10:16:14.351229 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 9041 win 2908 <=
nop,nop,timestamp 754810 754873>
> 10:16:14.842633 IP 10.0.0.40.2717338886 > 10.0.0.30.2049: 100 getattr fh =
Unknown/010006002C49FEF2BA4642939B2BF2B8322CCBCB0000000047219C8C00000000
> 10:16:14.851711 IP 10.0.0.30.2049 > 10.0.0.40.2717338886: reply ok 116 ge=
tattr DIR 40755 ids 0/0 sz 4096
> 10:16:14.856846 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 9157 win 2908 <=
nop,nop,timestamp 755272 755008>
> =
> =
> > > does somebody have such setup up and running and can tell his distro =
/ kernel and nfs-utils version ?
> > > maybe i change distro then.
> > =
> > I doubt that it is a distro-specific thing. As long as you have
> > nfs-utils-1.1.0 it should work. I don't have a 10.3 box
> > set up yet, but it works fine on Debian/unstable for me.
> =
> ok, will try this on debian.
> =
> > Maybe try adding the "no_root_squash" export option.
> no difference
> =
> > What does "ls -l /export" on the server show?
> nothing unusual. no errors, just the dirs/mountpoints
> =
> Thanks for your help!
> =
> regards
> roland
> =
> =
> > =
> > On Saturday October 27, devzero@web.de wrote:
> > > Hello !
> > > =
> > > with 2.6.22 i`m trying to export loopback mounted iso-images.
> > > =
> > > this is /etc/exports:
> > > =
> > > /export *(ro,crossmnt,subtree_check)
> > =
> > I recommend replacing subtree_check with no_subtree_check, but it
> > shouldn't make an important difference in this case.
> > =
> > =
> > This should work with nfs-utils 1.1.0 or later. With earlier releases
> > you need to explicitly export the subordinate filesystems too.
> > =
> > > =
> > > in /export, i have loopback mounted iso-images
> > > =
> > > after mounting on the client side under /mnt (tried one older and one=
recent system) , i`m getting:
> > > =
> > > vmhost:/mnt # ls -la
> > > /bin/ls: iso1: Input/output error
> > > /bin/ls: iso2 Input/output error
> > > /bin/ls: iso3: Input/output error
> > > total 10128
> > > drwxrwxrwt 18 root root 270336 Oct 26 08:45 .
> > > drwxrwxrwt 186 root root 20760 Oct 27 17:45 ..
> > > drwxr-xr-x 2 root root 16384 Jan 1 1970 iso1
> > > drwxr-xr-x 2 root root 16384 Jan 1 1970 iso2
> > > drwxr-xr-x 2 root root 16384 Jan 1 1970 iso3
> > > =
> > > vmhost:/mnt/iso1 # ls
> > > /bin/ls: .: Stale NFS file handle
> > > vmhost:/mnt/iso1 # ls -la
> > > /bin/ls: .: Input/output error
> > =
> > It is a little odd that the errors are inconsistent.
> > =
> > Can you find any log messages from mountd in syslog? What do they
> > say?
> > Also what does
> > cat /proc/fs/nfsd/exports
> > =
> > on the server show.
> > =
> > Finally, a tcpdump:
> > =
> > tcpdump -s 0 -w /tmp/tcpdump port 2049
> > =
> > while you run the experiment might help.
> > =
> > > =
> > > i`m unsure if i should blame suse here (it`s an opensuse 10.3 box whi=
ch seems to have nfs-utils 1.1.0)
> > > =
> > > does somebody have such setup up and running and can tell his distro =
/ kernel and nfs-utils version ?
> > > maybe i change distro then.
> > =
> > I doubt that it is a distro-specific thing. As long as you have
> > nfs-utils-1.1.0 it should work. I don't have a 10.3 box
> > set up yet, but it works fine on Debian/unstable for me.
> > =
> > Maybe try adding the "no_root_squash" export option.
> > What does "ls -l /export" on the server show?
> > =
> > NeilBrown
> > =
> =
> =
_____________________________________________________________________
Der WEB.DE SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen!
http://smartsurfer.web.de/?mc=3D100071&distributionid=3D000000000066
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: stale nfs file handle with exported loopback mounts
2007-10-31 20:46 devzero
@ 2007-10-31 20:57 ` J. Bruce Fields
0 siblings, 0 replies; 17+ messages in thread
From: J. Bruce Fields @ 2007-10-31 20:57 UTC (permalink / raw)
To: devzero; +Cc: Neil Brown, NFS
T24gV2VkLCBPY3QgMzEsIDIwMDcgYXQgMDk6NDY6MTNQTSArMDEwMCwgZGV2emVyb0B3ZWIuZGUg
d3JvdGU6Cj4gaGkgIQo+IAo+IGkgdHJpZWQgbGF0ZXN0IGdybWwgYnVpbGQgKGxlbm55L3NpZCkg
YXMgc2VydmVyIHRvZGF5IChzdXNlIDkuMyBhcyBjbGllbnQpLgo+IAo+IGVycm9yIHN0aWxsIGV4
aXN0aW5nLCBidXQgYSBsaXR0bGUgYml0IGRpZmZlcmVudDoKPiAKPiBpZiBpIGNkIHRvIHRoZSBs
b29wYmFjay1tb3VudCBkaXJgcyBvbiB0aGUgY2xpZW50LCBpIHNlZSB0aGUgY29udGVudHMgb2Yg
dGhlIHBhcmVudCBkaXJlY3RvcnksIGkuZS4gaSBnZXQgYSByZWN1cnNpb24uCgpUaGF0IHN1Z2dl
c3RzIHNvbWV0aGluZyBmdW5reSBpbiB0aGUgd2F5IHdlJ3JlIGlkZW50aWZ5aW5nIHRob3NlCmZp
bGVzeXN0ZW1zIGluIHRoZSBmaWxlaGFuZGxlLiAgSWYgeW91IGFkZCBhbiBleHBsaWNpdCBleHBv
cnQgZm9yIGVhY2gKb25lLCBlYWNoIHdpdGggaXRzIG93biAiZnNpZD14eXoiIG9wdGlvbiAod2l0
aCB4eXogd2hhdGV2ZXIgcG9zaXRpdmUKaW50ZWdlciB5b3UnZCBsaWtlLCBhcyBsb25nIGFzIGl0
J3MgZGlmZmljdWx0IGZvciBleHBvcnQpLCBkb2VzIHRoZQpwcm9ibGVtIGdvIGF3YXk/PwoKLS1i
LgoKPiAKPiByZWdhcmRzCj4gcm9sYW5kCj4gCj4gCj4gCj4gPiAtLS0tLVVyc3Byw7xuZ2xpY2hl
IE5hY2hyaWNodC0tLS0tCj4gPiBWb246IGRldnplcm9Ad2ViLmRlCj4gPiBHZXNlbmRldDogMzAu
MTAuMDcgMjE6MDU6NTYKPiA+IEFuOiBOZWlsIEJyb3duIDxuZWlsYkBzdXNlLmRlPgo+ID4gQ0M6
IE5GU0BsaXN0cy5zb3VyY2Vmb3JnZS5uZXQKPiA+IEJldHJlZmY6IFJlOiBbTkZTXSBzdGFsZSBu
ZnMgZmlsZSBoYW5kbGUgd2l0aCBleHBvcnRlZCBsb29wYmFjayBtb3VudHMKPiAKPiAKPiA+IAo+
ID4gPiBJIHJlY29tbWVuZCByZXBsYWNpbmcgc3VidHJlZV9jaGVjayB3aXRoIG5vX3N1YnRyZWVf
Y2hlY2ssIGJ1dCBpdAo+ID4gPiBzaG91bGRuJ3QgbWFrZSBhbiBpbXBvcnRhbnQgZGlmZmVyZW5j
ZSBpbiB0aGlzIGNhc2UuCj4gPiAKPiA+IG9rLCBpIGxlYXZlIGl0IGFzIGlzLgo+ID4gCj4gPiA+
IFRoaXMgc2hvdWxkIHdvcmsgd2l0aCBuZnMtdXRpbHMgMS4xLjAgb3IgbGF0ZXIuICBXaXRoIGVh
cmxpZXIgcmVsZWFzZXMKPiA+ID4geW91IG5lZWQgdG8gZXhwbGljaXRseSBleHBvcnQgdGhlIHN1
Ym9yZGluYXRlIGZpbGVzeXN0ZW1zIHRvby4KPiA+ID4gCj4gPiAKPiA+IG1oaCAtIG9wZW5zdXNl
IGRvZXNuYHQgaGF2ZSBuZnMtdXRpbHMgcGFja2FnZSwgYnV0IGl0IGhhcyBuZnMtY2xpZW50LTEu
MS4wLTggd2hpY2ggbG9va3MgbGlrZSB0aGV5IHJlcGFja2FnZWQgbmZzLXV0aWxzIDEuMS4wCj4g
PiAKPiA+ID4gSXQgaXMgYSBsaXR0bGUgb2RkIHRoYXQgdGhlIGVycm9ycyBhcmUgaW5jb25zaXN0
ZW50Lgo+ID4gCj4gPiBvaywgYnV0IG9ubHkgYSBtaW5vciBpc3N1ZSwgaWYgYW4gaXNzdWUgYXQg
YWxsLCBpc25gdCBpdCA/Cj4gPiAKPiA+ID4gQ2FuIHlvdSBmaW5kIGFueSBsb2cgbWVzc2FnZXMg
ZnJvbSBtb3VudGQgaW4gc3lzbG9nPyAgV2hhdCBkbyB0aGV5Cj4gPiA+IHNheT8KPiA+IAo+ID4g
eWVzLCBvbiB0aGUgY2xpZW50IGlgbSBnZXR0aW5nIDoKPiA+IAo+ID4gSnVuICAzIDIxOjM2OjAx
IGxpbnV4IGtlcm5lbDogbmZzX3VwZGF0ZV9pbm9kZTogaW5vZGUgbnVtYmVyIG1pc21hdGNoCj4g
PiBKdW4gIDMgMjE6MzY6MDEgbGludXgga2VybmVsOiBleHBlY3RlZCAoMDoxMS8weDIpLCBnb3Qg
KDA6MTEvMHgxMzg4MSkKPiA+IEp1biAgMyAyMTozNjowMSBsaW51eCBrZXJuZWw6IG5mc191cGRh
dGVfaW5vZGU6IGlub2RlIG51bWJlciBtaXNtYXRjaAo+ID4gSnVuICAzIDIxOjM2OjAxIGxpbnV4
IGtlcm5lbDogZXhwZWN0ZWQgKDA6MTEvMHgyKSwgZ290ICgwOjExLzB4MTM4ODEpCj4gPiBKdW4g
IDMgMjE6MzY6MTcgbGludXgga2VybmVsOiBuZnNfdXBkYXRlX2lub2RlOiBpbm9kZSBudW1iZXIg
bWlzbWF0Y2gKPiA+IEp1biAgMyAyMTozNjoxNyBsaW51eCBrZXJuZWw6IGV4cGVjdGVkICgwOjEx
LzB4MiksIGdvdCAoMDoxMS8weDEzODgxKQo+ID4gSnVuICAzIDIxOjM2OjE3IGxpbnV4IGtlcm5l
bDogbmZzX3VwZGF0ZV9pbm9kZTogaW5vZGUgbnVtYmVyIG1pc21hdGNoCj4gPiBKdW4gIDMgMjE6
MzY6MTcgbGludXgga2VybmVsOiBleHBlY3RlZCAoMDoxMS8weDIpLCBnb3QgKDA6MTEvMHgxMzg4
MSkKPiA+IEp1biAgMyAyMTozNjoyMCBsaW51eCBrZXJuZWw6IG5mc191cGRhdGVfaW5vZGU6IGlu
b2RlIG51bWJlciBtaXNtYXRjaAo+ID4gSnVuICAzIDIxOjM2OjIwIGxpbnV4IGtlcm5lbDogZXhw
ZWN0ZWQgKDA6MTEvMHgyKSwgZ290ICgwOjExLzB4MTM4ODEpCj4gPiBKdW4gIDMgMjE6MzY6MjAg
bGludXgga2VybmVsOiBuZnNfdXBkYXRlX2lub2RlOiBpbm9kZSBudW1iZXIgbWlzbWF0Y2gKPiA+
IAo+ID4gCj4gPiBubyBlcnJvciBtZXNzYWdlIG9uIHRoZSBzZXJ2ZXI6Cj4gPiBPY3QgMjYgMTA6
MDk6MzEgb3BlbnN1c2UxMDMgbW91bnRkWzQyOTNdOiBhdXRoZW50aWNhdGVkIHVubW91bnQgcmVx
dWVzdCBmcm9tIDEwLjAuMC40MDoxMDE0IGZvciAvbW50ICgvbW50KQo+ID4gT2N0IDI2IDEwOjEw
OjA3IG9wZW5zdXNlMTAzIG1vdW50ZFs0MjkzXTogYXV0aGVudGljYXRlZCBtb3VudCByZXF1ZXN0
IGZyb20gMTAuMC4wLjQwOjYxMiBmb3IgL21udCAoL21udCkKPiA+IE9jdCAyNiAxMDoxMDo0MyBv
cGVuc3VzZTEwMyBtb3VudGRbNDI5M106IGF1dGhlbnRpY2F0ZWQgdW5tb3VudCByZXF1ZXN0IGZy
b20gMTAuMC4wLjQwOjYyMyBmb3IgL21udCAoL21udCkKPiA+IE9jdCAyNiAxMDoxMDo1MiBvcGVu
c3VzZTEwMyBtb3VudGRbNDI5M106IGF1dGhlbnRpY2F0ZWQgbW91bnQgcmVxdWVzdCBmcm9tIDEw
LjAuMC40MDo2MjQgZm9yIC9tbnQgKC9tbnQpCj4gPiAKPiA+ID4gQWxzbyB3aGF0IGRvZXMKPiA+
ID4gICAgY2F0IC9wcm9jL2ZzL25mc2QvZXhwb3J0cwo+ID4gPiAKPiA+ID4gb24gdGhlIHNlcnZl
ciBzaG93Lgo+ID4gCj4gPiBvcGVuc3VzZTEwMzp+ICMgY2F0IC9wcm9jL2ZzL25mc2QvZXhwb3J0
cwo+ID4gIyBWZXJzaW9uIDEuMQo+ID4gIyBQYXRoIENsaWVudChGbGFncykgIyBJUHMKPiA+IC9t
bnQvaXNvMSAgICAgICAqKHJvLG5vX3Jvb3Rfc3F1YXNoLHN5bmMsd2RlbGF5LGNyb3NzbW50LHV1
aWQ9MmM0OWZlZjI6YmE0NjQyOTM6OWIyYmYyYjg6MzIyY2NiY2IpCj4gPiAvbW50L2lzbzMgICAg
ICAgKihybyxub19yb290X3NxdWFzaCxzeW5jLHdkZWxheSxjcm9zc21udCx1dWlkPTI3YWU5YzY3
OjBiNzk0YjM2OjhiNWU5ZTE3OjM3YjU2OWViKQo+ID4gL21udCAgICAqKHJvLG5vX3Jvb3Rfc3F1
YXNoLHN5bmMsd2RlbGF5LGNyb3NzbW50LHV1aWQ9MDgxNjRlZTQ6MmRiMTQxZWI6YWM5NjE3MDE6
NDljNzQzOTYpCj4gPiAvbW50L2lzbzIgICAgICAgKihybyxub19yb290X3NxdWFzaCxzeW5jLHdk
ZWxheSxjcm9zc21udCx1dWlkPTJhYWQ2ZWE1OmEwNWQ0NDQxOmI5NGM0OGU2OmU1ZDk5ODFlKQo+
ID4gCj4gPiAKPiA+ID4gRmluYWxseSwgYSB0Y3BkdW1wOgo+ID4gPiAKPiA+ID4gICB0Y3BkdW1w
IC1zIDAgLXcgL3RtcC90Y3BkdW1wIHBvcnQgMjA0OQo+ID4gPiAKPiA+ID4gd2hpbGUgeW91IHJ1
biB0aGUgZXhwZXJpbWVudCBtaWdodCBoZWxwLgo+ID4gCj4gPiBhaCAtIHRoaXMgc2VlbXMgdG8g
Z2l2ZSBhIGhpbnQsIGJ1dCBpIGRvbmB0IGhhdmUgYSBjbHVlIHdoeSB0aGUgc2VydmVyICgxMC4w
LjAuMzApIGlzIHRlbGxpbmcgdGhlIGNsaWVudCAoMTAuMC4wLjQwKSBhICJSUEMgVmVyc2lvbiBt
aXNtYXRjaCIuCj4gPiBJIGFsc28gdHJpZWQgIC0tbm8tbmZzLXZlcnNpb24gNCBmb3IgcnBjLm1v
dW50ZCAoc2V0dGluZyBpbiAvZXRjL3N5c2NvbmZpZy9uZnMpLCBidXQgdGhpcyBkaWRuYHQgbWFr
ZSBhIGRpZmZlcmVuY2UuCj4gPiAKPiA+IGhlcmUgaXMgdGhlIHRjcGR1bXAgb3V0cHV0IC0gaSBk
aWQgCj4gPiAKPiA+IC0gbW91bnQKPiA+IC0gbHMgLyBscyAtbGEgLyBjZCB0byBzdWJkaXJzCj4g
PiAKPiA+IDEwOjE1OjA2LjQ4MDU0MiBJUCAxMC4wLjAuNDAuMCA+IDEwLjAuMC4zMC4yMDQ5OiAw
IG51bGwKPiA+IDEwOjE1OjA2LjQ4MDU3MiBJUCAxMC4wLjAuMzAuMjA0OSA+IDEwLjAuMC40MC4w
OiByZXBseSBFUlIgMDogUlBDIFZlcnNpb24gbWlzbWF0Y2ggKDE2Nzc3MjE2MC0wKQo+ID4gMTA6
MTU6MDYuNDgwNzY1IElQIDEwLjAuMC40MC4xMDIyID4gMTAuMC4wLjMwLjIwNDk6IC4gYWNrIDIw
MzExMTQ5MTMgd2luIDE0NjAgPG5vcCxub3AsdGltZXN0YW1wIDY4NjEyOCA3Mzc5MTg+Cj4gPiAx
MDoxNTowNi40ODA4MjEgSVAgMTAuMC4wLjQwLjIwNzk4MDQ2NzggPiAxMC4wLjAuMzAuMjA0OTog
MTA4IGZzaW5mbyBmaCBVbmtub3duLzAxMDAwNzAwODEzODAxMDAwMDAwMDAwMDA4MTY0RUU0MkRC
MTQxRUJBQzk2MTcwMTQ5Qzc0Mzk2M0NEODE5MDgKPiA+IDEwOjE1OjA2LjQ4MDgzMyBJUCAxMC4w
LjAuMzAuMjA0OSA+IDEwLjAuMC40MC4xMDIyOiAuIGFjayAxMDggd2luIDE4MSA8bm9wLG5vcCx0
aW1lc3RhbXAgNzM3OTE4IDY4NjEyOD4KPiA+IDEwOjE1OjA2LjUxMzM2NSBJUCAxMC4wLjAuMzAu
MjA0OSA+IDEwLjAuMC40MC4yMDc5ODA0Njc4OiByZXBseSBvayA4NCBmc2luZm8gcnRtYXggNjU1
MzYgcnRwcmVmIDY1NTM2IHd0bWF4IDY1NTM2IHd0cHJlZiA2NTUzNiBkdHByZWYgNDA5Ngo+ID4g
MTA6MTU6MDYuNTE0NDA4IElQIDEwLjAuMC40MC4xMDIyID4gMTAuMC4wLjMwLjIwNDk6IC4gYWNr
IDg1IHdpbiAxNDYwIDxub3Asbm9wLHRpbWVzdGFtcCA2ODYxNjAgNzM3OTI2Pgo+ID4gMTA6MTU6
MDYuNTE0OTAyIElQIDEwLjAuMC40MC4yMDk2NTgxODk0ID4gMTAuMC4wLjMwLjIwNDk6IDEwOCBn
ZXRhdHRyIGZoIFVua25vd24vMDEwMDA3MDA4MTM4MDEwMDAwMDAwMDAwMDgxNjRFRTQyREIxNDFF
QkFDOTYxNzAxNDlDNzQzOTYzQ0Q4MTkwOAo+ID4gMTA6MTU6MDYuNTE1MjU2IElQIDEwLjAuMC4z
MC4yMDQ5ID4gMTAuMC4wLjQwLjIwOTY1ODE4OTQ6IHJlcGx5IG9rIDExNiBnZXRhdHRyIERJUiA0
MDc1NSBpZHMgMC8wIHN6IDQwOTYKPiA+IDEwOjE1OjA2LjU1Mzg2NSBJUCAxMC4wLjAuNDAuMTAy
MiA+IDEwLjAuMC4zMC4yMDQ5OiAuIGFjayAyMDEgd2luIDE0NjAgPG5vcCxub3AsdGltZXN0YW1w
IDY4NjIwMSA3Mzc5MjY+Cj4gPiAxMDoxNjowMy4wOTE3ODQgSVAgMTAuMC4wLjQwLjIxMTMzNTkx
MTAgPiAxMC4wLjAuMzAuMjA0OTogMTA4IGdldGF0dHIgZmggVW5rbm93bi8wMTAwMDcwMDgxMzgw
MTAwMDAwMDAwMDAwODE2NEVFNDJEQjE0MUVCQUM5NjE3MDE0OUM3NDM5NjQ3MjE5QzhDCj4gPiAx
MDoxNjowMy4wOTMzNjYgSVAgMTAuMC4wLjMwLjIwNDkgPiAxMC4wLjAuNDAuMjExMzM1OTExMDog
cmVwbHkgb2sgMTE2IGdldGF0dHIgRElSIDQwNzU1IGlkcyAwLzAgc3ogNDA5Ngo+ID4gMTA6MTY6
MDMuMDk2MDQ2IElQIDEwLjAuMC40MC4xMDIyID4gMTAuMC4wLjMwLjIwNDk6IC4gYWNrIDMxNyB3
aW4gMTQ2MCA8bm9wLG5vcCx0aW1lc3RhbXAgNzQzNjEwIDc1MjA3MT4KPiA+IDEwOjE2OjAzLjA5
NzM3MCBJUCAxMC4wLjAuNDAuMjEzMDEzNjMyNiA+IDEwLjAuMC4zMC4yMDQ5OiAxMTIgYWNjZXNz
IGZoIFVua25vd24vMDEwMDA3MDA4MTM4MDEwMDAwMDAwMDAwMDgxNjRFRTQyREIxNDFFQkFDOTYx
NzAxNDlDNzQzOTYwMDAwMDAxRiAwMDFmCj4gPiAxMDoxNjowMy4wOTgxNTYgSVAgMTAuMC4wLjMw
LjIwNDkgPiAxMC4wLjAuNDAuMjEzMDEzNjMyNjogcmVwbHkgb2sgMTI0IGFjY2VzcyBjIDAwMDMK
PiA+IDEwOjE2OjAzLjE1NjgxMiBJUCAxMC4wLjAuNDAuMTAyMiA+IDEwLjAuMC4zMC4yMDQ5OiAu
IGFjayA0NDEgd2luIDE0NjAgPG5vcCxub3AsdGltZXN0YW1wIDc0MzY1MSA3NTIwNzI+Cj4gPiAx
MDoxNjowOC45Njc4MDQgSVAgMTAuMC4wLjQwLjIxNDY5MTM1NDIgPiAxMC4wLjAuMzAuMjA0OTog
MTA4IGdldGF0dHIgZmggVW5rbm93bi8wMTAwMDcwMDgxMzgwMTAwMDAwMDAwMDAwODE2NEVFNDJE
QjE0MUVCQUM5NjE3MDE0OUM3NDM5NjAwMDAwMDAwCj4gPiAxMDoxNjowOC45NjgzMDUgSVAgMTAu
MC4wLjMwLjIwNDkgPiAxMC4wLjAuNDAuMjE0NjkxMzU0MjogcmVwbHkgb2sgMTE2IGdldGF0dHIg
RElSIDQwNzU1IGlkcyAwLzAgc3ogNDA5Ngo+ID4gMTA6MTY6MDguOTc0Mjg1IElQIDEwLjAuMC40
MC4xMDIyID4gMTAuMC4wLjMwLjIwNDk6IC4gYWNrIDU1NyB3aW4gMTQ2MCA8bm9wLG5vcCx0aW1l
c3RhbXAgNzQ5NDc3IDc1MzU0MD4KPiA+IDEwOjE2OjA4Ljk3NTczOSBJUCAxMC4wLjAuNDAuMjE2
MzY5MDc1OCA+IDEwLjAuMC4zMC4yMDQ5OiAxMzIgcmVhZGRpcnBsdXMgZmggVW5rbm93bi8wMTAw
MDcwMDgxMzgwMTAwMDAwMDAwMDAwODE2NEVFNDJEQjE0MUVCQUM5NjE3MDE0OUM3NDM5NjAwMDAw
MDAwIDUxMiBieXRlcyBAIDAKPiA+IDEwOjE2OjA4Ljk3NTk4NSBJUCAxMC4wLjAuMzAuMjA0OSA+
IDEwLjAuMC40MC4yMTYzNjkwNzU4OiByZXBseSBvayAxNDQ4IHJlYWRkaXJwbHVzCj4gPiAxMDox
NjowOC45NzY1MTAgSVAgMTAuMC4wLjMwLjIwNDkgPiAxMC4wLjAuNDAuMTY4NDEwODI4ODogcmVw
bHkgVW5rbm93biBycGMgcmVzcG9uc2UgY29kZT0yMDIxODU1ODYxIDM0MAo+ID4gMTA6MTY6MDgu
OTgyMjM4IElQIDEwLjAuMC40MC4xMDIyID4gMTAuMC4wLjMwLjIwNDk6IC4gYWNrIDIzNDUgd2lu
IDIxODQgPG5vcCxub3AsdGltZXN0YW1wIDc0OTQ3OSA3NTM1NDE+Cj4gPiAxMDoxNjowOC45ODQ0
MTUgSVAgMTAuMC4wLjQwLjIxODA0Njc5NzQgPiAxMC4wLjAuMzAuMjA0OTogMTE2IGxvb2t1cCBm
aCBVbmtub3duLzAxMDAwNzAwODEzODAxMDAwMDAwMDAwMDA4MTY0RUU0MkRCMTQxRUJBQzk2MTcw
MTQ5Qzc0Mzk2MDAwMDAwMDQgImlzbzMiCj4gPiAxMDoxNjowOC45ODQ5MzEgSVAgMTAuMC4wLjMw
LjIwNDkgPiAxMC4wLjAuNDAuMjE4MDQ2Nzk3NDogcmVwbHkgb2sgMjMyIGxvb2t1cCBmaCBVbmtu
b3duLzAxMDAwNjAwMjdBRTlDNjcwQjc5NEIzNjhCNUU5RTE3MzdCNTY5RUIwMDAwMDAwMTAwMDAw
MDAyMDAwMDQxRUQKPiA+IDEwOjE2OjA5LjAwODgxOSBJUCAxMC4wLjAuNDAuMjE5NzI0NTE5MCA+
IDEwLjAuMC4zMC4yMDQ5OiAxMDQgZ2V0YXR0ciBmaCBVbmtub3duLzAxMDAwNjAwMjdBRTlDNjcw
Qjc5NEIzNjhCNUU5RTE3MzdCNTY5RUIwMDAwMDAwRkREMjJDQzNDNzAwMkYwQUMKPiA+IDEwOjE2
OjA5LjAxMDk0NiBJUCAxMC4wLjAuMzAuMjA0OSA+IDEwLjAuMC40MC4yMTk3MjQ1MTkwOiByZXBs
eSBvayAxODggZ2V0YXR0ciBSRUcgMiBpZHMgNS8wIHN6IDAKPiA+IDEwOjE2OjA5LjAzMjU0MSBJ
UCAxMC4wLjAuNDAuMjIxNDAyMjQwNiA+IDEwLjAuMC4zMC4yMDQ5OiAxMjggZ2V0YXR0ciBmaCBV
bmtub3duLzAxMDAwNzAyODEzODAxMDAwMDAwMDAwMDA4MTY0RUU0MkRCMTQxRUJBQzk2MTcwMTQ5
Qzc0Mzk2RTY2RDA3MDAKPiA+IDEwOjE2OjA5LjAzMzQwNSBJUCAxMC4wLjAuMzAuMjA0OSA+IDEw
LjAuMC40MC4yMjE0MDIyNDA2OiByZXBseSBvayAxODggZ2V0YXR0ciBSRUcgMSBpZHMgMS8wIHN6
IDAKPiA+IDEwOjE2OjA5LjAzMzQ5MCBJUCAxMC4wLjAuNDAuMjIzMDc5OTYyMiA+IDEwLjAuMC4z
MC4yMDQ5OiAxMjggZ2V0YXR0ciBmaCBVbmtub3duLzAxMDAwNzAyODEzODAxMDAwMDAwMDAwMDA4
MTY0RUU0MkRCMTQxRUJBQzk2MTcwMTQ5Qzc0Mzk2RTU2RDA3MDAKPiA+IDEwOjE2OjA5LjAzNjgx
MSBJUCAxMC4wLjAuMzAuMjA0OSA+IDEwLjAuMC40MC4yMjMwNzk5NjIyOiByZXBseSBvayAxODgg
Z2V0YXR0ciBSRUcgMSBpZHMgMS8wIHN6IDAKPiA+IDEwOjE2OjA5LjAzNzgyMyBJUCAxMC4wLjAu
NDAuMjI0NzU3NjgzOCA+IDEwLjAuMC4zMC4yMDQ5OiAxMDggZ2V0YXR0ciBmaCBVbmtub3duLzAx
MDAwNzAwODEzODAxMDAwMDAwMDAwMDA4MTY0RUU0MkRCMTQxRUJBQzk2MTcwMTQ5Qzc0Mzk2MDAw
MDAwMDAKPiA+IDEwOjE2OjA5LjAzOTgxNyBJUCAxMC4wLjAuMzAuMjA0OSA+IDEwLjAuMC40MC4y
MjQ3NTc2ODM4OiByZXBseSBvayAxMTYgZ2V0YXR0ciBESVIgNDA3NTUgaWRzIDAvMCBzeiA0MDk2
Cj4gPiAxMDoxNjowOS4wNDA0MjMgSVAgMTAuMC4wLjQwLjIyNjQzNTQwNTQgPiAxMC4wLjAuMzAu
MjA0OTogMTEyIGdldGF0dHIgZmggVW5rbm93bi8wMTAwMDcwMDgxMzgwMTAwMDAwMDAwMDAwODE2
NEVFNDJEQjE0MUVCQUM5NjE3MDE0OUM3NDM5NjAwMDAwMDBGCj4gPiAxMDoxNjowOS4wNDA4NTYg
SVAgMTAuMC4wLjMwLjIwNDkgPiAxMC4wLjAuNDAuMjI2NDM1NDA1NDogcmVwbHkgb2sgMTg4IGdl
dGF0dHIgUkVHIDIgaWRzIDUvMCBzeiAwCj4gPiAxMDoxNjowOS4wNDE1OTAgSVAgMTAuMC4wLjQw
LjIyODExMzEyNzAgPiAxMC4wLjAuMzAuMjA0OTogMTE2IGxvb2t1cCBmaCBVbmtub3duLzAxMDAw
NzAwODEzODAxMDAwMDAwMDAwMDA4MTY0RUU0MkRCMTQxRUJBQzk2MTcwMTQ5Qzc0Mzk2MDAwMDAw
MDQgImlzbzIiCj4gPiAxMDoxNjowOS4wNDE5MjAgSVAgMTAuMC4wLjMwLjIwNDkgPiAxMC4wLjAu
NDAuMjI4MTEzMTI3MDogcmVwbHkgb2sgMjMyIGxvb2t1cCBmaCBVbmtub3duLzAxMDAwNjAwMkFB
RDZFQTVBMDVENDQ0MUI5NEM0OEU2RTVEOTk4MUUwMDAwMDAwMTAwMDAwMDAyMDAwMDQxRUQKPiA+
IDEwOjE2OjA5LjA0OTYzMyBJUCAxMC4wLjAuNDAuMjI5NzkwODQ4NiA+IDEwLjAuMC4zMC4yMDQ5
OiAxMDQgZ2V0YXR0ciBmaCBVbmtub3duLzAxMDAwNjAwMkFBRDZFQTVBMDVENDQ0MUI5NEM0OEU2
RTVEOTk4MUUwMDAwMDAwRjVGREM4NDQ1NDMyNkUxOTMKPiA+IDEwOjE2OjA5LjA0OTc4MSBJUCAx
MC4wLjAuMzAuMjA0OSA+IDEwLjAuMC40MC4yMjk3OTA4NDg2OiByZXBseSBvayAxODggZ2V0YXR0
ciBSRUcgMiBpZHMgNS8wIHN6IDAKPiA+IDEwOjE2OjA5LjA2MzIxOCBJUCAxMC4wLjAuNDAuMjMx
NDY4NTcwMiA+IDEwLjAuMC4zMC4yMDQ5OiAxMjggZ2V0YXR0ciBmaCBVbmtub3duLzAxMDAwNzAy
ODEzODAxMDAwMDAwMDAwMDA4MTY0RUU0MkRCMTQxRUJBQzk2MTcwMTQ5Qzc0Mzk2RTQ2RDA3MDAK
PiA+IDEwOjE2OjA5LjA3MjUwNSBJUCAxMC4wLjAuMzAuMjA0OSA+IDEwLjAuMC40MC4yMzE0Njg1
NzAyOiByZXBseSBvayAxODggZ2V0YXR0ciBSRUcgMSBpZHMgMS8wIHN6IDAKPiA+IDEwOjE2OjA5
LjA5MTY5OCBJUCAxMC4wLjAuNDAuMjMzMTQ2MjkxOCA+IDEwLjAuMC4zMC4yMDQ5OiAxMjQgZ2V0
YXR0ciBmaCBVbmtub3duLzAxMDAwNzAyODEzODAxMDAwMDAwMDAwMDA4MTY0RUU0MkRCMTQxRUJB
Qzk2MTcwMTQ5Qzc0Mzk2RTQ2RDA3MDAKPiA+IDEwOjE2OjA5LjA5MjAyMiBJUCAxMC4wLjAuMzAu
MjA0OSA+IDEwLjAuMC40MC4yMzMxNDYyOTE4OiByZXBseSBvayAxMTYgZ2V0YXR0ciBSRUcgMTAw
NjQ0IGlkcyAwLzAgc3ogMTA0ODU3Ngo+ID4gMTA6MTY6MDkuMTI4OTcxIElQIDEwLjAuMC40MC4y
MzQ4MjQwMTM0ID4gMTAuMC4wLjMwLjIwNDk6IDEyOCBnZXRhdHRyIGZoIFVua25vd24vMDEwMDA3
MDI4MTM4MDEwMDAwMDAwMDAwMDgxNjRFRTQyREIxNDFFQkFDOTYxNzAxNDlDNzQzOTZFMzZEMDcw
MAo+ID4gMTA6MTY6MDkuMTI5MzA0IElQIDEwLjAuMC4zMC4yMDQ5ID4gMTAuMC4wLjQwLjIzNDgy
NDAxMzQ6IHJlcGx5IG9rIDE4OCBnZXRhdHRyIFJFRyAxIGlkcyAxLzAgc3ogMAo+ID4gMTA6MTY6
MDkuMTg0MTU1IElQIDEwLjAuMC40MC4yMzY1MDE3MzUwID4gMTAuMC4wLjMwLjIwNDk6IDEyNCBn
ZXRhdHRyIGZoIFVua25vd24vMDEwMDA3MDI4MTM4MDEwMDAwMDAwMDAwMDgxNjRFRTQyREIxNDFF
QkFDOTYxNzAxNDlDNzQzOTZFMzZEMDcwMAo+ID4gMTA6MTY6MDkuMTg0NTgyIElQIDEwLjAuMC4z
MC4yMDQ5ID4gMTAuMC4wLjQwLjIzNjUwMTczNTA6IHJlcGx5IG9rIDExNiBnZXRhdHRyIFJFRyAx
MDA2NDQgaWRzIDAvMCBzeiAxMDQ4NTc2Cj4gPiAxMDoxNjowOS4xODkyMzQgSVAgMTAuMC4wLjQw
LjIzODE3OTQ1NjYgPiAxMC4wLjAuMzAuMjA0OTogMTE2IGxvb2t1cCBmaCBVbmtub3duLzAxMDAw
NzAwODEzODAxMDAwMDAwMDAwMDA4MTY0RUU0MkRCMTQxRUJBQzk2MTcwMTQ5Qzc0Mzk2MDAwMDAw
MDQgImlzbzEiCj4gPiAxMDoxNjowOS4xODk0MzUgSVAgMTAuMC4wLjMwLjIwNDkgPiAxMC4wLjAu
NDAuMjM4MTc5NDU2NjogcmVwbHkgb2sgMjMyIGxvb2t1cCBmaCBVbmtub3duLzAxMDAwNjAwMkM0
OUZFRjJCQTQ2NDI5MzlCMkJGMkI4MzIyQ0NCQ0IwMDAwMDAwMTAwMDAwMDAyMDAwMDQxRUQKPiA+
IDEwOjE2OjA5LjE5MzQ3NiBJUCAxMC4wLjAuNDAuMjM5ODU3MTc4MiA+IDEwLjAuMC4zMC4yMDQ5
OiAxMDQgZ2V0YXR0ciBmaCBVbmtub3duLzAxMDAwNjAwMkM0OUZFRjJCQTQ2NDI5MzlCMkJGMkI4
MzIyQ0NCQ0IwMDAwMDAwRjU4ODk2QTg4NEEwQzYyQjcKPiA+IDEwOjE2OjA5LjE5MzY1MiBJUCAx
MC4wLjAuMzAuMjA0OSA+IDEwLjAuMC40MC4yMzk4NTcxNzgyOiByZXBseSBvayAxODggZ2V0YXR0
ciBSRUcgMiBpZHMgNS8wIHN6IDAKPiA+IDEwOjE2OjA5LjE5NDkzNyBJUCAxMC4wLjAuNDAuMjQx
NTM0ODk5OCA+IDEwLjAuMC4zMC4yMDQ5OiAxMjggZ2V0YXR0ciBmaCBVbmtub3duLzAxMDAwNzAy
ODEzODAxMDAwMDAwMDAwMDA4MTY0RUU0MkRCMTQxRUJBQzk2MTcwMTQ5Qzc0Mzk2RTc2RDA3MDAK
PiA+IDEwOjE2OjA5LjE5NTAzMyBJUCAxMC4wLjAuMzAuMjA0OSA+IDEwLjAuMC40MC4yNDE1MzQ4
OTk4OiByZXBseSBvayAxODggZ2V0YXR0ciBSRUcgMSBpZHMgMS8wIHN6IDAKPiA+IDEwOjE2OjA5
LjE5NTIzMCBJUCAxMC4wLjAuNDAuMjQzMjEyNjIxNCA+IDEwLjAuMC4zMC4yMDQ5OiAxMjggZ2V0
YXR0ciBmaCBVbmtub3duLzAxMDAwNzAyODEzODAxMDAwMDAwMDAwMDA4MTY0RUU0MkRCMTQxRUJB
Qzk2MTcwMTQ5Qzc0Mzk2RTg2RDA3MDAKPiA+IDEwOjE2OjA5LjMyMzYzNSBJUCAxMC4wLjAuMzAu
MjA0OSA+IDEwLjAuMC40MC4xMDIyOiAuIGFjayAyNTcyIHdpbiA0MTYgPG5vcCxub3AsdGltZXN0
YW1wIDc1MzYyOSA3NDk1NjY+Cj4gPiAxMDoxNjowOS4zMjQzNDUgSVAgMTAuMC4wLjMwLjIwNDkg
PiAxMC4wLjAuNDAuMjQzMjEyNjIxNDogcmVwbHkgb2sgMTg4IGdldGF0dHIgUkVHIDEgaWRzIDEv
MCBzeiAwCj4gPiAxMDoxNjowOS4zNDE0NzUgSVAgMTAuMC4wLjQwLjI0NDg5MDM0MzAgPiAxMC4w
LjAuMzAuMjA0OTogMTI4IGdldGF0dHIgZmggVW5rbm93bi8wMTAwMDcwMjgxMzgwMTAwMDAwMDAw
MDAwODE2NEVFNDJEQjE0MUVCQUM5NjE3MDE0OUM3NDM5NkUyNkQwNzAwCj4gPiAxMDoxNjowOS4z
NDE1MjQgSVAgMTAuMC4wLjMwLjIwNDkgPiAxMC4wLjAuNDAuMTAyMjogLiBhY2sgMjcwMCB3aW4g
NDQ5IDxub3Asbm9wLHRpbWVzdGFtcCA3NTM2MzIgNzQ5NzA3Pgo+ID4gMTA6MTY6MDkuMzQxOTI4
IElQIDEwLjAuMC4zMC4yMDQ5ID4gMTAuMC4wLjQwLjI0NDg5MDM0MzA6IHJlcGx5IG9rIDE4OCBn
ZXRhdHRyIFJFRyAxIGlkcyAxLzAgc3ogMAo+ID4gMTA6MTY6MDkuMzQyMTE3IElQIDEwLjAuMC40
MC4yNDY1NjgwNjQ2ID4gMTAuMC4wLjMwLjIwNDk6IDEyNCBnZXRhdHRyIGZoIFVua25vd24vMDEw
MDA3MDI4MTM4MDEwMDAwMDAwMDAwMDgxNjRFRTQyREIxNDFFQkFDOTYxNzAxNDlDNzQzOTZFMjZE
MDcwMAo+ID4gMTA6MTY6MDkuMzQzNDA0IElQIDEwLjAuMC4zMC4yMDQ5ID4gMTAuMC4wLjQwLjI0
NjU2ODA2NDY6IHJlcGx5IG9rIDExNiBnZXRhdHRyIFJFRyAxMDA2NDQgaWRzIDAvMCBzeiAxMDQ4
NTc2Cj4gPiAxMDoxNjowOS4zODkzMTYgSVAgMTAuMC4wLjQwLjEwMjIgPiAxMC4wLjAuMzAuMjA0
OTogLiBhY2sgNTU3MyB3aW4gMjE4NCA8bm9wLG5vcCx0aW1lc3RhbXAgNzQ5NzUxIDc1MzYzMz4K
PiA+IDEwOjE2OjEzLjQ0OTUxMyBJUCAxMC4wLjAuNDAuMjQ4MjQ1Nzg2MiA+IDEwLjAuMC4zMC4y
MDQ5OiAxMDggZ2V0YXR0ciBmaCBVbmtub3duLzAxMDAwNzAwODEzODAxMDAwMDAwMDAwMDA4MTY0
RUU0MkRCMTQxRUJBQzk2MTcwMTQ5Qzc0Mzk2NDcyMTlDQjUKPiA+IDEwOjE2OjEzLjQ0OTgxNSBJ
UCAxMC4wLjAuMzAuMjA0OSA+IDEwLjAuMC40MC4yNDgyNDU3ODYyOiByZXBseSBvayAxMTYgZ2V0
YXR0ciBESVIgNDA3NTUgaWRzIDAvMCBzeiA0MDk2Cj4gPiAxMDoxNjoxMy40NTIzNDQgSVAgMTAu
MC4wLjQwLjEwMjIgPiAxMC4wLjAuMzAuMjA0OTogLiBhY2sgNTY4OSB3aW4gMjE4NCA8bm9wLG5v
cCx0aW1lc3RhbXAgNzUzOTQzIDc1NDY2MD4KPiA+IDEwOjE2OjEzLjQ1Mzk3MyBJUCAxMC4wLjAu
NDAuMjQ5OTIzNTA3OCA+IDEwLjAuMC4zMC4yMDQ5OiAxMDAgZ2V0YXR0ciBmaCBVbmtub3duLzAx
MDAwNjAwMkM0OUZFRjJCQTQ2NDI5MzlCMkJGMkI4MzIyQ0NCQ0I0NzIxOUM4QzAwMDAwMDAwNDcy
MTlDOEMKPiA+IDEwOjE2OjEzLjQ1NDE1NCBJUCAxMC4wLjAuMzAuMjA0OSA+IDEwLjAuMC40MC4y
NDk5MjM1MDc4OiByZXBseSBvayAxMTYgZ2V0YXR0ciBESVIgNDA3NTUgaWRzIDAvMCBzeiA0MDk2
Cj4gPiAxMDoxNjoxMy40NTYxOTQgSVAgMTAuMC4wLjQwLjI1MTYwMTIyOTQgPiAxMC4wLjAuMzAu
MjA0OTogMTA4IGdldGF0dHIgZmggVW5rbm93bi8wMTAwMDcwMDgxMzgwMTAwMDAwMDAwMDAwODE2
NEVFNDJEQjE0MUVCQUM5NjE3MDE0OUM3NDM5NjQ3MjE5QzhDCj4gPiAxMDoxNjoxMy40NTYzNjEg
SVAgMTAuMC4wLjMwLjIwNDkgPiAxMC4wLjAuNDAuMjUxNjAxMjI5NDogcmVwbHkgb2sgMTE2IGdl
dGF0dHIgRElSIDQwNzU1IGlkcyAwLzAgc3ogNDA5Ngo+ID4gMTA6MTY6MTMuNDU4MjgyIElQIDEw
LjAuMC40MC4yNTMyNzg5NTEwID4gMTAuMC4wLjMwLjIwNDk6IDExNiBsb29rdXAgZmggVW5rbm93
bi8wMTAwMDcwMDgxMzgwMTAwMDAwMDAwMDAwODE2NEVFNDJEQjE0MUVCQUM5NjE3MDE0OUM3NDM5
NjAwMDAwMDA0ICJpc28xIgo+ID4gMTA6MTY6MTMuNDU4NDYxIElQIDEwLjAuMC4zMC4yMDQ5ID4g
MTAuMC4wLjQwLjI1MzI3ODk1MTA6IHJlcGx5IG9rIDIzMiBsb29rdXAgZmggVW5rbm93bi8wMTAw
MDYwMDJDNDlGRUYyQkE0NjQyOTM5QjJCRjJCODMyMkNDQkNCMDAwMDAwMDEwMDAwMDAwMjAwMDA0
MUVECj4gPiAxMDoxNjoxMy41MTA2MzcgSVAgMTAuMC4wLjQwLjEwMjIgPiAxMC4wLjAuMzAuMjA0
OTogLiBhY2sgNjE1MyB3aW4gMjE4NCA8bm9wLG5vcCx0aW1lc3RhbXAgNzUzOTg1IDc1NDY2Mj4K
PiA+IDEwOjE2OjE0LjAzMDExMCBJUCAxMC4wLjAuNDAuMjU0OTU2NjcyNiA+IDEwLjAuMC4zMC4y
MDQ5OiAxMTIgYWNjZXNzIGZoIFVua25vd24vMDEwMDA3MDA4MTM4MDEwMDAwMDAwMDAwMDgxNjRF
RTQyREIxNDFFQkFDOTYxNzAxNDlDNzQzOTYwMDAwMDAxRiAwMDFmCj4gPiAxMDoxNjoxNC4wMzA5
MjcgSVAgMTAuMC4wLjMwLjIwNDkgPiAxMC4wLjAuNDAuMjU0OTU2NjcyNjogcmVwbHkgb2sgMTI0
IGFjY2VzcyBjIDAwMDMKPiA+IDEwOjE2OjE0LjAzMzQzNiBJUCAxMC4wLjAuNDAuMTAyMiA+IDEw
LjAuMC4zMC4yMDQ5OiAuIGFjayA2Mjc3IHdpbiAyMTg0IDxub3Asbm9wLHRpbWVzdGFtcCA3NTQ1
NDggNzU0ODA1Pgo+ID4gMTA6MTY6MTQuMDM0NzMyIElQIDEwLjAuMC40MC4yNTY2MzQzOTQyID4g
MTAuMC4wLjMwLjIwNDk6IDEwOCBnZXRhdHRyIGZoIFVua25vd24vMDEwMDA3MDA4MTM4MDEwMDAw
MDAwMDAwMDgxNjRFRTQyREIxNDFFQkFDOTYxNzAxNDlDNzQzOTYwMDAwMDAwMAo+ID4gMTA6MTY6
MTQuMDM0OTgwIElQIDEwLjAuMC4zMC4yMDQ5ID4gMTAuMC4wLjQwLjI1NjYzNDM5NDI6IHJlcGx5
IG9rIDExNiBnZXRhdHRyIERJUiA0MDc1NSBpZHMgMC8wIHN6IDQwOTYKPiA+IDEwOjE2OjE0LjAz
NzMxOSBJUCAxMC4wLjAuNDAuMjU4MzEyMTE1OCA+IDEwLjAuMC4zMC4yMDQ5OiAxMTIgYWNjZXNz
IGZoIFVua25vd24vMDEwMDA3MDA4MTM4MDEwMDAwMDAwMDAwMDgxNjRFRTQyREIxNDFFQkFDOTYx
NzAxNDlDNzQzOTYwMDAwMDAxRiAwMDFmCj4gPiAxMDoxNjoxNC4wMzc0ODYgSVAgMTAuMC4wLjMw
LjIwNDkgPiAxMC4wLjAuNDAuMjU4MzEyMTE1ODogcmVwbHkgb2sgMTI0IGFjY2VzcyBjIDAwMDMK
PiA+IDEwOjE2OjE0LjA0MDMyMyBJUCAxMC4wLjAuNDAuMjU5OTg5ODM3NCA+IDEwLjAuMC4zMC4y
MDQ5OiAxMzIgcmVhZGRpcnBsdXMgZmggVW5rbm93bi8wMTAwMDcwMDgxMzgwMTAwMDAwMDAwMDAw
ODE2NEVFNDJEQjE0MUVCQUM5NjE3MDE0OUM3NDM5NjAwMDAwMDAwIDUxMiBieXRlcyBAIDAKPiA+
IDEwOjE2OjE0LjA0MDU1NCBJUCAxMC4wLjAuMzAuMjA0OSA+IDEwLjAuMC40MC4yNTk5ODk4Mzc0
OiByZXBseSBvayAxNDQ4IHJlYWRkaXJwbHVzCj4gPiAxMDoxNjoxNC4wNDEwMjAgSVAgMTAuMC4w
LjMwLjIwNDkgPiAxMC4wLjAuNDAuMTY4NDEwODI4ODogcmVwbHkgVW5rbm93biBycGMgcmVzcG9u
c2UgY29kZT0yMDIxODU1ODYxIDM0MAo+ID4gMTA6MTY6MTQuMDQzNTgzIElQIDEwLjAuMC40MC4x
MDIyID4gMTAuMC4wLjMwLjIwNDk6IC4gYWNrIDgzMDUgd2luIDI5MDggPG5vcCxub3AsdGltZXN0
YW1wIDc1NDU1MCA3NTQ4MDg+Cj4gPiAxMDoxNjoxNC4wNDUxMDQgSVAgMTAuMC4wLjQwLjI2MTY2
NzU1OTAgPiAxMC4wLjAuMzAuMjA0OTogMTEyIGFjY2VzcyBmaCBVbmtub3duLzAxMDAwNzAwODEz
ODAxMDAwMDAwMDAwMDA4MTY0RUU0MkRCMTQxRUJBQzk2MTcwMTQ5Qzc0Mzk2MDAwMDAwMUYgMDAx
Zgo+ID4gMTA6MTY6MTQuMDQ1NDAyIElQIDEwLjAuMC4zMC4yMDQ5ID4gMTAuMC4wLjQwLjI2MTY2
NzU1OTA6IHJlcGx5IG9rIDEyNCBhY2Nlc3MgYyAwMDAzCj4gPiAxMDoxNjoxNC4wNDc4MzAgSVAg
MTAuMC4wLjQwLjI2MzM0NTI4MDYgPiAxMC4wLjAuMzAuMjA0OTogMTEyIGFjY2VzcyBmaCBVbmtu
b3duLzAxMDAwNzAwODEzODAxMDAwMDAwMDAwMDA4MTY0RUU0MkRCMTQxRUJBQzk2MTcwMTQ5Qzc0
Mzk2MDAwMDAwMUYgMDAxZgo+ID4gMTA6MTY6MTQuMDQ4MDM5IElQIDEwLjAuMC4zMC4yMDQ5ID4g
MTAuMC4wLjQwLjI2MzM0NTI4MDY6IHJlcGx5IG9rIDEyNCBhY2Nlc3MgYyAwMDAzCj4gPiAxMDox
NjoxNC4wOTkzODUgSVAgMTAuMC4wLjQwLjEwMjIgPiAxMC4wLjAuMzAuMjA0OTogLiBhY2sgODU1
MyB3aW4gMjkwOCA8bm9wLG5vcCx0aW1lc3RhbXAgNzU0NTkyIDc1NDgxMD4KPiA+IDEwOjE2OjE0
LjI5MzcxNCBJUCAxMC4wLjAuNDAuMjY1MDIzMDAyMiA+IDEwLjAuMC4zMC4yMDQ5OiAxMDggZ2V0
YXR0ciBmaCBVbmtub3duLzAxMDAwNzAwODEzODAxMDAwMDAwMDAwMDA4MTY0RUU0MkRCMTQxRUJB
Qzk2MTcwMTQ5Qzc0Mzk2MDAwMDAwMDAKPiA+IDEwOjE2OjE0LjI5NDAxOSBJUCAxMC4wLjAuMzAu
MjA0OSA+IDEwLjAuMC40MC4yNjUwMjMwMDIyOiByZXBseSBvayAxMTYgZ2V0YXR0ciBESVIgNDA3
NTUgaWRzIDAvMCBzeiA0MDk2Cj4gPiAxMDoxNjoxNC4yOTcyNjMgSVAgMTAuMC4wLjQwLjEwMjIg
PiAxMC4wLjAuMzAuMjA0OTogLiBhY2sgODY2OSB3aW4gMjkwOCA8bm9wLG5vcCx0aW1lc3RhbXAg
NzU0NzYzIDc1NDg3MT4KPiA+IDEwOjE2OjE0LjI5NzI3MiBJUCAxMC4wLjAuNDAuMjY2NzAwNzIz
OCA+IDEwLjAuMC4zMC4yMDQ5OiAxMTIgYWNjZXNzIGZoIFVua25vd24vMDEwMDA3MDA4MTM4MDEw
MDAwMDAwMDAwMDgxNjRFRTQyREIxNDFFQkFDOTYxNzAxNDlDNzQzOTYwMDAwMDAxRiAwMDFmCj4g
PiAxMDoxNjoxNC4yOTc0NjYgSVAgMTAuMC4wLjMwLjIwNDkgPiAxMC4wLjAuNDAuMjY2NzAwNzIz
ODogcmVwbHkgb2sgMTI0IGFjY2VzcyBjIDAwMDMKPiA+IDEwOjE2OjE0LjI5NzU4NiBJUCAxMC4w
LjAuNDAuMjY4Mzc4NDQ1NCA+IDEwLjAuMC4zMC4yMDQ5OiAxMTIgYWNjZXNzIGZoIFVua25vd24v
MDEwMDA3MDA4MTM4MDEwMDAwMDAwMDAwMDgxNjRFRTQyREIxNDFFQkFDOTYxNzAxNDlDNzQzOTYw
MDAwMDAxRiAwMDFmCj4gPiAxMDoxNjoxNC4yOTc5ODcgSVAgMTAuMC4wLjMwLjIwNDkgPiAxMC4w
LjAuNDAuMjY4Mzc4NDQ1NDogcmVwbHkgb2sgMTI0IGFjY2VzcyBjIDAwMDMKPiA+IDEwOjE2OjE0
LjMwMTE2NSBJUCAxMC4wLjAuNDAuMjcwMDU2MTY3MCA+IDEwLjAuMC4zMC4yMDQ5OiAxMDQgYWNj
ZXNzIGZoIFVua25vd24vMDEwMDA2MDAyQzQ5RkVGMkJBNDY0MjkzOUIyQkYyQjgzMjJDQ0JDQjAw
MDAwMDFGNDcyMTlDOEMwMDAwMDAwMCAwMDFmCj4gPiAxMDoxNjoxNC4zMDE0MDUgSVAgMTAuMC4w
LjMwLjIwNDkgPiAxMC4wLjAuNDAuMjcwMDU2MTY3MDogcmVwbHkgb2sgMTI0IGFjY2VzcyBjIDAw
MDMKPiA+IDEwOjE2OjE0LjM1MTIyOSBJUCAxMC4wLjAuNDAuMTAyMiA+IDEwLjAuMC4zMC4yMDQ5
OiAuIGFjayA5MDQxIHdpbiAyOTA4IDxub3Asbm9wLHRpbWVzdGFtcCA3NTQ4MTAgNzU0ODczPgo+
ID4gMTA6MTY6MTQuODQyNjMzIElQIDEwLjAuMC40MC4yNzE3MzM4ODg2ID4gMTAuMC4wLjMwLjIw
NDk6IDEwMCBnZXRhdHRyIGZoIFVua25vd24vMDEwMDA2MDAyQzQ5RkVGMkJBNDY0MjkzOUIyQkYy
QjgzMjJDQ0JDQjAwMDAwMDAwNDcyMTlDOEMwMDAwMDAwMAo+ID4gMTA6MTY6MTQuODUxNzExIElQ
IDEwLjAuMC4zMC4yMDQ5ID4gMTAuMC4wLjQwLjI3MTczMzg4ODY6IHJlcGx5IG9rIDExNiBnZXRh
dHRyIERJUiA0MDc1NSBpZHMgMC8wIHN6IDQwOTYKPiA+IDEwOjE2OjE0Ljg1Njg0NiBJUCAxMC4w
LjAuNDAuMTAyMiA+IDEwLjAuMC4zMC4yMDQ5OiAuIGFjayA5MTU3IHdpbiAyOTA4IDxub3Asbm9w
LHRpbWVzdGFtcCA3NTUyNzIgNzU1MDA4Pgo+ID4gCj4gPiAKPiA+ID4gPiBkb2VzIHNvbWVib2R5
IGhhdmUgc3VjaCBzZXR1cCB1cCBhbmQgcnVubmluZyBhbmQgY2FuIHRlbGwgaGlzIGRpc3RybyAv
IGtlcm5lbCBhbmQgbmZzLXV0aWxzIHZlcnNpb24gPwo+ID4gPiA+IG1heWJlIGkgY2hhbmdlIGRp
c3RybyB0aGVuLgo+ID4gPiAKPiA+ID4gSSBkb3VidCB0aGF0IGl0IGlzIGEgZGlzdHJvLXNwZWNp
ZmljIHRoaW5nLiAgQXMgbG9uZyBhcyB5b3UgaGF2ZQo+ID4gPiBuZnMtdXRpbHMtMS4xLjAgaXQg
c2hvdWxkIHdvcmsuICBJIGRvbid0IGhhdmUgYSAxMC4zIGJveAo+ID4gPiBzZXQgdXAgeWV0LCBi
dXQgaXQgd29ya3MgZmluZSBvbiBEZWJpYW4vdW5zdGFibGUgZm9yIG1lLgo+ID4gCj4gPiBvaywg
d2lsbCB0cnkgdGhpcyBvbiBkZWJpYW4uCj4gPiAKPiA+ID4gTWF5YmUgdHJ5IGFkZGluZyB0aGUg
Im5vX3Jvb3Rfc3F1YXNoIiBleHBvcnQgb3B0aW9uLgo+ID4gbm8gZGlmZmVyZW5jZQo+ID4gCj4g
PiA+IFdoYXQgZG9lcyAibHMgLWwgL2V4cG9ydCIgb24gdGhlIHNlcnZlciBzaG93Pwo+ID4gbm90
aGluZyB1bnVzdWFsLiBubyBlcnJvcnMsIGp1c3QgdGhlIGRpcnMvbW91bnRwb2ludHMKPiA+IAo+
ID4gVGhhbmtzIGZvciB5b3VyIGhlbHAhCj4gPiAKPiA+IHJlZ2FyZHMKPiA+IHJvbGFuZAo+ID4g
Cj4gPiAKPiA+ID4gCj4gPiA+IE9uIFNhdHVyZGF5IE9jdG9iZXIgMjcsIGRldnplcm9Ad2ViLmRl
IHdyb3RlOgo+ID4gPiA+IEhlbGxvICEKPiA+ID4gPiAKPiA+ID4gPiB3aXRoIDIuNi4yMiBpYG0g
dHJ5aW5nIHRvIGV4cG9ydCBsb29wYmFjayBtb3VudGVkIGlzby1pbWFnZXMuCj4gPiA+ID4gCj4g
PiA+ID4gdGhpcyBpcyAvZXRjL2V4cG9ydHM6Cj4gPiA+ID4gCj4gPiA+ID4gL2V4cG9ydCAqKHJv
LGNyb3NzbW50LHN1YnRyZWVfY2hlY2spCj4gPiA+IAo+ID4gPiBJIHJlY29tbWVuZCByZXBsYWNp
bmcgc3VidHJlZV9jaGVjayB3aXRoIG5vX3N1YnRyZWVfY2hlY2ssIGJ1dCBpdAo+ID4gPiBzaG91
bGRuJ3QgbWFrZSBhbiBpbXBvcnRhbnQgZGlmZmVyZW5jZSBpbiB0aGlzIGNhc2UuCj4gPiA+IAo+
ID4gPiAKPiA+ID4gVGhpcyBzaG91bGQgd29yayB3aXRoIG5mcy11dGlscyAxLjEuMCBvciBsYXRl
ci4gIFdpdGggZWFybGllciByZWxlYXNlcwo+ID4gPiB5b3UgbmVlZCB0byBleHBsaWNpdGx5IGV4
cG9ydCB0aGUgc3Vib3JkaW5hdGUgZmlsZXN5c3RlbXMgdG9vLgo+ID4gPiAKPiA+ID4gPiAKPiA+
ID4gPiBpbiAvZXhwb3J0LCBpIGhhdmUgbG9vcGJhY2sgbW91bnRlZCBpc28taW1hZ2VzCj4gPiA+
ID4gCj4gPiA+ID4gYWZ0ZXIgbW91bnRpbmcgb24gdGhlIGNsaWVudCBzaWRlIHVuZGVyIC9tbnQg
KHRyaWVkIG9uZSBvbGRlciBhbmQgb25lIHJlY2VudCBzeXN0ZW0pICwgaWBtIGdldHRpbmc6Cj4g
PiA+ID4gCj4gPiA+ID4gdm1ob3N0Oi9tbnQgIyBscyAtbGEKPiA+ID4gPiAvYmluL2xzOiBpc28x
OiBJbnB1dC9vdXRwdXQgZXJyb3IKPiA+ID4gPiAvYmluL2xzOiBpc28yICBJbnB1dC9vdXRwdXQg
ZXJyb3IKPiA+ID4gPiAvYmluL2xzOiBpc28zOiBJbnB1dC9vdXRwdXQgZXJyb3IKPiA+ID4gPiB0
b3RhbCAxMDEyOAo+ID4gPiA+IGRyd3hyd3hyd3QgICAxOCByb290IHJvb3QgIDI3MDMzNiBPY3Qg
MjYgMDg6NDUgLgo+ID4gPiA+IGRyd3hyd3hyd3QgIDE4NiByb290IHJvb3QgICAyMDc2MCBPY3Qg
MjcgMTc6NDUgLi4KPiA+ID4gPiBkcnd4ci14ci14ICAgIDIgcm9vdCByb290ICAgMTYzODQgSmFu
ICAxICAxOTcwIGlzbzEKPiA+ID4gPiBkcnd4ci14ci14ICAgIDIgcm9vdCByb290ICAgMTYzODQg
SmFuICAxICAxOTcwIGlzbzIKPiA+ID4gPiBkcnd4ci14ci14ICAgIDIgcm9vdCByb290ICAgMTYz
ODQgSmFuICAxICAxOTcwIGlzbzMKPiA+ID4gPiAKPiA+ID4gPiB2bWhvc3Q6L21udC9pc28xICMg
bHMKPiA+ID4gPiAvYmluL2xzOiAuOiBTdGFsZSBORlMgZmlsZSBoYW5kbGUKPiA+ID4gPiB2bWhv
c3Q6L21udC9pc28xICMgbHMgLWxhCj4gPiA+ID4gL2Jpbi9sczogLjogSW5wdXQvb3V0cHV0IGVy
cm9yCj4gPiA+IAo+ID4gPiBJdCBpcyBhIGxpdHRsZSBvZGQgdGhhdCB0aGUgZXJyb3JzIGFyZSBp
bmNvbnNpc3RlbnQuCj4gPiA+IAo+ID4gPiBDYW4geW91IGZpbmQgYW55IGxvZyBtZXNzYWdlcyBm
cm9tIG1vdW50ZCBpbiBzeXNsb2c/ICBXaGF0IGRvIHRoZXkKPiA+ID4gc2F5Pwo+ID4gPiBBbHNv
IHdoYXQgZG9lcwo+ID4gPiAgICBjYXQgL3Byb2MvZnMvbmZzZC9leHBvcnRzCj4gPiA+IAo+ID4g
PiBvbiB0aGUgc2VydmVyIHNob3cuCj4gPiA+IAo+ID4gPiBGaW5hbGx5LCBhIHRjcGR1bXA6Cj4g
PiA+IAo+ID4gPiAgIHRjcGR1bXAgLXMgMCAtdyAvdG1wL3RjcGR1bXAgcG9ydCAyMDQ5Cj4gPiA+
IAo+ID4gPiB3aGlsZSB5b3UgcnVuIHRoZSBleHBlcmltZW50IG1pZ2h0IGhlbHAuCj4gPiA+IAo+
ID4gPiA+IAo+ID4gPiA+IGlgbSB1bnN1cmUgaWYgaSBzaG91bGQgYmxhbWUgc3VzZSBoZXJlIChp
dGBzIGFuIG9wZW5zdXNlIDEwLjMgYm94IHdoaWNoIHNlZW1zIHRvIGhhdmUgbmZzLXV0aWxzIDEu
MS4wKQo+ID4gPiA+IAo+ID4gPiA+IGRvZXMgc29tZWJvZHkgaGF2ZSBzdWNoIHNldHVwIHVwIGFu
ZCBydW5uaW5nIGFuZCBjYW4gdGVsbCBoaXMgZGlzdHJvIC8ga2VybmVsIGFuZCBuZnMtdXRpbHMg
dmVyc2lvbiA/Cj4gPiA+ID4gbWF5YmUgaSBjaGFuZ2UgZGlzdHJvIHRoZW4uCj4gPiA+IAo+ID4g
PiBJIGRvdWJ0IHRoYXQgaXQgaXMgYSBkaXN0cm8tc3BlY2lmaWMgdGhpbmcuICBBcyBsb25nIGFz
IHlvdSBoYXZlCj4gPiA+IG5mcy11dGlscy0xLjEuMCBpdCBzaG91bGQgd29yay4gIEkgZG9uJ3Qg
aGF2ZSBhIDEwLjMgYm94Cj4gPiA+IHNldCB1cCB5ZXQsIGJ1dCBpdCB3b3JrcyBmaW5lIG9uIERl
Ymlhbi91bnN0YWJsZSBmb3IgbWUuCj4gPiA+IAo+ID4gPiBNYXliZSB0cnkgYWRkaW5nIHRoZSAi
bm9fcm9vdF9zcXVhc2giIGV4cG9ydCBvcHRpb24uCj4gPiA+IFdoYXQgZG9lcyAibHMgLWwgL2V4
cG9ydCIgb24gdGhlIHNlcnZlciBzaG93Pwo+ID4gPiAKPiA+ID4gTmVpbEJyb3duCj4gPiA+IAo+
ID4gCj4gPiAKPiAKPiAKPiBfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fX19fX18KPiBEZXIgV0VCLkRFIFNtYXJ0U3VyZmVyIGhp
bGZ0IGJpcyB6dSA3MCUgSWhyZXIgT25saW5la29zdGVuIHp1IHNwYXJlbiEKPiBodHRwOi8vc21h
cnRzdXJmZXIud2ViLmRlLz9tYz0xMDAwNzEmZGlzdHJpYnV0aW9uaWQ9MDAwMDAwMDAwMDY2Cj4g
Cj4gCj4gLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLQo+IFRoaXMgU0YubmV0IGVtYWlsIGlzIHNwb25zb3JlZCBi
eTogU3BsdW5rIEluYy4KPiBTdGlsbCBncmVwcGluZyB0aHJvdWdoIGxvZyBmaWxlcyB0byBmaW5k
IHByb2JsZW1zPyAgU3RvcC4KPiBOb3cgU2VhcmNoIGxvZyBldmVudHMgYW5kIGNvbmZpZ3VyYXRp
b24gZmlsZXMgdXNpbmcgQUpBWCBhbmQgYSBicm93c2VyLgo+IERvd25sb2FkIHlvdXIgRlJFRSBj
b3B5IG9mIFNwbHVuayBub3cgPj4gaHR0cDovL2dldC5zcGx1bmsuY29tLwo+IF9fX19fX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCj4gTkZTIG1haWxsaXN0ICAtICBO
RlNAbGlzdHMuc291cmNlZm9yZ2UubmV0Cj4gaHR0cHM6Ly9saXN0cy5zb3VyY2Vmb3JnZS5uZXQv
bGlzdHMvbGlzdGluZm8vbmZzCgotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tClRoaXMgU0YubmV0IGVtYWlsIGlz
IHNwb25zb3JlZCBieTogU3BsdW5rIEluYy4KU3RpbGwgZ3JlcHBpbmcgdGhyb3VnaCBsb2cgZmls
ZXMgdG8gZmluZCBwcm9ibGVtcz8gIFN0b3AuCk5vdyBTZWFyY2ggbG9nIGV2ZW50cyBhbmQgY29u
ZmlndXJhdGlvbiBmaWxlcyB1c2luZyBBSkFYIGFuZCBhIGJyb3dzZXIuCkRvd25sb2FkIHlvdXIg
RlJFRSBjb3B5IG9mIFNwbHVuayBub3cgPj4gaHR0cDovL2dldC5zcGx1bmsuY29tLwpfX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpORlMgbWFpbGxpc3QgIC0g
IE5GU0BsaXN0cy5zb3VyY2Vmb3JnZS5uZXQKaHR0cHM6Ly9saXN0cy5zb3VyY2Vmb3JnZS5uZXQv
bGlzdHMvbGlzdGluZm8vbmZzCg==
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: stale nfs file handle with exported loopback mounts
@ 2007-10-30 20:05 devzero
0 siblings, 0 replies; 17+ messages in thread
From: devzero @ 2007-10-30 20:05 UTC (permalink / raw)
To: Neil Brown; +Cc: NFS
> I recommend replacing subtree_check with no_subtree_check, but it
> shouldn't make an important difference in this case.
ok, i leave it as is.
> This should work with nfs-utils 1.1.0 or later. With earlier releases
> you need to explicitly export the subordinate filesystems too.
>
mhh - opensuse doesn`t have nfs-utils package, but it has nfs-client-1.1.0-8 which looks like they repackaged nfs-utils 1.1.0
> It is a little odd that the errors are inconsistent.
ok, but only a minor issue, if an issue at all, isn`t it ?
> Can you find any log messages from mountd in syslog? What do they
> say?
yes, on the client i`m getting :
Jun 3 21:36:01 linux kernel: nfs_update_inode: inode number mismatch
Jun 3 21:36:01 linux kernel: expected (0:11/0x2), got (0:11/0x13881)
Jun 3 21:36:01 linux kernel: nfs_update_inode: inode number mismatch
Jun 3 21:36:01 linux kernel: expected (0:11/0x2), got (0:11/0x13881)
Jun 3 21:36:17 linux kernel: nfs_update_inode: inode number mismatch
Jun 3 21:36:17 linux kernel: expected (0:11/0x2), got (0:11/0x13881)
Jun 3 21:36:17 linux kernel: nfs_update_inode: inode number mismatch
Jun 3 21:36:17 linux kernel: expected (0:11/0x2), got (0:11/0x13881)
Jun 3 21:36:20 linux kernel: nfs_update_inode: inode number mismatch
Jun 3 21:36:20 linux kernel: expected (0:11/0x2), got (0:11/0x13881)
Jun 3 21:36:20 linux kernel: nfs_update_inode: inode number mismatch
no error message on the server:
Oct 26 10:09:31 opensuse103 mountd[4293]: authenticated unmount request from 10.0.0.40:1014 for /mnt (/mnt)
Oct 26 10:10:07 opensuse103 mountd[4293]: authenticated mount request from 10.0.0.40:612 for /mnt (/mnt)
Oct 26 10:10:43 opensuse103 mountd[4293]: authenticated unmount request from 10.0.0.40:623 for /mnt (/mnt)
Oct 26 10:10:52 opensuse103 mountd[4293]: authenticated mount request from 10.0.0.40:624 for /mnt (/mnt)
> Also what does
> cat /proc/fs/nfsd/exports
>
> on the server show.
opensuse103:~ # cat /proc/fs/nfsd/exports
# Version 1.1
# Path Client(Flags) # IPs
/mnt/iso1 *(ro,no_root_squash,sync,wdelay,crossmnt,uuid=2c49fef2:ba464293:9b2bf2b8:322ccbcb)
/mnt/iso3 *(ro,no_root_squash,sync,wdelay,crossmnt,uuid=27ae9c67:0b794b36:8b5e9e17:37b569eb)
/mnt *(ro,no_root_squash,sync,wdelay,crossmnt,uuid=08164ee4:2db141eb:ac961701:49c74396)
/mnt/iso2 *(ro,no_root_squash,sync,wdelay,crossmnt,uuid=2aad6ea5:a05d4441:b94c48e6:e5d9981e)
> Finally, a tcpdump:
>
> tcpdump -s 0 -w /tmp/tcpdump port 2049
>
> while you run the experiment might help.
ah - this seems to give a hint, but i don`t have a clue why the server (10.0.0.30) is telling the client (10.0.0.40) a "RPC Version mismatch".
I also tried --no-nfs-version 4 for rpc.mountd (setting in /etc/sysconfig/nfs), but this didn`t make a difference.
here is the tcpdump output - i did
- mount
- ls / ls -la / cd to subdirs
10:15:06.480542 IP 10.0.0.40.0 > 10.0.0.30.2049: 0 null
10:15:06.480572 IP 10.0.0.30.2049 > 10.0.0.40.0: reply ERR 0: RPC Version mismatch (167772160-0)
10:15:06.480765 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 2031114913 win 1460 <nop,nop,timestamp 686128 737918>
10:15:06.480821 IP 10.0.0.40.2079804678 > 10.0.0.30.2049: 108 fsinfo fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743963CD81908
10:15:06.480833 IP 10.0.0.30.2049 > 10.0.0.40.1022: . ack 108 win 181 <nop,nop,timestamp 737918 686128>
10:15:06.513365 IP 10.0.0.30.2049 > 10.0.0.40.2079804678: reply ok 84 fsinfo rtmax 65536 rtpref 65536 wtmax 65536 wtpref 65536 dtpref 4096
10:15:06.514408 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 85 win 1460 <nop,nop,timestamp 686160 737926>
10:15:06.514902 IP 10.0.0.40.2096581894 > 10.0.0.30.2049: 108 getattr fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743963CD81908
10:15:06.515256 IP 10.0.0.30.2049 > 10.0.0.40.2096581894: reply ok 116 getattr DIR 40755 ids 0/0 sz 4096
10:15:06.553865 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 201 win 1460 <nop,nop,timestamp 686201 737926>
10:16:03.091784 IP 10.0.0.40.2113359110 > 10.0.0.30.2049: 108 getattr fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439647219C8C
10:16:03.093366 IP 10.0.0.30.2049 > 10.0.0.40.2113359110: reply ok 116 getattr DIR 40755 ids 0/0 sz 4096
10:16:03.096046 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 317 win 1460 <nop,nop,timestamp 743610 752071>
10:16:03.097370 IP 10.0.0.40.2130136326 > 10.0.0.30.2049: 112 access fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000001F 001f
10:16:03.098156 IP 10.0.0.30.2049 > 10.0.0.40.2130136326: reply ok 124 access c 0003
10:16:03.156812 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 441 win 1460 <nop,nop,timestamp 743651 752072>
10:16:08.967804 IP 10.0.0.40.2146913542 > 10.0.0.30.2049: 108 getattr fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439600000000
10:16:08.968305 IP 10.0.0.30.2049 > 10.0.0.40.2146913542: reply ok 116 getattr DIR 40755 ids 0/0 sz 4096
10:16:08.974285 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 557 win 1460 <nop,nop,timestamp 749477 753540>
10:16:08.975739 IP 10.0.0.40.2163690758 > 10.0.0.30.2049: 132 readdirplus fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439600000000 512 bytes @ 0
10:16:08.975985 IP 10.0.0.30.2049 > 10.0.0.40.2163690758: reply ok 1448 readdirplus
10:16:08.976510 IP 10.0.0.30.2049 > 10.0.0.40.1684108288: reply Unknown rpc response code=2021855861 340
10:16:08.982238 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 2345 win 2184 <nop,nop,timestamp 749479 753541>
10:16:08.984415 IP 10.0.0.40.2180467974 > 10.0.0.30.2049: 116 lookup fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439600000004 "iso3"
10:16:08.984931 IP 10.0.0.30.2049 > 10.0.0.40.2180467974: reply ok 232 lookup fh Unknown/0100060027AE9C670B794B368B5E9E1737B569EB0000000100000002000041ED
10:16:09.008819 IP 10.0.0.40.2197245190 > 10.0.0.30.2049: 104 getattr fh Unknown/0100060027AE9C670B794B368B5E9E1737B569EB0000000FDD22CC3C7002F0AC
10:16:09.010946 IP 10.0.0.30.2049 > 10.0.0.40.2197245190: reply ok 188 getattr REG 2 ids 5/0 sz 0
10:16:09.032541 IP 10.0.0.40.2214022406 > 10.0.0.30.2049: 128 getattr fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E66D0700
10:16:09.033405 IP 10.0.0.30.2049 > 10.0.0.40.2214022406: reply ok 188 getattr REG 1 ids 1/0 sz 0
10:16:09.033490 IP 10.0.0.40.2230799622 > 10.0.0.30.2049: 128 getattr fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E56D0700
10:16:09.036811 IP 10.0.0.30.2049 > 10.0.0.40.2230799622: reply ok 188 getattr REG 1 ids 1/0 sz 0
10:16:09.037823 IP 10.0.0.40.2247576838 > 10.0.0.30.2049: 108 getattr fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439600000000
10:16:09.039817 IP 10.0.0.30.2049 > 10.0.0.40.2247576838: reply ok 116 getattr DIR 40755 ids 0/0 sz 4096
10:16:09.040423 IP 10.0.0.40.2264354054 > 10.0.0.30.2049: 112 getattr fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000000F
10:16:09.040856 IP 10.0.0.30.2049 > 10.0.0.40.2264354054: reply ok 188 getattr REG 2 ids 5/0 sz 0
10:16:09.041590 IP 10.0.0.40.2281131270 > 10.0.0.30.2049: 116 lookup fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439600000004 "iso2"
10:16:09.041920 IP 10.0.0.30.2049 > 10.0.0.40.2281131270: reply ok 232 lookup fh Unknown/010006002AAD6EA5A05D4441B94C48E6E5D9981E0000000100000002000041ED
10:16:09.049633 IP 10.0.0.40.2297908486 > 10.0.0.30.2049: 104 getattr fh Unknown/010006002AAD6EA5A05D4441B94C48E6E5D9981E0000000F5FDC84454326E193
10:16:09.049781 IP 10.0.0.30.2049 > 10.0.0.40.2297908486: reply ok 188 getattr REG 2 ids 5/0 sz 0
10:16:09.063218 IP 10.0.0.40.2314685702 > 10.0.0.30.2049: 128 getattr fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E46D0700
10:16:09.072505 IP 10.0.0.30.2049 > 10.0.0.40.2314685702: reply ok 188 getattr REG 1 ids 1/0 sz 0
10:16:09.091698 IP 10.0.0.40.2331462918 > 10.0.0.30.2049: 124 getattr fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E46D0700
10:16:09.092022 IP 10.0.0.30.2049 > 10.0.0.40.2331462918: reply ok 116 getattr REG 100644 ids 0/0 sz 1048576
10:16:09.128971 IP 10.0.0.40.2348240134 > 10.0.0.30.2049: 128 getattr fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E36D0700
10:16:09.129304 IP 10.0.0.30.2049 > 10.0.0.40.2348240134: reply ok 188 getattr REG 1 ids 1/0 sz 0
10:16:09.184155 IP 10.0.0.40.2365017350 > 10.0.0.30.2049: 124 getattr fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E36D0700
10:16:09.184582 IP 10.0.0.30.2049 > 10.0.0.40.2365017350: reply ok 116 getattr REG 100644 ids 0/0 sz 1048576
10:16:09.189234 IP 10.0.0.40.2381794566 > 10.0.0.30.2049: 116 lookup fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439600000004 "iso1"
10:16:09.189435 IP 10.0.0.30.2049 > 10.0.0.40.2381794566: reply ok 232 lookup fh Unknown/010006002C49FEF2BA4642939B2BF2B8322CCBCB0000000100000002000041ED
10:16:09.193476 IP 10.0.0.40.2398571782 > 10.0.0.30.2049: 104 getattr fh Unknown/010006002C49FEF2BA4642939B2BF2B8322CCBCB0000000F58896A884A0C62B7
10:16:09.193652 IP 10.0.0.30.2049 > 10.0.0.40.2398571782: reply ok 188 getattr REG 2 ids 5/0 sz 0
10:16:09.194937 IP 10.0.0.40.2415348998 > 10.0.0.30.2049: 128 getattr fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E76D0700
10:16:09.195033 IP 10.0.0.30.2049 > 10.0.0.40.2415348998: reply ok 188 getattr REG 1 ids 1/0 sz 0
10:16:09.195230 IP 10.0.0.40.2432126214 > 10.0.0.30.2049: 128 getattr fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E86D0700
10:16:09.323635 IP 10.0.0.30.2049 > 10.0.0.40.1022: . ack 2572 win 416 <nop,nop,timestamp 753629 749566>
10:16:09.324345 IP 10.0.0.30.2049 > 10.0.0.40.2432126214: reply ok 188 getattr REG 1 ids 1/0 sz 0
10:16:09.341475 IP 10.0.0.40.2448903430 > 10.0.0.30.2049: 128 getattr fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E26D0700
10:16:09.341524 IP 10.0.0.30.2049 > 10.0.0.40.1022: . ack 2700 win 449 <nop,nop,timestamp 753632 749707>
10:16:09.341928 IP 10.0.0.30.2049 > 10.0.0.40.2448903430: reply ok 188 getattr REG 1 ids 1/0 sz 0
10:16:09.342117 IP 10.0.0.40.2465680646 > 10.0.0.30.2049: 124 getattr fh Unknown/01000702813801000000000008164EE42DB141EBAC96170149C74396E26D0700
10:16:09.343404 IP 10.0.0.30.2049 > 10.0.0.40.2465680646: reply ok 116 getattr REG 100644 ids 0/0 sz 1048576
10:16:09.389316 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 5573 win 2184 <nop,nop,timestamp 749751 753633>
10:16:13.449513 IP 10.0.0.40.2482457862 > 10.0.0.30.2049: 108 getattr fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439647219CB5
10:16:13.449815 IP 10.0.0.30.2049 > 10.0.0.40.2482457862: reply ok 116 getattr DIR 40755 ids 0/0 sz 4096
10:16:13.452344 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 5689 win 2184 <nop,nop,timestamp 753943 754660>
10:16:13.453973 IP 10.0.0.40.2499235078 > 10.0.0.30.2049: 100 getattr fh Unknown/010006002C49FEF2BA4642939B2BF2B8322CCBCB47219C8C0000000047219C8C
10:16:13.454154 IP 10.0.0.30.2049 > 10.0.0.40.2499235078: reply ok 116 getattr DIR 40755 ids 0/0 sz 4096
10:16:13.456194 IP 10.0.0.40.2516012294 > 10.0.0.30.2049: 108 getattr fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439647219C8C
10:16:13.456361 IP 10.0.0.30.2049 > 10.0.0.40.2516012294: reply ok 116 getattr DIR 40755 ids 0/0 sz 4096
10:16:13.458282 IP 10.0.0.40.2532789510 > 10.0.0.30.2049: 116 lookup fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439600000004 "iso1"
10:16:13.458461 IP 10.0.0.30.2049 > 10.0.0.40.2532789510: reply ok 232 lookup fh Unknown/010006002C49FEF2BA4642939B2BF2B8322CCBCB0000000100000002000041ED
10:16:13.510637 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 6153 win 2184 <nop,nop,timestamp 753985 754662>
10:16:14.030110 IP 10.0.0.40.2549566726 > 10.0.0.30.2049: 112 access fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000001F 001f
10:16:14.030927 IP 10.0.0.30.2049 > 10.0.0.40.2549566726: reply ok 124 access c 0003
10:16:14.033436 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 6277 win 2184 <nop,nop,timestamp 754548 754805>
10:16:14.034732 IP 10.0.0.40.2566343942 > 10.0.0.30.2049: 108 getattr fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439600000000
10:16:14.034980 IP 10.0.0.30.2049 > 10.0.0.40.2566343942: reply ok 116 getattr DIR 40755 ids 0/0 sz 4096
10:16:14.037319 IP 10.0.0.40.2583121158 > 10.0.0.30.2049: 112 access fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000001F 001f
10:16:14.037486 IP 10.0.0.30.2049 > 10.0.0.40.2583121158: reply ok 124 access c 0003
10:16:14.040323 IP 10.0.0.40.2599898374 > 10.0.0.30.2049: 132 readdirplus fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439600000000 512 bytes @ 0
10:16:14.040554 IP 10.0.0.30.2049 > 10.0.0.40.2599898374: reply ok 1448 readdirplus
10:16:14.041020 IP 10.0.0.30.2049 > 10.0.0.40.1684108288: reply Unknown rpc response code=2021855861 340
10:16:14.043583 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 8305 win 2908 <nop,nop,timestamp 754550 754808>
10:16:14.045104 IP 10.0.0.40.2616675590 > 10.0.0.30.2049: 112 access fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000001F 001f
10:16:14.045402 IP 10.0.0.30.2049 > 10.0.0.40.2616675590: reply ok 124 access c 0003
10:16:14.047830 IP 10.0.0.40.2633452806 > 10.0.0.30.2049: 112 access fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000001F 001f
10:16:14.048039 IP 10.0.0.30.2049 > 10.0.0.40.2633452806: reply ok 124 access c 0003
10:16:14.099385 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 8553 win 2908 <nop,nop,timestamp 754592 754810>
10:16:14.293714 IP 10.0.0.40.2650230022 > 10.0.0.30.2049: 108 getattr fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C7439600000000
10:16:14.294019 IP 10.0.0.30.2049 > 10.0.0.40.2650230022: reply ok 116 getattr DIR 40755 ids 0/0 sz 4096
10:16:14.297263 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 8669 win 2908 <nop,nop,timestamp 754763 754871>
10:16:14.297272 IP 10.0.0.40.2667007238 > 10.0.0.30.2049: 112 access fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000001F 001f
10:16:14.297466 IP 10.0.0.30.2049 > 10.0.0.40.2667007238: reply ok 124 access c 0003
10:16:14.297586 IP 10.0.0.40.2683784454 > 10.0.0.30.2049: 112 access fh Unknown/01000700813801000000000008164EE42DB141EBAC96170149C743960000001F 001f
10:16:14.297987 IP 10.0.0.30.2049 > 10.0.0.40.2683784454: reply ok 124 access c 0003
10:16:14.301165 IP 10.0.0.40.2700561670 > 10.0.0.30.2049: 104 access fh Unknown/010006002C49FEF2BA4642939B2BF2B8322CCBCB0000001F47219C8C00000000 001f
10:16:14.301405 IP 10.0.0.30.2049 > 10.0.0.40.2700561670: reply ok 124 access c 0003
10:16:14.351229 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 9041 win 2908 <nop,nop,timestamp 754810 754873>
10:16:14.842633 IP 10.0.0.40.2717338886 > 10.0.0.30.2049: 100 getattr fh Unknown/010006002C49FEF2BA4642939B2BF2B8322CCBCB0000000047219C8C00000000
10:16:14.851711 IP 10.0.0.30.2049 > 10.0.0.40.2717338886: reply ok 116 getattr DIR 40755 ids 0/0 sz 4096
10:16:14.856846 IP 10.0.0.40.1022 > 10.0.0.30.2049: . ack 9157 win 2908 <nop,nop,timestamp 755272 755008>
> > does somebody have such setup up and running and can tell his distro / kernel and nfs-utils version ?
> > maybe i change distro then.
>
> I doubt that it is a distro-specific thing. As long as you have
> nfs-utils-1.1.0 it should work. I don't have a 10.3 box
> set up yet, but it works fine on Debian/unstable for me.
ok, will try this on debian.
> Maybe try adding the "no_root_squash" export option.
no difference
> What does "ls -l /export" on the server show?
nothing unusual. no errors, just the dirs/mountpoints
Thanks for your help!
regards
roland
>
> On Saturday October 27, devzero@web.de wrote:
> > Hello !
> >
> > with 2.6.22 i`m trying to export loopback mounted iso-images.
> >
> > this is /etc/exports:
> >
> > /export *(ro,crossmnt,subtree_check)
>
> I recommend replacing subtree_check with no_subtree_check, but it
> shouldn't make an important difference in this case.
>
>
> This should work with nfs-utils 1.1.0 or later. With earlier releases
> you need to explicitly export the subordinate filesystems too.
>
> >
> > in /export, i have loopback mounted iso-images
> >
> > after mounting on the client side under /mnt (tried one older and one recent system) , i`m getting:
> >
> > vmhost:/mnt # ls -la
> > /bin/ls: iso1: Input/output error
> > /bin/ls: iso2 Input/output error
> > /bin/ls: iso3: Input/output error
> > total 10128
> > drwxrwxrwt 18 root root 270336 Oct 26 08:45 .
> > drwxrwxrwt 186 root root 20760 Oct 27 17:45 ..
> > drwxr-xr-x 2 root root 16384 Jan 1 1970 iso1
> > drwxr-xr-x 2 root root 16384 Jan 1 1970 iso2
> > drwxr-xr-x 2 root root 16384 Jan 1 1970 iso3
> >
> > vmhost:/mnt/iso1 # ls
> > /bin/ls: .: Stale NFS file handle
> > vmhost:/mnt/iso1 # ls -la
> > /bin/ls: .: Input/output error
>
> It is a little odd that the errors are inconsistent.
>
> Can you find any log messages from mountd in syslog? What do they
> say?
> Also what does
> cat /proc/fs/nfsd/exports
>
> on the server show.
>
> Finally, a tcpdump:
>
> tcpdump -s 0 -w /tmp/tcpdump port 2049
>
> while you run the experiment might help.
>
> >
> > i`m unsure if i should blame suse here (it`s an opensuse 10.3 box which seems to have nfs-utils 1.1.0)
> >
> > does somebody have such setup up and running and can tell his distro / kernel and nfs-utils version ?
> > maybe i change distro then.
>
> I doubt that it is a distro-specific thing. As long as you have
> nfs-utils-1.1.0 it should work. I don't have a 10.3 box
> set up yet, but it works fine on Debian/unstable for me.
>
> Maybe try adding the "no_root_squash" export option.
> What does "ls -l /export" on the server show?
>
> NeilBrown
>
_________________________________________________________________________
In 5 Schritten zur eigenen Homepage. Jetzt Domain sichern und gestalten!
Nur 3,99 EUR/Monat! http://www.maildomain.web.de/?mc=021114
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs
^ permalink raw reply [flat|nested] 17+ messages in thread
* stale nfs file handle with exported loopback mounts
@ 2007-10-27 16:13 devzero
2007-10-30 5:14 ` Neil Brown
0 siblings, 1 reply; 17+ messages in thread
From: devzero @ 2007-10-27 16:13 UTC (permalink / raw)
To: NFS
Hello !
with 2.6.22 i`m trying to export loopback mounted iso-images.
this is /etc/exports:
/export *(ro,crossmnt,subtree_check)
in /export, i have loopback mounted iso-images
after mounting on the client side under /mnt (tried one older and one recent system) , i`m getting:
vmhost:/mnt # ls -la
/bin/ls: iso1: Input/output error
/bin/ls: iso2 Input/output error
/bin/ls: iso3: Input/output error
total 10128
drwxrwxrwt 18 root root 270336 Oct 26 08:45 .
drwxrwxrwt 186 root root 20760 Oct 27 17:45 ..
drwxr-xr-x 2 root root 16384 Jan 1 1970 iso1
drwxr-xr-x 2 root root 16384 Jan 1 1970 iso2
drwxr-xr-x 2 root root 16384 Jan 1 1970 iso3
vmhost:/mnt/iso1 # ls
/bin/ls: .: Stale NFS file handle
vmhost:/mnt/iso1 # ls -la
/bin/ls: .: Input/output error
i`m unsure if i should blame suse here (it`s an opensuse 10.3 box which seems to have nfs-utils 1.1.0)
does somebody have such setup up and running and can tell his distro / kernel and nfs-utils version ?
maybe i change distro then.
any clues what`s going wrong here and how to resolve ?
regards
roland
_____________________________________________________________________
Der WEB.DE SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen!
http://smartsurfer.web.de/?mc=100071&distributionid=000000000066
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: stale nfs file handle with exported loopback mounts
2007-10-27 16:13 devzero
@ 2007-10-30 5:14 ` Neil Brown
0 siblings, 0 replies; 17+ messages in thread
From: Neil Brown @ 2007-10-30 5:14 UTC (permalink / raw)
To: devzero; +Cc: NFS
On Saturday October 27, devzero@web.de wrote:
> Hello !
>
> with 2.6.22 i`m trying to export loopback mounted iso-images.
>
> this is /etc/exports:
>
> /export *(ro,crossmnt,subtree_check)
I recommend replacing subtree_check with no_subtree_check, but it
shouldn't make an important difference in this case.
This should work with nfs-utils 1.1.0 or later. With earlier releases
you need to explicitly export the subordinate filesystems too.
>
> in /export, i have loopback mounted iso-images
>
> after mounting on the client side under /mnt (tried one older and one recent system) , i`m getting:
>
> vmhost:/mnt # ls -la
> /bin/ls: iso1: Input/output error
> /bin/ls: iso2 Input/output error
> /bin/ls: iso3: Input/output error
> total 10128
> drwxrwxrwt 18 root root 270336 Oct 26 08:45 .
> drwxrwxrwt 186 root root 20760 Oct 27 17:45 ..
> drwxr-xr-x 2 root root 16384 Jan 1 1970 iso1
> drwxr-xr-x 2 root root 16384 Jan 1 1970 iso2
> drwxr-xr-x 2 root root 16384 Jan 1 1970 iso3
>
> vmhost:/mnt/iso1 # ls
> /bin/ls: .: Stale NFS file handle
> vmhost:/mnt/iso1 # ls -la
> /bin/ls: .: Input/output error
It is a little odd that the errors are inconsistent.
Can you find any log messages from mountd in syslog? What do they
say?
Also what does
cat /proc/fs/nfsd/exports
on the server show.
Finally, a tcpdump:
tcpdump -s 0 -w /tmp/tcpdump port 2049
while you run the experiment might help.
>
> i`m unsure if i should blame suse here (it`s an opensuse 10.3 box which seems to have nfs-utils 1.1.0)
>
> does somebody have such setup up and running and can tell his distro / kernel and nfs-utils version ?
> maybe i change distro then.
I doubt that it is a distro-specific thing. As long as you have
nfs-utils-1.1.0 it should work. I don't have a 10.3 box
set up yet, but it works fine on Debian/unstable for me.
Maybe try adding the "no_root_squash" export option.
What does "ls -l /export" on the server show?
NeilBrown
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs
^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2007-11-10 15:14 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2007-10-31 22:50 stale nfs file handle with exported loopback mounts devzero
2007-11-01 4:26 ` Neil Brown
-- strict thread matches above, loose matches on Subject: below --
2007-11-10 15:14 devzero
2007-11-02 19:37 devzero
2007-11-02 19:42 ` J. Bruce Fields
2007-11-04 20:30 ` J. Bruce Fields
2007-11-05 9:59 ` Andreas Gruenbacher
2007-11-02 19:06 devzero
2007-11-02 19:23 ` J. Bruce Fields
2007-11-02 19:24 ` J. Bruce Fields
2007-10-31 22:19 devzero
2007-10-31 22:39 ` J. Bruce Fields
2007-10-31 20:46 devzero
2007-10-31 20:57 ` J. Bruce Fields
2007-10-30 20:05 devzero
2007-10-27 16:13 devzero
2007-10-30 5:14 ` Neil Brown
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.