From mboxrd@z Thu Jan 1 00:00:00 1970 From: Stanislav Kholmanskikh Date: Tue, 11 Oct 2016 15:03:33 +0300 Subject: [LTP] [PATCH] lsmod01: parse a copy of /proc/modules In-Reply-To: <2063556369.71871.1472485781200.JavaMail.zimbra@redhat.com> References: <1472468916-13152-1-git-send-email-stanislav.kholmanskikh@oracle.com> <20160829125050.GD30021@rei.lan> <57C43307.8000709@oracle.com> <8140813.24444.1472477694219.JavaMail.zimbra@redhat.com> <57C43FEF.50402@oracle.com> <57C440C0.5050307@oracle.com> <2063556369.71871.1472485781200.JavaMail.zimbra@redhat.com> Message-ID: <57FCD515.20202@oracle.com> List-Id: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: ltp@lists.linux.it Recovering an old thread. Please, see below: On 08/29/2016 06:49 PM, Jan Stancek wrote: > > > > ----- Original Message ----- >> From: "Stanislav Kholmanskikh" >> To: "Jan Stancek" >> Cc: "vasily isaenko" , ltp@lists.linux.it >> Sent: Monday, 29 August, 2016 4:03:44 PM >> Subject: Re: [LTP] [PATCH] lsmod01: parse a copy of /proc/modules >> >> >> >> On 08/29/2016 05:00 PM, Stanislav Kholmanskikh wrote: >>> >>> >>> On 08/29/2016 04:34 PM, Jan Stancek wrote: >>>> >>>> >>>> >>>> >>>> ----- Original Message ----- >>>>> From: "Stanislav Kholmanskikh" >>>>> To: "Cyril Hrubis" >>>>> Cc: "vasily isaenko" , ltp@lists.linux.it >>>>> Sent: Monday, 29 August, 2016 3:05:11 PM >>>>> Subject: Re: [LTP] [PATCH] lsmod01: parse a copy of /proc/modules >>>>> >>>>> >>>>> >>>>> On 08/29/2016 03:50 PM, Cyril Hrubis wrote: >>>>>> Hi! >>>>>>> In my environment, if TMPDIR is on NFSv4, this test case fails with: >>>>>>> >>>>>>> lsmod01 1 TFAIL : lsmod output different from /proc/modules. >>>>>>> 21c21 >>>>>>> < sunrpc 207591 28 >>>>>>> --- >>>>>>> > sunrpc 207591 29 >>>>>>> >>>>>>> To avoid such problems I separate the process of getting data from >>>>>>> /proc/modules and the process of parsing it in the pipe structure. >>>>>> >>>>>> So the sunrpc module gets its ref counter incremented from somewhere of >>>>>> the nfs kernel code once we open file on NFS? >>>>> >>>>> Looks so. I hava a share mounted from localhost: >>>>> >>>>> [root@skholman-m7 mnt]# mount|grep mnt >>>>> 127.0.0.1:/opt on /mnt type nfs >>>>> (rw,vers=4,addr=127.0.0.1,clientaddr=127.0.0.1) >>>>> [root@skholman-m7 mnt]# awk '{print $1, $2, $3}' /proc/modules|sort > >>>>> /tmp/not_nfs >>>>> [root@skholman-m7 mnt]# awk '{print $1, $2, $3}' /proc/modules|sort > nfs >>>>> [root@skholman-m7 mnt]# grep sunrpc nfs >>>>> sunrpc 207591 29 >>>>> [root@skholman-m7 mnt]# grep sunrpc /tmp/not_nfs >>>>> sunrpc 207591 28 >>>>> [root@skholman-m7 mnt]# >>>> >>>> And if you do that with just "cat /proc/modules", then there's no >>>> difference? >>>> Could it be that it's actually first write that takes extra ref? >>>> cat is reading in 65536 byte chunks for me, awk only 1024. >>> >>> Yes, there is no difference if I use "cat /proc/modules": >>> >>> [root@skholman-m7 mnt]# awk '{print $1, $2, $3}' /proc/modules|sort > nfs >>> [root@skholman-m7 mnt]# grep sunrpc nfs >>> sunrpc 207591 29 >>> [root@skholman-m7 mnt]# cat /proc/modules > temp >>> [root@skholman-m7 mnt]# awk '{print $1, $2, $3}' temp|sort > nfs >>> [root@skholman-m7 mnt]# grep sunrpc temp >>> sunrpc 207591 28 nfs,nfsd,lockd,nfs_acl,auth_rpcgss, Live >>> 0x00000000101ec000 >>> [root@skholman-m7 mnt]# >>> >>> As for 1024. lsmod also reads /proc/modules in 1024 bytes chunks. >> >> I suppose it's something related to using the pipe construction, since >> this change also "fixes" the issue: > > I'm suspecting this to be some kind of race between opening file on > nfs and reading /proc/modules at the same time: > > # sh -c "cat /proc/modules | cat > temp2"; grep sunrpc temp2 > sunrpc 300464 31 nfsv3,rpcsec_gss_krb5,nfsv4,nfs,nfsd,auth_rpcgss,nfs_acl,lockd, Live 0xffffffffa03a4000 > > # taskset -c 0 sh -c "cat /proc/modules | cat > temp2"; grep sunrpc temp2 > sunrpc 300464 30 nfsv3,rpcsec_gss_krb5,nfsv4,nfs,nfsd,auth_rpcgss,nfs_acl,lockd, Live 0xffffffffa03a4000 > > or > > # dd if=/proc/modules bs=1 | cat > temp2; grep sunrpc temp2 > 6332+0 records in > 6332+0 records out > 6332 bytes (6.3 kB) copied, 0.00288249 s, 2.2 MB/s > sunrpc 300464 30 nfsv3,rpcsec_gss_krb5,nfsv4,nfs,nfsd,auth_rpcgss,nfs_acl,lockd, Live 0xffffffffa03a4000 > > # dd if=/proc/modules bs=2 | cat > temp2; grep sunrpc temp2 > 3166+0 records in > 3166+0 records out > 6332 bytes (6.3 kB) copied, 0.00148174 s, 4.3 MB/s > sunrpc 300464 31 nfsv3,rpcsec_gss_krb5,nfsv4,nfs,nfsd,auth_rpcgss,nfs_acl,lockd, Live 0xffffffffa03a4000 > > or > > # cat /proc/modules | sh -c cat > temp2; grep sunrpc temp2 > sunrpc 300464 31 nfsv3,rpcsec_gss_krb5,nfsv4,nfs,nfsd,auth_rpcgss,nfs_acl,lockd, Live 0xffffffffa03a4000 > > # cat /proc/modules | sh -c "cat > temp2"; grep sunrpc temp2 > sunrpc 300464 30 nfsv3,rpcsec_gss_krb5,nfsv4,nfs,nfsd,auth_rpcgss,nfs_acl,lockd, Live 0xffffffffa03a4000 > Tried this all again. This is from a 4.1-based kernel in a VirtualBox host: [root@ol6-x64 mnt]# mount|grep mnt 127.0.0.1:/opt/ on /mnt type nfs (rw,vers=3,addr=127.0.0.1) [root@ol6-x64 mnt]# grep sunrpc /proc/modules sunrpc 329262 28 nfsv3,rpcsec_gss_krb5,nfsv4,nfs,nfsd,lockd,nfs_acl,auth_rpcgss, Live 0xffffffffa0366000 [root@ol6-x64 mnt]# dd if=/dev/zero of=file bs=512 While the above dd command is running, in a separate shell I see: [stas@ol6-x64 ~]$ lsmod|grep sunrpc sunrpc 329262 36 nfsv3,rpcsec_gss_krb5,nfsv4,nfs,nfsd,lockd,nfs_acl,auth_rpcgss [stas@ol6-x64 ~]$ [root@ol6-x64 mnt]# grep sunrpc /proc/modules sunrpc 329262 28 nfsv3,rpcsec_gss_krb5,nfsv4,nfs,nfsd,lockd,nfs_acl,auth_rpcgss, Live 0xffffffffa0366000 [root@ol6-x64 mnt]# dd if=/proc/modules bs=1 | cat > temp2; grep sunrpc temp2 2564+0 records in 2564+0 records out 2564 bytes (2.6 kB) copied, 0.00313203 s, 819 kB/s sunrpc 329262 29 nfsv3,rpcsec_gss_krb5,nfsv4,nfs,nfsd,lockd,nfs_acl,auth_rpcgss, Live 0xffffffffa0366000 [root@ol6-x64 mnt]# The same situation is with NFSv4. So it turns out that writing to a file on NFS increases the sunrpc module reference counter. I don't observe this with ext4.ko. So I find that my original patch is fine, it just needs a proper description. Thoughts?