* [PATCH] nfs-utils: mountd: empty export list should not be treated as a failure
@ 2008-02-12 5:13 Harshula
[not found] ` <1202793191.2890.36.camel-2WabGjdRN2LRvmHwrWB8BmjR7Gm6iKkz0E9HWUfgJXw@public.gmane.org>
0 siblings, 1 reply; 6+ messages in thread
From: Harshula @ 2008-02-12 5:13 UTC (permalink / raw)
To: Steve Dickson; +Cc: NFS list
Hi Steve,
This patch is against:
git://git.linux-nfs.org/projects/steved/nfs-utils.git
In mountd, if get_exportlist() (utils/mountd/mountd.c) returns NULL it
should not be considered a failure. It just means that there are no
exports on the system.
The practical problem with the current code is that a showmount -e
results in a syslog message from mountd that looks like:
rpc.mountd: export request from 10.250.100.2 failed.
References: SGI: PV977213
Reviewed-by: Greg Banks <gnb@sgi.com>
Signed-off-by: Harshula Jayasuriya <harshula@sgi.com>
---
utils/mountd/mountd.c | 11 ++++-------
1 file changed, 4 insertions(+), 7 deletions(-)
--- a/utils/mountd/mountd.c
+++ b/utils/mountd/mountd.c
@@ -254,10 +254,8 @@ mount_export_1_svc(struct svc_req *rqstp, void
*argp, exports *resp)
struct sockaddr_in *addr =
(struct sockaddr_in *) svc_getcaller(rqstp->rq_xprt);
- if ((*resp = get_exportlist()) == NULL)
- xlog(L_WARNING, "export request from %s failed.",
- inet_ntoa(addr->sin_addr));
-
+ xlog(D_CALL, "EXPORT1 request from %s.",
inet_ntoa(addr->sin_addr));
+ *resp = get_exportlist();
return 1;
}
@@ -267,9 +265,8 @@ mount_exportall_1_svc(struct svc_req *rqstp, void
*argp, exports *resp)
struct sockaddr_in *addr =
(struct sockaddr_in *) svc_getcaller(rqstp->rq_xprt);
- if ((*resp = get_exportlist()) == NULL)
- xlog(L_WARNING, "exportall request from %s failed.",
- inet_ntoa(addr->sin_addr));
+ xlog(D_CALL, "EXPORTALL1 request from %s.",
inet_ntoa(addr->sin_addr));
+ *resp = get_exportlist();
return 1;
}
cya,
#
^ permalink raw reply [flat|nested] 6+ messages in thread[parent not found: <1202793191.2890.36.camel-2WabGjdRN2LRvmHwrWB8BmjR7Gm6iKkz0E9HWUfgJXw@public.gmane.org>]
* nfs performance dropped when using 128 MB RAM [not found] ` <1202793191.2890.36.camel-2WabGjdRN2LRvmHwrWB8BmjR7Gm6iKkz0E9HWUfgJXw@public.gmane.org> @ 2008-02-12 8:33 ` Sagar Borikar [not found] ` <340C71CD25A7EB49BFA81AE8C8392667013239CA-WnaG5mLaJUuZWFWj0wbTyA@public.gmane.org_nt.nt.pmc-sierra.bc.ca> 2008-02-12 21:17 ` [PATCH] nfs-utils: mountd: empty export list should not be treated as a failure Steve Dickson 1 sibling, 1 reply; 6+ messages in thread From: Sagar Borikar @ 2008-02-12 8:33 UTC (permalink / raw) To: linux-nfs [-- Attachment #1: Type: text/plain, Size: 2747 bytes --] Hello Folks, I am not sure if this is the right place for this question. If not, I am sorry and request you to direct me to right mailing list. Currently I am working on a product which is derived from its ancestor with more or less similar feature set and few additions. Other than memory size, there are not hardware changes. The newer one is having 128 MB RAM while the ancestor was having 512 MB of RAM. The functionality carried forward is also similar except for nfs. Both have gigabit Ethernet interfaces I am finding that nfs write performance has drastically reduced say 1 MB/s or even it stalls for large file transfers to the box. On the contrary it works fine with 512 MB of RAM. The CPU utilization by nfsd as well as rpc.statd / rpc.mountd is also non existent. Rather nfsd most of the time is sleeping for the data. While trying to analyse through ethereal, I observe that the write requests by clients to the the server are periodic say around 1.5-2 minutes interval. Till then the tcpdump captures that the handshake is going on only for ack messages for wait requests. After then suddenly the burst write transfer starts and within 4-5 seconds when the buffers are full and no free memory is available, again it repeats the same cycle. When poking in the meminfo stats, I found that there are no dirty pages and write back buffers available before write bursts. As soon as the buffers are available, nfs sends requests for further data and meminfo stats are as it is. The free memory which is LowFree memory used by kernel data structures in our case as High free memory is zero, is very less when the buffers are not cleaned up. Once the burst data transfer finishes, cached memory equal to that of previous buffered gets cleared as expected. But Inactive memory is still high. I am attaching the tcpdump logs with this mail. I am using rsize / wsize as 64k. Tried various combinations of these sizes while mounting the nfs clients but it didn't help. I tried to check whether this problem is system wide but unfortunately CIFS and FTP writes are pretty fast. so only nfs performance is affected due to RAM size reduction. We have enabled socket buffer recycling in Ethernet driver which block 600k of data for recycling. Even just to get confirmation, I disabled the skb recycling, even then the result was same. I am not seeing this kind of behavior in other workstations with 128 MB of RAM and mips architecture. Can anyone let me know, why the write requests are buffered and periodically they get cleared? Also does nfs stack, requires special buffer availability for succeeding write requests? Thanks and Regards, Sagar Borikar PMC sierra [-- Attachment #2: tcpdump.log --] [-- Type: application/octet-stream, Size: 7937 bytes --] Mstening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes 11:12:35.674988 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 8161140 win 2896 <nop,nop,timestamp 18958133 515936722> 11:12:35.675005 IP 172.16.48.52.166575521 > 172.16.48.130.nfs: 2896 proc-2379056868 11:12:35.675013 IP 172.16.48.52.415833216 > 172.16.48.130.nfs: 1448 proc-3037336800 11:12:35.675394 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 8164036 win 2896 <nop,nop,timestamp 18958133 515936722> 11:12:35.675411 IP 172.16.48.52.2455467960 > 172.16.48.130.nfs: 1448 proc-3812267133 11:12:35.675419 IP 172.16.48.52.26272008 > 172.16.48.130.nfs: 1448 proc-3094064178 11:12:35.675636 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 8166932 win 2896 <nop,nop,timestamp 18958133 515936723> 11:12:35.675652 IP 172.16.48.52.817467000 > 172.16.48.130.nfs: 1448 proc-1456776907 11:12:35.675660 IP 172.16.48.52.3578191046 > 172.16.48.130.nfs: 1448 proc-1902769939 11:12:35.675924 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 8169828 win 2896 <nop,nop,timestamp 18958133 515936723> 11:12:35.675940 IP 172.16.48.52.3721596643 > 172.16.48.130.nfs: 1448 proc-974205282 11:12:35.675947 IP 172.16.48.52.2604580239 > 172.16.48.130.nfs: 1448 proc-1590512167 11:12:35.676169 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 8172724 win 2896 <nop,nop,timestamp 18958133 515936723> 11:12:35.676185 IP 172.16.48.52.4278242473 > 172.16.48.130.nfs: 1448 proc-3126856275 11:12:35.676193 IP 172.16.48.52.2419163184 > 172.16.48.130.nfs: 1448 proc-1815905253 11:12:35.676877 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 8175620 win 2896 <nop,nop,timestamp 18958134 515936724> 11:12:35.676896 IP 172.16.48.52.528592223 > 172.16.48.130.nfs: 1448 proc-3790581237 11:12:35.676904 IP 172.16.48.52.674712662 > 172.16.48.130.nfs: 1448 proc-1923312982 11:12:35.676913 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 8178516 win 2896 <nop,nop,timestamp 18958134 515936724> 11:12:35.676928 IP 172.16.48.52.2019061448 > 172.16.48.130.nfs: 1448 proc-3786701257 11:12:35.676935 IP 172.16.48.52.3938107219 > 172.16.48.130.nfs: 1448 proc-477133596 11:12:35.677420 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 8181412 win 2896 <nop,nop,timestamp 18958134 515936725> 11:12:35.677439 IP 172.16.48.52.3303078522 > 172.16.48.130.nfs: 1448 proc-416174347 11:12:35.677447 IP 172.16.48.52.992055196 > 172.16.48.130.nfs: 1448 proc-3167533369 11:12:35.677659 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 8184308 win 2896 <nop,nop,timestamp 18958134 515936725> 11:12:35.677675 IP 172.16.48.52.3782641007 > 172.16.48.130.nfs: 1448 proc-3744596530 11:12:35.677683 IP 172.16.48.52.2800514754 > 172.16.48.130.nfs: 1448 proc-1632170636 11:12:35.677956 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 8187204 win 2896 <nop,nop,timestamp 18958134 515936725> 11:12:35.677972 IP 172.16.48.52.2920219751 > 172.16.48.130.nfs: 1448 proc-167343746 11:12:35.677980 IP 172.16.48.52.1216585728 > 172.16.48.130.nfs: 1448 proc-2784742365 11:12:35.678228 IP 172.16.48.130.nfs > 172.16.48.52.2116931643: reply ok 140 11:12:35.678245 IP 172.16.48.52.798423794 > 172.16.48.130.nfs: 1448 proc-1298995762 11:12:35.678863 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 8194444 win 2896 <nop,nop,timestamp 18958134 515936725> 11:12:35.678880 IP 172.16.48.52.1705633946 > 172.16.48.130.nfs: 2896 proc-2178937653 11:12:35.678888 IP 172.16.48.52.865072077 > 172.16.48.130.nfs: 2896 proc-3935072703 11:12:35.679395 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 8197340 win 2896 <nop,nop,timestamp 18958134 515936727> 11:12:35.679412 IP 172.16.48.52.871577439 > 172.16.48.130.nfs: 2896 proc-3341172484 11:12:35.679640 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 8200236 win 2896 <nop,nop,timestamp 18958134 515936727> 11:12:35.679655 IP 172.16.48.52.1517075806 > 172.16.48.130.nfs: 2896 proc-2824690568 11:12:35.679925 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 8203132 win 2896 <nop,nop,timestamp 18958134 515936727> 11:12:35.679942 IP 172.16.48.52.2451863779 > 172.16.48.130.nfs: 2896 proc-2104363821 11:12:35.680173 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 8206028 win 2896 <nop,nop,timestamp 18958134 515936727> 11:12:35.680188 IP 172.16.48.52.2425756098 > 172.16.48.130.nfs: 2896 proc-4117019690 11:12:35.680777 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 8208924 win 2896 <nop,nop,timestamp 18958135 515936728> 11:12:35.680794 IP 172.16.48.52.2944274149 > 172.16.48.130.nfs: 2896 proc-2117355039 11:12:35.680798 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 8211820 win 2896 <nop,nop,timestamp 18958135 515936728> 11:12:35.680809 IP 172.16.48.52.1728566045 > 172.16.48.130.nfs: 2896 proc-2220972509 11:12:35.681317 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 8214716 win 2896 <nop,nop,timestamp 18958135 515936729> 11:12:35.681333 IP 172.16.48.52.1125632345 > 172.16.48.130.nfs: 2896 proc-1984113115 11:12:35.681554 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 8217612 win 2896 <nop,nop,timestamp 18958135 515936729> 11:12:35.681570 IP 172.16.48.52.2052363586 > 172.16.48.130.nfs: 2896 proc-3058647012 11:12:35.681851 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 8220508 win 2896 <nop,nop,timestamp 18958135 515936729> 11:12:35.681867 IP 172.16.48.52.3169922154 > 172.16.48.130.nfs: 2896 proc-1693529181 11:12:35.682000 IP 172.16.48.130.nfs > 172.16.48.52.2133708859: reply ok 140 11:12:35.682601 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 8226300 win 2896 <nop,nop,timestamp 18958135 515936729> 11:12:35.682617 IP 172.16.48.52.206440654 > 172.16.48.130.nfs: 2896 proc-2400289967 11:12:35.682625 IP 172.16.48.52.2681648301 > 172.16.48.130.nfs: 2896 proc-969898632 11:12:35.683132 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 8229196 win 2896 <nop,nop,timestamp 18958135 515936730> 11:12:35.683149 IP 172.16.48.52.166182463 > 172.16.48.130.nfs: 2896 proc-4137940490 11:12:35.683379 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 8232092 win 2896 <nop,nop,timestamp 18958135 515936730> 11:12:35.683395 IP 172.16.48.52.870457670 > 172.16.48.130.nfs: 2896 proc-460917824 11:12:35.683663 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 8234988 win 2896 <nop,nop,timestamp 18958135 515936731> 11:12:35.683678 IP 172.16.48.52.921558232 > 172.16.48.130.nfs: 2896 proc-1260781838 11:12:35.683905 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 8237884 win 2896 <nop,nop,timestamp 18958135 515936731> 11:12:35.683922 IP 172.16.48.52.2119784769 > 172.16.48.130.nfs: 2896 proc-4255658928 11:12:35.684570 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 8240780 win 2896 <nop,nop,timestamp 18958136 515936731> 11:12:35.684586 IP 172.16.48.52.2604214023 > 172.16.48.130.nfs: 2896 proc-4039840750 11:12:35.684594 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 8243676 win 2896 <nop,nop,timestamp 18958136 515936732> 11:12:35.684607 IP 172.16.48.52.2031761012 > 172.16.48.130.nfs: 2896 proc-4195605192 11:12:35.685115 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 8246572 win 2896 <nop,nop,timestamp 18958136 515936732> 11:12:35.685133 IP 172.16.48.52.3846229746 > 172.16.48.130.nfs: 2896 proc-2380498847 11:12:44.855773 IP 172.16.48.52.884 > 172.16.48.130.nfs: . ack 3396395833 win 501 <nop,nop,timestamp 515945904 18930424> 11:12:44.856003 IP 172.16.48.130.nfs > 172.16.48.52.884: . ack 1 win 0 <nop,nop,timestamp 18960429 515619303> 11:12:45.227822 IP 172.16.48.52.879 > 172.16.48.130.nfs: . ack 4074022534 win 729 <nop,nop,timestamp 515946276 18959710> 11:12:45.228049 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 1 win 0 <nop,nop,timestamp 18960522 515939986> 11:12:51.128765 IP 172.16.48.52.882 > 172.16.48.130.nfs: . ack 3871614401 win 501 <nop,nop,timestamp 515952177 18936137> 11:12:51.128972 IP 172.16.48.130.nfs > 172.16.48.52.882: . ack 1 win 0 <nop,nop,timestamp 18961997 515745571> 11:12:51.723781 IP 172.16.48.52.879 > 172.16.48.130.nfs: . ack 1 win 729 <nop,nop,timestamp 515952772 18960522> 11:12:51.723948 IP 172.16.48.130.nfs > 172.16.48.52.879: . ack 1 win 0 <nop,nop,timestamp 18962146 515939986> ^ permalink raw reply [flat|nested] 6+ messages in thread
[parent not found: <340C71CD25A7EB49BFA81AE8C8392667013239CA-WnaG5mLaJUuZWFWj0wbTyA@public.gmane.org_nt.nt.pmc-sierra.bc.ca>]
* Re: nfs performance dropped when using 128 MB RAM [not found] ` <340C71CD25A7EB49BFA81AE8C8392667013239CA-WnaG5mLaJUuZWFWj0wbTyA@public.gmane.org_nt.nt.pmc-sierra.bc.ca> @ 2008-02-12 18:25 ` J. Bruce Fields 2008-02-13 0:02 ` Sagar Borikar 2008-02-22 3:50 ` Sagar Borikar 1 sibling, 1 reply; 6+ messages in thread From: J. Bruce Fields @ 2008-02-12 18:25 UTC (permalink / raw) To: Sagar Borikar; +Cc: linux-nfs On Tue, Feb 12, 2008 at 12:33:41AM -0800, Sagar Borikar wrote: > Hello Folks, > > I am not sure if this is the right place for this question. If not, I am > sorry and request you to direct me to right mailing list. This is the right list, but you're replying to an unrelated message, which confuses those of us with threaded mail readers--no big deal, just please be more careful about that next time. > Currently I am working on a product which is derived from its ancestor > with more or less similar feature set and few additions. Other than > memory size, there are not hardware changes. The newer one is having 128 > MB RAM while the ancestor was having 512 MB of RAM. The functionality > carried forward is also similar except for nfs. Both have gigabit > Ethernet interfaces This is a server, not a client? What kernel are you using? --b. > I am finding that nfs write performance has drastically > reduced say 1 MB/s or even it stalls for large file transfers to the > box. On the contrary it works fine with 512 MB of RAM. The CPU > utilization by nfsd as well as rpc.statd / rpc.mountd is also non > existent. Rather nfsd most of the time is sleeping for the data. While > trying to analyse through ethereal, I observe that the write requests by > clients to the the server are periodic say around 1.5-2 minutes > interval. Till then the tcpdump captures that the handshake is going on > only for ack messages for wait requests. After then suddenly the burst > write transfer starts and within 4-5 seconds when the buffers are full > and no free memory is available, again it repeats the same cycle. When > poking in the meminfo stats, I found that there are no dirty pages and > write back buffers available before write bursts. As soon as the buffers > are available, nfs sends requests for further data and meminfo stats are > as it is. The free memory which is LowFree memory used by kernel data > structures in our case as High free memory is zero, is very less when > the buffers are not cleaned up. Once the burst data transfer finishes, > cached memory equal to that of previous buffered gets cleared as > expected. But Inactive memory is still high. I am attaching the tcpdump > logs with this mail. I am using rsize / wsize as 64k. Tried various > combinations of these sizes while mounting the nfs clients but it didn't > help. > I tried to check whether this problem is system wide but unfortunately > CIFS and FTP writes are pretty fast. so only nfs performance is affected > due to RAM size reduction. We have enabled socket buffer recycling in > Ethernet driver which block 600k of data for recycling. Even just to get > confirmation, I disabled the skb recycling, even then the result was > same. I am not seeing this kind of behavior in other workstations with > 128 MB of RAM and mips architecture. > Can anyone let me know, why the write requests are buffered and > periodically they get cleared? > Also does nfs stack, requires special buffer availability for succeeding > write requests? > > Thanks and Regards, > Sagar Borikar > PMC sierra > > ^ permalink raw reply [flat|nested] 6+ messages in thread
* RE: nfs performance dropped when using 128 MB RAM 2008-02-12 18:25 ` J. Bruce Fields @ 2008-02-13 0:02 ` Sagar Borikar 0 siblings, 0 replies; 6+ messages in thread From: Sagar Borikar @ 2008-02-13 0:02 UTC (permalink / raw) To: J. Bruce Fields; +Cc: linux-nfs On Tue, Feb 12, 2008 at 12:33:41AM -0800, Sagar Borikar wrote: > Hello Folks, > > I am not sure if this is the right place for this question. If not, I am > sorry and request you to direct me to right mailing list. This is the right list, but you're replying to an unrelated message, which confuses those of us with threaded mail readers--no big deal, just please be more careful about that next time. > Currently I am working on a product which is derived from its ancestor > with more or less similar feature set and few additions. Other than > memory size, there are not hardware changes. The newer one is having 128 > MB RAM while the ancestor was having 512 MB of RAM. The functionality > carried forward is also similar except for nfs. Both have gigabit > Ethernet interfaces This is a server, not a client? What kernel are you using? <Sagar> That's right. It is server and kernel is 2.6.18. I would appreciate any pointers / directions with this regard. Thanks > I am finding that nfs write performance has drastically > reduced say 1 MB/s or even it stalls for large file transfers to the > box. On the contrary it works fine with 512 MB of RAM. The CPU > utilization by nfsd as well as rpc.statd / rpc.mountd is also non > existent. Rather nfsd most of the time is sleeping for the data. While > trying to analyse through ethereal, I observe that the write requests by > clients to the the server are periodic say around 1.5-2 minutes > interval. Till then the tcpdump captures that the handshake is going on > only for ack messages for wait requests. After then suddenly the burst > write transfer starts and within 4-5 seconds when the buffers are full > and no free memory is available, again it repeats the same cycle. When > poking in the meminfo stats, I found that there are no dirty pages and > write back buffers available before write bursts. As soon as the buffers > are available, nfs sends requests for further data and meminfo stats are > as it is. The free memory which is LowFree memory used by kernel data > structures in our case as High free memory is zero, is very less when > the buffers are not cleaned up. Once the burst data transfer finishes, > cached memory equal to that of previous buffered gets cleared as > expected. But Inactive memory is still high. I am attaching the tcpdump > logs with this mail. I am using rsize / wsize as 64k. Tried various > combinations of these sizes while mounting the nfs clients but it didn't > help. > I tried to check whether this problem is system wide but unfortunately > CIFS and FTP writes are pretty fast. so only nfs performance is affected > due to RAM size reduction. We have enabled socket buffer recycling in > Ethernet driver which block 600k of data for recycling. Even just to get > confirmation, I disabled the skb recycling, even then the result was > same. I am not seeing this kind of behavior in other workstations with > 128 MB of RAM and mips architecture. > Can anyone let me know, why the write requests are buffered and > periodically they get cleared? > Also does nfs stack, requires special buffer availability for succeeding > write requests? > > Thanks and Regards, > Sagar Borikar > PMC sierra > > ^ permalink raw reply [flat|nested] 6+ messages in thread
* RE: nfs performance dropped when using 128 MB RAM [not found] ` <340C71CD25A7EB49BFA81AE8C8392667013239CA-WnaG5mLaJUuZWFWj0wbTyA@public.gmane.org_nt.nt.pmc-sierra.bc.ca> 2008-02-12 18:25 ` J. Bruce Fields @ 2008-02-22 3:50 ` Sagar Borikar 1 sibling, 0 replies; 6+ messages in thread From: Sagar Borikar @ 2008-02-22 3:50 UTC (permalink / raw) To: linux-nfs Folks, Ok, I was able to crack the issue but in some odd way. I found that pdflush was not able to flush the buffers in time because of which the next write requests were stalled at the client side. I did following changes to improve the performance. 1. Changed the number of pdflush threads from default 2 to 4 2. Changed the CACHESIZE from 1024 to 8192 in fs/nfsd/nfscache.c 3. Changed SVC block size from 32k to 64k in include/linux/nfsd/const.h After these changes, as expected I see much better performance but then there is overshoot of at the start with less write size and later it decreases. I captured the iozonelinux figures and they are as below Excel output is below: "Writer report" "4" "8" "16" "32" "64" "65536" 1006667 1414856 1437128 1606784 1580778 "131072" 801006 1427892 1232945 1191835 1232504 "262144" 8555 16856 10093 16721 12607 "524288" 8136 13017 8419 11485 16160 "1048576" 6546 9635 10923 10611 10995 "Re-writer report" "4" "8" "16" "32" "64" "65536" 1323668 1512831 1645809 1757521 1792708 "131072" 1119661 1299879 1367497 1423008 1427222 "262144" 3290 4173 3494 4158 4969 "524288" 3268 3398 3200 2899 3570 "1048576" 3126 3183 3646 3231 3890 "Reader report" "4" "8" "16" "32" "64" "65536" 10790 11167 11123 10842 9409 "131072" 9270 10640 9084 9480 10958 "262144" 10136 11011 10881 8924 8911 "524288" 10947 9303 8567 9290 10853 "1048576" 10290 9499 10956 11156 11063 "Re-Reader report" "4" "8" "16" "32" "64" "65536" 1770290 591736 686075 240800 236228 "131072" 753296 1219558 858200 1142548 1080169 "262144" 611654 308045 521012 403101 854802 "524288" 10834 9075 10613 9219 2725002 "1048576" 1410151 2341711 2635044 2869093 3004842 "Writer CPU utilization report (Zero values should be ignored)" "4" "8" "16" "32" "64" "65536" 0.79 0.44 0.71 0.56 0.57 "131072" 0.75 0.78 0.57 0.67 0.72 "262144" 0.61 0.55 0.56 0.61 0.56 "524288" 0.68 0.58 0.59 0.26 0.59 "1048576" 0.61 0.68 0.67 0.68 0.68 "Re-writer CPU utilization report (Zero values should be ignored)" "4" "8" "16" "32" "64" "65536" 0.70 0.66 0.59 0.52 0.70 "131072" 0.29 0.24 0.20 0.21 0.20 "262144" 0.25 0.22 0.19 0.20 0.20 "524288" 0.26 0.25 0.22 0.20 0.21 "1048576" 0.27 0.23 0.21 0.20 0.20 "Reader CPU utilization report (Zero values should be ignored)" "4" "8" "16" "32" "64" "65536" 1.17 0.92 1.14 0.93 0.75 "131072" 1.00 1.17 0.80 0.79 1.07 "262144" 1.17 1.11 0.88 0.73 0.77 "524288" 1.19 0.90 0.77 0.78 0.84 "1048576" 1.13 0.91 0.99 0.94 0.87 "Re-Reader CPU utilization report (Zero values should be ignored)" "4" "8" "16" "32" "64" "65536" 97.38 27.73 28.08 10.25 8.97 "131072" 41.15 57.14 33.57 43.21 44.13 "262144" 33.04 14.79 22.96 15.20 32.82 "524288" 1.15 0.81 0.93 0.75 93.97 "1048576" 71.14 99.75 99.18 99.22 99.61 I am surprised to see that write performance is better than read and more interestingly re-write performance is degraded compared to write performance. Any cluses on this anomaly? Thanks in advance Sagar -----Original Message----- From: linux-nfs-owner@vger.kernel.org [mailto:linux-nfs-owner@vger.kernel.org] On Behalf Of Sagar Borikar Sent: Tuesday, February 12, 2008 2:04 PM To: linux-nfs@vger.kernel.org Subject: nfs performance dropped when using 128 MB RAM Hello Folks, I am not sure if this is the right place for this question. If not, I am sorry and request you to direct me to right mailing list. Currently I am working on a product which is derived from its ancestor with more or less similar feature set and few additions. Other than memory size, there are not hardware changes. The newer one is having 128 MB RAM while the ancestor was having 512 MB of RAM. The functionality carried forward is also similar except for nfs. Both have gigabit Ethernet interfaces I am finding that nfs write performance has drastically reduced say 1 MB/s or even it stalls for large file transfers to the box. On the contrary it works fine with 512 MB of RAM. The CPU utilization by nfsd as well as rpc.statd / rpc.mountd is also non existent. Rather nfsd most of the time is sleeping for the data. While trying to analyse through ethereal, I observe that the write requests by clients to the the server are periodic say around 1.5-2 minutes interval. Till then the tcpdump captures that the handshake is going on only for ack messages for wait requests. After then suddenly the burst write transfer starts and within 4-5 seconds when the buffers are full and no free memory is available, again it repeats the same cycle. When poking in the meminfo stats, I found that there are no dirty pages and write back buffers available before write bursts. As soon as the buffers are available, nfs sends requests for further data and meminfo stats are as it is. The free memory which is LowFree memory used by kernel data structures in our case as High free memory is zero, is very less when the buffers are not cleaned up. Once the burst data transfer finishes, cached memory equal to that of previous buffered gets cleared as expected. But Inactive memory is still high. I am attaching the tcpdump logs with this mail. I am using rsize / wsize as 64k. Tried various combinations of these sizes while mounting the nfs clients but it didn't help. I tried to check whether this problem is system wide but unfortunately CIFS and FTP writes are pretty fast. so only nfs performance is affected due to RAM size reduction. We have enabled socket buffer recycling in Ethernet driver which block 600k of data for recycling. Even just to get confirmation, I disabled the skb recycling, even then the result was same. I am not seeing this kind of behavior in other workstations with 128 MB of RAM and mips architecture. Can anyone let me know, why the write requests are buffered and periodically they get cleared? Also does nfs stack, requires special buffer availability for succeeding write requests? Thanks and Regards, Sagar Borikar PMC sierra ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] nfs-utils: mountd: empty export list should not be treated as a failure [not found] ` <1202793191.2890.36.camel-2WabGjdRN2LRvmHwrWB8BmjR7Gm6iKkz0E9HWUfgJXw@public.gmane.org> 2008-02-12 8:33 ` nfs performance dropped when using 128 MB RAM Sagar Borikar @ 2008-02-12 21:17 ` Steve Dickson 1 sibling, 0 replies; 6+ messages in thread From: Steve Dickson @ 2008-02-12 21:17 UTC (permalink / raw) To: Harshula; +Cc: linux-nfs Harshula wrote: > In mountd, if get_exportlist() (utils/mountd/mountd.c) returns NULL it > should not be considered a failure. It just means that there are no > exports on the system. > > The practical problem with the current code is that a showmount -e > results in a syslog message from mountd that looks like: > > rpc.mountd: export request from 10.250.100.2 failed. Committed... thanks! steved. ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2008-02-22 3:50 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2008-02-12 5:13 [PATCH] nfs-utils: mountd: empty export list should not be treated as a failure Harshula
[not found] ` <1202793191.2890.36.camel-2WabGjdRN2LRvmHwrWB8BmjR7Gm6iKkz0E9HWUfgJXw@public.gmane.org>
2008-02-12 8:33 ` nfs performance dropped when using 128 MB RAM Sagar Borikar
[not found] ` <340C71CD25A7EB49BFA81AE8C8392667013239CA-WnaG5mLaJUuZWFWj0wbTyA@public.gmane.org_nt.nt.pmc-sierra.bc.ca>
2008-02-12 18:25 ` J. Bruce Fields
2008-02-13 0:02 ` Sagar Borikar
2008-02-22 3:50 ` Sagar Borikar
2008-02-12 21:17 ` [PATCH] nfs-utils: mountd: empty export list should not be treated as a failure Steve Dickson
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.