[PATCH v1 0/2] _rpc_dtablesize: decrease to fix excessive memory usage

[PATCH v1 0/2] _rpc_dtablesize: decrease to fix excessive memory usage

[Zhuohao Bai]

In the client code, the function _rpc_dtablesize() is used to determine the
memory allocation for the __svc_xports array.

However, some operating systems (including the recent Manjaro OS) can have
_SC_OPEN_MAX values as high as 1073741816, which can cause the __svc_xports
array to become too large. This results in the process being killed.

There is a limit to the maximum number of files. To avoid this problem, a
possible solution is to set the size to the lesser of 1024 and this value to
ensure that the array space for open files is not too large, thus preventing
the process from terminating.

Also discovered that some users have taken action on the issue. It is necessary
to address this for all users. Ultimately, we determined that adjusting the
size value of _rpc_dtablesize() and streamlining the existing user code would
be the most effective solution.


---
Changes in v1:
Clean up the existing code in user

---
Links: 
RFC:https://lore.kernel.org/linux-nfs/tencent_E6816C9AF53E61BA5E0A313BBE5E1D19B00A@qq.com/T/#u


Zhuohao Bai (2):
  _rpc_dtablesize: Decrease the value of size.
  _rpc_dtablesize: Cleaning up the existing code

 src/rpc_dtablesize.c | 2 ++
 src/svc.c            | 2 --
 2 files changed, 2 insertions(+), 2 deletions(-)

-- 
2.25.1

[PATCH v1 1/2] _rpc_dtablesize: Decrease the value of size.

[Zhuohao Bai]

To fix the bug caused by a Size value that is too large and leads to an array
that requires excessive memory, which subsequently results in the failure of
rpcbind to start properly.

Signed-off-by: Zhuohao Bai <zhuohao_bai@foxmail.com>
---
 src/rpc_dtablesize.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/rpc_dtablesize.c b/src/rpc_dtablesize.c
index bce97e8..12f80c1 100644
--- a/src/rpc_dtablesize.c
+++ b/src/rpc_dtablesize.c
@@ -41,7 +41,7 @@ _rpc_dtablesize(void)
 	static int size;
 
 	if (size == 0) {
-		size = sysconf(_SC_OPEN_MAX);
+		size = min(1024, sysconf(_SC_OPEN_MAX));
 	}
 	return (size);
 }
-- 
2.25.1

[PATCH v1 2/2] _rpc_dtablesize: Cleaning up the existing code

[Zhuohao Bai]

Some users have already taken steps to address this issue. To prevent
duplication, we need to remove the modified code and modify the rpc_dtablesize
function.

Signed-off-by: Zhuohao Bai <zhuohao_bai@foxmail.com>
---
 src/rpc_dtablesize.c | 4 +++-
 src/svc.c            | 2 --
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/src/rpc_dtablesize.c b/src/rpc_dtablesize.c
index 12f80c1..e88698f 100644
--- a/src/rpc_dtablesize.c
+++ b/src/rpc_dtablesize.c
@@ -41,7 +41,9 @@ _rpc_dtablesize(void)
 	static int size;
 
 	if (size == 0) {
-		size = min(1024, sysconf(_SC_OPEN_MAX));
+		size = sysconf(_SC_OPEN_MAX);
+		if (size > FD_SETSIZE)
+			size = FD_SETSIZE;
 	}
 	return (size);
 }
diff --git a/src/svc.c b/src/svc.c
index 3a8709f..9b932a5 100644
--- a/src/svc.c
+++ b/src/svc.c
@@ -657,8 +657,6 @@ svc_getreqset (readfds)
   assert (readfds != NULL);
 
   setsize = _rpc_dtablesize ();
-  if (setsize > FD_SETSIZE)
-    setsize = FD_SETSIZE;
   maskp = readfds->fds_bits;
   for (sock = 0; sock < setsize; sock += NFDBITS)
     {
-- 
2.25.1

Re: [PATCH v1 0/2] _rpc_dtablesize: decrease to fix excessive memory usage

[Steve Dickson]

On 10/31/23 9:13 AM, Zhuohao Bai wrote:
> In the client code, the function _rpc_dtablesize() is used to determine the
> memory allocation for the __svc_xports array.
> 
> However, some operating systems (including the recent Manjaro OS) can have
> _SC_OPEN_MAX values as high as 1073741816, which can cause the __svc_xports
> array to become too large. This results in the process being killed.
> 
> There is a limit to the maximum number of files. To avoid this problem, a
> possible solution is to set the size to the lesser of 1024 and this value to
> ensure that the array space for open files is not too large, thus preventing
> the process from terminating.
> 
> Also discovered that some users have taken action on the issue. It is necessary
> to address this for all users. Ultimately, we determined that adjusting the
> size value of _rpc_dtablesize() and streamlining the existing user code would
> be the most effective solution.
> 
> 
> ---
> Changes in v1:
> Clean up the existing code in user
> 
> ---
> Links:
> RFC:https://lore.kernel.org/linux-nfs/tencent_E6816C9AF53E61BA5E0A313BBE5E1D19B00A@qq.com/T/#u
> 
> 
> Zhuohao Bai (2):
>    _rpc_dtablesize: Decrease the value of size.
>    _rpc_dtablesize: Cleaning up the existing code
> 
>   src/rpc_dtablesize.c | 2 ++
>   src/svc.c            | 2 --
>   2 files changed, 2 insertions(+), 2 deletions(-)
> 
Both Committed... (tag: libtirpc-1-3-5-rc2)

steved.