Re: unable to run NFSD in container if "options sunrpc pool_mode=pernode"

[Mike Snitzer <snitzer@kernel.org>]

On Fri, May 23, 2025 at 06:19:03PM -0400, Mike Snitzer wrote:
> On Fri, May 23, 2025 at 02:40:17PM -0400, Jeff Layton wrote:
> > On Fri, 2025-05-23 at 14:29 -0400, Mike Snitzer wrote:
> > > I don't know if $SUBJECT ever worked... but with latest 6.15 or
> > > nfsd-testing if I just use pool_mode=global then all is fine.
> > > 
> > > If pool_mode=pernode then mounting the container's NFSv3 export fails.
> > > 
> > > I haven't started to dig into code yet but pool_mode=pernode works
> > > perfectly fine if NFSD isn't running in a container.
> > > 
> > > Mike
> > > 
> > > ps. yet another reason why pool_mode=pernode should be the default if
> > > more than 1 NUMA node ;)
> > 
> > Huh, strange. I've no idea why that would be. What kernel is this?
> 
> It is this 6.12.24 based frankenbeast-ish kernel:
> https://git.kernel.org/pub/scm/linux/kernel/git/snitzer/linux.git/log/?h=kernel-6.12.24/main-testing
> 
> Basically just 6.12.24 + NFS and NFSD sync'd through nfs-testing and
> nfsd-testing (so 6.15 NFS and NFSD going on 6.16).
> 
> But I also just verified that this kernel built on Chuck's
> nfsd-testing branch (with 2 extra patches) has the same issue:
> https://git.kernel.org/pub/scm/linux/kernel/git/snitzer/linux.git/log/?h=cel-nfsd-testing-6.16
> 
> Here is the NFS related config:
> 
> CONFIG_NETWORK_FILESYSTEMS=y
> CONFIG_NFS_FS=m
> # CONFIG_NFS_V2 is not set
> CONFIG_NFS_V3=m
> CONFIG_NFS_V3_ACL=y
> CONFIG_NFS_V4=m
> # CONFIG_NFS_SWAP is not set
> CONFIG_NFS_V4_1=y
> CONFIG_NFS_V4_2=y
> CONFIG_PNFS_FILE_LAYOUT=m
> CONFIG_PNFS_BLOCK=m
> CONFIG_PNFS_FLEXFILE_LAYOUT=m
> CONFIG_NFS_V4_1_IMPLEMENTATION_ID_DOMAIN="kernel.org"
> # CONFIG_NFS_V4_1_MIGRATION is not set
> CONFIG_NFS_V4_SECURITY_LABEL=y
> CONFIG_NFS_FSCACHE=y
> # CONFIG_NFS_USE_LEGACY_DNS is not set
> CONFIG_NFS_USE_KERNEL_DNS=y
> CONFIG_NFS_DEBUG=y
> CONFIG_NFS_DISABLE_UDP_SUPPORT=y
> # CONFIG_NFS_V4_2_READ_PLUS is not set
> CONFIG_NFSD=m
> # CONFIG_NFSD_V2 is not set
> CONFIG_NFSD_V3_ACL=y
> CONFIG_NFSD_V4=y
> CONFIG_NFSD_PNFS=y
> # CONFIG_NFSD_BLOCKLAYOUT is not set
> CONFIG_NFSD_SCSILAYOUT=y
> # CONFIG_NFSD_FLEXFILELAYOUT is not set
> # CONFIG_NFSD_V4_2_INTER_SSC is not set
> CONFIG_NFSD_V4_SECURITY_LABEL=y
> # CONFIG_NFSD_LEGACY_CLIENT_TRACKING is not set
> # CONFIG_NFSD_V4_DELEG_TIMESTAMPS is not set
> CONFIG_GRACE_PERIOD=m
> CONFIG_LOCKD=m
> CONFIG_LOCKD_V4=y
> CONFIG_NFS_ACL_SUPPORT=m
> CONFIG_NFS_COMMON=y
> CONFIG_NFS_COMMON_LOCALIO_SUPPORT=m
> CONFIG_NFS_LOCALIO=y
> CONFIG_NFS_V4_2_SSC_HELPER=y
> CONFIG_SUNRPC=m
> CONFIG_SUNRPC_GSS=m
> CONFIG_SUNRPC_BACKCHANNEL=y
> CONFIG_RPCSEC_GSS_KRB5=m
> CONFIG_RPCSEC_GSS_KRB5_ENCTYPES_AES_SHA1=y
> CONFIG_RPCSEC_GSS_KRB5_ENCTYPES_AES_SHA2=y
> CONFIG_SUNRPC_DEBUG=y
> CONFIG_SUNRPC_XPRT_RDMA=m
> 
> > FWIW, I just built a localio-enabled on a v6.12-uek kernel for our own
> > purposes yesterday and it's running pool_mode=pernode. It seemed to
> > work fine as a v3 DS, but I didn't test mounting the container's export
> > directly.
> 
> OK, but you were able to access the v3 DS just fine (assuming pNFS
> flexfiles layouts that point to your DS that is running NFSD in a
> container) ?
> 
> I'm using nfs-utils-2.8.2.  I don't see any nfsd threads running if I
> use "options sunrpc pool_mode=pernode".

Actually, I do see nfsd threads running.. just that if I try to issue
IO (using pNFS flexfiles to file on v3 DS) it hangs.

Mike