* [Regression linux-6.1.y]: NFSv4 server: sec=krb5p mount fails with EIO in 6.1.192
@ 2026-11-20 13:17 Max Vozeler
2026-11-21 10:28 ` Max Vozeler
0 siblings, 1 reply; 4+ messages in thread
From: Max Vozeler @ 2026-11-20 13:17 UTC (permalink / raw)
To: linux-nfs; +Cc: stable, regressions
#regzbot introduced: v6.1.190..v6.1.192
Hi,
after upgrading an NFS server (Debian bookworm) to 6.1.192
(Debian linux-image-6.1.0-55-amd64), NFSv4 clients fail to mount exports
using sec=krb5p with EIO.
Downgrading the server back to 6.1.190 (linux-image-6.1.0-54-amd64)
makes things work again.
Good: 6.1.190 (Debian linux-image-6.1.0-54-amd64)
Bad: 6.1.192 (Debian linux-image-6.1.0-55-amd64, linux-image-6.1.0-56-amd64)
sec=krb5i works on the exact same server, export, and Kerberos keytab;
only sec=krb5p fails.
# mount -vvv -t nfs4 -o sec=krb5p,vers=4.0 10.x.y.z:/<export> /mnt
mount.nfs4: timeout set for Fri Nov 20 08:10:07 2026
mount.nfs4: trying text-based options 'sec=krb5p,nfsvers=4.0,addr=10.x.y.z,clientaddr=10.x.y.a'
mount.nfs4: mount(2): Input/output error
mount.nfs4: mount system call failed
Server dmesg with rpcdebug enabled:
[42095.312947] svc: server 0000000039bc31cb, pool 0, transport 00000000cfa39484, inuse=2
[42095.312971] svc: server 0000000072d08ff5, pool 0, transport 00000000cfa39484, inuse=3
[42095.312975] svc: svc_authenticate (0)
[42095.324501] svc: server 0000000072d08ff5, pool 0, transport 00000000c60f4fcd, inuse=2
[42095.334305] svc: server 0000000072d08ff5, pool 0, transport 00000000c60f4fcd, inuse=2
[42095.334324] svc: svc_authenticate (6)
[42095.334331] svc: server 0000000039bc31cb, pool 0, transport 00000000c60f4fcd, inuse=3
[42095.334388] RPC: xs_local_send_request(1756) = 0
[42095.338142] RPC: krb5_encrypt returns 0
[42095.338144] RPC: krb5_encrypt returns 0
[42095.338148] RPC: krb5_encrypt returns 0
[42095.338148] RPC: krb5_encrypt returns 0
[42095.338151] RPC: krb5_encrypt returns 0
[42095.338151] RPC: krb5_encrypt returns 0
[42095.338154] RPC: krb5_encrypt returns 0
[42095.338154] RPC: krb5_encrypt returns 0
[42095.338156] RPC: krb5_encrypt returns 0
[42095.338156] RPC: krb5_encrypt returns 0
[42095.338158] RPC: krb5_encrypt returns 0
[42095.338158] RPC: krb5_encrypt returns 0
[42095.338160] RPC: gss_import_sec_context_kerberos: returning 0
[42095.338163] RPC: gss_get_mic_v2
[42095.338546] svc: server 0000000072d08ff5, pool 0, transport 00000000cfa39484, inuse=2
[42095.338554] svc: svc_authenticate (6)
[42095.338555] RPC: gss_verify_mic_v2
[42095.338557] RPC: gss_get_mic_v2
[42095.338557] svc: server 0000000039bc31cb, pool 0, transport 00000000cfa39484, inuse=3
[42095.338559] RPC: gss_verify_mic_v2
[42095.338573] nfsd4_exchange_id rqstp=0000000072d08ff5 exid=0000000096ec91a4 clname.len=27 clname.data=0000000035150242 ip_addr=10.x.y.50 flags 101, spa_how 1
[42095.338592] nfsd4_exchange_id seqid 0 flags 20001
[42095.338594] RPC: gss_get_mic_v2
[42095.338777] svc: server 0000000072d08ff5, pool 0, transport 00000000cfa39484, inuse=2
[42095.338783] svc: svc_authenticate (6)
[42095.338784] RPC: gss_verify_mic_v2
[42095.338786] RPC: gss_get_mic_v2
[42095.338786] svc: server 0000000039bc31cb, pool 0, transport 00000000cfa39484, inuse=3
[42095.338787] RPC: gss_verify_mic_v2
[42095.338790] nfsd4_exchange_id rqstp=0000000072d08ff5 exid=0000000096ec91a4 clname.len=27 clname.data=0000000035150242 ip_addr=10.x.y.50 flags 101, spa_how 1
[42095.338799] nfsd4_exchange_id seqid 0 flags 20001
[42095.338834] RPC: gss_get_mic_v2
[42095.338953] svc: server 0000000072d08ff5, pool 0, transport 00000000cfa39484, inuse=2
[42095.338958] svc: svc_authenticate (6)
[42095.338958] RPC: gss_verify_mic_v2
[42095.338960] RPC: gss_get_mic_v2
[42095.338960] svc: server 0000000039bc31cb, pool 0, transport 00000000cfa39484, inuse=3
[42095.338962] RPC: gss_verify_mic_v2
[42095.338982] check_slot_seqid enter. seqid 1 slot_seqid 0
[42095.338998] RPC: set up xprt to 10.x.y.50 (port 0) via tcp
[42095.339057] RPC: gss_get_mic_v2
[42095.339264] svc: server 0000000072d08ff5, pool 0, transport 00000000cfa39484, inuse=4
[42095.339267] svc: svc_authenticate (6)
[42095.339268] RPC: gss_verify_mic_v2
[42095.339270] RPC: gss_get_mic_v2
[42095.339271] RPC: gss_verify_mic_v2
[42095.339271] svc: server 0000000039bc31cb, pool 0, transport 00000000cfa39484, inuse=5
[42095.339274] __find_in_sessionid_hashtbl: 1795172831:1033077859:4:0
[42095.339284] nfsd4_sequence: slotid 0
[42095.339285] check_slot_seqid enter. seqid 1 slot_seqid 0
[42095.339287] alloc_cld_upcall: allocated xid 9
[42095.340556] --> nfsd4_store_cache_entry slot 000000008b186fa8
[42095.340560] RPC: gss_get_mic_v2
[42095.343136] svc: server 0000000072d08ff5, pool 0, transport 00000000f9548822, inuse=2
[42095.344680] svc: server 0000000072d08ff5, pool 0, transport 00000000f9548822, inuse=2
[42095.344687] svc: svc_authenticate (6)
[42095.344690] svc: server 0000000039bc31cb, pool 0, transport 00000000f9548822, inuse=3
[42095.344757] RPC: xs_local_send_request(1756) = 0
[42095.347090] RPC: krb5_encrypt returns 0
[42095.347092] RPC: krb5_encrypt returns 0
[42095.347096] RPC: krb5_encrypt returns 0
[42095.347096] RPC: krb5_encrypt returns 0
[42095.347099] RPC: krb5_encrypt returns 0
[42095.347099] RPC: krb5_encrypt returns 0
[42095.347101] RPC: krb5_encrypt returns 0
[42095.347101] RPC: krb5_encrypt returns 0
[42095.347103] RPC: krb5_encrypt returns 0
[42095.347104] RPC: krb5_encrypt returns 0
[42095.347105] RPC: krb5_encrypt returns 0
[42095.347105] RPC: krb5_encrypt returns 0
[42095.347107] RPC: gss_import_sec_context_kerberos: returning 0
[42095.347109] RPC: gss_get_mic_v2
[42095.347456] svc: server 0000000072d08ff5, pool 0, transport 00000000cfa39484, inuse=4
[42095.347468] svc: svc_authenticate (6)
[42095.347469] RPC: gss_verify_mic_v2
[42095.347471] RPC: gss_get_mic_v2
[42095.347471] svc: server 0000000039bc31cb, pool 0, transport 00000000cfa39484, inuse=5
[42095.347473] RPC: gss_unwrap_kerberos_v2
[42095.347646] RPC: bc_destroy xprt 00000000488733b3
[42095.348467] svc: server 0000000072d08ff5, pool 0, transport 000000005d77f59c, inuse=2
[42095.348472] svc: svc_authenticate (6)
[42095.348474] RPC: gss_verify_mic_v2
[42095.348481] RPC: gss_get_mic_v2
[42095.348483] RPC: gss_unwrap_kerberos_v2
[42095.348507] svc: server 0000000072d08ff5, pool 0, transport 000000005d77f59c, inuse=2
[42095.348713] svc: server 0000000072d08ff5, pool 0, transport 000000005d77f59c, inuse=2
[42095.348752] svc: server 0000000072d08ff5, pool 0, transport 000000005d77f59c, inuse=2
[42095.348758] svc: svc_authenticate (6)
[42095.348759] RPC: gss_verify_mic_v2
[42095.348761] RPC: gss_get_mic_v2
[42095.348763] RPC: gss_unwrap_kerberos_v2
[42095.348762] svc: server 0000000039bc31cb, pool 0, transport 000000005d77f59c, inuse=3
[42095.365858] svc: server 0000000072d08ff5, pool 0, transport 000000005d77f59c, inuse=2
[42095.365867] svc: svc_authenticate (6)
[42095.365870] RPC: gss_verify_mic_v2
[42095.365869] svc: server 0000000039bc31cb, pool 0, transport 000000005d77f59c, inuse=3
[42095.365877] RPC: gss_get_mic_v2
[42095.365877] svc: svc_authenticate (6)
[42095.365880] RPC: gss_delete_sec_context deleting 000000004f9c6594
[42095.365881] RPC: gss_verify_mic_v2
[42095.365892] RPC: gss_get_mic_v2
[42095.365894] RPC: gss_verify_mic_v2
[42095.365900] nfsd4_destroy_session: 1795172831:1033077859:4:0
[42095.365901] __find_in_sessionid_hashtbl: 1795172831:1033077859:4:0
[42095.365903] svc: server 0000000072d08ff5, pool 0, transport 000000005d77f59c, inuse=3
[42095.365928] RPC: gss_get_mic_v2
[42095.366086] svc: server 0000000072d08ff5, pool 0, transport 000000005d77f59c, inuse=2
[42095.366091] svc: svc_authenticate (6)
[42095.366092] RPC: gss_verify_mic_v2
[42095.366094] RPC: gss_get_mic_v2
[42095.366094] svc: server 0000000039bc31cb, pool 0, transport 000000005d77f59c, inuse=3
[42095.366095] RPC: gss_verify_mic_v2
[42095.366101] alloc_cld_upcall: allocated xid 10
[42095.367452] RPC: gss_get_mic_v2
[42095.367647] svc: server 0000000072d08ff5, pool 0, transport 000000005d77f59c, inuse=2
[42095.367653] svc: svc_authenticate (6)
[42095.367655] RPC: gss_verify_mic_v2
[42095.367655] svc: server 0000000039bc31cb, pool 0, transport 000000005d77f59c, inuse=3
[42095.367658] RPC: gss_get_mic_v2
[42095.367660] RPC: gss_delete_sec_context deleting 0000000023931779
Let me know if there is any useful information I can provide.
Best regards,
Max
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Regression linux-6.1.y]: NFSv4 server: sec=krb5p mount fails with EIO in 6.1.192
2026-11-20 13:17 [Regression linux-6.1.y]: NFSv4 server: sec=krb5p mount fails with EIO in 6.1.192 Max Vozeler
@ 2026-11-21 10:28 ` Max Vozeler
2026-11-21 12:30 ` Max Vozeler
0 siblings, 1 reply; 4+ messages in thread
From: Max Vozeler @ 2026-11-21 10:28 UTC (permalink / raw)
To: linux-nfs; +Cc: stable, regressions
#regzbot introduced: v6.1.190..v6.1.191
On Fri, Nov 20, 2026 at 02:17:43PM +0100, Max Vozeler wrote:
> after upgrading an NFS server (Debian bookworm) to 6.1.192
> (Debian linux-image-6.1.0-55-amd64), NFSv4 clients fail to mount exports
> using sec=krb5p with EIO.
>
> Downgrading the server back to 6.1.190 (linux-image-6.1.0-54-amd64)
> makes things work again.
>
> Good: 6.1.190 (Debian linux-image-6.1.0-54-amd64)
> Bad: 6.1.192 (Debian linux-image-6.1.0-55-amd64, linux-image-6.1.0-56-amd64)
Reproduced with vanilla 6.1.191 and 6.1.192
Good: 6.1.190
Bad: 6.1.191, 6.1.192
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Regression linux-6.1.y]: NFSv4 server: sec=krb5p mount fails with EIO in 6.1.192
2026-11-21 10:28 ` Max Vozeler
@ 2026-11-21 12:30 ` Max Vozeler
2026-11-21 12:54 ` Max Vozeler
0 siblings, 1 reply; 4+ messages in thread
From: Max Vozeler @ 2026-11-21 12:30 UTC (permalink / raw)
To: Chuck Lever, Sasha Levin, Greg Kroah-Hartman, Chris Mason,
Jeff Layton
Cc: linux-nfs, stable, regressions
[-- Attachment #1: Type: text/plain, Size: 1428 bytes --]
#regzbot introduced: 82cf21374fb7ee55b37be9afb6704ba0eeffb236
On Sat, Nov 21, 2026 at 11:28:53AM +0100, Max Vozeler wrote:
> On Fri, Nov 20, 2026 at 02:17:43PM +0100, Max Vozeler wrote:
> > after upgrading an NFS server (Debian bookworm) to 6.1.192
> > (Debian linux-image-6.1.0-55-amd64), NFSv4 clients fail to mount exports
> > using sec=krb5p with EIO.
> >
> > Downgrading the server back to 6.1.190 (linux-image-6.1.0-54-amd64)
> > makes things work again.
> >
> > Good: 6.1.190 (Debian linux-image-6.1.0-54-amd64)
> > Bad: 6.1.192 (Debian linux-image-6.1.0-55-amd64, linux-image-6.1.0-56-amd64)
>
> Reproduced with vanilla 6.1.191 and 6.1.192
>
> Good: 6.1.190
> Bad: 6.1.191, 6.1.192
bisect points to
commit 82cf21374fb7ee55b37be9afb6704ba0eeffb236
Author: Chuck Lever <chuck.lever@oracle.com>
Date: Mon Jan 2 12:07:07 2023 -0500
SUNRPC: Convert unwrap_priv_data() to use xdr_stream
[ Upstream commit 42140718ea26c47fb06c679451a6aa6703545136 ]
Done as part of hardening the server-side RPC header decoding path.
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Stable-dep-of: a919c5c88769 ("SUNRPC: svcauth_gss: enforce krb5 token minimum length")
Signed-off-by: Sasha Levin <sashal@kernel.org>
This seems to have been part of a larger series, only parts of which
were backported for a919c5c88769 ?
Best regards,
Max
[-- Attachment #2: bisect.log --]
[-- Type: text/plain, Size: 1637 bytes --]
git bisect start
# status: waiting for both good and bad commits
# bad: [08889840163ebcdb7e24bf7ec311bc2e6fc3aaea] Linux 6.1.191
git bisect bad 08889840163ebcdb7e24bf7ec311bc2e6fc3aaea
# status: waiting for good commit(s), bad commit known
# good: [79807e970724c1f90cee1316361d61b80db4eef6] Linux 6.1.190
git bisect good 79807e970724c1f90cee1316361d61b80db4eef6
# good: [91c2a5d201daf66fa3f42b8a383bb89174bde2c8] serial: core: fix infinite loop in handle_tx() for PORT_UNKNOWN
git bisect good 91c2a5d201daf66fa3f42b8a383bb89174bde2c8
# bad: [ab8c8e027fa8f9314340a96999dd3309b7f80d61] fbdev: ssd1307fb: defer I2C transfers from damage callbacks
git bisect bad ab8c8e027fa8f9314340a96999dd3309b7f80d61
# good: [35b46918d5deee01fade7e2f2075a7c238a2a0e8] cifs: prevent readdir from changing file size due to stale directory metadata
git bisect good 35b46918d5deee01fade7e2f2075a7c238a2a0e8
# good: [91abd8383fd0588d1fcc31120d6344150a64ba5a] net: sched: cls_api: add filter counter
git bisect good 91abd8383fd0588d1fcc31120d6344150a64ba5a
# bad: [ed126ca71d4d949ad99f40b974a8f7685e808f8b] net/packet: defer vmalloc TX_RING free until skbs finish
git bisect bad ed126ca71d4d949ad99f40b974a8f7685e808f8b
# bad: [82cf21374fb7ee55b37be9afb6704ba0eeffb236] SUNRPC: Convert unwrap_priv_data() to use xdr_stream
git bisect bad 82cf21374fb7ee55b37be9afb6704ba0eeffb236
# good: [8da6f765098d46e5b3178ef4b2f8273c9ba1d3bb] SUNRPC: Rename automatic variables in unwrap_priv_data()
git bisect good 8da6f765098d46e5b3178ef4b2f8273c9ba1d3bb
# first bad commit: [82cf21374fb7ee55b37be9afb6704ba0eeffb236] SUNRPC: Convert unwrap_priv_data() to use xdr_stream
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: [Regression linux-6.1.y]: NFSv4 server: sec=krb5p mount fails with EIO in 6.1.192
2026-11-21 12:30 ` Max Vozeler
@ 2026-11-21 12:54 ` Max Vozeler
0 siblings, 0 replies; 4+ messages in thread
From: Max Vozeler @ 2026-11-21 12:54 UTC (permalink / raw)
To: Chuck Lever, Sasha Levin, Greg Kroah-Hartman, Chris Mason,
Jeff Layton
Cc: linux-nfs, stable, regressions
[Resend to @kernel.org for Chuck and Chris; previous mail bounced]
On Sat, Nov 21, 2026 at 01:30:49PM +0100, Max Vozeler wrote:
> #regzbot introduced: 82cf21374fb7ee55b37be9afb6704ba0eeffb236
>
> On Sat, Nov 21, 2026 at 11:28:53AM +0100, Max Vozeler wrote:
> > On Fri, Nov 20, 2026 at 02:17:43PM +0100, Max Vozeler wrote:
> > > after upgrading an NFS server (Debian bookworm) to 6.1.192
> > > (Debian linux-image-6.1.0-55-amd64), NFSv4 clients fail to mount exports
> > > using sec=krb5p with EIO.
> > >
> > > Downgrading the server back to 6.1.190 (linux-image-6.1.0-54-amd64)
> > > makes things work again.
> > >
> > > Good: 6.1.190 (Debian linux-image-6.1.0-54-amd64)
> > > Bad: 6.1.192 (Debian linux-image-6.1.0-55-amd64, linux-image-6.1.0-56-amd64)
> >
> > Reproduced with vanilla 6.1.191 and 6.1.192
> >
> > Good: 6.1.190
> > Bad: 6.1.191, 6.1.192
>
> bisect points to
>
> commit 82cf21374fb7ee55b37be9afb6704ba0eeffb236
> Author: Chuck Lever <chuck.lever@oracle.com>
> Date: Mon Jan 2 12:07:07 2023 -0500
>
> SUNRPC: Convert unwrap_priv_data() to use xdr_stream
>
> [ Upstream commit 42140718ea26c47fb06c679451a6aa6703545136 ]
>
> Done as part of hardening the server-side RPC header decoding path.
>
> Reviewed-by: Jeff Layton <jlayton@kernel.org>
> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
> Stable-dep-of: a919c5c88769 ("SUNRPC: svcauth_gss: enforce krb5 token minimum length")
> Signed-off-by: Sasha Levin <sashal@kernel.org>
>
> This seems to have been part of a larger series, only parts of which
> were backported for a919c5c88769 ?
>
> Best regards,
> Max
> git bisect start
> # status: waiting for both good and bad commits
> # bad: [08889840163ebcdb7e24bf7ec311bc2e6fc3aaea] Linux 6.1.191
> git bisect bad 08889840163ebcdb7e24bf7ec311bc2e6fc3aaea
> # status: waiting for good commit(s), bad commit known
> # good: [79807e970724c1f90cee1316361d61b80db4eef6] Linux 6.1.190
> git bisect good 79807e970724c1f90cee1316361d61b80db4eef6
> # good: [91c2a5d201daf66fa3f42b8a383bb89174bde2c8] serial: core: fix infinite loop in handle_tx() for PORT_UNKNOWN
> git bisect good 91c2a5d201daf66fa3f42b8a383bb89174bde2c8
> # bad: [ab8c8e027fa8f9314340a96999dd3309b7f80d61] fbdev: ssd1307fb: defer I2C transfers from damage callbacks
> git bisect bad ab8c8e027fa8f9314340a96999dd3309b7f80d61
> # good: [35b46918d5deee01fade7e2f2075a7c238a2a0e8] cifs: prevent readdir from changing file size due to stale directory metadata
> git bisect good 35b46918d5deee01fade7e2f2075a7c238a2a0e8
> # good: [91abd8383fd0588d1fcc31120d6344150a64ba5a] net: sched: cls_api: add filter counter
> git bisect good 91abd8383fd0588d1fcc31120d6344150a64ba5a
> # bad: [ed126ca71d4d949ad99f40b974a8f7685e808f8b] net/packet: defer vmalloc TX_RING free until skbs finish
> git bisect bad ed126ca71d4d949ad99f40b974a8f7685e808f8b
> # bad: [82cf21374fb7ee55b37be9afb6704ba0eeffb236] SUNRPC: Convert unwrap_priv_data() to use xdr_stream
> git bisect bad 82cf21374fb7ee55b37be9afb6704ba0eeffb236
> # good: [8da6f765098d46e5b3178ef4b2f8273c9ba1d3bb] SUNRPC: Rename automatic variables in unwrap_priv_data()
> git bisect good 8da6f765098d46e5b3178ef4b2f8273c9ba1d3bb
> # first bad commit: [82cf21374fb7ee55b37be9afb6704ba0eeffb236] SUNRPC: Convert unwrap_priv_data() to use xdr_stream
Context: https://lore.kernel.org/regressions/awBIdxOJ4XSXuc7S@dp.vpn.hinterhof.net/T/#rd62f582e83e56dd3f5e8b430de3de07bee6b60b7
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2026-11-21 12:54 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-20 13:17 [Regression linux-6.1.y]: NFSv4 server: sec=krb5p mount fails with EIO in 6.1.192 Max Vozeler
2026-11-21 10:28 ` Max Vozeler
2026-11-21 12:30 ` Max Vozeler
2026-11-21 12:54 ` Max Vozeler
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox