bpf, sockmap: FIONREAD returns 0 for TCP sockets in a sockmap without a verdict program

bpf, sockmap: FIONREAD returns 0 for TCP sockets in a sockmap without a verdict program

[Mattia Meleleo]

Hi,

in OpenTelemetry eBPF instrumentation we use a sockhash to track
outgoing TCP sockets for trace context propagation. Sockets are added
from a sock_ops program, and the map only has an sk_msg program
attached on the egress side - there is no ingress verdict program.

Since commit 929e30f93125 ("bpf, sockmap: Fix FIONREAD for sockmap"),
ioctl(FIONREAD) returns 0 for these sockets even though read() returns
data. tcp_bpf_ioctl() answers from psock->msg_tot_len, which only
accounts for bytes in ingress_msg. Without a verdict program, data
never lands there: it stays in sk_receive_queue and is read through
the tcp_bpf_recvmsg() fallback, so FIONREAD always reports 0.

The commit message explains that sk_receive_queue is intentionally not
counted because a verdict program may redirect or drop its contents.
Without a verdict program, however, all of that data is readable, and
applications that use FIONREAD to size their reads (nginx, Java, .NET)
hang or truncate transfers once their sockets are in the map.

Observed on mainline and on 6.12.75+, 6.6.128+, 6.18.14+.

Reproducer: https://github.com/mmat11/fionread-repro

  pre-insert     FIONREAD=4096 (expect 4096) OK
  in-sockhash    FIONREAD=0 (expect 4096) BROKEN
  post-delete    FIONREAD=4096 (expect 4096) OK

Is this intended for psocks without a verdict program? If not, would
falling back to tcp_inq() (plus msg_tot_len) in that case be an
acceptable fix? Happy to send a patch and test it.

Thanks,
Mattia

Re: bpf, sockmap: FIONREAD returns 0 for TCP sockets in a sockmap without a verdict program

[John Fastabend]

On Fri, Jul 03, 2026 at 06:10:16PM +0200, Mattia Meleleo wrote:
>Hi,
>
>in OpenTelemetry eBPF instrumentation we use a sockhash to track
>outgoing TCP sockets for trace context propagation. Sockets are added
>from a sock_ops program, and the map only has an sk_msg program
>attached on the egress side - there is no ingress verdict program.

This is how we use sockmap in many cases as well. I just haven't
rolled out any actual workload tests recently sadly. I expect we
would also hit this.

>
>Since commit 929e30f93125 ("bpf, sockmap: Fix FIONREAD for sockmap"),
>ioctl(FIONREAD) returns 0 for these sockets even though read() returns
>data. tcp_bpf_ioctl() answers from psock->msg_tot_len, which only
>accounts for bytes in ingress_msg. Without a verdict program, data
>never lands there: it stays in sk_receive_queue and is read through
>the tcp_bpf_recvmsg() fallback, so FIONREAD always reports 0.
>
>The commit message explains that sk_receive_queue is intentionally not
>counted because a verdict program may redirect or drop its contents.
>Without a verdict program, however, all of that data is readable, and
>applications that use FIONREAD to size their reads (nginx, Java, .NET)
>hang or truncate transfers once their sockets are in the map.
>
>Observed on mainline and on 6.12.75+, 6.6.128+, 6.18.14+.

agh because fixes tag it was backported everywhere :(

>
>Reproducer: https://github.com/mmat11/fionread-repro
>
>  pre-insert     FIONREAD=4096 (expect 4096) OK
>  in-sockhash    FIONREAD=0 (expect 4096) BROKEN
>  post-delete    FIONREAD=4096 (expect 4096) OK
>
>Is this intended for psocks without a verdict program? If not, would
>falling back to tcp_inq() (plus msg_tot_len) in that case be an
>acceptable fix? Happy to send a patch and test it.

Its a bug. It needs a fix. Please send a fix I'll review and
test on our dogfooding env as well. Would you also add a selftest
here so we can capture this behavior is fairly common for apps to
do this.