sock_ops: calling bpf_sock_ops_cb_flags_set() for already established sockets

sock_ops: calling bpf_sock_ops_cb_flags_set() for already established sockets

[Alan Maguire]

hi folks

I've run into a few cases where users have wanted to enable additional
tcp-bpf sock_ops events for a socket _after_ connection establishment.
The problem is that to set the flags to enable additional events, we
have to be in the context of a sock_ops program, and as I understand it,
by default only events early in the socket lifetime are enabled by
default (such as connection established/accepted). As a consequence, if
we do not catch one of those early events, the sock_ops program will not
run and we miss the opportunity to enable more sock_ops events. This can
be a problem for boot-time connections like iSCSI where we are too late
to catch connection establishment.

I can see a few possibilities:

- support setting sock_ops event flags via a socket iterator. This would
mean that the user can always set per-socket flags on
already-established sockets by iterating over existing sockets,
selecting those of interest.
- supporting setting event flags via setsockopt(). In fact we wouldn't
need to fully support setting event flags via "real" setsockopt(); we
could simply use a cgroup/setsockopt program and allow
bpf_sock_ops_cb_flags_set() to run in the cgroup/setsockopt context
(with additional checks to ensure it is indeed a tcp socket).

Do either/both of these seem reasonable, or is there a better way to
tackle this? Thanks!

Alan

Re: sock_ops: calling bpf_sock_ops_cb_flags_set() for already established sockets

[Martin KaFai Lau]

On 12/1/23 9:01 AM, Alan Maguire wrote:
> hi folks
> 
> I've run into a few cases where users have wanted to enable additional
> tcp-bpf sock_ops events for a socket _after_ connection establishment.
> The problem is that to set the flags to enable additional events, we
> have to be in the context of a sock_ops program, and as I understand it,
> by default only events early in the socket lifetime are enabled by
> default (such as connection established/accepted). As a consequence, if
> we do not catch one of those early events, the sock_ops program will not
> run and we miss the opportunity to enable more sock_ops events. This can
> be a problem for boot-time connections like iSCSI where we are too late
> to catch connection establishment.
> 
> I can see a few possibilities:
> 
> - support setting sock_ops event flags via a socket iterator. This would
> mean that the user can always set per-socket flags on
> already-established sockets by iterating over existing sockets,
> selecting those of interest.
> - supporting setting event flags via setsockopt(). In fact we wouldn't
> need to fully support setting event flags via "real" setsockopt(); we
> could simply use a cgroup/setsockopt program and allow
> bpf_sock_ops_cb_flags_set() to run in the cgroup/setsockopt context
> (with additional checks to ensure it is indeed a tcp socket).
> 
> Do either/both of these seem reasonable, or is there a better way to
> tackle this? Thanks!

The lock_sock() is held in both cases (iterator / cgroup-setsockopt), so should 
be ok. I guess it depends on the usecase. If it does not have a hold on the 
socket fd, socket iterator may be the only way.