RE: [PATCH bpf-next v2 1/3] bpf: Make errors of bpf_tcp_check_syncookie distinguishable

[John Fastabend <john.fastabend@gmail.com>]

Maxim Mikityanskiy wrote:
> bpf_tcp_check_syncookie returns ambiguous error codes in some cases. The
> list below shows various error conditions and matching error codes:
> 
> 1. NULL socket: -EINVAL.
> 
> 2. Invalid packet: -EINVAL, -ENOENT.
> 
> 3. Bad cookie: -ENOENT.
> 
> 4. Cookies are not in use: -EINVAL, -ENOENT.
> 
> 5. Good cookie: 0.
> 
> As we see, the same error code may correspond to multiple error
> conditions, making them undistinguishable, and at the same time one
> error condition may return different codes, although it's typically
> handled in the same way.
> 
> This patch reassigns error codes of bpf_tcp_check_syncookie and
> documents them:
> 
> 1. Invalid packet or NULL socket: -EINVAL;
> 
> 2. Bad cookie: -EACCES.
> 
> 3. Cookies are not in use: -ENOENT.
> 
> 4. Good cookie: 0.
> 
> This change allows XDP programs to make smarter decisions based on error
> code, because different error conditions are now easily distinguishable.

I'm missing the point here it still looks a bit like shuffling
around of code. What do you gain, whats the real bug? Are you
trying to differentiate between an overflow condition and a valid
syncookie? But I don't think you said this anywhere.

At the moment EINVAL tells me somethings wrong with the input or
configuration, although really any app that cares checked the
sysctl flag right?

ENOENT tells me either recent overflow or cookie is invalid. If
there is no '!ack || rst || syn' then I can either learn that
directly from the program (why would a real program through
these at the helper), but it also falls into the incorrect
cookie in some sense.

> 
> Backward compatibility shouldn't suffer because of these reasons:
> 
> 1. The specific error codes weren't documented. The behavior that used
>    to be documented (0 is good cookie, negative values are errors) still
>    holds. Anyone who relied on implementation details should have
>    understood the risks.

I'll disagree, just because a user facing API doesn't document its
behavior very well doesn't mean users should some how understand the
risks. Ideally we would have done better with error codes up front,
but thats old history. If a user complains that this breaks a real
application it would likely be reason to revert it.

At least I would remove this from the commit.

> 
> 2. Two known usecases (classification of ACKs with cookies that initial
>    new connections, SYN flood protection) take decisions which don't
>    depend on specific error codes:
> 
>      Traffic classification:
>        ACK packet is new, error == 0: classify as NEW.
>        ACK packet is new, error < 0: classify as INVALID.
> 
>      SYN flood protection:
>        ACK packet is new, error == 0: good cookie, XDP_PASS.
>        ACK packet is new, error < 0: bad cookie, XDP_DROP.
> 
>    As Lorenz Bauer confirms, their implementation of traffic classifier
>    won't break, as well as the kernel selftests.
> 
> 3. It's hard to imagine that old error codes could be used for any
>    useful decisions.
> 
> Signed-off-by: Maxim Mikityanskiy <maximmi@nvidia.com>
> Reviewed-by: Tariq Toukan <tariqt@nvidia.com>
> ---
>  include/uapi/linux/bpf.h       | 18 ++++++++++++++++--
>  net/core/filter.c              |  6 +++---
>  tools/include/uapi/linux/bpf.h | 18 ++++++++++++++++--
>  3 files changed, 35 insertions(+), 7 deletions(-)
> 
> diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> index 16a7574292a5..4d2d4a09bf25 100644
> --- a/include/uapi/linux/bpf.h
> +++ b/include/uapi/linux/bpf.h
> @@ -3575,8 +3575,22 @@ union bpf_attr {
>   * 		*th* points to the start of the TCP header, while *th_len*
>   * 		contains **sizeof**\ (**struct tcphdr**).
>   * 	Return
> - * 		0 if *iph* and *th* are a valid SYN cookie ACK, or a negative
> - * 		error otherwise.
> + *		0 if *iph* and *th* are a valid SYN cookie ACK.
> + *
> + *		On failure, the returned value is one of the following:
> + *
> + *		**-EACCES** if the SYN cookie is not valid.
> + *
> + *		**-EINVAL** if the packet or input arguments are invalid.
> + *
> + *		**-ENOENT** if SYN cookies are not issued (no SYN flood, or SYN
> + *		cookies are disabled in sysctl).
> + *
> + *		**-EOPNOTSUPP** if the kernel configuration does not enable SYN
> + *		cookies (CONFIG_SYN_COOKIES is off).
> + *
> + *		**-EPROTONOSUPPORT** if the IP version is not 4 or 6 (or 6, but
> + *		CONFIG_IPV6 is disabled).
>   *
>   * long bpf_sysctl_get_name(struct bpf_sysctl *ctx, char *buf, size_t buf_len, u64 flags)
>   *	Description
> diff --git a/net/core/filter.c b/net/core/filter.c
> index a06931c27eeb..18559b5828a3 100644
> --- a/net/core/filter.c
> +++ b/net/core/filter.c
> @@ -6998,10 +6998,10 @@ BPF_CALL_5(bpf_tcp_check_syncookie, struct sock *, sk, void *, iph, u32, iph_len
>  		return -EINVAL;
>  
>  	if (!sock_net(sk)->ipv4.sysctl_tcp_syncookies)
> -		return -EINVAL;
> +		return -ENOENT;

I wouldn't change this.

>  
>  	if (!th->ack || th->rst || th->syn)
> -		return -ENOENT;
> +		return -EINVAL;

not sure if this is useful change and it is bpf program visible.

>  
>  	if (tcp_synq_no_recent_overflow(sk))
>  		return -ENOENT;
> @@ -7032,7 +7032,7 @@ BPF_CALL_5(bpf_tcp_check_syncookie, struct sock *, sk, void *, iph, u32, iph_len
>  	if (ret > 0)
>  		return 0;
>  
> -	return -ENOENT;
> +	return -EACCES;

This one might have a valid argument to differentiate between an
overflow condition and an invalid cookie. But, curious what do you
do with this info?

Thanks,
John