From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 57C7465D; Sat, 1 Jul 2023 03:11:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EB0D5C433C9; Sat, 1 Jul 2023 03:11:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1688181062; bh=KFkohFkvFbHLLBwnHmlfBxLR+78ltfSpH1Qop5HD2dQ=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=q6IJ3MaABN+Ni0mPcl2k0KwvIFP+CVc+sM3t5E5YSdujLUe+5z4Ap6b+mJQTRFf6N VewBaSeKOvSH8LU6WVZuKTu4Na8zm95W9kq/Gx4ur3Cg8kcx/pF82huI33U7djrza6 B2UJunjdwGO7AJmlg25wPB6k3FclLHBvJGDkM+ngG9AL66RsdwyCp3W/ECLJ569q9w A1a/oHp3STHmwwcT9lBd5EKkQsEmX///pklBp7xltyvg/zY7LNpA2MXkvi6bjNPM2R CH1blQ4inbLtqsyh/h7hpMH9oXh5eHfvhthJW5jyUZrS60aeOMt3yh0dZpMmoaHLMB C2rCMBde8lU0g== Date: Fri, 30 Jun 2023 20:11:00 -0700 From: Jakub Kicinski To: John Fastabend Cc: Toke =?UTF-8?B?SMO4aWxhbmQtSsO4cmdlbnNlbg==?= , Stanislav Fomichev , Alexei Starovoitov , Donald Hunter , bpf , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , KP Singh , Hao Luo , Jiri Olsa , Network Development Subject: Re: [RFC bpf-next v2 11/11] net/mlx5e: Support TX timestamp metadata Message-ID: <20230630201100.0bb9b1f3@kernel.org> In-Reply-To: <649f78b57358c_30943208c4@john.notmuch> References: <20230622195757.kmxqagulvu4mwhp6@macbook-pro-8.dhcp.thefacebook.com> <649637e91a709_7bea820894@john.notmuch> <20230624143834.26c5b5e8@kernel.org> <649b581ded8c1_75d8a208c@john.notmuch> <20230628115204.595dea8c@kernel.org> <87y1k2fq9m.fsf@toke.dk> <649f78b57358c_30943208c4@john.notmuch> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Fri, 30 Jun 2023 17:52:05 -0700 John Fastabend wrote: > Toke H=C3=B8iland-J=C3=B8rgensen wrote: > > Jakub Kicinski writes: > > > Sorry but this is not going to happen without my nack. DPDK was a much > > > cleaner bifurcation point than trying to write datapath drivers in BP= F. > > > Users having to learn how to render descriptors for all the NICs > > > and queue formats out there is not reasonable. Isovalent hired =20 >=20 > I would expect BPF/driver experts would write the libraries for the > datapath API that the network/switch developer is going to use. I would > even put the BPF programs in kernel and ship them with the release > if that helps. >=20 > We have different visions on who the BPF user is that writes XDP > programs I think. Yes, crucially. What I've seen talking to engineers working on TC/XDP BPF at Meta (and I may not be dealing with experts, Martin would have a broader view) is that they don't understand basics like s/g or details of checksums. I don't think it is reasonable to call you, Maxim, Nik and co. "users". We're risking building system so complex normal people will _need_ an overlay on top to make it work. > > > a lot of former driver developers so you may feel like it's a good > > > idea, as a middleware provider. But for the rest of us the matrix > > > of HW x queue format x people writing BPF is too large. If we can =20 >=20 > Its nice though that we have good coverage for XDP so the matrix > is big. Even with kfuncs though we need someone to write support. > My thought is its just a question of if they write it in BPF > or in C code as a reader kfunc. I suspect for these advanced features > its only a subset at least upfront. Either way BPF or C you are > stuck finding someone to write that code. Right, but kernel is a central point where it can be written, reviewed, cross-optimized and stored. > > > write some poor man's DPDK / common BPF driver library to be selected > > > at linking time - we can as well provide a generic interface in > > > the kernel itself. Again, we never merged explicit DPDK support,=20 > > > your idea is strictly worse. =20 > >=20 > > I agree: we're writing an operating system kernel here. The *whole > > point* of an operating system is to provide an abstraction over > > different types of hardware and provide a common API so users don't have > > to deal with the hardware details. =20 >=20 > And just to be clear what we sacrifice then is forwards/backwards > portability. Forward compatibility is also the favorite word of HW vendors when=20 they create proprietary interfaces. I think it's incorrect to call cutting functionality out of a project forward compatibility. If functionality is moved the surface of compatibility is different. > If its a kernel kfunc we need to add a kfunc for > every field we want to read and it will only be available then. > Further, it will need some general agreement that its useful for > it to be added. A hardware vendor wont be able to add some arbitrary > field and get access to it. So we lose this by doing kfuncs. We both know how easy it is to come up with useful HW, so I'm guessing this is rhetorical. > Its pushing complexity into the kernel that we maintain in kernel > when we could push the complexity into BPF and maintain as user > space code and BPF codes. Its a choice to make I think. Right, and I believe having the code in the kernel, appropriately integrated with the drivers is beneficial. The main argument against=20 it is that in certain environments kernels are old. But that's a very destructive argument.