From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.3 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30EDCC3A589 for ; Thu, 15 Aug 2019 15:21:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 031C2206C1 for ; Thu, 15 Aug 2019 15:21:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=fomichev-me.20150623.gappssmtp.com header.i=@fomichev-me.20150623.gappssmtp.com header.b="YoynTap7" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728766AbfHOPVD (ORCPT ); Thu, 15 Aug 2019 11:21:03 -0400 Received: from mail-pf1-f193.google.com ([209.85.210.193]:41334 "EHLO mail-pf1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728700AbfHOPVD (ORCPT ); Thu, 15 Aug 2019 11:21:03 -0400 Received: by mail-pf1-f193.google.com with SMTP id 196so1477757pfz.8 for ; Thu, 15 Aug 2019 08:21:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fomichev-me.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=gWCpJv0MdsDcCjYfH3QHT9RFuWsfjx5IZK9e9mVsAXA=; b=YoynTap7BbTEYNR98yeMXMzDCfOXQqvgHIvw9T4f5smrgxjtf6N2/WreLJ60l+vU8n sQRM31Vvc9uMxiWvB7x0jIAhk3t5WryONIJ8hhIjS3dLDVABCIZzRE+TXCzCiZjfaGuG f1jJN3z+bCNepyPs3HAWSeBLbaPAolhXY+2s/SjPXh3pZmoAX7dKANCwBtAQ3iQ2sDKb ltS7eqL83EnQjGI7hoUZx+ELNE9WJZT48PAtQx+EQRhCdrwPvkO/1LYmAayjIzWxCk15 4gi+1qPX5o4KCrVHoRblzeR0s13OMlq9rqGOKoUHeTH6R88neSEhC2M8lFhg4fMh0CXI x/Xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=gWCpJv0MdsDcCjYfH3QHT9RFuWsfjx5IZK9e9mVsAXA=; b=o6nfm2GFpT64WycyClR+ov10Sri3L6AnmDtktYRGyB8a52fDGIpuHourl6JOLxGK8t NcL6gzGYSOTiO3kVUg7+2JfbDSZb/VB1n5ACmzjaNc3koT0MBh+BqGDwGui4usfR+jr1 0M+PMkTKAAlZ/pTFAWgxWs6yWygdcqMu8jQUueZ3lJcdajZUvTCKWBzpc3PqV9FC/Slv MhPYrnBeSik6h/kngmcpwzUX0Rn5xswV0IWGepRGg5djrFmUxtOxQnqT9E5IVIp6r7go hWnFN4mqerRR32c54twcTmu7F6+JUFbQBJH1JYDfMDiAhAbLnwxhpRnbVy5rjEVdteNd +31A== X-Gm-Message-State: APjAAAVmVUthXS6uyUCbTx+kbW6dkk9xuO7Sj62GP3lW0pABe3UhylnQ Hs4khE2XTqUZhYFul0KbMNaNrg== X-Google-Smtp-Source: APXvYqw55yeAxnoRONyct3JREtm0sRJmsrCOOytkTUf8aXAzcDNBNx85nZ3TKKmLqMaScwLjx2UGkg== X-Received: by 2002:aa7:8f2e:: with SMTP id y14mr5970297pfr.113.1565882462524; Thu, 15 Aug 2019 08:21:02 -0700 (PDT) Received: from localhost ([2601:646:8f00:18d9:d0fa:7a4b:764f:de48]) by smtp.gmail.com with ESMTPSA id r75sm3268036pfc.18.2019.08.15.08.21.01 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 15 Aug 2019 08:21:01 -0700 (PDT) Date: Thu, 15 Aug 2019 08:21:00 -0700 From: Stanislav Fomichev To: Toshiaki Makita Cc: Alexei Starovoitov , Daniel Borkmann , Martin KaFai Lau , Song Liu , Yonghong Song , "David S. Miller" , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend , Jamal Hadi Salim , Cong Wang , Jiri Pirko , netdev@vger.kernel.org, bpf@vger.kernel.org, William Tu Subject: Re: [RFC PATCH bpf-next 00/14] xdp_flow: Flow offload to XDP Message-ID: <20190815152100.GN2820@mini-arch> References: <20190813120558.6151-1-toshiaki.makita1@gmail.com> <20190814170715.GJ2820@mini-arch> <14c4a876-6f5d-4750-cbe4-19622f64975b@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <14c4a876-6f5d-4750-cbe4-19622f64975b@gmail.com> User-Agent: Mutt/1.12.1 (2019-06-15) Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org On 08/15, Toshiaki Makita wrote: > On 2019/08/15 2:07, Stanislav Fomichev wrote: > > On 08/13, Toshiaki Makita wrote: > > > * Implementation > > > > > > xdp_flow makes use of UMH to load an eBPF program for XDP, similar to > > > bpfilter. The difference is that xdp_flow does not generate the eBPF > > > program dynamically but a prebuilt program is embedded in UMH. This is > > > mainly because flow insertion is considerably frequent. If we generate > > > and load an eBPF program on each insertion of a flow, the latency of the > > > first packet of ping in above test will incease, which I want to avoid. > > Can this be instead implemented with a new hook that will be called > > for TC events? This hook can write to perf event buffer and control > > plane will insert/remove/modify flow tables in the BPF maps (contol > > plane will also install xdp program). > > > > Why do we need UMH? What am I missing? > > So you suggest doing everything in xdp_flow kmod? You probably don't even need xdp_flow kmod. Add new tc "offload" mode (bypass) that dumps every command via netlink (or calls the BPF hook where you can dump it into perf event buffer) and then read that info from userspace and install xdp programs and modify flow tables. I don't think you need any kernel changes besides that stream of data from the kernel about qdisc/tc flow creation/removal/etc. But, I haven't looked at the series deeply, so I might be missing something :-) > I also thought about that. There are two phases so let's think about them separately. > > 1) TC block (qdisc) creation / eBPF load > > I saw eBPF maintainers repeatedly saying eBPF program loading needs to be > done from userland, not from kernel, to run the verifier for safety. > However xdp_flow eBPF program is prebuilt and embedded in kernel so we may > allow such programs to be loaded from kernel? I currently don't have the will > to make such an API as loading can be done with current UMH mechanism. > > 2) flow insertion / eBPF map update > > Not sure if this needs to be done from userland. One concern is that eBPF maps can > be modified by unrelated processes and we need to handle all unexpected state of maps. > Such handling tends to be difficult and may cause unexpected kernel behavior. > OTOH updating maps from kmod may reduces the latency of flow insertion drastically. Latency from the moment I type 'tc filter add ...' to the moment the rule is installed into the maps? Does it really matter? Do I understand correctly that both of those events (qdisc creation and flow insertion) are triggered from tcf_block_offload_cmd (or similar)? > Alexei, Daniel, what do you think? > > Toshiaki Makita