From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.9 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7C2EC18E5B for ; Mon, 16 Mar 2020 21:01:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8EA28206C0 for ; Mon, 16 Mar 2020 21:01:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="F/mKIGm2" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732571AbgCPVBi (ORCPT ); Mon, 16 Mar 2020 17:01:38 -0400 Received: from mail-pg1-f195.google.com ([209.85.215.195]:38653 "EHLO mail-pg1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732607AbgCPVBh (ORCPT ); Mon, 16 Mar 2020 17:01:37 -0400 Received: by mail-pg1-f195.google.com with SMTP id x7so10422914pgh.5 for ; Mon, 16 Mar 2020 14:01:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=FykSs6ArRQ1VTW0edGQU9r+oRpdoLevjlWnrbNH8SzQ=; b=F/mKIGm2VSDsMdeG2UN7tdnBaLkXajwusX1FSQGsszz2c2mkMYP00hgIofJTgrQDW1 SlYcjtn0dgAw7izmdBv71zLnPNljO/N2mb5wfEaVhtU0iQ3ICwa7+j1vTDbBcOdxXrmc L4SVReMbOZrT0y9fUuU3TP449id5RqEw/5/u0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=FykSs6ArRQ1VTW0edGQU9r+oRpdoLevjlWnrbNH8SzQ=; b=dRsfb/ThK6Qpwe2DPsTUBnICBWVZP6vt7YW3G9RbkUBIwtApnIgp1d0MVditjNA4Ci xR25UZG7QB9yuYPfjeYGbOl8XXN29eTv7nZkGzke9nhbKg5rND2ykryPt+wRXxK4BjXC 2QCanNy91dYM7a+EJ91OnpduuB0uo2+whr77h19TTKGB5pFCTL3S99QDPzfgu+1k+vZX 1LLNP7Ir/wE/cTfVpoIrixDX4R/PJiZvypy3KduiA1ybLJSRYiiMJCIat93Q00jrvoLJ DKQuzALG1wAQkDWY8Wv3Ok4+nvtq+wEpQO5ALTFH22uPtL/rY9y09SNFHUw9oTMdWg76 dGSg== X-Gm-Message-State: ANhLgQ35KxLSA1IRpeE7zGw5e2deYsDpf3HFZ8ZKG+q4QG8xaGGRgqE8 GESnF7gNnLoq0UPx7go+i4e+7A== X-Google-Smtp-Source: ADFU+vspzRBYngPdmN2Sb7yi3ynFWRC4K625gLz2x5YvOEj/9r9JVqHRh4cG4ZtIaq7Mzq78lwob+g== X-Received: by 2002:a63:7c4:: with SMTP id 187mr1570289pgh.369.1584392495839; Mon, 16 Mar 2020 14:01:35 -0700 (PDT) Received: from www.outflux.net (smtp.outflux.net. [198.145.64.163]) by smtp.gmail.com with ESMTPSA id p4sm747093pfg.163.2020.03.16.14.01.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Mar 2020 14:01:34 -0700 (PDT) Date: Mon, 16 Mar 2020 14:01:33 -0700 From: Kees Cook To: Jakub Kicinski Cc: "Bird, Tim" , "shuah@kernel.org" , "luto@amacapital.net" , "wad@chromium.org" , "linux-kselftest@vger.kernel.org" , "netdev@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "kernel-team@fb.com" Subject: Re: [PATCH v2 0/4] kselftest: add fixture parameters Message-ID: <202003161356.6CD6783@keescook> References: <20200314005501.2446494-1-kuba@kernel.org> <202003132049.3D0CDBB2A@keescook> <20200316130416.4ec9103b@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200316130416.4ec9103b@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com> Sender: linux-kselftest-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org On Mon, Mar 16, 2020 at 01:04:16PM -0700, Jakub Kicinski wrote: > Variant sounds good too, although the abbreviation would be VAR? > Which isn't ideal. > > But I really don't care so whoever cares the most please speak up :P Let's go with "variant" and just spell it out. > > BTW - Fuego has a similar feature for naming a collection of test > > parameters with specific values (if I understand this proposed > > feature correctly). Fuego's feature was named a long time ago > > (incorrectly, I think) and it continues to bug me to this day. > > It was named 'specs', and after giving it considerable thought > > I've been meaning to change it to 'variants'. > > > > Just a suggestion for consideration. The fact that Fuego got this > > wrong is what motivates my suggestion today. You have to live > > with this kind of stuff a long time. :-) > > > > We ran into some issues in Fuego with this concept, that motivate > > the comments below. I'll use your 'instance' terminology in my comments > > although the terminology is different in Fuego. > > > > > Also a change in reporting: > > > > > > struct __fixture_params_metadata no_param = { .name = "", }; > > > > > > Let's make ".name = NULL" here, and then we can detect instantiation: > > > > > > printf("[ RUN ] %s%s%s.%s\n", f->name, p->name ? "." : "", > > > p->name ?: "", t->name); > > Do I have to make it NULL or is it okay to test p->name[0] ? > That way we can save one ternary operator from the litany.. I did consider Tim's idea of having them all say 'default', but since the bulk of tests aren't going to have variants, I don't want to spam the report with words I have to skip over. And empty-check (instead of NULL) is fine by me. > To me global.default.XYZ is a mouthful. so in my example (perhaps that > should have been part of the cover letter) I got: > > [ RUN ] global.keysizes <= non-fixture test > [ OK ] global.keysizes > [ RUN ] tls_basic.base_base <= fixture: "tls_basic", no params > [ OK ] tls_basic.base_base > [ RUN ] tls12.sendfile <= fixture: "tls", param: "12" > [ OK ] tls12.sendfile > [ RUN ] tls13.sendfile <= fixture: "tls", param: "13" > [ OK ] tls13.sendfile (same fixture, diff param) > > And users can start inserting underscores themselves if they really > want. (For TLS I was considering different ciphers but they don't impact > testing much.) The reason I'd like a dot is just for lay-person grep-ability and to avoid everyone needing to remember to add separator prefixes -- there should just be a common one. e.g. searching for "tls13" in the tree wouldn't find the test (since it's actually named "tls" and "13" is separate places). (I mean, sure, searching for "tls" is also insane, but I think I made my point.) -Kees -- Kees Cook