From mboxrd@z Thu Jan  1 00:00:00 1970
From: Luca Boccassi <luca.boccassi@gmail.com>
Subject: Re: [PATCH 1/7] member: implement main API
Date: Sun, 03 Sep 2017 00:49:34 +0100
Message-ID: <1504396174.21816.0.camel@gmail.com>
References: <1503361193-36699-1-git-send-email-yipeng1.wang@intel.com>
 <1503361193-36699-2-git-send-email-yipeng1.wang@intel.com>
 <1503396136.6638.1.camel@gmail.com>
 <51975604-23a5-ad97-eb0a-62716cb764df@intel.com>
 <1503568544.12733.4.camel@gmail.com>
 <02c6c322-a35e-c674-9f3c-c270dfe54e54@intel.com>
 <1504356923.26853.4.camel@gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Cc: "dev@dpdk.org" <dev@dpdk.org>
To: Ferruh Yigit <ferruh.yigit@intel.com>, Yipeng Wang <yipeng1.wang@intel.com>
Return-path: <dev-bounces@dpdk.org>
Received: from mail-wm0-f68.google.com (mail-wm0-f68.google.com [74.125.82.68])
 by dpdk.org (Postfix) with ESMTP id 616807CAB
 for <dev@dpdk.org>; Sun,  3 Sep 2017 01:49:37 +0200 (CEST)
Received: by mail-wm0-f68.google.com with SMTP id 187so3050153wmn.0
 for <dev@dpdk.org>; Sat, 02 Sep 2017 16:49:37 -0700 (PDT)
In-Reply-To: <1504356923.26853.4.camel@gmail.com>
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>

On Sat, 2017-09-02 at 13:55 +0100, Luca Boccassi wrote:
> On Thu, 2017-08-24 at 11:32 +0100, Ferruh Yigit wrote:
> > On 8/24/2017 10:55 AM, Luca Boccassi wrote:
> > > On Thu, 2017-08-24 at 10:35 +0100, Ferruh Yigit wrote:
> > > > On 8/22/2017 11:02 AM, Luca Boccassi wrote:
> > > > > On Mon, 2017-08-21 at 17:19 -0700, Yipeng Wang wrote:
> > > > > > Membership library is an extension and generalization of a
> > > > > > traditional
> > > > > > filter (for example Bloom Filter) structure. In general,
> > > > > > the
> > > > > > Membership
> > > > > > library is a data structure that provides a "set-summary"
> > > > > > and
> > > > > > responds
> > > > > > to set-membership queries of whether a certain element
> > > > > > belongs to
> > > > > > a
> > > > > > set(s). A membership test for an element will return the
> > > > > > set
> > > > > > this
> > > > > > element
> > > > > > belongs to or not-found if the element is never inserted
> > > > > > into
> > > > > > the
> > > > > > set-summary.
> > > > > >=20
> > > > > > The results of the membership test is not 100% accurate.
> > > > > > Certain
> > > > > > false positive or false negative probability could exist.
> > > > > > However,
> > > > > > comparing to a "full-blown" complete list of elements, a
> > > > > > "set-
> > > > > > summary"
> > > > > > is memory efficient and fast on lookup.
> > > > > >=20
> > > > > > This patch add the main API definition.
> > > > > >=20
> > > > > > Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
> > > > > > ---
> > > > > > =C2=A0lib/Makefile=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0|=C2=A0=C2=A0=C2=A02 =
+
> > > > > > =C2=A0lib/librte_eal/common/eal_common_log.c=C2=A0=C2=A0=C2=A0|=
=C2=A0=C2=A0=C2=A01 +
> > > > > > =C2=A0lib/librte_eal/common/include/rte_log.h=C2=A0=C2=A0|=C2=
=A0=C2=A0=C2=A01 +
> > > > > > =C2=A0lib/librte_member/Makefile=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0|=C2=A0=C2=A048=
 +++
> > > > > > =C2=A0lib/librte_member/rte_member.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0| 357
> > > > > > +++++++++++++++++++++
> > > > > > =C2=A0lib/librte_member/rte_member.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0| 518
> > > > > > +++++++++++++++++++++++++++++++
> > > > > > =C2=A0lib/librte_member/rte_member_version.map |=C2=A0=C2=A015 =
+
> > > > > > =C2=A07 files changed, 942 insertions(+)
> > > > > > =C2=A0create mode 100644 lib/librte_member/Makefile
> > > > > > =C2=A0create mode 100644 lib/librte_member/rte_member.c
> > > > > > =C2=A0create mode 100644 lib/librte_member/rte_member.h
> > > > > > =C2=A0create mode 100644
> > > > > > lib/librte_member/rte_member_version.map
> > > > > >=20
> > > > > > diff --git a/lib/librte_member/Makefile
> > > > > > b/lib/librte_member/Makefile
> > > > > > new file mode 100644
> > > > > > index 0000000..997c825
> > > > > > --- /dev/null
> > > > > > +++ b/lib/librte_member/Makefile
> > > > > > @@ -0,0 +1,48 @@
> > > > > > +#=C2=A0=C2=A0=C2=A0BSD LICENSE
> > > > > > +#
> > > > > > +#=C2=A0=C2=A0=C2=A0Copyright(c) 2017 Intel Corporation. All ri=
ghts
> > > > > > reserved.
> > > > > > +#=C2=A0=C2=A0=C2=A0All rights reserved.
> > > > > > +#
> > > > > > +#=C2=A0=C2=A0=C2=A0Redistribution and use in source and binary=
 forms,
> > > > > > with
> > > > > > or
> > > > > > without
> > > > > > +#=C2=A0=C2=A0=C2=A0modification, are permitted provided that t=
he
> > > > > > following
> > > > > > conditions
> > > > > > +#=C2=A0=C2=A0=C2=A0are met:
> > > > > > +#
> > > > > > +#=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0* Redistributions of source cod=
e must retain the
> > > > > > above
> > > > > > copyright
> > > > > > +#=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0notice, this list o=
f conditions and the following
> > > > > > disclaimer.
> > > > > > +#=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0* Redistributions in binary for=
m must reproduce the
> > > > > > above
> > > > > > copyright
> > > > > > +#=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0notice, this list o=
f conditions and the following
> > > > > > disclaimer
> > > > > > in
> > > > > > +#=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0the documentation a=
nd/or other materials provided
> > > > > > with
> > > > > > the
> > > > > > +#=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0distribution.
> > > > > > +#=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0* Neither the name of Intel Cor=
poration nor the
> > > > > > names
> > > > > > of
> > > > > > its
> > > > > > +#=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0contributors may be=
 used to endorse or promote
> > > > > > products
> > > > > > derived
> > > > > > +#=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0from this software =
without specific prior written
> > > > > > permission.
> > > > > > +#
> > > > > > +#=C2=A0=C2=A0=C2=A0THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT =
HOLDERS AND
> > > > > > CONTRIBUTORS
> > > > > > +#=C2=A0=C2=A0=C2=A0"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTI=
ES,
> > > > > > INCLUDING,
> > > > > > BUT
> > > > > > NOT
> > > > > > +#=C2=A0=C2=A0=C2=A0LIMITED TO, THE IMPLIED WARRANTIES OF MERCH=
ANTABILITY
> > > > > > AND
> > > > > > FITNESS FOR
> > > > > > +#=C2=A0=C2=A0=C2=A0A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO =
EVENT SHALL
> > > > > > THE
> > > > > > COPYRIGHT
> > > > > > +#=C2=A0=C2=A0=C2=A0OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIR=
ECT,
> > > > > > INDIRECT,
> > > > > > INCIDENTAL,
> > > > > > +#=C2=A0=C2=A0=C2=A0SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGE=
S
> > > > > > (INCLUDING,
> > > > > > BUT
> > > > > > NOT
> > > > > > +#=C2=A0=C2=A0=C2=A0LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS=
 OR
> > > > > > SERVICES;
> > > > > > LOSS OF
> > > > > > USE,
> > > > > > +#=C2=A0=C2=A0=C2=A0DATA, OR PROFITS; OR BUSINESS INTERRUPTION)=
 HOWEVER
> > > > > > CAUSED
> > > > > > AND
> > > > > > ON ANY
> > > > > > +#=C2=A0=C2=A0=C2=A0THEORY OF LIABILITY, WHETHER IN CONTRACT, S=
TRICT
> > > > > > LIABILITY,
> > > > > > OR
> > > > > > TORT
> > > > > > +#=C2=A0=C2=A0=C2=A0(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING=
 IN ANY WAY
> > > > > > OUT
> > > > > > OF
> > > > > > THE USE
> > > > > > +#=C2=A0=C2=A0=C2=A0OF THIS SOFTWARE, EVEN IF ADVISED OF THE PO=
SSIBILITY
> > > > > > OF
> > > > > > SUCH
> > > > > > DAMAGE.
> > > > > > +
> > > > > > +include $(RTE_SDK)/mk/rte.vars.mk
> > > > > > +
> > > > > > +# library name
> > > > > > +LIB =3D librte_member.a
> > > > > > +
> > > > > > +CFLAGS +=3D $(WERROR_FLAGS) -I$(SRCDIR) -O3
> > > > > > +
> > > > >=20
> > > > > This breaks reproducibility as the output directory will be
> > > > > included
> > > > > before the source directory, causing a race - please do
> > > > > something
> > > > > like:
> > > > >=20
> > > > > CFLAGS :=3D -I$(SRCDIR) $(CFLAGS)
> > > > > CFLAGS +=3D $(WERROR_FLAGS) -O3
> > > >=20
> > > > Can we remove "-I$(SRCDIR)" completely by first installing
> > > > headers
> > > > and
> > > > later compiling objects, all using $(RTE_OUT) only?
> > > >=20
> > > > Do you think can this work?
> > >=20
> > > I'm not sure, it might - but given Bruce's effort to port to
> > > Meson
> > > I'm
> > > not sure it's worth spending a lot of time doing big refactoring
> > > of
> > > the
> > > existing build system
> >=20
> > Not big refactoring, following seems worked for me, if you would
> > like
> > to
> > test:
> >=20
> > diff --git a/mk/rte.lib.mk b/mk/rte.lib.mk
> > index 13115d146..643da47da 100644
> > --- a/mk/rte.lib.mk
> > +++ b/mk/rte.lib.mk
> > @@ -59,14 +59,19 @@ endif
> >=20
> >=20
> > =C2=A0_BUILD =3D $(LIB)
> > -_INSTALL =3D $(INSTALL-FILES-y) $(SYMLINK-FILES-y)
> > $(RTE_OUTPUT)/lib/$(LIB)
> > +PREINSTALL =3D $(SYMLINK-FILES-y)
> > +_INSTALL =3D $(INSTALL-FILES-y) $(RTE_OUTPUT)/lib/$(LIB)
> > =C2=A0_CLEAN =3D doclean
> >=20
> > =C2=A0.PHONY: all
> > =C2=A0all: install
> >=20
> > =C2=A0.PHONY: install
> > +ifeq ($(SYMLINK-FILES-y),)
> > =C2=A0install: build _postinstall
> > +else
> > +install: _preinstall build _postinstall
> > +endif
> >=20
> > =C2=A0_postinstall: build
>=20
> Easier than I would have thought :-)
>=20
> I'll give that patch a spin through the repro build grinder next
> week,
> thanks.

Thanks Ferruh, this works in making the build reproducible with the
current CI as an alternative with a fixed build path.

But on the other hand, it will cause the compiler to always embed the
custom build directory in the path to the header in the DWARF .debug
files, rather than the "repository" path:

readelf --debug-dump=3Dline /tmp/69c02db5050024b530a14555b49528c419419f.deb=
ug

<cut>
=C2=A0The Directory Table (offset 0x1b):
=C2=A0 1=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0dpdk/lib/librte_eal/linuxapp/eal
=C2=A0 2=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0/usr/include/x86_64-linux-gnu/bits
=C2=A0 3=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0dpdk/lib/librte_eal/common
=C2=A0 4=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0dpdk/debian/build/shared-root/include
=C2=A0 5=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0dpdk/debian/build/shared-root/include=
/generic
=C2=A0 6=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0/usr/lib/gcc/x86_64-linux-gnu/7/inclu=
de
=C2=A0 7=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0/usr/include/x86_64-linux-gnu/sys
=C2=A0 8=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0/usr/include

=C2=A0The File Name Table (offset 0x15a):
=C2=A0 Entry Dir=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0Time=C2=A0=C2=A0=C2=A0=C2=A0S=
ize=C2=A0=C2=A0=C2=A0=C2=A0Name
=C2=A0 1=C2=A0=C2=A0=C2=A0=C2=A0=C2=A01=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A00=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A00=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0eal.c
=C2=A0 2=C2=A0=C2=A0=C2=A0=C2=A0=C2=A02=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A00=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A00=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0stdio2.h
=C2=A0 3=C2=A0=C2=A0=C2=A0=C2=A0=C2=A03=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A00=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A00=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0eal_filesystem.h
=C2=A0 4=C2=A0=C2=A0=C2=A0=C2=A0=C2=A02=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A00=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A00=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0fcntl2.h
=C2=A0 5=C2=A0=C2=A0=C2=A0=C2=A0=C2=A04=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A00=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A00=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0rte_atomic.h
=C2=A0 6=C2=A0=C2=A0=C2=A0=C2=A0=C2=A05=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A00=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A00=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0rte_atomic.h
=C2=A0 7=C2=A0=C2=A0=C2=A0=C2=A0=C2=A02=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A00=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A00=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0string3.h
=C2=A0 8=C2=A0=C2=A0=C2=A0=C2=A0=C2=A04=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A00=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A00=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0rte_cycles.h
=C2=A0 9=C2=A0=C2=A0=C2=A0=C2=A0=C2=A04=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A00=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A00=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0rte_random.h
=C2=A0 10=C2=A0=C2=A0=C2=A0=C2=A06=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A00=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A00=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0xmmintrin.h
=C2=A0 11=C2=A0=C2=A0=C2=A0=C2=A04=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A00=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A00=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0rte_eal_memconfig.h
=C2=A0 12=C2=A0=C2=A0=C2=A0=C2=A04=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A00=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A00=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0rte_lcore.h

<cut>

note 4 and 5 items in the directory tables, they point to debian/build-
shared-root which is the output directory we use in the package build.=20

This means that 2 builds that are otherwise exactly the same (same
machine type, same build dependencies) will not be binary identical due
to that difference.

On one hand it's a much smaller and nicer patch and there's the Meson
work which will make both obsolete, on the other hand it's not very
reproducibl-y. What do you think?

--=20
Kind regards,
Luca Boccassi