From mboxrd@z Thu Jan 1 00:00:00 1970 From: Vincent JARDIN Subject: Re: [dpdk-announce] important design choices - statistics - ABI Date: Wed, 17 Jun 2015 15:21:28 +0200 Message-ID: <55817458.5020104@6wind.com> References: <9092314.MoyqUJ5VU2@xps13> <20150617103521.GB24677@hmsreliant.think-freely.org> <55816489.5080706@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Cc: "dev@dpdk.org" To: Panu Matilainen , Neil Horman , Thomas Monjalon Return-path: Received: from mail-wi0-f173.google.com (mail-wi0-f173.google.com [209.85.212.173]) by dpdk.org (Postfix) with ESMTP id 2885DC444 for ; Wed, 17 Jun 2015 15:21:34 +0200 (CEST) Received: by wiga1 with SMTP id a1so139486599wig.0 for ; Wed, 17 Jun 2015 06:21:34 -0700 (PDT) In-Reply-To: <55816489.5080706@redhat.com> List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 17/06/2015 14:14, Panu Matilainen wrote: > (initially accidentally sent to announce, resending to dev) > > On 06/17/2015 01:35 PM, Neil Horman wrote: >> On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote: >>> Hi all, >>> >>> Sometimes there are some important discussions about architecture or >>> design >>> which require opinions from several developers. Unfortunately, we cannot >>> read every threads. Maybe that using the announce mailing list will help >>> to bring more audience to these discussions. >>> Please note that >>> - the announce@ ML is moderated to keep a low traffic, >>> - every announce email is forwarded to dev@ ML. >>> In case you want to reply to this email, please use dev@dpdk.org >>> address. >>> >>> There were some debates about software statistics disabling. >>> Should they be always on or possibly disabled when compiled? >>> We need to take a decision shortly and discuss (or agree) this proposal: >>> http://dpdk.org/ml/archives/dev/2015-June/019461.html >>> >>> During the development of the release 2.0, there was an agreement to >>> keep >>> ABI compatibility or to bring new ABI while keeping old one during >>> one release. >>> In case it's not possible to have this transition, the (exceptional) >>> break >>> should be acknowledged by several developers. >>> http://dpdk.org/doc/guides-2.0/rel_notes/abi.html >>> There were some interesting discussions but not a lot of participants: >>> http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus=8461 >>> >>> >>> During the current development cycle for the release 2.1, the ABI >>> question >>> arises many times in different threads. >>> To add the hash key size field, it is proposed to use a struct >>> padding gap: >>> http://dpdk.org/ml/archives/dev/2015-June/019386.html >>> To support the flow director for VF, there is no proposal yet: >>> http://dpdk.org/ml/archives/dev/2015-June/019343.html >>> To add the speed capability, it is proposed to break ABI in the >>> release 2.2: >>> http://dpdk.org/ml/archives/dev/2015-June/019225.html >>> To support vhost-user multiqueues, it is proposed to break ABI in 2.2: >>> http://dpdk.org/ml/archives/dev/2015-June/019443.html >>> To add the interrupt mode, it is proposed to add a build-time option >>> CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking >>> binary: >>> http://dpdk.org/ml/archives/dev/2015-June/018947.html >>> To add the packet type, there is a proposal to add a build-time option >>> CONFIG_RTE_NEXT_ABI common to every ABI breaking features: >>> http://dpdk.org/ml/archives/dev/2015-June/019172.html >>> We must also better document how to remove a deprecated ABI: >>> http://dpdk.org/ml/archives/dev/2015-June/019465.html >>> The ABI compatibility is a new constraint and we need to better >>> understand >>> what it means and how to proceed. Even the macros are not yet well >>> documented: >>> http://dpdk.org/ml/archives/dev/2015-June/019357.html >>> >>> Thanks for your attention and your participation in these important >>> choices. >>> >> >> Thomas- >> Just to re-iterate what you said earlier, and what was discussed >> in the >> previous ABI discussions >> >> 1) ABI stability was introduced to promote DPDK's ability to be >> included with >> various linux and BSD distributions. Distributions, by and large, favor >> building libraries as DSO's, favoring security and updatability in >> favor of all >> out performance. >> >> 2) The desire was to put DPDK developers in a mindset whereby ABI >> stability was >> something they needed to think about during development, as the DPDK >> exposes >> many data structures and instances that cannot be changed without >> breaking ABI >> >> 3) The versioning mechanism was introduced to allow for backward >> compatibility >> during periods in which we needed to support both an old an new ABI >> >> 4) As Stephan and others point out, its not expected that we will >> always be able >> to maintain ABI, and as such an easy library versioning mechanism was >> introduced >> to prevent the loading of an incompatible library with an older >> application >> >> 5) The ABI policy was introduced to create a method by which new ABI >> facets >> could be scheduled while allowing distros to prepare their downstream >> users for >> the upcomming changes. >> >> >> It seems to me, looking back over these last few months, that we're >> falling down >> a bit on our use of (3). I've seen several people take advantage of >> the ABI >> scheduled updates, but no one has tried the versioning interface, and >> as a >> result patches are getting delayed, which was never my intent. Not >> sure whats >> to be done about that, but we should probably address it. Is use of the >> versionnig interface just too hard or convoluted? > > To me it seems that by far the biggest problem with ABI stability in > DPDK is features requiring changes to public structs (often directly > allocated and accessed by apps), which is something the symbol > versioning doesn't directly help with, you'd need to version the structs > too. > > One only needs to glance at the glibc documentation on how to accomplish > it [1] to see it gets rather involved. Glibc promises backwards > compatibility for life, so the effort is justified. However in where > DPDK we're talking about extending compatibility for a few months by > minimum requirement, people are unlikely to bother. > > [1] https://sourceware.org/glibc/wiki/Development/Versioning_A_Structure Does it means that it requires a specific engineering so we are back to the needs of using two layers: a- the direct calls to "non ABI stable layers" (current librte*) for those we can/are fine to use it, b- the calls thru such layers that guarantee the ABI stabilities (a librte_compat?) for those who needs it. Could both be exposed by the distributions so they can be consumed? Thank you, Vincent