From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.1 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5DC0C433DF for ; Fri, 26 Jun 2020 09:37:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7280021531 for ; Fri, 26 Jun 2020 09:37:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ILs95nAV" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726384AbgFZJhy (ORCPT ); Fri, 26 Jun 2020 05:37:54 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:57496 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725280AbgFZJhy (ORCPT ); Fri, 26 Jun 2020 05:37:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593164272; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=mIo8l5VfRJsWL8p3RKk3Ay5PZgeekc+3GNjG0us+Ooc=; b=ILs95nAVhRQH7ELpHIkTND5ogcGK4pcmiVQnQuFBVyVrYM16qWjj0kdnPkXr83BkmqN6Ho HkFK8K4T50ADI6tIIvk+g5nruXJuVdXxtJ/loNaAVIYUzphg1e3Rvrwvq+M/tayJ8aol81 q9tuoPNVxKQtBjeTTo7PplF+2NjKpHo= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-4-6j3q-ILdMjyG6UIQWFjEOw-1; Fri, 26 Jun 2020 05:37:49 -0400 X-MC-Unique: 6j3q-ILdMjyG6UIQWFjEOw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E3FB8800C64; Fri, 26 Jun 2020 09:37:47 +0000 (UTC) Received: from krava (unknown [10.40.193.90]) by smtp.corp.redhat.com (Postfix) with SMTP id 235AD60E1C; Fri, 26 Jun 2020 09:37:45 +0000 (UTC) Date: Fri, 26 Jun 2020 11:37:45 +0200 From: Jiri Olsa To: Alexey Budankov Cc: Arnaldo Carvalho de Melo , Namhyung Kim , Alexander Shishkin , Peter Zijlstra , Ingo Molnar , Andi Kleen , linux-kernel Subject: Re: [PATCH v8 01/13] tools/libperf: avoid moving of fds at fdarray__filter() call Message-ID: <20200626093745.GM2719003@krava> References: <0781a077-aa82-5b4a-273e-c17372a72b93@linux.intel.com> <3d36dc7a-4249-096c-7554-80e6d290eac5@linux.intel.com> <20200625171405.GL2719003@krava> <688910f3-289e-d63e-79e3-0a17a6df0e9e@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <688910f3-289e-d63e-79e3-0a17a6df0e9e@linux.intel.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Jun 25, 2020 at 10:32:29PM +0300, Alexey Budankov wrote: > > On 25.06.2020 20:14, Jiri Olsa wrote: > > On Wed, Jun 24, 2020 at 08:19:32PM +0300, Alexey Budankov wrote: > >> > >> On 17.06.2020 11:35, Alexey Budankov wrote: > >>> > >>> Skip fds with zeroed revents field from count and avoid fds moving > >>> at fdarray__filter() call so fds indices returned by fdarray__add() > >>> call stay the same and can be used for direct access and processing > >>> of fd revents status field at entries array of struct fdarray object. > >>> > >>> Signed-off-by: Alexey Budankov > >>> --- > >>> tools/lib/api/fd/array.c | 11 +++++------ > >>> tools/perf/tests/fdarray.c | 20 ++------------------ > >>> 2 files changed, 7 insertions(+), 24 deletions(-) > >>> > >>> diff --git a/tools/lib/api/fd/array.c b/tools/lib/api/fd/array.c > >>> index 58d44d5eee31..97843a837370 100644 > >>> --- a/tools/lib/api/fd/array.c > >>> +++ b/tools/lib/api/fd/array.c > >>> @@ -93,22 +93,21 @@ int fdarray__filter(struct fdarray *fda, short revents, > >>> return 0; > >>> > >>> for (fd = 0; fd < fda->nr; ++fd) { > >>> + if (!fda->entries[fd].revents) > >>> + continue; > >>> + > >> > >> So it looks like this condition also filters out non signaling events fds, not only > >> control and others fds, and this should be somehow avoided so such event related fds > >> would be counted. Several options have been proposed so far: > >> > >> 1) Explicit typing of fds via API extension and filtering based on the types: > >> a) with separate fdarray__add_stat() call > >> b) with type arg of existing fdarray__add() call > >> c) various memory management design is possible > >> > >> 2) Playing tricks with fd positions inside entries and assumptions on fdarray API calls ordering > >> - looks more like a hack than a designed solution > >> > >> 3) Rewrite of fdarray class to allocate separate object for every added fds > >> - can be replaced with nonscrewing of fds by __filter() > >> > >> 4) Distinct between fds types at fdarray__filter() using .revents == 0 condition > >> - seems to have corner cases and thus not applicable > >> > >> 5) Extension of fdarray__poll(, *arg_ptr, arg_size) with arg of fds array to atomically poll > >> on fdarray_add()-ed fds and external arg fds and then external arg fds processing > >> > >> 6) Rewrite of fdarray class on epoll() call basis > >> - introduces new scalability restrictions for Perf tool > > > > hum, how many fds for polling do you expect in your workloads? > > Currently it is several hundreds so default of 1K is easily hit and > "Profile a Large Number of PMU Events on Multi-Core Systems" section [1] > recommends: > > soft nofile 65535 > hard nofile 65535 I'm confused, are you talking about file descriptors limit now? this wont be affected by epoll change.. what do I miss? I thought your concern was fs.epoll.max_user_watches, which has default value that seems to be enough: $ cat /proc/sys/fs/epoll/max_user_watches 3169996 jirka > > for for /etc/security/limits.conf settings. > > ~Alexey > > [1] https://software.intel.com/content/www/us/en/develop/documentation/vtune-cookbook/top/configuration-recipes/profiling-hardware-without-sampling-drivers.html > > > > > jirka > > >