From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.lttng.org (lists.lttng.org [167.114.26.123]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 157A5C433EF for ; Wed, 15 Jun 2022 14:16:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=lists.lttng.org; s=default; t=1655302563; bh=09ZgoX2hYtRZtxPckXwvL43r1G641LjCLyGK6CXk+SQ=; h=Date:To:Cc:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=KoZt0Cpk4LfBfjA0Iu+sY255Bg66mQxHy5H4tky7YvrORIIuO+fkCq4MUz+0pytwW tGHuOuLPp4m9w73sR3hUbJf/n/y8o+KvBbIkoqtOLg5vB7jYES1VeMmusg0ewlAbqK kql+5VebC5DBo/hQyH138XUekzKK/UwgJolOoSJFlScP4WcwzogPsqv+HUU+7Wmp6M zCQXa/8OKsnB3zlMQXRBWwMOFLOCwQ0S7Dpe7M0L0Fu43KN4IkP7mFzencmr/yAumK FknOCWPmKEJwACNvdx4IX2UXpju2vj44P/SOFfANl+aNv9ZAbcnw48fcGz3+bUB/Y0 2AylgGCyT2nYg== Received: from lists-lttng01.efficios.com (localhost [IPv6:::1]) by lists.lttng.org (Postfix) with ESMTP id 4LNS5Z2btZz8th; Wed, 15 Jun 2022 10:16:02 -0400 (EDT) Received: from mail.efficios.com (mail.efficios.com [167.114.26.124]) by lists.lttng.org (Postfix) with ESMTPS id 4LNS5X2nmhz8X6 for ; Wed, 15 Jun 2022 10:16:00 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mail.efficios.com (Postfix) with ESMTP id ABB5D409EA9 for ; Wed, 15 Jun 2022 10:15:58 -0400 (EDT) Received: from mail.efficios.com ([127.0.0.1]) by localhost (mail03.efficios.com [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id dFUAsVKch22G; Wed, 15 Jun 2022 10:15:58 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mail.efficios.com (Postfix) with ESMTP id 4B5E3409EA8; Wed, 15 Jun 2022 10:15:58 -0400 (EDT) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.efficios.com 4B5E3409EA8 X-Virus-Scanned: amavisd-new at efficios.com Received: from mail.efficios.com ([127.0.0.1]) by localhost (mail03.efficios.com [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id knOj5b3fsyr8; Wed, 15 Jun 2022 10:15:58 -0400 (EDT) Received: from mail03.efficios.com (mail03.efficios.com [167.114.26.124]) by mail.efficios.com (Postfix) with ESMTP id 41CB4409E19; Wed, 15 Jun 2022 10:15:58 -0400 (EDT) Date: Wed, 15 Jun 2022 10:15:58 -0400 (EDT) To: Minlan Wang Cc: lttng-dev Message-ID: <396090936.61897.1655302558168.JavaMail.zimbra@efficios.com> In-Reply-To: <20220615034900.GA187807@localhost.localdomain>+5075995E84B158A6> References: <20220614035533.GA174967@localhost.localdomain> <765519142.60031.1655221996546.JavaMail.zimbra@efficios.com> <20220615034900.GA187807@localhost.localdomain> MIME-Version: 1.0 X-Originating-IP: [167.114.26.124] X-Mailer: Zimbra 8.8.15_GA_4272 (ZimbraWebClient - FF100 (Linux)/8.8.15_GA_4257) Thread-Topic: urcu workqueue thread uses 99% of cpu while workqueue is empty Thread-Index: qESOu78Fq/8Ruep1pSS6IoI/2YqI1g== Subject: Re: [lttng-dev] urcu workqueue thread uses 99% of cpu while workqueue is empty X-BeenThere: lttng-dev@lists.lttng.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: LTTng development list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Mathieu Desnoyers via lttng-dev Reply-To: Mathieu Desnoyers Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: lttng-dev-bounces@lists.lttng.org Sender: "lttng-dev" ----- On Jun 14, 2022, at 11:49 PM, Minlan Wang wangminlan@szsandstone.com wrote: > Hi, Mathieu, > The commit on branch stable-0.12 correponds to the tarball we downloaded is > this: > > commit d5277e807192178ddb79f56ecbbd5ac3c4994f60 (HEAD -> v0.12.1.b, tag: > v0.12.1) > Author: Mathieu Desnoyers > Date: Wed Apr 22 08:51:41 2020 -0400 > > Version 0.12.1 > > Signed-off-by: Mathieu Desnoyers > > The OS we are using is CentOS Linux release 7.9.2009 (Core), not CentOS 8.2 > as mentioned before. And the kernel version is: 3.10.0-1160.el7.x86_64. One more thing: did you apply any changes on top of the v0.12.1 userspace rcu tag ? Also, can you provide the output of "./configure" on your system ? I'm also interested in knowing the contents of include/urcu/config.h after building liburcu. Thanks, Mathieu > > On Tue, Jun 14, 2022 at 11:53:16AM -0400, Mathieu Desnoyers wrote: >> Also, I notice that you appear to be using an internal liburcu API (not public) >> from outside of the liburcu project, which is not really expected. > We are trying to move some linux kernel module function into userspace, and > found that the urcu internal workqueue.h has all the things we need for a > replace for kernel workqueue, so we decided to give it a try. > >> >> If your process forks without exec, make sure you wire up the equivalent of >> rculfhash pthread_atfork functions which call urcu_workqueue_pause_worker(), >> urcu_workqueue_resume_worker() and urcu_workqueue_create_worker(). > There's no fork/exec in the process who is calling alloc_workqueue, and the > threads who are enqueue work into the workqueue is created by calling > pthread_create. > >> >> Also, can you validate of you have many workqueue worker threads trying to >> dequeue from the same workqueue in parallel ? This is unsupported and would >> cause the kind of issues you are observing here. > The workqueue thread is created by calling urcu_workqueue_create in the code > below, and it is the only thread which will dequeue work from the workqueue. > Though, there are multiple threads who will enqueue work by calling > urcu_workqueue_queue_work(wq, work, work->func). > --- > static void workqueue_init_fn(struct urcu_workqueue *workqueue, void *priv) > { > pthread_t tid; > const char *name; > char thread_name[16] = {0}; > > if (!priv) > return; > > name = (const char *)priv; > tid = pthread_self(); > > memcpy(thread_name, name, 15); > if (pthread_setname_np(tid, thread_name)) { > pr_err("failed to set thread name for workqueue %s\n", name); > } > > urcu_memb_register_thread(); > } > > static void workqueue_finalize_fn(struct urcu_workqueue *workqueue, void > *priv) > { > urcu_memb_unregister_thread(); > if (priv) > free(priv); > } > > struct workqueue_struct *alloc_workqueue(const char *fmt, > unsigned int flags, > int max_active, ...) > { > const char *name; > > name = strdup(fmt); > if (!name) { > pr_err("failed to dup name for workqueue %s\n", fmt); > return NULL; > } > > return urcu_workqueue_create(0, -1, (void *)name, > NULL, /* grace */ > workqueue_init_fn, /* init */ > workqueue_finalize_fn, /* finalize */ > NULL, /* before wait */ > NULL, /* after wake up */ > NULL, /* before pasue */ > NULL); /* after resume */ > } > --- > > B.R > Minlan -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com _______________________________________________ lttng-dev mailing list lttng-dev@lists.lttng.org https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev