From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DBA6BEB64DD for ; Tue, 11 Jul 2023 21:39:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230006AbjGKVjW (ORCPT ); Tue, 11 Jul 2023 17:39:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41432 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229537AbjGKVjV (ORCPT ); Tue, 11 Jul 2023 17:39:21 -0400 Received: from mail-pl1-x630.google.com (mail-pl1-x630.google.com [IPv6:2607:f8b0:4864:20::630]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4A9DC127; Tue, 11 Jul 2023 14:39:20 -0700 (PDT) Received: by mail-pl1-x630.google.com with SMTP id d9443c01a7336-1b89d47ffb6so32097155ad.2; Tue, 11 Jul 2023 14:39:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689111560; x=1691703560; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:sender :from:to:cc:subject:date:message-id:reply-to; bh=KtT4gFz6TTEFRseSk+gcMtJFveuGAwBprTn2TmvM3QM=; b=qVGDef4XTDiy2pQQAH3Cwnbtqh8ZKWeDR0YpW5oBVmIf8Ft1FfiQFB9j/nigTkpw8I 82vXU59i3SdogLeNz/ABKCqINgA4PTDr9/ZdUPp6ZUB83Z6MdD4YHZx6qWrTB/+YO5SG xt3gCozG7WIu1lWUXRr3EpK+TzZRAcw2ll3vth9VDJ1GAaRvrA2V8kIgGX9kdM2Wgg3l stzdpWOedBvZ0uhTUpLtvsrBCTVN9HRCFZfzCFKFQAEkzZQTdhvlZETvKEQOgCpBxb1y lgv7BtP9O20Qjyp/f7+CQeQ3LbV13czXSXJgwRbcUJP+TfmtuZqC2GkNl1o/MlSqRGEW v5GA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689111560; x=1691703560; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:sender :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=KtT4gFz6TTEFRseSk+gcMtJFveuGAwBprTn2TmvM3QM=; b=hm8E4CWDWpbVZZ/+chXcJ7tdA4AR/wwkc6zVZU92wh0wHs5qBdenkc+UEB6mFSZ4XZ 8SJ5UAwVhW5mwC6+OrNEvwivGpCBvhuDzKI4tId4ng9Yf8BzmutY01tDT7m0gnqOu+yg m4gMsrzsZj+OnXjDY/tAi1tuwoo2Vo420xOaPzZTe6yyrrR4a6eV/vF2zr0h0q6aB3pw xF6n9AtDZhrtv7gkZteIzebInWjRv6qvq8LQ5D3Yemyds4oLFwI3y9BBiiJ9H8tu51Ij OIsAk5FHDmvvpEKkrbWZET54CKfnJQisQEuxfcY0+Za2rbh1vSW1QW0WFRHkoO4+rLHG cQbw== X-Gm-Message-State: ABy/qLY2MGoYPaQXvDgloKfTQQOhhNGNIDy5sCwAO/D0tbP/5sjIELhQ 4P4dsfoJXbL45Ui7piOqFVc= X-Google-Smtp-Source: APBJJlHvfrCF++DlafNT3B6O6EffAGKpXCrh8newwF9nz4kDCviiDR8GuVNpvkxYKklGu+rZ5VQk5Q== X-Received: by 2002:a17:903:24e:b0:1b6:80f0:d969 with SMTP id j14-20020a170903024e00b001b680f0d969mr15429164plh.11.1689111559501; Tue, 11 Jul 2023 14:39:19 -0700 (PDT) Received: from localhost ([2620:10d:c090:400::5:9374]) by smtp.gmail.com with ESMTPSA id bd5-20020a170902830500b001b8761c739csm2362090plb.271.2023.07.11.14.39.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Jul 2023 14:39:19 -0700 (PDT) Sender: Tejun Heo Date: Tue, 11 Jul 2023 11:39:17 -1000 From: Tejun Heo To: Geert Uytterhoeven Cc: Lai Jiangshan , "torvalds@linux-foundation.org" , Peter Zijlstra , Linux Kernel Mailing List , kernel-team@meta.com, Linux PM list , DRI Development , linux-rtc@vger.kernel.org, linux-riscv , netdev , Linux Fbdev development list , Linux MMC List , "open list:LIBATA SUBSYSTEM (Serial and Parallel ATA drivers)" , Linux-Renesas Subject: Re: Consider switching to WQ_UNBOUND messages (was: Re: [PATCH v2 6/7] workqueue: Report work funcs that trigger automatic CPU_INTENSIVE mechanism) Message-ID: References: <20230511181931.869812-1-tj@kernel.org> <20230511181931.869812-7-tj@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-fbdev@vger.kernel.org Hello, On Tue, Jul 11, 2023 at 04:06:22PM +0200, Geert Uytterhoeven wrote: > On Tue, Jul 11, 2023 at 3:55 PM Geert Uytterhoeven wrote: > > > > Hi Tejun, > > > > On Fri, May 12, 2023 at 9:54 PM Tejun Heo wrote: > > > Workqueue now automatically marks per-cpu work items that hog CPU for too > > > long as CPU_INTENSIVE, which excludes them from concurrency management and > > > prevents stalling other concurrency-managed work items. If a work function > > > keeps running over the thershold, it likely needs to be switched to use an > > > unbound workqueue. > > > > > > This patch adds a debug mechanism which tracks the work functions which > > > trigger the automatic CPU_INTENSIVE mechanism and report them using > > > pr_warn() with exponential backoff. > > > > > > v2: Drop bouncing through kthread_worker for printing messages. It was to > > > avoid introducing circular locking dependency but wasn't effective as it > > > still had pool lock -> wci_lock -> printk -> pool lock loop. Let's just > > > print directly using printk_deferred(). > > > > > > Signed-off-by: Tejun Heo > > > Suggested-by: Peter Zijlstra > > > > Thanks for your patch, which is now commit 6363845005202148 > > ("workqueue: Report work funcs that trigger automatic CPU_INTENSIVE > > mechanism") in v6.5-rc1. > > > > I guess you are interested to know where this triggers. > > I enabled CONFIG_WQ_CPU_INTENSIVE_REPORT=y, and tested > > the result on various machines... > > > OrangeCrab/Linux-on-LiteX-VexRiscV with ht16k33 14-seg display and ssd130xdrmfb: > > > > workqueue: check_lifetime hogged CPU for >10000us 4 times, consider > > switching to WQ_UNBOUND > > workqueue: drm_fb_helper_damage_work hogged CPU for >10000us 1024 > > times, consider switching to WQ_UNBOUND > > workqueue: fb_flashcursor hogged CPU for >10000us 128 times, > > consider switching to WQ_UNBOUND > > workqueue: ht16k33_seg14_update hogged CPU for >10000us 128 times, > > consider switching to WQ_UNBOUND > > workqueue: mmc_rescan hogged CPU for >10000us 128 times, consider > > switching to WQ_UNBOUND > > Got one more after a while: > > workqueue: neigh_managed_work hogged CPU for >10000us 4 times, > consider switching to WQ_UNBOUND I wonder whether the right thing to do here is somehow scaling the threshold according to the relative processing power. It's difficult to come up with a threshold which works well across the latest & fastest and really tiny CPUs. I'll think about it some more but if you have some ideas, please feel free to suggest. Thanks. -- tejun