From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E82CC4332F for ; Tue, 20 Dec 2022 21:37:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234154AbiLTVhQ (ORCPT ); Tue, 20 Dec 2022 16:37:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52664 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230048AbiLTVhO (ORCPT ); Tue, 20 Dec 2022 16:37:14 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A84AC13D2D for ; Tue, 20 Dec 2022 13:36:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1671572186; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=knEFZp/Br2lwUYLZXfbzI9ZSnZOaTBCXhLfkLB5psow=; b=ecXMKQbnA8nLHUESey+9ksl4iB+FZTfuZJB2YTIus9GJ7OcbCIKFvBFSBoBRF7YT/dRUiJ YXuyNuS9kaVJbN5i9N+Nkg+ehSCPYIX1dQc4WSjearWrWzpMZc0Wvm5UUEVOlEMZ8edEbZ 0OWOhHRu/4sj0AUhW2yw1zqxeNZWaXs= Received: from mail-yw1-f199.google.com (mail-yw1-f199.google.com [209.85.128.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-655-gyuawPVkMSibkicDDlBk1A-1; Tue, 20 Dec 2022 16:36:24 -0500 X-MC-Unique: gyuawPVkMSibkicDDlBk1A-1 Received: by mail-yw1-f199.google.com with SMTP id 00721157ae682-45e2e1d6d2fso8139917b3.0 for ; Tue, 20 Dec 2022 13:36:24 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=knEFZp/Br2lwUYLZXfbzI9ZSnZOaTBCXhLfkLB5psow=; b=RBs9CfbrGhUWBLm1trk7HbRmE3o6l4W0DAbQJjHOuxoAQKOFUQMzswexDGTCKGpwQh VD25W6CrcxgpoCjVD45DwElbBA/thSe3TqWKXZEqgbYOiOqsaYChAuHwKCq4w+aNUVmD +bclv+lrrhuJuT1rW2mjN+QMYkgVgsxmaYE6y3X5F/FsaCxY6N+YLgG9bTaImGcbSt9W cKyK1+kAwiUYVjjEm490xyjATKbE6Wt6w7cZbaK2NLn35Tr+o6Jae9t64q1prgE2ULfY LZv6NCg/3sX34tS95uuRiIZHyKl2HBCNtFbmIfopLVfcLADxQ+Q/pYPDP9XTLa+fgr6A Svnw== X-Gm-Message-State: AFqh2kqNabXOjTvPRL7DaID72vtJUvWeFRkEXpqz80kk5joqPUpBx2FV MCiKdUyQ4aJcmZIW8qAyYOwwn2IGLypR+SM1n7tqwRlEuMuVTtEmSG8F+/NvzAWmFCICecDGexc 2zMicXtihMLjlOyx0t7MbArWGazU= X-Received: by 2002:a25:1341:0:b0:750:1a06:a065 with SMTP id 62-20020a251341000000b007501a06a065mr3477884ybt.27.1671572183797; Tue, 20 Dec 2022 13:36:23 -0800 (PST) X-Google-Smtp-Source: AMrXdXsPohFHhtEKKvf6gK73YNmbJSdo8yOT0tgD3pKborKAYbk0JfGgQt+ZczDVgb1x59kgQv6r9Q== X-Received: by 2002:a25:1341:0:b0:750:1a06:a065 with SMTP id 62-20020a251341000000b007501a06a065mr3477861ybt.27.1671572183536; Tue, 20 Dec 2022 13:36:23 -0800 (PST) Received: from fedora (modemcable149.19-202-24.mc.videotron.ca. [24.202.19.149]) by smtp.gmail.com with ESMTPSA id u25-20020a37ab19000000b006feea093006sm9457393qke.124.2022.12.20.13.36.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Dec 2022 13:36:22 -0800 (PST) Date: Tue, 20 Dec 2022 16:36:20 -0500 From: Adrien Thierry To: Krzysztof Kozlowski Cc: "Rafael J. Wysocki" , Len Brown , Pavel Machek , Greg Kroah-Hartman , Kevin Hilman , Ulf Hansson , Daniel Lezcano , Lorenzo Pieralisi , Sudeep Holla , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Brian Masney , linux-rt-users@vger.kernel.org Subject: Re: [PATCH v2 0/5] PM: Fixes for Realtime systems Message-ID: References: <20221219151503.385816-1-krzysztof.kozlowski@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20221219151503.385816-1-krzysztof.kozlowski@linaro.org> Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org Hi Krzysztof, Thanks for looking into this! I tested your patchset on the QDrive3 on a CentOS Stream 9 RT kernel (I couldn't test it on mainline because the latest RT patchset only supports 6.1 which is missing some bits needed to boot QDrive3). It fixes the PSCI cpuidle issue I was encountering in [1]. However, I may have found another code path that triggers a similar issue: BUG: sleeping function called from invalid context at kernel/locking/spinlock_rt.c:46 in_atomic(): 1, irqs_disabled(): 128, non_block: 0, pid: 113, name: kworker/4:2 preempt_count: 1, expected: 0 RCU nest depth: 0, expected: 0 4 locks held by kworker/4:2/113: #0: ffff09b0c2376928 ((wq_completion)pm){+.+.}-{0:0}, at: process_one_work+0x1f4/0x7c0 #1: ffff800008bf3dd0 ((work_completion)(&genpd->power_off_work)){+.+.}-{0:0}, at: process_one_work+0x1f4/0x7c0 #2: ffff09b0c2e44860 (&genpd->rslock){....}-{2:2}, at: genpd_lock_rawspin+0x20/0x30 #3: ffff09b0c6696a20 (&dev->power.lock){+.+.}-{2:2}, at: dev_pm_qos_flags+0x2c/0x60 irq event stamp: 170 hardirqs last enabled at (169): [] _raw_spin_unlock_irq+0x48/0xc4 hardirqs last disabled at (170): [] _raw_spin_lock_irqsave+0xb0/0xfc softirqs last enabled at (0): [] copy_process+0x68c/0x1500 softirqs last disabled at (0): [<0000000000000000>] 0x0 Preemption disabled at: [] genpd_lock_rawspin+0x20/0x30 CPU: 4 PID: 113 Comm: kworker/4:2 Tainted: G X --------- --- 5.14.0-rt14+ #2 Hardware name: Qualcomm SA8540 ADP (DT) Workqueue: pm genpd_power_off_work_fn Call trace: dump_backtrace+0xb4/0x12c show_stack+0x1c/0x70 dump_stack_lvl+0x98/0xd0 dump_stack+0x14/0x2c __might_resched+0x180/0x220 rt_spin_lock+0x74/0x11c dev_pm_qos_flags+0x2c/0x60 genpd_power_off.part.0.isra.0+0xac/0x2d0 genpd_power_off_work_fn+0x68/0x8c process_one_work+0x2b8/0x7c0 worker_thread+0x15c/0x44c kthread+0xf8/0x104 ret_from_fork+0x10/0x20 This happens consistently during boot. But on the mainline kernel, this code path has changed: genpd_power_off no longer calls dev_pm_qos_flags. So it might not happen on mainline. I hope to be able to test your patchset again soon on mainline with the next version of the RT patchset (which should be able to boot the QDrive3). Best, Adrien [1] https://lore.kernel.org/all/20220615203605.1068453-1-athierry@redhat.com/