From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E01BCC433E0 for ; Thu, 25 Jun 2020 12:04:36 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AA95B2067D for ; Thu, 25 Jun 2020 12:04:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="sKc3XHzg" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AA95B2067D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Type: Content-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=EckY4P6y/TSMA9u2h0f5cCZs8mxSllN13fYajtvfMrc=; b=sKc3XHzgoYmJeZFxEokbrZ0mu mZmNZg43+J0zqJzA9O/nieOqZQ6yWvy9ojw7qKkBh26pPLGWtY/y2XBKK+LPPDEIKMIqdct8F+Sjd 4zd+G/ar85qlktZTW7+kMtVyiNK1hiXqKhPx7dinu12igkaCg3cgF716r0AYvWDXjuz8q0bluwEQJ 0l3KuJlETOgNGHObqp0OtjIXErH97cpaGhI+20TFRw/sNplnCH1l5hbym8zRIc9bAjRZmIb/RdMne o03oztkzIlAcVZMMH4biWl2KuYAlFz6FBqqr31rL/kCFQsr90U6m8EiY3kqyUddEf17ma3lp6URxx 5hd5XG1Bw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1joQag-00077q-F5; Thu, 25 Jun 2020 12:02:50 +0000 Received: from foss.arm.com ([217.140.110.172]) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1joQad-000768-CP for linux-arm-kernel@lists.infradead.org; Thu, 25 Jun 2020 12:02:48 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 45E371FB; Thu, 25 Jun 2020 05:02:45 -0700 (PDT) Received: from [10.37.12.83] (unknown [10.37.12.83]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8CC8E3F73C; Thu, 25 Jun 2020 05:02:42 -0700 (PDT) Subject: Re: brocken devfreq simple_ondemand for Odroid XU3/4? To: Kamil Konieczny , Sylwester Nawrocki References: <20200623164733.qbhua7b6cg2umafj@macmini.local> <20200623191129.GA4171@kozik-lap> <85f5a8c0-7d48-f2cd-3385-c56d662f2c88@arm.com> <4a72fcab-e8da-8323-1fbe-98a6a4b3e0f1@arm.com> <4c3b01af-2337-1eba-4675-6488105144c8@samsung.com> From: Lukasz Luba Message-ID: <6f8b1119-62b1-942d-cfde-6f1e9a28c40c@arm.com> Date: Thu, 25 Jun 2020 13:02:40 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.9.0 MIME-Version: 1.0 In-Reply-To: <4c3b01af-2337-1eba-4675-6488105144c8@samsung.com> Content-Language: en-US X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Willy Wolff , "linux-samsung-soc@vger.kernel.org" , linux-pm@vger.kernel.org, "linux-kernel@vger.kernel.org" , Krzysztof Kozlowski , Chanwoo Choi , Kyungmin Park , Kukjin Kim , MyungJoo Ham , linux-arm-kernel@lists.infradead.org Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 6/25/20 12:30 PM, Kamil Konieczny wrote: > Hi Lukasz, > > On 25.06.2020 12:02, Lukasz Luba wrote: >> Hi Sylwester, >> >> On 6/24/20 4:11 PM, Sylwester Nawrocki wrote: >>> Hi All, >>> >>> On 24.06.2020 12:32, Lukasz Luba wrote: >>>> I had issues with devfreq governor which wasn't called by devfreq >>>> workqueue. The old DELAYED vs DEFERRED work discussions and my patches >>>> for it [1]. If the CPU which scheduled the next work went idle, the >>>> devfreq workqueue will not be kicked and devfreq governor won't check >>>> DMC status and will not decide to decrease the frequency based on low >>>> busy_time. >>>> The same applies for going up with the frequency. They both are >>>> done by the governor but the workqueue must be scheduled periodically. >>> >>> As I have been working on resolving the video mixer IOMMU fault issue >>> described here: https://patchwork.kernel.org/patch/10861757 >>> I did some investigation of the devfreq operation, mostly on Odroid U3. >>> >>> My conclusions are similar to what Lukasz says above. I would like to add >>> that broken scheduling of the performance counters read and the devfreq >>> updates seems to have one more serious implication. In each call, which >>> normally should happen periodically with fixed interval we stop the counters, >>> read counter values and start the counters again. But if period between >>> calls becomes long enough to let any of the counters overflow, we will >>> get wrong performance measurement results. My observations are that >>> the workqueue job can be suspended for several seconds and conditions for >>> the counter overflow occur sooner or later, depending among others >>> on the CPUs load. >>> Wrong bus load measurement can lead to setting too low interconnect bus >>> clock frequency and then bad things happen in peripheral devices. >>> >>> I agree the workqueue issue needs to be fixed. I have some WIP code to use >>> the performance counters overflow interrupts instead of SW polling and with >>> that the interconnect bus clock control seems to work much better. >>> >> >> Thank you for sharing your use case and investigation results. I think >> we are reaching a decent number of developers to maybe address this >> issue: 'workqueue issue needs to be fixed'. >> I have been facing this devfreq workqueue issue ~5 times in different >> platforms. >> >> Regarding the 'performance counters overflow interrupts' there is one >> thing worth to keep in mind: variable utilization and frequency. >> For example, in order to make a conclusion in algorithm deciding that >> the device should increase or decrease the frequency, we fix the period >> of observation, i.e. to 500ms. That can cause the long delay if the >> utilization of the device suddenly drops. For example we set an >> overflow threshold to value i.e. 1000 and we know that at 1000MHz >> and full utilization (100%) the counter will reach that threshold >> after 500ms (which we want, because we don't want too many interrupts >> per sec). What if suddenly utilization drops to 2% (i.e. from 5GB/s >> to 250MB/s (what if it drops to 25MB/s?!)), the counter will reach the >> threshold after 50*500ms = 25s. It is impossible just for the counters >> to predict next utilization and adjust the threshold. [...] > > irq triggers for underflow and overflow, so driver can adjust freq > Probably possible on some platforms, depends on how many PMU registers are available, what information can be can assign to them and type of interrupt. A lot of hassle and still - platform and device specific. Also, drivers should not adjust the freq, governors (different types of them with different settings that they can handle) should do it. What the framework can do is to take this responsibility and provide generic way to monitor the devices (or stop if they are suspended). That should work nicely with the governors, which try to predict the next best frequency. From my experience the more fluctuating intervals the governors are called, the more odd decisions they make. That's why I think having a predictable interval i.e. 100ms is something desirable. Tuning the governors is easier in this case, statistics are easier to trace and interpret, solution is not to platform specific, etc. Kamil do you have plans to refresh and push your next version of the workqueue solution? Regards, Lukasz _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel