From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47694C433E0 for ; Thu, 25 Jun 2020 12:14:41 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 074022065D for ; Thu, 25 Jun 2020 12:14:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="sX1P7fot"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="gZ9lm5GJ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 074022065D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=samsung.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:References:In-Reply-To:MIME-Version:Date:Message-ID: From:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=esKQUvcp067rAMmYCavdCMGdmCnx2vJSnCWeUnGhPpI=; b=sX1P7fotgJyw7ixXHd5p9I7GI AtVz/AItL/MmjwNxyPmy1qzTeejaTVtvDmny1o/qVIP4UkaJB9nk/cLTXeURQpaCRRq8DyFWmT5Q0 VsFNwSWnQ1J14GSeT8TC6fJ1g+PQQA+0IdWq9oUzBQe1JhJXH2gIyKv05c7B/95gzhRm3L99LWnu3 FLdd1L/+3Gc2cwMyp1GM6I0t7cQhZGbFBh0vBYhXTdhfMqt9EkzRgQHGjEoqy8b/h8+K02pWV+QHb aJqQrQ85Eh3LJp+l5tNUmUDeny5vH8sUTaYPPQLtKFXgh1e9TIyTJeCzsZsigkwd7FPmso8rTrHmY K7NhL7Feg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1joQk8-0008FL-6O; Thu, 25 Jun 2020 12:12:36 +0000 Received: from mailout1.w1.samsung.com ([210.118.77.11]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1joQk5-0008Eh-Py for linux-arm-kernel@lists.infradead.org; Thu, 25 Jun 2020 12:12:35 +0000 Received: from eucas1p2.samsung.com (unknown [182.198.249.207]) by mailout1.w1.samsung.com (KnoxPortal) with ESMTP id 20200625121232euoutp015d014ba85c3e1e2d0322585cc95f0ee5~byFMFHC6n2315523155euoutp01f for ; Thu, 25 Jun 2020 12:12:32 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.w1.samsung.com 20200625121232euoutp015d014ba85c3e1e2d0322585cc95f0ee5~byFMFHC6n2315523155euoutp01f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1593087152; bh=r2YT8qcUQvVggMMJZuYcGzmqkKrEtSaC/jUBzXklzHg=; h=Subject:To:Cc:From:Date:In-Reply-To:References:From; b=gZ9lm5GJX8zSYn6hxUqOx/nDBRXXrhKft0P5LrBDJOxgoyNLXqQ5TmGaGtIVK0ltJ FzoVajLwqkDd5k6XCsuCuH0Pd6ib1f/5XxGWZ1EkDlesdpnhPiGFNoB9Ca2zWH02JU 8zc/unirqQEa/DMOYhbNdFw9SjhGLhnZXBd5RDVA= Received: from eusmges2new.samsung.com (unknown [203.254.199.244]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20200625121231eucas1p25a695661397003624facaafce7eaba5c~byFL2OKhJ1012410124eucas1p2y; Thu, 25 Jun 2020 12:12:31 +0000 (GMT) Received: from eucas1p1.samsung.com ( [182.198.249.206]) by eusmges2new.samsung.com (EUCPMTA) with SMTP id 75.98.05997.FA494FE5; Thu, 25 Jun 2020 13:12:31 +0100 (BST) Received: from eusmtrp2.samsung.com (unknown [182.198.249.139]) by eucas1p1.samsung.com (KnoxPortal) with ESMTPA id 20200625121231eucas1p15f9882e3d46aabd71835dbdba4c3e651~byFLbvHvC2321623216eucas1p1-; Thu, 25 Jun 2020 12:12:31 +0000 (GMT) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eusmtrp2.samsung.com (KnoxPortal) with ESMTP id 20200625121231eusmtrp236613c8ea77d6351fe1c2131faa508da~byFLa6jhA2442224422eusmtrp2U; Thu, 25 Jun 2020 12:12:31 +0000 (GMT) X-AuditID: cbfec7f4-677ff7000000176d-e9-5ef494afc5bc Received: from eusmtip2.samsung.com ( [203.254.199.222]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id 87.C4.06314.FA494FE5; Thu, 25 Jun 2020 13:12:31 +0100 (BST) Received: from [106.120.51.18] (unknown [106.120.51.18]) by eusmtip2.samsung.com (KnoxPortal) with ESMTPA id 20200625121230eusmtip2466d09df0997094c011652c8f475c6ad~byFKzbT-P0139301393eusmtip2-; Thu, 25 Jun 2020 12:12:30 +0000 (GMT) Subject: Re: brocken devfreq simple_ondemand for Odroid XU3/4? To: Lukasz Luba , Sylwester Nawrocki From: Kamil Konieczny Message-ID: Date: Thu, 25 Jun 2020 14:12:30 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.8.0 MIME-Version: 1.0 In-Reply-To: <6f8b1119-62b1-942d-cfde-6f1e9a28c40c@arm.com> Content-Language: en-US X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrDKsWRmVeSWpSXmKPExsWy7djPc7rrp3yJMzi8z8hi44z1rBbXvzxn teh//JrZ4vz5DewWZ5vesFtsenyN1eLyrjlsFp97jzBazDi/j8liYVMLu8XtxhVsFofftLNa fDvxiNGB12PNvDWMHjtn3WX32LSqk81j85J6j74tqxg9Pm+SC2CL4rJJSc3JLEst0rdL4Mr4 d2w6S8EDlYrOTVuYGhifyHYxcnJICJhIrDu/m62LkYtDSGAFo8S/1u+sEM4XRolPp14wQzif GSWW7O9igWmZvOYaVGI5o0TT5UWsIAkhgbeMEtP7fUFsYQE7ifnXdoHFRQRCJNbNnQfWwCzw k1ni/aYTzCAJNgF9iYNnT4JN5QVq+DvjJiOIzSKgKvFnVgcTiC0qECFxvHsyO0SNoMTJmU+A 6jk4OAWsJd6tFQUJMwuIS9x6Mp8JwpaX2P52DjPEoS/ZJc4fVYWwXSTunrrLBGELS7w6voUd wpaR+L8TpJcLyG5mlDjdMJUdwulhlNj7ZQsbRJW1xOePB9hAFjMLaEqs36UPEXaUeHvsDTNI WEKAT+LGW0GIG/gkJm2bDhXmlehoE4KoVpV4fqoH6gRpia7/61gnMCrNQvLYLCTfzELyzSyE vQsYWVYxiqeWFuempxYb5aWW6xUn5haX5qXrJefnbmIEprHT/45/2cG460/SIUYBDkYlHt4D E7/ECbEmlhVX5h5ilOBgVhLhdTp7Ok6INyWxsiq1KD++qDQntfgQozQHi5I4r/Gil7FCAumJ JanZqakFqUUwWSYOTqkGxqYghUV2vaHBRe9X1D9XmbKYdbGpVlfbtcfTFhmk/THmPLx+/dQj tfz/opsSo7dtm+nNy/fM7qLZS3fb7r9OEqeNyje6TTjdcLVIwW9tkt6tVyu558XJypXOUmSJ TRa8EmL4c2f5DKNsVd86dpezfnf+Hl6TyV7+oe3uJlWTEwsrDnEqqjP0K7EUZyQaajEXFScC AN29E6pfAwAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrPIsWRmVeSWpSXmKPExsVy+t/xe7rrp3yJM2jrELXYOGM9q8X1L89Z Lfofv2a2OH9+A7vF2aY37BabHl9jtbi8aw6bxefeI4wWM87vY7JY2NTCbnG7cQWbxeE37awW 3048YnTg9Vgzbw2jx85Zd9k9Nq3qZPPYvKTeo2/LKkaPz5vkAtii9GyK8ktLUhUy8otLbJWi DS2M9AwtLfSMTCz1DI3NY62MTJX07WxSUnMyy1KL9O0S9DL+HZvOUvBApaJz0xamBsYnsl2M nBwSAiYSk9dcY+5i5OIQEljKKDHx4CcmiIS0ROPp1VC2sMSfa11sILaQwGtGie830kFsYQE7 ifnXdrGC2CICIRKXu88wggxiFvjNLNH77CPU1O/MEl+u7wXrZhPQlzh49iQLiM0L1P13xk1G EJtFQFXiz6wOsG2iAhESLff/sEPUCEqcnPkEqJ6Dg1PAWuLdWlGQMLOAusSfeZeYIWxxiVtP 5jNB2PIS29/OYZ7AKDQLSfcsJC2zkLTMQtKygJFlFaNIamlxbnpusaFecWJucWleul5yfu4m RmDkbjv2c/MOxksbgw8xCnAwKvHwHpj4JU6INbGsuDL3EKMEB7OSCK/T2dNxQrwpiZVVqUX5 8UWlOanFhxhNgX6byCwlmpwPTCp5JfGGpobmFpaG5sbmxmYWSuK8HQIHY4QE0hNLUrNTUwtS i2D6mDg4pRoYPU9dknrJ+bF42r6Ev7s4JzCaq1/4/qZ+nlFI31+/3f7a/rPZ9OJXxIYpvDb+ Vn1i32rby/5zZLs4OHO3y6mJnmPYclR+goyU8vTp7JzRm37ve/qUvXVzwpetqWrdThUhT0y7 56w+2b5BLLtslmrii38xM7XK+y23rd19LmXl9TUJp19WaT3lVmIpzkg01GIuKk4EAKzmvUDy AgAA X-CMS-MailID: 20200625121231eucas1p15f9882e3d46aabd71835dbdba4c3e651 X-Msg-Generator: CA X-RootMTR: 20200624103308eucas1p188a5fe3cee1916d9430c9971c2dab3a3 X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20200624103308eucas1p188a5fe3cee1916d9430c9971c2dab3a3 References: <20200623164733.qbhua7b6cg2umafj@macmini.local> <20200623191129.GA4171@kozik-lap> <85f5a8c0-7d48-f2cd-3385-c56d662f2c88@arm.com> <4a72fcab-e8da-8323-1fbe-98a6a4b3e0f1@arm.com> <4c3b01af-2337-1eba-4675-6488105144c8@samsung.com> <6f8b1119-62b1-942d-cfde-6f1e9a28c40c@arm.com> X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Willy Wolff , "linux-samsung-soc@vger.kernel.org" , linux-pm@vger.kernel.org, Bartlomiej Zolnierkiewicz , "linux-kernel@vger.kernel.org" , Krzysztof Kozlowski , Chanwoo Choi , Kyungmin Park , Kukjin Kim , MyungJoo Ham , linux-arm-kernel@lists.infradead.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 25.06.2020 14:02, Lukasz Luba wrote: > > > On 6/25/20 12:30 PM, Kamil Konieczny wrote: >> Hi Lukasz, >> >> On 25.06.2020 12:02, Lukasz Luba wrote: >>> Hi Sylwester, >>> >>> On 6/24/20 4:11 PM, Sylwester Nawrocki wrote: >>>> Hi All, >>>> >>>> On 24.06.2020 12:32, Lukasz Luba wrote: >>>>> I had issues with devfreq governor which wasn't called by devfreq >>>>> workqueue. The old DELAYED vs DEFERRED work discussions and my patches >>>>> for it [1]. If the CPU which scheduled the next work went idle, the >>>>> devfreq workqueue will not be kicked and devfreq governor won't check >>>>> DMC status and will not decide to decrease the frequency based on low >>>>> busy_time. >>>>> The same applies for going up with the frequency. They both are >>>>> done by the governor but the workqueue must be scheduled periodically. >>>> >>>> As I have been working on resolving the video mixer IOMMU fault issue >>>> described here: https://patchwork.kernel.org/patch/10861757 >>>> I did some investigation of the devfreq operation, mostly on Odroid U3. >>>> >>>> My conclusions are similar to what Lukasz says above. I would like to add >>>> that broken scheduling of the performance counters read and the devfreq >>>> updates seems to have one more serious implication. In each call, which >>>> normally should happen periodically with fixed interval we stop the counters, >>>> read counter values and start the counters again. But if period between >>>> calls becomes long enough to let any of the counters overflow, we will >>>> get wrong performance measurement results. My observations are that >>>> the workqueue job can be suspended for several seconds and conditions for >>>> the counter overflow occur sooner or later, depending among others >>>> on the CPUs load. >>>> Wrong bus load measurement can lead to setting too low interconnect bus >>>> clock frequency and then bad things happen in peripheral devices. >>>> >>>> I agree the workqueue issue needs to be fixed. I have some WIP code to use >>>> the performance counters overflow interrupts instead of SW polling and with >>>> that the interconnect bus clock control seems to work much better. >>>> >>> >>> Thank you for sharing your use case and investigation results. I think >>> we are reaching a decent number of developers to maybe address this >>> issue: 'workqueue issue needs to be fixed'. >>> I have been facing this devfreq workqueue issue ~5 times in different >>> platforms. >>> >>> Regarding the 'performance counters overflow interrupts' there is one >>> thing worth to keep in mind: variable utilization and frequency. >>> For example, in order to make a conclusion in algorithm deciding that >>> the device should increase or decrease the frequency, we fix the period >>> of observation, i.e. to 500ms. That can cause the long delay if the >>> utilization of the device suddenly drops. For example we set an >>> overflow threshold to value i.e. 1000 and we know that at 1000MHz >>> and full utilization (100%) the counter will reach that threshold >>> after 500ms (which we want, because we don't want too many interrupts >>> per sec). What if suddenly utilization drops to 2% (i.e. from 5GB/s >>> to 250MB/s (what if it drops to 25MB/s?!)), the counter will reach the >>> threshold after 50*500ms = 25s. It is impossible just for the counters >>> to predict next utilization and adjust the threshold. [...] >> >> irq triggers for underflow and overflow, so driver can adjust freq >> > > Probably possible on some platforms, depends on how many PMU registers > are available, what information can be can assign to them and type of > interrupt. A lot of hassle and still - platform and device specific. > Also, drivers should not adjust the freq, governors (different types > of them with different settings that they can handle) should do it. > > What the framework can do is to take this responsibility and provide > generic way to monitor the devices (or stop if they are suspended). > That should work nicely with the governors, which try to predict the > next best frequency. From my experience the more fluctuating intervals > the governors are called, the more odd decisions they make. > That's why I think having a predictable interval i.e. 100ms is something > desirable. Tuning the governors is easier in this case, statistics > are easier to trace and interpret, solution is not to platform specific, > etc. > > Kamil do you have plans to refresh and push your next version of the > workqueue solution? I do not, as Bartek takes over my work, +CC Bartek -- Best regards, Kamil Konieczny Samsung R&D Institute Poland _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel