From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.0 required=3.0 tests=DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9004DC433E1 for ; Wed, 24 Jun 2020 08:03:30 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4FC8320FC3 for ; Wed, 24 Jun 2020 08:03:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="jNBnroU+"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="tZx5N0RS" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4FC8320FC3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=c2cYrFL8Do8+xyqzu6pihv6PD7zz2vMq4qPF4ohvQM0=; b=jNBnroU+ev7tSaoVdnHOFuzUo QKmPHwAHh7asInjdttL/PkjPVz7Lhhe7OYXJ+J2AD58kK87iqudXdrZDJpBZ+CunKQI1fQ5WqNoSX 9PzxyU+55klrECFBZTrPAM4cmJsaI//4h918USdHLz6lfyOBb09HaSSkw6GY+XHq68YYMDzAg3w+b 2WRrHNFptJ9t+3ZTOVu+rY8zqWCRJAR89JX2ZEMXE+uNyzPloKMqpzLcDkjU+aQrkD1JCrgh+oZ/b rByAiGzljqU0ybrA/vSG4egEKHEQhVPMOhepaTo43wJfKF1jHvTvGZUHjlBpNjrVvONvpjgYDc5Rv numYQLJkA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jo0LT-0000XG-BH; Wed, 24 Jun 2020 08:01:23 +0000 Received: from mail-wr1-x42b.google.com ([2a00:1450:4864:20::42b]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jo0LR-0000WR-30 for linux-arm-kernel@lists.infradead.org; Wed, 24 Jun 2020 08:01:22 +0000 Received: by mail-wr1-x42b.google.com with SMTP id h5so1286404wrc.7 for ; Wed, 24 Jun 2020 01:01:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=ff095JW8oYzVPcA6w3J52BpgeRH7KWoM7ayC0lmndtU=; b=tZx5N0RSb6jkzNmNahEEiKJwPzEuT/scDoWx8lSV93588W1idC3T/cFC8g0oUwR+dJ SqUv97R3K6VZqs32bMmDkMsqtvLpbxQFvTnHOn1jmHzkoM8tFQI2dMdribE8bDMsqtxe zVSWen0pguU8L0jyZ3sb8Vfd0ApMEYEII0noWPOecy8tBdxCLtBKjY3utTraZiAQ/oRb FICEPmpsp4a5cJtfO57nA8HeyihnePvJENwzAsJDII0XjQ4IZzsQCNekTatH2Mpb0SKK p8zZm9+2ICDJE9L+c77ST3xFMu5/6wI+gmCxzoFjRJTOV5atRuIjxkTB3Y+nk0wtk8lt f7ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=ff095JW8oYzVPcA6w3J52BpgeRH7KWoM7ayC0lmndtU=; b=XfIeV6KmAYjMt5lvKg0zaMHlWl5IYZ2jOLKkE6e8qAs8TBaLqCSRvvn8B1UnuG00TQ FpG+22F3vfjIb3EYvNjZMQ+aLNE4p3sFrHVWWkIdaJsK9Ma1rhdpf7WFjWfCvGq+A/9L jfdT6sn0Eq15ClBaslWRiUeMFIVQrQxCoWEwC1wcC1cPQbpwCv2RqsX5JzguCm4q6bWs wfEDS97/QR0SVNvW1gqb+cTgASYOazaCcEsGrZsL8178qKtIgzptAYj6Sqaqn5In5Ub4 LUiZ/VE0HyoRkudJlEglQsWkis96bz8hUBwkprVT5p/lj4wkYScCuJhUIlIGSm0nKO+v ireA== X-Gm-Message-State: AOAM532rT6AcSl2zw3hys/MrjaxrMV022WuDF3bPjVcOm3bTFOo9rqIe eEMJ1XHeRGBekPMiiYMeaZw= X-Google-Smtp-Source: ABdhPJyXAEGE/s3H+cIQzr4JLHjCH60NZodTnxqdileYvj08Qui4Sanh+mSVawVIq9fLlkUa9yJP3Q== X-Received: by 2002:a05:6000:1cf:: with SMTP id t15mr14115855wrx.180.1592985679537; Wed, 24 Jun 2020 01:01:19 -0700 (PDT) Received: from macmini.local (181.4.199.77.rev.sfr.net. [77.199.4.181]) by smtp.gmail.com with ESMTPSA id a10sm26396204wrm.21.2020.06.24.01.01.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jun 2020 01:01:18 -0700 (PDT) Date: Wed, 24 Jun 2020 10:01:17 +0200 From: Willy Wolff To: Krzysztof Kozlowski Subject: Re: brocken devfreq simple_ondemand for Odroid XU3/4? Message-ID: <20200624080117.fzgowkpgyhs6tbzx@macmini.local> References: <20200623164733.qbhua7b6cg2umafj@macmini.local> <20200623191129.GA4171@kozik-lap> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200623191129.GA4171@kozik-lap> X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "linux-samsung-soc@vger.kernel.org" , linux-pm@vger.kernel.org, "linux-kernel@vger.kernel.org" , Chanwoo Choi , Kyungmin Park , MyungJoo Ham , Kukjin Kim , Lukasz Luba , linux-arm-kernel@lists.infradead.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Krzysztof, Thanks to look at it. mem_gov is /sys/class/devfreq/10c20000.memory-controller/governor Here some numbers after increasing the running time: Running using simple_ondemand: Before: From : To : 165000000 206000000 275000000 413000000 543000000 633000000 728000000 825000000 time(ms) * 165000000: 0 0 0 0 0 0 0 4 4528600 206000000: 5 0 0 0 0 0 0 0 57780 275000000: 0 5 0 0 0 0 0 0 50060 413000000: 0 0 5 0 0 0 0 0 46240 543000000: 0 0 0 5 0 0 0 0 48970 633000000: 0 0 0 0 5 0 0 0 47330 728000000: 0 0 0 0 0 0 0 0 0 825000000: 0 0 0 0 0 5 0 0 331300 Total transition : 34 After: From : To : 165000000 206000000 275000000 413000000 543000000 633000000 728000000 825000000 time(ms) * 165000000: 0 0 0 0 0 0 0 4 5098890 206000000: 5 0 0 0 0 0 0 0 57780 275000000: 0 5 0 0 0 0 0 0 50060 413000000: 0 0 5 0 0 0 0 0 46240 543000000: 0 0 0 5 0 0 0 0 48970 633000000: 0 0 0 0 5 0 0 0 47330 728000000: 0 0 0 0 0 0 0 0 0 825000000: 0 0 0 0 0 5 0 0 331300 Total transition : 34 With a running time of: LITTLE => 283.699 s (680.877 c per mem access) big => 284.47 s (975.327 c per mem access) And when I set to the performance governor: Before: From : To : 165000000 206000000 275000000 413000000 543000000 633000000 728000000 825000000 time(ms) 165000000: 0 0 0 0 0 0 0 5 5099040 206000000: 5 0 0 0 0 0 0 0 57780 275000000: 0 5 0 0 0 0 0 0 50060 413000000: 0 0 5 0 0 0 0 0 46240 543000000: 0 0 0 5 0 0 0 0 48970 633000000: 0 0 0 0 5 0 0 0 47330 728000000: 0 0 0 0 0 0 0 0 0 * 825000000: 0 0 0 0 0 5 0 0 331350 Total transition : 35 After: From : To : 165000000 206000000 275000000 413000000 543000000 633000000 728000000 825000000 time(ms) 165000000: 0 0 0 0 0 0 0 5 5099040 206000000: 5 0 0 0 0 0 0 0 57780 275000000: 0 5 0 0 0 0 0 0 50060 413000000: 0 0 5 0 0 0 0 0 46240 543000000: 0 0 0 5 0 0 0 0 48970 633000000: 0 0 0 0 5 0 0 0 47330 728000000: 0 0 0 0 0 0 0 0 0 * 825000000: 0 0 0 0 0 5 0 0 472980 Total transition : 35 With a running time of: LITTLE: 68.8428 s (165.223 c per mem access) big: 71.3268 s (244.549 c per mem access) I see some transition, but not occuring during the benchmark. I haven't dive into the code, but maybe it is the heuristic behind that is not well defined? If you know how it's working that would be helpfull before I dive in it. I run your test as well, and indeed, it seems to work for large bunch of memory, and there is some delay before making a transition (seems to be around 10s). When you kill memtester, it reduces the freq stepwisely every ~10s. Note that the timing shown above account for the critical path, and the code is looping on reading only, there is no write in the critical path. Maybe memtester is doing writes and devfreq heuristic uses only write info? Cheers, Willy On 2020-06-23-21-11-29, Krzysztof Kozlowski wrote: > On Tue, Jun 23, 2020 at 09:02:38PM +0200, Krzysztof Kozlowski wrote: > > On Tue, 23 Jun 2020 at 18:47, Willy Wolff wrote: > > > > > > Hi everybody, > > > > > > Is DVFS for memory bus really working on Odroid XU3/4 board? > > > Using a simple microbenchmark that is doing only memory accesses, memory DVFS > > > seems to not working properly: > > > > > > The microbenchmark is doing pointer chasing by following index in an array. > > > Indices in the array are set to follow a random pattern (cutting prefetcher), > > > and forcing RAM access. > > > > > > git clone https://github.com/wwilly/benchmark.git \ > > > && cd benchmark \ > > > && source env.sh \ > > > && ./bench_build.sh \ > > > && bash source/scripts/test_dvfs_mem.sh > > > > > > Python 3, cmake and sudo rights are required. > > > > > > Results: > > > DVFS CPU with performance governor > > > mem_gov = simple_ondemand at 165000000 Hz in idle, should be bumped when the > > > benchmark is running. > > > - on the LITTLE cluster it takes 4.74308 s to run (683.004 c per memory access), > > > - on the big cluster it takes 4.76556 s to run (980.343 c per moemory access). > > > > > > While forcing DVFS memory bus to use performance governor, > > > mem_gov = performance at 825000000 Hz in idle, > > > - on the LITTLE cluster it takes 1.1451 s to run (164.894 c per memory access), > > > - on the big cluster it takes 1.18448 s to run (243.664 c per memory access). > > > > > > The kernel used is the last 5.7.5 stable with default exynos_defconfig. > > > > Thanks for the report. Few thoughts: > > 1. What trans_stat are saying? Except DMC driver you can also check > > all other devfreq devices (e.g. wcore) - maybe the devfreq events > > (nocp) are not properly assigned? > > 2. Try running the measurement for ~1 minutes or longer. The counters > > might have some delay (which would require probably fixing but the > > point is to narrow the problem). > > 3. What do you understand by "mem_gov"? Which device is it? > > +Cc Lukasz who was working on this. > > I just run memtester and more-or-less ondemand works (at least ramps > up): > > Before: > /sys/class/devfreq/10c20000.memory-controller$ cat trans_stat > From : To > : 165000000 206000000 275000000 413000000 543000000 633000000 728000000 825000000 time(ms) > * 165000000: 0 0 0 0 0 0 0 0 1795950 > 206000000: 1 0 0 0 0 0 0 0 4770 > 275000000: 0 1 0 0 0 0 0 0 15540 > 413000000: 0 0 1 0 0 0 0 0 20780 > 543000000: 0 0 0 1 0 0 0 1 10760 > 633000000: 0 0 0 0 2 0 0 0 10310 > 728000000: 0 0 0 0 0 0 0 0 0 > 825000000: 0 0 0 0 0 2 0 0 25920 > Total transition : 9 > > > $ sudo memtester 1G > > During memtester: > /sys/class/devfreq/10c20000.memory-controller$ cat trans_stat > From : To > : 165000000 206000000 275000000 413000000 543000000 633000000 728000000 825000000 time(ms) > 165000000: 0 0 0 0 0 0 0 1 1801490 > 206000000: 1 0 0 0 0 0 0 0 4770 > 275000000: 0 1 0 0 0 0 0 0 15540 > 413000000: 0 0 1 0 0 0 0 0 20780 > 543000000: 0 0 0 1 0 0 0 2 11090 > 633000000: 0 0 0 0 3 0 0 0 17210 > 728000000: 0 0 0 0 0 0 0 0 0 > * 825000000: 0 0 0 0 0 3 0 0 169020 > Total transition : 13 > > However after killing memtester it stays at 633 MHz for very long time > and does not slow down. This is indeed weird... > > Best regards, > Krzysztof _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel