From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 24164C433EF for ; Mon, 27 Sep 2021 13:39:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 07A1660230 for ; Mon, 27 Sep 2021 13:39:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234557AbhI0NlZ (ORCPT ); Mon, 27 Sep 2021 09:41:25 -0400 Received: from mail.kernel.org ([198.145.29.99]:37626 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234487AbhI0NlY (ORCPT ); Mon, 27 Sep 2021 09:41:24 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id F1FD060230; Mon, 27 Sep 2021 13:39:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1632749987; bh=I0VLqLGVdYksZCLnthzXJO1Cc3oa8W2vUufNiQFgCcM=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=A2bGGNVaCvQPVjMNFJNXEapq6P9b55VHUoFGwfauwXxYZSPnPwJDaBvoQPbB6RTyB z9Si/dqh8TvNXK+WoYM45rmh+CEYlSDo4oyRbYuMUpjwxZ8MHnVqV9uVnPvLvJLTQK g+rMSylZqEF43nv+UIMge2S86T7FNn9a5CvyJyJAxLX7SeMBGt/+X8AHtg1PhSt7SF l4CxJOQnA8NE9Is47lSkONwAR4KZtwwa7z17jA7LV+AKhCxMQorrRNh5OWK1QM9vlz 5KBeK/larkhCFk2rBFqvnHcwrRH2r6H770osFxkIuziLP0ehDlJqfT2Jx5VO2rv1RP LduYzMKmadBRQ== Date: Mon, 27 Sep 2021 15:39:42 +0200 From: Mauro Carvalho Chehab To: Greg Kroah-Hartman Cc: Linux Doc Mailing List , linux-kernel@vger.kernel.org, Jonathan Corbet Subject: Re: [PATCH 0/8] (REBASED) get_abi.pl undefined: improve precision and performance Message-ID: <20210927153942.75bbb9cf@coco.lan> In-Reply-To: References: <20210927105553.105f22c5@coco.lan> X-Mailer: Claws Mail 4.0.0 (GTK+ 3.24.30; x86_64-redhat-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-doc@vger.kernel.org Em Mon, 27 Sep 2021 11:23:20 +0200 Greg Kroah-Hartman escreveu: > On Mon, Sep 27, 2021 at 10:55:53AM +0200, Mauro Carvalho Chehab wrote: > > Em Thu, 23 Sep 2021 19:13:04 +0200 > > Greg Kroah-Hartman escreveu: > > > > > On Thu, Sep 23, 2021 at 05:41:11PM +0200, Mauro Carvalho Chehab wrote: > > > > Hi Greg, > > > > > > > > As requested, this is exactly the same changes, rebased on the top of > > > > driver-core/driver-core-next. > > > > > > > > - > > > > > > > > It follows a series of improvements for get_abi.pl. it is on the top of driver-core/driver-core-next. > > > > > > > > With such changes, on my development tree, the script is taking 6 seconds to run > > > > on my desktop: > > > > > > > > $ !1076 > > > > $ time ./scripts/get_abi.pl undefined |sort >undefined_after && cat undefined_after| perl -ne 'print "$1\n" if (m#.*/(\S+) not found#)'|sort|uniq -c|sort -nr >undefined_symbols; wc -l undefined_after undefined_symbols > > > > > > > > real 0m6,292s > > > > user 0m5,640s > > > > sys 0m0,634s > > > > 6838 undefined_after > > > > 808 undefined_symbols > > > > 7646 total > > > > > > > > And 7 seconds on a Dell Precision 5820: > > > > > > > > $ time ./scripts/get_abi.pl undefined |sort >undefined && cat undefined| perl -ne 'print "$1\n" if (m#.*/(\S+) not found#)'|sort|uniq -c|sort -nr >undefined_symbols; wc -l undefined; wc -l undefined_symbols > > > > > > > > real 0m7.162s > > > > user 0m5.836s > > > > sys 0m1.329s > > > > 6548 undefined > > > > 772 undefined_symbols > > > > > > > > Both tests were done against this tree (based on today's linux-next): > > > > > > > > $ https://git.kernel.org/pub/scm/linux/kernel/git/mchehab/devel.git/log/?h=get_abi_undefined-latest > > > > > > > > It should be noticed that, as my tree has several ABI fixes, the time to run the > > > > script is likely less than if you run on your tree, as there will be less symbols to > > > > be reported, and the algorithm is optimized to reduce the number of regexes > > > > when a symbol is found. > > > > > > > > Besides optimizing and improving the seek logic, this series also change the > > > > debug logic. It how receives a bitmap, where "8" means to print the regexes > > > > that will be used by "undefined" command: > > > > > > > > $ time ./scripts/get_abi.pl undefined --debug 8 >foo > > > > real 0m17,189s > > > > user 0m13,940s > > > > sys 0m2,404s > > > > > > > > $wc -l foo > > > > 18421939 foo > > > > > > > > $ cat foo > > > > ... > > > > /sys/kernel/kexec_crash_loaded =~ /^(?^:^/sys/.*/iio\:device.*/in_voltage.*_scale_available$)$/ > > > > /sys/kernel/kexec_crash_loaded =~ /^(?^:^/sys/.*/iio\:device.*/out_voltage.*_scale_available$)$/ > > > > /sys/kernel/kexec_crash_loaded =~ /^(?^:^/sys/.*/iio\:device.*/out_altvoltage.*_scale_available$)$/ > > > > /sys/kernel/kexec_crash_loaded =~ /^(?^:^/sys/.*/iio\:device.*/in_pressure.*_scale_available$)$/ > > > > ... > > > > > > > > On other words, on my desktop, the /sys match is performing >18M regular > > > > expression searches, which takes 6,2 seconds (or 17,2 seconds, if debug is > > > > enabled and sent to an area on my nvme storage). > > > > > > Better, it's down to 10 minutes on my machine now: > > > > > > real 10m39.218s > > > user 10m37.742s > > > sys 0m0.775s > > > > A lot better, but not clear why it is still taking ~40x more than here... > > It could well be due to the other ABI changes yet to be applied > > (I'll submit it probably later today), but it could also be related to > > something else. Could this be due to disk writes? > > Disk writes to where for what? This is a very fast disk (nvme raid > array) It's also a very "big" system, with lots of sysfs files: > > $ find /sys/devices/ -type f | wc -l > 44334 Ok. Maybe that partially explains why it is taking so long, as the number of regex to compare will increase (not linearly). > compared to my laptop that only has 17k entries in /sys/devices/ > > I'll run this updated script on my laptop later today and give you some > numbers. Ok, thanks! > And any Documentation/ABI/ updates you might have I'll gladly > take as well. I'll be submitting it soon enough. Got sidetracked by a regression on my INBOX due to a fetchmail regression[1]. > thanks, > > greg k-h [1] https://gitlab.com/fetchmail/fetchmail/-/issues/39 Thanks, Mauro