From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AE969ECAAD3 for ; Mon, 5 Sep 2022 17:22:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237123AbiIERWH (ORCPT ); Mon, 5 Sep 2022 13:22:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53912 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230479AbiIERWD (ORCPT ); Mon, 5 Sep 2022 13:22:03 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E099D52FC4 for ; Mon, 5 Sep 2022 10:22:02 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 7C4F161369 for ; Mon, 5 Sep 2022 17:22:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D76DEC433D6; Mon, 5 Sep 2022 17:22:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1662398521; bh=pW6cPiKz1ABqdGUp/M/nDAZUMJVaiT/7PREpOhe2/2Q=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=C88GljmRRQAQXH2uSpuZTjqlfOqA5PMvLsWwMvzLfMiUP9kM67sXvkQ2+LmDURY8X N3xekTP8OzTr4X+qy+NjK8TwIQy+ECtDFCRAjUOdivEv9cUzbKMV+QNUgfcmJ8PW2X cHM3+s6kYourVSadJBWcE6GC0RsyJQ+j/yoAMbV+uP3861b42/qKF92qKcICwZ6RZE Ud4ZmaUQmvRZMgp6VupyWhGcCZYw0POm213FWAUk3nAI5t+NqzwcBAXl3HREyQvSWJ Rxdy1u798vcgYJBvFT78fTfYWGXBWWZPKDx0vpUzcsUH/UgOmY8QkLp+qo30vqVsbP 5u096LZhLLkhQ== Received: from ip-185-104-136-29.ptr.icomera.net ([185.104.136.29] helo=wait-a-minute.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1oVFnL-0088Cr-0D; Mon, 05 Sep 2022 18:21:59 +0100 Date: Mon, 05 Sep 2022 18:21:57 +0100 Message-ID: <87o7vtrb2i.wl-maz@kernel.org> From: Marc Zyngier To: Liao Chang Cc: , , Subject: Re: [PATCH V2] irqchip/gic-v3-its: Reclaim the dangling bits in LPI maps In-Reply-To: <20220831023332.191368-1-liaochang1@huawei.com> References: <20220831023332.191368-1-liaochang1@huawei.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.104.136.29 X-SA-Exim-Rcpt-To: liaochang1@huawei.com, tglx@linutronix.de, linux-kernel@vger.kernel.org, john.garry@huawei.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 31 Aug 2022 03:33:32 +0100, Liao Chang wrote: > > Following interrupt allocation process leads to some interrupts are > mapped in the low-level domain(Arm ITS), but they have never mapped > at the higher level. > > irq_domain_alloc_irqs_hierarchy(.., nr_irqs, ...) > its_irq_domain_alloc(..., nr_irqs, ...) > its_alloc_device_irq(..., nr_irqs, ...) > bitmap_find_free_region(..., get_count_order(nr_irqs)) > > Since ITS domain finds a region of zero bits, the length of which must > aligned to the power of two. If nr_irqs is 30, the length of zero bits > is actually 32, but the first 30 bits are really mapped. > > On teardown, the low-level domain only free these interrupts that > actually mapped, and leave last interrupts dangling in the ITS domain. > Thus the ITS device resources are never freed. On device driver reload, > dangling interrupts prevent ITS domain from allocating enough resource. > > irq_domain_free_irqs_hierarchy(..., nr_irqs, ...) > its_irq_domain_free(..., irq_base + i, 1) > bitmap_release_region(..., irq_base + i, get_count_order(1)) > > John reported this problem to LKML and Marc provided a solution and fix > it in the generic code, see the discussion from Link tag. Marc's patch > fix John's problem, but does not take care of some corner case, look one > example below. > > Step1: 32 interrupts allocated in LPI domain, but return the first 30 to > higher driver. > > 111111111111111111111111111111 11 > |<------------0~29------------>|30,31| > > Step2: interrupt #16~28 are released one by one, then #0~15 and #29~31 > still be there. > > 1111111111111111 0000000000000 1 11 > |<-----0~15----->|<---16~28--->|29|30,31| > > Step#: on driver teardown, generic code will invoke ITS domain code > twice. The first time, #0~15 will be released, the second one, only #29 > will be released(1 align to power of two). > > 0000000000000000 0000000000000 0 11 > |<-----0~15----->|<---16~28--->|29|30,31| Which driver is doing this? This really looks like a driver bug to only free a portion of its MSI allocation, and that's definitely not something that is commonly done. Even worse, this can result in some LPIs being released behind the driver's back, exactly due to this power-of-two alignment. It seems to me that you are trying to solve a problem that only exists for a buggy driver. Please point me to the upstream code that has such behaviour and explain why this can't be fixed in that driver itself. Thanks, M. -- Without deviation from the norm, progress is not possible.