From mboxrd@z Thu Jan 1 00:00:00 1970 From: Will Deacon Subject: Re: [PATCH v2 1/3] iommu/arm-smmu-v3: put off the execution of TLBI* to reduce lock confliction Date: Wed, 18 Oct 2017 13:58:49 +0100 Message-ID: <20171018125849.GD4077@arm.com> References: <1505221238-9428-1-git-send-email-thunder.leizhen@huawei.com> <1505221238-9428-2-git-send-email-thunder.leizhen@huawei.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Content-Disposition: inline In-Reply-To: <1505221238-9428-2-git-send-email-thunder.leizhen-hv44wF8Li93QT0dZR+AlfA@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Errors-To: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org To: Zhen Lei Cc: Kefeng Wang , linux-kernel , Jinyue Li , iommu , Libin , Hanjun Guo , linux-arm-kernel List-Id: iommu@lists.linux-foundation.org Hi Thunder, On Tue, Sep 12, 2017 at 09:00:36PM +0800, Zhen Lei wrote: > Because all TLBI commands should be followed by a SYNC command, to make > sure that it has been completely finished. So we can just add the TLBI > commands into the queue, and put off the execution until meet SYNC or > other commands. To prevent the followed SYNC command waiting for a long > time because of too many commands have been delayed, restrict the max > delayed number. > > According to my test, I got the same performance data as I replaced writel > with writel_relaxed in queue_inc_prod. > > Signed-off-by: Zhen Lei > --- > drivers/iommu/arm-smmu-v3.c | 42 +++++++++++++++++++++++++++++++++++++----- > 1 file changed, 37 insertions(+), 5 deletions(-) If we want to go down the route of explicit command batching, I'd much rather do it by implementing the iotlb_range_add callback in the driver, and have a fixed-length array of batched ranges on the domain. We could potentially toggle this function pointer based on the compatible string too, if it shows only to benefit some systems. Will From mboxrd@z Thu Jan 1 00:00:00 1970 From: will.deacon@arm.com (Will Deacon) Date: Wed, 18 Oct 2017 13:58:49 +0100 Subject: [PATCH v2 1/3] iommu/arm-smmu-v3: put off the execution of TLBI* to reduce lock confliction In-Reply-To: <1505221238-9428-2-git-send-email-thunder.leizhen@huawei.com> References: <1505221238-9428-1-git-send-email-thunder.leizhen@huawei.com> <1505221238-9428-2-git-send-email-thunder.leizhen@huawei.com> Message-ID: <20171018125849.GD4077@arm.com> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org Hi Thunder, On Tue, Sep 12, 2017 at 09:00:36PM +0800, Zhen Lei wrote: > Because all TLBI commands should be followed by a SYNC command, to make > sure that it has been completely finished. So we can just add the TLBI > commands into the queue, and put off the execution until meet SYNC or > other commands. To prevent the followed SYNC command waiting for a long > time because of too many commands have been delayed, restrict the max > delayed number. > > According to my test, I got the same performance data as I replaced writel > with writel_relaxed in queue_inc_prod. > > Signed-off-by: Zhen Lei > --- > drivers/iommu/arm-smmu-v3.c | 42 +++++++++++++++++++++++++++++++++++++----- > 1 file changed, 37 insertions(+), 5 deletions(-) If we want to go down the route of explicit command batching, I'd much rather do it by implementing the iotlb_range_add callback in the driver, and have a fixed-length array of batched ranges on the domain. We could potentially toggle this function pointer based on the compatible string too, if it shows only to benefit some systems. Will From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752625AbdJRM7a (ORCPT ); Wed, 18 Oct 2017 08:59:30 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:39828 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752421AbdJRM6q (ORCPT ); Wed, 18 Oct 2017 08:58:46 -0400 Date: Wed, 18 Oct 2017 13:58:49 +0100 From: Will Deacon To: Zhen Lei Cc: Joerg Roedel , linux-arm-kernel , iommu , Robin Murphy , linux-kernel , Hanjun Guo , Libin , Jinyue Li , Kefeng Wang Subject: Re: [PATCH v2 1/3] iommu/arm-smmu-v3: put off the execution of TLBI* to reduce lock confliction Message-ID: <20171018125849.GD4077@arm.com> References: <1505221238-9428-1-git-send-email-thunder.leizhen@huawei.com> <1505221238-9428-2-git-send-email-thunder.leizhen@huawei.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1505221238-9428-2-git-send-email-thunder.leizhen@huawei.com> User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Thunder, On Tue, Sep 12, 2017 at 09:00:36PM +0800, Zhen Lei wrote: > Because all TLBI commands should be followed by a SYNC command, to make > sure that it has been completely finished. So we can just add the TLBI > commands into the queue, and put off the execution until meet SYNC or > other commands. To prevent the followed SYNC command waiting for a long > time because of too many commands have been delayed, restrict the max > delayed number. > > According to my test, I got the same performance data as I replaced writel > with writel_relaxed in queue_inc_prod. > > Signed-off-by: Zhen Lei > --- > drivers/iommu/arm-smmu-v3.c | 42 +++++++++++++++++++++++++++++++++++++----- > 1 file changed, 37 insertions(+), 5 deletions(-) If we want to go down the route of explicit command batching, I'd much rather do it by implementing the iotlb_range_add callback in the driver, and have a fixed-length array of batched ranges on the domain. We could potentially toggle this function pointer based on the compatible string too, if it shows only to benefit some systems. Will