Re: [PATCHv3] iommu/arm-smmu: Optimize ->tlb_flush_walk() for qcom implementation

From: Sai Prakash Ranjan <saiprakash.ranjan@codeaurora.org>
To: Will Deacon <will@kernel.org>
Cc: Robin Murphy <robin.murphy@arm.com>,
	Joerg Roedel <joro@8bytes.org>,
	iommu@lists.linux-foundation.org,
	linux-arm-kernel@lists.infradead.org,
	linux-kernel@vger.kernel.org, linux-arm-msm@vger.kernel.org,
	Doug Anderson <dianders@chromium.org>,
	Krishna Reddy <vdumpa@nvidia.com>,
	Thierry Reding <treding@nvidia.com>,
	Tomasz Figa <tfiga@chromium.org>,
	Bjorn Andersson <bjorn.andersson@linaro.org>
Subject: Re: [PATCHv3] iommu/arm-smmu: Optimize ->tlb_flush_walk() for qcom implementation
Date: Wed, 11 Aug 2021 11:39:54 +0530	[thread overview]
Message-ID: <6f013c7eb690d40091f7c503ef640711@codeaurora.org> (raw)

On 2021-08-10 23:38, Will Deacon wrote:
> On Tue, Aug 03, 2021 at 11:09:17AM +0530, Sai Prakash Ranjan wrote:
>> On 2021-08-02 21:13, Will Deacon wrote:
>> > On Wed, Jun 23, 2021 at 07:12:01PM +0530, Sai Prakash Ranjan wrote:
>> > > diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.c
>> > > b/drivers/iommu/arm/arm-smmu/arm-smmu.c
>> > > index d3c6f54110a5..f3845e822565 100644
>> > > --- a/drivers/iommu/arm/arm-smmu/arm-smmu.c
>> > > +++ b/drivers/iommu/arm/arm-smmu/arm-smmu.c
>> > > @@ -341,6 +341,12 @@ static void arm_smmu_tlb_add_page_s1(struct
>> > > iommu_iotlb_gather *gather,
>> > >  				  ARM_SMMU_CB_S1_TLBIVAL);
>> > >  }
>> > >
>> > > +static void arm_smmu_tlb_inv_walk_impl_s1(unsigned long iova,
>> > > size_t size,
>> > > +				     size_t granule, void *cookie)
>> > > +{
>> > > +	arm_smmu_tlb_inv_context_s1(cookie);
>> > > +}
>> > > +
>> > >  static void arm_smmu_tlb_inv_walk_s2(unsigned long iova, size_t size,
>> > >  				     size_t granule, void *cookie)
>> > >  {
>> > > @@ -388,6 +394,12 @@ static const struct iommu_flush_ops
>> > > arm_smmu_s1_tlb_ops = {
>> > >  	.tlb_add_page	= arm_smmu_tlb_add_page_s1,
>> > >  };
>> > >
>> > > +const struct iommu_flush_ops arm_smmu_s1_tlb_impl_ops = {
>> > > +	.tlb_flush_all	= arm_smmu_tlb_inv_context_s1,
>> > > +	.tlb_flush_walk	= arm_smmu_tlb_inv_walk_impl_s1,
>> > > +	.tlb_add_page	= arm_smmu_tlb_add_page_s1,
>> > > +};
>> >
>> > Hmm, dunno about this. Wouldn't it be a lot cleaner if the
>> > tlb_flush_walk
>> > callbacks just did the right thing based on the smmu_domain (maybe in
>> > the
>> > arm_smmu_cfg?) rather than having an entirely new set of ops just
>> > because
>> > they're const and you can't overide the bit you want?
>> >
>> > I don't think there's really an awful lot qcom-specific about the
>> > principle
>> > here -- there's a trade-off between over-invalidation and invalidation
>> > latency. That happens on the CPU as well.
>> >
>> 
>> Sorry didn't understand, based on smmu_domain what? How do we make
>> this implementation specific? Do you mean something like a quirk?
>> The reason we didn't make this common was because nvidia folks weren't
>> so happy with that, you can find the discussion in this thread [1].
>> 
>> [1] 
>> https://lore.kernel.org/lkml/20210609145315.25750-1-saiprakash.ranjan@codeaurora.org/
> 
> The ->tlb_flush_walk() callbacks take a 'void *cookie' which, for this
> driver, is a 'struct arm_smmu_domain *'. From that, you can get to the
> 'struct arm_smmu_cfg' which could have something as coarse as:
> 
> 	bool	flush_walk_prefer_tlbiasid;
> 
> which you can set when you initialise the domain (maybe in the
> ->init_context callback?). It shouldn't affect anybody else.
> 

Ah ok, you meant a new flag in arm_smmu_cfg, right getting it from 
cookie
is no big deal but nonetheless thanks for detailing it. I have made the
changes and sent a v4 after testing.

Thanks,
Sai

-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a 
member
of Code Aurora Forum, hosted by The Linux Foundation