Re: [PATCH] iommu/io-pgtable-arm: Optimize partial walk flush for large scatter-gather list

Sai Prakash Ranjan
2021-06-12T02:46:47Z

Hi Krishna,

On 2021-06-11 22:19, Krishna Reddy wrote:
> Hi Sai,
>> >> > No, the unmap latency is not just in some test case written, the
>> >> > issue is very real and we have workloads where camera is reporting
>> >> > frame drops because of this unmap latency in the order of 100s of
>> milliseconds.
>
>> Not exactly, this issue is not specific to camera. If you look at the
>> numbers in the
>> commit text, even for the test device its the same observation. It
>> depends on
>> the buffer size we are unmapping which affects the number of TLBIs
>> issue. I am
>> not aware of any such HW side bw issues for camera specifically on
>> QCOM
>> devices.
>
> It is clear that reducing number of TLBIs reduces the umap API
> latency. But, It is
> at the expense of throwing away valid tlb entries.
> Quantifying the impact of arbitrary invalidation of valid tlb entries
> at context level is not straight forward and
> use case dependent. The side-effects might be rare or won't be known
> until they are noticed.

Right but we won't know until we profile the specific usecases or try
them
in generic workload to see if they affect the performance. Sure, over
invalidation is a concern where multiple buffers can be mapped to same
context
and the cache is not usable at the time for lookup and such but we don't
do it
for small buffers and only for large buffers which means thousands of
TLB entry
mappings in which case TLBIASID is preferred (note: I mentioned the HW
team
recommendation to use it for anything greater than 128 TLB entries) in
my earlier
reply. And also note that we do this only for partial walk flush, we are
not
arbitrarily changing all the TLBIs to ASID based.

> Can you provide more details on How the unmap latency is causing
> camera to drop frames?
> Is unmap performed in the perf path?

I am no camera expert but from what the camera team mentioned is that
there is a thread which frees memory(large unused memory buffers)
periodically which ends up taking around 100+ms and causing some camera
test
failures with frame drops. Parallel efforts are already being made to
optimize
this usage of thread but as I mentioned previously, this is *not a
camera
specific*, lets say someone else invokes such large unmaps, it's going
to face
the same issue.

> If unmap is queued and performed on a back ground thread, would it
> resolve the frame drops?

Not sure I understand what you mean by queuing on background thread but
with
that or not, we still do the same number of TLBIs and hop through
iommu->io-pgtable->arm-smmu to perform the the unmap, so how will that
help?

Thanks,
Sai

--
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a
member