RE: [Xen-devel] [PATCH] modify the IO_TLB_SEGSIZE to io_tlb_segsize configurable as flexible requirement about SW-IOMMU.

Wang, Xiaoming
2015-02-12T00:55:58Z

Dear Wilk:

> -----Original Message-----
> From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com]
> Sent: Thursday, February 12, 2015 4:49 AM
> To: Wang, Xiaoming
> Cc: David Vrabel; linux-mips@linux-mips.org; pebolle@tiscali.nl; Zhang,
> Dongxing; lauraa@codeaurora.org; d.kasatkin@samsung.com;
> heiko.carstens@de.ibm.com; linux-kernel@vger.kernel.org; ralf@linux-
> mips.org; chris@chris-wilson.co.uk; takahiro.akashi@linaro.org;
> linux@horizon.com; xen-devel@lists.xenproject.org;
> boris.ostrovsky@oracle.com; Liu, Chuansheng; akpm@linux-foundation.org
> Subject: Re: [Xen-devel] [PATCH] modify the IO_TLB_SEGSIZE to
> io_tlb_segsize configurable as flexible requirement about SW-IOMMU.
>
> On Wed, Feb 11, 2015 at 08:38:29AM +0000, Wang, Xiaoming wrote:
> > Dear David
> >
> > > -----Original Message-----
> > > From: David Vrabel [mailto:david.vrabel@citrix.com]
> > > Sent: Tuesday, February 10, 2015 5:46 PM
> > > To: Wang, Xiaoming; Konrad Rzeszutek Wilk
> > > Cc: linux-mips@linux-mips.org; pebolle@tiscali.nl; Zhang, Dongxing;
> > > lauraa@codeaurora.org; d.kasatkin@samsung.com;
> > > heiko.carstens@de.ibm.com; linux-kernel@vger.kernel.org; ralf@linux-
> > > mips.org; chris@chris-wilson.co.uk; takahiro.akashi@linaro.org;
> > > david.vrabel@citrix.com; linux@horizon.com; xen-
> > > devel@lists.xenproject.org; boris.ostrovsky@oracle.com; Liu,
> > > Chuansheng; akpm@linux-foundation.org
> > > Subject: Re: [Xen-devel] [PATCH] modify the IO_TLB_SEGSIZE to
> > > io_tlb_segsize configurable as flexible requirement about SW-IOMMU.
> > >
> > > On 06/02/15 00:10, Wang, Xiaoming wrote:
> > > >
> > > >
> > > >> -----Original Message-----
> > > >> From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com]
> > > >> Sent: Friday, February 6, 2015 3:33 AM
> > > >> To: Wang, Xiaoming
> > > >> Cc: ralf@linux-mips.org; boris.ostrovsky@oracle.com;
> > > >> david.vrabel@citrix.com; linux-mips@linux-mips.org; linux-
> > > >> kernel@vger.kernel.org; xen-devel@lists.xenproject.org;
> > > >> akpm@linux- foundation.org; linux@horizon.com;
> > > >> lauraa@codeaurora.org; heiko.carstens@de.ibm.com;
> > > >> d.kasatkin@samsung.com; takahiro.akashi@linaro.org;
> > > >> chris@chris-wilson.co.uk; pebolle@tiscali.nl; Liu, Chuansheng;
> > > >> Zhang, Dongxing
> > > >> Subject: Re: [PATCH] modify the IO_TLB_SEGSIZE to io_tlb_segsize
> > > >> configurable as flexible requirement about SW-IOMMU.
> > > >>
> > > >> On Fri, Feb 06, 2015 at 07:01:14AM +0800, xiaomin1 wrote:
> > > >>> The maximum of SW-IOMMU is limited to 2^11*128 = 256K.
> > > >>> While in different platform and different requirements this
> > > >>> seems
> > > improper.
> > > >>> So modify the IO_TLB_SEGSIZE to io_tlb_segsize as configurable
> > > >>> is make
> > > >> sense.
> > > >>
> > > >> More details please. What is the issue you are hitting?
> > > >>
> > > > Example:
> > > > If 1M bytes are requied. There has an error like.
> > >
> > > Instead of allowing the bouncing of such large buffers, could the
> > > gadget driver be modified to submit the buffers to the hardware in
> smaller chunks?
> > >
> > > David
> >
> > Our target is try to make IO_TLB_SEGSIZE configurable.
> > Neither 256 bytes or 1M bytes seems suitable value, I think.
> > It's better to use the tactics something like kmem_cache_create in
> > kmalloc function.
> > But SW-IOMMU seems more lighter.
> > So we choose variable rather than function.
>
> Would it be possible to understand why the gadget needs such large buffer?
> That is irrespective of the patchset you are proposing.
>
> In regards to the pathchset - I don't see anything fundamentally wrong with
> the patch. What I am afraid is that this fixes the symptoms instead of the
> underlaying problem. The problem I think is that with this large 1MB requests
> you risk of using the SWIOTLB bounce buffer which can result in poor
> performance.
>
> So eventually somebody will have to figure out why the performance is poor
> and have a hard time figuring what is wrong - as the symptoms have been
> removed.
>
> Hence looking at potentially using an scatter gather mechanism and chop up
> the requests in smaller sizes might be an better option. But I don't know?
> Perhaps you are more familiar with the gadget and could tell me why it needs
> an 1MB size request?
>
>
The 1M size is requested when doing flash fastboot in
system/core/fastbootd/commands/flash.c defined by Google.
I listed a partial code from flash.c here.
#define BUFFER_SIZE 1024 * 1024
int current_size = MIN(size - written, BUFFER_SIZE);
(gpt_mmap(&input, written + skip, current_size, data_fd))
mapping->size = ALIGN(size + location_diff, PAGE_SIZE);

> >
> > Xiaoming.