Re: [PATCH 4/5] block: introduce helpers for allocating IO buffers from slab

Re: [PATCH 4/5] block: introduce helpers for allocating IO buffers from slab

[Ming Lei]

On Thu, Oct 18, 2018 at 05:22:19PM +0200, Christoph Hellwig wrote:
> On Thu, Oct 18, 2018 at 08:11:23AM -0700, Matthew Wilcox wrote:
> > On Thu, Oct 18, 2018 at 04:42:07PM +0200, Christoph Hellwig wrote:
> > > This all seems quite complicated.
> > > 
> > > I think the interface we'd want is more one that has a little
> > > cache of a single page in the queue, and a little bitmap which
> > > sub-page size blocks of it are used.
> > > 
> > > Something like (pseudo code minus locking):
> > > 
> > > void *blk_alloc_sector_buffer(struct block_device *bdev, gfp_t gfp)
> > > {
> > > 	unsigned block_size = block_size(bdev);
> > > 
> > > 	if (blocksize >= PAGE_SIZE)
> > > 		return (void *)__get_free_pages(gfp, get_order(blocksize));
> > > 
> > > 	if (bdev->fragment_cache_page) {
> > > 		[ <find fragment in bdev->fragment_cache_page using
> > > 		  e.g. bitmap and return if found]
> > > 	}
> > > 
> > > 	bdev->fragment_cache_page = (void *)__get_free_page(gfp);
> > > 	goto find_again;
> > > }
> > 
> > This looks a lot like page_frag_alloc() except I think page_frag_alloc()
> > may be more efficient.
> 
> Oh, nice. Sounds like XFS should just use page_frag_alloc.  I'll give
> it a spin.

XFS or other fs can use page_frag_alloc() directly, seems not necessary to
introduce this change in block layer any more given 512-aligned buffer
should be fine everywhere.

The only benefit to make it as block helper is that the offset or size
can be checked with q->dma_alignment.

Dave/Jens, do you think which way is better? Put allocation as block
helper or fs uses page_frag_alloc() directly for allocating 512*N-byte 
buffer(total size is less than PAGE_SIZE)?

Thanks,
Ming

Re: [PATCH 4/5] block: introduce helpers for allocating IO buffers from slab

[Jens Axboe]

On 10/18/18 8:53 PM, Ming Lei wrote:
> On Thu, Oct 18, 2018 at 05:22:19PM +0200, Christoph Hellwig wrote:
>> On Thu, Oct 18, 2018 at 08:11:23AM -0700, Matthew Wilcox wrote:
>>> On Thu, Oct 18, 2018 at 04:42:07PM +0200, Christoph Hellwig wrote:
>>>> This all seems quite complicated.
>>>>
>>>> I think the interface we'd want is more one that has a little
>>>> cache of a single page in the queue, and a little bitmap which
>>>> sub-page size blocks of it are used.
>>>>
>>>> Something like (pseudo code minus locking):
>>>>
>>>> void *blk_alloc_sector_buffer(struct block_device *bdev, gfp_t gfp)
>>>> {
>>>> 	unsigned block_size = block_size(bdev);
>>>>
>>>> 	if (blocksize >= PAGE_SIZE)
>>>> 		return (void *)__get_free_pages(gfp, get_order(blocksize));
>>>>
>>>> 	if (bdev->fragment_cache_page) {
>>>> 		[ <find fragment in bdev->fragment_cache_page using
>>>> 		  e.g. bitmap and return if found]
>>>> 	}
>>>>
>>>> 	bdev->fragment_cache_page = (void *)__get_free_page(gfp);
>>>> 	goto find_again;
>>>> }
>>>
>>> This looks a lot like page_frag_alloc() except I think page_frag_alloc()
>>> may be more efficient.
>>
>> Oh, nice. Sounds like XFS should just use page_frag_alloc.  I'll give
>> it a spin.
> 
> XFS or other fs can use page_frag_alloc() directly, seems not necessary to
> introduce this change in block layer any more given 512-aligned buffer
> should be fine everywhere.
> 
> The only benefit to make it as block helper is that the offset or size
> can be checked with q->dma_alignment.
> 
> Dave/Jens, do you think which way is better? Put allocation as block
> helper or fs uses page_frag_alloc() directly for allocating 512*N-byte 
> buffer(total size is less than PAGE_SIZE)?

I'd greatly prefer having the FS use that directly, seems kind of
pointless to provide an abstraction for that at that point.

-- 
Jens Axboe

Re: [PATCH 4/5] block: introduce helpers for allocating IO buffers from slab

[Dave Chinner]

On Fri, Oct 19, 2018 at 10:53:49AM +0800, Ming Lei wrote:
> On Thu, Oct 18, 2018 at 05:22:19PM +0200, Christoph Hellwig wrote:
> > On Thu, Oct 18, 2018 at 08:11:23AM -0700, Matthew Wilcox wrote:
> > > On Thu, Oct 18, 2018 at 04:42:07PM +0200, Christoph Hellwig wrote:
> > > > This all seems quite complicated.
> > > > 
> > > > I think the interface we'd want is more one that has a little
> > > > cache of a single page in the queue, and a little bitmap which
> > > > sub-page size blocks of it are used.
> > > > 
> > > > Something like (pseudo code minus locking):
> > > > 
> > > > void *blk_alloc_sector_buffer(struct block_device *bdev, gfp_t gfp)
> > > > {
> > > > 	unsigned block_size = block_size(bdev);
> > > > 
> > > > 	if (blocksize >= PAGE_SIZE)
> > > > 		return (void *)__get_free_pages(gfp, get_order(blocksize));
> > > > 
> > > > 	if (bdev->fragment_cache_page) {
> > > > 		[ <find fragment in bdev->fragment_cache_page using
> > > > 		  e.g. bitmap and return if found]
> > > > 	}
> > > > 
> > > > 	bdev->fragment_cache_page = (void *)__get_free_page(gfp);
> > > > 	goto find_again;
> > > > }
> > > 
> > > This looks a lot like page_frag_alloc() except I think page_frag_alloc()
> > > may be more efficient.
> > 
> > Oh, nice. Sounds like XFS should just use page_frag_alloc.  I'll give
> > it a spin.
> 
> XFS or other fs can use page_frag_alloc() directly, seems not necessary to
> introduce this change in block layer any more given 512-aligned buffer
> should be fine everywhere.
> 
> The only benefit to make it as block helper is that the offset or size
> can be checked with q->dma_alignment.
> 
> Dave/Jens, do you think which way is better? Put allocation as block
> helper or fs uses page_frag_alloc() directly for allocating 512*N-byte 
> buffer(total size is less than PAGE_SIZE)?

Cristoph has already said he's looking at using page_frag_alloc()
directly in XFS....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com