Re: [PATCH v2] tcp: splice as many packets as possible at once

Jarek Poplawski
2009-02-03T09:41:29Z

On Mon, Feb 02, 2009 at 11:50:17PM -0800, David Miller wrote:
> From: Jarek Poplawski <jarkao2@gmail.com>
> Date: Mon, 2 Feb 2009 08:43:58 +0000
>
> > On Mon, Feb 02, 2009 at 12:18:54AM -0800, David Miller wrote:
> > > Allocating 4096 or 8192 bytes for a 1500 byte frame is wasteful.
> >
> > I mean allocating chunks of cached pages similarly to sk_sndmsg_page
> > way. I guess the similar problem is to be worked out in any case. But
> > it seems doing it on the linear area requires less changes in other
> > places.
>
> This is a very interesting idea, but it has some drawbacks:
>
> 1) Just like any other allocator we'll need to find a way to
> handle > PAGE_SIZE allocations, and thus add handling for
> compound pages etc.
>
> And exactly the drivers that want such huge SKB data areas
> on receive should be converted to use scatter gather page
> vectors in order to avoid multi-order pages and thus strains
> on the page allocator.

I guess compound pages are handled by put_page() enough, but I don't
think they should be main argument here, and I agree: scatter gather
should be used where possible.

>
> 2) Space wastage and poor packing can be an issue.
>
> Even with SLAB/SLUB we get poor packing, look at Evegeniy's
> graphs that he made when writing his NTA patches.

I'm a bit lost here: could you "remind" the way page space would be
used/saved in your paged variant e.g. for ~1500B skbs?

>
> Now, when choosing a way to move forward, I'm willing to accept a
> little bit of the issues in #2 for the sake of avoiding the
> issues in #1 above.
>
> Jarek, note that we can just keep your current splice() copy hacks in
> there. And as a result we can have an easier to handle migration
> path. We just do the page RX allocation conversions in the drivers
> where performance really matters, for hardware a lot of people have.
>
> That's a lot smoother and has less issues that converting the system
> wide SKB allocator upside down.
>

Yes, this looks reasonable. On the other hand, I think it would be
nice to get some opinions of slab folks (incl. Evgeniy) on the expected
efficiency of such a solution. (It seems releasing with put_page() will
always have some cost with delayed reusing and/or waste of space.)

Jarek P.