memcpy() avoidance using decode(bufferlist)

memcpy() avoidance using decode(bufferlist)

[David Disseldorp]

Hi,

A question regarding bufferlist usage:

I'm working on the cmpext librados C API, and want to decode the OSD
response directly into a (char *buf/size_t len) buffer provided by the
caller.

I figured this would be straightforward via:
  bl.push_back(buffer::create_static(buf, len));
  ...
  ::decode_nohead(iter.get_remaining(), bl, iter);

However, the decoded data doesn't make it into buf, unless an extra copy
occurs via:
  bl.copy(0, bl.length(), buf);

Any suggestions on a clean way to avoid the extra copy here?

Cheers, David

Re: memcpy() avoidance using decode(bufferlist)

[Casey Bodley]

On 04/22/2016 09:53 AM, David Disseldorp wrote:
> Hi,
>
> A question regarding bufferlist usage:
>
> I'm working on the cmpext librados C API, and want to decode the OSD
> response directly into a (char *buf/size_t len) buffer provided by the
> caller.
>
> I figured this would be straightforward via:
>    bl.push_back(buffer::create_static(buf, len));
>    ...
>    ::decode_nohead(iter.get_remaining(), bl, iter);
>
> However, the decoded data doesn't make it into buf, unless an extra copy
> occurs via:
>    bl.copy(0, bl.length(), buf);
>
> Any suggestions on a clean way to avoid the extra copy here?
>
> Cheers, David
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Hi David,

The decode logic is appending data to the end of the given bufferlist, 
which is why it's skipping over the buffer pointer that you provide. 
Internally, buffer::list is allocating its own 'buffer::ptr 
append_buffer' to batch these appends. This append_buffer is not 
currently exposed by the interface, but if there was a way to provide 
your own buffer::ptr from buffer::create_static(), you could 
hypothetically use that to avoid the copy.

Casey

Re: memcpy() avoidance using decode(bufferlist)

[David Disseldorp]

Thanks for the tips Casey!

On Fri, 22 Apr 2016 14:05:15 -0400, Casey Bodley wrote:

> This append_buffer is not 
> currently exposed by the interface, but if there was a way to provide 
> your own buffer::ptr from buffer::create_static(), you could 
> hypothetically use that to avoid the copy.

That's a pity. In that case, I guess I'll keep the extra memcpy until
there's an API to directly provide the bufferlist destination buffer
here.

Cheers, David