controlling erasure code chunk size

[Loic Dachary <loic@dachary.org>]

[-- Attachment #1: Type: text/plain, Size: 953 bytes --]

[cc' ceph-devel]

Hi Sam,

Here is how chunks are expected to be aligned:

https://github.com/ceph/ceph/blob/4c4e1d0d470beba7690d1c0e39bfd1146a25f465/src/osd/ErasureCodePluginJerasure/ErasureCodeJerasure.cc#L365

 unsigned alignment = k*w*packetsize*sizeof(int);
  if ( ((w*packetsize*sizeof(int))%LARGEST_VECTOR_WORDSIZE) )
    alignment = k*w*packetsize*LARGEST_VECTOR_WORDSIZE;
  return alignment;

If you are going to encode small objects, it may very well lead to oversized chunks if packetsize is large. At the moment the default is 3072

https://github.com/ceph/ceph/blob/4c4e1d0d470beba7690d1c0e39bfd1146a25f465/src/common/config_opts.h#L406

A value I picked when experimenting with 1MB objects encoding ( http://dachary.org/?p=2594 ). 

I'm not entirely sure why the alignment is calculated the way it is. Andreas certainly has a better understanding on this topic.

Cheers

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 263 bytes --]