CEPH filesystem development
 help / color / mirror / Atom feed
From: Janne Grunau <j@jannau.net>
To: Andreas Joachim Peters <Andreas.Joachim.Peters@cern.ch>
Cc: "ceph-devel@vger.kernel.org" <ceph-devel@vger.kernel.org>
Subject: Re: v2 aligned buffer changes for erasure codes
Date: Thu, 18 Sep 2014 15:23:55 +0200	[thread overview]
Message-ID: <20140918132355.GD25730@jannau.net> (raw)
In-Reply-To: <3472A07E6605974CBC9BC573F1BC02E4AE7D1488@CERNXCHG44.cern.ch>

On 2014-09-18 13:01:03 +0000, Andreas Joachim Peters wrote:
> 
> >> For encoding there is normally a single buffer split 'virtually' 
> >> into
> >> k pieces. To make all pieces starting at an aligned address one 
> >> needs
> >> to align the chunk size to e.g. 16*k.
> 
> >I don't get that. How is the buffer splitted? into k (+ m) chunk size
> >parts? As long as the start and the length are both 16 (or 32) byte
> >aligned all parts are properly aligned too. I don't see where the k
> >comes into play.
> 
> The original data block to encode has to be split into k equally long 
> pieces. Each piece is given as one of the k input buffers to the 
> erasure code algorithm producing m output buffers and each piece has 
> to have an aligned starting address and length.
> 
> If you deal with 128 byte data input buffers for k=4 it splits like
> 
> offset=00 len=32 as chunk1
> offset=32 len=32 as chunk2
> offset=64 len=32 as chunk3
> offset=96 len=32 as chunk4
> 
> If the desired IO size would be 196 bytes the 32 byte alignment 
> requirement blows this buffer up to 256 bytes:
> 
> offset=00 len=64 as chunk1
> offset=64 len=64 as chunk2
> offset=128 len=64 as chunk3
> offset=196 len=64 as chunk4

I fail to see how the 32 * k is related to alignment. It's only used for 
to pad the total size so it becomes a mulitple of k * 32. That is ok 
since we want k 32-byte aligned chunks. The alignment for each chunk is 
just 32-bytes.

Janne

  reply	other threads:[~2014-09-18 13:23 UTC|newest]

Thread overview: 41+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-09-15 15:55 [PATCH 1/3] buffer: add an aligned buffer with less alignment than a page Janne Grunau
2014-09-15 15:55 ` [PATCH 2/3] ec: make use of added aligned buffers Janne Grunau
2014-09-15 17:20   ` Loic Dachary
2014-09-15 23:56     ` Ma, Jianpeng
2014-09-16  0:02       ` Sage Weil
2014-09-16  0:08         ` Ma, Jianpeng
2014-09-16  6:47           ` Loic Dachary
2014-09-16  6:59             ` Ma, Jianpeng
2014-09-16  7:55               ` Loic Dachary
2014-09-16  8:23                 ` Ma, Jianpeng
2014-09-15 15:55 ` [PATCH 3/3] ceph_erasure_code_benchmark: align the encoding input Janne Grunau
2014-09-15 16:46 ` [PATCH 1/3] buffer: add an aligned buffer with less alignment than a page Loic Dachary
2014-09-18 10:33 ` v2 aligned buffer changes for erasure codes Janne Grunau
2014-09-18 10:33   ` [PATCH v2 1/3] buffer: add an aligned buffer with less alignment than a page Janne Grunau
2014-09-18 10:33   ` [PATCH v2 2/3] ec: use 32-byte aligned buffers Janne Grunau
2014-09-19  9:47     ` Loic Dachary
2014-09-18 10:33   ` [PATCH v2 3/3] ceph_erasure_code_benchmark: align the encoding input Janne Grunau
2014-09-18 12:18   ` v2 aligned buffer changes for erasure codes Andreas Joachim Peters
2014-09-18 12:34     ` Andreas Joachim Peters
2014-09-18 12:53       ` Janne Grunau
2014-09-19  9:18       ` Loic Dachary
2014-09-18 12:40     ` Janne Grunau
2014-09-18 13:01       ` Andreas Joachim Peters
2014-09-18 13:23         ` Janne Grunau [this message]
2014-09-18 14:47           ` Andreas Joachim Peters
2014-09-29 12:34 ` [PATCH v3 0/4] buffer alignment for erasure code SIMD Janne Grunau
2014-09-29 12:34   ` [PATCH v3 1/4] buffer: add an aligned buffer with less alignment than a page Janne Grunau
2014-09-29 13:12     ` Loic Dachary
2014-10-02 12:09       ` Janne Grunau
2014-09-29 13:27     ` Loic Dachary
2014-10-02 12:12       ` Janne Grunau
2014-10-02 14:17         ` Loic Dachary
2014-09-29 12:34   ` [PATCH v3 2/4] erasure code: use a function for the chunk mapping index Janne Grunau
2014-09-29 12:34   ` [PATCH v3 3/4] erasure code: use 32-byte aligned buffers Janne Grunau
2014-09-29 12:34   ` [PATCH v3 4/4] ceph_erasure_code_benchmark: use 32-byte aligned input Janne Grunau
2014-09-29 13:15   ` [PATCH v3 0/4] buffer alignment for erasure code SIMD Loic Dachary
2014-09-29 15:18   ` Milosz Tanski
2014-09-29 15:24     ` C++11 Sage Weil
2014-09-29 15:44       ` C++11 Milosz Tanski
2014-09-29 17:56       ` C++11 Wido den Hollander
2014-10-02 12:15     ` [PATCH v3 0/4] buffer alignment for erasure code SIMD Janne Grunau

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20140918132355.GD25730@jannau.net \
    --to=j@jannau.net \
    --cc=Andreas.Joachim.Peters@cern.ch \
    --cc=ceph-devel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox