Re: [PATCH v3 3/3] NFSD: Add support for encoding multiple segments

J. Bruce Fields
2015-04-15T19:56:20Z

On Wed, Apr 15, 2015 at 03:32:02PM -0400, Anna Schumaker wrote:
> I just ran some more tests comparing the directio case across
> different filesystem types. These tests used three 1G files: 100%
> data, 100% hole, and mixed file with alternating 4k data and hole
> segments. The mixed case seems to be consistently slower compared to
> NFS v4.1, and I'm at a loss for anything I could do to make it faster.
> Here are my numbers:

Have you tried the implementation we discussed that always returns a
single segment covering the whole requested range, by treating holes as
data if necessary when they don't cover the whole range?

(Also: I assume it's the same as before, but: when you post test
results, could you repost if necessary:

- what the actual test is
- what the hardware/software setup is on client and server

so that we have reproduceable results for posterity's sake.)

Interesting that "Mixed" is a little slower even before READ_PLUS.

And I guess we should really report this to ext4 people, looks like they
may have a bug.

--b.

>
> ###########
> # #
> # XFS #
> # #
> ###########
>
>
> NFS v4.1:
> Trial
> |---------|---------|---------|---------|---------|---------|---------|
> | | 1 | 2 | 3 | 4 | 5 | Average |
> |---------|---------|---------|---------|---------|---------|---------|
> | Data | 1.883s | 1.808s | 1.781s | 1.685s | 1.591s | 1.746s |
> | Hole | 1.815s | 1.635s | 1.682s | 1.698s | 1.653s | 1.697s |
> | Mixed | 2.089s | 2.024s | 1.970s | 1.925s | 2.049s | 2.011s |
> |---------|---------|---------|---------|---------|---------|---------|
>
>
> NFS v4.2:
> Trial
> |---------|---------|---------|---------|---------|---------|---------|
> | | 1 | 2 | 3 | 4 | 5 | Average |
> |---------|---------|---------|---------|---------|---------|---------|
> | Data | 1.849s | 1.879s | 1.852s | 1.799s | 1.781s | 1.832s |
> | Hole | 0.668s | 0.600s | 0.611s | 0.619s | 0.617s | 0.623s |
> | Mixed | 5.913s | 5.811s | 5.952s | 5.962s | 5.806s | 5.889s |
> |---------|---------|---------|---------|---------|---------|---------|
>
>
>
>
>
> ############
> # #
> # EXT4 #
> # #
> ############
>
>
> NFS v4.1:
> Trial
> |---------|---------|---------|---------|---------|---------|---------|
> | | 1 | 2 | 3 | 4 | 5 | Average |
> |---------|---------|---------|---------|---------|---------|---------|
> | Data | 2.637s | 1.823s | 1.792s | 1.816s | 2.000s | 2.014s |
> | Hole | 1.734s | 1.743s | 1.709s | 1.761s | 1.871s | 1.764s |
> | Mixed | 5.465s | 2.158s | 2.254s | 2.676s | 2.422s | 2.995s |
> |---------|---------|---------|---------|---------|---------|---------|
>
>
> NFS v4.2:
> Trial
> |---------|---------|---------|---------|---------|---------|---------|
> | | 1 | 2 | 3 | 4 | 5 | Average |
> |---------|---------|---------|---------|---------|---------|---------|
> | Data | 1.934s | 1.783s | 1.800s | 2.010s | 1.982s | 1.902s |
> | Hole | 63.568s | 63.423s | 64.671s | 66.190s | 65.985s | 64.767s |
> | Mixed | 6.010s | 5.798s | 6.146s | 6.460s | 6.720s | 6.225s |
> |---------|---------|---------|---------|---------|---------|---------|
>
>
>
>
>
> #############
> # #
> # BTRFS #
> # #
> #############
>
>
> NFS v4.1:
> Trial
> |---------|---------|---------|---------|---------|---------|---------|
> | | 1 | 2 | 3 | 4 | 5 | Average |
> |---------|---------|---------|---------|---------|---------|---------|
> | Data | 2.386s | 1.952s | 1.832s | 1.818s | 1.826s | 1.963s |
> | Hole | 1.759s | 1.717s | 1.754s | 1.621s | 1.708s | 1.712s |
> | Mixed | 2.889s | 2.272s | 2.778s | 2.277s | 2.255s | 2.494s |
> |---------|---------|---------|---------|---------|---------|---------|
>
>
> NFS v4.2:
> Trial
> |---------|---------|---------|---------|---------|---------|---------|
> | | 1 | 2 | 3 | 4 | 5 | Average |
> |---------|---------|---------|---------|---------|---------|---------|
> | Data | 2.586s | 1.816s | 2.022s | 1.862s | 1.975s | 2.052s |
> | Hole | 0.646s | 0.659s | 0.669s | 0.628s | 0.605s | 0.641s |
> | Mixed | 8.555s | 8.553s | 7.904s | 8.567s | 8.286s | 8.373s |
> |---------|---------|---------|---------|---------|---------|---------|