Re: slowness when cp respectively send/receiving on top of dm-crypt

[Christoph Anton Mitterer <calestyo@scientia.net>]

[-- Attachment #1: Type: text/plain, Size: 3061 bytes --]

On Fri, 2015-11-27 at 20:00 +0100, Henk Slager wrote:
> As far as I can guess this is transfers between Seagate Archive 8TB
> SMR drives.
Yes it is,... and I though about SMR being the reason at first, too,
but:
- As far as I understood SMR, it shouldn't kick in when I do what is
mostly streaming data. Okay I don't know exactly how btrfs writes it's
data, but when I send/receive 7GB I'd have expected that a great deal
of it is just sequential writing.

- When these disks move data from their non shingled areas to the
shingled ones, that - or at least that's my impression - produces some
typical sounds from the mechanical movements, which I didn't hear

- Bust most importantly,... if the reason was SMR, why should always
when no IO happens dmcrypt_write be at basically full CPU.


> I think you know this:
> https://www.mail-archive.com/linux-btrfs%40vger.kernel.org/msg47341.h
> tml
> and certainly this:
> https://bugzilla.kernel.org/show_bug.cgi?id=93581
I knew the later, but as you've mentioned, the USB/SATA bridge probably
save me from it. Interesting that USB3 is still slow enough not to get
caught ;)

But thanks for the hint nevertheless :)



> > There seems to be a repeating schema like this:
> > - First, there is some heavy disk IO (200-250 M/s), mostly on btrfs
> I think it is MByte/s (and not Mbit/s) right?
Oh yes.. it's whatever iotop shows (not sure whether the use MB or MiB)


> I must say that adding compression (compress-force=zlib mount option)
> makes the whole transferchain tend to not pipeline.
Ah? Well if I'd have known that in advance ^^ (although I just use
compress)...
Didn't marketing tell people that compression may even speed up IO
because the CPUs are so much faster than the disks?

>  Just dm-crypt I
> have not seen it on Core-i7 systems. A test between 2 modern SSDs
> (SATA3 connected ) is likely needed to see if there really is
> tendency
> for hiccups in processing/pipelining. On kernels 3.11 to 4.0 I have
> seen and experienced far from optimal behavior, but with 4.3 it is
> quite OK, although I use large bcache which can mitigate HDD seeks
> quite well.
I remember that in much earlier times, there was something about dm-
crypt that is used just a single thread or so for IO...forgot the
details though.

> >Not sure if this is something that could be optimised or maybe it's
> > even a non issue that happens for example while many small files
> > are
> > read/written (the data consists of both, many small files as well
> > as
> > many big files), which may explain why sometimes the actual IO goes
> > up
> > to large >200M/s or at least > 150M/s and sometimes it caps at
> > around
> > 40-80M/s
> Indeed typical behavior of SMR drive
Well it's not that I wanted to complain... I can live with that
speeds... I just thought that there may be some bad playing between dm-
crypt and btrfs, which could have been shown by these periods in which
nothing seems to happen except dmcrypt_write doing stuff.


Cheers,
Chris

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5313 bytes --]