From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from tansi.org (ns.km10532-04.keymachine.de [87.118.102.195]) by mail.saout.de (Postfix) with ESMTP for ; Tue, 9 Feb 2010 17:13:16 +0100 (CET) Received: from gatewagner.dyndns.org (84-74-164-239.dclient.hispeed.ch [84.74.164.239]) by tansi.org (Postfix) with ESMTP id 2DE3E2128001 for ; Tue, 9 Feb 2010 17:13:16 +0100 (CET) Date: Tue, 9 Feb 2010 17:14:48 +0100 From: Arno Wagner Message-ID: <20100209161448.GA23112@tansi.org> References: <20100208011654.GA4927@southpole.se> <20100208235416.GA8981@southpole.se> <20100209002806.GA14310@tansi.org> <20100209144829.GA4211@southpole.se> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="h31gzZEtNLTqOjlF" Content-Disposition: inline In-Reply-To: <20100209144829.GA4211@southpole.se> Subject: Re: [dm-crypt] Poor performane (idle cpu) [SOLVED; problem with "pv"] List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: dm-crypt@saout.de --h31gzZEtNLTqOjlF Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tue, Feb 09, 2010 at 03:48:29PM +0100, Jakob Sandgren wrote: > On Tue, Feb 09, 2010 at 01:28:06AM +0100, Arno Wagner wrote: > > On Tue, Feb 09, 2010 at 12:54:16AM +0100, Jakob Sandgren wrote: > > > Hi, > > > > > > (please keep me on CC since I'm not subscribed yet) > > > > > > To me it seems like there is some serious flaw within kcryptd that > > > ends up to wait for "something" instead of sending enough requests to > > > the disks to make sure it has data to decrypt. What do you think? > > > > The same thing. > > > > Here is a reference test (I have notebook disks in this server): > > > > Raw read: 54MB/s 14% CPU > > Read with decrypt: 53MB/s 65% CPU > > For reference this is the exact output of my benchmark, maybe there > are some difference in setup or benchmark? > > OOOOOooops! While putting toghether this information I actually found > the cause of the problem, it was my benchark that was wrong! > > This was the benchmark I used to get the performance was: > dd if=/dev/mapper/bench1 bs=4M iflag=direct |pv | dd of=/dev/null > and the number reported by "pv" during the run was ~75MB/s and dd > reported the same number when finished. > > Changing this to: > dd if=/dev/mapper/bench1 bs=4M iflag=direct of=/dev/null count=1000 > gave a more correct number; 125MB/s > > > I was not aware of that piping the data through pv would cause such a > big degradation in performance. > Ah, yes. I think it is historic and uses some things that are slow. I have my own tool (wcs for "wc-stream") that is a bit faster and provides real-time speeds on a vt100. It is really small. Sources are attached. > > That would mean the crypto is pretty slow on your new CPU. > > As a reference, my 53MB/s at 65% CPU is on an 2800MHz Athlon > > 64 X2 5600+ with aes-cbc-plain. > > > > Here is an OpenSSL crypto speed test: > > openssl speed -evp aes-256-cbc > > [...] > > The 'numbers' are in 1000s of bytes per second processed. > > type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes > > aes-256-cbc 71848.00k 98649.49k 110187.78k 113646.25k 114666.15k > > > > You might want to compare this with the numbers on your CPU. > > The numbers from my system (Core I7) are below > > root@mvh:~# openssl speed -evp aes-256-cbc > ... > The 'numbers' are in 1000s of bytes per second processed. > type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes > aes-256-cbc 110769.28k 118629.67k 120600.15k 121138.86k 121206.10k > > I has now been able to get a 175MB/sec from my main raid partition. Good. Arno -- Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name GnuPG: ID: 1E25338F FP: 0C30 5782 9D93 F785 E79C 0296 797F 6B50 1E25 338F ---- Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans If it's in the news, don't worry about it. The very definition of "news" is "something that hardly ever happens." -- Bruce Schneier --h31gzZEtNLTqOjlF Content-Type: text/x-csrc; charset=us-ascii Content-Disposition: attachment; filename="wcs.c" /* "wc-follow": wc with incremental output every second or so. * Line positioning is done like in dd_rescue with * "optimistic positioning" i.e. the hope that we have a terminal that * understands vt100 positioning. * * (C) Arno Wagner 2008. Distributed under * The GNU public license v2 * * Version 1.0 * * Compile with "gcc -O6 -o wcs wcs.c" */ #include #include #include const char* up = "\x1b[A"; //] const char* down = "\n"; const char* right = "\x1b[C"; //] char * usage() { return("\n" " Prints out running count and rate statistics about the data\n" " read on stdin to stderr. Does use cursor positioning, which\n" " should work on most terminals (vt100 or later).\n" "\n" " stdin is copied through to stdout, much like in tee.\n" " \n" " This programm does not support any commandline arguments.\n" "\n" "\n"); } void printlong(long long l) { if (l < 1000000L) fprintf(stderr, "%7.3f kB", l/1000.0); else if (l < 1000000000) fprintf(stderr, "%7.3f MB", l/1000000.0); else if (l < 1000000000000LL) fprintf(stderr, "%7.3f GB", l/1000000000.0); else fprintf(stderr, "%7.3f TB", l/1000000000000.0); } int main(int argc, char ** argv) { char buf[4096]; long long cnt = 0; time_t to, tn, ts, dt; int read_count; double rate; if (argc > 1) { fprintf(stderr, "%s", usage()); exit(1); } to = time(NULL); ts = to; fprintf(stderr, down); while (1) { read_count = read (0, buf, sizeof(buf)); if (read_count < 0 && errno == EINTR) continue; // not an error if (read_count < 0) { perror("Abnormal condition in read from stdin:"); exit(1); } if (read_count > 0) { cnt += read_count; if (fwrite (buf, 1, read_count, stdout) != read_count) { perror("Error in wcs writing to stdout:"); exit(1); } } tn = time(NULL); if (tn != to || read_count == 0) { to = tn; fprintf(stderr, "%s read: ", up); printlong(cnt); fprintf(stderr, " [ %12Ld B] avg: ", cnt); // Calculate the rate dt = tn - ts; rate = cnt / (double)dt; printlong((long long) rate); fprintf(stderr, "/sec [ %5d sec]%s", dt, down); } if (read_count == 0) break; // we are done } } --h31gzZEtNLTqOjlF--