From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.saout.de ([127.0.0.1]) by localhost (mail.saout.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id d8VFDEWUYZ_T for ; Wed, 8 Feb 2012 16:26:47 +0100 (CET) Received: from v4.tansi.org (ns.km33513-03.keymachine.de [87.118.94.3]) by mail.saout.de (Postfix) with ESMTP for ; Wed, 8 Feb 2012 16:26:46 +0100 (CET) Received: from gatewagner.dyndns.org (84-74-163-71.dclient.hispeed.ch [84.74.163.71]) by v4.tansi.org (Postfix) with ESMTPA id 44F31204D29 for ; Wed, 8 Feb 2012 16:26:46 +0100 (CET) Date: Wed, 8 Feb 2012 16:26:45 +0100 From: Arno Wagner Message-ID: <20120208152645.GA17166@tansi.org> References: <1328567304.90926.YahooMailNeo@web29602.mail.ird.yahoo.com> <20120207081135.GC17107@tansi.org> <20120207083334.GA18260@tansi.org> <1328713497.10971.YahooMailNeo@web29605.mail.ird.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1328713497.10971.YahooMailNeo@web29605.mail.ird.yahoo.com> Subject: Re: [dm-crypt] Re : Poor performances with nfs and Kernel 3.x List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: dm-crypt@saout.de Have you looked at CPU load on the server? Maybe the difference is just the CPU-power that NFS consumes and decryption is already CPU-Limited before. The low non-encrypted performance, compared to local performance, is also a bit fishy. Pure speculation, but maybe something is consuming CPU on the server or the CPU is slowed-down in some way. If this not a CPU-power issue, then the CPU should have significant idle percentage during encrypted accesses. Arno On Wed, Feb 08, 2012 at 03:04:57PM +0000, Mickael wrote: > >De?: Arno Wagner > >??: dm-crypt@saout.de > >Cc?: > >Envoy? le : Mardi 7 f?vrier 2012 9h33 > >Objet?: Re: [dm-crypt] Poor performances with nfs and Kernel 3.x > > > >O.k., no such obervation with 3.2.2 on client and server. I got > > > >root /gate/tmp>cat ttt | wcs > /dev/null > >read:? 2.147 GB [? 2147483648 B]? ? avg:? 69.274 MB/sec [? ? 31 sec] > > > >(wcs, a.k.a. wc-stream is a small tool I wrote to do real-time > >monitoring of pipeline throughput and byte count, sources > >below.) This is with some ramp-up and almost 100% CPU load > >on the (slower) server. > > > >I noticed though that nfsiostat is not the right tool to measure, as > >it gives you performance over the whole time the device has been > >mounted, i.e. the throughput number keeps being updated in > >real-time. > > > >Maybe re-run with wcs as shown above (compile instructions are in > >the header of wcs.c) and also monitor CPU usage on the server > >while this is rrunning, e.g. with "top". I am not saying you > >imagine the issue, but lets be sure the measurement is good. > > > >If you do not want to compile anything, you could also use > >something like "time cat ttt > /dev/null" and calculate > >throughput manually. > > > >Arno > > > >----- CUTed ----- > > > > > Hello Arno, > Here are the new tests with wcs: results are still the same. > > /tmp is the tmp directory on the client (7200 rpm HD) > /CRYPT is a crypted partition on the server (Green HD ~5600 rpm) > /SAV is a non-crypted partition on the server (Green HD ~5600 rpm) > CPU monitored with htop > > > *** Server with Kernel 3.0.0-15-server #26 > > * From client, using NFS on crypted partition: > > cat /tmp/testfile.avi | wcs > /mnt/CRYPT/testfile.avi > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 41.476 MB/sec [??? 49 sec] > > cat /mnt/CRYPT/testfile.avi | wcs > /tmp/testfile.avi > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 29.454 MB/sec [??? 69 sec]??? <--------------- > > ----------- > > * From client, using NFS on non-crypted partition > > cat /tmp/testfile.avi | wcs > /mnt/SAV/testfile.avi > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 72.583 MB/sec [??? 28 sec] > > cat /mnt/SAV/testfile.avi | wcs > /tmp/testfile.avi > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 65.559 MB/sec [??? 31 sec] > > ----------- > > > * From Server: > > cat /mnt/CRYPT/testfile.avi | wcs > /dev/null > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 72.583 MB/sec [??? 28 sec] > > cat /mnt/SAV/testfile.avi | wcs > /dev/null > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 67.744 MB/sec [??? 30 sec] > > ----------- > > cat /mnt/CRYPT/testfile.avi | wcs > /mnt/SAV/testfile.avi > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 56.454 MB/sec [??? 36 sec] > > cat /mnt/SAV/testfile.avi | wcs > /mnt/CRYPT/testfile.avi > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 54.928 MB/sec [??? 37 sec] > > > > =============================== > > *** Server with Kernel 2.6.38-13-server #52 > > * From client, using NFS on crypted partition: > > cat /tmp/testfile.avi | wcs > /mnt/CRYPT/testfile.avi > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 54.928 MB/sec [??? 37 sec]??? CPU=100% > > cat /mnt/CRYPT/testfile.avi | wcs > /tmp/testfile.avi > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 81.293 MB/sec [??? 25 sec]??? CPU=98%??? ??? <------------------- > > ----------- > > * From client, using NFS on non-crypted partition > > cat /tmp/testfile.avi | wcs > /mnt/SAV/testfile.avi > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 63.510 MB/sec [??? 32 sec]??? CPU=50% > > cat /mnt/SAV/testfile.avi | wcs > /tmp/testfile.avi > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 63.510 MB/sec [??? 32 sec]??? CPU=25% > > > > * From Server: > > cat /mnt/CRYPT/testfile.avi | wcs > /dev/null > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 84.680 MB/sec [??? 24 sec]??? CPU=98% > > cat /mnt/SAV/testfile.avi | wcs > /dev/null > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 67.744 MB/sec [??? 30 sec]??? CPU=25% > > ----------- > > cat /mnt/CRYPT/testfile.avi | wcs > /mnt/SAV/testfile.avi > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 52.111 MB/sec [??? 39 sec]??? CPU=50%-100% > > cat /mnt/SAV/testfile.avi | wcs > /mnt/CRYPT/testfile.avi > ?read:?? 2.032 GB [?? 2032330895 B]??? avg:? 47.264 MB/sec [??? 43 sec]??? CPU=75%-100% > _______________________________________________ > dm-crypt mailing list > dm-crypt@saout.de > http://www.saout.de/mailman/listinfo/dm-crypt > -- Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name GnuPG: ID: 1E25338F FP: 0C30 5782 9D93 F785 E79C 0296 797F 6B50 1E25 338F ---- One of the painful things about our time is that those who feel certainty are stupid, and those with any imagination and understanding are filled with doubt and indecision. -- Bertrand Russell