From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <mbroz@redhat.com>
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28])
 by mail.saout.de (Postfix) with ESMTP
 for <dm-crypt@saout.de>; Sat, 14 May 2011 17:31:03 +0200 (CEST)
Message-ID: <4DCEA032.1030207@redhat.com>
Date: Sat, 14 May 2011 17:30:58 +0200
From: Milan Broz <mbroz@redhat.com>
MIME-Version: 1.0
References: <BANLkTimMGF-fKzoyrrWKd-f4TR_ohANRbw@mail.gmail.com>
In-Reply-To: <BANLkTimMGF-fKzoyrrWKd-f4TR_ohANRbw@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Subject: Re: [dm-crypt] FSB saturation?
List-Id: <dm-crypt.saout.de>
List-Unsubscribe: <http://www.saout.de/mailman/options/dm-crypt>,
 <mailto:dm-crypt-request@saout.de?subject=unsubscribe>
List-Archive: <http://www.saout.de/pipermail/dm-crypt>
List-Post: <mailto:dm-crypt@saout.de>
List-Help: <mailto:dm-crypt-request@saout.de?subject=help>
List-Subscribe: <http://www.saout.de/mailman/listinfo/dm-crypt>,
 <mailto:dm-crypt-request@saout.de?subject=subscribe>
To: Kyle Rose <krose@krose.org>
Cc: dm-crypt <dm-crypt@saout.de>

On 05/14/2011 04:36 PM, Kyle Rose wrote:
> For the past week I've been trying to figure out why my
> dm-crypt-on-RAID6 performance is so terrible. To wit:
> 
> root@yupa:/source# hdparm -t /dev/md1
> 
> /dev/md1: <---the RAID 6 block device
>  Timing buffered disk reads: 1262 MB in  3.00 seconds = 420.19 MB/sec
> 
> root@yupa:/source# hdparm -t /dev/mapper/yupa
> 
> /dev/mapper/yupa: <---the dm-crypt device with /dev/md1 as the underlying store
>  Timing buffered disk reads: 370 MB in  3.00 seconds = 123.26 MB/sec

IIRC hdparm is doing direct io of size 2MB.

Can you try do the same but with dd (just dd from device without direct mode).
like "dd if=/dev/md1 of=/dev/null bs=64k count=4096"

Gives it better throughput now? (on 2.6.38 and above)

Milan