From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o881DAml005855 for ; Tue, 7 Sep 2010 20:13:10 -0500 Received: from EXHUB018-3.exch018.msoutlookonline.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 7B48717B815E for ; Tue, 7 Sep 2010 18:13:53 -0700 (PDT) Received: from EXHUB018-3.exch018.msoutlookonline.net (exhub018-3.exch018.msoutlookonline.net [64.78.17.18]) by cuda.sgi.com with ESMTP id CXnGCzsYCawtMgrt for ; Tue, 07 Sep 2010 18:13:53 -0700 (PDT) From: Brady Chang Date: Tue, 7 Sep 2010 18:16:43 -0700 Subject: Re: fragmentation question Message-ID: In-Reply-To: Content-Language: en MIME-Version: 1.0 List-Id: XFS Filesystem from SGI List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: multipart/mixed; boundary="===============6726082990017508048==" Sender: xfs-bounces@oss.sgi.com Errors-To: xfs-bounces@oss.sgi.com To: "xfs@oss.sgi.com" --===============6726082990017508048== Content-Language: en Content-Type: multipart/alternative; boundary="_000_C8AC320B6CB6bchanggreenplumcom_" --_000_C8AC320B6CB6bchanggreenplumcom_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable By the way, the os is RHEL 5.5 kernel 2.6.18-194.11.1.el5 thanks in advance. -Brady On 9/7/10 5:45 PM, "Brady Chang" wrote: Hello All, I have an issue with fragmentation on a particular device thanks for any advice. -Brady I have a Dell r510 with 12 disks 2xraid 5 (6 disks each) raid group1: 48 GB carved out for os mounted as / remaining space 2.7 TB for xfs mounted as /data1 raid group2: 48 GB for swap remaining space 2.7 TB for xfs mounted as /data2 The strange thing is that /data1 never gets fragmented where as /data2 is b= adly fragmented. I believe increase allocsize would help, but not sure how to explain why /d= ata2(/dev/sdd) always gets fragmented and not /data1(/dev/sdb) It's a data warehouse application. the I/O is balanced between /data1 and = /data2: output of xfs_db [root@sdw4 data1]# xfs_db -c frag -r /dev/sdb actual 14353, ideal 13702, fragmentation factor 4.54% [root@sdw4 data1]# xfs_db -c frag -r /dev/sdd actual 408674, ideal 13719, fragmentation factor 96.64% df output /dev/sdb 2.7T 967G 1.8T 36% /data1 /dev/sdd 2.7T 1.1T 1.7T 39% /data2 LABEL=3D/data1 /data1 xfs allocsize=3D1048576,logbufs=3D8,no= atime,nodiratime 0 0 LABEL=3D/data2 /data2 xfs allocsize=3D1048576,logbufs=3D8,no= atime,nodiratime 0 0 raid config output: [root@sdw4 data1]# omreport storage vdisk List of Virtual Disks in the System Controller PERC H700 Integrated (Slot 4) ID : 0 Status : Ok Name : boot State : Ready Hot Spare Policy violated : Not Assigned Virtual Disk Bad Blocks : No Secured : Not Applicable Progress : Not Applicable Layout : RAID-5 Size : 48.99 GB (52602470400 bytes) Device Name : /dev/sda Bus Protocol : SAS Media : HDD Read Policy : No Read Ahead Write Policy : Force Write Back Cache Policy : Not Applicable Stripe Element Size : 128 KB Disk Cache Policy : Disabled ID : 1 Status : Ok Name : data1 State : Ready Hot Spare Policy violated : Not Assigned Virtual Disk Bad Blocks : No Secured : Not Applicable Progress : Not Applicable Layout : RAID-5 Size : 2,742.89 GB (2945150484480 bytes) Device Name : /dev/sdb Bus Protocol : SAS Media : HDD Read Policy : No Read Ahead Write Policy : Force Write Back Cache Policy : Not Applicable Stripe Element Size : 128 KB Disk Cache Policy : Disabled ID : 2 Status : Ok Name : swap State : Ready Hot Spare Policy violated : Not Assigned Virtual Disk Bad Blocks : No Secured : Not Applicable Progress : Not Applicable Layout : RAID-5 Size : 48.99 GB (52602470400 bytes) Device Name : /dev/sdc Bus Protocol : SAS Media : HDD Read Policy : No Read Ahead Write Policy : Force Write Back Cache Policy : Not Applicable Stripe Element Size : 128 KB Disk Cache Policy : Disabled ID : 3 Status : Ok Name : data2 State : Ready Hot Spare Policy violated : Not Assigned Virtual Disk Bad Blocks : No Secured : Not Applicable Progress : Not Applicable Layout : RAID-5 Size : 2,742.89 GB (2945150484480 bytes) Device Name : /dev/sdd Bus Protocol : SAS Media : HDD Read Policy : No Read Ahead Write Policy : Force Write Back Cache Policy : Not Applicable Stripe Element Size : 128 KB Disk Cache Policy : Disabled --_000_C8AC320B6CB6bchanggreenplumcom_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Re: fragmentation question By the way, the os is RHEL 5.5 kernel 2.6.18-194.11.1.el5

thanks in advance.
-Brady


On 9/7/10 5:45 PM, "Brady Chang" <bchang@greenplum.com> wrote:

Hello All,
I have an issue with fragmentation on a particular device
thanks for any advice.

-Brady

I have a Dell r510 with 12 disks
2xraid 5 (6 disks each)
raid group1:
48 GB   carved out for os mounted as /
remaining space  2.7 TB for xfs mounted as /data1
raid group2:
48 GB  for swap
remaining space 2.7 TB for xfs mounted as /data2

The strange thing is that /data1 never gets fragmented where as /data2 is b= adly fragmented.
I believe increase allocsize would help, but not sure how to explain why /d= ata2(/dev/sdd) always gets fragmented and not /data1(/dev/sdb)

It's a data warehouse application.  the I/O is balanced between /data1= and /data2:
output of xfs_db
[root@sdw4 data1]# xfs_db -c frag -r /dev/sdb
actual 14353, ideal 13702, fragmentation factor 4.54%
[root@sdw4 data1]# xfs_db -c frag -r /dev/sdd
actual 408674, ideal 13719, fragmentation factor 96.64%
df output
/dev/sdb            =   2.7T  967G  1.8T  36% /data1
/dev/sdd            =   2.7T  1.1T  1.7T  39% /data2

LABEL=3D/data1        /data1   = ;  xfs     allocsize=3D1048576,logbufs=3D8,no= atime,nodiratime 0 0
LABEL=3D/data2        /data2   = ;  xfs     allocsize=3D1048576,logbufs=3D8,no= atime,nodiratime 0 0


raid config output:
[root@sdw4 data1]# omreport storage vdisk
List of Virtual Disks in the System

Controller PERC H700 Integrated (Slot 4)
ID             =            : 0
Status            &n= bsp;       : Ok
Name            &nbs= p;         : boot
State            &nb= sp;        : Ready
Hot Spare Policy violated : Not Assigned
Virtual Disk Bad Blocks   : No
Secured            &= nbsp;      : Not Applicable
Progress            =       : Not Applicable
Layout            &n= bsp;       : RAID-5
Size            &nbs= p;         : 48.99 GB (5260247= 0400 bytes)
Device Name           &nb= sp;   : /dev/sda
Bus Protocol           &n= bsp;  : SAS
Media            &nb= sp;        : HDD
Read Policy           &nb= sp;   : No Read Ahead
Write Policy           &n= bsp;  : Force Write Back
Cache Policy           &n= bsp;  : Not Applicable
Stripe Element Size       : 128 KB
Disk Cache Policy         : Disable= d

ID             =            : 1
Status            &n= bsp;       : Ok
Name            &nbs= p;         : data1
State            &nb= sp;        : Ready
Hot Spare Policy violated : Not Assigned
Virtual Disk Bad Blocks   : No
Secured            &= nbsp;      : Not Applicable
Progress            =       : Not Applicable
Layout            &n= bsp;       : RAID-5
Size            &nbs= p;         : 2,742.89 GB (2945= 150484480 bytes)
Device Name           &nb= sp;   : /dev/sdb
Bus Protocol           &n= bsp;  : SAS
Media            &nb= sp;        : HDD
Read Policy           &nb= sp;   : No Read Ahead
Write Policy           &n= bsp;  : Force Write Back
Cache Policy           &n= bsp;  : Not Applicable
Stripe Element Size       : 128 KB
Disk Cache Policy         : Disable= d

ID             =            : 2
Status            &n= bsp;       : Ok
Name            &nbs= p;         : swap
State            &nb= sp;        : Ready
Hot Spare Policy violated : Not Assigned
Virtual Disk Bad Blocks   : No
Secured            &= nbsp;      : Not Applicable
Progress            =       : Not Applicable
Layout            &n= bsp;       : RAID-5
Size            &nbs= p;         : 48.99 GB (5260247= 0400 bytes)
Device Name           &nb= sp;   : /dev/sdc
Bus Protocol           &n= bsp;  : SAS
Media            &nb= sp;        : HDD
Read Policy           &nb= sp;   : No Read Ahead
Write Policy           &n= bsp;  : Force Write Back
Cache Policy           &n= bsp;  : Not Applicable
Stripe Element Size       : 128 KB
Disk Cache Policy         : Disable= d

ID             =            : 3
Status            &n= bsp;       : Ok
Name            &nbs= p;         : data2
State            &nb= sp;        : Ready
Hot Spare Policy violated : Not Assigned
Virtual Disk Bad Blocks   : No
Secured            &= nbsp;      : Not Applicable
Progress            =       : Not Applicable
Layout            &n= bsp;       : RAID-5
Size            &nbs= p;         : 2,742.89 GB (2945= 150484480 bytes)
Device Name           &nb= sp;   : /dev/sdd
Bus Protocol           &n= bsp;  : SAS
Media            &nb= sp;        : HDD
Read Policy           &nb= sp;   : No Read Ahead
Write Policy           &n= bsp;  : Force Write Back
Cache Policy           &n= bsp;  : Not Applicable
Stripe Element Size       : 128 KB
Disk Cache Policy         : Disable= d
--_000_C8AC320B6CB6bchanggreenplumcom_-- --===============6726082990017508048== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs --===============6726082990017508048==--