* sw and su for hardware RAID10 (w/ LVM) @ 2014-03-11 4:56 Ray Van Dolson 2014-03-12 11:37 ` Stan Hoeppner 0 siblings, 1 reply; 4+ messages in thread From: Ray Van Dolson @ 2014-03-11 4:56 UTC (permalink / raw) To: xfs RHEL6.x + XFS that comes w/ Red Hat's scalable file system add on. We have two PowerVault MD3260e's each configured with a 30 disk RAID10 (15 RAID groups) exposed to our server. Segment size is 128K (in Dell's world I'm not sure if this means my stripe width is 128K*15?) Have set up a concatenated LVM volume on top of these two "virtual disks" (with lvcreate -i 2). By default LVM says it's used a stripe width of 64K. # lvs -o path,size,stripes,stripe_size Path LSize #Str Stripe /dev/agsfac_vg00/lv00 100.00t 2 64.00k Unsure if these defaults should be adjusted. I'm trying to figure out the appropriate sw/su values to use per: http://xfs.org/index.php/XFS_FAQ#Q:_How_to_calculate_the_correct_sunit.2Cswidth_values_for_optimal_performance Am considering either just going with defaults (XFS should pull from LVM I think) or doing something like sw=2,su=128K. However, maybe I should be doing sw=2,su=1920K? And perhaps my LVM stripe width should be adjusted? Thanks, Ray _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: sw and su for hardware RAID10 (w/ LVM) 2014-03-11 4:56 sw and su for hardware RAID10 (w/ LVM) Ray Van Dolson @ 2014-03-12 11:37 ` Stan Hoeppner 2014-03-13 14:23 ` Ray Van Dolson 0 siblings, 1 reply; 4+ messages in thread From: Stan Hoeppner @ 2014-03-12 11:37 UTC (permalink / raw) To: Ray Van Dolson, xfs On 3/10/2014 11:56 PM, Ray Van Dolson wrote: > RHEL6.x + XFS that comes w/ Red Hat's scalable file system add on. We > have two PowerVault MD3260e's each configured with a 30 disk RAID10 (15 > RAID groups) exposed to our server. Segment size is 128K (in Dell's > world I'm not sure if this means my stripe width is 128K*15?) 128KB must be the stripe unit. > Have set up a concatenated LVM volume on top of these two "virtual > disks" (with lvcreate -i 2). This is because you created a 2 stripe array, not a concatenation. > By default LVM says it's used a stripe width of 64K. > > # lvs -o path,size,stripes,stripe_size > Path LSize #Str Stripe > /dev/agsfac_vg00/lv00 100.00t 2 64.00k from lvcreate(8) -i, --stripes Stripes Gives the number of stripes... > Unsure if these defaults should be adjusted. > > I'm trying to figure out the appropriate sw/su values to use per: > > http://xfs.org/index.php/XFS_FAQ#Q:_How_to_calculate_the_correct_sunit.2Cswidth_values_for_optimal_performance > > Am considering either just going with defaults (XFS should pull from > LVM I think) or doing something like sw=2,su=128K. However, maybe I > should be doing sw=2,su=1920K? And perhaps my LVM stripe width should > be adjusted? Why don't you first tell us what you want? You say at the top that you created a concatenation, but at the bottom you say LVM stripe. So first tell us which one you actually want, because the XFS alignment is radically different for each. Then tell us why you must use LVM instead of md. md has fewer problems/limitations for stripes and concat than LVM, and is much easier to configure. -- Stan _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: sw and su for hardware RAID10 (w/ LVM) 2014-03-12 11:37 ` Stan Hoeppner @ 2014-03-13 14:23 ` Ray Van Dolson 2014-03-14 0:11 ` Stan Hoeppner 0 siblings, 1 reply; 4+ messages in thread From: Ray Van Dolson @ 2014-03-13 14:23 UTC (permalink / raw) To: Stan Hoeppner; +Cc: xfs On Wed, Mar 12, 2014 at 06:37:13AM -0500, Stan Hoeppner wrote: > On 3/10/2014 11:56 PM, Ray Van Dolson wrote: > > RHEL6.x + XFS that comes w/ Red Hat's scalable file system add on. We > > have two PowerVault MD3260e's each configured with a 30 disk RAID10 (15 > > RAID groups) exposed to our server. Segment size is 128K (in Dell's > > world I'm not sure if this means my stripe width is 128K*15?) > > 128KB must be the stripe unit. > > > Have set up a concatenated LVM volume on top of these two "virtual > > disks" (with lvcreate -i 2). > > This is because you created a 2 stripe array, not a concatenation. > > > By default LVM says it's used a stripe width of 64K. > > > > # lvs -o path,size,stripes,stripe_size > > Path LSize #Str Stripe > > /dev/agsfac_vg00/lv00 100.00t 2 64.00k > > from lvcreate(8) > > -i, --stripes Stripes > Gives the number of stripes... > > > Unsure if these defaults should be adjusted. > > > > I'm trying to figure out the appropriate sw/su values to use per: > > > > http://xfs.org/index.php/XFS_FAQ#Q:_How_to_calculate_the_correct_sunit.2Cswidth_values_for_optimal_performance > > > > Am considering either just going with defaults (XFS should pull from > > LVM I think) or doing something like sw=2,su=128K. However, maybe I > > should be doing sw=2,su=1920K? And perhaps my LVM stripe width should > > be adjusted? > > Why don't you first tell us what you want? You say at the top that you > created a concatenation, but at the bottom you say LVM stripe. So first > tell us which one you actually want, because the XFS alignment is > radically different for each. > > Then tell us why you must use LVM instead of md. md has fewer > problems/limitations for stripes and concat than LVM, and is much easier > to configure. Yes, misused the term concatenation. Striping is what I'm afer (want to use all of my LUNs equally). I don't know that I necessarily need to use LVM here. No need for snapshots, just after the best "performance" for multiple NAS sourced (via Samba) sequential write or read streams (but not read/write at the same time). My setup is as follows right now: MD3260_1 -> Disk Group 0 (RAID10 - 15 RG's, 128K segment size) -> 2 Virtual Disks (one per controller) MD3260_2 -> Disk Group 0 (RAID10 - 15 RG's, 128K segment size) -> 2 Virtual Disks (one per controller) So I see four equally sized LUNs on my RHEL box, each with one active path and one passive path (using Linux MPIO). I'll set up a striped md array across these four LUNs using a 128K chunk size. Things work pretty well with the xfs default, so may stick with that, but to try and get it as "right" as possible, I'm thinking I should be using a su=128k value, but am not sure on the sw value. It's either: - 4 (four LUNs as far as my OS is concerned) - 30 (15 RAID groups per MD3260) I'm thinking probably 4 is the right answer since the RAID groups on my PowerVaults are all abstracted. Ray _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: sw and su for hardware RAID10 (w/ LVM) 2014-03-13 14:23 ` Ray Van Dolson @ 2014-03-14 0:11 ` Stan Hoeppner 0 siblings, 0 replies; 4+ messages in thread From: Stan Hoeppner @ 2014-03-14 0:11 UTC (permalink / raw) To: Ray Van Dolson; +Cc: xfs On 3/13/2014 9:23 AM, Ray Van Dolson wrote: > On Wed, Mar 12, 2014 at 06:37:13AM -0500, Stan Hoeppner wrote: >> On 3/10/2014 11:56 PM, Ray Van Dolson wrote: >>> RHEL6.x + XFS that comes w/ Red Hat's scalable file system add on. We >>> have two PowerVault MD3260e's each configured with a 30 disk RAID10 (15 >>> RAID groups) exposed to our server. Segment size is 128K (in Dell's >>> world I'm not sure if this means my stripe width is 128K*15?) >> >> 128KB must be the stripe unit. >> >>> Have set up a concatenated LVM volume on top of these two "virtual >>> disks" (with lvcreate -i 2). >> >> This is because you created a 2 stripe array, not a concatenation. >> >>> By default LVM says it's used a stripe width of 64K. >>> >>> # lvs -o path,size,stripes,stripe_size >>> Path LSize #Str Stripe >>> /dev/agsfac_vg00/lv00 100.00t 2 64.00k >> >> from lvcreate(8) >> >> -i, --stripes Stripes >> Gives the number of stripes... >> >>> Unsure if these defaults should be adjusted. >>> >>> I'm trying to figure out the appropriate sw/su values to use per: >>> >>> http://xfs.org/index.php/XFS_FAQ#Q:_How_to_calculate_the_correct_sunit.2Cswidth_values_for_optimal_performance >>> >>> Am considering either just going with defaults (XFS should pull from >>> LVM I think) or doing something like sw=2,su=128K. However, maybe I >>> should be doing sw=2,su=1920K? And perhaps my LVM stripe width should >>> be adjusted? >> >> Why don't you first tell us what you want? You say at the top that you >> created a concatenation, but at the bottom you say LVM stripe. So first >> tell us which one you actually want, because the XFS alignment is >> radically different for each. >> >> Then tell us why you must use LVM instead of md. md has fewer >> problems/limitations for stripes and concat than LVM, and is much easier >> to configure. > > Yes, misused the term concatenation. Striping is what I'm afer (want > to use all of my LUNs equally). Striping does not guarantee equal distribution of data. > I don't know that I necessarily need to use LVM here. No need for > snapshots, just after the best "performance" for multiple NAS sourced > (via Samba) sequential write or read streams (but not read/write at the > same time). > > My setup is as follows right now: You need to figure out exactly what you have because none of what you state below makes any sense whatsoever. > MD3260_1 -> Disk Group 0 (RAID10 - 15 RG's, 128K segment size) -> 2 > Virtual Disks (one per controller) > MD3260_2 -> Disk Group 0 (RAID10 - 15 RG's, 128K segment size) -> 2 > Virtual Disks (one per controller) This doesn't tell us how many disks are in each RAID10. That is required information. It does not show the relationship between a "Virtual Disk" and a RAID10 array. > So I see four equally sized LUNs on my RHEL box, each with one active > path and one passive path (using Linux MPIO). This does not tell us the size of each LUN. That is required information. The number and size of the RAID10 arrays is also required. > I'll set up a striped md array across these four LUNs using a 128K > chunk size. This is wrong, no matter what the actual numbers are for everything up above, because the stripe unit of a nested stripe is always equal to the stripe width of the constituent arrays it stitches together. > Things work pretty well with the xfs default, so may stick with that, > but to try and get it as "right" as possible, I'm thinking I should be > using a su=128k value, but am not sure on the sw value. It's either: You only align XFS if you have an allocation heavy workload where the file sizes are smaller but very close to stripe width, or many times larger than stripe width. You *appear* to currently have a 3.84 MB stripe width. That is yet to be confirmed by you. > - 4 (four LUNs as far as my OS is concerned) My guess is that 2 of the 4 point to the same RAID10, both to the entire capacity, or to half the capacity. If you have 2 carvings of each RAID10 that are called "Virtual Disks" which are each mapped out a different controller as a distinct LUN, then this is completely wrong for what you want to accomplish. > - 30 (15 RAID groups per MD3260) So Dell calls the 15 RAID1 mirror of a RAID10 "RAID groups"? What does it call the RAID10 itself? > I'm thinking probably 4 is the right answer since the RAID groups on my > PowerVaults are all abstracted. If you are unable to figure out exactly how your RAIDs, VGs, and LUNs are configured, do not under any circumstances align XFS. All it will do is decrease performance. The default 4KB alignment is far, far better than misalignment. -- Stan _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2014-03-14 0:12 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2014-03-11 4:56 sw and su for hardware RAID10 (w/ LVM) Ray Van Dolson 2014-03-12 11:37 ` Stan Hoeppner 2014-03-13 14:23 ` Ray Van Dolson 2014-03-14 0:11 ` Stan Hoeppner
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.