xfsprogs 2.x vs 3.x logsize changed

xfsprogs 2.x vs 3.x logsize changed

[Łukasz Oleś]

Hi,

I'm upgrading xfsprogs from 2.10.1 to the latest 3.1.4 version. I noticed that 
when I'm creating large lvm volume (2T) the log size is almost 1G in the old 
version it was 128M.  
I know I can manipulate this value with -lsize option, but I'm wondering why 
this difference is so huge? 

On this volume I have one sparse file which is exported by iSCSI Target.  I 
have script which calculates  for me "seek" value for dd command and now it 
returns me wrong values.
Can I stay with the old log size or maybe there are some good reasons to use 
new values?

Regards,
Łukasz Oleś

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: xfsprogs 2.x vs 3.x logsize changed

[Michael Monnerie]

[-- Attachment #1.1: Type: Text/Plain, Size: 809 bytes --]

On Mittwoch, 17. November 2010 Łukasz Oleś wrote:
> Can I stay with the old log size or maybe there are some good reasons
> to use  new values?

I believe the new defaults are there because it results in faster 
metadata operations when there are lots of them. I'd guess if you only 
ever have one writer it wouldn't matter much, but in case you have 
several parallel writers it could make a difference. But I'm not a dev, 
maybe they can confirm or clarify.

-- 
mit freundlichen Grüssen,
Michael Monnerie, Ing. BSc

it-management Internet Services: Protéger
http://proteger.at [gesprochen: Prot-e-schee]
Tel: +43 660 / 415 6531

// ****** Radiointerview zum Thema Spam ******
// http://www.it-podcast.at/archiv.html#podcast-100716
// 
// Haus zu verkaufen: http://zmi.at/langegg/

[-- Attachment #1.2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: xfsprogs 2.x vs 3.x logsize changed

[Łukasz Oleś]

On Wednesday 17 of November 2010 18:38:29 Eric Sandeen wrote:
> On 11/17/10 8:49 AM, Łukasz Oleś wrote:
> > Hi,
> > 
> > I'm upgrading xfsprogs from 2.10.1 to the latest 3.1.4 version. I noticed
> > that when I'm creating large lvm volume (2T) the log size is almost 1G
> > in the old version it was 128M.
> > I know I can manipulate this value with -lsize option, but I'm wondering
> > why this difference is so huge?
> > 
> > On this volume I have one sparse file which is exported by iSCSI Target. 
> > I have script which calculates  for me "seek" value for dd command and
> > now it returns me wrong values.
> > Can I stay with the old log size or maybe there are some good reasons to
> > use new values?
> > 
> > Regards,
> > Łukasz Oleś
> 
> commit a6634fba3dec4a92f0a2c4e30c80b634c0576ad5
> Author: David Chinner <dgc@sgi.com>
> Date:   Sun May 10 21:31:22 2009 +0200
> 
>     mkfs: allow to make larger logs
> 
>     Increase the maximum log size supported by mkfs.
> 
>     The log size can be increased easily in mkfs by changing a few
>     defines and a couple of types to allow the log size to increase to
>     just under 2GB. Theoretically, the log size can be made much, much
>     larger than this (up to 2^32 sectors), but going beyond 2^31
>     *bytes* causes integer overflow issues in the kernel log code.
> 
>     We use a maximum size of just under 2GB as exactly 2GB will cause
>     overflow issues on the first transaction reservation.
> 
>     Maximum log size is now reached at a fs size of ~4TB.
> 
>     Signed-off-by: Dave Chinner <dgc@sgi.com>
>     Signed-off-by: Christoph Hellwig <hch@lst.de>
> 
> 
> did:
> 
> @@ -1737,10 +1737,10 @@ _("size %s specified for log subvolume is too
> large, maximum is %lld blocks\n"), logblocks = 0;
>         } else if (loginternal && !logsize) {
>                 /*
> -                * logblocks grows from min_logblocks to XFS_MAX_LOG_BLOCKS
> -                * at 128GB
> -                *
> -                * 2048 = 128GB / MAX_LOG_BYTES
> +                * With a 2GB max log size, default to maximum size
> +                * at 4TB. This keeps the same ratio from the older
> +                * max log size of 128M at 256GB fs size. IOWs,
> +                * the ratio of fs size to log size is 2048:1.
>                  */
>                 logblocks = (dblocks << blocklog) / 2048;
>                 logblocks = logblocks >> blocklog;
> 
> old MAX was:
> 
> -#define XFS_MAX_LOG_BYTES      (128 * 1024 * 1024)
> 
> In your case ... I'm not sure a larger log is going to help, I think
> keeping it smaller is fine if you wish.
> 
> Dave may have a different opinion though :)
> 
> -Eric

Ok, so it's this commit http://git.kernel.org/?p=fs/xfs/xfsprogs-
dev.git;a=commitdiff;h=a6634fba3dec4a92f0a2c4e30c80b634c0576ad5

Would it be safe if I just revert this commit?  It looks safe for me, but 
maybe other things depends on it and I will break something.

Regards,
Łukasz Oleś

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: xfsprogs 2.x vs 3.x logsize changed

[Eric Sandeen]

On 11/18/10 3:49 AM, Łukasz Oleś wrote:

> Ok, so it's this commit http://git.kernel.org/?p=fs/xfs/xfsprogs-
> dev.git;a=commitdiff;h=a6634fba3dec4a92f0a2c4e30c80b634c0576ad5
> 
> Would it be safe if I just revert this commit?  It looks safe for me, but 
> maybe other things depends on it and I will break something.

I would just specify a smaller log on the commandline if that
is what you prefer, rather than changing the code.

-Eric
 
> Regards,
> Łukasz Oleś
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: xfsprogs 2.x vs 3.x logsize changed

[Dave Chinner]

On Wed, Nov 17, 2010 at 03:49:04PM +0100, Łukasz Oleś wrote:
> Hi,
> 
> I'm upgrading xfsprogs from 2.10.1 to the latest 3.1.4 version. I noticed that 
> when I'm creating large lvm volume (2T) the log size is almost 1G in the old 
> version it was 128M.  
> I know I can manipulate this value with -lsize option, but I'm wondering why 
> this difference is so huge? 

Many workloads were demonstrated to have substantially better
performance with larger logs, even on small filesystems. At 4TB,
most people are using RAID of some kind, so larger logs are quite
beneficial here.

> On this volume I have one sparse file which is exported by iSCSI Target.  I 
> have script which calculates  for me "seek" value for dd command and now it 
> returns me wrong values.
> Can I stay with the old log size or maybe there are some good reasons to use 
> new values?

Staying with the old log size is just fine - it'll behave exactly
the same as it does now. There are two main things hat make a larger
log size attractive:

	1. log size determines maximum transaction parallelism, so
	   so smaller logs may limit operational concurrency. A
	   128MB log typically allows ~250 concurrent transactions
	   on a 1TB, 4k block size filesystem.
	2. larger logs allow the filesystem to soak up larger burst
	   of metadata modifications without needing to write back
	   dirty metadata.

The downside ot a larger log is that recovery can take longer after
a crash.

Anyway, if you are having no problems at 128MB, then just use
that...

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs