[PATCH 0/3] xfs: remaining 3.11 merge patches

[PATCH 0/3] xfs: remaining 3.11 merge patches

[Dave Chinner]

Hi Ben,

I figured it was easiest for you to just repost all three remainng
patches for the 3.11 merge window that I have in one set. The dquot
log res patch has been updated to address Chandra's review.

Cheers,

Dave.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 1/3] xfs: update mount options documentation

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Because it's horribly out of date.

And mark various deprecated options as deprecated and give them a
removal date.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Mark Tinguely <tinguely@sgi.com>
---
 Documentation/filesystems/xfs.txt | 317 +++++++++++++++++++++++++-------------
 1 file changed, 209 insertions(+), 108 deletions(-)

diff --git a/Documentation/filesystems/xfs.txt b/Documentation/filesystems/xfs.txt
index 83577f0..12525b1 100644
--- a/Documentation/filesystems/xfs.txt
+++ b/Documentation/filesystems/xfs.txt
@@ -18,6 +18,8 @@ Mount Options
 =============
 
 When mounting an XFS filesystem, the following options are accepted.
+For boolean mount options, the names with the (*) suffix is the
+default behaviour.
 
   allocsize=size
 	Sets the buffered I/O end-of-file preallocation size when
@@ -25,97 +27,128 @@ When mounting an XFS filesystem, the following options are accepted.
 	Valid values for this option are page size (typically 4KiB)
 	through to 1GiB, inclusive, in power-of-2 increments.
 
-  attr2/noattr2
-	The options enable/disable (default is disabled for backward
-	compatibility on-disk) an "opportunistic" improvement to be
-	made in the way inline extended attributes are stored on-disk.
-	When the new form is used for the first time (by setting or
-	removing extended attributes) the on-disk superblock feature
-	bit field will be updated to reflect this format being in use.
+	The default behaviour is for dynamic end-of-file
+	preallocation size, which uses a set of heuristics to
+	optimise the preallocation size based on the current
+	allocation patterns within the file and the access patterns
+	to the file. Specifying a fixed allocsize value turns off
+	the dynamic behaviour.
+
+  attr2
+  noattr2
+	The options enable/disable an "opportunistic" improvement to
+	be made in the way inline extended attributes are stored
+	on-disk.  When the new form is used for the first time when
+	attr2 is selected (either when setting or removing extended
+	attributes) the on-disk superblock feature bit field will be
+	updated to reflect this format being in use.
+
+	The default behaviour is determined by the on-disk feature
+	bit indicating that attr2 behaviour is active. If either
+	mount option it set, then that becomes the new default used
+	by the filesystem.
 
 	CRC enabled filesystems always use the attr2 format, and so
 	will reject the noattr2 mount option if it is set.
 
-  barrier
-	Enables the use of block layer write barriers for writes into
-	the journal and unwritten extent conversion.  This allows for
-	drive level write caching to be enabled, for devices that
-	support write barriers.
+  barrier (*)
+  nobarrier
+	Enables/disables the use of block layer write barriers for
+	writes into the journal and for data integrity operations.
+	This allows for drive level write caching to be enabled, for
+	devices that support write barriers.
 
   discard
-	Issue command to let the block device reclaim space freed by the
-	filesystem.  This is useful for SSD devices, thinly provisioned
-	LUNs and virtual machine images, but may have a performance
-	impact.
-
-  dmapi
-	Enable the DMAPI (Data Management API) event callouts.
-	Use with the "mtpt" option.
-
-  grpid/bsdgroups and nogrpid/sysvgroups
-	These options define what group ID a newly created file gets.
-	When grpid is set, it takes the group ID of the directory in
-	which it is created; otherwise (the default) it takes the fsgid
-	of the current process, unless the directory has the setgid bit
-	set, in which case it takes the gid from the parent directory,
-	and also gets the setgid bit set if it is a directory itself.
-
-  ihashsize=value
-	In memory inode hashes have been removed, so this option has
-	no function as of August 2007. Option is deprecated.
-
-  ikeep/noikeep
-	When ikeep is specified, XFS does not delete empty inode clusters
-	and keeps them around on disk. ikeep is the traditional XFS
-	behaviour. When noikeep is specified, empty inode clusters
-	are returned to the free space pool. The default is noikeep for
-	non-DMAPI mounts, while ikeep is the default when DMAPI is in use.
-
-  inode64
-	Indicates that XFS is allowed to create inodes at any location
-	in the filesystem, including those which will result in inode
-	numbers occupying more than 32 bits of significance.  This is
-	the default allocation option. Applications which do not handle
-	inode numbers bigger than 32 bits, should use inode32 option.
+  nodiscard (*)
+	Enable/disable the issuing of commands to let the block
+	device reclaim space freed by the filesystem.  This is
+	useful for SSD devices, thinly provisioned LUNs and virtual
+	machine images, but may have a performance impact.
+
+	Note: It is currently recommended that you use the fstrim
+	application to discard unused blocks rather than the discard
+	mount option because the performance impact of this option
+	is quite severe.
+
+  grpid/bsdgroups
+  nogrpid/sysvgroups (*)
+	These options define what group ID a newly created file
+	gets.  When grpid is set, it takes the group ID of the
+	directory in which it is created; otherwise it takes the
+	fsgid of the current process, unless the directory has the
+	setgid bit set, in which case it takes the gid from the
+	parent directory, and also gets the setgid bit set if it is
+	a directory itself.
+
+  filestreams
+	Make the data allocator use the filestreams allocation mode
+	across the entire filesystem rather than just on directories
+	configured to use it.
+
+  ikeep
+  noikeep (*)
+	When ikeep is specified, XFS does not delete empty inode
+	clusters and keeps them around on disk.  When noikeep is
+	specified, empty inode clusters are returned to the free
+	space pool.
 
   inode32
-	Indicates that XFS is limited to create inodes at locations which
-	will not result in inode numbers with more than 32 bits of
-	significance. This is provided for backwards compatibility, since
-	64 bits inode numbers might cause problems for some applications
-	that cannot handle large inode numbers.
-
-  largeio/nolargeio
+  inode64 (*)
+	When inode32 is specified, it indicates that XFS limits
+	inode creation to locations which will not result in inode
+	numbers with more than 32 bits of significance.
+
+	When inode64 is specified, it indicates that XFS is allowed
+	to create inodes at any location in the filesystem,
+	including those which will result in inode numbers occupying
+	more than 32 bits of significance. 
+
+	inode32 is provided for backwards compatibility with older
+	systems and applications, since 64 bits inode numbers might
+	cause problems for some applications that cannot handle
+	large inode numbers.  If applications are in use which do
+	not handle inode numbers bigger than 32 bits, the inode32
+	option should be specified.
+
+
+  largeio
+  nolargeio (*)
 	If "nolargeio" is specified, the optimal I/O reported in
-	st_blksize by stat(2) will be as small as possible to allow user
-	applications to avoid inefficient read/modify/write I/O.
-	If "largeio" specified, a filesystem that has a "swidth" specified
-	will return the "swidth" value (in bytes) in st_blksize. If the
-	filesystem does not have a "swidth" specified but does specify
-	an "allocsize" then "allocsize" (in bytes) will be returned
-	instead.
-	If neither of these two options are specified, then filesystem
-	will behave as if "nolargeio" was specified.
+	st_blksize by stat(2) will be as small as possible to allow
+	user applications to avoid inefficient read/modify/write
+	I/O.  This is typically the page size of the machine, as
+	this is the granularity of the page cache.
+
+	If "largeio" specified, a filesystem that was created with a
+	"swidth" specified will return the "swidth" value (in bytes)
+	in st_blksize. If the filesystem does not have a "swidth"
+	specified but does specify an "allocsize" then "allocsize"
+	(in bytes) will be returned instead. Otherwise the behaviour
+	is the same as if "nolargeio" was specified.
 
   logbufs=value
-	Set the number of in-memory log buffers.  Valid numbers range
-	from 2-8 inclusive.
-	The default value is 8 buffers for filesystems with a
-	blocksize of 64KiB, 4 buffers for filesystems with a blocksize
-	of 32KiB, 3 buffers for filesystems with a blocksize of 16KiB
-	and 2 buffers for all other configurations.  Increasing the
-	number of buffers may increase performance on some workloads
-	at the cost of the memory used for the additional log buffers
-	and their associated control structures.
+	Set the number of in-memory log buffers.  Valid numbers
+	range from 2-8 inclusive.
+
+	The default value is 8 buffers.
+
+	If the memory cost of 8 log buffers is too high on small
+	systems, then it may be reduced at some cost to performance
+	on metadata intensive workloads. The logbsize option below
+	controls the size of each buffer and so is also relevent to
+	this case.
 
   logbsize=value
-	Set the size of each in-memory log buffer.
-	Size may be specified in bytes, or in kilobytes with a "k" suffix.
-	Valid sizes for version 1 and version 2 logs are 16384 (16k) and
-	32768 (32k).  Valid sizes for version 2 logs also include
-	65536 (64k), 131072 (128k) and 262144 (256k).
-	The default value for machines with more than 32MiB of memory
-	is 32768, machines with less memory use 16384 by default.
+	Set the size of each in-memory log buffer.  The size may be
+	specified in bytes, or in kilobytes with a "k" suffix.
+	Valid sizes for version 1 and version 2 logs are 16384 (16k)
+	and 32768 (32k).  Valid sizes for version 2 logs also
+	include 65536 (64k), 131072 (128k) and 262144 (256k). The
+	logbsize must be an integer multiple of the log
+	stripe unit configured at mkfs time.
+
+	The default value for for version 1 logs is 32768, while the
+	default value for version 2 logs is MAX(32768, log_sunit).
 
   logdev=device and rtdev=device
 	Use an external log (metadata journal) and/or real-time device.
@@ -124,16 +157,11 @@ When mounting an XFS filesystem, the following options are accepted.
 	optional, and the log section can be separate from the data
 	section or contained within it.
 
-  mtpt=mountpoint
-	Use with the "dmapi" option.  The value specified here will be
-	included in the DMAPI mount event, and should be the path of
-	the actual mountpoint that is used.
-
   noalign
-	Data allocations will not be aligned at stripe unit boundaries.
-
-  noatime
-	Access timestamps are not updated when a file is read.
+	Data allocations will not be aligned at stripe unit
+	boundaries. This is only relevant to filesystems created
+	with non-zero data alignment parameters (sunit, swidth) by
+	mkfs.
 
   norecovery
 	The filesystem will be mounted without running log recovery.
@@ -144,8 +172,14 @@ When mounting an XFS filesystem, the following options are accepted.
 	the mount will fail.
 
   nouuid
-	Don't check for double mounted file systems using the file system uuid.
-	This is useful to mount LVM snapshot volumes.
+	Don't check for double mounted file systems using the file
+	system uuid.  This is useful to mount LVM snapshot volumes,
+	and often used in combination with "norecovery" for mounting
+	read-only snapshots.
+
+  noquota
+	Forcibly turns off all quota accounting and enforcement
+	within the filesystem.
 
   uquota/usrquota/uqnoenforce/quota
 	User disk quota accounting enabled, and limits (optionally)
@@ -160,24 +194,64 @@ When mounting an XFS filesystem, the following options are accepted.
 	enforced.  Refer to xfs_quota(8) for further details.
 
   sunit=value and swidth=value
-	Used to specify the stripe unit and width for a RAID device or
-	a stripe volume.  "value" must be specified in 512-byte block
-	units.
-	If this option is not specified and the filesystem was made on
-	a stripe volume or the stripe width or unit were specified for
-	the RAID device at mkfs time, then the mount system call will
-	restore the value from the superblock.  For filesystems that
-	are made directly on RAID devices, these options can be used
-	to override the information in the superblock if the underlying
-	disk layout changes after the filesystem has been created.
-	The "swidth" option is required if the "sunit" option has been
-	specified, and must be a multiple of the "sunit" value.
+	Used to specify the stripe unit and width for a RAID device
+	or a stripe volume.  "value" must be specified in 512-byte
+	block units. These options are only relevant to filesystems
+	that were created with non-zero data alignment parameters.
+
+	The sunit and swidth parameters specified must be compatible
+	with the existing filesystem alignment characteristics.  In
+	general, that means the only valid changes to sunit are
+	increasing it by a power-of-2 multiple. Valid swidth values
+	are any integer multiple of a valid sunit value.
+
+	Typically the only time these mount options are necessary if
+	after an underlying RAID device has had it's geometry
+	modified, such as adding a new disk to a RAID5 lun and
+	reshaping it.
 
   swalloc
 	Data allocations will be rounded up to stripe width boundaries
 	when the current end of file is being extended and the file
 	size is larger than the stripe width size.
 
+  wsync
+	When specified, all filesystem namespace operations are
+	executed synchronously. This ensures that when the namespace
+	operation (create, unlink, etc) completes, the change to the
+	namespace is on stable storage. This is useful in HA setups
+	where failover must not result in clients seeing
+	inconsistent namespace presentation during or after a
+	failover event.
+
+
+Deprecated Mount Options
+========================
+
+  delaylog/nodelaylog
+	Delayed logging is the only logging method that XFS supports
+	now, so these mount options are now ignored.
+
+	Due for removal in 3.12.
+
+  ihashsize=value
+	In memory inode hashes have been removed, so this option has
+	no function as of August 2007. Option is deprecated.
+
+	Due for removal in 3.12.
+
+  irixsgid
+	This behaviour is now controlled by a sysctl, so the mount
+	option is ignored.
+
+	Due for removal in 3.12.
+
+  osyncisdsync
+  osyncisosync
+	O_SYNC and O_DSYNC are fully supported, so there is no need
+	for these options any more.
+
+	Due for removal in 3.12.
 
 sysctls
 =======
@@ -189,15 +263,20 @@ The following sysctls are available for the XFS filesystem:
 	in /proc/fs/xfs/stat.  It then immediately resets to "0".
 
   fs.xfs.xfssyncd_centisecs	(Min: 100  Default: 3000  Max: 720000)
-  	The interval at which the xfssyncd thread flushes metadata
-  	out to disk.  This thread will flush log activity out, and
-  	do some processing on unlinked inodes.
+	The interval at which the filesystem flushes metadata
+	out to disk and runs internal cache cleanup routines.
 
-  fs.xfs.xfsbufd_centisecs	(Min: 50  Default: 100	Max: 3000)
-	The interval at which xfsbufd scans the dirty metadata buffers list.
+  fs.xfs.filestream_centisecs	(Min: 1  Default: 3000  Max: 360000)
+	The interval at which the filesystem ages filestreams cache
+	references and returns timed-out AGs back to the free stream
+	pool.
 
-  fs.xfs.age_buffer_centisecs	(Min: 100  Default: 1500  Max: 720000)
-	The age at which xfsbufd flushes dirty metadata buffers to disk.
+  fs.xfs.speculative_prealloc_lifetime
+		(Units: seconds   Min: 1  Default: 300  Max: 86400)
+	The interval at which the background scanning for inodes
+	with unused speculative preallocation runs. The scan
+	removes unused preallocation from clean inodes and releases
+	the unused space back to the free pool.
 
   fs.xfs.error_level		(Min: 0  Default: 3  Max: 11)
 	A volume knob for error reporting when internal errors occur.
@@ -254,9 +333,31 @@ The following sysctls are available for the XFS filesystem:
 	by the xfs_io(8) chattr command on a directory to be
 	inherited by files in that directory.
 
+  fs.xfs.inherit_nodefrag	(Min: 0  Default: 1  Max: 1)
+	Setting this to "1" will cause the "nodefrag" flag set
+	by the xfs_io(8) chattr command on a directory to be
+	inherited by files in that directory.
+
   fs.xfs.rotorstep		(Min: 1  Default: 1  Max: 256)
 	In "inode32" allocation mode, this option determines how many
 	files the allocator attempts to allocate in the same allocation
 	group before moving to the next allocation group.  The intent
 	is to control the rate at which the allocator moves between
 	allocation groups when allocating extents for new files.
+
+Deprecated Sysctls
+==================
+
+  fs.xfs.xfsbufd_centisecs	(Min: 50  Default: 100	Max: 3000)
+	Dirty metadata is now tracked by the log subsystem and
+	flushing is driven by log space and idling demands. The
+	xfsbufd no longer exists, so this syctl does nothing.
+
+	Due for removal in 3.14.
+
+  fs.xfs.age_buffer_centisecs	(Min: 100  Default: 1500  Max: 720000)
+	Dirty metadata is now tracked by the log subsystem and
+	flushing is driven by log space and idling demands. The
+	xfsbufd no longer exists, so this syctl does nothing.
+
+	Due for removal in 3.14.
-- 
1.8.3.2

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 1/3] xfs: update mount options documentation

[Ben Myers]

On Wed, Jul 10, 2013 at 07:03:59AM +1000, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
> 
> Because it's horribly out of date.
> 
> And mark various deprecated options as deprecated and give them a
> removal date.
> 
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> Reviewed-by: Mark Tinguely <tinguely@sgi.com>
> ---
>  Documentation/filesystems/xfs.txt | 317 +++++++++++++++++++++++++-------------
>  1 file changed, 209 insertions(+), 108 deletions(-)
> 
> diff --git a/Documentation/filesystems/xfs.txt b/Documentation/filesystems/xfs.txt
> index 83577f0..12525b1 100644
> --- a/Documentation/filesystems/xfs.txt
> +++ b/Documentation/filesystems/xfs.txt
> @@ -18,6 +18,8 @@ Mount Options
>  =============
>  
>  When mounting an XFS filesystem, the following options are accepted.
> +For boolean mount options, the names with the (*) suffix is the
> +default behaviour.
>  
>    allocsize=size
>  	Sets the buffered I/O end-of-file preallocation size when
> @@ -25,97 +27,128 @@ When mounting an XFS filesystem, the following options are accepted.
>  	Valid values for this option are page size (typically 4KiB)
>  	through to 1GiB, inclusive, in power-of-2 increments.
>  
> -  attr2/noattr2
> -	The options enable/disable (default is disabled for backward
> -	compatibility on-disk) an "opportunistic" improvement to be
> -	made in the way inline extended attributes are stored on-disk.
> -	When the new form is used for the first time (by setting or
> -	removing extended attributes) the on-disk superblock feature
> -	bit field will be updated to reflect this format being in use.
> +	The default behaviour is for dynamic end-of-file
> +	preallocation size, which uses a set of heuristics to
> +	optimise the preallocation size based on the current
> +	allocation patterns within the file and the access patterns
> +	to the file. Specifying a fixed allocsize value turns off
> +	the dynamic behaviour.
> +
> +  attr2
> +  noattr2
> +	The options enable/disable an "opportunistic" improvement to
> +	be made in the way inline extended attributes are stored
> +	on-disk.  When the new form is used for the first time when
> +	attr2 is selected (either when setting or removing extended
> +	attributes) the on-disk superblock feature bit field will be
> +	updated to reflect this format being in use.
> +
> +	The default behaviour is determined by the on-disk feature
> +	bit indicating that attr2 behaviour is active. If either
> +	mount option it set, then that becomes the new default used
> +	by the filesystem.
>  
>  	CRC enabled filesystems always use the attr2 format, and so
>  	will reject the noattr2 mount option if it is set.
>  
> -  barrier
> -	Enables the use of block layer write barriers for writes into
> -	the journal and unwritten extent conversion.  This allows for
> -	drive level write caching to be enabled, for devices that
> -	support write barriers.
> +  barrier (*)
> +  nobarrier
> +	Enables/disables the use of block layer write barriers for
> +	writes into the journal and for data integrity operations.
> +	This allows for drive level write caching to be enabled, for
> +	devices that support write barriers.
>  
>    discard
> -	Issue command to let the block device reclaim space freed by the
> -	filesystem.  This is useful for SSD devices, thinly provisioned
> -	LUNs and virtual machine images, but may have a performance
> -	impact.
> -
> -  dmapi
> -	Enable the DMAPI (Data Management API) event callouts.
> -	Use with the "mtpt" option.
> -
> -  grpid/bsdgroups and nogrpid/sysvgroups
> -	These options define what group ID a newly created file gets.
> -	When grpid is set, it takes the group ID of the directory in
> -	which it is created; otherwise (the default) it takes the fsgid
> -	of the current process, unless the directory has the setgid bit
> -	set, in which case it takes the gid from the parent directory,
> -	and also gets the setgid bit set if it is a directory itself.
> -
> -  ihashsize=value
> -	In memory inode hashes have been removed, so this option has
> -	no function as of August 2007. Option is deprecated.
> -
> -  ikeep/noikeep
> -	When ikeep is specified, XFS does not delete empty inode clusters
> -	and keeps them around on disk. ikeep is the traditional XFS
> -	behaviour. When noikeep is specified, empty inode clusters
> -	are returned to the free space pool. The default is noikeep for
> -	non-DMAPI mounts, while ikeep is the default when DMAPI is in use.
> -
> -  inode64
> -	Indicates that XFS is allowed to create inodes at any location
> -	in the filesystem, including those which will result in inode
> -	numbers occupying more than 32 bits of significance.  This is
> -	the default allocation option. Applications which do not handle
> -	inode numbers bigger than 32 bits, should use inode32 option.
> +  nodiscard (*)
> +	Enable/disable the issuing of commands to let the block
> +	device reclaim space freed by the filesystem.  This is
> +	useful for SSD devices, thinly provisioned LUNs and virtual
> +	machine images, but may have a performance impact.
> +
> +	Note: It is currently recommended that you use the fstrim
> +	application to discard unused blocks rather than the discard
> +	mount option because the performance impact of this option
> +	is quite severe.
> +
> +  grpid/bsdgroups
> +  nogrpid/sysvgroups (*)
> +	These options define what group ID a newly created file
> +	gets.  When grpid is set, it takes the group ID of the
> +	directory in which it is created; otherwise it takes the
> +	fsgid of the current process, unless the directory has the
> +	setgid bit set, in which case it takes the gid from the
> +	parent directory, and also gets the setgid bit set if it is
> +	a directory itself.
> +
> +  filestreams
> +	Make the data allocator use the filestreams allocation mode
> +	across the entire filesystem rather than just on directories
> +	configured to use it.
> +
> +  ikeep
> +  noikeep (*)
> +	When ikeep is specified, XFS does not delete empty inode
> +	clusters and keeps them around on disk.  When noikeep is
> +	specified, empty inode clusters are returned to the free
> +	space pool.
>  
>    inode32
> -	Indicates that XFS is limited to create inodes at locations which
> -	will not result in inode numbers with more than 32 bits of
> -	significance. This is provided for backwards compatibility, since
> -	64 bits inode numbers might cause problems for some applications
> -	that cannot handle large inode numbers.
> -
> -  largeio/nolargeio
> +  inode64 (*)
> +	When inode32 is specified, it indicates that XFS limits
> +	inode creation to locations which will not result in inode
> +	numbers with more than 32 bits of significance.
> +
> +	When inode64 is specified, it indicates that XFS is allowed
> +	to create inodes at any location in the filesystem,
> +	including those which will result in inode numbers occupying
> +	more than 32 bits of significance. 
> +
> +	inode32 is provided for backwards compatibility with older
> +	systems and applications, since 64 bits inode numbers might
> +	cause problems for some applications that cannot handle
> +	large inode numbers.  If applications are in use which do
> +	not handle inode numbers bigger than 32 bits, the inode32
> +	option should be specified.
> +
> +
> +  largeio
> +  nolargeio (*)
>  	If "nolargeio" is specified, the optimal I/O reported in
> -	st_blksize by stat(2) will be as small as possible to allow user
> -	applications to avoid inefficient read/modify/write I/O.
> -	If "largeio" specified, a filesystem that has a "swidth" specified
> -	will return the "swidth" value (in bytes) in st_blksize. If the
> -	filesystem does not have a "swidth" specified but does specify
> -	an "allocsize" then "allocsize" (in bytes) will be returned
> -	instead.
> -	If neither of these two options are specified, then filesystem
> -	will behave as if "nolargeio" was specified.
> +	st_blksize by stat(2) will be as small as possible to allow
> +	user applications to avoid inefficient read/modify/write
> +	I/O.  This is typically the page size of the machine, as
> +	this is the granularity of the page cache.
> +
> +	If "largeio" specified, a filesystem that was created with a
> +	"swidth" specified will return the "swidth" value (in bytes)
> +	in st_blksize. If the filesystem does not have a "swidth"
> +	specified but does specify an "allocsize" then "allocsize"
> +	(in bytes) will be returned instead. Otherwise the behaviour
> +	is the same as if "nolargeio" was specified.
>  
>    logbufs=value
> -	Set the number of in-memory log buffers.  Valid numbers range
> -	from 2-8 inclusive.
> -	The default value is 8 buffers for filesystems with a
> -	blocksize of 64KiB, 4 buffers for filesystems with a blocksize
> -	of 32KiB, 3 buffers for filesystems with a blocksize of 16KiB
> -	and 2 buffers for all other configurations.  Increasing the
> -	number of buffers may increase performance on some workloads
> -	at the cost of the memory used for the additional log buffers
> -	and their associated control structures.
> +	Set the number of in-memory log buffers.  Valid numbers
> +	range from 2-8 inclusive.
> +
> +	The default value is 8 buffers.
> +
> +	If the memory cost of 8 log buffers is too high on small
> +	systems, then it may be reduced at some cost to performance
> +	on metadata intensive workloads. The logbsize option below
> +	controls the size of each buffer and so is also relevent to
> +	this case.
>  
>    logbsize=value
> -	Set the size of each in-memory log buffer.
> -	Size may be specified in bytes, or in kilobytes with a "k" suffix.
> -	Valid sizes for version 1 and version 2 logs are 16384 (16k) and
> -	32768 (32k).  Valid sizes for version 2 logs also include
> -	65536 (64k), 131072 (128k) and 262144 (256k).
> -	The default value for machines with more than 32MiB of memory
> -	is 32768, machines with less memory use 16384 by default.
> +	Set the size of each in-memory log buffer.  The size may be
> +	specified in bytes, or in kilobytes with a "k" suffix.
> +	Valid sizes for version 1 and version 2 logs are 16384 (16k)
> +	and 32768 (32k).  Valid sizes for version 2 logs also
> +	include 65536 (64k), 131072 (128k) and 262144 (256k). The
> +	logbsize must be an integer multiple of the log
> +	stripe unit configured at mkfs time.
> +
> +	The default value for for version 1 logs is 32768, while the
> +	default value for version 2 logs is MAX(32768, log_sunit).
>  
>    logdev=device and rtdev=device
>  	Use an external log (metadata journal) and/or real-time device.
> @@ -124,16 +157,11 @@ When mounting an XFS filesystem, the following options are accepted.
>  	optional, and the log section can be separate from the data
>  	section or contained within it.
>  
> -  mtpt=mountpoint
> -	Use with the "dmapi" option.  The value specified here will be
> -	included in the DMAPI mount event, and should be the path of
> -	the actual mountpoint that is used.
> -
>    noalign
> -	Data allocations will not be aligned at stripe unit boundaries.
> -
> -  noatime
> -	Access timestamps are not updated when a file is read.
> +	Data allocations will not be aligned at stripe unit
> +	boundaries. This is only relevant to filesystems created
> +	with non-zero data alignment parameters (sunit, swidth) by
> +	mkfs.
>  
>    norecovery
>  	The filesystem will be mounted without running log recovery.
> @@ -144,8 +172,14 @@ When mounting an XFS filesystem, the following options are accepted.
>  	the mount will fail.
>  
>    nouuid
> -	Don't check for double mounted file systems using the file system uuid.
> -	This is useful to mount LVM snapshot volumes.
> +	Don't check for double mounted file systems using the file
> +	system uuid.  This is useful to mount LVM snapshot volumes,
> +	and often used in combination with "norecovery" for mounting
> +	read-only snapshots.
> +
> +  noquota
> +	Forcibly turns off all quota accounting and enforcement
> +	within the filesystem.
>  
>    uquota/usrquota/uqnoenforce/quota
>  	User disk quota accounting enabled, and limits (optionally)
> @@ -160,24 +194,64 @@ When mounting an XFS filesystem, the following options are accepted.
>  	enforced.  Refer to xfs_quota(8) for further details.
>  
>    sunit=value and swidth=value
> -	Used to specify the stripe unit and width for a RAID device or
> -	a stripe volume.  "value" must be specified in 512-byte block
> -	units.
> -	If this option is not specified and the filesystem was made on
> -	a stripe volume or the stripe width or unit were specified for
> -	the RAID device at mkfs time, then the mount system call will
> -	restore the value from the superblock.  For filesystems that
> -	are made directly on RAID devices, these options can be used
> -	to override the information in the superblock if the underlying
> -	disk layout changes after the filesystem has been created.
> -	The "swidth" option is required if the "sunit" option has been
> -	specified, and must be a multiple of the "sunit" value.
> +	Used to specify the stripe unit and width for a RAID device
> +	or a stripe volume.  "value" must be specified in 512-byte
> +	block units. These options are only relevant to filesystems
> +	that were created with non-zero data alignment parameters.
> +
> +	The sunit and swidth parameters specified must be compatible
> +	with the existing filesystem alignment characteristics.  In
> +	general, that means the only valid changes to sunit are
> +	increasing it by a power-of-2 multiple. Valid swidth values
> +	are any integer multiple of a valid sunit value.
> +
> +	Typically the only time these mount options are necessary if
> +	after an underlying RAID device has had it's geometry
> +	modified, such as adding a new disk to a RAID5 lun and
> +	reshaping it.
>  
>    swalloc
>  	Data allocations will be rounded up to stripe width boundaries
>  	when the current end of file is being extended and the file
>  	size is larger than the stripe width size.
>  
> +  wsync
> +	When specified, all filesystem namespace operations are
> +	executed synchronously. This ensures that when the namespace
> +	operation (create, unlink, etc) completes, the change to the
> +	namespace is on stable storage. This is useful in HA setups
> +	where failover must not result in clients seeing
> +	inconsistent namespace presentation during or after a
> +	failover event.
> +
> +
> +Deprecated Mount Options
> +========================
> +
> +  delaylog/nodelaylog
> +	Delayed logging is the only logging method that XFS supports
> +	now, so these mount options are now ignored.
> +
> +	Due for removal in 3.12.
> +
> +  ihashsize=value
> +	In memory inode hashes have been removed, so this option has
> +	no function as of August 2007. Option is deprecated.
> +
> +	Due for removal in 3.12.
> +
> +  irixsgid
> +	This behaviour is now controlled by a sysctl, so the mount
> +	option is ignored.
> +
> +	Due for removal in 3.12.
> +
> +  osyncisdsync
> +  osyncisosync
> +	O_SYNC and O_DSYNC are fully supported, so there is no need
> +	for these options any more.
> +
> +	Due for removal in 3.12.

I finally read through Documentation/ABI/README, and I agree this all seems
fairly reasonable with respect to that doc.

I do agree that it would be good to add these into the
Documentation/ABI/obsolete directory.  The only other concern that I have is
that the doc is saying that they want two years notice.  3.12 might be a little
short for that timeframe.  Anyway we can cross that bridge when we come to it.

Applied.

-Ben

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 1/3] xfs: update mount options documentation

[Dave Chinner]

On Tue, Jul 09, 2013 at 04:37:03PM -0500, Ben Myers wrote:
> On Wed, Jul 10, 2013 at 07:03:59AM +1000, Dave Chinner wrote:
> > From: Dave Chinner <dchinner@redhat.com>
> > 
> > Because it's horribly out of date.
> > 
> > And mark various deprecated options as deprecated and give them a
> > removal date.
> > 
> > Signed-off-by: Dave Chinner <dchinner@redhat.com>
> > Reviewed-by: Mark Tinguely <tinguely@sgi.com>
> > ---
> >  Documentation/filesystems/xfs.txt | 317 +++++++++++++++++++++++++-------------
> >  1 file changed, 209 insertions(+), 108 deletions(-)
> > 
> > diff --git a/Documentation/filesystems/xfs.txt b/Documentation/filesystems/xfs.txt
> > index 83577f0..12525b1 100644
> > --- a/Documentation/filesystems/xfs.txt
> > +++ b/Documentation/filesystems/xfs.txt
> > @@ -18,6 +18,8 @@ Mount Options
> >  =============
> >  
> >  When mounting an XFS filesystem, the following options are accepted.
> > +For boolean mount options, the names with the (*) suffix is the
> > +default behaviour.
> >  
> >    allocsize=size
> >  	Sets the buffered I/O end-of-file preallocation size when
> > @@ -25,97 +27,128 @@ When mounting an XFS filesystem, the following options are accepted.
> >  	Valid values for this option are page size (typically 4KiB)
> >  	through to 1GiB, inclusive, in power-of-2 increments.
> >  
> > -  attr2/noattr2
> > -	The options enable/disable (default is disabled for backward
> > -	compatibility on-disk) an "opportunistic" improvement to be
> > -	made in the way inline extended attributes are stored on-disk.
> > -	When the new form is used for the first time (by setting or
> > -	removing extended attributes) the on-disk superblock feature
> > -	bit field will be updated to reflect this format being in use.
> > +	The default behaviour is for dynamic end-of-file
> > +	preallocation size, which uses a set of heuristics to
> > +	optimise the preallocation size based on the current
> > +	allocation patterns within the file and the access patterns
> > +	to the file. Specifying a fixed allocsize value turns off
> > +	the dynamic behaviour.
> > +
> > +  attr2
> > +  noattr2
> > +	The options enable/disable an "opportunistic" improvement to
> > +	be made in the way inline extended attributes are stored
> > +	on-disk.  When the new form is used for the first time when
> > +	attr2 is selected (either when setting or removing extended
> > +	attributes) the on-disk superblock feature bit field will be
> > +	updated to reflect this format being in use.
> > +
> > +	The default behaviour is determined by the on-disk feature
> > +	bit indicating that attr2 behaviour is active. If either
> > +	mount option it set, then that becomes the new default used
> > +	by the filesystem.
> >  
> >  	CRC enabled filesystems always use the attr2 format, and so
> >  	will reject the noattr2 mount option if it is set.
> >  
> > -  barrier
> > -	Enables the use of block layer write barriers for writes into
> > -	the journal and unwritten extent conversion.  This allows for
> > -	drive level write caching to be enabled, for devices that
> > -	support write barriers.
> > +  barrier (*)
> > +  nobarrier
> > +	Enables/disables the use of block layer write barriers for
> > +	writes into the journal and for data integrity operations.
> > +	This allows for drive level write caching to be enabled, for
> > +	devices that support write barriers.
> >  
> >    discard
> > -	Issue command to let the block device reclaim space freed by the
> > -	filesystem.  This is useful for SSD devices, thinly provisioned
> > -	LUNs and virtual machine images, but may have a performance
> > -	impact.
> > -
> > -  dmapi
> > -	Enable the DMAPI (Data Management API) event callouts.
> > -	Use with the "mtpt" option.
> > -
> > -  grpid/bsdgroups and nogrpid/sysvgroups
> > -	These options define what group ID a newly created file gets.
> > -	When grpid is set, it takes the group ID of the directory in
> > -	which it is created; otherwise (the default) it takes the fsgid
> > -	of the current process, unless the directory has the setgid bit
> > -	set, in which case it takes the gid from the parent directory,
> > -	and also gets the setgid bit set if it is a directory itself.
> > -
> > -  ihashsize=value
> > -	In memory inode hashes have been removed, so this option has
> > -	no function as of August 2007. Option is deprecated.
> > -
> > -  ikeep/noikeep
> > -	When ikeep is specified, XFS does not delete empty inode clusters
> > -	and keeps them around on disk. ikeep is the traditional XFS
> > -	behaviour. When noikeep is specified, empty inode clusters
> > -	are returned to the free space pool. The default is noikeep for
> > -	non-DMAPI mounts, while ikeep is the default when DMAPI is in use.
> > -
> > -  inode64
> > -	Indicates that XFS is allowed to create inodes at any location
> > -	in the filesystem, including those which will result in inode
> > -	numbers occupying more than 32 bits of significance.  This is
> > -	the default allocation option. Applications which do not handle
> > -	inode numbers bigger than 32 bits, should use inode32 option.
> > +  nodiscard (*)
> > +	Enable/disable the issuing of commands to let the block
> > +	device reclaim space freed by the filesystem.  This is
> > +	useful for SSD devices, thinly provisioned LUNs and virtual
> > +	machine images, but may have a performance impact.
> > +
> > +	Note: It is currently recommended that you use the fstrim
> > +	application to discard unused blocks rather than the discard
> > +	mount option because the performance impact of this option
> > +	is quite severe.
> > +
> > +  grpid/bsdgroups
> > +  nogrpid/sysvgroups (*)
> > +	These options define what group ID a newly created file
> > +	gets.  When grpid is set, it takes the group ID of the
> > +	directory in which it is created; otherwise it takes the
> > +	fsgid of the current process, unless the directory has the
> > +	setgid bit set, in which case it takes the gid from the
> > +	parent directory, and also gets the setgid bit set if it is
> > +	a directory itself.
> > +
> > +  filestreams
> > +	Make the data allocator use the filestreams allocation mode
> > +	across the entire filesystem rather than just on directories
> > +	configured to use it.
> > +
> > +  ikeep
> > +  noikeep (*)
> > +	When ikeep is specified, XFS does not delete empty inode
> > +	clusters and keeps them around on disk.  When noikeep is
> > +	specified, empty inode clusters are returned to the free
> > +	space pool.
> >  
> >    inode32
> > -	Indicates that XFS is limited to create inodes at locations which
> > -	will not result in inode numbers with more than 32 bits of
> > -	significance. This is provided for backwards compatibility, since
> > -	64 bits inode numbers might cause problems for some applications
> > -	that cannot handle large inode numbers.
> > -
> > -  largeio/nolargeio
> > +  inode64 (*)
> > +	When inode32 is specified, it indicates that XFS limits
> > +	inode creation to locations which will not result in inode
> > +	numbers with more than 32 bits of significance.
> > +
> > +	When inode64 is specified, it indicates that XFS is allowed
> > +	to create inodes at any location in the filesystem,
> > +	including those which will result in inode numbers occupying
> > +	more than 32 bits of significance. 
> > +
> > +	inode32 is provided for backwards compatibility with older
> > +	systems and applications, since 64 bits inode numbers might
> > +	cause problems for some applications that cannot handle
> > +	large inode numbers.  If applications are in use which do
> > +	not handle inode numbers bigger than 32 bits, the inode32
> > +	option should be specified.
> > +
> > +
> > +  largeio
> > +  nolargeio (*)
> >  	If "nolargeio" is specified, the optimal I/O reported in
> > -	st_blksize by stat(2) will be as small as possible to allow user
> > -	applications to avoid inefficient read/modify/write I/O.
> > -	If "largeio" specified, a filesystem that has a "swidth" specified
> > -	will return the "swidth" value (in bytes) in st_blksize. If the
> > -	filesystem does not have a "swidth" specified but does specify
> > -	an "allocsize" then "allocsize" (in bytes) will be returned
> > -	instead.
> > -	If neither of these two options are specified, then filesystem
> > -	will behave as if "nolargeio" was specified.
> > +	st_blksize by stat(2) will be as small as possible to allow
> > +	user applications to avoid inefficient read/modify/write
> > +	I/O.  This is typically the page size of the machine, as
> > +	this is the granularity of the page cache.
> > +
> > +	If "largeio" specified, a filesystem that was created with a
> > +	"swidth" specified will return the "swidth" value (in bytes)
> > +	in st_blksize. If the filesystem does not have a "swidth"
> > +	specified but does specify an "allocsize" then "allocsize"
> > +	(in bytes) will be returned instead. Otherwise the behaviour
> > +	is the same as if "nolargeio" was specified.
> >  
> >    logbufs=value
> > -	Set the number of in-memory log buffers.  Valid numbers range
> > -	from 2-8 inclusive.
> > -	The default value is 8 buffers for filesystems with a
> > -	blocksize of 64KiB, 4 buffers for filesystems with a blocksize
> > -	of 32KiB, 3 buffers for filesystems with a blocksize of 16KiB
> > -	and 2 buffers for all other configurations.  Increasing the
> > -	number of buffers may increase performance on some workloads
> > -	at the cost of the memory used for the additional log buffers
> > -	and their associated control structures.
> > +	Set the number of in-memory log buffers.  Valid numbers
> > +	range from 2-8 inclusive.
> > +
> > +	The default value is 8 buffers.
> > +
> > +	If the memory cost of 8 log buffers is too high on small
> > +	systems, then it may be reduced at some cost to performance
> > +	on metadata intensive workloads. The logbsize option below
> > +	controls the size of each buffer and so is also relevent to
> > +	this case.
> >  
> >    logbsize=value
> > -	Set the size of each in-memory log buffer.
> > -	Size may be specified in bytes, or in kilobytes with a "k" suffix.
> > -	Valid sizes for version 1 and version 2 logs are 16384 (16k) and
> > -	32768 (32k).  Valid sizes for version 2 logs also include
> > -	65536 (64k), 131072 (128k) and 262144 (256k).
> > -	The default value for machines with more than 32MiB of memory
> > -	is 32768, machines with less memory use 16384 by default.
> > +	Set the size of each in-memory log buffer.  The size may be
> > +	specified in bytes, or in kilobytes with a "k" suffix.
> > +	Valid sizes for version 1 and version 2 logs are 16384 (16k)
> > +	and 32768 (32k).  Valid sizes for version 2 logs also
> > +	include 65536 (64k), 131072 (128k) and 262144 (256k). The
> > +	logbsize must be an integer multiple of the log
> > +	stripe unit configured at mkfs time.
> > +
> > +	The default value for for version 1 logs is 32768, while the
> > +	default value for version 2 logs is MAX(32768, log_sunit).
> >  
> >    logdev=device and rtdev=device
> >  	Use an external log (metadata journal) and/or real-time device.
> > @@ -124,16 +157,11 @@ When mounting an XFS filesystem, the following options are accepted.
> >  	optional, and the log section can be separate from the data
> >  	section or contained within it.
> >  
> > -  mtpt=mountpoint
> > -	Use with the "dmapi" option.  The value specified here will be
> > -	included in the DMAPI mount event, and should be the path of
> > -	the actual mountpoint that is used.
> > -
> >    noalign
> > -	Data allocations will not be aligned at stripe unit boundaries.
> > -
> > -  noatime
> > -	Access timestamps are not updated when a file is read.
> > +	Data allocations will not be aligned at stripe unit
> > +	boundaries. This is only relevant to filesystems created
> > +	with non-zero data alignment parameters (sunit, swidth) by
> > +	mkfs.
> >  
> >    norecovery
> >  	The filesystem will be mounted without running log recovery.
> > @@ -144,8 +172,14 @@ When mounting an XFS filesystem, the following options are accepted.
> >  	the mount will fail.
> >  
> >    nouuid
> > -	Don't check for double mounted file systems using the file system uuid.
> > -	This is useful to mount LVM snapshot volumes.
> > +	Don't check for double mounted file systems using the file
> > +	system uuid.  This is useful to mount LVM snapshot volumes,
> > +	and often used in combination with "norecovery" for mounting
> > +	read-only snapshots.
> > +
> > +  noquota
> > +	Forcibly turns off all quota accounting and enforcement
> > +	within the filesystem.
> >  
> >    uquota/usrquota/uqnoenforce/quota
> >  	User disk quota accounting enabled, and limits (optionally)
> > @@ -160,24 +194,64 @@ When mounting an XFS filesystem, the following options are accepted.
> >  	enforced.  Refer to xfs_quota(8) for further details.
> >  
> >    sunit=value and swidth=value
> > -	Used to specify the stripe unit and width for a RAID device or
> > -	a stripe volume.  "value" must be specified in 512-byte block
> > -	units.
> > -	If this option is not specified and the filesystem was made on
> > -	a stripe volume or the stripe width or unit were specified for
> > -	the RAID device at mkfs time, then the mount system call will
> > -	restore the value from the superblock.  For filesystems that
> > -	are made directly on RAID devices, these options can be used
> > -	to override the information in the superblock if the underlying
> > -	disk layout changes after the filesystem has been created.
> > -	The "swidth" option is required if the "sunit" option has been
> > -	specified, and must be a multiple of the "sunit" value.
> > +	Used to specify the stripe unit and width for a RAID device
> > +	or a stripe volume.  "value" must be specified in 512-byte
> > +	block units. These options are only relevant to filesystems
> > +	that were created with non-zero data alignment parameters.
> > +
> > +	The sunit and swidth parameters specified must be compatible
> > +	with the existing filesystem alignment characteristics.  In
> > +	general, that means the only valid changes to sunit are
> > +	increasing it by a power-of-2 multiple. Valid swidth values
> > +	are any integer multiple of a valid sunit value.
> > +
> > +	Typically the only time these mount options are necessary if
> > +	after an underlying RAID device has had it's geometry
> > +	modified, such as adding a new disk to a RAID5 lun and
> > +	reshaping it.
> >  
> >    swalloc
> >  	Data allocations will be rounded up to stripe width boundaries
> >  	when the current end of file is being extended and the file
> >  	size is larger than the stripe width size.
> >  
> > +  wsync
> > +	When specified, all filesystem namespace operations are
> > +	executed synchronously. This ensures that when the namespace
> > +	operation (create, unlink, etc) completes, the change to the
> > +	namespace is on stable storage. This is useful in HA setups
> > +	where failover must not result in clients seeing
> > +	inconsistent namespace presentation during or after a
> > +	failover event.
> > +
> > +
> > +Deprecated Mount Options
> > +========================
> > +
> > +  delaylog/nodelaylog
> > +	Delayed logging is the only logging method that XFS supports
> > +	now, so these mount options are now ignored.
> > +
> > +	Due for removal in 3.12.
> > +
> > +  ihashsize=value
> > +	In memory inode hashes have been removed, so this option has
> > +	no function as of August 2007. Option is deprecated.
> > +
> > +	Due for removal in 3.12.
> > +
> > +  irixsgid
> > +	This behaviour is now controlled by a sysctl, so the mount
> > +	option is ignored.
> > +
> > +	Due for removal in 3.12.
> > +
> > +  osyncisdsync
> > +  osyncisosync
> > +	O_SYNC and O_DSYNC are fully supported, so there is no need
> > +	for these options any more.
> > +
> > +	Due for removal in 3.12.
> 
> I finally read through Documentation/ABI/README, and I agree this all seems
> fairly reasonable with respect to that doc.
> 
> I do agree that it would be good to add these into the
> Documentation/ABI/obsolete directory.

I think that it is more relevant to document it alongside the XFS
mount option/systune documentation ;)

> The only other concern that I have is that the doc is saying that
> they want two years notice.  3.12 might be a little short for that
> timeframe.  Anyway we can cross that bridge when we come to it.

The "two year" directive is that items added to the "stable" ABI
group will remain stable for at least 2 years. It does not say
anything about how long an item should remain in the obselete state
before being removed.  The rule of thumb has been generally applied
is that one year of warning between first deprecation notices and
removal is sufficient.

But it's a moot argument - the most recently deprecated mount option
in the list has been issuing warnings (i.e. telling uses directly
that it is obsolete) since:

commit 242d621964dd8641df53f7d51d4c6ead655cc5a6
Author: Christoph Hellwig <hch@infradead.org>
Date:   Wed Aug 24 05:57:51 2011 +0000

    xfs: deprecate the nodelaylog mount option

Which was release in 3.1 on October 24 2011. So by the time 3.12
releases (roughly 6 months from now), it will have been giving
deprecation warnings for over 2 years and so we are fine to remove
it.

Besides, what is in Documentation/ABI are guidelines - not hard
rules that must be strictly followed. It is up to *our discretion*
as to how we apply them to XFS. So let's treat them as intended - as
guidelines - because the world won't end if we don't follow them
strictly to the letter....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 2/3] xfs: remove local fork format handling from xfs_bmapi_write()

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

The conversion from local format to extent format requires
interpretation of the data in the fork being converted, so it cannot
be done in a generic way. It is up to the caller to convert the fork
format to extent format before calling into xfs_bmapi_write() so
format conversion can be done correctly.

The code in xfs_bmapi_write() to convert the format is used
implicitly by the attribute and directory code, but they
specifically zero the fork size so that the conversion does not do
any allocation or manipulation. Move this conversion into the
shortform to leaf functions for the dir/attr code so the conversions
are explicitly controlled by all callers.

Now we can remove the conversion code in xfs_bmapi_write.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Mark Tinguely <tinguely@sgi.com>
---
 fs/xfs/xfs_attr_leaf.c  |   2 +
 fs/xfs/xfs_bmap.c       | 199 ++++++++++++++++++++----------------------------
 fs/xfs/xfs_bmap.h       |   1 +
 fs/xfs/xfs_dir2_block.c |  20 +++--
 4 files changed, 98 insertions(+), 124 deletions(-)

diff --git a/fs/xfs/xfs_attr_leaf.c b/fs/xfs/xfs_attr_leaf.c
index 31d3cd1..b800fbc 100644
--- a/fs/xfs/xfs_attr_leaf.c
+++ b/fs/xfs/xfs_attr_leaf.c
@@ -690,6 +690,8 @@ xfs_attr_shortform_to_leaf(xfs_da_args_t *args)
 	sf = (xfs_attr_shortform_t *)tmpbuffer;
 
 	xfs_idata_realloc(dp, -size, XFS_ATTR_FORK);
+	xfs_bmap_local_to_extents_empty(dp, XFS_ATTR_FORK);
+
 	bp = NULL;
 	error = xfs_da_grow_inode(args, &blkno);
 	if (error) {
diff --git a/fs/xfs/xfs_bmap.c b/fs/xfs/xfs_bmap.c
index 8904284..05c698c 100644
--- a/fs/xfs/xfs_bmap.c
+++ b/fs/xfs/xfs_bmap.c
@@ -1161,6 +1161,24 @@ xfs_bmap_extents_to_btree(
  * since the file data needs to get logged so things will stay consistent.
  * (The bmap-level manipulations are ok, though).
  */
+void
+xfs_bmap_local_to_extents_empty(
+	struct xfs_inode	*ip,
+	int			whichfork)
+{
+	struct xfs_ifork	*ifp = XFS_IFORK_PTR(ip, whichfork);
+
+	ASSERT(XFS_IFORK_FORMAT(ip, whichfork) == XFS_DINODE_FMT_LOCAL);
+	ASSERT(ifp->if_bytes == 0);
+	ASSERT(XFS_IFORK_NEXTENTS(ip, whichfork) == 0);
+
+	xfs_bmap_forkoff_reset(ip->i_mount, ip, whichfork);
+	ifp->if_flags &= ~XFS_IFINLINE;
+	ifp->if_flags |= XFS_IFEXTENTS;
+	XFS_IFORK_FMT_SET(ip, whichfork, XFS_DINODE_FMT_EXTENTS);
+}
+
+
 STATIC int				/* error */
 xfs_bmap_local_to_extents(
 	xfs_trans_t	*tp,		/* transaction pointer */
@@ -1174,9 +1192,12 @@ xfs_bmap_local_to_extents(
 				   struct xfs_inode *ip,
 				   struct xfs_ifork *ifp))
 {
-	int		error;		/* error return value */
+	int		error = 0;
 	int		flags;		/* logging flags returned */
 	xfs_ifork_t	*ifp;		/* inode fork pointer */
+	xfs_alloc_arg_t	args;		/* allocation arguments */
+	xfs_buf_t	*bp;		/* buffer for extent block */
+	xfs_bmbt_rec_host_t *ep;	/* extent record pointer */
 
 	/*
 	 * We don't want to deal with the case of keeping inode data inline yet.
@@ -1185,68 +1206,65 @@ xfs_bmap_local_to_extents(
 	ASSERT(!(S_ISREG(ip->i_d.di_mode) && whichfork == XFS_DATA_FORK));
 	ifp = XFS_IFORK_PTR(ip, whichfork);
 	ASSERT(XFS_IFORK_FORMAT(ip, whichfork) == XFS_DINODE_FMT_LOCAL);
+
+	if (!ifp->if_bytes) {
+		xfs_bmap_local_to_extents_empty(ip, whichfork);
+		flags = XFS_ILOG_CORE;
+		goto done;
+	}
+
 	flags = 0;
 	error = 0;
-	if (ifp->if_bytes) {
-		xfs_alloc_arg_t	args;	/* allocation arguments */
-		xfs_buf_t	*bp;	/* buffer for extent block */
-		xfs_bmbt_rec_host_t *ep;/* extent record pointer */
-
-		ASSERT((ifp->if_flags &
-			(XFS_IFINLINE|XFS_IFEXTENTS|XFS_IFEXTIREC)) == XFS_IFINLINE);
-		memset(&args, 0, sizeof(args));
-		args.tp = tp;
-		args.mp = ip->i_mount;
-		args.firstblock = *firstblock;
-		/*
-		 * Allocate a block.  We know we need only one, since the
-		 * file currently fits in an inode.
-		 */
-		if (*firstblock == NULLFSBLOCK) {
-			args.fsbno = XFS_INO_TO_FSB(args.mp, ip->i_ino);
-			args.type = XFS_ALLOCTYPE_START_BNO;
-		} else {
-			args.fsbno = *firstblock;
-			args.type = XFS_ALLOCTYPE_NEAR_BNO;
-		}
-		args.total = total;
-		args.minlen = args.maxlen = args.prod = 1;
-		error = xfs_alloc_vextent(&args);
-		if (error)
-			goto done;
-
-		/* Can't fail, the space was reserved. */
-		ASSERT(args.fsbno != NULLFSBLOCK);
-		ASSERT(args.len == 1);
-		*firstblock = args.fsbno;
-		bp = xfs_btree_get_bufl(args.mp, tp, args.fsbno, 0);
-
-		/* initialise the block and copy the data */
-		init_fn(tp, bp, ip, ifp);
-
-		/* account for the change in fork size and log everything */
-		xfs_trans_log_buf(tp, bp, 0, ifp->if_bytes - 1);
-		xfs_bmap_forkoff_reset(args.mp, ip, whichfork);
-		xfs_idata_realloc(ip, -ifp->if_bytes, whichfork);
-		xfs_iext_add(ifp, 0, 1);
-		ep = xfs_iext_get_ext(ifp, 0);
-		xfs_bmbt_set_allf(ep, 0, args.fsbno, 1, XFS_EXT_NORM);
-		trace_xfs_bmap_post_update(ip, 0,
-				whichfork == XFS_ATTR_FORK ? BMAP_ATTRFORK : 0,
-				_THIS_IP_);
-		XFS_IFORK_NEXT_SET(ip, whichfork, 1);
-		ip->i_d.di_nblocks = 1;
-		xfs_trans_mod_dquot_byino(tp, ip,
-			XFS_TRANS_DQ_BCOUNT, 1L);
-		flags |= xfs_ilog_fext(whichfork);
+	ASSERT((ifp->if_flags & (XFS_IFINLINE|XFS_IFEXTENTS|XFS_IFEXTIREC)) ==
+								XFS_IFINLINE);
+	memset(&args, 0, sizeof(args));
+	args.tp = tp;
+	args.mp = ip->i_mount;
+	args.firstblock = *firstblock;
+	/*
+	 * Allocate a block.  We know we need only one, since the
+	 * file currently fits in an inode.
+	 */
+	if (*firstblock == NULLFSBLOCK) {
+		args.fsbno = XFS_INO_TO_FSB(args.mp, ip->i_ino);
+		args.type = XFS_ALLOCTYPE_START_BNO;
 	} else {
-		ASSERT(XFS_IFORK_NEXTENTS(ip, whichfork) == 0);
-		xfs_bmap_forkoff_reset(ip->i_mount, ip, whichfork);
+		args.fsbno = *firstblock;
+		args.type = XFS_ALLOCTYPE_NEAR_BNO;
 	}
-	ifp->if_flags &= ~XFS_IFINLINE;
-	ifp->if_flags |= XFS_IFEXTENTS;
-	XFS_IFORK_FMT_SET(ip, whichfork, XFS_DINODE_FMT_EXTENTS);
+	args.total = total;
+	args.minlen = args.maxlen = args.prod = 1;
+	error = xfs_alloc_vextent(&args);
+	if (error)
+		goto done;
+
+	/* Can't fail, the space was reserved. */
+	ASSERT(args.fsbno != NULLFSBLOCK);
+	ASSERT(args.len == 1);
+	*firstblock = args.fsbno;
+	bp = xfs_btree_get_bufl(args.mp, tp, args.fsbno, 0);
+
+	/* initialise the block and copy the data */
+	init_fn(tp, bp, ip, ifp);
+
+	/* account for the change in fork size and log everything */
+	xfs_trans_log_buf(tp, bp, 0, ifp->if_bytes - 1);
+	xfs_idata_realloc(ip, -ifp->if_bytes, whichfork);
+	xfs_bmap_local_to_extents_empty(ip, whichfork);
 	flags |= XFS_ILOG_CORE;
+
+	xfs_iext_add(ifp, 0, 1);
+	ep = xfs_iext_get_ext(ifp, 0);
+	xfs_bmbt_set_allf(ep, 0, args.fsbno, 1, XFS_EXT_NORM);
+	trace_xfs_bmap_post_update(ip, 0,
+			whichfork == XFS_ATTR_FORK ? BMAP_ATTRFORK : 0,
+			_THIS_IP_);
+	XFS_IFORK_NEXT_SET(ip, whichfork, 1);
+	ip->i_d.di_nblocks = 1;
+	xfs_trans_mod_dquot_byino(tp, ip,
+		XFS_TRANS_DQ_BCOUNT, 1L);
+	flags |= xfs_ilog_fext(whichfork);
+
 done:
 	*logflagsp = flags;
 	return error;
@@ -1323,25 +1341,6 @@ xfs_bmap_add_attrfork_extents(
 }
 
 /*
- * Block initialisation function for local to extent format conversion.
- *
- * This shouldn't actually be called by anyone, so make sure debug kernels cause
- * a noticable failure.
- */
-STATIC void
-xfs_bmap_local_to_extents_init_fn(
-	struct xfs_trans	*tp,
-	struct xfs_buf		*bp,
-	struct xfs_inode	*ip,
-	struct xfs_ifork	*ifp)
-{
-	ASSERT(0);
-	bp->b_ops = &xfs_bmbt_buf_ops;
-	memcpy(bp->b_addr, ifp->if_u1.if_data, ifp->if_bytes);
-	xfs_trans_buf_set_type(tp, bp, XFS_BLFT_BTREE_BUF);
-}
-
-/*
  * Called from xfs_bmap_add_attrfork to handle local format files. Each
  * different data fork content type needs a different callout to do the
  * conversion. Some are basic and only require special block initialisation
@@ -1381,9 +1380,9 @@ xfs_bmap_add_attrfork_local(
 						 flags, XFS_DATA_FORK,
 						 xfs_symlink_local_to_remote);
 
-	return xfs_bmap_local_to_extents(tp, ip, firstblock, 1, flags,
-					 XFS_DATA_FORK,
-					 xfs_bmap_local_to_extents_init_fn);
+	/* should only be called for types that support local format data */
+	ASSERT(0);
+	return EFSCORRUPTED;
 }
 
 /*
@@ -4907,20 +4906,19 @@ xfs_bmapi_write(
 	orig_mval = mval;
 	orig_nmap = *nmap;
 #endif
+	whichfork = (flags & XFS_BMAPI_ATTRFORK) ?
+		XFS_ATTR_FORK : XFS_DATA_FORK;
 
 	ASSERT(*nmap >= 1);
 	ASSERT(*nmap <= XFS_BMAP_MAX_NMAP);
 	ASSERT(!(flags & XFS_BMAPI_IGSTATE));
 	ASSERT(tp != NULL);
 	ASSERT(len > 0);
-
-	whichfork = (flags & XFS_BMAPI_ATTRFORK) ?
-		XFS_ATTR_FORK : XFS_DATA_FORK;
+	ASSERT(XFS_IFORK_FORMAT(ip, whichfork) != XFS_DINODE_FMT_LOCAL);
 
 	if (unlikely(XFS_TEST_ERROR(
 	    (XFS_IFORK_FORMAT(ip, whichfork) != XFS_DINODE_FMT_EXTENTS &&
-	     XFS_IFORK_FORMAT(ip, whichfork) != XFS_DINODE_FMT_BTREE &&
-	     XFS_IFORK_FORMAT(ip, whichfork) != XFS_DINODE_FMT_LOCAL),
+	     XFS_IFORK_FORMAT(ip, whichfork) != XFS_DINODE_FMT_BTREE),
 	     mp, XFS_ERRTAG_BMAPIFORMAT, XFS_RANDOM_BMAPIFORMAT))) {
 		XFS_ERROR_REPORT("xfs_bmapi_write", XFS_ERRLEVEL_LOW, mp);
 		return XFS_ERROR(EFSCORRUPTED);
@@ -4933,37 +4931,6 @@ xfs_bmapi_write(
 
 	XFS_STATS_INC(xs_blk_mapw);
 
-	if (XFS_IFORK_FORMAT(ip, whichfork) == XFS_DINODE_FMT_LOCAL) {
-		/*
-		 * XXX (dgc): This assumes we are only called for inodes that
-		 * contain content neutral data in local format. Anything that
-		 * contains caller-specific data in local format that needs
-		 * transformation to move to a block format needs to do the
-		 * conversion to extent format itself.
-		 *
-		 * Directory data forks and attribute forks handle this
-		 * themselves, but with the addition of metadata verifiers every
-		 * data fork in local format now contains caller specific data
-		 * and as such conversion through this function is likely to be
-		 * broken.
-		 *
-		 * The only likely user of this branch is for remote symlinks,
-		 * but we cannot overwrite the data fork contents of the symlink
-		 * (EEXIST occurs higher up the stack) and so it will never go
-		 * from local format to extent format here. Hence I don't think
-		 * this branch is ever executed intentionally and we should
-		 * consider removing it and asserting that xfs_bmapi_write()
-		 * cannot be called directly on local format forks. i.e. callers
-		 * are completely responsible for local to extent format
-		 * conversion, not xfs_bmapi_write().
-		 */
-		error = xfs_bmap_local_to_extents(tp, ip, firstblock, total,
-					&bma.logflags, whichfork,
-					xfs_bmap_local_to_extents_init_fn);
-		if (error)
-			goto error0;
-	}
-
 	if (*firstblock == NULLFSBLOCK) {
 		if (XFS_IFORK_FORMAT(ip, whichfork) == XFS_DINODE_FMT_BTREE)
 			bma.minleft = be16_to_cpu(ifp->if_broot->bb_level) + 1;
diff --git a/fs/xfs/xfs_bmap.h b/fs/xfs/xfs_bmap.h
index 5f469c3..1cf1292 100644
--- a/fs/xfs/xfs_bmap.h
+++ b/fs/xfs/xfs_bmap.h
@@ -172,6 +172,7 @@ void	xfs_bmap_trace_exlist(struct xfs_inode *ip, xfs_extnum_t cnt,
 #endif
 
 int	xfs_bmap_add_attrfork(struct xfs_inode *ip, int size, int rsvd);
+void	xfs_bmap_local_to_extents_empty(struct xfs_inode *ip, int whichfork);
 void	xfs_bmap_add_free(xfs_fsblock_t bno, xfs_filblks_t len,
 		struct xfs_bmap_free *flist, struct xfs_mount *mp);
 void	xfs_bmap_cancel(struct xfs_bmap_free *flist);
diff --git a/fs/xfs/xfs_dir2_block.c b/fs/xfs/xfs_dir2_block.c
index e59f5fc..53b9aa2 100644
--- a/fs/xfs/xfs_dir2_block.c
+++ b/fs/xfs/xfs_dir2_block.c
@@ -29,6 +29,7 @@
 #include "xfs_dinode.h"
 #include "xfs_inode.h"
 #include "xfs_inode_item.h"
+#include "xfs_bmap.h"
 #include "xfs_buf_item.h"
 #include "xfs_dir2.h"
 #include "xfs_dir2_format.h"
@@ -1167,13 +1168,15 @@ xfs_dir2_sf_to_block(
 	__be16			*tagp;		/* end of data entry */
 	xfs_trans_t		*tp;		/* transaction pointer */
 	struct xfs_name		name;
+	struct xfs_ifork	*ifp;
 
 	trace_xfs_dir2_sf_to_block(args);
 
 	dp = args->dp;
 	tp = args->trans;
 	mp = dp->i_mount;
-	ASSERT(dp->i_df.if_flags & XFS_IFINLINE);
+	ifp = XFS_IFORK_PTR(dp, XFS_DATA_FORK);
+	ASSERT(ifp->if_flags & XFS_IFINLINE);
 	/*
 	 * Bomb out if the shortform directory is way too short.
 	 */
@@ -1182,22 +1185,23 @@ xfs_dir2_sf_to_block(
 		return XFS_ERROR(EIO);
 	}
 
-	oldsfp = (xfs_dir2_sf_hdr_t *)dp->i_df.if_u1.if_data;
+	oldsfp = (xfs_dir2_sf_hdr_t *)ifp->if_u1.if_data;
 
-	ASSERT(dp->i_df.if_bytes == dp->i_d.di_size);
-	ASSERT(dp->i_df.if_u1.if_data != NULL);
+	ASSERT(ifp->if_bytes == dp->i_d.di_size);
+	ASSERT(ifp->if_u1.if_data != NULL);
 	ASSERT(dp->i_d.di_size >= xfs_dir2_sf_hdr_size(oldsfp->i8count));
+	ASSERT(dp->i_d.di_nextents == 0);
 
 	/*
 	 * Copy the directory into a temporary buffer.
 	 * Then pitch the incore inode data so we can make extents.
 	 */
-	sfp = kmem_alloc(dp->i_df.if_bytes, KM_SLEEP);
-	memcpy(sfp, oldsfp, dp->i_df.if_bytes);
+	sfp = kmem_alloc(ifp->if_bytes, KM_SLEEP);
+	memcpy(sfp, oldsfp, ifp->if_bytes);
 
-	xfs_idata_realloc(dp, -dp->i_df.if_bytes, XFS_DATA_FORK);
+	xfs_idata_realloc(dp, -ifp->if_bytes, XFS_DATA_FORK);
+	xfs_bmap_local_to_extents_empty(dp, XFS_DATA_FORK);
 	dp->i_d.di_size = 0;
-	xfs_trans_log_inode(tp, dp, XFS_ILOG_CORE);
 
 	/*
 	 * Add block 0 to the inode.
-- 
1.8.3.2

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 3/3] xfs: dquot log reservations are too small

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

During review of the separate project quota inode patches, it became
obvious that the dquot log reservation calculation underestimated
the number dquots that can be modified in a transaction. This has
it's roots way back in the Irix quota implementation.

That is, when quotas were first implemented in XFS, it only
supported user and project quotas as Irix did not have group quotas.
Hence the worst case operation involving dquot modification was
calculated to involve 2 user dquots and 1 project dquot or 1 user
dequot and 2 project dquots. i.e. 3 dquots. This was determined back
in 1996, and has remained unchanged ever since.

However, back in 2001, the Linux XFS port dropped all support for
project quota and implmented group quotas over the top. This was
effectively done with a search-and-replace of project with group,
and as such the log reservation was not changed. However, with the
advent of group quotas, chmod and rename now could modify more than
3 dquots in a single transaction - both could modify 4 dquots. Hence
this log reservation has been wrong for a long time.

In 2005, project quota support was reintroduced into Linux, but it
was implemented to be mutually exclusive to group quotas and so this
didn't add any new changes to the dquot log reservation. Hence when
project quotas were in use (rather than group quotas) the log
reservation was again valid, just like in the Irix days.

Now, with the addition of the separate project quota inode, group
and project quotas are no longer mutually exclusive, and hence
operations can now modify three dquots per inode where previously it
was only two. The worst case here is the rename transaction, which
can allocate/free space on two different directory inodes, and if
they have different uid/gid/prid configurations and are world
writeable, then rename can actually modify 6 different dquots now.

Further, the dquot log reservation doesn't take into account the
space used by the dquot log format structure that precedes the dquot
that is logged, and hence further underestimates the worst case
log space required by dquots during a transaction. This has been
missing since the first commit in 1996.

Hence the worst case log reservation needs to be increased from 3 to
6, and it needs to take into account a log format header for each of
those dquots.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Mark Tinguely <tinguely@sgi.com>
---
 fs/xfs/xfs_quota.h       | 25 +++++++++++++++++++++----
 fs/xfs/xfs_trans_dquot.c |  9 ++++-----
 2 files changed, 25 insertions(+), 9 deletions(-)

diff --git a/fs/xfs/xfs_quota.h b/fs/xfs/xfs_quota.h
index c3483ba..40d508b 100644
--- a/fs/xfs/xfs_quota.h
+++ b/fs/xfs/xfs_quota.h
@@ -108,11 +108,28 @@ typedef struct xfs_dqblk {
 	{ XFS_DQ_FREEING,	"FREEING" }
 
 /*
- * In the worst case, when both user and group quotas are on,
- * we can have a max of three dquots changing in a single transaction.
+ * We have the possibility of all three quota types being active at once, and
+ * hence free space modification requires modification of all three current
+ * dquots in a single transaction. For this case we need to have a reservation
+ * of at least 3 dquots.
+ *
+ * However, a chmod operation can change both UID and GID in a single
+ * transaction, resulting in requiring {old, new} x {uid, gid} dquots to be
+ * modified. Hence for this case we need to reserve space for at least 4 dquots.
+ *
+ * And in the worst case, there's a rename operation that can be modifying up to
+ * 4 inodes with dquots attached to them. In reality, the only inodes that can
+ * have their dquots modified are the source and destination directory inodes
+ * due to directory name creation and removal. That can require space allocation
+ * and/or freeing on both directory inodes, and hence all three dquots on each
+ * inode can be modified. And if the directories are world writeable, all the
+ * dquots can be unique and so 6 dquots can be modified....
+ *
+ * And, of course, we also need to take into account the dquot log format item
+ * used to describe each dquot.
  */
-#define XFS_DQUOT_LOGRES(mp)	(sizeof(xfs_disk_dquot_t) * 3)
-
+#define XFS_DQUOT_LOGRES(mp)	\
+	((sizeof(struct xfs_dq_logformat) + sizeof(struct xfs_disk_dquot)) * 6)
 
 /*
  * These are the structures used to lay out dquots and quotaoff
diff --git a/fs/xfs/xfs_trans_dquot.c b/fs/xfs/xfs_trans_dquot.c
index 3ba64d5..db041a5 100644
--- a/fs/xfs/xfs_trans_dquot.c
+++ b/fs/xfs/xfs_trans_dquot.c
@@ -291,11 +291,10 @@ xfs_trans_mod_dquot(
 
 
 /*
- * Given an array of dqtrx structures, lock all the dquots associated
- * and join them to the transaction, provided they have been modified.
- * We know that the highest number of dquots (of one type - usr OR grp),
- * involved in a transaction is 2 and that both usr and grp combined - 3.
- * So, we don't attempt to make this very generic.
+ * Given an array of dqtrx structures, lock all the dquots associated and join
+ * them to the transaction, provided they have been modified.  We know that the
+ * highest number of dquots of one type - usr, grp OR prj - involved in a
+ * transaction is 2 so we don't need to make this very generic.
  */
 STATIC void
 xfs_trans_dqlockedjoin(
-- 
1.8.3.2

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 0/3] xfs: remaining 3.11 merge patches

[Ben Myers]

Hey Dave,

On Wed, Jul 10, 2013 at 07:03:58AM +1000, Dave Chinner wrote:
> I figured it was easiest for you to just repost all three remainng
> patches for the 3.11 merge window that I have in one set. The dquot
> log res patch has been updated to address Chandra's review.

Much appreciated!  

Thanks,
	Ben

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 0/3] xfs: remaining 3.11 merge patches

[Ben Myers]

On Tue, Jul 09, 2013 at 04:09:41PM -0500, Ben Myers wrote:
> On Wed, Jul 10, 2013 at 07:03:58AM +1000, Dave Chinner wrote:
> > I figured it was easiest for you to just repost all three remainng
> > patches for the 3.11 merge window that I have in one set. The dquot
> > log res patch has been updated to address Chandra's review.
> 
> Much appreciated!  

Applied the last two.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs