Re: [PATCH 0/2] Make task doing heavy writing killable

[Jan Kara <jack@suse.cz>]

On Mon 14-11-11 19:59:12, Wu Fengguang wrote:
> On Mon, Nov 14, 2011 at 07:10:28PM +0800, Jan Kara wrote:
> > 
> >   Hello,
> > 
> >   these two patches aim at making task waiting in balance_dirty_pages()
> > killable.  This is desirable because otherwise if filesystem stops accepting
> > writes (e.g. if device has been removed or other serious error condidion) we
> > have a task stuck in D state forever.
> 
> Agreed totally. I myself has run into such conditions and get very
> annoyed not being able to kill the hard throttled tasks -- they just
> stuck there for ever if the error condition does not change.
> 
> >   I'm not sure who should merge these two patches... Al, Fengguang?
> 
> I'd like to do it -- otherwise there will obviously be merge conflicts.
  Good.

> Actually I also queued a patch to do this (attached). Your patches do
> better on TASK_KILLABLE and the use of signal_pending() in write
> routines, while mine goes further to add the break to various
> filesystems.  How about combining them together?
> 
> Subject: writeback: quit throttling when fatal signal pending
> Date: Wed Sep 08 17:40:22 CST 2010
> 
> This allows quick response to Ctrl-C etc. for impatient users.
> 
> It's necessary to abort the generic_perform_write() and other filesystem
> write loops too, to avoid large write + SIGKILL combination exceeding
> the dirty limit and possibly strange OOM.
> 
> It mainly helps the rare bdi/global dirty exceeded cases.
> In the normal case of not exceeded, it will quit the loop anyway.
> 
> 		if (fatal_signal_pending(current) &&
> 		    nr_reclaimable + nr_writeback <= dirty_thresh)
> 			break;
> 
> Reviewed-by: Neil Brown <neilb@suse.de>
> Reviewed-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
> cc: Chris Mason <chris.mason@oracle.com>
> cc: Anton Altaparmakov <aia21@cantab.net>
> Signed-off-by: Wu Fengguang <fengguang.wu@intel.com>
> ---
>  fs/btrfs/file.c       |    4 ++++
>  fs/btrfs/ioctl.c      |    4 ++++
>  fs/btrfs/relocation.c |    4 ++++
>  fs/buffer.c           |    4 ++++
>  fs/ntfs/attrib.c      |    2 ++
>  fs/ntfs/file.c        |    6 ++++++
>  mm/filemap.c          |    3 ++-
>  mm/page-writeback.c   |    2 ++
>  8 files changed, 28 insertions(+), 1 deletion(-)
> 
> --- linux-next.orig/mm/page-writeback.c	2011-11-13 20:37:11.000000000 +0800
> +++ linux-next/mm/page-writeback.c	2011-11-13 20:37:57.000000000 +0800
> @@ -1185,6 +1185,8 @@ pause:
>  		 */
>  		if (task_ratelimit)
>  			break;
> +		if (fatal_signal_pending(current))
> +			break;
>  	}
>  
>  	if (!dirty_exceeded && bdi->dirty_exceeded)
> --- linux-next.orig/mm/filemap.c	2011-11-13 20:03:23.000000000 +0800
> +++ linux-next/mm/filemap.c	2011-11-13 20:37:17.000000000 +0800
> @@ -2463,7 +2463,8 @@ again:
>  		written += copied;
>  
>  		balance_dirty_pages_ratelimited(mapping);
> -
> +		if (fatal_signal_pending(current))
> +			break;
>  	} while (iov_iter_count(i));
>  
>  	return written ? written : status;
  Above two hunks are already part of my patches (effectively). The hunks
below look good but I guess they should be split to per-filesystem patches
and sent to filesystem maintainers for merging. The changes are independent
so there's no reason to do them in one big patch or skip fs maintainers...

									Honza

> --- linux-next.orig/fs/btrfs/file.c	2011-11-13 20:37:11.000000000 +0800
> +++ linux-next/fs/btrfs/file.c	2011-11-13 20:37:17.000000000 +0800
> @@ -1274,6 +1274,10 @@ static noinline ssize_t __btrfs_buffered
>  						   dirty_pages);
>  		if (dirty_pages < (root->leafsize >> PAGE_CACHE_SHIFT) + 1)
>  			btrfs_btree_balance_dirty(root, 1);
> +		if (fatal_signal_pending(current)) {
> +			ret = -EINTR;
> +			break;
> +		}
>  		btrfs_throttle(root);
>  
>  		pos += copied;
> --- linux-next.orig/fs/btrfs/ioctl.c	2011-11-13 20:03:23.000000000 +0800
> +++ linux-next/fs/btrfs/ioctl.c	2011-11-13 20:37:17.000000000 +0800
> @@ -1115,6 +1115,10 @@ int btrfs_defrag_file(struct inode *inod
>  
>  		defrag_count += ret;
>  		balance_dirty_pages_ratelimited_nr(inode->i_mapping, ret);
> +		if (fatal_signal_pending(current)) {
> +			ret = -EINTR;
> +			goto out_ra;
> +		}
>  
>  		if (newer_than) {
>  			if (newer_off == (u64)-1)
> --- linux-next.orig/fs/btrfs/relocation.c	2011-11-13 20:03:23.000000000 +0800
> +++ linux-next/fs/btrfs/relocation.c	2011-11-13 20:37:17.000000000 +0800
> @@ -3009,6 +3009,10 @@ static int relocate_file_extent_cluster(
>  
>  		index++;
>  		balance_dirty_pages_ratelimited(inode->i_mapping);
> +		if (fatal_signal_pending(current)) {
> +			ret = -EINTR;
> +			break;
> +		}
>  		btrfs_throttle(BTRFS_I(inode)->root);
>  	}
>  	WARN_ON(nr != cluster->nr);
> --- linux-next.orig/fs/buffer.c	2011-11-13 20:03:23.000000000 +0800
> +++ linux-next/fs/buffer.c	2011-11-13 20:37:17.000000000 +0800
> @@ -2255,6 +2255,10 @@ static int cont_expand_zero(struct file 
>  		err = 0;
>  
>  		balance_dirty_pages_ratelimited(mapping);
> +		if (fatal_signal_pending(current)) {
> +			err = -EINTR;
> +			goto out;
> +		}
>  	}
>  
>  	/* page covers the boundary, find the boundary offset */
> --- linux-next.orig/fs/ntfs/attrib.c	2011-11-13 20:03:23.000000000 +0800
> +++ linux-next/fs/ntfs/attrib.c	2011-11-13 20:37:17.000000000 +0800
> @@ -2588,6 +2588,8 @@ int ntfs_attr_set(ntfs_inode *ni, const 
>  		unlock_page(page);
>  		page_cache_release(page);
>  		balance_dirty_pages_ratelimited(mapping);
> +		if (fatal_signal_pending(current))
> +			return -EINTR;
>  		cond_resched();
>  	}
>  	/* If there is a last partial page, need to do it the slow way. */
> --- linux-next.orig/fs/ntfs/file.c	2011-11-13 20:03:23.000000000 +0800
> +++ linux-next/fs/ntfs/file.c	2011-11-13 20:37:17.000000000 +0800
> @@ -278,6 +278,10 @@ do_non_resident_extend:
>  		 * files.
>  		 */
>  		balance_dirty_pages_ratelimited(mapping);
> +		if (fatal_signal_pending(current)) {
> +			err = -EINTR;
> +			goto init_err_out;
> +		}
>  		cond_resched();
>  	} while (++index < end_index);
>  	read_lock_irqsave(&ni->size_lock, flags);
> @@ -2054,6 +2058,8 @@ static ssize_t ntfs_file_buffered_write(
>  		if (unlikely(status))
>  			break;
>  		balance_dirty_pages_ratelimited(mapping);
> +		if (fatal_signal_pending(current))
> +			break;
>  		cond_resched();
>  	} while (count);
>  err_out:

-- 
Jan Kara <jack@suse.cz>
SUSE Labs, CR