stable.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	stable@vger.kernel.org, Junxiao Bi <junxiao.bi@oracle.com>,
	Dave Chinner <david@fromorbit.com>,
	"joyce.xue" <xuejiufei@huawei.com>,
	Ming Lei <ming.lei@canonical.com>,
	Trond Myklebust <trond.myklebust@primarydata.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Linus Torvalds <torvalds@linux-foundation.org>
Subject: [PATCH 3.16 069/127] mm: clear __GFP_FS when PF_MEMALLOC_NOIO is set
Date: Tue, 28 Oct 2014 11:35:04 +0800	[thread overview]
Message-ID: <20141028033423.870460996@linuxfoundation.org> (raw)
In-Reply-To: <20141028033420.925922046@linuxfoundation.org>

3.16-stable review patch.  If anyone has any objections, please let me know.

------------------

From: Junxiao Bi <junxiao.bi@oracle.com>

commit 934f3072c17cc8886f4c043b47eeeb1b12f8de33 upstream.

commit 21caf2fc1931 ("mm: teach mm by current context info to not do I/O
during memory allocation") introduces PF_MEMALLOC_NOIO flag to avoid doing
I/O inside memory allocation, __GFP_IO is cleared when this flag is set,
but __GFP_FS implies __GFP_IO, it should also be cleared.  Or it may still
run into I/O, like in superblock shrinker.  And this will make the kernel
run into the deadlock case described in that commit.

See Dave Chinner's comment about io in superblock shrinker:

Filesystem shrinkers do indeed perform IO from the superblock shrinker and
have for years.  Even clean inodes can require IO before they can be freed
- e.g.  on an orphan list, need truncation of post-eof blocks, need to
wait for ordered operations to complete before it can be freed, etc.

IOWs, Ext4, btrfs and XFS all can issue and/or block on arbitrary amounts
of IO in the superblock shrinker context.  XFS, in particular, has been
doing transactions and IO from the VFS inode cache shrinker since it was
first introduced....

Fix this by clearing __GFP_FS in memalloc_noio_flags(), this function has
masked all the gfp_mask that will be passed into fs for the processes
setting PF_MEMALLOC_NOIO in the direct reclaim path.

v1 thread at: https://lkml.org/lkml/2014/9/3/32

Signed-off-by: Junxiao Bi <junxiao.bi@oracle.com>
Cc: Dave Chinner <david@fromorbit.com>
Cc: joyce.xue <xuejiufei@huawei.com>
Cc: Ming Lei <ming.lei@canonical.com>
Cc: Trond Myklebust <trond.myklebust@primarydata.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

---
 include/linux/sched.h |    6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -1947,11 +1947,13 @@ extern void thread_group_cputime_adjuste
 #define tsk_used_math(p) ((p)->flags & PF_USED_MATH)
 #define used_math() tsk_used_math(current)
 
-/* __GFP_IO isn't allowed if PF_MEMALLOC_NOIO is set in current->flags */
+/* __GFP_IO isn't allowed if PF_MEMALLOC_NOIO is set in current->flags
+ * __GFP_FS is also cleared as it implies __GFP_IO.
+ */
 static inline gfp_t memalloc_noio_flags(gfp_t flags)
 {
 	if (unlikely(current->flags & PF_MEMALLOC_NOIO))
-		flags &= ~__GFP_IO;
+		flags &= ~(__GFP_IO | __GFP_FS);
 	return flags;
 }
 



  parent reply	other threads:[~2014-10-28  3:35 UTC|newest]

Thread overview: 129+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-10-28  3:33 [PATCH 3.16 000/127] 3.16.7-stable review Greg Kroah-Hartman
2014-10-28  3:33 ` [PATCH 3.16 001/127] btrfs: wake up transaction thread from SYNC_FS ioctl Greg Kroah-Hartman
2014-10-28  3:33 ` [PATCH 3.16 002/127] btrfs: Fix a deadlock in btrfs_dev_replace_finishing() Greg Kroah-Hartman
2014-10-28  3:33 ` [PATCH 3.16 003/127] Btrfs: add missing compression property remove in btrfs_ioctl_setflags Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 005/127] Btrfs: dont do async reclaim during log replay Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 006/127] Btrfs: try not to ENOSPC on " Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 007/127] Btrfs: cleanup error handling in build_backref_tree Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 008/127] Btrfs: fix build_backref_tree issue with multiple shared blocks Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 009/127] Btrfs: fix race in WAIT_SYNC ioctl Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 010/127] fs: Add a missing permission check to do_umount Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 011/127] usb: pch_udc: usb gadget device support for Intel Quark X1000 Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 012/127] pci_ids: Add support for Intel Quark ILB Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 013/127] kvm: x86: fix stale mmio cache bug Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 014/127] kvm: fix potentially corrupt mmio cache Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 015/127] KVM: do not bias the generation number in kvm_current_mmio_generation Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 016/127] KVM: s390: unintended fallthrough for external call Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 017/127] kvm: dont take vcpu mutex for obviously invalid vcpu ioctls Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 018/127] x86,kvm,vmx: Preserve CR4 across VM entry Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 019/127] x86/intel/quark: Switch off CR4.PGE so TLB flush uses CR3 instead Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 020/127] spi: dw-mid: respect 8 bit mode Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 021/127] spi: dw-mid: check that DMA was inited before exit Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 022/127] HID: rmi: check sanity of the incoming report Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 023/127] mpc85xx_edac: Make L2 interrupt shared too Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 024/127] regmap: debugfs: fix possbile NULL pointer dereference Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 025/127] regmap: fix NULL pointer dereference in _regmap_write/read Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 026/127] regmap: fix possible ZERO_SIZE_PTR pointer dereferencing error Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 027/127] be2iscsi: check ip buffer before copying Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 028/127] mptfusion: enable no_write_same for vmware scsi disks Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 029/127] regulator: ltc3589: fix broken voltage transitions Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 030/127] qla2xxx: fix kernel NULL pointer access Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 031/127] qla2xxx: Use correct offset to req-q-out for reserve calculation Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 032/127] qla2xxx: Fix shost use-after-free on device removal Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 033/127] dmaengine: fix xor sources continuation Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 034/127] firmware_class: make sure fw requests contain a name Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 035/127] arm64: debug: dont re-enable debug exceptions on return from el1_dbg Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 036/127] Drivers: hv: util: Properly pack the data for file copy functionality Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 037/127] Drivers: hv: vmbus: Cleanup vmbus_post_msg() Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 038/127] Drivers: hv: vmbus: Cleanup vmbus_teardown_gpadl() Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 039/127] Drivers: hv: vmbus: Cleanup vmbus_close_internal() Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 040/127] Drivers: hv: vmbus: Cleanup vmbus_establish_gpadl() Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 041/127] Drivers: hv: vmbus: Fix a bug in vmbus_open() Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 042/127] Drivers: hv: vmbus: Cleanup hv_post_message() Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 043/127] mei: bus: fix possible boundaries violation Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 044/127] m68k: Disable/restore interrupts in hwreg_present()/hwreg_write() Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 045/127] Fixing lease renewal Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 046/127] Documentation: lzo: document part of the encoding Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 047/127] Revert "lzo: properly check for overruns" Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 048/127] lzo: check for length overrun in variable length encoding Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 049/127] tty: omap-serial: fix division by zero Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 050/127] NFSv4: Fix lock recovery when CREATE_SESSION/SETCLIENTID_CONFIRM fails Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 051/127] NFSv4: fix open/lock state recovery error handling Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 052/127] NFSv4.1: Fix an NFSv4.1 state renewal regression Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 053/127] nfsd4: reserve adequate space for LOCK op Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 054/127] NFS: Fix an uninitialised pointer Oops in the writeback error path Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 055/127] NFS: Fix a bogus warning in nfs_generic_pgio Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 056/127] iwlwifi: mvm: disable BT Co-running by default Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 057/127] iwlwifi: Add missing PCI IDs for the 7260 series Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 058/127] spi: dw-mid: terminate ongoing transfers at exit Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 059/127] PCI: mvebu: Fix uninitialized variable in mvebu_get_tgt_attr() Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 060/127] PCI: Add missing MEM_64 mask in pci_assign_unassigned_bridge_resources() Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 061/127] PCI: Increase IBM ipr SAS Crocodile BARs to at least system page size Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 062/127] PCI: Generate uppercase hex for modalias interface class Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 063/127] rt2800: correct BBP1_TX_POWER_CTRL mask Greg Kroah-Hartman
2014-10-28  3:34 ` [PATCH 3.16 064/127] Revert "ath9k_hw: reduce ANI firstep range for older chips" Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 065/127] Bluetooth: Fix HCI H5 corrupted ack value Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 066/127] Bluetooth: Fix incorrect LE CoC PDU length restriction based on HCI MTU Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 067/127] Bluetooth: Fix issue with USB suspend in btusb driver Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 068/127] Bluetooth: Fix setting correct security level when initiating SMP Greg Kroah-Hartman
2014-10-28  3:35 ` Greg Kroah-Hartman [this message]
2014-10-28  3:35 ` [PATCH 3.16 071/127] kernel: add support for gcc 5 Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 073/127] futex: Ensure get_futex_key_refs() always implies a barrier Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 074/127] powerpc/iommu/ddw: Fix endianness Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 075/127] powerpc/eeh: Clear frozen device state in time Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 076/127] ima: fix fallback to use new_sync_read() Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 077/127] ima: provide flag to identify new empty files Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 078/127] arm64: compat: fix compat types affecting struct compat_elf_prpsinfo Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 079/127] ALSA: pcm: use the same dma mmap codepath both for arm and arm64 Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 081/127] ALSA: emu10k1: Fix deadlock in synth voice lookup Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 082/127] ALSA: ALC283 codec - Avoid pop noise on headphones during suspend/resume Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 083/127] ALSA: usb-audio: Add support for Steinberg UR22 USB interface Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 084/127] ALSA: hda - hdmi: Fix missing ELD change event on plug/unplug Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 085/127] ALSA: hda - Fix inverted LED gpio setup for Lenovo Ideapad Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 086/127] ALSA: hda - Add missing terminating entry to SND_HDA_PIN_QUIRK macro Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 087/127] ARM: at91/dt: Fix typo regarding can0_clk Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 088/127] ARM: at91: fix at91sam9263ek DT mmc pinmuxing settings Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 089/127] ARM: at91/PMC: dont forget to write PMC_PCDR register to disable clocks Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 090/127] ARM: Kirkwood: Fix DT based DSA Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 091/127] ARM: mvebu: Netgear RN104: Use Hardware BCH ECC Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 092/127] ARM: mvebu: Netgear RN2120: " Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 093/127] ARM: mvebu: Netgear RN102: " Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 094/127] ARM: dts: imx28-evk: Let i2c0 run at 100kHz Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 095/127] ecryptfs: avoid to access NULL pointer when write metadata in xattr Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 096/127] xfs: ensure WB_SYNC_ALL writeback handles partial pages correctly Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 097/127] sparc64: Do not disable interrupts in nmi_cpu_busy() Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 098/127] sparc64: Fix pcr_ops initialization and usage bugs Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 099/127] sparc32: dma_alloc_coherent must honour gfp flags Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 100/127] sparc64: sun4v TLB error power off events Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 101/127] sparc64: Fix corrupted thread fault code Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 102/127] sparc64: find_node adjustment Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 103/127] sparc64: Move request_irq() from ldc_bind() to ldc_alloc() Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 104/127] sparc: Let memset return the address argument Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 105/127] sparc: bpf_jit: fix support for ldx/stx mem and SKF_AD_VLAN_TAG Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 106/127] sparc: bpf_jit: fix loads from negative offsets Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 107/127] sparc64: Fix reversed start/end in flush_tlb_kernel_range() Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 108/127] sparc64: Fix lockdep warnings on reboot on Ultra-5 Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 109/127] sparc64: Fix FPU register corruption with AES crypto offload Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 110/127] sparc64: Do not define thread fpregs save area as zero-length array Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 111/127] sparc64: Fix hibernation code refrence to PAGE_OFFSET Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 112/127] sparc64: correctly recognise M6 and M7 cpu type Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 113/127] sparc64: support M6 and M7 for building CPU distribution map Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 114/127] sparc64: cpu hardware caps support for sparc M6 and M7 Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 115/127] sparc64: T5 PMU Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 116/127] sparc64: Switch to 4-level page tables Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 117/127] sparc64: Define VA hole at run time, rather than at compile time Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 118/127] sparc64: Adjust KTSB assembler to support larger physical addresses Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 119/127] sparc64: Fix physical memory management regressions with large max_phys_bits Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 120/127] sparc64: Use kernel page tables for vmemmap Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 121/127] sparc64: Increase MAX_PHYS_ADDRESS_BITS to 53 Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 122/127] sparc64: Adjust vmalloc region size based upon available virtual address bits Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 123/127] sparc64: sparse irq Greg Kroah-Hartman
2014-10-28  3:35 ` [PATCH 3.16 124/127] sparc64: Kill unnecessary tables and increase MAX_BANKS Greg Kroah-Hartman
2014-10-28  3:36 ` [PATCH 3.16 125/127] sparc64: Increase size of boot string to 1024 bytes Greg Kroah-Hartman
2014-10-28  3:36 ` [PATCH 3.16 126/127] sparc64: Fix register corruption in top-most kernel stack frame during boot Greg Kroah-Hartman
2014-10-28  3:36 ` [PATCH 3.16 127/127] sparc64: Implement __get_user_pages_fast() Greg Kroah-Hartman
2014-10-28  3:51 ` [PATCH 3.16 000/127] 3.16.7-stable review Greg Kroah-Hartman
2014-10-28 13:57   ` Steven Rostedt
2014-10-28 14:06     ` Greg Kroah-Hartman
2014-10-28 15:14 ` Guenter Roeck
2014-10-28 16:15 ` Shuah Khan

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20141028033423.870460996@linuxfoundation.org \
    --to=gregkh@linuxfoundation.org \
    --cc=akpm@linux-foundation.org \
    --cc=david@fromorbit.com \
    --cc=junxiao.bi@oracle.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=ming.lei@canonical.com \
    --cc=stable@vger.kernel.org \
    --cc=torvalds@linux-foundation.org \
    --cc=trond.myklebust@primarydata.com \
    --cc=xuejiufei@huawei.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).